Philip Holland

Rejoining VIEWS UK (or just joining for the first time)?

Recently I reported that the VIEWS web site had been closed down, but that the newsletter archive was still available on sasCommunity.org. I have now been able to store the original VIEWS News PDF files on my blog site, which can be accessed via a new VIEWS UK forum. This forum is available on request to all registered blog members, and those with access will be notified of any new content.

Some of the new content will include previously unpublished indexes I used to help me write the Formats, Options & Functions articles in the newsletters, which allowed me to see in a single view which features had already been described and when. I will also be including an authors list soon. All features that could help you find information that is difficult to find using standard searches. I may even try to publish some new issues of VIEWS News soon too!

You might be thinking that, as I had emailed VIEWS members in the past, I could use that email list to include them automatically again. However, this new VIEWS environment has been built on an existing blog site, and, although theoretically I could register all the previous VIEWS members using their email addresses, I don’t know all of their real names and would, therefore, have to guess their new user names, so I don’t think forcing anyone to register on my blog site is at all appropriate. This is the reason I’m asking everyone interested to register themselves instead.

SO, whether you have been a VIEWS member in the past, or not, here is how you can join the new VIEWS UK forum:

If you are not already registered on my blog site, then join via the registration page.
Once you have registered, then email views-uk@hollandnumerics.org.uk, or use the site’s Contact Us link, to say you would like to join the VIEWS UK forum.
I will manually update your blog profile to give you the required configuration and reply to your registered email address as soon as you have access to the new forum. You will then see VIEWS UK in the forum list when you next log into the blog site.

Looking for a SAS job or wanting to promote your SAS recruitment agency?

I’ve recently opened up new opportunities for two groups of registered members:

Recruiters wanting to promote their agencies.
SAS programmers looking for their next SAS job.

All of the forums in [Holland Numerics: Blog and Forums] now allow messages to be sent by registered members directly and privately to anyone posting a topic in a forum. This means that, if you post a new topic in the SAS Skills Discussion Forum (which is accessible to all registered members) describing your recruitment agency, or your own SAS programming skills, other registered members will be able to send you messages. The facility also removes the necessity to reveal your phone number or email address to anyone you don’t need to, as the messages will be collected on the blog site, and you will be notified by email of new messages there, and any forum topic will display a list of your own messages at the bottom of the web page.

However, please think carefully about what you include in your topic, as your agency profile, or your SAS job prospects, could be positively or negatively affected by what you write!

Note that there may also be a delay in the topic appearing if you are writing your first topic in the forum, as I will have to approve it. This is intended to protect you from spam postings. But as soon as it is visible to all registered members, then a [Send a Message] link will be seen in the topic under your name.

New app updates available in Google Play and Amazon Appstore

Android 8.0 is coming very soon, so I have started preparing my apps for this new platform, while still maintaining the same functionality for Android versions back to 2.3.3 (Gingerbread). However, I will not be further updating my Chrome apps, as the Chrome Web Store has stopped accepting new apps, and is likely to stop hosting existing apps soon too.

The first paid-for app to be updated for Android 8.0, including 6 more questions, is:

Studio: Do This in SAS? (Google Play)
Studio: How Do You Do This in SAS? (Amazon Appstore)

and the 2 free apps, including samples from the other apps:

How Do You Do This in SAS? (Google Play and Amazon Appstore)
Consulting: Do This in SAS? (Google Play)
Consulting: How Do You Do This in SAS? (Amazon Appstore)

Update to more of my Android apps will follow shortly.

This is quick reminder that I still have the following apps available in Google Play:

Data Steps: Do This in SAS?
SQL: Do This in SAS?
Graphs: Do This in SAS?
Platforms: Do This in SAS?
EG: Do This in SAS?
Macros: Do This in SAS?
Efficiency: Do This in SAS?
Studio: Do This in SAS?

These apps are also available for Android users in the Amazon Appstore:

Data Steps: How Do You Do This in SAS?
SQL: How Do You Do This in SAS?
Graphs: How Do You Do This in SAS?
Platforms: How Do You Do This in SAS?
EG: How Do You Do This in SAS?
Macros: How Do You Do This in SAS?
Efficiency: How Do You Do This in SAS?
Studio: How Do You Do This in SAS?

There are also free apps, which you can find by searching in each app store:

How Do You Do This in SAS? (Google Play and Amazon Appstore)
Consulting: Do This in SAS? (Google Play)
Consulting: How Do You Do This in SAS? (Amazon Appstore)

All these apps are updated frequently, and the paid-for apps are priced at less than US$3 based on the number of questions that are answered, and within the apps you can ask new questions that can be added in future updates. It doesn’t matter which platform you use, as the same content is present in the corresponding Google Play and Amazon Appstore apps.

Non-SAS Debugging That Uses My Preferred SAS Debugging Techniques

The morale of this story is that, once you have proved all the likely reasons for the failure are incorrect, the only remaining reason, however improbable, is the cause!

Note that, as you are reading this post, the problem described below has definitely been resolved!

Over the last few weeks I have been trying to configure my new Linux server to include RAID1 (mirrored) drives, but, while it should only have caused brief service outages for reboots, some of the down times have been rather too long. I have now decided to implement regular copies of the drives instead.

The most recent outage was somewhat baffling, as my attempt to include the RAID1 drives had failed again, but I’d followed my well-practised back out procedure. Unfortunately the server hardware appeared to be up and running, I could access both servers on the device using local IP addresses, but only one, the Apache server and not the WordPress server, using the hollandnumerics.org.uk web address.

A systematic approach to resolving the problem would be needed:

Check the server monitors to make sure the software was running:
- Both servers were running.
  - The problem was not a failure to start the server software on reboot.
Open the web pages for each server using their local IP addresses:
- The web pages for both servers opened.
  - Both servers were up and accepting connections.
Open the web pages for each server using their web addresses:
- The Apache server pages opened, but the WordPress server pages timed out.
  - Maybe a problem with the WordPress port?
Check the WordPress server access log for my attempts to open web pages:
- Only local addresses were recorded as connecting to WordPress.
  - Maybe a problem with the WordPress port in the router?
Check the event log on the router:
- All connections to the WordPress port are being blocked.
  - Maybe a fault in the Port Forwarding configuration in the router?
Check the Port Forwarding configuration in the router:
- All expected entries present.
  - Maybe a fault has developed in the supplied entry used with the WordPress port?
Replace the supplied WordPress entry with a user-modified copy of the working Apache entry, and then open web pages for the WordPress server using their web addresses:
- The web pages for the WordPress server opened successfully.
  - Problem resolved!

The key actions used to resolve this problem have much in common with those actions listed in my post “How Do You Successfully Debug Your Program?“:

Work through the problem testing systematically:
- Make changes one at a time to be certain you know which one fixes the problem, and also allows you to back out the update if it doesn’t.
- Test after each change.
- Review the results from every test.
Consider the likely and the unlikely causes of the problem.
Most importantly: Read the logs!

Premium SAS Support for Commercial Clients

Free membership gives you access to the SAS Skills Discussion Forum for non-programming questions about SAS. Programming and Recruiter memberships, which require monthly low-cost subscriptions, give you access to the same forum and to the SAS Programming Forum, where you can ask SAS programming questions. However, all questions and answers are visible to all members with access to each forum, which may not be quite what you are looking for.

To fulfill any need for confidential correspondence you may have seen a recent addition to the blog menu. The Premium SAS help desk service, called [sas.answers @ Holland Numerics] (which has actually been around since 2002), is different. It does not require even Free membership of the blog, but it does require a separate subscription payable using PayPal. What makes this service special is that it provides an in-house web-based interface for you to ask questions, which automatically generates an SMS message and an email to Holland Numerics, with a copy of the email sent back to you too. There is also a service level agreement to provide an answer to every question within 48 hours, and that answer will be privately sent directly back to the sending subscriber! Please note that, if a monthly subscription is not necessary, then a single question can be asked instead for a reduced fee.

Full details of the [sas.answers @ Holland Numerics] help desk service can be found here.

How Do You Successfully Debug Your Program?

For a long while now I have been reading discussions on a range of social media platforms where programmers have been asking for help with programs that don’t behave as expected. Many of these programmers have certificates relating to the relevant programming languages, yet appear to have never been taught how to debug a program. To me the ability to resolve issues in a program is the most vital skill a programmer of every programming languages has to possess, otherwise they simply cannot call themselves a “programmer” in that language. Without this skill I would only call them a “code writer”.

So is this ability to resolve issues in a program a difficult skill to learn? I would say no, because it just involves the following a set of simple instructions, with a little observation and logic. Note that these instructions are not at all specific to SAS programming, or indeed to any other programming language:

Talk through the logic of the program with a colleague. Quite often you can spot potential problems yourself while trying to explain the logic to another programmer.
Run the program, and then check the log for error and warning messages, as there should be none!
Look for unexpected empty intermediate data sets in the log.
Test your code with small data subsets first, so that you can follow the processing path of individual data records. These subsets ought to be special sample records which will exercise as much of the logic within the program as possible.
Print out intermediate data sets, especially if these are not kept at the end of the program.
Print out intermediate data values at different places in the data steps to show the effect of different sections of the code.
Fix any issues in the program and then start again from step #2 until no more problems are found.
Remove, or at least comment out, any additional debugging code used.

However, at this point I must emphasize that not all programming problems can be resolved using these steps, but these steps should be tried first before asking for help.

So how should help be requested? By following steps 1-6 above you will have already collected a lot of information that could be vital for another programmer to resolve your problem, so don’t ignore any of it, remembering that any particular bit of information may be the most important bit required to solve your problem. Therefore, most, if not all, of the following information should be included in every request for assistance:

A description of the expected output from your program.
The reason you chose to write the program in this way.
All of the program code, because what you think is the cause of any problems may not be the only cause.
All of the sample data records, or the real data records if they are causing problems and can be posted.
The warning and error messages from the log (at the very least!).

Comments can be made by registered members in a copy of this post in the SAS Skills Discussion Forum.

Have You Forgotten Your Password?

The registration page on my blog site encourages new members to create at least a “Medium” strength password. This is good for the security of their personal information, but not so good if they follow the general security advice by not writing that password on a sticky note and attaching it to their computer screen or tablet. Unfortunately, no-one has a perfect memory, and the Password Reset functionality in WordPress is dependent on a number of different factors too, including the site’s email setup, the email address used by the site, the rules of the email server used by the site, and the rules used by the member’s email server. Actually it is quite astonishing that it works at all, which is why I’d like to make you all aware of a more reliable method, at least as a last resort, in case you ever forget your password:

Go to my blog site at http://blog.hollandnumerics.org.uk and click on the [Password Reset] link in the menu (under [Member Login]).
Type in the email address you used when you registered on my blog site, or the most recently email address you’ve registered since joining, and click the [RESET PASSWORD] button.
You should receive an email containing a link to the password reset page within a few minutes, and using that link should allow you to change your password to something you can remember.
If you don’t receive an email within a few minutes, or the link in the email doesn’t seem to work, then you can contact the site administrator (that’s me!) by using the [Contact us] link, remembering to include the same email address you used on the site.
I will send you back a freshly generated and fiendishly strong password by email with instructions to use it immediately to log in.
Once you have logged in use the [Edit Profile] link in the menu (under [Member Login]) to update your password to something you can remember.

Note that I can check whether it really is you trying to update your password, and it will only be sent back to your registered email address. Note also that I’m based in the UK, so don’t expect a lightning fast response when it is 3am GMT. In return I will try to send you your new password at a reasonable time of the day for you too.

Please note that if you just want to change your password, but can still log in, starting at step #6 will work just fine for you!

My blog and forums are open for business in their new premises!

On 28 March 2017 I completed the migration of my blog and forums to a new server. You will not have to do anything different to access the new server, as it is all managed transparently by my router.

The new environment is protected by an uninterruptible power supply, and has an increase in its available disk space. I’m hoping that the chance of future interruptions has been dramatically reduced.

Alongside the blog are 2 forums where you can ask questions:

(1) The SAS Skills Discussion Forum is included when you register for the Free membership, and is where you can ask questions about SAS skills, but not SAS coding problems.

(2) The SAS Programming Forum, which has a very small subscription cost, is where you can ask SAS coding questions, including those from homework and interviews. This forum also includes a growing SAS course in the form of small topics. There are currently 3 main sections to the SAS course: Data Steps, PROC SQL and macros, but more are being developed.

Preventing Spam and SAS Blog Server Downtime – and how you can help!

On 13 March 2017 my blog site started getting user registrations from spam bots all with first name equal to last name. I was leaving to travel to the SAS Global Forum in Orlando the following day, so I quickly set the default membership level to “Pending” to prevent too much damage, as “Pending” members only have the same blog site access as unregistered visitors. Recently new members may have noticed a delay in getting full access to the site, and this is because your membership is initially “Pending” until I manually change it to “Active”. The following day more spam registrations occurred, but then on the day after that my blog server decided to go into stand-by mode, shutting down the web site, for reasons I still don’t understand. Under normal circumstances I would have been upset by this, but, given the likely influx of many more spam registrations, it was actually a relief, as I would have limited access to WiFi while travelling, and so I was able to relax a little.On 1 April 2017, when I’d arrived at the conference, I asked my daughter to restart the blog server, so it would be available for new registrations again, and I was then able to track the unwelcome new members more easily using the hotel and conference WiFi.

Where were the IP addresses for spam registrations located?

Former Soviet Union – 53.5%
EU and UK – 22.0%
North America – 11.5%
Rest of the World – 8.5%
Unknown – 4.5%

The IP addresses of the spam registrations suggest they are mostly from countries in the former Soviet Union, although otherwise spread around the world.

Email domains in spam registrations:

mail.ru – 31.5%
yandex.com – 6.0%
yandex.ru – 5.0%
other *.ru, *.ua, *.ee – 3.0%
other *.top – 16.5%
other email domains – 39.0%

At first sight you would assume that the former Soviet email addresses matched the former Soviet locations, but that was definitely not the case, as *.ru and yandex.com email addresses were associated with locations in countries spread right across the rest of the globe. It appears that these IP addresses are being used remotely for spam registration, possibly without even the knowledge of their owners. Wordfence, a security company for WordPress sites, has published an article about how some home routers could be compromised, how to check for this vulnerability, and how to protect yourself. I strongly recommend that you read this article and protect yourself along with WordPress users like myself! This is how you can help.

So what have I done to resolve all of these problems? First of all I have ordered a new server and uninterruptible power supply, so that going into stand-by mode and power cuts should not interrupt your connection to the server in the future, other than in exceptional circumstances. The site should be moved over to the new server in the next few weeks. Secondly I installed a new WordPress plug-in called “WP Spam-Shield” on 14 April 2017, which has miraculously stopped all of the spam registrations! As a consequence I have now reset the default setting for new membership from “Pending” to “Active”, so new members won’t have to wait for me to activate their accounts.

What happened in Orlando?

As usual I ran a prize draw this year at SAS Global Forum in Orlando for a copy of my latest book, which was won by Matthew Hoolsema from Carnegie Mellon University from 49 draw entries.

Matthew Hoolsema won a copy of my book in Orlando

The sad part was that my well-thumbed sample copy, which allowed everyone to see what was in the book, was taken during the conference, so I will have to replace it with another brand new copy before my next free draw, instead of using that new copy as a prize. I must admit that I find it extremely annoying when a company can pay $100s to $1,000s for the conference registration, travel and accommodation, but nothing for a $40 book!

The conference’s Kick-Back party was held at Disney Hollywood Studios at the end of the 2nd full day of the conference after the public had left. I’d last visited this park in 1999, when it was called MGM Studios. Some of the “exciting” rides, which I have never enjoyed, were open, and my favourite show from 1999, “Indiana Jones Stunt Spectacular”, was still there, but disappointingly closed for the evening! Fortunately, unlike previous Kick-Back parties, the noise levels were low enough to permit normal conversations, so my voice was still OK the following morning for my presentation. You may remember that in March I said that I was presenting “Making Validation of Graphs Easier: The Benefits of ODS Graphics” at the conference on 5 April 2017. The video recording of my presentation can now be viewed on the SAS web site, along with several of the other presentations, and my paper, slides and sample code can be downloaded from this blog site.

Next year SAS Global Forum will be in Denver, Colorado. I’ve never been there before, so I’m looking for some suitable topics for new presentations. Any suggestions?

Why don’t SAS development teams talk to each other?

We all know that SAS develops its software in separate teams, but it can be really annoying when it becomes apparent that several associated teams haven’t planned together how a SAS procedure will work.

I’m going to take as an example PROC IMPORT, which is part of Base SAS, but is also included in SAS/ACCESS. When you run the following program all the variables created begin with VAR, i.e. VAR1, VAR2, VAR3, VAR4, VAR5, etc., and this would also be true for DBMS=TAB and DLM:

PROC IMPORT FILE = "test.csv" DBMS = CSV REPLACE;
  GETNAMES = NO;
RUN;

However, using similar PROC IMPORT code for DBMS=EXCEL, in SAS/ACCESS for PC Files, will create variables beginning with F, i.e. F1, F2, F3, F4, F5, etc.:

PROC IMPORT FILE = "test.xls" DBMS = EXCEL REPLACE;
  GETNAMES = NO;
RUN;

More shocking though is using PROC IMPORT code for DBMS=XLS or XLSX in UNIX or Windows, in SAS/ACCESS for PC Files, as this will create variables with no prefix at all, i.e. A, B, C, D, E, etc.:

PROC IMPORT FILE = "test.xls" DBMS = XLS REPLACE;
  GETNAMES = NO;
RUN;

This inconsistency even extends to using GETNAMES = YES too when there are multiple columns with the same label.

If you want to import a CSV file, instead of an Excel file, or indeed import an Excel file in UNIX, then the subsequent processing step will have to be updated to use the new variable names (annoying!). Why can’t the procedure be consistent, or, at least, have a parameter, like PREFIX=, that allows users to choose the prefix?

Back in SAS 6 there were monumental problems caused by each SAS procedure having to manage their own output destinations, and each one doing it in a slightly different way, so ODS (Output Delivery System) was introduced in SAS 7 to make the output interface separate from the procedures. SAS now need to develop standards for parameters and defaults, just like they did for output!

To vote for this to be included in the next SASware Ballot just click here and vote up the idea!

Warning: you may need to use a desktop browser to vote up the idea! 🙁

Do you have any examples of similar inconsistencies in SAS?