Holland Numerics: Blog and Forums

Premium SAS Support for Commercial Clients

Free membership gives you access to the SAS Skills Discussion Forum for non-programming questions about SAS. Programming and Recruiter memberships, which require monthly low-cost subscriptions, give you access to the same forum and to the SAS Programming Forum, where you can ask SAS programming questions. However, all questions and answers are visible to all members with access to each forum, which may not be quite what you are looking for.

To fulfill any need for confidential correspondence you may have seen a recent addition to the blog menu. The Premium SAS help desk service, called [sas.answers @ Holland Numerics] (which has actually been around since 2002), is different. It does not require even Free membership of the blog, but it does require a separate subscription payable using PayPal. What makes this service special is that it provides an in-house web-based interface for you to ask questions, which automatically generates an SMS message and an email to Holland Numerics, with a copy of the email sent back to you too. There is also a service level agreement to provide an answer to every question within 48 hours, and that answer will be privately sent directly back to the sending subscriber! Please note that, if a monthly subscription is not necessary, then a single question can be asked instead for a reduced fee.

Full details of the [sas.answers @ Holland Numerics] help desk service can be found here.

How Do You Successfully Debug Your Program?

For a long while now I have been reading discussions on a range of social media platforms where programmers have been asking for help with programs that don’t behave as expected. Many of these programmers have certificates relating to the relevant programming languages, yet appear to have never been taught how to debug a program. To me the ability to resolve issues in a program is the most vital skill a programmer of every programming languages has to possess, otherwise they simply cannot call themselves a “programmer” in that language. Without this skill I would only call them a “code writer”.

So is this ability to resolve issues in a program a difficult skill to learn? I would say no, because it just involves the following a set of simple instructions, with a little observation and logic. Note that these instructions are not at all specific to SAS programming, or indeed to any other programming language:

Talk through the logic of the program with a colleague. Quite often you can spot potential problems yourself while trying to explain the logic to another programmer.
Run the program, and then check the log for error and warning messages, as there should be none!
Look for unexpected empty intermediate data sets in the log.
Test your code with small data subsets first, so that you can follow the processing path of individual data records. These subsets ought to be special sample records which will exercise as much of the logic within the program as possible.
Print out intermediate data sets, especially if these are not kept at the end of the program.
Print out intermediate data values at different places in the data steps to show the effect of different sections of the code.
Fix any issues in the program and then start again from step #2 until no more problems are found.
Remove, or at least comment out, any additional debugging code used.

However, at this point I must emphasize that not all programming problems can be resolved using these steps, but these steps should be tried first before asking for help.

So how should help be requested? By following steps 1-6 above you will have already collected a lot of information that could be vital for another programmer to resolve your problem, so don’t ignore any of it, remembering that any particular bit of information may be the most important bit required to solve your problem. Therefore, most, if not all, of the following information should be included in every request for assistance:

A description of the expected output from your program.
The reason you chose to write the program in this way.
All of the program code, because what you think is the cause of any problems may not be the only cause.
All of the sample data records, or the real data records if they are causing problems and can be posted.
The warning and error messages from the log (at the very least!).

Comments can be made by registered members in a copy of this post in the SAS Skills Discussion Forum.

Have You Forgotten Your Password?

The registration page on my blog site encourages new members to create at least a “Medium” strength password. This is good for the security of their personal information, but not so good if they follow the general security advice by not writing that password on a sticky note and attaching it to their computer screen or tablet. Unfortunately, no-one has a perfect memory, and the Password Reset functionality in WordPress is dependent on a number of different factors too, including the site’s email setup, the email address used by the site, the rules of the email server used by the site, and the rules used by the member’s email server. Actually it is quite astonishing that it works at all, which is why I’d like to make you all aware of a more reliable method, at least as a last resort, in case you ever forget your password:

Go to my blog site at http://blog.hollandnumerics.org.uk and click on the [Password Reset] link in the menu (under [Member Login]).
Type in the email address you used when you registered on my blog site, or the most recently email address you’ve registered since joining, and click the [RESET PASSWORD] button.
You should receive an email containing a link to the password reset page within a few minutes, and using that link should allow you to change your password to something you can remember.
If you don’t receive an email within a few minutes, or the link in the email doesn’t seem to work, then you can contact the site administrator (that’s me!) by using the [Contact us] link, remembering to include the same email address you used on the site.
I will send you back a freshly generated and fiendishly strong password by email with instructions to use it immediately to log in.
Once you have logged in use the [Edit Profile] link in the menu (under [Member Login]) to update your password to something you can remember.

Note that I can check whether it really is you trying to update your password, and it will only be sent back to your registered email address. Note also that I’m based in the UK, so don’t expect a lightning fast response when it is 3am GMT. In return I will try to send you your new password at a reasonable time of the day for you too.

Please note that if you just want to change your password, but can still log in, starting at step #6 will work just fine for you!

My blog and forums are open for business in their new premises!

On 28 March 2017 I completed the migration of my blog and forums to a new server. You will not have to do anything different to access the new server, as it is all managed transparently by my router.

The new environment is protected by an uninterruptible power supply, and has an increase in its available disk space. I’m hoping that the chance of future interruptions has been dramatically reduced.

Alongside the blog are 2 forums where you can ask questions:

(1) The SAS Skills Discussion Forum is included when you register for the Free membership, and is where you can ask questions about SAS skills, but not SAS coding problems.

(2) The SAS Programming Forum, which has a very small subscription cost, is where you can ask SAS coding questions, including those from homework and interviews. This forum also includes a growing SAS course in the form of small topics. There are currently 3 main sections to the SAS course: Data Steps, PROC SQL and macros, but more are being developed.

Preventing Spam and SAS Blog Server Downtime – and how you can help!

On 13 March 2017 my blog site started getting user registrations from spam bots all with first name equal to last name. I was leaving to travel to the SAS Global Forum in Orlando the following day, so I quickly set the default membership level to “Pending” to prevent too much damage, as “Pending” members only have the same blog site access as unregistered visitors. Recently new members may have noticed a delay in getting full access to the site, and this is because your membership is initially “Pending” until I manually change it to “Active”. The following day more spam registrations occurred, but then on the day after that my blog server decided to go into stand-by mode, shutting down the web site, for reasons I still don’t understand. Under normal circumstances I would have been upset by this, but, given the likely influx of many more spam registrations, it was actually a relief, as I would have limited access to WiFi while travelling, and so I was able to relax a little.On 1 April 2017, when I’d arrived at the conference, I asked my daughter to restart the blog server, so it would be available for new registrations again, and I was then able to track the unwelcome new members more easily using the hotel and conference WiFi.

Where were the IP addresses for spam registrations located?

Former Soviet Union – 53.5%
EU and UK – 22.0%
North America – 11.5%
Rest of the World – 8.5%
Unknown – 4.5%

The IP addresses of the spam registrations suggest they are mostly from countries in the former Soviet Union, although otherwise spread around the world.

Email domains in spam registrations:

mail.ru – 31.5%
yandex.com – 6.0%
yandex.ru – 5.0%
other *.ru, *.ua, *.ee – 3.0%
other *.top – 16.5%
other email domains – 39.0%

At first sight you would assume that the former Soviet email addresses matched the former Soviet locations, but that was definitely not the case, as *.ru and yandex.com email addresses were associated with locations in countries spread right across the rest of the globe. It appears that these IP addresses are being used remotely for spam registration, possibly without even the knowledge of their owners. Wordfence, a security company for WordPress sites, has published an article about how some home routers could be compromised, how to check for this vulnerability, and how to protect yourself. I strongly recommend that you read this article and protect yourself along with WordPress users like myself! This is how you can help.

So what have I done to resolve all of these problems? First of all I have ordered a new server and uninterruptible power supply, so that going into stand-by mode and power cuts should not interrupt your connection to the server in the future, other than in exceptional circumstances. The site should be moved over to the new server in the next few weeks. Secondly I installed a new WordPress plug-in called “WP Spam-Shield” on 14 April 2017, which has miraculously stopped all of the spam registrations! As a consequence I have now reset the default setting for new membership from “Pending” to “Active”, so new members won’t have to wait for me to activate their accounts.

What happened in Orlando?

As usual I ran a prize draw this year at SAS Global Forum in Orlando for a copy of my latest book, which was won by Matthew Hoolsema from Carnegie Mellon University from 49 draw entries.

Matthew Hoolsema won a copy of my book in Orlando

The sad part was that my well-thumbed sample copy, which allowed everyone to see what was in the book, was taken during the conference, so I will have to replace it with another brand new copy before my next free draw, instead of using that new copy as a prize. I must admit that I find it extremely annoying when a company can pay $100s to $1,000s for the conference registration, travel and accommodation, but nothing for a $40 book!

The conference’s Kick-Back party was held at Disney Hollywood Studios at the end of the 2nd full day of the conference after the public had left. I’d last visited this park in 1999, when it was called MGM Studios. Some of the “exciting” rides, which I have never enjoyed, were open, and my favourite show from 1999, “Indiana Jones Stunt Spectacular”, was still there, but disappointingly closed for the evening! Fortunately, unlike previous Kick-Back parties, the noise levels were low enough to permit normal conversations, so my voice was still OK the following morning for my presentation. You may remember that in March I said that I was presenting “Making Validation of Graphs Easier: The Benefits of ODS Graphics” at the conference on 5 April 2017. The video recording of my presentation can now be viewed on the SAS web site, along with several of the other presentations, and my paper, slides and sample code can be downloaded from this blog site.

Next year SAS Global Forum will be in Denver, Colorado. I’ve never been there before, so I’m looking for some suitable topics for new presentations. Any suggestions?

Why don’t SAS development teams talk to each other?

We all know that SAS develops its software in separate teams, but it can be really annoying when it becomes apparent that several associated teams haven’t planned together how a SAS procedure will work.

I’m going to take as an example PROC IMPORT, which is part of Base SAS, but is also included in SAS/ACCESS. When you run the following program all the variables created begin with VAR, i.e. VAR1, VAR2, VAR3, VAR4, VAR5, etc., and this would also be true for DBMS=TAB and DLM:

PROC IMPORT FILE = "test.csv" DBMS = CSV REPLACE;
  GETNAMES = NO;
RUN;

However, using similar PROC IMPORT code for DBMS=EXCEL, in SAS/ACCESS for PC Files, will create variables beginning with F, i.e. F1, F2, F3, F4, F5, etc.:

PROC IMPORT FILE = "test.xls" DBMS = EXCEL REPLACE;
  GETNAMES = NO;
RUN;

More shocking though is using PROC IMPORT code for DBMS=XLS or XLSX in UNIX or Windows, in SAS/ACCESS for PC Files, as this will create variables with no prefix at all, i.e. A, B, C, D, E, etc.:

PROC IMPORT FILE = "test.xls" DBMS = XLS REPLACE;
  GETNAMES = NO;
RUN;

This inconsistency even extends to using GETNAMES = YES too when there are multiple columns with the same label.

If you want to import a CSV file, instead of an Excel file, or indeed import an Excel file in UNIX, then the subsequent processing step will have to be updated to use the new variable names (annoying!). Why can’t the procedure be consistent, or, at least, have a parameter, like PREFIX=, that allows users to choose the prefix?

Back in SAS 6 there were monumental problems caused by each SAS procedure having to manage their own output destinations, and each one doing it in a slightly different way, so ODS (Output Delivery System) was introduced in SAS 7 to make the output interface separate from the procedures. SAS now need to develop standards for parameters and defaults, just like they did for output!

To vote for this to be included in the next SASware Ballot just click here and vote up the idea!

Warning: you may need to use a desktop browser to vote up the idea! 🙁

Do you have any examples of similar inconsistencies in SAS?

My published Training Course list is for companies, the SAS course is for individuals

I published a list of available training courses for 2017 in January this year, but this course list is actually intended for companies, and not for individuals, as the courses have a fixed price no matter how many people attend them. All the training materials are personalised and supplied in digital form, and so can be duplicated for that client to re-use internally.

Individuals looking for SAS training should subscribe to the SAS course in the SAS Programming Forum instead, which is specifically targeted at individual SAS learners and programmers, and is priced as a low-cost monthly subscription. However, the SAS Programming Forum is not just there for the SAS course, but can be used to get answers to any SAS-related questions you may have too, even if they are from homework or interview questions.

See “Making Graphs Easier to Validate – The Benefits of ODS Graphics” at SAS Global Forum in Orlando

Are you attending SAS Global Forum in April this year? If you are then you have the opportunity to see me present “Making Graphs Easier to Validate – The Benefits of ODS Graphics” on Wednesday 5 April from 1100-1150hr. I’ll be in the Americas Seminar Room on Dolphin Level 5 at the Walt Disney World Swan and Dolphin Resort in Orlando, Florida. In this presentation I’ll be talking about how to make your graphs easier to validate by using ODS Graphics and a lot of common sense!

However, maybe you are not going to be in Orlando in April, but you and your colleagues would still like to see me present on this topic. What can you do about that? Well it should still be possible, because I’m currently developing a 1/2 day training session which I’m calling “Defensive SAS Programming”. This will include “Making Graphs Easier to Validate – The Benefits of ODS Graphics”, but also “Writing Reusable Macros” and another new topic “The Art of Defensive Programming: How to Cope with Unseen Data”.

You’ll see from the link above that I don’t have to be in the room with you to present, therefore I could present through your company’s video-conferencing system from my own office at home to your conference room wherever you are instead. The “Available” and “Coming Soon” training sessions can all be booked in advance, but the “Coming Soon” training sessions will be developed specifically for you, if you are the first to request them.

Going back to “Making Graphs Easier to Validate – The Benefits of ODS Graphics”, this presentation and paper will be available for download from the Product Shop shortly after it is presented for the first time, but may also be available from the SAS Global Forum site before the conference too!

I’m looking forward to SAS Global Forum, and hope to see you there, or elsewhere in the near future!

There is now a new Training Course list for 2017

There is now a new Training Course list for 2017, which can be downloaded from here. The courses available in 2016 are still there, but I’m developing some new SAS-related courses, based on the SAS course, which you can accelerate to production status by requesting them:

½ day SAS Data Step training
½ day SAS PROC SQL training
½ day SAS macros training
½ day Defensive SAS Programming training

Your interest in any of these courses will result in them being developed as priority tasks!

Fast – Cheap – Good – Pick any 2!

Fast - Cheap - Good I’m sure you’ve seen this sign many times before, but it is a very important statement!

At Holland Numerics we strive to provide good service at all times, so we will never even consider providing fast service cheap, because it will not be good! That said, we can provide good service fast, but it won’t be cheap, and good service cheap, but it won’t be fast.

Sometimes we have to sit back and consider Oscar Wilde’s view (from his play “Lady Windemere’s Fan”) that at his point in time a cynic, but now everyone, knows the price of everything, and the value of nothing! This is particularly relevant for consultancies like Holland Numerics trying to determine appropriate pricing for consulting services.

That is why the SAS Programming Forum is accessed for a small subscription, not because we need it to be an income stream, but because we want to make certain that subscribers really want to access the valuable SAS-related information I have added over several months in that forum. I like to think this is perfectly exemplified by another quote, this time from my all-time favourite novel “Time Enough for Love” by Robert A Heinlein:

“Anything free is worth what you pay for it.”

After a period of instability my SAS blog should have more availability, and may be quicker too!

Following my SAS blog server crash in September I have experienced occasional access interruptions from a few minutes to several hours since then. There have been various reasons for the down time, including WordPress plugin updates causing incompatibilities, server software updates, accidental disconnecting of the router, and my internet service provider temporarily interrupting their service to me.

To correct the last problem I decided to change my broadband connection from ADSL to a “Superfast” fibre, which meant changing my provider. This has now been completed, but the changeover process did more than change my internet service provider. While running the two broadband connections in parallel I discovered some issues in the setup of the network gateway, which had itself caused a number of interruptions. I can’t guarantee that there won’t be more short interruptions in the future, as I’m in the process of reorganising the cabling in my office, but I wanted to assure everyone that, if you have difficulty connecting, then wait a short while before trying again, as I will be back, and the blog could be quicker too!