Apologies for the 16 hour server outage today. Note to self: if an installation manual doesn’t mention something important, then don’t try it out in production!

Loading

Apologies for the 16 hour server outage on my SAS-related blog site today. I was trying to install Jitsi Meet, an open-source alternative to Zoom, on my server:

  • While it is possible to organise moderated meeting using the hosted meet.jit.si servers, installing a self-hosted copy of the open-source Jitsi Meet software alongside WordPress seemed to me to be a good idea, so that SAS training could easily be held locally.
  • By default standard self-hosted Jitsi Meet uses Nginx, not Apache, web servers, and assumes a fully qualified hostname, e.g. meet.hollandnumerics.org.uk.
  • What I wanted to create was an installation of Jitsi Meet in a sub-folder on my existing Apache web site, e.g. hollandnumerics.org.uk/meet, which is also part of a Bitnami stack, where software is installed specifically for use by the web server. This use case was not mentioned anywhere in the installation manual, although I did find a number of web forums that described similar environments.
  • That is where the problems started:
    1. To move Jitsi Meet into a sub-folder I had to edit the existing web configuration, the web pages to display the meetings, and the scripts to start the software.
    2. I also had to generate a self-signed SSL certificate for localhost.
    3. I had to reboot the server and restart the web service several times.
    4. Finally I discovered that some of my changes were lost whenever the Jitsi Meet software was updated.
  • At no point did I ever get Jitsi Meet to a point where I could run a test meeting, so I uninstalled it all.
  • Having uninstalled Jitsi Meet, I then couldn’t restart the web server to access WordPress, because something was holding onto port 443 (the secure HTTP port). At this point it was fast approaching midnight, so I decided to sleep on it.
  • Finally, this morning, I found that, when Jitsi Meet had been installed, it had added “Listen 443” commands to the default Apache installation, which conflicted with my Bitnami stack Apache server. Once these were removed my WordPress site suddenly reappeared!

Lessons learned:

  • When installing software you don’t fully understand, if the installation instructions have no information explaining what you should do, don’t just do it anyway!

Thank you for your patience!

Apologies for the 9 hour server outage today, but I now know more about Apache and PHP!

Loading

Apologies for the 9 hour server outage on my SAS-related blog site today, but I now know more about the relationships between Apache and PHP:

  1. The server’s Ubuntu operating system was upgraded from 16.04 to 18.04 on 18Aug2019.
  2. The Apache WordPress server was unaffected, but my Apache cloud server refused to start due to a PHP incompatibility.
  3. The cloud server software was then upgraded to use PHP7.2 instead of PHP7.0, but the server still refused to start.
  4. More PHP7.2 modules were downloaded and installed.
  5. The PHP configuration in the Apache cloud server was changed from PHP7.0 to PHP7.2, and it now started.
  6. At some point in the updates the listening ports on the Apache WordPress server were also updated, so, when it was automatically restarted at midnight, it immediately failed due to duplicate ports.
  7. Finally, this morning, I corrected the listening ports on the Apache WordPress server, which is how I’ve been able to write this blog post.

Thank you for your patience!

Scheduled blog server maintenance in the evening of 17Jul2018 (UK time)

Loading

My blog server runs WordPress software, which I can update hourly while the server is running, if necessary. However, the underlying Apache, PHP and MySQL software requires server downtime to update, so I have scheduled their updating to take place in the evening (UK time) of Tuesday 17 July 2018. I am hoping it will only take 1-2 hours to complete, but, having never done this big an update before, I can’t be more precise.

Beware of the 14th of June as the power supply for my SAS Blog server is being upgraded

Loading

On the 14Jun2018 my power supply is being upgraded, which may cause server interruptions between 1200hr and 1600hr (UK time). I have an interruptible power supply attached to the blog server, but, while I’ve tested it for short interruptions, I can’t guarantee that the power supply will smooth over all of the disconnections for power upgrade.

Please accept my apologies if you are impacted.

If there is any interruption (to quote the movies) I will be back!!

My blog and forums are open for business in their new premises!

Loading

On 28 March 2017 I completed the migration of my blog and forums to a new server. You will not have to do anything different to access the new server, as it is all managed transparently by my router.

The new environment is protected by an uninterruptible power supply, and has an increase in its available disk space. I’m hoping that the chance of future interruptions has been dramatically reduced.

Alongside the blog are 2 forums where you can ask questions:

(1) The SAS Skills Discussion Forum is included when you register for the Free membership, and is where you can ask questions about SAS skills, but not SAS coding problems.

(2) The SAS Programming Forum, which has a very small subscription cost, is where you can ask SAS coding questions, including those from homework and interviews. This forum also includes a growing SAS course in the form of small topics. There are currently 3 main sections to the SAS course: Data Steps, PROC SQL and macros, but more are being developed.

Preventing Spam and SAS Blog Server Downtime – and how you can help!

Loading

On 13 March 2017 my blog site started getting user registrations from spam bots all with first name equal to last name. I was leaving to travel to the SAS Global Forum in Orlando the following day, so I quickly set the default membership level to “Pending” to prevent too much damage, as “Pending” members only have the same blog site access as unregistered visitors. Recently new members may have noticed a delay in getting full access to the site, and this is because your membership is initially “Pending” until I manually change it to “Active”. The following day more spam registrations occurred, but then on the day after that my blog server decided to go into stand-by mode, shutting down the web site, for reasons I still don’t understand. Under normal circumstances I would have been upset by this, but, given the likely influx of many more spam registrations, it was actually a relief, as I would have limited access to WiFi while travelling, and so I was able to relax a little.On 1 April 2017, when I’d arrived at the conference, I asked my daughter to restart the blog server, so it would be available for new registrations again, and I was then able to track the unwelcome new members more easily using the hotel and conference WiFi.

Where were the IP addresses for spam registrations located?

  • Former Soviet Union – 53.5%
  • EU and UK – 22.0%
  • North America – 11.5%
  • Rest of the World – 8.5%
  • Unknown – 4.5%

The IP addresses of the spam registrations suggest they are mostly from countries in the former Soviet Union, although otherwise spread around the world.

Email domains in spam registrations:

  • mail.ru – 31.5%
  • yandex.com – 6.0%
  • yandex.ru – 5.0%
  • other *.ru, *.ua, *.ee – 3.0%
  • other *.top – 16.5%
  • other email domains – 39.0%

At first sight you would assume that the former Soviet email addresses matched the former Soviet locations, but that was definitely not the case, as *.ru and yandex.com email addresses were associated with locations in countries spread right across the rest of the globe. It appears that these IP addresses are being used remotely for spam registration, possibly without even the knowledge of their owners. Wordfence, a security company for WordPress sites, has published an article about how some home routers could be compromised, how to check for this vulnerability, and how to protect yourself. I strongly recommend that you read this article and protect yourself along with WordPress users like myself! This is how you can help.

So what have I done to resolve all of these problems? First of all I have ordered a new server and uninterruptible power supply, so that going into stand-by mode and power cuts should not interrupt your connection to the server in the future, other than in exceptional circumstances. The site should be moved over to the new server in the next few weeks. Secondly I installed a new WordPress plug-in called “WP Spam-Shield” on 14 April 2017, which has miraculously stopped all of the spam registrations! As a consequence I have now reset the default setting for new membership from “Pending” to “Active”, so new members won’t have to wait for me to activate their accounts.

After a period of instability my SAS blog should have more availability, and may be quicker too!

Loading

Following my SAS blog server crash in September I have experienced occasional access interruptions from a few minutes to several hours since then. There have been various reasons for the down time, including WordPress plugin updates causing incompatibilities, server software updates, accidental disconnecting of the router, and my internet service provider temporarily interrupting their service to me.

To correct the last problem I decided to change my broadband connection from ADSL to a “Superfast” fibre, which meant changing my provider. This has now been completed, but the changeover process did more than change my internet service provider. While running the two broadband connections in parallel I discovered some issues in the setup of the network gateway, which had itself caused a number of interruptions. I can’t guarantee that there won’t be more short interruptions in the future, as I’m in the process of reorganising the cabling in my office, but I wanted to assure everyone that, if you have difficulty connecting, then wait a short while before trying again, as I will be back, and the blog could be quicker too!

Apologies for my blog server crash – normal service has now been restored

Loading

I suspect many of you will have a “bucket list”, a list of places you would really like to visit and activities you would like to do. I have bucket list myself, which included visiting the Alhambra in Granada in Southern Spain.  Last week my wife and I joined a tour of Andalusia, which included a visit to the Alhambra on the first day, where we enjoyed the splendour of the Moorish palace.

Little did I know at the time, but just after we had left for our tour my blog server switched itself off. I have still not determined the exact reason for the crash, but I think it could have been a local power cut that caused the shutdown. Anyway, I was 900 miles away with no remote access to the server, so I had to relax and enjoy the rest of the tour. On returning to the UK it was simply a matter of restarting the server, and normal service was restored in a few minutes.

I will conclude with some simple questions for you. I could have bought and installed an uninterruptible power supply to the server, which would have, hopefully, prevented the crash. However, I would not have been able to tell you about my bucket list!

My questions for you are:

  • What is the value of a blog, more or less than the cost of purchasing and running an uninterruptible power supply, or flying back to fix it?
  • Would a week where the blog is unavailable affect its value?
  • Should I have told you all about the crash immediately, or when it was fixed?