Title: So Your Code is NOT Critical?

Most of the articles about critical software consider, quite properly, the "obvious" mission critical areas such as control systems for Nuclear Power Stations, medical equipment and Air Traffic Control. I am not for one moment implying that these are not far more serious than the experience I have just suffered but I would like to show how a very much wider range of software is critical to someone than perhaps most of us realise.

Like many small business I have a presence on the Internet, this is in the form of a domain name, a Webb site and email facilities. In keeping with most small businesses a dial up account provided by one of the many ISP's in this country implements this. I pay for these services which include the ISP taking care of the incoming and outgoing emails. My incoming mail is addressed to but this needs to finish up at , clearly the ISP has a range of databases and other tables which are used to sort this out. As most ISP's have this problem I assumed that the software for such actions would be well established, robust and reliable, as would the software for keeping the databases up to date.

For over two years the system had worked, there were some lost emails but my ISP always maintained A) that he had not lost them and B) he was not guaranteeing to deliver all my mail any way. As I had a much lower incidence of problems with the email than with Royal Mail I tolerated the situation until the end of last month. Indeed I have gradually moved to a situation where most of my business communication is now done by email except for the invoice and the cheque.

One morning, just as I was leaving for an important appointment, I had a phone call from a supplier to check my email address as some mail to me had been bounced. I gave an alternate email address without thinking and rushed to get my train. That night I checked my email, including sending myself an email and everything seemed to be working. At 7pm the next day I got a phone call from one of my clients, he was having emails to me bounced. As this was one of my very computer literate clients I thought something must be wrong so I called my ISP "help line". With the normal patronising attitude of these guys I was told everything was working well and that probably my email software was badly configured! Alternatively it might be that my clients had the wrong address, my advice that they had been using the same address successfully for 18 months was not deemed to be proof that they actually had the correct address.

The next day I had four calls in a short period of time to tell me my email was being bounced, one from our own Peter Hamilton. Now when people of Peter's experience have trouble with my email it is real trouble, this time the "help desk" admitted to a "slight problem with the server" which would be fixed within the hour. It took 8 days before they fixed it and the service was poor for about 4 more days.

The reason for all this, they admitted that they wanted to update the system for maintaining the various databases etc that handled the domain names. However the new software was only tested on a small set of accounts and it appeared that when it was run on the full set many had been corrupted. This is what we think happened, based on the announcements made by the ISP.

So here we have a small "housekeeping" task that destroys the email service to several hundred small businesses for at least 7 days and in practice for longer. No ones life was endangered, no one nearly crashed a car but the impact was considerable. I have had to spend a long time checking with people that might have sent me emails, I have had to resubscribe to lists that automatically delete me when an email bounces. I have had to explain to at least one client that I have not stopped trading. All in all I wasted about 4 days of my time, spent a lot of time on the phone trying to get real information out of the "help line" and generally been properly messed about. At my charging rate the cost to me alone in time was about 5 years subscription to my ISP. I may have lost business, I will never know who tried to email me and took the bounce as a sign that I had ceased trading.

And as I say this is not "critical" code.

There are other lessons to be learnt from this fiasco, many are best reported in other forums but one seems to be very important. There is a maxim in business "Never single source a key resource", I slipped up and single sourced my email provision. I now have a completely independent email address, with so many free ISP's it is trivial to get an extra email post box. If addresses like seem not to have the right aura it is worth noting that many professional organisations offer their members mail forwarding. The IOP and the IEE do and I believe many others do. Hence it is easy to set up which points to a less salubrious address. I have an IEE email address, but guess where it used to point, yes which is where all my email was being bounced.

I would be very tempted to talk with my lawyers. Bouncing of mail sent to a business is close to defamation of character. It carries a strong implication that the business may no longer be trading. What makes this case more significant is the IP's initial refusal to accept any responsibility. Add in over a week of bouncing mail and we have a serious degree of incompetence.

By the way, this further emphasises my belief that you should always include alternative ways of contact in your signature. At least potential clients have a way of checking who is incompetent Francis.