Title: Don’t Brush Bugs Under The Carpet

Author: Bob Schmidt

Date: 04 November 2018 17:36:29 +00:00 or Sun, 04 November 2018 17:36:29 +00:00

Summary: Silas S.Brown presents an allegorical lesson on bug reports.

Body: 

Once upon a time, there was a doctor who was so wonderfully good at curing the sick that he couldn’t be replaced. Unfortunately, he also had a psychopathic irrational hatred for red clothes. If any patient came in wearing red clothes, he would kill the patient instead of curing them.

Question: what should be done about this?

• Option A: get rid of the doctor and try to find a replacement, no matter how long that takes and how many patients go without treatment in the meantime.
• Option B: try to fix the doctor. Get a competent analyst to help him figure out why he goes crazy at red clothes and address it so he doesn’t do it again.
• Option C: post a guard outside the doctor’s office. The guard will not let anyone in if they’re wearing red clothing.
• Option D: post a warning sign outside the doctor’s office, hoping that wearers of red clothing will read the sign and change their clothes before entering.
• Option E: do nothing. I personally will not wear red clothes, and I’m the only patient that matters, so problem solved.

Unfortunately, I have twice now encountered European software users who seem to think the answer is E. In both cases, the issue was that they can cause a program to crash (only on the Microsoft Windows platform) by giving it input that involves an accented letter. Once they realised that it was the accented letter causing the crash, they simply said “Oh yes, we shouldn’t expect English software to cope with our accents, we’ll just change our input and you don’t have to fix anything.” They weren’t even willing to show me what the original input was (and I couldn’t reproduce the bug by typing accented letters myself, so I needed a copy of their input for investigation and I wasn’t going to get it).

Mapping the doctor analogy onto software:

• Option A (replace the doctor) might be best if there is a good alternative implementation available, or if you have reason to believe that you really can do better with a rewrite, but remember that a rewrite can introduce new bugs, and if it’s you who wrote the first version (and not so long ago) then how do you know you’ve improved enough not to make the same mistakes a second time?
• Option B (fix the doctor) is the best option if you really can fix it. But it’s hard to analyse what’s going on if all you are getting is vague reports from users who are not willing to share their data with you. My second user was on the point of sharing his data, but he didn’t want to share the original, so he tried to create a ‘cleaned-up, simplified-down’ version, and in the process he discovered that it was a problem with an accented letter, so he said: “Don’t worry, I’ve solved it” and I haven’t heard any more from him. Frustrating!
• Option C (guard the doctor) is probably the best option if you cannot fix your code: simply add guard code around it that restricts the input. Unfortunately it’s a bit of a compromise solution if the guard code has to ‘over-block’: if I have a bug that only manifests itself in some very specific situation that involves a particular accented letter, then I could say ‘insist on ASCII only’, but that would be putting extra restrictions on my users that are not entirely necessary. I already know my code works perfectly well with many non-ASCII situations; it just goes wrong in some particular case that people are refusing to tell me.
• Option D (warn about the doctor) is a very poor option. People won’t read the sign. It’s like putting a comment on a function that says ‘may crash if X’ and nobody ever reads the comment. Perhaps a warning would be better than nothing, but not very much better than nothing.
• Option E (do nothing) might be OK for a personal private doctor (some quick program you wrote just to use yourself on your own computer), as saying ‘but I would never give it that input anyway’ could be true if you are the only user. But even then I’d still be worried about my future self and at least write a simple guard, even if it’s nothing more than an assert or two, just to get a good ‘fail: you need to take a look at this’ message if I ever forget it had that limitation. And I certainly don’t think production code that is used in cancer-research labs around the world should take Option E, and I’m worried to see my European users thinking otherwise.

I’m probably ‘preaching to the converted’, telling this to developers. It is our users who need to learn the value of being willing to give us all the information we need to properly diagnose a bug, instead of giving up as soon as they personally have a workaround. I can understand settling for a personal workaround if the developer is not interested in fixing it properly, but I wouldn’t want to do that when dealing with a developer who IS interested. Perhaps they’re just not used to having developers take an interest in them.

Anyway, I hope that this ‘doctor story’ can at least help somebody explain to a user the need to get things properly fixed, at least if that user is fluent enough in your language to be able to listen to your explanation.

Silas S. Brown Silas is a partially-sighted Computer Science post-doc in Cambridge who currently works in part-time assistant tuition. He has been an ACCU member since 1994.

Notes: 

More fields may be available via dynamicdata ..