Title: Delivering Bad News from QA

Author: Martin Moene

Date: 07 November 2016 16:59:01 +00:00 or Mon, 07 November 2016 16:59:01 +00:00

Summary: Silas S. Brown describes how not to report your senior colleague’s bug.

Body: 

As I type this on a vintage 1999 Psion Revo (remember them?), my Cantonese wife of 18 months is tapping away into LibreOffice on her old laptop (whose malware-infested Windows I upgraded to Linux shortly into our marriage), on the kitchen table of my parents’ house in a West Dorset village (we’re visiting them this week), occasionally calling me over to check some aspect of English grammar. She is writing to a Nature journal about a C program I wrote which we hope will speed up cancer research teams across the world. But just last week, that same program caused her to feel so bullied by a senior colleague that she resigned, took her notice period off sick and doesn’t want to go anywhere near a research lab again.

About a month previously, my wife had been asked to check batches of molecular primers for a battery of DNA tests and ended up waiting on into the night for the lab’s computer to finish simulating their mutual interactions, then bringing home the Visual Basic 6 program which we tried to run on WINE but it got nowhere all weekend (I later found her input was one or two orders of magnitude larger than what that program was designed to take). So I sat her down and fired off a string of stupid non-biologist questions until I’d coaxed out a specification that I could use as a starting point for re-implementing the thing, except my version was in C with bit-pattern techniques (making full use of 64-bit registers to test many DNA bases at a time) and OpenMP parallelisation. Because this ran literally thousands of times faster than their Visual Basic affair, it opened up the possibility of simulating on larger scales and automating more aspects of the pooling process; I added the extra features they needed, and it seemed none of the other pieces of software out there had features in just the right combination for that kind of cash-strapped cancer research lab (I heard the principal investigator was trying to employ three assistants on a grant that was meant to be for one, so they obviously needed to conserve resources at every turn and that’s why optimisation was important). Everyone seemed excited that I’d been able to write what they needed.

But then it turned sour. We tested the software by asking it to check the mutual interactions in a set of pools that had been meticulously derived by hand by one of that lab’s senior academics a couple of years earlier, and it pointed out 11 amplicon overlaps [1]. That academic was supposed to have eliminated all amplicon overlaps, and had done a pretty good job of getting rid of hundreds of them, but the fact that there’d been 11 left called into question their work and potentially the lab’s results for the last 2 years. We carefully checked if my software report was incorrect. It wasn’t. My wife went to the lab and basically said ‘look how good our software is: it found these 11 mistakes in Dr X’s work’. And said senior academic took things very badly and, I’m told, began the workplace bullying that led to my wife’s resignation as previously mentioned.

I should have insisted on going to her lab myself to present the software and take Q&A. Not because I’m macho (which I’m not over much), nor because my visual impairment deprives the bullies of one of their dimensions of expression, nor because I have the experience of being physically bullied by dozens of boys in primary school and therefore ended up with a wider sense of perspective, but simply because, as a programmer, I know that bugs can happen to anybody. I could have included in my presentation tales of Cambridge professor Sir Maurice Wilkes finding a bug in his second EDSAC program in the late 1940s, and no doubt other examples to show that to err is human (no matter how clever you are) and it’s good for all of us to run our work through some form of automated testing, and that sanity-checking tools that work well are good no matter who we are (note the ‘we’; it sounds better than ‘you’ when the subject is human error). One does not join an organisation and within months walk up to seniority (especially non-programmers) and say ‘I found a bug in your work’ (so there). One says ‘I was trying to put your work through this compiler and it said this; do you think that’s a problem?’ or some such. (Make sure to say it was the compiler that found the problem, not you. You might have thought finding problems gets you credit, but it might instead get you some wrath.) Non-programmers are not used to bug reports, so we have to be tactful especially if we’ve just developed a new QA tool that’s throwing up newly-discovered problems (after all, it’s also possible the tool is simply wrong to flag up all that). Unfortunately I didn’t think to warn my wife about all this in advance; she didn’t have previous experience delivering such reports to senior non-programmers and she naturally put the proverbial foot in it. I honestly don’t know how far their project is going to get now she’s left, and let’s not even think about how many more patients die while things are delayed. At least we’re still trying to get the software past peer review so other labs can start using it (I asked an old classmate who now works in a commercial bio lab and she basically said ‘sounds good but tell me when it’s gone through peer review’; many labs have a policy of not being the first to try new tools, so having it published in a medical journal is important).

Once our Nature paper is ready to submit, we take the daily village bus (which my late grandmother campaigned for and subsequently couldn’t use) to visit the public library of the small nearby town (around which my parents lived all their lives), where we have to battle with Windows against a time limit – shorter than expected because somebody else had booked the computer, and they’ve locked down the configuration so I can’t turn up the print size or type in the Dvorak keyboard layout, and accidentally muttering ‘for goodness’ sake’ gets me told off for swearing in the library – but we somehow get it done. Two days later, I am able to briefly get a signal in the village by holding my mobile phone against a tree on the hillside (the capacitative coupling seems to help) and we see Nature thought our paper was too specialist for them. Their response is a model of how to say this the right way: they specifically said they didn’t doubt the technical quality of our work or its interest to others working with primer pools; they just didn’t think the advances presented would have sufficiently significant immediate impact on the broader readership. They wanted to return it to us as soon as possible so it can be sent elsewhere without delay. My wife is very happy about this and says we will look at it once we’re back in Cambridge (perhaps we’ve done enough of rushing it from the rurals). We’ll probably try the Oxford Bioinformatics journal (I wouldn’t want to task our very own Frances Buontempo with assembling a medically-qualified peer review team for an Overload paper).

If we could all train ourselves to be as nice as those Nature editors when reporting problems within an organisation, perhaps our professional relationships would be a tiny bit better. Perhaps.

Postscript: Shortly after we returned, my father succumbed to his illness and passed away. We immediately made a second visit to Dorset and wrote the second paper while there, which my wife decided we should send to Nucleic Acids Research. That editor felt it’s still off-topic but was kind enough to transfer it to another journal which might be appropriate; their decision is still pending.

Note

[1] An amplicon overlap is the error of placing primers for two overlapping sections of the DNA into the same pool.

Notes: 

More fields may be available via dynamicdata ..