Title: Writing For Publication in the Real World

Recently, the ACCU alert e-mail list carried a note that a publisher was looking for authors to help finish a book on C. The book was aimed at intermediate to advanced programmers and the time-line for finished work was short. I felt I was qualified to write on one of the topics noted in the list, so I wrote an email describing my background and some general information about how I would approach the project.

The publisher replied requesting an outline of the chapter that went three layers deep. This required more thought than I had given the effort, so I went back to the drawing board and reconsidered what should be included in a chapter on sparse matrices. First and foremost, the chapter should go through the process of building a sparse matrix. Next, but also necessary, the chapter needed to address how to traverse the matrix so that it would be useful. Finally, matrix manipulations would be nice. I laid this information out to three layers of depth and sent it on to the publisher.

The publisher thought my plan was great and said they would send me a contract with a due date a little more than 3 weeks hence. Additionally, the work would need to be done over the Christmas holidays and the Y2K transition. I knew that my real job and real life would swiftly intrude on the time I could spend on the book, so I desperately started to evaluate what kind of sparse matrix I would develop. As many of you may know, like anything else, there are lots of ways to program a sparse matrix, but I had to choose one because I was limited to 40 pages and a tight time constraint.

At the same time, I wanted to work on an interesting project which would be challenging to me so that I could get as excited and interested in the project as possible. Maybe excited is too strong. I guess I just didn't want to fall asleep reviewing my own code. I hunted around on the Internet and noticed that there were lots of good libraries for sparse matrix implementation, which suggested that my efforts would be doomed to obscurity unless my matrix outperformed some of the ones already there. Since I was not building a comprehensive library of matrix code, and I wouldn't have much time for performance, that was out.

On the other hand, all of the sparse matrix libraries I saw were two-dimensional. Now this makes sense as many of the applications of a sparse matrix are in the area of mechanical engineering and they are generally used to solve lots of equations with lots of variables. My strongest experience with sparse matrices however was in the field of Artificial Life. In this field 3-dimensional matrices were of high value and N-dimensional matrices would allow monitoring of 3-dimensional spaces against multiple variables (population density, time, etc). So, I resolved to write an N dimensional sparse matrix implementation.

Note to self: Never bite off more than you can chew.

Of course, this was a huge effort. I thought my skill and expertise, particularly with this type of code, would dramatically help and speed up the process. Since I was going to be using C and not C++, the sparse matrix stuff I had done in the past 4-5 years was not going to be useful (it was all in C++). I went back to the drawing board and decided to abandon everything I remember about sparse matrices and rebuild from the ground up.

I came up with an inventive method (I think) of handling the N dimensions and drew nice pictures describing my thinking. I then drew up a 6x6 matrix (2 dimensions are much easier to draw) showing all the links between all the components. YES! I was ready to code.

I created an include file which gave my paltry structures. These structures were really deceptively simple. Suddenly I realized that I hadn't written any of the 40 pages for the chapter. Suddenly 40 pages wasn't seeming like a restriction anymore, but a huge void. I switched gears and wrote about why, when, and how to use sparse matrices - all the good stuff that comes at the beginning of a chapter which hacks skip to go right to the code. I feel safe noting this is hack behavior, as I have done it often enough. On the other hand it is worthwhile, because even the hacks end up reading it after they have digested the code.

Back to the code, and I had little better than a week left to finish the whole thing. My evenings were consumed with writing, coding, and testing. My train rides to and from work were opportunities to work on algorithms (do you have any idea how hard it is to find batteries for old PowerBooks? - but that is another story) using the old paper and pencil approach. In a lather of activity my code grew substantially and the chapter material, which described the algorithms and coding process used, started to fill up the 40 pages. Suddenly, the due date was upon me. Christmas had come and gone with nary an improvement, Y2K had puffed into non-existence (YEAH! Competent programming!) and I had until midnight to finish.

I was suddenly transported back to college with images of caffeine-induced inspirations and dramatic dashes between my dorm room and the mainframe printing room spinning through my head. Only now, it wasn't roommates, it was a spouse and children who called for my attention. It wasn't other classes demanding output, but real work and family obligations that reduced my time.

Finally, in a frenzy of activity the code was completed and run through the bare minimum of tests, text was completed and grammatical suggestions addressed on the most base level. I was a day and a half late. Fortunately, the due date was Friday at midnight and I was finally done Sunday afternoon. The publisher had not been in the office over the weekend, so all was well.

I sent a note explaining that I was fine with the text, but had concerns with the code, which had not been adequately tested. The publisher replied that I would have a chance to review the book chapter later in the month and could make code modifications then.

I returned to my code and found glaring errors: Tests that failed with disturbing frequency, sections that simply didn't work, and error codes that were missed. I was ashamed and embarrassed that I had sent this code on. I continued to work on it and just as the tests were coming to a close and changes were working, the publisher sent e-mail.

I dreaded the message. Summoning courage and rallying my cyber hounds, I opened the message; it contained a huge list of complaints from the technical editor who couldn't get my code to work. I fired off a quick response letting the editor know I was aware of the problems and hard at work trying to fix them. Once again, this was a Friday. By Sunday morning the code was working and I had a test suite that rivaled the code itself in line count. I put it all together and sent it off to the publisher once more.

Since then, things have been mighty quiet. I'm guessing others are being asked to refine their code, editing is being done, and various and sundry other polishing processes are ongoing. Sometime soon (probably by the time you read this article), I will have given my chapter its final review and the book will be in print. Please look at my poor chapter on sparse matrices in the book C-Unleashed, find any mistakes, and offer suggestions.

Also, if you would, share your own chapter, article, or book writing experiences. I hope that most books are written with more time than I had. Without a doubt this is a learning experience and I have come to the conclusion that most programming authors must be terribly underpaid for their efforts (not that I'm complaining for myself).

Editor's note:

I have published this as an insight into the process by which many of these multi-authored compilations get produced. I wonder if readers have thoughts on this process.