Title: Building Sculptures out of Holes

I've been working on an embedded system lately. My specific problem was to install an operating system (in my case GNU/Linux) on a device that, though i386-compatible, doesn't have a cdrom or floppy drive. There's no video card, no keyboard interface, just a serial port, an Ethernet and an ISDN interface. It has a 15 MB flash disk that can be programmed via the serial port. The system's BIOS has a remote control feature that is executed on startup and reads 15 MB of binary data for the disk. The only way to install the operating system is to supply a complete image of an already set up and configured system as one file ready to be programmed into the flash disk by the BIOS utility. Take a file, program the disk, toggle power and the system runs. In theory. In practice, this file must come equipped with a properly compiled and installed kernel, a boot loader, a partition table, and so on.

Under Linux, with a few tricks, you can treat a file just like a disk (i.e. a block device). Then, to create the operating system image, use "dd if=/dev/zero of=image bs=512 count=31296" (for this special system) and you get a file called "image" with the desired size consisting only of zeros. You partition it with fdisk, mount it, copy the kernel onto it and install the boot loader (LILO). fdisk and lilo, the LInux LOader map installer, took about 10 seconds each on my machine, dd took another 3 seconds. This was clearly too much, so I decided to speed it up. After experimenting a day or two without promising results, I figured that I had to write my own partitioning tool. This could be kept very simple, since it didn't have to do protocol with a real hard disk. It was just a matter of writing a few bytes at a fixed position in the file. That done, the whole image creation process took about 13 seconds, 10 seconds less than before. This seemed to be the sound barrier because replacing lilo with an own implementation was far to much work. And optimising away the 3 seconds of dd wouldn't pay off, right?

Wrong. I was curious. I had the idea of replacing dd with a small program that would just create the 15 MB file as a file without real content. Even without the zeros. In Unix jargon, this is called a file with holes. You create a new file, fseek to some position p, write just one byte and close it again. What you get is an empty file with p+1 bytes in size. On the disk, however, this file would take almost no space, at least if your file system supports that. With that change, I anticipated a total runtime of about 10 seconds for the image creation. When I tested, it took only about 3 seconds! Clearly, the file with holes has sped up lilo.

Why do I tell you this story? Because I found it just funny how my curiosity about these files with holes led me to this non-obvious optimisation. This was clearly a matter of coincidence. In the last issue of CVu, in his "Professionalism in Programming" column, Pete Goodliffe wrote about curiosity being one of the key qualities of a good programmer. Could it be that this curiosity more often than not leads to such lucky coincidences, such solutions that can't be planned? solutions that can't be designed? Could it be that some of the best features in software out there were found by accident by a curious programmer? It's clear that one doesn't become a good programmer just by having luck. But maybe one gets luck just by being curious.