Title: TXNUL, A reprise

In C Vu 11.2 (January 1999) Anthony Pleace asked how to capture data from the serial port. He mentioned that he had found a program called TXNUL on C Vu disc 10.1 but could not find an associated article.

In his reply Brian Bramer said he had looked and could not find anything, and Francis agreed that he did not think there was an associated article. In fact, on page 13 of C Vu 10.1 there is a letter from Pete Disdale (pete@pdmail.demon.co.uk) in which he mentions his attached program TXNUL. This was in response to a letter from Lawrence McHugh in the preceding issue.

Here are some extracts from the correspondence:-

In experimenting with programming the serial port of a PC I have noticed that I cannot send an ASCII NULL (0x00), it seems to me logical, when you write to a transmit register that the fact of writing alone should be enough to cause the device to transmit the character regardless of what it is (even a 0x00) however I have found that it does not transmit this character.

How then can, for instance, an executable program (which may contain data which is (0x00) be transmitted without port stripping out the NULLs. To illustrate this problem generate a file containing the data "1 2 3 \0x0 4 5 6" i.e. the characters 123,456 separated with a NULL. Connect two PCs together using the com ports, set up the ports with the mode COMx: command and try to copy the file from one to the other using the DOS copy <file> COMx: command. Lo and behold the file comes out as "123456" with the NULL stripped out.

Question: am I doing something wrong or what? How can I transmit an ASCII NULL from the serial port?

The above letter provoked several replies in CVu 10.1. On page 13, the letter from Pete Disdale...

Hello Francis,

In C Vu 9.6, Lawrence McHugh asked a couple of questions, the first concerning chopping up of rods algorithmically and the second on how to send an ASCII NUL character via the serial port on a PC.

Francis replied, "Lawrence's second problem is more tractable but I leave it to the low-level experts to come up with advice (if you know the answer please send it in rather than just assume someone else will)".

On the grounds that (1) the former requires more grey cells that I can muster right now, (2) I'm pretty low-level, if not expert and (3) like everyone else I'm assuming that I'm the only respondent, I'll have a go at the second.

Lawrence doesn't say exactly how he is "... experimenting with programming the serial port...". My assumption from the way he states his problem ("... when you write to a transmit register...") is that he is in fact writing directly to the appropriate UART register using an out instruction. If this is indeed the case, I cannot see the problem - it works!

Conversely, if he is using the serial port services provided by DOS or the ROM BIOS, I can well imagine he is getting odd results - I have never managed to use these successfully!

In my experience, "programming the serial port" on a PC (or more accurately in the8250/16550/8259 PIC) is an all or nothing affair if the aim is to produce a useful serial link; one has to be prepared to write all the code the BIOS or OS author left out. This means invoking the interrupt facilities offered by the UART and providing the handlers to process them, buffering the I/O etc.

That said, there is no reason I can see why transmitting or receiving NULs should be a problem, even using a simple polling mechanism. As a (I hope) is demonstrated by the actual program (TXNUL) which writes and reads back the sequence of characters that Lawrence suggested.

This program requires a single PC with a COM1 or COM2, and a strap between the Tx and Rx pins (2/3) of the selected serial port (this is easily achieved using a patch box, which is IMHO an indispensable bit of kit if one is playing serial comms). It could naturally be re-hashed to run on two PCs - a transmitter and a receiver - via a null modem (2/3-crossover) cable.

The program defaults to using COM1 (base UART address 0x3f8), but if run as 'TXNUL 2' will use COM2 (at 0x2f8). Apologies in advance for the ASM-like programming style - I would not usually write this type of code in C, so cribbed/translated it from my ASM buffered serial I/O lib. Hope it helps.

The following is an extract from the letter from Silas Brown on page 16...

Re: sending NULL characters over PC serial port: I have tried this at 9600 baud between an Olivetti PCS 286S PC (unknown serial port type) running MS-DOS 5.0 (using mode and copy) and a Psion 3a running its native comms application. When I asked the Psion to translate Enter to ASCII 0, the PC did receive it correctly. I tried copying binary files in both directions without trouble, except that the EOF character, ASCII 26, tended to truncate them prematurely, and sometimes there were curious "0" digits (ASCII 48) cropping up from nowhere. Of course, if you really wanted to transfer binary files between computers, the best thing to do would be to use comms software with error checking protocols, to make sure that they arrived correctly. PD software with XModem should be widely available, and, although it pads files with ASCII 27 (sic) to multiples of 128 bites in length, it is a start.

And finally from Fred Johnson on page 19...

Now, I would like to pass on a few comments to Lawrence McHugh. First of all, welcome to the world of the uncharted territories. The area of serial comms should be labelled with a sign saying "here be dragons" as you have already discovered. May I first request that you refer to ASCII 0 as NUL, (the recommended ASCII term) not NULL. Presently, these are the same on some systems, but this state of affair is may not always be so (NULL may be defined as an integer of two bytes or more length, not a single byte).

Starting with a COPY command in DOS, this may have switches set for either ASCII or BINARY transfer with /A or /B (MSDOS Users Guide, note default is /B) and MODE may be set for 7 or 8 bit ASCII, with differing results. Having used COPY, there is then the problem of accurately displaying the contents of the file. Since NUL is a directive to a printer or display device to do nothing, then nothing is printed on finding '/0' in a file. Similarly, most text editors ignore '/0 '.

The only solution is to use some form of hex editor (e.g. XTREE View mode with HEX display selected) or write your own display routine to display hex bytes.

Using your example, I found that the '/0' is indeed transmitted and received but not displayed using DOS TYPE or PRINT commands or using the DOS EDIT program.

It is advisable to stop thinking in terms of ASCII (or any other character set, as in some other systems characters are 16 bits in length e.g. Unicode) and only use hex bytes unless you wish to limit yourself in what you wish to achieve or to invite failure on other platforms.

I found in the early stages, a dumb terminal like a CIFER T5 or a WYSE 50 (both in VT100 mode) which allow display of all control codes from a menu setting, and also display the status of the control lines (RTS, CTS etc.) an invaluable tool for development of basic transmission routines. Having established clean transmit routines, I was then able to develop reception routines using my own transmissions for testing.

You do not say which compiler you are using, but some advise not using the built-in BIOS serial routines within DOS due to their very low speed, a statement borne out by a few tests.

Quote from Borland 'C' v5.0 BC5/DOC.Helpme.Wri

Q: How come bioscom() doesn't work on my computer?

A: The bioscom() functioning uses DOS interrupt 0x14 directly, and thus bioscom()'s functionality is tied directly to the BIOS of your computer. MSDOS support for the serial communications port might be inadequate in several respects for high-performance serial I/O applications. First, MSDOS provides no portable way to test for the existence or status of a particular serial port in a system. If a program "opens" COM2 and write data to it, and the physical COM2 adapt to isn't present in the system, the program may simply hang. Similarly, if the serial port exists, but no character has been received and the program attempts to read a character, the program hangs until one is available. There is no traditional function call to check if a character is waiting. MSDOS also provides no portable method to initialise the communication adapter to a particular baud rate, word length, and parity. An application must resort to ROM BIOS calls, manipulate the hardware directly, or rely on the user to configure the port properly with the MODE command before running the application that uses it. Because of all the problems mentioned above, we strongly recommend getting a third party communications package when attempting to do serial communications, or downloading the example program SERIAL.ZIP from our BBS at (408) 439-9096.

For example, using a '486' computer clocked at 25 MHz driving two COM ports, characters started being lost at 38400 baud even though the serial ports used the faster 16550 UART. When using assembler or 'C 'interrupt driven routines an old '286' based machines clocked at 12 MHz was able to drive three COM ports at 38400 baud with no loss of data using the older 8250 UART. Setting the port speeds and handshaking may be achieved using 'C' routines that allow more freedom than the DOS MODE command. This requires access to a data sheet on the port for setting the UART to values not supported by DOS.

Your choices seem to be either using assembler or 'C' based interrupt driven routines in both of which you provide a large enough circular input buffer to hold received characters until you can get around to dealing with them and emptying the buffer.

The next point to consider is the protocol to be used. I have come across four distinctly different types in use, each with its own pros and cons. In the first, data is sent in fixed length packets, with short packets padded with extra characters (often '/0' is used for this character). They first packet is sent which contains the number of packets to be sent, to allow the receiving software to know when the message is complete (e.g. Kermit).

In the second, data is sent in variable length messages with a message start segment containing the length of the message. This is used in some serially controlled machines.

The third system goes back to using ASCII characters only (despite earlier comments) with messages starting with STX and ending with ETX. Data bytes are sent as ASCII equivalents rather than non-printing characters, i.e. hex 1A is sent as ASCII '1' followed by ASCII 'A ', and then restored to original in the receiving program.

The last system transmitted a single 8 bit character and waited for the same character to be echoed from the receiver before sending the next character.

Whichever one you use, the first essential is to produce clean transmission and reception routines capable of handling whatever baud rates and handshaking you choose. Unless this is achieved, problems may occur which lead to encoding and decoding routines being suspect when the real culprits are the above mentioned routines losing essential data.

Thanks for the time taken putting that together. Unfortunately this is yet another example of someone who left his/her name out of the file. Please, when you send files as attachments make sure you put your name in it somewhere.