Title: Don’t Assume Any Non-Zero exit() Will Fail!

Author: Bob Schmidt

Date: 06 July 2018 17:13:44 +01:00 or Fri, 06 July 2018 17:13:44 +01:00

Summary: Silas S. Brown shares his finding on process exit codes.

Body: 

I recently came across a little ‘gotcha’ on a BSD system which also affects GNU/Linux systems. I was using make to run a series of commands including a Python script, and I assumed that make would stop if the Python script exitted with non-zero status. In the Python script I had done os.system() to run a shell command, and, if the result is non-zero, then sys.exit() with that result to fail the Python script as well:

  err = os.system("...")
  if err: sys.exit(err) # DON'T DO THIS!

Except make carried on regardless. It didn't stop at the failure.

It turned out that os.system() was returning 256. And in the bash shell on both BSD and Linux, the command:

 python -c 'import sys; sys.exit(256)' || echo fail

prints nothing.

The POSIX standard wait() and waitpid() calls provide only the lowest 8 bits of the exit status in WEXITSTATUS, so an exit status of 256 would be read as 0. The newer waitid() call returns 32 bits of exit status, but obviously not all system tools use it. (The underlying C standard defines only EXIT_SUCCESS and EXIT_FAILURE as 0 and 1; it says nothing about how platforms treat other values.)

But why was os.system returning 256 in the first place? As can be seen from:

  >>> import os
  >>> os.system("exit 3")
  768

the return value of os.system() is not (on Unix/Linux) the straightforward exit code of the program it called. So we turn to the Python docs:

On Unix, the return value [of os.system()] is the exit status of the process encoded in the format specified for wait()

which links to a page saying:

exit status indication: a 16-bit number, whose low byte is the signal number that killed the process, and whose high byte is the exit status (if the signal number is zero); the high bit of the low byte is set if a core file was produced.

Incidentally the man page of the underlying C system() function on BSD also says, “

The system() function returns the exit status of the shell as returned by

waitpid(2)” which says you’re supposed to call WEXITSTATUS to get the exit status from it if WIFEXITED is true.

So to be more portable, you have to do something like:

  if (err & 0xFF) == 0: err >>= 8
  sys.exit(err)

or check what platform you’re on and act accordingly.

On Windows, os.system() straightforwardly returns the exit code of the command interpreter. But not all Windows command shells actually pass on the exit code of the program they ran. cmd.exe does, but the older command.com on Win9x systems didn’t, and more worryingly third-party ‘power shells’ and such are not guaranteed to (the Python docs say check their documentation). So the only way you can guarantee to find out the real exit code is to run the program directly with os.spawn*, or use the (new in Python 2.4) subprocess module, instead of using a system shell.

Notes: 

More fields may be available via dynamicdata ..