Journal Articles

CVu Journal Vol 13, #3 - Jun 2001 + Professionalism in Programming, from CVu journal
Browse in : All > Journals > CVu > 133 (12)
All > Journal Columns > Professionalism (40)
Any of these categories - All of these categories

Note: when you create a new publication type, the articles module will automatically use the templates user-display-[publicationtype].xt and user-summary-[publicationtype].xt. If those templates do not exist when you try to preview or display a new article, you'll get this warning :-) Please place your own templates in themes/yourtheme/modules/articles . The templates will get the extension .xt there.

Title: Professionalism in Programming #8

Author: Administrator

Date: 06 June 2001 13:15:46 +01:00 or Wed, 06 June 2001 13:15:46 +01:00

Summary: 

The Programmer's Toolbox

Body: 

To be a productive workman you need a good set of tools. A plumber has a collection of items in his toolbox that will support him in whatever task he encounters. Not only the existence, but also the quality of these tools is a vital issue: a good craftsman is only as good as his set of tools. If our plumber has a set of dodgy compression valves there will be water everywhere, no matter how good he is.

The same is true of programming. If we want to do a good job, we need to be backed up by an appropriate kit of tools. Tools in which we have confidence, know how to use and which are fit for the jobs we will encounter.

In the past few columns we have covered a number of issues that relate to some particular tools. Here we will broach the subject of software tools as a whole. Programming is a field that simply cannot do without tools. From day to day we use tools without much of a thought for them. We can take our compiler for granted in much the same way we take a tin opener for granted - it is fine whilst it works, but as soon as it goes wrong (or we need to open an odd shaped tin) we are stuck, no matter how fancy the tin opener is. A cheap, basic tin opener that works is better than some pretentious contraption that does not.

Why bother with tools?

So we cannot get by without using a certain number of software tools[1] in order to create programs. Other tools we can get by without, but they still have a genuine use. In order to improve our productivity, quality, and therefore professionalism, it is good to pay a little attention to what tools we are using and what they really do.

Some help you write code. Some help you write good code. Some pretend to write code for you.

If you understand how your tool works, and which tool to use for which job, you are better able to produce code that has no nasty surprises, and produce it more quickly. We need to be clear in our minds why we actually use tools: tools do not do our work for us - they help us to do it. This is a reason why I am generally wary of "wizard" generated code - do you honestly understand all of the pages of code the wizard has generated and are you able to safely extend it? If not, can you really have confidence in what the tool is doing?

Different tools have different focuses, ranging from a very specific task (e.g. calculate the bit rate of an mpeg stream based on a couple of input parameters) to the scope of an entire project (e.g. a collaborative project management environment). Some tasks are naturally more tool-able than others.

Programmers have wildly varying attitudes towards using and selecting tools. In use, some just plug away at a tool until it does something like what they want and just stick with that. (Just because you've run it without generating an error does that mean the tool has done exactly what you wanted?) Some programmers, on the other hand, carefully read the documentation to find out exactly what can be done. What is the right approach?

Some programmers encounter a lengthy task and laboriously do it by hand, others write a tool in a scripting language to do the job automatically, others spend hours searching for a pre-written tool to do the job for them. What is the right approach?

Part of becoming a more 'professional' programmer is understanding that balance, and being able to make the distinction. It is true that everyone is different and everyone works differently. But if you saw someone converting their C code into assembler by hand on a day-to-day basis, you might begin to question their sanity.

Different tools work in different ways, and obviously the platform and environment they fit into play a huge part in this. Command line Unix tools are very different from highly graphical Windows-based utilities, but that does not make them any the less useful (in fact, it can make them an awful lot more powerful). A thorough understanding of your development platform enables you to get the most from the tools you use. Do you prefer all the tools glued together (in a monolithic GUI that takes forever to set up just as you like it), or packaged as discrete command line entities (with complicated and disparate interfaces that are not easy to understand)? This to a large extent will depend on your mindset and prior experience. I would challenge each of us to try working the other way on a reasonably sized project to fully understand pros and cons[2] of each.

Using tools

It is important to understand the platform you are working on and what it can do for (or with) you. One huge benefit of command line Unix tools is in their use as filters when "piped" together - splicing small individual tools together to make a larger utility. If you do not know much about this, I urge you to read up on it[3]. Understanding the power of the tools and how they inter-operate brings significant benefits.

Even just knowing the general sorts of tools that exist, where you can find them and what you can expect from them will make you better prepared to use a tool. It will help you be able to make the choice between searching for a tool for a particular task, writing it yourself or (if it is a truly one off, fairly short task) doing it by hand. Being ready to try a new tool and to take the time to learn how to use it is beneficial. You might need to find new tools if you start a new project/platform, you have a new problem to solve, or your old tools become deprecated.

Reading the manuals and supporting documentation for your tools may pay for itself at the end of the day. Do you even know where to find full documentation on each tool you use? GUI tools give the impression of never hiding esoteric functionality because all the options are available in a dialogue box - you do not need to read the docs. In experience this is not quite how it works, though.

Finally, tools seem to develop at an incredible rate. In this industry technology changes fast. Some tools develop at a much faster rate than others. It is important to understand what is going on with the tools that we use - so that we do not get out of date, and therefore have a possibly buggy and unsupported toolkit. But this should be done cautiously - not necessarily chasing the latest version blindly. The bleeding edge can be painful!

What tools?

Over the years various tools have been developed to scratch particular itches, the needs that often cropped up. When a task has had to be done many times, then you can bet someone has written a tool for it. Most of these are available for the software engineer to use (thank goodness for the internet!).

Having access to the source code for your tools can be instrumental in diagnosing problems that you encounter, allowing you to work out exactly what the tool does and how it behaves. This might be a deciding factor in your choice of some tools. However, the factor that will most likely affect whether you'll adopt a particular tool is how well it fits into your personal development practice, followed by the cost, how configurable it is (does it enforce a working style that you do not like), is it easy to use, is it well documented, is it reliable and does it have a proven track record?

Exactly what comprises your toolkit will depend on your line of work. The available tools for embedded platforms are rarely as rich as for desktop machines. However, we'll consider some of the more common components below. Some are obvious, some perhaps not so.

Source code editor

Perhaps the most obvious tool we use. Also perhaps the most fundamental tool - the programmer will spend most of their time using this piece of software. But that is why it should be very carefully chosen. The One True Source Editor is an age old debate which does not need stirring here, but you should use a source editor that you find comfortable and does what you require. Just because an editor is embedded in your visual IDE does not mean that it is the best editor for your purposes. On the other hand, you may find that having it integrated is an incredible boon. For source editing, I personally require at least the following from my source editor:

  • Syntax colouring (with support for many languages)

  • Simple syntax checking (e.g. highlighting mismatched braces)

  • Good incremental search facilities

  • Macro recording

  • Highly configurable

  • Works across every platform that I use.

My requirements and choice of editor may not be yours.

e.g. vim, emacs, IDE editors, UltraEdit

Compiler

The next most taken for granted tool. What else can convert our source code into machine code? Since we use this tool so pervasively, is not it important that we are sure how to use it? Do we really know all the options that it has? Do we understand what level of optimisation we require, and how that might change our code?[4] It is true that in many places of work there is a specific buildmaster who ensures that tools in the build process are used correctly, but that is not an excuse to be unfamiliar with the compiler.

As another example, do you compile with all warnings switched on? There really is no excuse not to (perhaps only if you are maintaining legacy code that is already full of warnings). The warnings highlight potential errors and (when removed) give you extra confidence in your code. Also, is the compiler ISO standard [ISO] compliant by default? Does it have any non-standard extensions, and do you know what they are and how to avoid them?

Often we use a "cross compiler" when writing code for an embedded environment. This simply means that the target platform is different from the development one (after all, it is hard to run visual C++ on a washing machine).

The compiler is often considered a single part of a larger development tool chain, including the linker, assembler, debugger, profiler, simulator and other object-file manipulators.

e.g. gcc, Visual C++, Borland C++ builder

Debugger

The availability of a good quality debugger and understanding how to use it can save hours of development time chasing surprising behaviour. It allows you to investigate paths of execution in your code, break into it, investigate variable values, set breakpoints and generally dissect your running program. It is an order of magnitude more sophisticated than using printf statements!

e.g. gdb, ddd, IDE debuggers

Profiler

Used when code runs unacceptably slowly. The profiler times sections of running code and identifies potential bottlenecks. This tools is invaluable for optimisation and will prevent you from speeding up code that is only run once in a blue moon.

e.g. gprof, IDE profilers

Code validators

Code validators come in two varieties, static and dynamic. The former can be run over code in a similar way to a compiler, to identify possible problem areas in the code. Lint is a good example of such a program - performing static checking for a series of common coding errors in C. A lot of this kind of functionality tends to be built into modern compilers, but there are still a number of very valuable tools available to complement this.

Dynamic validators modify the code as it is compiled, and then perform checking at run time. Memory allocation/bounds checkers are an example of this - ensuring that all dynamically allocated memory is freed appropriately, and that array accesses do not occur out of bounds. Again, the value of these types of tools cannot be stressed - they can save hours of legwork finding obscure bugs. They can be much more useful than a debugger in some situations.

e.g. Lint, Insure++, MemProf

Build environment

The tools that comprise the build environment are more than just the compiler and linker. The kind of tools that are used are the "make" build tool, or the build portions of your IDE. They automate the compilation process. Many Unix open source projects use the autoconf/automake tools to simplify the build environment.

e.g. make, IDE build tools, automake/autoconf

Fault tracking

A good shared database allowing colleagues to log, query, assign, reply to, and fix faults is essential to ensuring the quality of a product. Without something like this we cannot ensure that we have addressed every issue that has cropped up. Capturing and storing this information is useful when looking back over code at a later date.

e.g. Bugzilla, Matrix

Software components/libraries

Yes, these are tools! Reusing software components and finding libraries that do what we need prevents us from having to reinvent the wheel.

e.g. STL, Boost, Rogue Wave libraries

Source navigation tools

There is a whole field of tools to help you delve into and understand code, either presenting it in some graphical map, or enabling you to search/navigate through the code base. This can be especially useful on large code bases or when entering a project that is well established.

Good example of such tools are LXR, SNiFF+, the McCabe toolset and the venerable ctags.

We should also consider Unix source level tools such as diff, which allows us to compare differences in files (to see how it has been modified by someone else, for example) and grep for searching for information in files, plus locate/find to discover where in the build tree files are. (Naturally there are GUI equivalents on most platforms.)

Source control and project management

We will not dwell on source control tools here, since we covered it in the previous column [Goodliffe7]. Management and work collaboration tools may be fundamental to a company is method of working allowing you to report and track work against the schedule, possibly encompassing fault tracking.

e.g. CVS et al. MS Project, Achievo

Source generation

There are a number of tools that exist in order to generate code for us. Some of them, as I mentioned earlier, do not necessarily fill me with confidence. However, in the past I have written Perl scripts to generate code automatically. Having written the generator I trusted the code generated.

There are a number of other interesting code generators. For example, Yacc is a Unix LALR(1) parser generator. You use it to generate code to parse well formed input (by defining the input grammar). It spits out a C parser with hooks that you can write. Bison is a similar tool.

There is a class of code generation tools that help you design user interfaces, and then spit out the workhorse backend code. These are especially used for complex GUI toolkits like MFC. I am inclined to think that if the library needs a tool to do so much legwork, it implies the library is fundamentally broken in the first place.

Documentation tools

Documentation is invaluable to code that is well engineered. Documentation needs to be read as well as written. Good online help systems (backed up by a quality bookshelf) are critical[5]. The use of manuals, understanding how to web search well, as well as reading appropriate newsgroups and FAQs all contribute to your pool of available knowledge.

We also need to be able to write good documentation. We mentioned code documentation tools in a previous article [Goodliffe5]. We also need a good document processor (word processor or some such).

Testing toolchain

Appropriate testing is vital to producing reliable, high quality software. It is often neglected - and largely because it is "too much work", distracting attention away from the important task of writing code. This is one of the most dangerous threats to good software engineering. There are tools that help automate unit testing. This brings a host of advantages (we'll talk about this in a later column). Good testing is one of the tenets of the Extreme Programming movement.

As well as automated testing, tools can be available for the generation of test data or test cases, and for simulation of the target platform, perhaps with options to simulate particular error conditions (low memory, high load, etc).

e.g. Jtest

Others

And naturally there are many more tools we use, for example OO analysis/design tools to help us compose object oriented systems, tools for evaluating code metrics and to investigate the quality of our code There are also scripting languages with which we can write other tools as needed (a little Unix shell goes a long way, but understanding the right language for the right job is an essential bit of background knowledge for the aspiring professional programmer).

Conclusion

It is enlightening for us to evaluate the set of tools that we use. Do we really know how to use our current tools? Are there any other tools we really require? Are we fully using the ones available to us?

At the end of the day, a tool is only as good as its user. The maxim bad workman blames his tools contains a lot of truth. A poor programmer will produce poor code no matter how many tools they have used. In fact, sometimes tools help provide spectacularly worse code. Fostering a professional, responsible attitude towards our toolbox will make us better programmers.

References

[ISO] ISO Standard X3J16/96-0225 WG21/N1043. International Standard for Information Systems - Programming Language C++.

[Goodliffe7] Pete Goodliffe. Professionalism in Programming #7: Practising safe source. C Vu, Volume 13 No 2. ISSN: 1354-3164.

[Goodliffe5] Pete Goodliffe. Professionalism in Programming #5: Documenting Code. C Vu, Volume 12 No 6. ISSN: 1354-3164.



[1] By 'tools' we mean small programs used as 'utilities' in our development work. Programs to build programs - if that is not too philosophical.

[2] It seems to me that a lot of GUI tools are less flexible, more expensive versions of command line tools. They're not all bad, though!

[3] man bash is a good place to start, search for "Pipelines".

[4] It is important - it will affect how surprisingly the code runs in the debugger.

[5] Good systems support is something we all dream of, too.

Notes: 

More fields may be available via dynamicdata ..