Title: More Include File Rules

Rule 16: Avoid paths in #include

File locations vary platform to platform: is socket.h in include/ or include/sys/ ? As a general rule it is wise to avoid include paths inside the source code. By placing part of the include path in the source code you are tying the physical locations of source files into the logical source code. If you choose to change your directory structure at a later date you will need to change your source file, which impacts reusablability: not only must the re-user have a copy of your file but they must follow your directory structure.

Any time you reference the directory layout of the system you are coupling your code to the physical directory structure, this is especially true when the parent directory is referenced: e.g. code such as #include "../../utils/helper.h" imposes a lot of directory structure on a project.

Include paths specified outside of the code, through configuration setting, command line options or environment variables break the dependency between file location and file content - it is sensible to use them. On occasions it is necessary to specify part of an include path (e.g. socket.h) but this should not be more than two levels and certainly should not reference the parent directory. (Microsoft Visual C++ 4.0 contained a bug which caused it to corrupt make-files which included '.' paths.)

Rule 17: Break rule 16 when using namespaces

Having said all the above I have been thinking about Kevlin's suggestion that namespaces be placed in a sub-directory and I think he is right. Classically, a project would place most of its headers in a few directories: the system headers, the compiler's headers, third party libraries and one application include. Inevitable the application directory becomes crowded. Now that namespaces provide a way of subdividing the application modules I think echoing these modules in subdirectories is a good idea.

When a system is divided into modules, e.g. ProjectX is divided into gui and xml modules, it makes sense to sub-divide the interfaces (header files) into their own spaces, ProjectX/include/gui and ProjectX/include/xml rather than dump them all in one directory such as ProjectX/include.

Once we have done this we have a choice of either

Using option 1, if we where to add a new module we would need to change the include path and perhaps effect the entire system. Option 2 follows the both the "least modification" and "only add code" rules, because no global change would be needed and only the files which used the new module would be effected.

Rule 18: Know the difference between local and remote includes

The #include is really two directives: #include "foo.h" and #include <bar.h>: local and remote includes. Traditionally, local meant: search for includes in the same directory as the current compilation unit, i.e. "local is here"; remote meant: search for includes from some include-path - but all this is implementation defined (see K&R or Hatton.)

This could cause problems where the current compilation unit (say bar.cpp) included another file remotely. (say <bar.h>) which itself, included a file locally (say,. "foo.h"). Two problems would occur:

To complicate things further, not only is this compiler dependent, but all compilers allow multiple ways of setting include paths, e.g. Microsoft allows "additional include directories" as part of the pre-processor, set for the project (stored in the .DSP file) while "include directories" is set in options/tools and is the same for all project on this machine whether you are compiling the Nuclear Reactor control project, or you Lottery Random number generator.

Some things, like stdio.h are part of the compiler environment regardless of project, some things are always part of the project and some things depend: imagine two versions of the STL on your system: a small fast version (perfect for choosing lottery numbers) and a slow but bug-free version (perfect for fault-tolerant systems).

At some point in your system, the logic in the file must meet the physical file system and directory layout (a subject worthy of an article in its own right.). I have always viewed the glue that bound the two together to be the make-file. The make-file knows how files fit together with each other and the standard gubbins.

My view on include files was very clear:

Rule 18: only use "local include" where the file is local to the includer, chances are it will never been seen by anything else,

Rule 19: use <remote include> for everything else, let the make-file ensure the right thing is seen.

In this world, the Microsoft "additional includes" are part of the makefile and make sense, but they use the "local" mechanism.

I cannot give you a solid rule here: in my mind the jury is still out. Given a blank sheet of paper I

As mentioned above, the search for header files is implementation dependent. For example, looking for #include "fred.h" Microsoft C++ (5.0) searches:

While Sun's C++ (4.2) compiler searches:

Obviously very different from Microsoft, however, GNU C++ (actually egcs 1.1.2 in this case) complicates things further with a multitude of extra options, e.g. two search paths are allowed. Generally, GCC seems closer to Sun but the hard coded paths can be varied at compile time so the same version may actually search in different places.

While I always knew differences existed only when researching this article did I realise how different they can be. Microsoft's upward-recursion algorithm seems particularly dangerous.

Microsoft's compiler offers nothing to help debug include problems, Sun provides an -H option which will produce a list of include files as they are included making it possible to follow the logic. If memory serves GCC/egcs lists which files included which file when some compilation errors occur.

One of the more infuriating problems is when you can see something is included but the compiler will not recognise it. For example:

// file: fred.h
class Fred { ... }
// file: Flintstones.cpp
#include <willma.h>
#include <fred.h>
Fred fred; // error: undeclared identifier

The most common reasons for this are:

The quickest way to find out what is happening to your file is to add a message which identifies it at compile time, e.g.

#pragma message("This is fred.h")

These can be sprinkled around liberally, but remember you will want to remove them before check in. The best places to put these are:

If you place a message inside the include guard remember that you should only see it once for each compilation unit, e.g. each .cpp file, but you will most likely see it multiple times during a complete build because multiple compilation units will probably include the file.

In practice the pragma message is the most powerful tool for debugging header file issues for most problems. If your compiler does not support pragma message, try #error. Or, add an actual syntax error to the file, these are seldom fatal to the compile but will show you what is happening

Variations on the "it is included" theme are:

One common error is the type redefinition. This happens when you declare something, e.g. class Table, and something you have previously included declares typedef... Table. Hopefully my rules on include file order should help avoid this. However, this is not always possible.

Developers do not often make this mistake in their own code, but third party code, be it OS headers, bought in libraries, or in house libraries usually contain the other copy. (Hopefully, with namespaces this kind of thing will become less frequent.)

If your lucky you can place your cursor on the offending definition, hit F1 and the on-line help will tell you about the OS type of the name - at which point you must give in gracefully and rename yours. Unfortunately, much that is defined in OS headers is not documented, at which point grepping the code may help.

A quicker way to find the first declaration is too cheat. Add a quick declaration before all your includes, something obscure like: typedef short <Offending identifier>, and then recompile. Normally this will give you the same error, but this time on the first declaration. Once you have removed your extra declaration you will know which items conflict and resolve the conflict.

A particularly nasty form of this involves macros with the same name as types or members, e.g. Sun defines a macro called minor in sysmacros.h, once this file has been included somewhere you will have problems compiling code with CORBA exceptions as these define a member called minor.

As I said right at the start of these articles, including a header file is very primitive. If you write:

// file: fred.h
class Fred {
  ...
// <eof>
// file: flintstones.h
#include <fred.h>
  ...
// <eof>

You will at least get an unexpected end of file error, and possibly a lot of other strange syntax errors. It is obvious here but it is easy to miss because the file it shows up in, in this case, flintstones.h, is a long way from fred.h. Potentially, flintstones.h did not include fred.h directly, but included family.h, which included "characters.h".

Again cut and paste programming is a common cause of this error; legwork, and a few pragma messages are the best attack.

One short cut which you can make is to comment out everything (#includes and source code) in the .cpp file and re-introduce the includes one at a time until the error appears. Check the file you just included, if you are lucky it is guilty, if not repeat the process with its include's too.

Hi Allan,

I enjoyed your "Include Files" article in Overload issue 39 (September 2000), and would like to share some comments.

Re Rule 2: You argue that forward-declaring the standard "string" typedef yourself is impractical, but it's much stronger than that: It's illegal. The only thing you can do yourself in namespace std is declare specializations of standard templates; you can't declare standard names like basic_string.

Re Rule 3: I agree with the rule, but I don't think your rationale is quite right. You write, "I can use forward declarations with pointers and references, but I[sic] not with [objects]," which is correct if you're talking about function definitions but isn't correct if you're just talking about function declarations (which is what you seem to be talking about in the context of the other Rules). For example, most people are surprised that the following code is a complete header and does not require a definition for class X, because you only need the definition when you need to generate code that must know the size of X (e.g., allocate stack space for a local object or a function value parameter) or call a member function of X (e.g., copy it):

// ...usual include guards here and at
// the end...
  void g( X&, double );
  void h( X*, double );
  class Y {
  public:
    X& f( X& ); 
// doesn't need definition of X
    X* f( X* ); // ditto
    X  f( X  ); // ditto
  };
  inline X& Y::f( X& r ) {
    g( r, 1.0 ); 
// use the reference: still don't need 
// definition of X
  }
  inline X* Y::f( X* p ) {
    h( p, 1.0 ); // use the pointer:
// still don't need definition of X
  }
// can't define Y::f(X) without X's 
// definition, though, because that
// would actually ask the compiler to
// generate code to call the copy 
// constructor and know the object's size

See also Item 26 in Exceptional C++

Re Rule 9: I don't understand this rule. When you say, "Think about include guards, especially for libraries," I can't see how this can apply to non-header files, yet Rule 8 just said that every header should have include guards (and that's right). What is Rule 9 intended to apply to?

Incidentally, I strongly disagree with Lakos' external include guards on two grounds:

Re Rule 10: No, never! Even if you disagree with me above and go ahead with external guard, your objections really do apply. If you forget (or mistype) the external guards in even one place...

Re Rules 12-14: Agreed. It's also worth noting that header files should always be written such that they do not depend on #include file ordering (e.g., being included before vs. after any other header files). This actually impacts namespaces and it's a reason I argue that using declarations should never appear anywhere in a header file, not even in a namespace (see my article Migrating To Namespaces in Dr Dobbs Journal, 25(10), October 2000).

Herb

First, thanks for taking the time to read my thoughts and pass on your comments. Interestingly, John Merrells has part two of the piece in the works for Overload 40, in this I added a paragraph as a result of reading Exceptional C++ to clarify rules 2-3. I agree completely with your comments on my rule 3 - forward declarations can be used as long as no code is generated; my writing was a little confused.

In rule 2 (concerning forward declarations in separate files) I was trying to use the string type as an example of how forward declarations could get complex. Unfortunately I chose a bad example. In fact I didn't know re-opening std namespace was illegal, I should have done but I didn't. I wonder how many developers actually know this? Now you've brought it to my attention it's obvious. Perhaps more importantly: do any compilers enforce this rule? I suspect few if any. I suggest we shout it loud: "don't add anything to std namespace."

(This is kind of unfortunate, to my mind, one of the strongest advantages of a namespace over a class is that it can be re-opened and items added but many novices first encounter with namespaces will be with std, one or both points may well be lost on them.)

Rule 9: Reading it again I see I should have phrased it as "Think about external include guards, especially within libraries." My objective was to explore the Lakos argument about external include guards. Like yourself I have problems with Lakos' argument, while he presents a convincing case for improving compile times increasingly I agree with you. In the long run I think external guards make the source code more difficult to read and increase the coupling between a header file and the file's clients. Yet the benefits, if any, will on be apparent on very large projects.

I deliberately said "think about" because I didn't want to give a definite rule, I wanted to make people aware of the argument, to this end your comments are an excellent contribution to the case. Rule 10: I never intended to advocate the removal of internal include guards. We agree here: always leave them in even if you have external guards!

I find it very interesting that you agree with my rules 12 to 14 "order includes with most inflexible first, most flexible last." These rules where the genesis of the article, they have become a fixture of my coding style. However, they go directly against Lakos' advice which states that you should order from local to system. (Not only this, Kevlin's article in the same issue of Overload upholds the Lakos rule.) I hope we have started a debate here.

I hope you'll forgive this novice-writer a few slips and enjoy my continuation of the subject in Overload 40. Once again, many thanks for your comments.

Allan