Title: C++ Patterns Source Cohesion and Decoupling

Large-scale structure in C++ programs is dominated by use of the preprocessor for file inclusion. However, the relationship between physical source, such as source and header files^[1], and logical source, such as declarations and definitions, is not always a happy one. The same can be said of Bjarne Stroustrup's view of this C legacy in C++: "I'd like to see Cpp abolished" [Stroustrup1994]. Evidently there would be little love lost here if this were to happen.

The preprocessor is blunt tool for expressing module relationships, based as it is on semantically-unconstrained textual inclusion:

There is no language-recognized module structure for a system, so physical partitioning is notionally optional and based strongly on habit and convention - always a topic for heated debate. As perceived by the programmer, this leads to unnecessary incidental complexity when working with source - whether it is writing it, compiling it, or versioning it.

If unmanaged, build times can rise exponentially with the number of source and header files. In smaller projects this offers plenty of opportunities for coffee breaks and meetings, but on larger projects it introduces the bind of overnight builds.

An unwanted side effect of preprocessor usage is that legitimate code may be terrorized and harassed by wild macros.

These are common forces and consequences in development. Unmanaged physical dependencies thus make systems harder to understand and harder to build, putting the brakes on rapid, incremental development and turnaround. The pattern language presented in this paper documents and links a set of practices that balance these forces.

Table 1 lists the patterns described here in thumbnail form, providing a summary of the problem and solution for each. Other key patterns used in this paper are described toward the end in thumbnail form in Table 2.

Figure 1. Patterns and their successors for organizing C++ source and header files.

The patterns may be organized into a pattern language, as shown in Figure 1. Cohesive Header defines the natural entry point into the language, the remaining arrows showing the subsequent flow through the language.

The patterns are presented using a brief, low ceremony pattern form: pattern name followed by intent; a description of the problem, identifying the forces; a detailed description of the solution, identifying the consequences.

What partitioning criteria should be used in allocating abstractions to header files in a system?

It is possible for a programmer to develop a small project in a single file. However, although this apparently simplifies tool usage - compilation, linkage, versioning, etc. - it does not scale well: Comprehensibility falls rapidly with size, editing becomes increasingly awkward and Cobol-like, and opportunities for developer teams, parallel development, separate compilation, fine-grained versioning, reuse, unit testing, etc. are all but eliminated. Such a seductively easy route leads quickly to a dead end.

Separate source files in part resolve these issues, but introduce the need for different source files to share common function declarations and type definitions. The temptation is to revert to a monolithic header file that includes all declarations and definitions needed across a project. As with the single source file approach, such a "project.h" requires a particular presence of mind to ensure the internal organization of the file does not degenerate into cryptic spaghetti^[2], a secret dish known only to those initiated in its history and development.

Dividing the declarations and definitions of a project across many header files increases the opportunity for separate development, and many of the benefits that it brings. However, partitioning alone is insufficient to bring in all of the benefits, and may also be used inadvertently as a tool to reduce comprehensib-ility: The partitioning may be coincidental or inconsistent, making the apparent structure of the system difficult for a programmer to navigate, modify, and extend. The resulting header files may be large and still difficult to comprehend, offering apparently unrelated concepts in a single place, i.e. "kitchensink.h" (because they contain "everything but the kitchen sink").

Breaking declarations and definitions across multiple header files will lead to implied dependencies between header files, where one declaration depends on another. This can be awkward to manage in source files as the programmer must be aware of and track such subtle relationships, ensuring that they #include the right files in the right order.

Organize a header around a single concept, such as a class or set of services. The header should declare or include everything it needs to be self-contained, but no more: It should also be a Minimal Header. A Source/Header Correspondence ensures that the implementation of features offered in a Cohesive Header is easy to find. An Idempotent Header ensures that multiple inclusion of a header is transparent in non-hierarchical inclusion graphs (i.e. directed acyclic graphs).

A Cohesive Header represents a physical interface that corresponds to the logical structure. It represents a unit of common release and change. For instance, a Cohesive Header may be organized around a single class and its dependent functions. It would include any associated operators and algorithms, helper classes, and namespace wrapping, as well as its own inclusion of any headers whose contents it depended on. Thus to gain access to the full use of a class a programmer need only determine which Cohesive Header corresponds to that concept and include it.

Cohesive Header supports and is supported by the Interface Principle [Sutter2000], where "for a class X, all functions, including free functions, that both 'mention' X and 'are supplied with' X are logically part of X, because they form part of the interface of X".

A Cohesive Header should be named after its principal concept, i.e. a class or general purpose, followed by a suffix, such as .hpp or .hh. The suffix is generally dependent on the project and the platform culture. Although the standard header files omit suffices, e.g. <string> and <list>, such files cannot be associated with applications or found conveniently using wildcard matching. Many source code editors use syntax highlighting based on identifying the language through the suffix, and the absence of a suffix introduces an inconvenience. The .h suffix is generally not recommended because of its more standard association with C header files, so that C++ headers cannot be readily distinguished from C headers by either programmers or tools.

Where namespaces are used to represent subsystems or packages, as opposed to simply wrapping up a single class and its dependent functions, the physical directory structure should correspond to the logical source structure. This again makes it easier to find and comprehend the header in question, and should also be made apparent in the inclusion by client code, i.e. #include <subsystem/concept.hpp> rather than simply #include <concept.hpp>. Such partitioning was relatively common before the introduction of C++ namespaces, and is reinforced by their addition as well as the adoption of the same model as the basis of Java's package mechanism.

Partitioning a system's logical interfaces across Cohesive Headers will result in more header files than other partitioning criteria, which means that there are more files for the programmer to be aware of and to manage. However, each of these files will be smaller, more cohesive, and - therefore - easier to comprehend and manage, e.g. change history will clearly reflect the stability of and rate of change of a concept. If the compiler's own file management is limited, many small header files on a large project can have a knock-on effect on the build time as the compiler will be performing more file opening, reading, and closing operations.

As with many forms of source partitioning, Cohesive Headers is a convention that requires judgment - i.e. whether or not the contents of a given header are genuinely cohesive. If adopted by developers, it is a convention that is enforced beyond the control of the compiler.

What should be the relationship between header files that declare the abstractions in a system and the source files that define them? Having partitioned the physical view of the logical elements a system into Cohesive Headers, header file clients can now easily locate functionality for inclusion. But what about programmers working on the actual defining source code that realizes the features in Cohesive Header? It should be simple for them to locate definition code for reading, modification, or extension based on the header, as well as know which header to modify if a change requires that they be kept in synch.

Ada and Modula-2 have a clear correspondence between the exported view of a package or module, but C++'s preprocessor does not enforce any model, focusing only on the composition of a translation unit, by inclusion and macro expansion, for further compilation. It would be equally legitimate for a system to have its function and data definitions grouped into a single source file as it would be for each individual function or data item to be assigned its own source file. The former will create a slab of code that defies software engineering principles and the latter will create a finely fragmented system structure that is overwhelming for both programmers and compilers, leading to astonishingly long build times.

For each Cohesive Header define a corresponding source file with the definitions to the declared abstractions. The source file includes the header and has the same base name as the header.

Source/Header Correspondence effectively splits an abstraction into two: Its declarative or export part, and its defining or implementation part. This longstanding practice follows the structure enforced by languages such as Ada and Modula-2. The two files should have the same base name and differ only in the appropriately corresponding suffices, i.e. .hpp and .cpp or .hh and .cc. The source file is also a unit of information hiding within which the use of anonymous namespaces or the deprecated use of static can keep additional implementation abstractions private. Source/Header Correspondence makes the source structure self-evident and easy to work with.

However, with inline functions and the include-all model for template compilation, C++ does not follow a perfect specification/implementation separation. Source/Header Correspondence may be further generalized to include files for inlines and headers for templates, so that a header file provides function and data declarations and class definitions only, and includes source for template definitions and inline function definitions as separate files. Such a separation of concerns simplifies source management.

A source file includes its own corresponding header file, as well as any other system and project files required for its definitions. The recommended order for inclusion in the source file first #include the corresponding header, next any headers developed for the project, and finally any other system or third party headers. Within each of these groupings any consistent ordering, such as alphabetical, will suffice. This reinforces a Cohesive Header by ensuring that headers are genuinely self-contained, not accidentally relying on features they do not themselves include.

What steps should be taken to deal with header files that include common header files, which may lead to multiple inclusion problems?

It is common for header files in projects to require the same abstraction, and therefore as Cohesive Headers to include them. Any other source or header file that includes these will then be including a common header file at least twice. This can potentially cause problems: In a single translation unit a class or function, including inline functions, may be defined only once; multiple inclusion leads to multiple definitions, and therefore failed compilation. For instance:

// timer.hpp
#include "callback.hpp"
....

// viewer.hpp
#include "callback.hpp"
....

// main.cpp
#include "timer.hpp"
#include "viewer.hpp"
....

Such directed acyclic graphs of dependencies are inevitable in any well-factored project, so it is often impossible as well as impractical to mandate that repeated inclusion "shall not happen". Alternatively if header files include nothing and source files are required to resolve all of the dependencies, the problem can be resolved technically. However, the header files are no longer Cohesive Headers and a source file programmer is required to know and document explicitly not only what the header filesdependencies of each include file are, but also the required ordering. This makes header file usage cumbersome and unpleasant; there is more to learn, and more to miss. In particular, the programmer must be aware of the private section of each class and dependencies of each inline function or template. This is acceptable for the developer of the header file, but not for its user. It simply adds complexity and means that can break in a build.

A header file should have the same effect, no matter how many times it is included. Use a Header Guard to ensure that its contents are expanded out only the first time a header is included in a translation unit.

An Idempotent Header may be included multiple times in producing the same translation unit without harm. This allows Cohesive Header partitioning and simplifies the use of header files from the point of view of its users, ensuring that they are self-contained and work out of the box.

The portable and conventional way to implement an Idempotent Header is to use a Header Guard, but some compilers support a #pragma that ensures that the body of a header is expanded out by the preprocessor only the first time it is encountered. In principle, there are portable cases where an Idempotent Header need not use a Header Guard: Declarations can be repeated without the ill effect caused by repeated definitions. However, for consistency and surprise-reduction a Header Guard is always recommended as the support for an Idempotent Header.

What mechanism can be used to prevent multiple inclusion of a header file leading to multiple expansion of its contents, and hence multiple definition problems?

Idempotent Headers simplify the use of headers, but the provider of the header file must ensure that such headers genuinely are idempotent, avoiding the problems arising from multiple inclusion of definitions. As this support is not automatic within the language, programmers are required to provide it themselves, and the conventions used should be consistent across a project.

Test for the definition of a macro - based on the capitalized version of the header's base name - only including the content of the header if it is undefined. Define the macro only if the content is included. A conditional compilation guard therefore surrounds the body of the header.

The basic schema for a Header Guard is

#ifndef HEADERBASENAME_INCLUDED
#define HEADERBASENAME_INCLUDED
.... // body of header
#endif

There are many different conventions for naming the guard macro, and any convention adopted should meet the following criteria:

Macros should follow convention by being all in uppercase, i.e. HEADERBASENAME_INCLUDED is OK, but not HeaderBaseName_Included.

Leading underscores followed an uppercase letter are reserved for use by the compiler and should be avoided, i.e. _HEADERBASENAME_INCLUDED should not be used. The same is true for any use of double underscores, i.e. HEADERBASENAME__INCLUDED should also be avoided.

Any directory name required for inclusion should also be encoded in the name to avoid clashes with other files with the same base name, i.e. SUBSYSTEM_HEADERBASENAME_INCLUDED.

Any suffix used should consider the effect of moving header files across platforms where the header suffix might be changed in line with any compiler-required changes to the suffix of the source file, i.e. HEADERBASENAME_H and HEADERBASENAME_HPP suggest they are tied to a particular header suffix whereas HEADERBASENAME_INCLUDED and HEADERBASEDNAME_HEADER do not. The suffix can be omitted altogether, but note that the macro then looks like any other macro as opposed to a header guard, i.e. HEADERBASENAME.

The convention should be used consistently across a project.

This last point is important as it conforms to the principle of least astonishment, especially when external guards are used for larger files: Although repeated inclusion with a Header Guard does no harm, the file is still read by the compiler, which can add significant time for large headers and, on some systems, can exhaust the number of file handles available to the compiler. In such cases, a build optimization is possible by using redundant external guards around an include [Lakos1996]:

#ifndef HEADERBASENAME_INCLUDED
#include "headerbasename.hpp"
#endif

All header files should, regardless, define Header Guards. The programmer must remember to do this consistently, and this can be helped by using boilerplates or editor scripting.

How can a Cohesive Header include no more information than is strictly necessary when its classes and functions mention types defined in other header files?

A programmer, assuming that a header is not only self-sufficient but also sufficient, may assume that all the headers included by the header are genuine dependencies of the header's contents. However, it is often the case that certain headers are included by force of habit or laziness to include certain utility files elsewhere, e.g. #include <iostream> or #include <string>. Such headers are often not required by either the header or many of the source files. This makes the dependency graph for a system confused, so that listing #include directives does not give an accurate reflection of the necessary and logical dependencies in a system. It also unnecessarily increases the build time for a given header.

In other cases, although all of the headers included are Cohesive Headers and their concepts are used in the including header, e.g. use of a class name, inclusion of a full header is not always necessary to allow the code to compile. In these cases the inclusion is mostly redundant and creates a compile-time overhead.

Include only the headers that define non-class types and the headers for classes whose definition is genuinely required, e.g. base classes and data members by value. For each other class that is used, e.g. as a function argument or a pointer member, provide a Forward Declaration.

A Minimal Header supports Cohesive Header principles by ensuring that what is defined by a header is necessary and sufficient, that it is standalone. It is the source file programmer's responsibility to include any other header files they see fit, but it is not the job of the header file programmer to cater for a "just in case" need or to predict other possible inclusion requirements of the header file client.

In projects that have come to rely on this, making the change will involve an initial refactoring that breaks a lot of compilation. However, the required changes will be easy to identify and make, and following that change the system build time will be faster and header files more self descriptive.

Header files with inlines and template definitions will typically require many more inclusions because they require implementation detail that would otherwise be in a source file. For inline functions, it is best to be sure that they are genuinely required and that they do have a measurable positive effect on the runtime performance or footprint of the system, otherwise they are simply an excuse for laziness at the expense of dependency management and build times. For templates, techniques such as hoisting [Carroll-1995] can be used in reducing the inclusions required to allow compilation of a template.

How can a class be referred to in a header without causing its full definition to be included for compilation?

Not all types that are included in a header are fully used in that header, e.g. a type name may be used to declare a prototype of as a pointer data member. For a class included for use as a base class or whose instances are manipulated in inline or template functions, the full definition is clearly required. For these other cases, where only a mention of the name is required, inclusion of the full definition seems wasteful.

It is possible to forward declare a class or struct name for use in function prototypes, class static and extern data declarations, pointer or reference data members, and some template parameters. However, this does not apply to other type names, e.g. enum or typedef names. Thus, for classes (and structs) declare only the name in the header file, placing it after any included files. In the defining source file include the appropriate header file for all forward-declared types that are used. A Forward Declaring Header simplifies forward declaration for complex type names.

For classes defined in a namespace, the namespace can be reopened for the forward declaration:

namespace subsystem
{
    class concept;
}

Forward Declaring Headers can make this less of a chore.

Forward Declaration supports Minimal Headers based on the principle of sufficiency, leading to improvements in build times and reduction of recompilation when headers are changed. The onus is now placed on the file including the header file to include the full definition, assuming it is needed (not always the case).

Forward Declaration's role as a dependency management idiom is the basis of the Opaque Type idiom, which in turn, and along with Handle/Body, forms the basis of the Cheshire Cat idiom. Another technique for eliminating dependencies arising from private implementation detail is at the logical rather than the source level: Defining an Interface Class uses inheritance to eliminate any private section, and thus any dependencies from it.

How can Forward Declarations for complex types, such as template definitions with defaulted parameters, be declared in a Minimal Header without hardwiring and duplicating the declaration information over the system?

For instance, if a programmer wished to declare a prototype that took an ostream from the standard library as one of its arguments without including any of the sizeable I/O stream headers, a Forward Declaration would suggest itself as the solution:

namespace std
{
    class ostream;
}

However, this will not work because ostream is a typedef name that hides a more complex type:

namespace std
{
    ....
    template<
        typename char_type,
        typename traits = 
            char_traits<char_type> >
      class basic_ostream;
    typedef basic_ostream<char> ostream;
    ....
}

Declarations of this complexity make Forward Declarations by the including programmer impractical and error-prone, as well as susceptible to minor changes, e.g. if a default template parameter were to change.

For such complex types, provide an additional header file that comprises only Forward Declarations. Clients now include the Forward Declaring Header rather than providing their own Forward Declarations.

A Forward Declaring Header requires the anticipation of the programmer providing the principal header. The Forward Declaring Header includes the relevant declarations, dealing with namespace and template parameter issues, for the user to include directly. The principal header file can itself include the Forward Declaring Header, helping to ensure that its Forward Declarations are in synch with the actual definitions, and simplifying the management of what is now an increased number of files.

The role of the header file as a Forward Declaring Header should be made clear by use of an appropriate convention, such as appending fwd to the base name. For example, #include <iosfwd>. If a consistent and clear convention is followed, users will be more aware of these header files, and will be more likely to use them to reduce their dependencies and build times.