Title: An Introduction to FastFormat (Part 3): Solving Real Problems, Quickly

Author: 

Date: 10 June 2009 09:56:00 +01:00 or Wed, 10 June 2009 09:56:00 +01:00

Summary: A good library must be useful in practice. Matthew Wilson looks at usability and extendability.

Body: 

This article, the third and last of the current series on the FastFormat formatting library, discusses several use cases, from real world projects and discussion forums, that illustrate how the library can be used to achieve concise, transparent application code while utilising its flexibility and performance advantages. It is about solving real formatting problems, quickly: both in speed of development and speed of executed code.

Along the way, we'll look at some of the more esoteric aspects of the application layer, customisation of the format specification defect handling, and consider cases where suppressing unused argument exceptions is useful.

Introduction

This article is divided into two halves. The first half describes six use cases, four of which are from real applications, four of which demonstrate improvements to application code, and four of which involve performance benefits. (Not the same four.) I'll present performance measures for each scenario for which it is relevant, putting more flesh on the performance characteristics suggested in the Yaffle scenario from part 1 [FF1] . The second half of the article is a mix of miscellaneous but useful tips for taking your use of the library further. Along the way we'll see examples of FastFormat interoperability with MFC, ATL, and the Pantheios logging library [PAN] .

The first two scenarios are pedagogical, contrasting the use of FastFormat for formatting columnar floating-point data, and in formatting according to absolute tabulations. The benefit of using FastFormat in these cases is primarily in performance.

The next two are extracts from client codebases, and involve server connection logging and database insert statement preparation. These demonstrate improvements both in application code transparency and performance.

The last two scenarios do not involve performance improvements. (Well, they might, but that's unimportant.) Rather, they involve substantial improvements in the transparency of application code by dint of FastFormat's expressiveness and flexibility.

FastFormat in action

Several of the scenarios described here are extracted from clients' work, and so names have been changed and types simplified. Also, the samples assume the inclusion of the fastformat/ff.hpp header to alias the fastformat namespace to ff, and the inclusion of whatever other headers are required for the various types and libraries involved. The full source of all the programs are included with the FastFormat distribution (version 0.4+).

Except where stated otherwise, the performance of each scenario was ascertained by invoking each of the statements 10,000 times, and repeating that loop three times, taking the times on the third outer iteration to minimise environment effects (since we're writing to stdout). The output was piped to a bit-bucket program - just a getchar() loop - to remove the latency of writing to the console/terminal from the measured times. The results are presented in milliseconds. The tests were conducted on Mac OS-X with GCC 4.0 (32-bit), on Linux with GCC 4.1 (64-bit), and on Windows with Visual C++ 9 (32-bit).

Floating-point columns

The first scenario is based on a question about C++ formatting on StackOverflow [SO] , which asked how to do the following using the IOStreams:

      printf("%-14.3f%-14.3f\n", 12345.12345,  
      12345.12345);  

resulting in the output (where ⋅ represents a space):

      12345.123⋅⋅⋅⋅⋅12345.123⋅⋅⋅⋅⋅  

The implementations for IOStreams, Boost.Format, and FastFormat are shown in Listing 1. Note that I've added enclosing square braces as an aid to verifying that they all produce identical output. (This was done simply by running them all through uniq, a little trick I learned from a UNIX guru long ago.)

// Streams printf( "[%-14.3f%-14.3f]\n" , 12345.12345 , 12345.12345 ); // IOStreams std::cout << '[' << std::setiosflags(std::ios::fixed) << std::left << std::setprecision(3) << std::setw(14) << 12345.12345 << std::setw(14) << 12345.12345 << ']' << std::endl; // FastFormat.Format ff::fmtln( std::cout , "[{0}{1}]" , ff::to_f(12345.12345, -14, 3) , ff::to_f(12345.12345, -14, 3) ); // Boost.Format std::cout << boost::format("[%-14.3f%-14.3f]\n") % 12345.12345 % 12345.12345;

Listing 1

All present clear and transparent code except, in my opinion, the IOStreams, due to the verbosity of the code and the inconsistent semantics between the manipulators setprecision (sticky) and setw (non-sticky). Of course, some may argue that the explicit nature of something like setprecision is far more transparent than an arcane squiggle such as "%-14.3f". So the issue of transparency here is somewhat subjective.

Totally objective, however, are the performance results, in milliseconds, are shown in Table 1. It's no big surprise to see that Streams is the standout performer here (since all the others form their floating-point arguments in terms of sprintf()).

Times (in ms) for 10,000 invocations of the Floating-Point Columns scenario Library GCC 4.0 (32) GCC 4.1 (64) VC++ 9 (32) Streams 14.7 33.5 43.9 IOStreams 81.1 91.8 152.2 Boost.Format 103.3 104.0 223.8 FastFormat.Format 21.6 42.5 64.7 Boost.Format (1-arg) 109.7 105.6 224.9 FastFormat.Format (1-arg) 13.4 27.2 38.9

Table 1

Since Boost.Format and FastFormat.Format both allow an argument to be reused, we can simplify the statements and just have one argument, as in Listing 2 (floating-point column solutions reusing a single parameter). While there's little realism for the current scenario, there are cases where it is useful to use arguments multiple times, so it's interesting to see the effect. The times are shown in the bottom two rows of Table 1.

// FastFormat.Format ff::fmtln( std::cout , "[{0}{0}]" , ff::to_f(12345.12345, -14, 3) ); // Boost.Format std::cout << boost::format("[%1$-14.3f%1$-14.3f]\n") % 12345.12345;

Listing 2

As expected, FastFormat's time drops, making it even faster than Streams for this edge case. The Boost.Format times stay almost exactly the same, so we may assume that it must perform the argument conversion twice.

Tabulations

One of Boost.Format's advanced features is the ability to apply absolute tabulations, something none of the other examined libraries is able to do. The example given on the library's website, assumes three vectors of strings, such that when used with the following statement

      std::cout   
        << boost::format("%1%, %2%, %|30t|%3%\n")  
          % forenames[i]  
          % surnames[i]  
          % tels[i];  

the output is as follows:

      Marc-François Michel, Durand, 0123 456 789  
      Jean, de Lattre de Tassigny,  0987 654 321  

As I mentioned in part 1 [FF1 ], FastFormat is able to support absolute tabulations with a little indirection. The above output can be obtained as shown in Listing 3 (synthesising absolute tabluations with FastFormat.Format)..

std::string scratch; ff::fmtln( std::cout , "{0,40,,<}{1}" , ff::fmt( scratch , "{0}, {1}, " , forenames[i] , surnames[i] ) , tels[i] );

Listing 3

Clearly it's not as transparent as the Boost.Format statement, but it's not opaque either. And given the relative performances (Table 2), the picture's not too bad.

Times (in ms) for 10,000 invocations of the Tabulations scenario Library GCC 4.0 (32) GCC 4.1 (64) VC++ 9 (32) Boost.Format 237.2 141.8 366.8 FastFormat.Format 37.0 46.0 149.1

Table 2

Server connection Log

This next scenario is extracted from a client's proprietary internetworking server (UNIX and Windows), which writes out connection event logs containing connection identifier, addresses + port, time and bytes transferred. Consider the following fictionalised structure:

      struct connection_t  
      {  
        std::string     connectionId;  
        struct in_addr  remoteAddress;  
        struct in_addr  localAddress;  
        unsigned short  port;  
        unsigned long   numBytesTransferred;  
        struct tm       completionTime;  
      } conn;  
 
      conn.remoteAddress.s_addr = htonl(0xC0A8A0f7);  
      conn.localAddress.s_addr  = htonl(0x7f000001);  
      conn.port                 = 5651;  
      conn.numBytesTransferred  = 102401;  
      conn.completionTime       = . . . // now
      conn.connectionId         = "channel-1";  

Currently, the format of the log is (though this may change):

      <id> <time> <remote-addr> <local-addr> <port>  
      <bytes>  

giving an output along the lines of:

      channel-1 May 03 03:50:41 2009 192.168.160.247  
      127.0.0.1 5651 102401  

This can be achieved simply with both FastFormat APIs, as shown in Listings 4 and 5. In this case I think the FastFormat.Format version is more transparent, and makes reordering of the replacement parameters a trivial matter (as suggested by the non-sequential format string in the example).

• Listing 4 shows a FastFormat.Write implementation of the Server Connection Log scenario

void log_connection(connection_t const& conn) { ff::writeln( stm , conn.connectionId , " " , conn.completionTime , " " , conn.remoteAddress , " " , conn.localAddress , " " , conn.port , " " , conn.numBytesTransferred ); }

Listing 4

• Listing 5 shows a FastFormat.Format implementation of the Server Connection Log scenario

void log_connection(connection_t const& conn) { ff::fmtln( std::cout , "{0} {5} {1} {2} {3} {4}" , conn.connectionId , conn.remoteAddress , conn.localAddress , conn.port , conn.numBytesTransferred , conn.completionTime ); }

Listing 5

With Streams, IOStreams or any other library that does not support the automatic insertion of struct in_addr and struct tm, providing equivalent functionality is going to involve extra effort. Consider the Streams version (Listing 6), which is pretty close to the original implementation. There's considerably more code, and it's clear why the log format was originally difficult to change.

void log_connection(connection_t const& conn) { char time[21]; size_t n0 = strftime( &time[0], STLSOFT_NUM_ELEMENTS(time) , "%b %d %H:%M:%S %Y" , &conn.completionTime); STLSOFT_ASSERT( n0 < STLSOFT_NUM_ELEMENTS(time)); uint32_t ra_l = ntohl(conn.remoteAddress.s_addr); uint32_t la_l = ntohl(conn.localAddress.s_addr); fprintf( stdout , "%.*s %.*s %d.%d.%d.%d %d.%d.%d.%d %d %lu\n" , int(conn.connectionId.size()) , conn.connectionId.data() , int(n0), time , ((ra_l >> 24) & 0xff), ((ra_l >> 16) & 0xff), ((ra_l >> 8) & 0xff), ((ra_l >> 0) & 0xff) , ((la_l >> 24) & 0xff), ((la_l >> 16) & 0xff), ((la_l >> 8) & 0xff), ((la_l >> 0) & 0xff) , conn.port , conn.numBytesTransferred ); }

Listing 6

The brittleness of the Streams example format string can be obviated by using IOStreams, although it does require the definition of two inserters (Listing 7: IOStream Inserters for struct tm and struct in_addr) in order to cut down on the amount of application code.

inline std::ostream& operator <<( std::ostream& stm , struct tm const& t ) { char time[21]; size_t n0 = strftime( &time[0], STLSOFT_NUM_ELEMENTS(time) , "%b %d %H:%M:%S %Y" , &t); STLSOFT_ASSERT(n0 < STLSOFT_NUM_ELEMENTS(time)); return stm.write(time, n0); } inline std::ostream& operator <<( std::ostream& stm , struct in_addr const& addr ) { uint32_t ra = ntohl(addr.s_addr); return stm << ((ra >> 24) & 0xff) << '.' << ((ra >> 16) & 0xff) << '.' << ((ra >> 8) & 0xff) << '.' << ((ra >> 0) & 0xff); }

Listing 7

With these, we can now write a much improved version using IOStreams (Listing 8).

void log_connection(connection_t const& conn) { std::cout << conn.connectionId << ' ' << conn.connectionTime << ' ' << conn.remoteAddress << ' ' << conn.localAddress << ' ' << conn.port << ' ' << conn.numBytesTransferred << std::endl; }

Listing 8

It's worth noting that the compatibility for struct tm and struct in_addr that is afforded to FastFormat (and other libraries, such as Pantheios [PAN] ) by STLSoft's string access shims (see [FF2] , [XSTLv1] , [IC++] for more details) is automatic. You don't have to define anything to make use of it, merely ensure the right #includes are made. This is a clear win for FastFormat over IOStreams in the case of struct in_addr, since the dotted-decimal format (e.g. 127.0.0.1) is widely accepted. However, with struct tm, it is only convenient by accident, since the string access shim format - a là strftime() - is equivalent to that required by the server log. If any other format was required, then you'd either have to write an inserter function or customise via the filter_type mechanism (see [FF2] ), each of which is equivalent to the effort of defining an inserter.

The relative performances are shown in Table 3. It's no surprise that the IOStreams solution fairs poorly, but it is interesting to see how FastFormat (in either guise) is on a par, or even better in some cases, than Streams in a non-trivial real-world example. Given the substantial differences in transparency of the code, FastFormat would appear to be a clear winner in this case.

Times (in ms) for 10,000 invocations of the Tabulations scenario Library GCC 4.0 (32) GCC 4.1 (64) VC++ 9 (32) Streams 53.0 59.5 86.6 IOStreams 94.4 111.3 248.5 FastFormat.Format 56.0 55.0 86.7 FastFormat.Write 49.7 47.8 86.5

Table 3

Database insert statement

This next scenario is from a client's (UNIX) codebase. The particular code is to form a database insert statement, part of a high volume data processing subsystem. The client does, er, financial things, and they're extremely cagey about their work, so I hope you'll forgive the heavy obfuscation of names and types.

The original implementation was done using std::stringstream to form the statement, along the lines shown in Listing 9.

const int intNaN = 0x7fffffff; class BusinessAdaptor { . . . public: std::string m_tablename; int m_id_1; std::stringstream m_ss; std::stringstream m_slice; }; std::string BusinessAdapter::insertRecord( const BusinessRecord& r ) { m_ss.str(""); m_ss << "insert[" << m_tablename << ";("; m_ss << makeSlice(r.member_1) << ";"; m_ss << makeSlice(r.member_2) << ";"; m_ss << makeSlice(r.member_3) << ";"; m_ss << makeSlice(r.member_4) << ";"; m_ss << makeSlice(m_id_1) << ";"; m_ss << makeSlice(r.member_5) << ";"; . . . // same for members 6 => 18 m_ss << makeSlice(r.member_19) << ")]"; return m_ss.str(); }

Listing 9

The member_?? variables are all integers. The makeSlice() helper function converts an integer to a string except where it is equal to the application-defined sentinel constant intNan, in which case it is converted to the string "0N", as in:

      std::string BusinessAdapter::makeSlice(  
        const int val)  
      {  
        m_slice.str("");  
        if ( val == intNaN )  
          m_slice << "0N";  
        else  
          m_slice << val;  
        return m_slice.str();  
      }  
 

The client is a heavy (and happy) user of a customised version of Pantheios [PAN] , and wished to know whether there were similar performance gains to be had in a more general way in the manipulation of strings. This was at a time before FastFormat had been released (and this was a primary impetus to my getting it out there), and I was able to show them the performance speed-up shown in Table 4; in this case the times are measured in microseconds for 10,000 invocations (since there's no I/O).

Times (in µs) for 10,000 invocations of the Database Insert Statement scenario Library GCC 4.0 (32) GCC 4.1 (64) VC++ 9 (32) IOStreams 1210 1546 5378 FastFormat.Format 533 456 569 FastFormat.Write 468 396 530

Table 4

The two FastFormat implementations are shown in Listings 10 and 11. Neither can be said to be significantly more transparent than the original. Actually, in this case I'd concede that FastFormat.Format is less transparent, simply because having 21 replacement parameters in a format string is a challenge to maintenance. None of the solutions are easy on the eye, probably because a string is being built from 21 arguments. The primary discriminant in this case is performance. I would observe that at least the FastFormat.Format version has an extra level of error-checking over the other two, since if there are too many or too few arguments, an exception will be thrown.

• Listing 10 shows a FastFormat.Format implementation of the Database Inserter Statement scenario

std::string BusinessAdapter::insertRecord( const BusinessRecord& r ) { std::string result; ff::fmt( result , "insert[{0};({1};{2};{3};{4};{5};{6};{7};{8};{9}; {10};{11};{12};{13};{14};{15};{16};{17};{18};{19}; {20})]" , m_tablename , make_slice(r.member_1) , make_slice(r.member_2) , make_slice(r.member_3) , make_slice(r.member_4) , make_slice(m_id_1) , make_slice(r.member_5) . . . // same for members 6 => 18 , make_slice(r.member_19) ); return result; }

Listing 10

• Listing 11 shows a FastFormat.Write implementation of the Database Inserter Statement scenario

std::string BusinessAdapter::insertRecord( const BusinessRecord& r ) { std::string result; ff::write( result , "insert[", m_tablename , ";(", make_slice(r.member_1) , ";", make_slice(r.member_2) , ";", make_slice(r.member_3) , ";", make_slice(r.member_4) , ";", make_slice(m_id_1) , ";", make_slice(r.member_5) . . . // same for members 6 => 18 , ";", make_slice(r.member_19) , ")]" ); return result; }

Listing 11

Readers of part 2 [FF2] should recognise aspects of the implementation of the make_slice() inserter function (Listing 12), which provides equivalent semantics to the original makeSlice() but uses shim strings [STLv] , [FF1] to avoid memory allocations, and to reuse the existing FastFormat integer-to-string conversions.

stlsoft::basic_shim_string<char, 20> make_slice(const int val) { if(intNaN == val) { return stlsoft::basic_shim_string<char, 20>("0N"); } else { return fastformat::filters::filter_type(val, &val, static_cast<char const*>(0)); } }

Listing 12

CComBSTR, std::string, std::wstring and CString

This example is heavily edited from a client's codebase. Please don't try to understand the functions' original purposes, just focus on the formulation of the result string. Listing 13 shows the old version of GetFilter() (along with some other things, representative of the original code, that are required just to get the snippet to compile)

#define atFilter 1 #define strFilterSeparator L"-" enum dimension_t; HRESULT XXGetFilterEx_( dimension_t dimension , int newIndex , std::string* fg , std::string* fv ); // Assign to *fg and *fv int offset_to_new_index_( int filter , dimension_t dimension , int index ); // calc index, e.g. 'return index + 1;' HRESULT GetFilter( dimension_t dimension , short index , BSTR* filter ) { std::string szFG; std::string szFV; int newIndex = offset_to_new_index_( atFilter, dimension, index); XXGetFilterEx_(dimension, newIndex, &szFG, &szFV); CString flt(szFG.c_str()); // +1 flt += strFilterSeparator; // +2 flt += szFV.c_str(); // +1 CComBSTR filter(flt); // +1 *filter = filter.Detach(); return S_OK; }

Listing 13

The comments indicate the number of memory allocations, involved in a typical invocation. The type transitions in this case are std::string -> CString (x2), wchar_t const* -> CString, and CString -> CComBSTR. Listing 14 shows the new version of GetFilter() that is implemented in terms of FastFormat.Write, using the sink for CComBSTR.

HRESULT GetFilter(dimension_t dimension, short index, BSTR* filter) { . . . // as before CComBSTR filter; *filter = fastformat::write( filter , szFG , winstl::w2m(strFilterSeparator) , szFV ).Detach(); // +1 return S_OK; }

Listing 14

As you can see, the original five statements have been reduced to two, and the number of memory allocations have been reduced from five to one. Were it not for the need to compress the main statement into the narrow display confines of this magazine, it would be evident that it's also more transparent than the original. There's no performance test for this case - the main aim in the change was to increase transparency and maintainability, and aiding in the project-wide task of removing dependency on the MFC library (incl. CString).

MessageBox

This last example, another extract from a commercial project, is also about improvements to transparency. The original code in this case was far too big to include here, and I really didn't want to have to think up all the obfuscated names. Furthermore, it did not have the same level of functionality, so only the new version is shown (Listing 15). The code comes from a Windows GUI application that needs to process files and, as shown, report to the user if the file cannot be accessed. For localisation purposes, message strings, and windows error strings, are obtained at runtime.

void ProcessFile( HINSTANCE hinst , HWND parent , LPCTSTR fileName ) { WIN32_FIND_DATA fd; HANDLE h = ::FindFirstFile(fileName, &fd); if(INVALID_HANDLE_VALUE == h) { DWORD err = ::GetLastError(); ff::windows_resource_bundle bundle(hinst); ff::ignore_unreferenced_arguments_scope scoper; ff::fmt( ff::sinks::MessageBox(parent, "Problem", MB_ICONWARNING) , bundle[IDS_FMT_MISSING_FILE] , filename , err , winstl::error_desc(err)); } else { // . . . do something useful with file data } }

Listing 15

Assume fileName is "abc.def", and that no such file exists. Further assume that the module designated by hinst has a string resource with the identifier IDS_FMT_MISSING_FILE whose value is

      "The file '{0}' could not be processed: {2}"  

In that case, a message box will be displayed, as a child of the parent window, with the type MB_ICONWARNING (the yellow triangle with an exclamation mark in it), and the message

    "The file 'abc.def' could not be processed: The system cannot find the file specified"  

Obviously, there's a lot going on here: formatting, looking up resources, looking up error code strings. Let's break it down according to the separate FastFormat components at play.

First, an instance of the fastformat::windows_resource_bundle class is declared, taking hinst in its constructor. This class represents a façade over the Windows Resources API functions, providing a simple mapping of id to string, throwing an exception if a given id does not represent a string resource.

Second, the creator function fastformat::sinks::MessageBox() [XSTLv1] constructs an instance of the sink class fastformat::sinks::MessageBox_sink, which will receive the formatted statement results and then invoke the Windows function MessageBox() to display the message. In this case, the two arguments required are the filename ({0}) and a temporary instance of the winstl::error_desc class ({2}), which is used to elicit the string form of an error code via the Windows FormatMessage() function. Note that we remember the error code associated with the failure to open/stat the file before doing any error display processing, since any subsequent Windows API failure would change the thread's last error code, and lead to a potentially misleading cause being presented to the user.

Third, you may have noticed that there are actually three format arguments: filename, err, and winstl::error_desc(err), but the example format string contains just two replacement parameters. By default, all format specification defect conditions result in the throwing of an exception (derived from fastformat::fastformat_exception). The purpose of the scoper instance of the succinctly named fastformat::ignore_unreferenced_arguments_scope class is to suppress this, and allow the string to be formatted despite the mismatch. During its lifetime it suppresses the throwing of a fastformat::unreferenced_argument_exception. (We'll discuss how this works later.)

We do this because we want to be able to use different resource strings without breaking the application. This is usually for localisation purposes, but may also be to give more information in debug builds (such as the numeric value of the error code err). For example, for some locales we might want to change IDS_FMT_MISSING_FILE to:

    "The file '{0}' could not be processed"  

And in debug builds we might want to use the format string:

    "The file '{0}' could not be processed: error code {1}: {2}"  

Note that scoper only suppresses exceptions with unreferenced arguments, however. If the format changed to

    "The file '{0}' could not be processed: please inform {3}"  

then a fastformat::missing_argument_exception would be thrown, and we don't want to squash that. (If we did, we'd declare an instance of fastformat::ignore_missing_arguments_scope, with the effect that {3} would be replaced with the empty string.)

Windows format strings

One last note on format strings on Windows. Windows message files (used via FormatMessage) and MFC (used via AfxFormatString*()) use a different format syntax, where the format string would instead be

      "The file '%1' could not be processed: %3"  

As a convenience, the windows_resource_bundle class is able to use these format strings, and performs a translation if the original contains one or more Windows replacement parameters and zero FastFormat replacement parameters. This allows an easy upgrade from MFC-based resource formatting to FastFormat, which is particularly useful if your application is localised to several locales.

Format specification defect handlers

Let's now consider the format specification defect handler mechanism. As discussed in part 1 [FF1] , format specification defects involve both badly formed format strings, and a failure to match all replacement parameters to all given arguments. We'll consider only the mismatch case; the two aspects follow the same pattern.

Control of mismatch behaviour involves an enumeration, a handler function prototype, a structure, and four API functions, to get+set the handler for thread+process. Thread handlers, if set, take precedence over process ones; in single-threaded builds they are the same.

The relevant aspects of the API are shown in Listing 16. (I apologise for the long names.)

enum ff_replacement_code_t { FF_REPLACEMENTCODE_SUCCESS = 0 , FF_REPLACEMENTCODE_MISSING_ARGUMENT , FF_REPLACEMENTCODE_UNREFERENCED_ARGUMENT }; typedef int (*fastformat_mismatchedHandler_t)( void* param , ff_replacement_code_t code , size_t numParameters , int parameterIndex , ff_string_slice_t* slice , void* reserved0 , size_t reserved1 , void* reserved2 ); struct ff_mismatched_handler_info_t { fastformat_mismatchedHandler_t handler; void* param; }; ff_mismatched_handler_info_t fastformat_getThreadMismatchedHandler(); ff_mismatched_handler_info_t fastformat_setProcessMismatchedHandler( fastformat_mismatchedHandler_t handler , void* param ); . . . // same for process handlers

Listing 16

With this, as shown in Listing 17, we're in a position to see how the fastformat::ignore_unreferenced_arguments_scope class works.

// in namespace fastformat class ignore_unreferenced_arguments_scope : private mismatched_arguments_scope_base { public: typedef ignore_unreferenced_arguments_scope class_type; typedef mismatched_arguments_scope_base parent_class_type; public: ignore_unreferenced_arguments_scope() : parent_class_type(class_type::handler, get_this_()) {} private: void* get_this_() throw() { return this; } static int handler( void* param , ff_replacement_code_t code , size_t numParameters , int parameterIndex , ff_string_slice_t* slice , void* reserved0 , size_t reserved1 , void* reserved2 ) { class_type* pThis = static_cast<class_type*>(param); if(FF_REPLACEMENTCODE_UNREFERENCED_ARGUMENT == code) { return +1; // Ignore unreferenced argument } else { return pThis->parent_class_type:: handle_default(param, code, numParameters, parameterIndex, slice, reserved0, reserved1, reserved2); } } }

Listing 17

The class is derived (privately) from the class fastformat::mismatched_arguments_scope_base, which handles (de-)registration of a derived class instance and its handler method for the duration of its lifetime, via the class fastformat_setThreadMismatchedHandler(). (The use of get_this_() is an old trick for avoiding compiler warnings about use of a partially constructed instance in its own member initialiser list; see [IC++] .)

The meat of this component is in the derived class's handler() method, which intercepts an unreferenced argument code, and instructs the FastFormat replacement engine to ignore it. All other codes are passed, via the parent class's handle_default(), to the previous handler, if any, in the chain. The consequence of this is that, for the lifetime of scoper, any unreferenced arguments will not cause an fastformat::unreferenced_argument_exception to be thrown.

You're not limited to the scoping classes provided with the distribution. FastFormat allows you to customise its behaviour in light of ill-formed format strings and/or of mismatched arguments, on a per-process and/or per-thread basis, to do whatever funky things your heart desires.

Choosing output sinks

A last note on output sinks. If you're determined to squeeze out every last cycle, you might choose to use stdout as your sink rather than std::cout, as in:

      #include <fastformat/sinks/FILE.hpp>  
      //#include <fastformat/sinks/ostream.hpp>  
 
      FILE* stm = stdout;  
 
      // FastFormat.Format  
      ff::fmtln(  
        stm  
      , "[{0}{1}]"  
      , ff::to_f(12345.12345, -14, 3)  
      , ff::to_f(12345.12345, -14, 3)  
      );  

The inconvenience with this is that you'll likely have to declare a sink variable, as shown in the example, because stdout is often not an instance at all, but rather some #define into part of an implementation-defined structure, such as (&_iob[1]) for Visual C++, or (&_streams[1]) with Borland. The same is needed for stderr.

Note that the gains, if any, are strongly platform-dependent. On Linux it gives between 5% and 10% improvement, on Mac OS-X between 2% and 6%. On Windows, with VC++ 9 it actually slows down by a small margn. Obviously, the advice is to get it working (with std::cout) and then optimise (with stdout) if you really need to. That's only likely to be if you're writing to a file - using an arbitrary FILE* handle rather than an arbitrary fstream instance - since console/terminal output is far too much affected by other factors for such low-percentage performance improvements to be significant.

FastFormat for logging?

Several people have enquired about the use of FastFormat for application logging. As we've seen in the Server Connection Log example, for some kinds of logging it's a good solution. For what I call application logging, however - the presence of statements throughout the code that allows an interested observer to follow what is happening now, and what has already happened, at a high level of granularity - the use of FastFormat.Format, or any other replacement-based API [FF1] , is not advisable. The same goes for any library that is less than 100% type-safe. The reason is that the programmer should be able to have full confidence that an application logging statement will be processed and emit output. This is because many uses of log statements are in places in the code are impossible, or exceedingly difficult, to test, and usually these statements are the ones you most need to be able to rely on.

I've mentioned my other, older, logging API library, Pantheios, a couple of times in this article series. I hope to write an article about that at some time in the future, and will go into more detail about the how/why/when/what of logging. For the moment, however, I'll show you a sneaky trick that allows you to use FastFormat.Write (or FastFormat.Format, if you must) with Pantheios.

The Pantheios application layer contains severity level pseudo-constant symbols that are actually stateless global instances of specialisations of a severity level class template:

      namespace Pantheios  
      {  
        namespace  
        {  
          static level<PANTHEIOS_SEV_DEBUG> debug;  
          . . . // and so on  
          static level<SEV_ALERT> alert;  
          static level<SEV_EMERGENCY> emergency;  
        }  
      }  

These are not declared const, even though no-one should be attempting any mutations of them, precisely so they can be used as 'sinks' to FastFormat, by defining the following overload of fastformat::sinks::fmt_slices [FF2] :

// in namespace fastformat::sinks template <int L> pantheios::level<L>& fmt_slices( pantheios::level<L>& sink , int /* flags */ , size_t /* cchTotal */ , size_t numResults , ff_string_slice_t const* results) { STLSOFT_STATIC_ASSERT(sizeof( pantheios::pan_slice_t) == sizeof(fastformat::ff_string_slice_t)); STLSOFT_STATIC_ASSERT(offsetof( pantheios::pan_slice_t, len) == offsetof(fastformat::ff_string_slice_t, len)); pantheios::pantheios_log_n( sink , numResults , stlsoft::sap_cast<pantheios::pan_slice_t const*>(results) ); return sink; }

Listing 18

This 'works' because the definitions of ff_string_slice_t and pan_slice_t - the thing ff_string_slice_t was copied from in the first place - are identical, and therefore binary compatible. The two static asserts [IC++] are there to ensure that any changes that invalidate that assumption are not missed. Then it's as simple as casting from one array of string alices to the other, and passing to the core Pantheios logging function.

This allows code such as the following:

      catch(std::exception& x)  
      {  
        ff::write(pan::warning,  
           "Something bad has happened: ", x);  
      }  

and

      HRESULT GetFilter(dimension_t dimension,  
         short index, BSTR* filter)  
      {  
        // NOTE: can pass 'dimension' variable   
        // directly if string access shims are  
        // defined for the dimension_t enum  
        ff::write(pan::debug, "GetFilter(dimension=",  
           dimension, "; index=", index, ", ...)");  

As I said at the start of this section, there are several reasons to prefer using a proper logging solution, such as Pantheios, but this technique will get you a fair way along to a good solution.

Summary

This article completes the introduction to FastFormat, a C++ library that applies advanced generic conversion techniques to provide robust, flexible and efficient formatting, providing answers to the deficiencies of the current standard and widely used third-party libraries. It is in ongoing development, and readers are invited to use, criticise and contribute, as they see fit.

References

[FF1] 'An Introduction to FastFormat, part 1: The State of the Art', Matthew Wilson, Overload #89, February 2009; http://accu.org/index.php/journals/c249/

[FF2] 'An Introduction to FastFormat, part 2: Custom Argument and Sink Types', Matthew Wilson, Overload #90, April 2009; http://accu.org/index.php/journals/c251/

[IC++] Imperfect C++, Matthew Wilson, Addison-Wesley 2004; http://www.imperfectcplusplus.com/

[PAN] The Pantheios Logging API Library, http://www.pantheios.org/; to see why it's the best choice in C++ logging APIs, check out http://www.pantheios.org/performance.html#sweet-spot, which shows graphically how Pantheios can be up to two-orders of magnitude faster than the rest.

[PragProg] The Pragmatic Programmer, Dave Thomas and Andy Hunt, Addison-Wesley, 2000; http://www.pragmaticbookshelf.com/

[SO] http://www.stackoverflow.com/questions/586410/

[XSTLv1] Extended STL, volume 1, Matthew Wilson, Addison-Wesley 2007; http://www.extendedstl.com/

Notes: 

More fields may be available via dynamicdata ..