Title: Patterns in C - Part 5: REACTOR

This final part of the series will step outside the domain of standard C and investigate a pattern for event-driven applications. The REACTOR pattern decouples different responsibilities and allows applications to demultiplex and dispatch events from potentially many clients.

In order to simplify maintenance of large systems, the diagnostics of the individual subsystems are gathered in one, central unit. Each subsystem connects to the diagnostics server using TCP/IP. As TCP/IP is a connection-oriented protocol, the clients (the different subsystems) have to request a connection at the server. Once a connection is established, a client may send diagnostics messages at any time.

The brute-force approach is to scan for connection requests and diagnostics messages from the clients one by one as illustrated in the activity diagram in Figure 1.

Figure 1. Eternal loop to scan for different events

Even in this strongly simplified example, there are several potential problems. By intertwining the application logic with the networking code and the code for event dispatching, several unrelated responsibilities have been built into one module. Such a design is likely to lead to serious maintenance, testability, and scalability problems by violating a fundamental design principle.

The single responsibility principle states that "a class should have only one reason to change" [Martin]. The goal of this principle is close to that of the open-closed principle treated below: both strive to protect existing code from modifications. When violating the single responsibility principle, a module gets more reasons to change and modifications to it become more likely. Worse, the different responsibilities absorbed by a single module may become coupled and interact with each other making modifications and testing of the module more complicated.

The single responsibility principle is basically about cohesion. It is useful and valuable on many levels of abstraction, not at least in a procedural context; simply replacing the word "class" with "function" enables us to analyze algorithms like the one above with respect to this principle.

By violating the single responsibility principle, the module in the example above will be hard to maintain; it is code that one never wants to dig into in the future. Unfortunately, on collision course with that wish is the fact that the event loop above violates the open-closed principle [Martin]; new functionality cannot be added without modifying existing code. Related to our example, a diagnostics server typically allows a technician to connect and query stored information. Introducing that functionality would double the logic in the event loop. Clearly, this design makes the code expensive to modify.

To make things worse, the solution above fails to scale in terms of performance as well. As all events are scanned serially, even in case timeouts are used, valuable time is wasted doing nothing.

The potential performance problem above may be derived from the failure of taking the concurrent nature of the problem into account. One approach to address this problem is by introducing multiple threads. The diagnostics server keeps its event loop, but the loop is now reduced to scan for connection requests. As soon as a connection is requested, a new thread is allocated exclusively for handling messages on that new connection.

The multithreading approach fails to address the overall design issue as it still violates both the single responsibility principle and the open-closed principle. Although the code for scanning and dispatching diagnostics messages is moved out of the event loop, adding a new server-port still requires modifications to existing code.

From a design perspective threads didn't improve anything. In fact, even with respect to performance, this solution may due to context switches and synchronization actually perform worse than the initial single-threaded approach.

The sheer complexity inherent in designing and implementing multithreaded applications is a further argument for discarding this solution.

Summarizing the experience so far, the design fails as it assumes three different responsibilities. This problem is bound to be worse as the design violates the open-closed principle, making modifications to existing code more likely.

Summarizing the ideal solution, it should scale well, encapsulate and decouple the different responsibilities, and be able to serve multiple clients simultaneously without introducing the liabilities of multithreading. The REACTOR pattern realizes this solution by encapsulating each service of the application logic in event handlers and separating out the code for event demultiplexing.

The intent of the REACTOR pattern is: "The REACTOR architectural pattern allows event-driven applications to demultiplex and dispatch service requests that are delivered to an application from one or more clients [POSA]."

Figure 2. Structure of the REACTOR Pattern

The roles of the involved participants are:

In its description of REACTOR, Pattern-Oriented Software Architecture [POSA] defines a further collaborator, the Synchronous Event Demultiplexer. The Synchronous Event Demultiplexer is called by the Reactor in order to wait for events to occur on the registered Handles.

A synchronous event demultiplexer is often provided by the operating system. This example will use poll() (select() and Win32's WaitForMultipleObjects() are other functions available on common operating systems), which works with any descriptor.

The code interacting with poll() will only be provided as a sketch, because the POSIX specific details are outside the scope of this article. The complete sample code, used in this article, is available from my homepage [source].

The collaboration between an EventHandler and the Reactor is similar to the interaction between an observer and its subject in the design pattern OBSERVER [GoF]. This relationship between these two patterns indicates that the techniques used to realize OBSERVER in C [Petersen2] may serve equally well to implement the REACTOR.

In order to decouple the Reactor from its event handlers and still enable the Reactor to notify them, each concrete event handler must correspond to a unique instance. In our OBSERVER implementation, the FIRST-CLASS ADT pattern [Petersen1] was put to work to solve this problem. As all concrete event handlers have to be abstracted as one, general type realizing the EventHandler interface, void* is chosen as "the general type" to be registered at the Reactor (please refer to the previous part in this series - Reference [Petersen2] - for the rationale and technical reasons behind the void* abstraction). These decisions enable a common interface for all event handlers:

/* The type of a handle is system specific - this example uses UNIX I/O handles, which are plain integer values. */
typedef int Handle;

/* All interaction from Reactor to an event handler goes through function pointers with the following signatures: */
typedef Handle (*getHandleFunc)
   (void* instance);
typedef void (*handleEventFunc)
   (void* instance);

typedef struct
{
  void* instance;
  getHandleFunc getHandle;
  handleEventFunc handleEvent;
} EventHandler;

#include "EventHandler.h"

void Register(EventHandler* handler);
void Unregister(EventHandler* handler);

#include "EventHandler.h"

struct DiagnosticsServer
{
   Handle listeningSocket;
   EventHandler eventHandler;
  
   /* Other attributes here... */
};
/* Implementation of the EventHandler interface. */
static Handle getServerSocket(void* instance)
{
  const DiagnosticsServerPtr server = 
     instance;
  return server->listeningSocket;
}

static void handleConnectRequest(void* instance)
{
  DiagnosticsServerPtr server = instance;
  
  /* The server gets notified as a new connection request arrives. 
     Add code for accepting the new connection and creating a client here... */
}

DiagnosticsServerPtr createServer
   (unsigned int tcpPort)
{
  DiagnosticsServerPtr newServer = 
     malloc(sizeof *newServer);

  if(NULL != newServer) {
  /* Code for creating the server socket here.
     The real code should look for a failure,
     etc. */
      newServer->listeningSocket = 
         createServerSocket(tcpPort);

      /* Successfully created -> register the
         listening socket. */
       newServer->eventHandler.instance = 
          newServer;
       newServer->eventHandler.getHandle =
          getServerSocket;
       newServer->eventHandler.handleEvent =
          handleConnectRequest;

      Register(&newServer->eventHandler);
   }
   return newServer;
}

void destroyServer
   (DiagnosticsServerPtr server)
{
   /* Before deleting the server we have to
      unregister at the Reactor. */
   Unregister(&server->eventHandler);

   free(server);
}

When implementing the concrete event handlers as a FIRST-CLASS ADT, the functions for creating and destructing the ADT serves well to encapsulate the registration handling. The advantage is the combination of loose dependencies with information hiding as a client does not even have to know about the usage and interactions with the Reactor.

Another attractive property is that the internals of the server, in our example the handle, is encapsulated within the getServerSocket function. Sure, we are giving the Reactor a way to fetch it, but the Reactor is considered a well-trusted collaborator and we are actively giving it access by registering our event handler. There is no way for any other module to mistakenly fetch the handle and corrupt the associated resource.

The details of the REACTOR implementation are platform specific as they depend upon the available synchronous event demultiplexers. In case the operating system provides more than one synchronous event demultiplexer (e.g. select() and poll()), a concrete Reactor may be implemented for each one of them and the linker used to chose either one of them depending on the problem at hand. This technique is referred to as link-time polymorphism.

Each Reactor implementation has to decide upon the number of reactors required by the surrounding application. In the most common case, the application can be structured around one, single Reactor. In this case, the interface in Listing 2 (Reactor.h) will serve well. An application requiring more than one Reactor should consider making the Reactor itself a FIRST-CLASS ADT. This second variation complicates the clients slightly as references to the Reactor ADT have to be maintained and passed around in the system.

Independent of the system specific demultiplexing mechanism, a Reactor has to maintain a collection of registered, concrete event handlers. In its simplest form, this collection may simply be an array. This approach serves well in case the maximum number of clients is known in advance.

#include "Reactor.h"
#include <poll.h>
/* Other include files omitted... */

/* Bind an event handler to the struct used to
   interface poll(). */
typedef struct
{
   EventHandler handler;
   struct pollfd fd;
} HandlerRegistration;

static HandlerRegistration registeredHandlers[MAX_NO_OF_HANDLES];

/* Add a copy of the given handler to the first free position in registeredHandlers. */
static void addToRegistry(EventHandler* handler);
  /* Identify the event handler in the registeredHandlers and remove it. */
static void removeFromRegistry(EventHandler* 
   handler);

/* Implementation of the Reactor interface used for registrations.*/
void Register(EventHandler* handler)
{
   assert(NULL != handler);
   addToRegistry(handler);
}

void Unregister(EventHandler* handler)
{
   assert(NULL != handler);
   removeFromRegistry(handler);
}

The reactive event loop is the core of the Reactor and its responsibilities are to control the demultiplexing and dispatch the detected events to the registered, concrete event handlers. The event loop is contained within the HandleEvents() function and is typically invoked from the main() function.

int main(void){
   const unsigned int serverPort = 0xC001;
   DiagnosticsServerPtr server = 
      createServer(serverPort);
   if(NULL == server) {
      error("Failed to create the server");
   }
   /* Enter the eternal reactive event loop.*/
   for(;;){
      HandleEvents();
   }
}

void HandleEvents(void);

With the interface in place, we can move on and implement the event loop itself. The listing below extends Listing 4.

#include "ReactorEventLoop.h"
/* The code from Listing 4 go here (omitted)... */

/* Add a copy of all registered handlers to the given array. */
static size_t buildPollArray
   (struct pollfd* fds);

  /* Identify the event handler corresponding to the given 
     descriptor in the registeredHandlers. */
static EventHandler* findHandler(int fd);

static void dispatchSignalledHandles(
      const struct pollfd* fds,
      size_t noOfHandles)
{
   /* Loop through all handles. Upon detection of a handle signalled by poll, 
      its corresponding event handler is fetched and invoked. */
   size_t i = 0;
   for(i = 0; i < noOfHandles; ++i) {
      /* Detect all signalled handles and invoke their corresponding 
         event handlers. */
      if((POLLRDNORM | POLLERR) & 
         fds[i].revents) {
         EventHandler* signalledHandler =
         findHandler(fds[i].fd);

         if(NULL != signalledHandler){
            signalledHandler-> handleEvent
            (signalledHandler->instance);
         }
      }
   }
}
/*Implementation of the reactive event loop.*/
void HandleEvents(void)
{
   /* Build the array required to interact with poll().*/
   struct pollfd fds[MAX_NO_OF_HANDLES] = {0};
   const size_t noOfHandles = 
      buildPollArray(fds);
/*Invoke the synchronous event demultiplexer*/
   if(0 < poll(fds, noOfHandles, INFTIM)){
   /* Identify all signalled handles and
   invoke the event handler associated with 
   each one. */
      dispatchSignalledHandles(fds,
          noOfHandles);
   }
   else{
      error("Poll failure");
   }
}

In my previous article [Petersen2], I discussed strategies for managing changed registrations during the notification of observers. The alternative of forbidding changed registrations is, unlike the OBSERVER pattern, not an option for a REACTOR implementation. In the example used in this article, the server reacts to the notification by creating a new client, which must be registered shall it ever be activated again. This leaves only one option for a REACTOR implementer: ensure that it works.

One solution is to maintain a separate array to interact with the synchronous event demultiplexer as illustrated above. This array is never modified in the event loop. However, this solution has the consequence that handles unregistered during the current event loop may be marked as signalled in the separate array. The code simply has to check for this case and ignore such handles, as illustrated by the function dispatchSignalledHandles in Listing 7 above.

The code uses the handle alone as identification. In cases resources are disposed and created during the same event loop, there is, depending on platform, a possibility that the handle ID's are re-used; a signalled handle in the copy may belong to an unregistered event handler, but due to a new registration using the re-cycled handle ID, the new event handler may be erroneously invoked. If this is an issue, the problem may be prevented by introducing further book-keeping data. For example, a second array containing the identities of the handles unregistered during the current event loop makes it possible to identify the case described above and thus avoid it.

The design in the example above does only allow applications to register for one type of event (read-events). The event type is even hardcoded in the Reactor and it is a simple solution sufficient for applications without any need for further event detection. The REACTOR pattern, however, is not limited to one type of event. The pattern scales well to support different types of events.

Pattern-Oriented Software Architecture [POSA] describes two general strategies for dispatching event notifications:

Although the mechanisms used to implement them are related, there are differences between these two patterns. The main difference is in the notification mechanism. As a Subject changes its state in an OBSERVER implementation, all its dependents (observers) are notified. In a REACTOR implementation, this relationship is one to one - a detected event leads the Reactor to notify exactly one dependent (EventHandler).

One typical liability of the OBSERVER pattern is that the cohesion of the subject is lowered; besides serving its central purpose, a subject also takes on the responsibility of managing and notifying observers. With this respect, a Reactor differs significantly as its whole raison d'être is to dispatch events to its registered handlers.

The main consequences of applying the REACTOR pattern are:

The REACTOR pattern simplifies event-driven applications by decoupling the different responsibilities, encapsulated in separate modules.

There is much more to the REACTOR pattern than described in this article. Particularly several variations that all come with different benefits and trade-offs. For an excellent in-depth treatment of the REACTOR and other patterns in the domain, I recommend the book Pattern-Oriented Software Architecture, volume 2 [POSA].

Many thanks to Drago Krznaric and André Saitzkoff for their feedback.