Title: Enlarging on "A Problem of Access" in C Vu December 2001 Volume 13 Number 6

Author: 

Date: 03 October 2002 13:15:54 +01:00 or Thu, 03 October 2002 13:15:54 +01:00

Summary: 

Body: 

We are developing a component based system where one central component is the server.

Among other things, the server holds some very large containers of objects (thus these containers are preferably constructed only once - at the time the server component is coming up.) A number of client components can refer to the container object simultaneously. They may inspect it (shared mode - container and its objects are read only) when there are no exclusive (mutating) mode clients, but if any client wants to change any such containers or objects within, it has to have an exclusive lock on the container first (exclusive or mutating mode ) and there must be no pre-existing exclusive or shared mode clients. This way, there is no (meaning "minimum"!) space for surprises.

One such container is Dimension, which holds a great many members (millions). The following abstract class is exposed to outside clients (other components):

// Interface class 
class CDimension_I { 
  public: 
    virtual Id GetId() = 0; 
    virtual void setId(Id id) = 0; 
    // other interface methods 
};
// This class holds millions of members organized in a tree structure 
class CBaseDimension : public CDimension_I, public Resource_m { 
  public: 
    Id GetId(); 
    void setId(Id id); 
    //other helpers and concrete implementations 
}; 

Client contexts are stored in objects of class Session.

CBaseDimension& cbd = ...; 
Session* pSession = ...; 
// ... 
ResLock_m* pResLock = new ResLock_m( 
    dynamic_cast<Resource_m*>(&cbd),
    pSession,
    LT_SHARED); //locks in shared mode 
// inspect the container, traverse the data structures it holds etc. 
delete pResLock; // unlock the above locked resource 

A number of issues rear their ugly heads:

Okay, let us take them in turn. First some definitions:

enum LOCK_TYPE { 
  LT_NOLOCK, 
  LT_EXCLUSIVE, 
  LT_SHARED 
}; 
template<LOCK_TYPE lckType> 
struct LockTraits {}; 

template<> struct 
LockTraits<LT_SHARED> {}; 

template<> struct 
LockTraits<LT_EXCLUSIVE> {}; 

This is similar to the Int2Type template explained in "Modern C++ Design" by Andrei Alexandrescu. We essentially convert an enum into a type. Converting enums to types helps the C++ compiler track bugs for you. This is immensely helpful as you shall soon see.

Next, separate all the methods of the shared class into const and non- const methods.

Whenever, a client code asks about SHARED access, give a const reference to the shared object.

Only when it requests an EXCLUSIVE access, then only give out non- const reference, essentially allowing client code to call any method.

A singleton object of type CGetResource grants these accesses.

class CDimension_I { 
  public: 
    virtual Id GetId() const = 0; // note the const 
    virtual void setId(Id dimId); // note the absence of const 
    // other interface methods 
}; 

const CDimension_I* inspectDim = ...; 
CDimension_I* mutateDim = ...; 

inspectDim->GetId();  // ok 
inspectDim->setId(3); // compilation error, can't call a non-const method on a const object 
mutateDim->GetId();   // ok, a non-const can call a const method 
mutateDim->setId(3);  // ok 

For more in-depth treatment of this pattern, please see a very neat article by Kevlin Henney in CUJ - "C++ Experts Forum" online article "From Mechanism to Method: Further Qualifications".

template<class DimT, class LckType> 
class CdimWrapper { 
  protected: 
    DimT* m_pDim; 
    CSessionBase* m_pSession; 
  public: 
    CDimWrapper(DimT* pDim, 
                CSessionBase* pSession, 
                LockTraits<LT_SHARED>) 
      : m_pDim(pDim), 
        m_pSession(pSession) { 
    // Pass the buck onto session class. 
    // This takes care of multiple locking 
    m_pSession->Lock( 
      dynamic_cast<Resource_m*>(pDim), 
      LockTraits<LT_SHARED>()); 
    } 
}; 

class CBaseDimShr 
  : public CDimWrapper<const CBaseDimension, LockTraits<LT_SHARED> > { 
  public: 
    CBaseDimShr(const CBaseDimension* pDim, 
                CSessionBase* pSession, 
                CMapShr* pMap, 
                LockTraits<LT_SHARED>) 
      : CDimWrapper<const CBaseDimension, LockTraits<LT_SHARED> >( 
          pDim, pSession, pMap, LockTraits<LT_SHARED>()) {
    } 
    const CBaseDimension* operator->() const { 
      return m_pDim; 
    } 
}; 

class PtrBaseDimShr { 
  const CBaseDimShr* m_pLwDim; // note the const 
  // private copy ctor and assignment op not shown 
  public: 
    PtrBaseDimShr(const CBaseDimShr* pLwDim) 
      : m_pLwDim(pLwDim) { 
    }
    bool operator!() { 
      return m_pLwDim == 0; 
    }
    operator const void*() const { 
      return m_pLwDim; 
    }
    const CBaseDimShr& operator->() const { 
      return *m_pLwDim; 
    }
    ~PtrBaseDimShr() { 
      delete m_pLwDim; 
    } 
};

class CBaseDimEx 
  : public CDimWrapper<CBaseDimension, LockTraits<LT_EXCLUSIVE> > { 
  public: 
    CBaseDimEx(CBaseDimension* pDim, 
               CSessionBase* pSession, 
               CMapEx* pMap, 
               LockTraits<LT_EXCLUSIVE>) 
      : CDimWrapper<CBaseDimension, LockTraits<LT_EXCLUSIVE> >( 
          pDim, pSession, pMap, LockTraits<LT_EXCLUSIVE>()) { 
    }
    CBaseDimension* operator->() { 
      return m_pDim; 
    } 
}; 

class PtrBaseDimEx { 
  CBaseDimEx* m_pLwDim; 
  // private copy ctor and assignment op not shown 
  public: 
    PtrBaseDimEx(CBaseDimEx* pLwDim) 
      : m_pLwDim( pLwDim) { 
    }
    bool operator!() { 
      return m_pLwDim == 0; 
    }
    operator void*() { 
     return m_pLwDim; 
    }
    CBaseDimEx& operator->() { 
      return *m_pLwDim; 
    }
    ~PtrBaseDimEx() { 
      delete m_pLwDim; 
    } 
}; 

class CGetResource { 
  // This class takes care of point 3 
  // (allow multiple SHARED locks or a single EXCLUSIVE) 
  const CBaseDimShr* GetBaseDim(LockTraits<LT_SHARED>, 
                                BaseDimHandle hDimHdl, 
                                CSessionBase *session_ptr); 
  CBaseDimEx* GetBaseDim(LockTraits<LT_EXCLUSIVE>, 
                         BaseDimHandle hDimHdl, 
                         CSessionBase *session_ptr); 
  ......... 
}; 

Now a client acquires the shared object as follows:

void ClientClass::getBaseDimName(CSessionBase *pSession, 
                                 CGetResource* pGetRes, 
                                 BaseDimHandle hDim ) { 
  // original code 
  // CBaseDimension* pDim = ..... 
  PtrBaseDimShr pDim = pGetRes->GetBaseDim( LockTraits<LT_SHARED>(), hDim, pSession); 
} 

When the above function exits (either with a return statement or by throwing an exception), this scheme guarantees that all locks are released. Also, all the locking mechanism is neatly hidden. The programmer just has to know

1 He is trying to get a shared resource ( LockTraits<> ) and

2 The call might sleep

Now with all this in place, suppose somebody does the following:

PtrBaseDimShr pDim = pGetRes->GetBaseDim( LockTraits<LT_EXCLUSIVE>(), hDim, pSession); 

As LockTraits<LT_EXCLUSIVE> is a class (an empty class), there is no function that satisfies the above line. The compiler will kick in an error.

Now, the programmer corrects the mistake and obtains a shared pointer.

Id id = 4; 
pDim->setId(id); // this again will kick in a compiler error. 

Let us trace this call to see what is going on. Trying to resolve the above call, as pDim is not a plain pointer, the compiler applies the operator -> found in class PtrBaseDimShr, which returns a const object of "CBaseDimShr". Reapplying the operator -> again, which returns a pointer to const object of type CBaseDimension*. So in essence, the above setId() call is invoked on a const object of type CBaseDimension. However, as this is a non-const method, the call fails to compile.

Similarly, tracing through an exclusive mode pointer, we can indeed see that the above call goes through fine.

This design was grafted onto a legacy code, so we did many other nice things so it all worked quite well. The only other thing that needed to be changed was the declaration of the pointers. (PtrBaseDimShr instead of CBaseDimension* etc.). As the code was quite huge, nearly 20000 lines and still growing, we never needed to know the logic so as to know where to place the lock calls etc. We just mechanically replaced the above declarations (using nifty VIM macros) and the work got reduced to just few keystrokes. The unit testing of the locking scheme was done using a Perl script (which is an altogether different story - will do it some other time).

One queer thing I notice again and again. After all the core design was coded in place, it was just a mechanical/routine matter to extend it. The system just developed and extended itself. I remember a similar experience while developing a schema language using Lex, Bison and C++. This way of things is indeed very gratifying. We did complete our part well before 15 days of the release deadline, leaving everybody happy and some more spare time for myself to linux it away.

Notes: 

More fields may be available via dynamicdata ..