Title: Working with GNU Export Maps

Author: 

Date: 04 June 2007 11:56:00 +01:00 or Mon, 04 June 2007 11:56:00 +01:00

Summary: Taking control over the symbols exported from shared libraries built with the GNU toolchain.

Body: 

Recently I've been preparing version 2 of an internal shared library for my company and I needed to make sure that both the old version of the library and the new one could be loaded into a process at the same time, without causing problems. I knew from previous work that the answer was probably GNU export maps, but my memory was distinctly rusty on the details. Having been to the discussion on writing for the ACCU at the conference, it occurred to me that other people might also find some notes on the topic useful.

Exporting C++ from a shared library

When exporting symbols from a shared library, the GNU ELF shared library linker behaves in a significantly different way to the Microsoft Windows linker. On Windows, nothing is exported from a DLL unless the programmer explicitly requests it. The GNU ELF linker, on the other hand, exports everything by default.

The GNU ELF default undoubtedly makes initial C++ application development simpler; hands up everyone who has at some point struggled to export a class from a DLL, because it uses an STL container... There's a cost to that initial simplicity though. A C++ shared library will typically contain a large number of symbols. When an application is linked against that library, the compiler and linker generate a reference for each of those symbols. When the library is loaded at run time, each of those references has to be bound to the corresponding symbol in the shared library.

Let's take a look at a trivial example (Listing 1).

// spaceship.h
#include <string>
#include <vector>

namespace scifi
{
class Spaceship
{
public:
  Spaceship( std::string const& name );
  ~Spaceship();
  void stabliseIonFluxers();
  void initiateHyperwarp();
private:
  Spaceship( Spaceship const& );
  Spaceship& operator=( Spaceship const& );
private:
  typedef unsigned int            FluxLevel;
  typedef std::vector<FluxLevel>  FluxLevels;
private:
  void doSomethingInternal();
  FluxLevel checkFluxLevel( size_t ionFluxerIdx );
private:
  std::string m_name;
  FluxLevels  m_fluxLevels;
};
}

// spaceship.cpp omitted for brevity    

// testflight.cpp
#include "spaceship.h"
int main( int, char** )
{
  scifi::Spaceship*  ship = new scifi::Spaceship(
      "Beagle" );
  ship->stabiliseIonFluxers();
  ship->initiateHyperwarp();
  delete ship;
  return 0;
}

If we build spaceship.cpp into a shared library and testflight.cpp into an executable linked against that library, we can examine what happens at runtime, using the LD_DEBUG environment variable.

> g++ -shared -fPIC spaceship.cpp -o 
libspaceship.so.1 -Wl,-soname=libspaceship.so.1
> ln -s libspaceship.so.1 libspaceship.so
> g++ testflight.cpp -L. -lspaceship -o testflight
> export LD_DEBUG=symbols
> export LD_LIBRARY_PATH=.
> ./testflight   

This produces a lot of output. Digging through it, we see some things that we are expecting to be resolved to our library, like those shown in Figure 1.

5975: symbol=_ZN9SpaceshipC1ERKSs;  lookup in file=./testflight
5975: symbol=_ZN9SpaceshipC1ERKSs;  lookup in file=./libspaceship.so.1
5975: symbol=_ZN9Spaceship19stabiliseIonFluxersEv;  lookup in file=./testflight
5975: symbol=_ZN9Spaceship19stabiliseIonFluxersEv;  lookup in file=./libspaceship.so.1

But we also see a lot more that we might not have expected, like those in Figure 2.

5975: symbol=_ZNSt6vectorIjSaIjEEC1IiEET_S3_RKS0_;  lookup in file=./testflight
5975: symbol=_ZNSt6vectorIjSaIjEEC1IiEET_S3_RKS0_;  lookup in file=./libspaceship.so.1
5975: symbol=_ZNSt18_Vector_alloc_baseIjSaIjELb1EE11_M_allocateEj;  lookup in file=./testflight
5975: symbol=_ZNSt18_Vector_alloc_baseIjSaIjELb1EE11_M_allocateEj;  lookup in file=./libspaceship.so.1

In total, there are twenty one symbols that get resolved to our library. That's quite a lot of fix-ups for such a small amount of code. It's worse than it immediately looks, as well, because each lookup is done by doing a string compare against each possible function in each library, until a match is found, so the number of symbols exported from our library affects not just how many symbols have to be fixed up by the executable, but how many string matches have to be done for each fix-up. Imagine how that scales up for a real C++ library.

Notice how even though we didn't deliberately export them from our library, there are quite a lot of symbols from STL instantiations being looked up there; in fact they swamp the symbols we actually intended to export.

We can use nm to look at what's being exported from our library:

> nm -g -D -C --defined-only libspaceship.so.1    

As with the output generated by LD_DEBUG, we see some symbols that relate directly to our class, as shown in Figure 3, and we also see many other symbols relating to the STL classes we used in the implementation, like those shown in Figure 4.

0000132a T scifi::Spaceship::checkFluxLevel(unsigned int)
0000131e T scifi::Spaceship::initiateHyperwarp()
00001324 T scifi::Spaceship::doSomethingInternal()
00001318 T scifi::Spaceship::stabiliseIonFluxers()
00001134 T scifi::Spaceship::Spaceship(std::string const&)
00001270 T scifi::Spaceship::~Spaceship()

00001330 W std::allocator<unsigned int>::allocator()
00001336 W std::allocator<unsigned int>::~allocator()
0000154a W std::_Vector_base<unsigned int, std::allocator<unsigned int> 
>::_Vector_base(std::allocator<unsigned int> const&)
0000147c W std::_Vector_base<unsigned int, std::allocator<unsigned int> 
>::~_Vector_base() 000014e6 W std::__simple_alloc<unsigned int, 
std::__default_alloc_template<true, 0> >::deallocate(unsigned int*, 
unsigned int)

In amongst the noise of the weakly defined STL template instantiations being exported from our library, notice how our private member functions are also exported.

Remember that we did not build with debug information. Also don't be fooled into thinking that it's because the header file listed them; remember that there's nothing special about header files in C++; the compiler and linker don't even know they exist, so even if we use idioms like Cheshire Cat or abstract base classes, all that implementation detail will still be exported and available for perusal by anyone who cares to run freely available tools like nm over the shared library.

For some projects, this could represent an unacceptable IP leakage.

If those problems don't concern you, there is another issue you might like to consider.

Let's imagine that we have successfully deployed version 1 of our spaceship library. It's being used in a few places and is perhaps referenced by a few other shared libraries. Consider what happens if we now want to do a version 2, which isn't compatible. Obviously, we'll build it into libspaceship.so.2, with the SONAME set appropriately, so we're versioned and everything is OK, right?

Not quite. When the dynamic linker is resolving symbols, it simply searches the list of modules, in order. There is no information in the symbol to say which library it ought to be loaded from. So let's imagine that one part of our application is linked against libspaceship.so.2, but another part hasn't been updated yet and still links against libspaceship.so.1. If libspaceship.so.1 gets loaded first, then whenever the Spaceship constructor is searched for, the one in libspaceship.so.1 will always be found. If we then try to use a facility that we added to libspaceship.so.2, disaster will ensue.

Fortunately, there is a mechanism which can solve both problems. What we need to do is take control over which symbols are exported from our library. The GNU tool-chain offers a few ways of doing this. One involves decorating the code with __attribute__ ((visibility("xxx")) tags; another, introduced with GCC 4.0, uses #pragma GCC visibility, but I'm going to focus on GNU Export Maps, sometimes called Version Scripts. This is partly because I don't like adding large amounts of tool-chain specific decoration to my code and partly because, at present, as far as I know, only export maps can help with versioning symbols.

An export map is simply a text file listing which symbols should be exported and which should not. A really simple example to export one 'C' function called foo from a shared library would look like this:

{
  global:
    foo;
  local:
    *;
};
    

Unfortunately, the situation for C++ is, inevitably, slightly more complex... you've guessed, of course: name mangling! Export maps are used by the linker, by which time the compiler has mangled the names. The good news is that the GNU linker understands the GNU compiler's C++ name mangling, we just have to tell it that the symbols are C++.

So for our spaceship, we might write:

{
  global:
    extern "C++" {
      *scifi::Spaceship;
      scifi::Spaceship::Spaceship*;
      scifi::Spaceship::?Spaceship*;
      scifi::Spaceship::stabliseIonFluxers*;
      scifi::Spaceship::initiateHyperwarp*
    };
  local
    *;
};    

A few points need explaining here.

The first entry exports the typeinfo for the class. In this example, it's not strictly necessary, but I have included it to show how. See the sidebar entitled 'Exporting TypeInfo' for an explanation of why you might need to do so.

Tilde (~) is not a valid character in an export map, so in the export line for the destructor, we replace it with a single character wildcard.

Next, notice how within the extern C++ block, every entry ends with a semi-colon except the last. This is not a typo! The syntax is defined that way.

Lastly, the wildcards on the end of the function names are because the full function name includes its signature and we don't want to have to write it out here.

Let's build our example using the example export map and see what happens. This is done by passing an extra option to the linker:

> g++ -shared spaceship.cpp -o libspaceship.so.1 
-Wl,-soname=libspaceship.so.1 -Wl,
--version-script=spaceship.expmap
> g++ testflight.cpp -L. -lspaceship -o testflight    

First the output of nm, to show that we are now only exporting what we actually want to (see Figure 5).

> nm -g -D -C --defined-only libspaceship.so.1
00000b4e T scifi::Spaceship::initiateHyperwarp()
00000b48 T scifi::Spaceship::stabiliseIonFluxers()
00000a02 T scifi::Spaceship::Spaceship(std::string const&)
00000964 T scifi::Spaceship::Spaceship(std::string const&)
00000af4 T scifi::Spaceship::~Spaceship()
00000aa0 T scifi::Spaceship::~Spaceship()

All of the implementation details of our class are now safely hidden away as they should be and only our public interface is visible outside the library.

Obviously, the first thing we must do is run the test harness to check that we haven't broken anything by restricting the exports. It runs without any problems, so we can use LD_DEBUG again to see what difference it has made to the runtime behaviour. Filtering the output to show only those symbols that were resolved to the spaceship library, this time we get Figure 6.

13421: symbol=_ZN5scifi9SpaceshipC1ERKSs;  lookup in file=./testflight
13421: symbol=_ZN5scifi9SpaceshipC1ERKSs;  lookup in file=./libspaceship.so.1
13421: symbol=_ZN5scifi9Spaceship19stabiliseIonFluxersEv;  lookup in file=./testflight
13421: symbol=_ZN5scifi9Spaceship19stabiliseIonFluxersEv;  lookup in file=./libspaceship.so.1      13421: symbol=_ZN5scifi9Spaceship17initiateHyperwarpEv;  lookup in file=./testflight
13421: symbol=_ZN5scifi9Spaceship17initiateHyperwarpEv;  lookup in file=./libspaceship.so.1 
13421: symbol=_ZN5scifi9SpaceshipD1Ev;  lookup in file=./testflight
13421: symbol=_ZN5scifi9SpaceshipD1Ev;  lookup in file=./libspaceship.so.1 

This looks more like we'd want this time: the only things being resolved to the library are the functions that make up the library's public interface. What's more, if you compare the raw output of the two runs, you'll notice that there are fewer lookups being performed in total, because we no longer have the weak symbols to be resolved when our library is loaded.

Symbol versioning

At its simplest, this requires a simple addition to the export map:

SPACESHIP_1.0 {
  global:
    extern "C++" {
      *scifi::Spaceship;
      scifi::Spaceship::Spaceship*;
      scifi::Spaceship::~Spaceship*;
      scifi::Spaceship::stabliseIonFluxers*;
      scifi::Spaceship::initiateHyperwarp*
    };
  local
    *;
};    

In order to see the effect this has, we need to use objdump, rather than nm, because nm does not display symbol versioning information. First, if we look at the symbols exported from libspaceship.so.1, we can see that they are all now marked with a version. objdump doesn't have a filtering option equivalent to nm's --defined-only, so I have picked out just the relevant lines from its output in Figure 7.

> objdump -T -C libspaceship.so.1
libspaceship.so.1:     file format elf32-i386
DYNAMIC SYMBOL TABLE: 
00000a92 g    DF .text  0000009d  SPACESHIP_1.0 scifi::Spaceship::Spaceship(std::string const&) 
00000bde g    DF .text  00000005  SPACESHIP_1.0 scifi::Spaceship::initiateHyperwarp() 
00000b84 g    DF .text  00000054  SPACESHIP_1.0 scifi::Spaceship::~Spaceship() 
00000000 g    DO *ABS*  00000000  SPACESHIP_1.0 SPACESHIP_1.0 
00000b30 g    DF .text  00000054  SPACESHIP_1.0 scifi::Spaceship::~Spaceship() 
000009f4 g    DF .text  0000009d  SPACESHIP_1.0 scifi::Spaceship::Spaceship(std::string const&)

Notice how each export is now marked with the version string we gave. Also, there is a single extra absolute symbol which states that this shared library provides this version of the ABI.

Now let's look at the imports in the test executable. Again, I'm going to pick out just the entries (Figure 8) that relate to libspaceship. If you do this for yourself, you'll see a lot more entries for glibc and libstdc++.

> objdump -x testflight
Dynamic Section:   NEEDED      libspaceship.so.1 
Version References:   
required from libspaceship.so.1:     0x04a15a30 0x00 03 SPACESHIP_1.0 
SYMBOL TABLE: 
00000000       F *UND*  00000005          _ZN5scifi9Spaceship17initiateHyperwarpEv@@SPACESHIP_1.0 
00000000       F *UND*  00000054          _ZN5scifi9SpaceshipD1Ev@@SPACESHIP_1.0 
00000000       F *UND*  0000009d          _ZN5scifi9SpaceshipC1ERKSs@@SPACESHIP_1.0 
00000000       F *UND*  00000005           _ZN5scifi9Spaceship19stabiliseIonFluxersEv@@SPACESHIP_1.0

Not only does the executable state, as usual, that it needs libspaceship, but it now states that it needs a version of libspaceship that provides the right version of the ABI. In addition and more importantly, each symbol being imported from our library is now marked with the required version.

There are some more sophisticated things that can be done with symbol versioning, such as marking which minor version of an interface symbols were introduced in, by having more than one section in the export map. See the reference at the end for more information on this.

Making it easier

There's only one slight fly in the ointment. For a trivial example, that export map looks fine, but maintaining it for a real library could quickly become painful.

One answer, that works for some circumstances, is to think carefully about how much you actually need to export from your shared library. If you are programming to interfaces expressed as abstract base classes, then you probably also have factory functions to create instances of implementation classes, which return a pointer to the interface. In that case, it often turns out the the only thing that needs to be exported from the shared library is that factory function. On one project that I work on, we have a number of shared libraries that are loaded dynamically at run time that have this property. Because the libraries are loaded dynamically and the factory function is found at runtime using dlsym(), the factory function can have the same name in every such component and so we can generate the export map at build time and don't have to maintain them at all.

That is not always a sensible or workable approach though; if you are writing a C++ class library, then you need to export the classes.

By deciding to write an export map, we have, in effect, created the same situation that we have on Windows: we are starting with an empty 'global' section in our export map, so that nothing is exported and we're now trying to get a list of what to export. On Windows, this is often done by decorating the code with some special directives for the Microsoft tool-chain. (Note that these directives occur in a different place in the source code to the GNU __attribute__ directive.) These are often hidden away in a macro, because different directives are needed when building the library and when linking against it:

#if defined( _WIN32 )
#  if defined(SPACESHIP_EXPORT_INTERFACE)
#    define SPACESHIP_API __declspec(dllexport)
#  else
#    define SPACESHIP_API __declspec(dllimport)
#  endif
#else
#  define SPACESHIP_API
#endif
class SPACESHIP_API Spaceship
{
  // etc
};
void SPACESHIP_API myGlobalFunction();    

If all the classes that are to be exported are marked this way, then it ought to be possible to write a tool that will generate the export map for us.

An approach that I've been experimenting with is to use regular expression matching to find interesting declarations (namespaces, classes / structures and functions) in header files. Tracking instances of opening and closing block braces allows the generation of scoped names in the export map. Of course, it's not possible (as far as I'm aware) to write a regular expression that will correctly identify function declarations in the general case, but we can spot global functions that are to be exported by the presence of the SPACESHIP_API tag and if we avoid writing any serious implementation inside our exportable class declarations, then we can spot function declarations within the class declaration body. Looking out for public / protected / private tags allows us to avoid exporting implementation functions.

Anyone fancy a summer project?

References

• How To Write Shared Libraries: Ulrich Drepper, 2005
• Linux man and info pages, nm, objdump, ld: various

Notes: 

More fields may be available via dynamicdata ..