Title: new & delete

Author: 

Date: 01 February 1994 08:56:00 +00:00 or Tue, 01 February 1994 08:56:00 +00:00

Summary: 

Body: 

There seems to be much ignorance and confusion about the C++ dynamic memory management tools. As these have been changed and further refined by the July meeting of XJ16/WG21 (the correct designation of the joint ANSI and ISO committees standardising C++) I am going to put aside the other things that I have listed for articles and focus on this.

First of all there are four (not two) operators involved with associated families of functions. The operators are strictly called:

new operator
new[] operator  (read as new array)
delete operator
delete[] operator  (delete array)

The new operator is a built-in action of the language and does two things. It calls a possibly overloaded operator new function and then calls a constructor to initialise memory.

The terminology used in technical papers about the memory management facilities could hardly be worse chosen. From here onwards I will refer to the new (new[], delete, delete[]) operation when I mean the built-in memory management facilities. I will refer to the functions that are (primarily provided by libraries or users) designed to raw memory as the new (new[] ...) function.   In these terms you can describe the action of the new[] operation as:

Call the appropriate version of the new[] function and then call the relevant constructor once for each element of the array. (place information somewhere about the number of elements in the array).

The other two operations are

delete operation:

Call the destructor for an object of the type pointed to and then call the appropriate overload of the delete function.

delete[] operation:

Call the destructor for an object of the type pointed to for each element of the array (find the array size via information squirreled away by the compiler implementation of the new[] function. Then call the appropriate version of the delete function.

Please understand that you cannot change the memory management operations, you can only change the functions that they use. Each operation directly calls at least two functions (one of them may be called many times).  In the case of the new and new[] operation failures of the new or new[] function must be dealt with. More about this later.

Many programmers seem to be unaware that standard implementations of C++ provide at least two versions of the new function. The first obtains memory from some heap (in some environments this might not always be an operating system's heap) and returns a void*. The second (often called placement new) simply relays a user provided pointer as a void*. In other words the programmer can provide memory. E.g.

char * cp;
cp=(void *)malloc(20*sizeof(int));
int * ip=new (cp) int[20];

At least a passing knowledge of this form can help you understand some error messages generated by syntactically incorrect statements using new. Sometimes the compiler thinks you are trying to write a placement use when you have simply got in a tangle trying to create a complex object with new.

Likewise an explicit call of a destructor does not free up the underlying memory (though it does release resources used by the destructed object).

As there is no efficient mechanism for the compiler to determine your intent with regards allocation and de­allocation of resources it is essential that you make correct use of the memory management tools of C++. Many programmers have become slack about matching new with delete and new[] with delete[] (and indeed the rules have been in a state of flux over the last ten years, which has hardly helped).

Simple objects that do not attach resources to themselves and which have destructors that do nothing are relatively robust. If you mix the various memory management tools (including mixing new with free or malloc with delete) nothing will happen to warn you that you have it wrong.

As soon as you start writing objects that attach resources (more memory, files, serial ports etc.) your bad habits will cause problems. Again you may not notice them as small memory leaks when you have plenty to spare, failure to close a file and so on will often prove less than fatal. However the problem is still there and one day it will matter.

This leads to the first of my rules:

Never use a pointer to handle both single objects created with new and array objects created with new[]. Doing so is bound to lead to undetected faults in your product. (If necessary you can always have an array of one object.)

A substantive change

For a committee that constantly rejects proposed changes (such as attempts to bring the for initialiser statement into line with normal programmer expectation - but that is another story) XJ16/WG21 had little hesitation in invalidating all prior usage of new with its decisions of July '93. Quite apart from splitting off the array versions as operations in their own right, they also declared that the normal behaviour for a failure of memory allocation would be to throw an xalloc exception. This behaviour will replace the previous form that returns a NULL pointer.

Of course, in the committees defence I must point out that implementors will elect to provide the previous behaviour as a compiler option. However it also means that you must carefully document all your in-house libraries (and vendors will need to document theirs for you) so that you know which behaviour to expect. If you have never used a class specific version of new/delete or replaced/overloaded the global versions this should still concern you. Commercial libraries may have done so. Worse than this, commercial libraries may be relying on the now replaced methods.

If you think you have any breathing space to come to terms with this change I have news for you; the new Borland C++ compiler already defaults to throwing an xalloc exception.

For many years you are going to have to carefully document the failure behaviour of new in your programs. You will also need to check the behaviour in all the third party libraries you use. (There are other implications to using exceptions, but this is not the place to explore them.)

Writing Your Own operator new()

Let me focus on a simple value class to illustrate the reasons for both writing your own new function and for the added ability to write your own new[] function, class Complex will do nicely for this. The size of instances of this class would normally be small. Programmers happily return actual Complex values from functions. In all aspects its expected behaviour is like a built-in.

If you treat it like a built-in and rarely use new to create instances everything will be fine (and this is the way I would expect it to be used, but I need a well understood user defined type which is small and of fixed size) but what happens if you start creating dynamic instances of Complex?

Very soon you find that your program is spending most of its time allocating and de-allocating memory. Eventually you might wonder if you could not do better by taking a block of memory and managing it directly. As you are looking for speed efficiency you would probably get a block from the heap and structure it into a linked list of freespace blocks with each block large enough to hold a single instance of Complex.

Now the job of your Complex scope new function will be to get the address of the next free block and return it as a void*.

What happens if there is no free blocks? Perhaps you then get another large chunk from the OS. Or may be you expect this to be rare and are willing to revert to the built-in version. If the latter you are going to need to be much more careful with writing your version of the delete function for Complex. You can still manage as long as you keep static information about the address range for your standard version which can be used by your delete function to discriminate between normal and abnormal dynamic allocation for Complex.

If you create your own new function for Complex then you must write your own version of the corresponding delete function. This will have to return the released block to your free list (or if you are going for the clever version, it will need to check which version of new allocated it and either return the released memory to the free list or to the OS)

The prototypes for straightforward user written version of the new function and the delete function are:

void * operator new(size_t size) 
void * operator delete(void *ptr)

One problem with this is what happens if you cannot get enough memory? The first thing is to get the address of the new_handler function. A typedef is useful here:

typedef void (*PEH)();

PEH is a pointer to a function with no parameters returning a void*

then

PEH  currentHandler=set_new_handler(@@@); 

where @@@ represents the address of the next handler after this one that you want to try - usually NULL. Now if currentHandler is not NULL you call it and try again.

If it is NULL (i.e. there is no handler) your new function must either return 0 or throw an xalloc exception.

If you called a handler and it has successfully found some memory, do not forget to reset the memory handler before returning from your new function.

The above outline is only the tip of the iceberg as regards writing your own new and delete functions but it would be an excellent exercise to try one for yourself. Send your attempts to Mike and we will find some suitable award for the one which exhibits the best quality code.

At this stage you might ask why the previous rules specified that arrays of objects (Complex in the case we are looking at) revert to the use of the global version of the new function?

The reason is that your class based version may not meet the need. Indeed if you follow the suggested plan for a new function for Complex it definitely will not work for arrays. Arrays must occupy contiguous space and apart from memory alignment requirements the programmer is entitled to expect adjacent members of arrays to be adjacent in memory. This will not be the case if we use our linked free list - the link pointers will be taking up space.

Until July '93 you had no mechanism for providing a class based version of the new function for an array of class objects. This restriction began to seriously annoy some programmers who wanted to provide sophisticated memory management on a class by class basis. The solution was to introduce to extra functions, new[] and delete[] that would be called by the new[] operation and the delete[] operation. These now exist in global form and can be replaced, overloaded and provided within a class scope quite independently of the new and delete functions.

I am not going to go into detail about these two functions because I have not had enough time to give them much detailed thought. But the important point is to remember that they are provided to get raw memory for an array. They are not responsible for calling constructors/destructors. That job is out of your hands, it is done automatically by the new[] operation and delete[] operation.

Hunting around for some way to try to help those with little experience of memory management functions I came up with the following concept:

There is something rather like a global virtual function for new that is called by the C++ new operation. Actually it is more like an overloaded virtual function. When I use the new operation it will select the version of the new function appropriate to the set of parameters that is found in the nearest enclosing scope of the object being created.

Similar functions exist for the other three operations (new[], delete and delete[])

Another Use.

When I first started programming in Basic (yes I did for a time :-) I came across various mechanisms used by interpreters to support strings. One method assigned 4 bytes to each string variable. The first two were a pointer to the memory where the actual string was and the second pair gave the length of the string. There was a large heap of memory that was used as string space and a defined space for the four byte handles. Periodically the machine would go into a huddle while it garbage collected its string space. This is quite a complicated process because the location of the handles must not be changed though the strings can and will be.

One possible mechanism for garbage collection is to keep a free store list for your string area and set some trigger mechanism to clean up storage when a request for string storage fails. The clean-up routine will be a new_handler specific to the relevant class (I hesitate to talk about a String class as there is already one of these per programmer :-)

Now if you are going to do this kind of memory management, your versions of the new functions will use there own new_handler(s) before calling that currently returned by a call to set_new_handler(). Why do I hint that there might be more than one special new_handler()? Well you would initially look for space to meet the requirement (using which ever is your chosen algorithm to select from a free list). If this fails you would call a handler that unified adjacent blocks found in your free list. Only if this failed would you proceed to garbage collect your string space.

This mechanism is not specific to storage for strings but can be used by any class that is constantly re-allocating memory. A good example of this would be a matrix class. A large enough matrix would probably be worth allocating via global new but smaller matrices would be better handled in a reserved block of raw memory for the matrix class. As matrices vary in size this raw memory would be a good candidate for garbage collection.

Think carefully about the costs for garbage collection. Its purpose is to handle limited memory resources efficiently. The result will be a definite time penalty not only for the garbage collection itself (which normally eliminates this as an option for real time systems) but for the extra level of indirection needed to allow relocation of your object. None-the-less if you are faced with much allocation and de-allocation of memory for objects of variable size you must either do the garbage collection yourself, tolerate fragmented memory or hand the responsibility for garbage collection/fragmented memory to the operating system.

Of course your application may be an operating system in which case you are going to have to face up to the problem and learn about the appropriate data structures and algorithms yourself. Experience of operating systems to date suggest that there is plenty of scope for improvement among those programming these.

I already have an ever growing stack of things to write about so I am going to hand over the task of writing and then writing about a garbage collection memory management system for class Matrix to you. I hope that one or more of you will come up with a quality product. Even if you cannot manage an all singing, all dancing Matrix memory management system I can assure you that you will learn a lot from trying to write one. It is not as difficult as it may seem.

Another Problem

Arithmetic classes (a subset of value-classes) usually opt to return instances by value. For example operator + () normally returns a value not a reference. For classes with small object size this can be lived with - indeed it is probably efficient for speed and not bad for memory use. But some of these classes cannot work efficiently if their arithmetic functions return by value. However any one who has ever tried to write such a class will be completely familiar with the problem of trying to return by reference (dangling references are only one of the problems).

I have a way of fixing this problem so that, for example, the arithmetic operations for a matrix class can return references without placing unenforceable syntactic restraints on their use.

Before I tell you my solution I invite you to suggest one of your own. To make the problem clear I want to fill out the following:

class Matrix {
    // constructor to produce an a by b Matrix 
    Matrix (int a, int b);
    // constructor to create an unitialised 
    // matrix 
    Matrix ();
    ...
    Matrix & operator + (const Matrix &) const; 
}

So that

int main () {
    Matrix ml, m2(3,4), m3(3,4), m4(3,4);
    ... 
    //code stuffing some values into m2,m3,m4
    ml=m2+m3+m4;
    ...
}

Notes: 

More fields may be available via dynamicdata ..