Title: Francis' Scribbles

You will have to wait to the end to understand that heading. The story starts with one of the test students for my book who after three days of bashing his head against a brick wall finally emailed me with a problem that I could distil down to why this code will not compile:

#include <iostream>
using namespace std;
double distance(int, int);
int main() {
  int abc(0), xyz(0);
  cout << distance(abc, xyz);
}

Try it. I can fix the problem this time by using ::distance(abc, xyz). But how on earth would an inexperienced programmer know to try that? And how could anything in the iostream header conceivably cause a problem?

A little experience will lead you to guess that distance is used as a function name somewhere in the Standard C++ Library. The error messages generated (all five of them) will give you a clue that the iterator header is part of the problem. Presumably the iostream header includes that (and a little thought will show that that inclusion is almost necessary).

The instinctive moral drawn by not a few C++ experts is that that the above code just demonstrates the evils of using directives. Sorry, I beg to differ. Suppose that your code needs to use the distance function from the Standard C++ Library as well as a distance function from a third party library. You might be of the school of thought that prefers using declarations.

Try replacing the using directive in the above code with:

using std::cout;
using std::distance;

which is the style advocated by Andy Koenig in Accelerated C++. Now you get the same errors but the only way to fix them is to remove the using std::distance; declaration.

As an aside, please note the difference between using declarations (that effectively make the name concerned behave as if declared in the current scope) and usingdirectives that simply make all the names in the specified namespace available for overloading and for use without qualification. A using directive is NOT equivalent to providing using declarations for all the names declared in the relevant namespace.

Faced with, to me, unnatural behaviour from all the compilers I had to hand (that for once all agreed with each other, Comeau, CodeWarrior, Borland and MinGW) meant that I had to try to find out what was happening.

First I looked up distance in my copy of Nico Josuttis The C++ Standard Library. Here I found:

template <typename InputIterator>
iterator_traits<InputIterator>::difference_type
  distance(InputIterator pos1,
           InputIterator pos2);

Well, the error messages mention iterator_traits so that looked as if I was moving forward. But why was it trying to instantiate iterator_traits<int> for a template function that could clearly never be called because an exactly matching non-template function declaration was already in scope and must always be preferred?

The answer to that question is simple: the compiler must generate the overload set first before determining a best match. That seems reasonable and avoiding special cases makes sense. But why is it trying to place a function it cannot instantiate into the overload set? Hang on to that question for a moment and have a look at this code:

template <typename T>
class x_traits;

template<typename T>
x_traits<T>::return_type foo(T, T);

int foo(int, int);
int main(){
  foo(1, 2);
}

Do you think that code should compile? It does but only after I remember to add typename before the return type of the foo function template, without a murmur and it selects the correct version of foo(). However notice that I have only declared x_traits, I have not defined it. So what happens when I add the definition? For example:

template <typename T>
class x_traits {
  typedef typename T::return_type return_type;
};

It depends where I place that definition. If I place it before main(), the code fails to compile. If after, everything is fine. In other words as long as the compiler cannot try to instantiate the return type while generating an overload set all will work as desired. However the moment we provide that extra information, the compiler grabs it and refuses to compile the code. Do you feel something is not quite right?

I am grateful to John Spicer of EDG for having dug away into the past to find an explanation of what is clearly silly. Here is what the C++ Standard has to say on the subject (and it took several people quite a lot of winnowing down to find that this was the problem):

I have emphasised two sections.

In 1996 clause 14 of what was to become the C++ Standard was being reworked editorially. One of the changes made was to systematically replace 'generate' with 'instantiate' and another was to replace 'template function' with 'function template specialization'. These were perfectly sensible editorial changes because 'generate' was almost always used as synonym for 'instantiate', and there is no such beast as a 'template function.'

Before that date the first piece of emphasized text read: "the template-arguments (deduced and/or explicit) are used to generate a single template function"

That is substantially more vague and would not have led implementors to try to actually instantiate the function rather than just use its signature. Those two substitutions were generally benign but together leads to the current position where many high quality compilers are rejecting the code I started with.

However the second piece of emphasised text leads to looking further to understand what it means to say that 'argument deduction fails.' This leads to14.8.2/4, which explicitly says: "When all template arguments have been deduced, all uses of template parameters in non-deduced contexts are replaced with the corresponding deduced argument values. If the substitution results in an invalid type, as described above, type deduction fails."

The problem is that the failure occurs whilst the compiler is trying to instantiate a template because that is the only way that it can follow the requirements of 14.8.2. But why are compilers treating that failure as making the users source ill-formed? I do not think that the compilers are doing the right thing here. If I understand the text correctly, the very first error in trying to instantiate the return type should cause the compiler to silently remove the function template from the list of overload candidates.

That means that compiler implementors have to be more careful of the context in which a template instantiation is taking place.

I think not. 14.8.2's requirements re generating an overload have some serious things to say about export.

In order to determine if argument deduction for a function template such as std::distance has succeeded the compiler has to see the relevant template definitions. For example consider:

template <typename T>
class x_traits;

template<typename T>
x_traits<T>::return_type & foo(T, T);
int foo(int, long);
int main(){
  foo(1, 2);
}

Now foo<int> is the best match if it is allowed to be part of the overload set. However under the requirements of 14.8.2/4 it should not be considered if x_traits<int> cannot be instantiated. However the compiler cannot know the answer to that question until it can see the definition of x_traits.

Under the inclusion model for templates that will not happen (I think) but once we bring export into play all the bets are off.

This all started because a novice was faced with code that would not work. That happened because I, as an author, had checked my code, moved on, re-organised a library and did not check that the re-organisation had not broken anything. I knew in my own mind that it could not have done. In one sense I was right, because it should not have done but that is not really an excuse.

I guess that this was not the first time that the issue has arisen somewhere during the last five years. I guess that it has just been treated as a reason for avoiding using directives (and possibly declarations).

We all know that C++ templates are problematical and so we do not make enough effort to understand when things go wrong. I think we do have a real problem hiding within the novice's code not compiling. I think we have to address that problem. I do not think that it is going to prove at all easy.

I think that there is a subtle but serious break in the overload rules when function templates that have templated types dependant on their deduced type arguments. I would hate to be a compiler implementor faced with this problem but I think that we should not just dismiss it.

It is a fundamental precept of programming in C++ that the meaning of code should not silently change because we make a definition visible. In my opinion the current rules for overload resolution break that precept.