Title: A Touch Of Abstraction

"Abstraction Is Evil!" So claimed Jim Coplien in his keynote talk at last year's JaCC conference. Although simply dropped in as a soundbite, with no elaboration, the statement created a definite buzz among the audience. During the rest of the conference there was much informal debate on the point, both involving Cope and between other delegates. In the main Cope's statement was seen as a deliberately provocative attack on one of the keystones of software development. But how much substance is there to the claim? What follows is a brief look at some of the issues.

A dictionary definition describes abstraction as:

This seems relevant to how we think about abstraction as applied to software development. However, it seems a little unguided: we can take from it that abstraction offers a way to manage complexity, but no more. The definition does continue to discuss the essential role abstraction plays in classification, where objects with the same abstracted properties are arranged in a class. This gives us a little more purpose. We can now relate this to a software specific definition, taken from the glossary of the UML specification:

This stresses the role of abstraction in supporting partitioning and separation of concerns, but raises an interesting and important point - the importance of the perspective of the viewer. An abstraction is sensitive to context. In fact, there are no such things as *the* essential characteristics of an entity, the characteristics that are significant are determined by context. For example, take colour (itself a classic abstraction). If I want my living room painted blue, telling the decorator just that would be a poor use of abstraction. Instead, with both of us referring to the same colour chart, I tell him Velvet Sky. But when I tell friends what colour the room is going to be, Velvet Sky is inappropriate: I tell them blue. This is a different level of abstraction, but it is precise enough to enable them to advise me whether the colour will clash horribly with my sofa.

There is a further dimension to abstraction in software design that we have not covered so far: the concept of variability. Ironically it is given very good coverage in Cope's Multi-Paradigm Design for C++ [Coplien], in which he discusses both commonality and variability analysis. The concepts of commonality and variability relate closely to abstraction. Commonality relates to classification, as previously discussed. But if abstraction reveals commonalities among entities, then it must also be concerned with variabilities. Indeed, an abstraction that captures an aspect of commonality without identifying some associated variability does not prove to be very interesting. So it is worth noting that abstraction seeks to identify commonalities *and* variabilities.

It is an indication of how deep-rooted our use of abstraction is that we take its benefits for granted, and are surprised by Cope's attack. Abstraction is a fundamental analysis and design tool, which underpins much of what we do. In analysis we seek to develop a model of the problem domain. This by definition involves abstraction, as a model is an abstracted representation. Note that by this I do not necessarily imply a UML supported process, this model may simply exist in a single programmer's head. When this model is further developed during the "design phase" (again not focusing on any specific process), abstraction plays a key role in structuring systems. Abstraction, as we know, supports partitioning. While the concepts are separate, they tend to go hand in hand. In the common Layers architectural pattern [Buschmann-], for example, an application is decomposed into groups of subtasks that form a set of layers: each layer presents an interface to the layer above which represents a certain level of abstraction. It is possible to produce many more examples of the use of abstraction to aid software development. So, what is wrong with abstraction?

Returning to the dictionary definition, it indicates that abstraction is about ignorance. The process of abstraction involves removing details that are deemed irrelevant based on some perspective. Actually we usually do not remove anything, instead we hide these details behind an interface, and keep them secret. Cope describes abstraction as "selective lying". This may appear to be deliberately twisting the strengths of abstraction - managing complexity and supporting separation of concerns. But there is a point. The intention is, of course, that the details that have been chosen to be hidden by an abstraction are not important to the users of that abstraction, and can be safely hidden. But the process is unreliable, and it is possible for the designer to make the wrong decision about what is relevant, and therefore abstract away important information. Recall that abstractions are defined relative to a context: the context must be well understood for the appropriate abstraction to be developed. What is more, how the context is likely to change in the future needs to be understood in order that the abstraction be flexible enough to remain valid as the system develops. Often this is not the case, and abstractions that are too rigid can obstruct rather then ease the evolution of a system.

Creating abstractions that are precise, cohesive and with clear interfaces is difficult, and often developers will at least partially fail in the task. Some of these concerns are expressed very eloquently by Richard Gabriel in his book Patterns Of Software [Gabriel], in a chapter that explores the drawbacks of abstraction. He focuses on the problems that can occur through the overuse of abstraction, first pointing out that such overuse is encouraged by much education and literature in the field. He envisages a scenario where abstraction is used at all levels, resulting in a system that is made up of many complex inter-related, possibly ad hoc, abstractions. This can lead to several problems in addition to the difficulty of creating good abstractions. The more an abstraction is shared, the more difficult it is to make any changes to its interface. Abstractions are intended to hide implementation detail, but in reality some implementation detail can often leak out. Even if this does not occur, the act of hiding implementation detail can be problematic. Where this has been done inappropriately, it can lead to situations where users of an abstraction make (false) assumptions about the implementation.

It should be noted that Gabriel does not condemn abstraction per se, but its misuse, particularly where it reduces the "habitability" of a software system. He defines habitability as the characteristic of a piece of software to enable developers to live comfortably in and repair or modify code and design. This recognises the fact that a software system, like a building, rarely remains static after its initial construction, but instead develops in ways that are not originally anticipated. Building software out of grand, complex abstractions makes such evolution difficult, as later developers struggle to fully understand the existing system. On the other hand, a system developed with smaller, more natural abstractions will be more understandable and allow a smoother evolution path.

Cope makes the case for compression over abstraction. In doing this he draws directly from Gabriel's definition of compression. Gabriel uses the analogy of a poet striving to produce statements which are as concise and economical as possible, while still conveying all the intended meaning. The difference may seem to be one of emphasis: abstraction has a similar aim, though compression perhaps suggests a more holistic approach. Gabriel elaborates by defining compression as "the characteristic of a piece of text that the meaning of any part of it is 'larger' than that piece has by itself". In other words, each part draws on its context to produce a text that is dense with meaning. In software, or specifically object-oriented languages, Gabriel suggests that compression can be created through inheritance. When the definition of a subclass bases some of its meaning on the definition of its superclass(es), the result is a compressed definition that depends on details taken from its context. Thus in an object-oriented language, compression can lead to compact and efficiently expressed programs. Interestingly, Gabriel does not draw as clear a line as Cope between abstraction and compression, presenting compression as a mechanism for achieving abstraction, with equal potential for overuse. The result is systems with poor habitability caused by deep inheritance hierarchies, where knowledge of the whole hierarchy is needed to work with any part of it. Cope on the other hand suggests compression as an alternative to abstraction. I feel that with his favouring of compression he is suggesting that we should break out of a mind set that leads us to see abstraction as the only useful tool for constructing software systems, that there are other ways of solving design problems. As such it is in support of his whole point. But how compression, considered separately from abstraction, assists us is unclear.

So, is abstraction evil? Well, it makes a good soundbite, and as such should never, of course, be taken at face value. The statement is itself an example of abstraction, as there are a great many of concepts hidden behind it. The arguments against abstraction tend to focus on the ease with which it can be misused or misapplied. But abstraction is a powerful tool, and so it is inevitable that its misuse is both easy and highly problematic. It is not difficult to see that abstraction can have a dark side, and it is wise to be aware of this, however, I find it impossible to condemn it. As for Cope, though a careful reading of the opening chapters of Multi-Paradigm Design for C++ makes his latest statement a little hard to reconcile, he maintains that he absolutely means it. The debate, I'm sure, is set to continue.