On CMM, Formalism and Creativity

Disclaimer: as usual, opinions within this article are those of ‘No Bugs’ Bunny, and do not necessarily coincide with opinions of translator and Overload editors; please also keep in mind that translation difficulties from Lapine (like those described in [LoganBerry]) might have prevented us from providing an exact translation. In addition, both the translator and Overload expressly disclaim all responsibility from any action or inaction resulting from reading this article.

Thlayli lay meth methrah nao
(Bigwig is a poor storyteller)

Creativity and formalism

Today I will try to touch on quite a sensitive issue, related to the subtle relationship between creativity and formalism, which my fellow rabbits can often feel but which is usually quite difficult to write down. While a few years ago it was argued (see, for example, [Konrad05]) that agile development can co-exist with formal methodologies like CMM, the question about the co-existence of formalism with creativity has not been addressed in the rabbit literature yet.

I will certainly not argue that creativity is always a good thing. In many cases I am personally really afraid of excessive creativity. For example, I certainly don’t want to fly on a plane which has been serviced by mechanics who were excessively creative (such as the one described in [JA8119]), or to be operated on by a surgeon who’s just had a fancy idea about how to make things so much better and wants to try it on me. On the other hand, in many cases creativity is highly desirable – the guy who invented the wheel did need to be creative.

Now to the question of formalism. Let’s consider a team or organization which does need to be creative. What level of formalism is optimal for such a team? Should this team be in a state of complete anarchy? Or should it be a perfectly working machine where everything is done ‘by the book’? Intuitively, it is quite clear that the latter is not the right way to inspire creativity, but unfortunately way too often management doesn’t realize this and measures success, not in terms of a successful end product, but in terms of ‘how organized the process is’. Let us consider this whole problem using CMM – Capability Maturity Model – as an example of such a management approach.

On CMM

CMM originated at the end of 1980s in a book by Watts Humphrey [Humphrey89]. From the very beginning, CMM was closely related to the US Department of Defense (and in particular the US Air Force). While the idea of improving software quality is certainly commendable, especially within military applications, it seems that the temptation to make developers march in formation was too strong to overcome, and this temptation eventually found its place within CMM. Later, around the end of the 1990s, CMM has been replaced with CMMI – Capability Maturity Model Integration – which tends to cover much more than the original, including ‘CMMI for Development’, a direct successor of the original CMM. Also the formally separate but ideologically similar ‘People CMM’ was released. Ironically, about the same time the very same Watts Humphrey realized that CMM doesn’t really work in practice and came up with an alternative model known as ‘Personal Software Process’, which has evolved into ‘Team Software Process’, a.k.a. PSP/TSP. While the main promoter and owner of the ‘CMM’ trademark (SEI of Carnegie Mellon University) considers PSP/TSP as a valid (and recommended) CMM/CMMI implementation, for the purposes of this article we will consider it separate and specifically comment on it later.

From the point of view of management, CMM if often considered as a kind of ‘holy grail’, where it is enough to pre-build organizational processes and procedures and then the project will march ahead towards the bright dawn of success; unfortunately, experience shows it is certainly not guaranteed. In addition, CMM (similar to ISO9001) is often seen as a prerequisite to obtaining certain government contracts, as well as a way to get some kind of certification to show clients that the organization is a ‘good one’. Research conducted by SEI shows improvements in productivity, at least in some cases. On the other hand, it often faces harsh criticism (again, similar to ISO9001) from both developers and CIOs, ranging from ‘CIOs who look to CMM for guarantees won’t find them’ [Koch04] and ‘In fact, the study found that Level 5 companies on average had higher defect rates than anyone else.’ [Koch04] to ‘At worst, the CMM is a whitewash that obscures the true dynamics of software engineering, suppresses alternative models.’ [Bach94] Opinions of fellow rabbit developers are often even harsher, which is why I won’t be able to quote them here. In short, it is quite a controversial subject.

Repeatability and replaceability

The key idea behind CMM is repeatability: as [P-CMM] says, ‘A fundamental premise of the process maturity framework is that a practice cannot be improved if it cannot be repeated’. Even this statement is not really obvious, but let’s see where it leads. For the purposes of this article, we will ignore many other implications of repeatability, concentrating on only one of them: for the process to be repeatable, people need to be replaceable. CMM states it in terms of ‘exceptional individuals’ [P-CMM, page 12]: ‘...in low-maturity organizations, their results depend largely on the skills of exceptional individuals...’ (and in CMM speak, ‘low-maturity’ is pretty close to ‘double-plus ungood’).

Now let’s try to put ourselves in the shoes of a manager: I am a manager, and have an exceptional individual on the team, great! But after reading a book on CMM, I’m starting to wonder: what happens if she leaves? All repeatability goes out of the window. Therefore to comply with CMM I (as a manager) need to eliminate all dependencies on irreplaceable developers. If done with caution, it is not a bad thing to eliminate dependencies, but unfortunately way too often management misreads it and eliminates exceptional individuals completely, using CMM as an excuse. Add that it is way too often aligned with the inclinations of a not-so-good manager who is not competent enough to manage the project, and we see the classical case of injelititis [Parkinson57]: where ‘...the head of the organization is second-rate, he will see to it that his immediate staff are all third-rate; and they will, in turn, see to it that their subordinates are fourth-rate.’ Regardless of whether it was intention or not of CMM to achieve such a situation, it certainly helps managers to reach it.

Exceptional vs average

Even if the manager doesn’t suffer from injelititis and doesn’t eliminate everybody who’s smarter than him, applying CMM is problematic in certain areas. If processes are strictly adhered to (the thing which CMM strongly advocates), what will these exceptional individuals do within a CMM-compliant organization? By the definition of CMM compliance, such individuals are required to obey the same processes as everybody else, and if these processes are rigid enough there is no room to apply their exceptional abilities; as a rule of thumb, exceptional individuals either leave such organizations, or find a way to bypass the processes (which essentially ruins CMM). In short: if management tries to fit an exceptional individual into an existing process (making the exceptional individual just another ‘cog in the machine’ without regard to her exceptional abilities), it doesn’t work.

Therefore, by going ahead with the very spirit of CMM towards repeatability and replaceability, it will inevitably eliminate exceptional individuals from the team, which leads towards teams consisting only of about-average individuals who’re fine with the role of ‘cogs in the machine’. The approach of about-average individuals works great in those fields where creativity is not an issue (there is no argument that air mechanics should be easily and directly replaceable), but what about areas where creativity is necessary? What about a replaceable Einstein or Enrico Fermi?

Are creativity and CMM are incompatible

While it is obvious at an intuitive level that the approach of average individuals won’t work where high levels of creativity are necessary, it is interesting to note that it is possible to provide a more formal demonstration of it. Let’s take a look at one of the criteria used to measure innovation, namely the granting of patents. In most jurisdictions, for patents (which are undoubtedly one way to acknowledge innovation) there is a so-called ‘average engineer’ test: usually, a patent cannot be granted if it is obvious to ‘a person of ordinary skill in the art’ (which is essentially legalese for an ‘average engineer’). It implies that inventions cannot be made if you have only ‘average engineers’ on board. As we saw above, replaceability in practice means ‘average engineers’ (ie not using exceptional abilities of exceptional individuals, which is about the same thing), and as CMM implies replaceability, it should not possible to generate any patents within truly CMM-compliant organizations. Q.E.D.

Obviously, this ‘proof’ is not a strict one, and in fact some of organizations which are formally CMM-level 5 do produce patents; however my fellow rabbits who went through CMM and ISO9001 audits feel that was only because CMM compliance (just like ISO9001 compliance) is merely a formality which has nothing to do with realities of software development in an organization; therefore, formal compliance has nothing to do with adhering to the spirit of CMM. Also of interest is that, as with any text which is vague enough, all kinds of interpretations are possible, so it is perfectly feasible to create formally correct interpretations of ‘what CMM means’, with fundamentally different conclusions (which essentially is what was done when ‘Personal Software Process’/‘Team Software Process’ were designed). What is important is that the interpretation described above is certainly by far the most common one when it comes to real managers.

Process-centered vs people-centered

At this point a much more important question arises: does all this mean that projects which require creativity should be completely non-structured, informal, and in a state of anarchy? Not really. It just means that teams which need creativity should be managed not in a process-centered way (like CMM advocates), but in a people-centered way. In other words, whenever creativity is necessary, a manager should not carve processes in stone and then fit people into those processes (essentially making people ‘cogs in the machine’), but the whole paradigm of management should be completely opposite: one will need to build a team of individuals, then build processes around this team, and adjust the processes when the needs of the team (or the team itself) change. We can compare these two approaches in software project management (‘process-centered’ and ‘people-centered’) to two fundamentally different approaches in software development itself: a ‘process-centered’ approach, with processes essentially carved in stone, is similar to the (in)famous ‘waterfall’ methodology, while a ‘people-centered’ one implies dynamic processes with multiple iterations, similar to ‘agile’ development methods.

Such ‘people-centered’ approaches are in fact nothing new: despite being admittedly more difficult for the manager, they have been used many times with tremendous success. [VirtuosoTeams] (BTW, I strongly recommended it for fellow rabbits who’re interested in team leading or management) describes in detail several such teams, including the team behind the ‘Manhattan project’. With a dozen Nobel laureates on board, and a task of epic creativity proportions, it certainly wasn’t anywhere close to CMM past level 2. Still, Colonel Leslie Groves, while certainly struggling with the management of such a ‘virtuoso team’, had managed to find a way to make things work, and (despite his military background) his approach was certainly ‘people-centric’, and not ‘process-centric’ as prescribed by CMM. While the moral grounds of the ‘Manhattan project’ are debatable, from a technical and management point of view it certainly was a tremendous success.

References

[Adams] http://en.wikipedia.org/wiki/Lapine_language

[Bach94] ‘The Immaturity of CMM’, James Bach, American Programmer, 1994, http://www.satisfice.com/articles/cmm.shtml

[Humphrey89] Watts S. Humphrey Managing the Software Process, Addison-Wesley, 1989

[JA8119] http://en.wikipedia.org/wiki/Japan_Airlines_Flight_123

[Konrad05] ‘Agile CMMI: No Oxymoron’, Mike Konrad, James W. Over, 2005, http://www.drdobbs.com/184415287

[Koch04] ‘Software Quality: Bursting the CMM Hype’, Christopher Koch, http://www.cio.com/article/32138/Software_Quality_Bursting_the_CMM_Hype?page=1&taxonomyId=3000

[Loganberry] David ‘Loganberry’, Frithaes! – an Introduction to Colloquial Lapine!, http://www.loganberry.furtopia.org/bnb/lapine/unit14.html

[P-CMM] People Capability Maturity Model Version 2.0, Second Edition

[Parkinson57] Parkinson’s law, and other studies in administration, Houghton Mifflin, 1957. Chapter 8, Injelititis, or Palsied Paralysis

[VirtuosoTeams] Virtuoso Teams: Lessons from teams that changed their worlds, Andy Boynton, Bill Fischer, FT Press, 2005