Title: In Response to Extreme Programming

One of the dirty little secrets of software development is; specifications do not work. The Victoria novel approach to software development, so beloved by out-sourcing companies is fatally flawed. To write a comprehensive spec you have to implement most of the system. Performing comprehensive analysis is a sure-fire way to get bogged down on the starting blocks. The only complete specification you will ever have is the program code.

XP copes with this issue in two ways. Firstly, specifications are presented as a set of stories, a story is half use case, half feature. You have a pile (literally, your advised to write these on CRC cards) and the top priorities come at the top and the lesser ones at the bottom. This also allows XP to cope with changes and additions to the spec, you simply add and remove stories (CRC cards) as required.

Secondly, XP accepts that the code is the best source of specification and documentation.

Like McCarthy, Beck is a big, big fan of iterative development. The "release early, release often" school has been gaining ground for several years and is widely used in the Open Source community.

I long ago came to realise that large monolithic release just do not work. They are easy to understand and fit into a water-fall development strategy but increase the risks, if you miss a release everything in that release is lost. With a short release cycle, adding a few features at a time, you minimise these risks.

Short release cycles do have a downside: organisations that are change averse will require a major cultural shift. Also, every release carries over heads, these are usually fairly fixed for each release (e.g. customer sign off, release notes, source code labelling) and if you increase the number of releases you will increase the amount of time spent on these activities. Further, every release carries a risk factor, a series of small incremental releases could entail more cumulative risk than one big release.

Most developers accept testing as a necessary evil. XP has an interesting take on this: write your tests before you code and write them as automated tests. I cannot say I disagree with either of these but I do not think it is as easy as Beck makes out. A large part of XP depends on you being able to write automated tests that can be run repeatedly and quickly. Sometimes this will not work: I've been on several projects where it just is not possible to automate the tests e.g. data is not repeatable because it comes from a market feed or a environment sensor; and I've also written programs with run times of several hours.

Writing the tests before coding should confer some of the benefits of prototyping because you are forced to explore the problem before constructing the solution.

Beck claims there are four variables in XP: Cost, time, quality and scope^[2]: the objective of the planning game is to bring these into equilibrium for the next iteration cycle. Beck encourages a team approach, together with the customer, to the planning game. Beck does not pretend, as so many managers do, that requirements are independent of the time it will take to implement them. I am sure most developers have faced a high priority requirement X which will take N weeks to implement, and low priority requirement Y with will take N hours: sometimes a little jam today makes a long wait more attractive.

This is not to say always we do the easy things first: Beck is right to say tackle the difficult bits first, this way you avoid hitting a show stopper later and have some easy work to look forward to.

This is probably the XP idea that gets the most attention, and is undoubtedly not without merit.

Personally, I am not sure its always applicable. Sometimes we just require a good old think, maybe a fiddle and a re-think. I expect working in a pair would make you more prone to get something working without necessarily giving it in depth thinking. But as XP advocates simple design, this is not an issue.

In part, I suspect it depends on the temperament of developers: for many programming in pairs will bring a discipline, for others it will seem like a bind. Maybe to pair, or not to pair, for any given piece of work, should be the decision taken at the time by a developer.

While I agree with the sentiment of this idea, this is where I have my biggest problem with XP. Let me take it as two items to start of with.

Simple design: Beck says "produce the absolute simplest solution" and re-work it next time. For me this raises so many questions. What is the simplest design? Lets start throwing away some stuff:

I cannot accept that it is always in the best interest of the system to look the other way and do something very simple every time. Exception handling in particular is notorious difficult to retro-fit yet can, in the long run significantly enhance the understandability and hence maintainability of a system.

I strongly believe that if we pursue the "simplest design possible" we will end up with code which lacks elegance and extendibility.

Potentially, this rule violates Bertrand Meyer's "open-closed" rule: the code will not be open to extension (as this is the absolute simplest possible why should it be?), and will not be closed to other modules because if another module needs to re-use it we can just refactor it then.

I am pleased Beck and others are making refactoring respectable, indeed, almost a buzzword. However, I am not sure that refactoring is possible without some semblance of change to XP principles. Faced with several thousand lines of tag-and-switch code (done that way because it is simple, and lets face it, in a small program a switch statement is easier to follow than virtual methods) I may not be able to refactor to any significant degree.

I suspect that this simple design and refactor is a substitute for prototyping. It amounts to the same thing: write something then throw it away when we have learned about the problem. Compare the development cycles for both:

My biggest problem with all of this is one of my own rules of thumb built up over several years. Get it right first time, because no matter how much you or your boss thinks you can come back and sort it out later chances are you will not get the time.

To give Beck his due I suspect that his definition of simplest design possible may not be as absolute as mine. An individual's definition of simplest possible will depend on their experience and views. Kevlin Henny has suggested that a solution that computes but is not architecturally elegant is not a solution that works. If we moderate our definition of simplest possible we may over come many of the problems raised here.

Beck is equally absolute on the matter of eliminating duplicate code as part of refactoring. It is worth noting that John Lakos makes a good case for the use of duplicate code.

Any team adopting XP would be well advised to prepare for these battles.

Collective ownership seems like a good idea but again I have some worries about it. Could it actually be an excuse for no-ownership? Where an entire team owns a product that is a good thing, when the new version ships the whole team have something to celebrate, if flaws occur then its a flaw in the team's baby.

But when it comes to collective ownership of the entire code base I see several problems:

I think some degree of collective ownership is a good thing. It is worth having an under-study for each developer^[3] - if only so we can take holidays. - spreading knowledge round is a good thing but I think the idea that every developer is responsible for every line of code is, erh, too extreme.

This is one of the key points of XP which limits its scalability. If every developer must be able to maintain every line the maximum number of lines in the project is equal to the number of lines the least able developer can handle.

Lawyers have been trying to write precise English that covers all situations for a few hundred years and yet we still have arguments in the courts about what the law means. Actually having a real live customer who you can just ask things of is a real boon. However, this requires a change in how the company operates and bills. If every time the customer tells you something you have to raise a change request you will not get anywhere. You must also be able to have a frank discussion with a customer. I remember one project where after a discussion with the "architect" I went talk to an onsite customer about a problem but had to ask questions so that they did not become aware that we had uncovered a major deficiency.

If companies really want to use an on-site customer, they often have to change their way of thinking and billing. This may be impossible unless the developers and customers actually work for the same company.

Most of my professional career has been spent in a production state. I have no problems with this rule, it imposes a certain discipline and is not without it's draw-backs. For example, you may not get the time to perform a major refactoring, or, to explore a new technology that is potentially useful^[4].

Sometimes this is not possible. I recently worked on a system where we had six months to prepare for a new software release. Both our software and the clients went live on the same day, releasing ours before theirs was pointless, but to release later carried a very real prospect that customer would use a competitor.

Beck advocates a strict 40-hour week^[5]: Developers are, to some degree, brought up in a culture of long hours, all night hacks, and racing deadlines. Sometimes, these are necessary, sometimes, just sometimes they can make a difference. Overall though, doing much more than forty hours a week on a regular basis takes its toll. On the subject of holidays he is right again, a break for a week or two can do wonders for the creative process. Just because I am away from the code-face does not mean my brain is diminishing, if anything, I've used these opportunities to look at my problems in a different light and seek inspiration.

Beck talks of the importance of a team which fits together, he is right about this, a team which thinks alike, shares a common vision and respects one another will be an order of magnitude more productive than a team of individuals - the whole being more than the sum of the individuals and all that. However, Beck seems too quick to fire people from the team, his method of getting a team to gel is to fire the people who do not fit. This is the wrong way of approaching the problem, albeit, it is extreme. Not only does much employment law prevent this it is not the right way to treat people. Sometimes it is the only way to deal with a situation, but generally, if you go round firing people who do not fit your practising management by terror. Beck does not talk about recruitment, surely it is better to hire people who fit in? Have the whole team involved in the interview process, is everyone happy with hiring this person?

Beck recognises that best projects are all seated physically close together, and sometimes the best way to achieve this is physically move the desks yourself. However, in the bigger companies I have worked for this just is not on, period. Yes, companies with this attitude may deserve to fail.

Beck also advocates developers taking responsibility for the process: ownership of the process is an important part of owning the final product. But this is not without problems.

Some organisations have a process, and they can be very attached to the process, especially if confers ISO900X or CMM level N status to the organisation. Secondly, this could lead to navel gazing. Is the program broken or the process?

Small companies can usually be more flexible with this kind of thing while bigger companies less flexible, after all: you may not be the only development team in the organisation and what works for one may not work for another, large companies frequently prefer lowest-common-denominator approaches. Changing the process can bring you into all sorts of office politics that potentially kill the project before you start.

Finally here, I agree on his view on toys and food but I do not think you can legislate for them.

Moving the furniture, owning the process, playing with toys are all part of binding a team building and empowering. The shared vision / shared goals concepts which run through Beck and McCarthy's work is where I believe the silver bullet is be found.