Title: Professionalism in Programming #12

Booch describes OO programming as "a method of implementation in which programs are organised as co-operative collections of objects, each which represents an instance of some class, and whose classes are all members of a hierarchy of classes united via inheritance relationships." [Booch]. It is another imperative methodology.

This is very much a data-centred model. We think about the life of the data and how it moves about, rather than the sequence of steps that need to be taken to get the job done. 'Objects' (the data) have behaviour (they do things) and state (which changes when they do things). This is implemented at language level by methods on classes of objects.

OO programming started with Simula around 1970, and is recently popularised with C++ and Java. One of the few 'pure' OO programming languages is Smalltalk. These days there are many other OO languages. A number are structured languages with fashionable OO 'bolt-ons'.

This is a declarative programming methodology based on typed lambda-calculus, a more mathematical model of programming. You work with values, functions and functional forms. Functional programs are generally compact and elegant, although seldom compiled. They are therefore reliant on a language executor. The program's performance is governed by this executor - they can be quite slow and memory-hungry^[2].

The structured and OO methodologies are far more popular in mainstream use than any declarative languages, although that doesn't diminish the usefulness of this breed of programming. They have different strong points and uses.

Common functional programming languages are Lisp (although it does contain non-functional elements), Scheme, ML, and Haskell.

This is another declarative methodology, in which you provide the executor with a set of axioms (rules) and a goal statement. A set of built-in inference rules (over which the programmer has no control) are applied to determine whether the axioms are sufficient to ensure the truth of the goal statement. The program execution is essentially the proof of the goal statement.

The best known logic programming language is Prolog.

It's a starting point, but really an anti-process. Here there is no process, or it is undocumented. Everybody works to their own agenda, and hopefully something useful will drop out the end.

If an organisation doesn't know how it builds software then it's in an unforgivable state. Even if it's a small organisation who think theydo not need it. There is no guarantee the software will be delivered on time, since there's no accountability. Who can guarantee that all the features will be implemented?

A lot of 'open source' software is created using this chaotic method^[3]. If you have an infinite number of monkeys and an infinite number of computers you might eventually get a program out - however it isn't feasible to wait the requisite infinite amount of time. Even back of napkin designs are a step towards a more formal, predictable development process.

This case is anarchy. Individual employees may work hard and may eventually produce something of value, however this outcome cannot be seriously relied upon. The team is likely to be very inefficient, and will probably never deliver anything of value.

The 'waterfall' is the classic software development lifecycle. It is much criticised for its simplicity (or even for being old fashioned), however practically every other development process is in some way based on it. It is modelled after a more conventional engineering life cycle, and was described by W.W. Royce in 1970 [Royce].

It is a simple idea; the development process is broken up into a number of stages, which run one after the other. This is likened to a waterfall because of the steady irreversible flow from one stage to another. Just as water always flows downward toward the river, the development always flows downward through the stages toward release.

This is the traditional waterfall illustration:

You can see the five standard stages. These are described in the Stages of development sidebar. Once a stage is successfully completed progression is made via some "gating process" (usually a review meeting) to the next stage. The output of most stages is a document; a requirements specification, a design specification, or some such. If the review finds an error, it is generally fed back up-stream setting that stage back again.

Following this model you can't easily backtrack to make changes; its like a salmon expending massive amounts of time and energy swimming back upstream. This means that the process is not helpful when changes are made late in the development process. The requirements must be fixed before system design, and it is difficult to accommodate too many alterations after the process is underway. Generally problems at the design stage are not discovered until system testing.

In its favour, though, it is simple to manage and is the basis for most other development models.

Although this sounds like development only partaken by consenting adults, it actually stands for Structured Systems Analysis and Design Methodology. It is a structured and rigorous method following the waterfall approach. It covers analysis and design, not implementation and testing. It is a well defined "open" standard, heavily used in UK government organisations. SSADM consists of five main steps (each subdivided into many other procedures), which for our purposes are self-descriptive: (i) Feasibility study, (ii) Requirements analysis, (iii) Requirements specification, (iv) Logical system specification, and (v) Physical design.

Despite there being many years of research and experience in software development processes, the waterfall is still a standard reference model since it has a clear logic to it - it is not right to perform implementation before requirements analysis or any design work. However, with the waterfall it has been hard to evaluate the entire system until the waterfall is complete.

It is also hard to show what is being developed to a customer until the integration phase has completed and the system is in a ready state. The prototyping approach attempts to work around this limitation. It helps to explore and evaluate implementation as development progresses, and to refine unknown or ambiguous requirements.

The essence of the prototyping process is to create a number of (possibly throw-away) prototypes of the system under development. Each prototype is evaluated, shown to the customer, and the feedback is used to shape the next prototype. This continues until enough is known to develop and deploy the 'real' product.

We see an analogy with other industries here. If you were developing a new car you'd create many prototypes until you hit on exactly the right design. We aim to do the same with our software. However, there is an important difference that must be observed. When building a car, the major cost is in the production, not the development. It works the other way around with software. You can make multiple copies of the code for free, the development is the costly part. For this reason the prototyping cycle needs to be controlled, it can't be repeated an unlimited number of times.

The prototypes are developed quickly in very high-level languages. The use of automated tool support can speed prototype production immensely. The prototypes are proofs of concept, so efficiency, stability, or a complete feature set are not primary concerns. For this reason, prototyping works best for systems with an emphasis on the user interface.

The danger with prototyping is the temptation to release the inefficient, quickly developed, not fully thought out prototype code. This is especially true when a project is running out of time and the 'real' development might not get done in time. It needs very careful management to work.

All the recent 'advances' on the waterfall approach are broadly variations on a theme. The improvement is performing development in an 'iterative and incremental' manner. That is many trips (iterations) round a small development lifecycle run back to back (incrementally), adding more and more functionality to the system until it is complete. Each single run of a 'mini lifecycle' tends to follow the waterfall model. Each of the phases of the waterfall therefore gets executed more than once. At each iteration end is a 'software release'.

Incremental development is neither a top-down nor bottom-up approach. During each iteration of the process the system grows and subsequent design work can be done based on the proof of validity of the existing design and implementation. There is a parallel to prototyping here, but we're not so focused on quick hacks that work as 'proof of concepts'. With this approach each stage is less complex and easier to manage - and process progress is more easily monitored; you know how much of the system is built and integrated.

This kind of process works better for projects whose requirements are less understood at the start - it is more resilient to change and saves lengthy re-design and re-implementation of the entire system that you'd encounter in the waterfall approach.

The spiral model, proposed by Barry Boehm in 1988 [Boehm], is an example of this. The development process is modelled as a spiral. It starts in the centre and fans outwards towards the later stages of

he process. We start working on a very rough notion of the system, becoming more detailed over time as we enter later stages of the spiral. Each 360 degree turn of the spiral sees us go through a single 'waterfall'.

Features are defined and implemented in order of decreasing priority; the most important facilities are created as soon as possible. This is a way of managing risk, it is safer because as you inch nearer to the ship date you can be sure that the majority of the system is complete. In fact, it is very pragmatic common sense approach. Engineers will not be spending 80% of their time on the trifling (but fun) 20% of the system.

Boehm's spiral is split into four quadrants which describe four distinct phases: (i) objective setting (where specific objectives for this phase are identified), (ii) risk assessment and reduction (the key risks are identified, analysed and information is sought to reduce these risks), (iii) development and validation (appropriate model is chosen for next phase of development), and (iv) planning (the project is reviewed and plans drawn up for the next round of the spiral).

There are a great many other processes we could look at; they are all very similar to those above, and yet distinct. There are several modified waterfalls (for example with overlapping phases, or waterfalls with subprojects, each managed as waterfalls). There is the evolutionary prototyping approach where you start with an initial concept, design and implement an initial prototype, then iteratively refine the prototype until it is acceptable, complete and release this, perhaps planning to include some throw away prototypes in the process.

In staged delivery development we follow a sequential process up to the architectural design, and then implement the separate components showing them to the customer as each is completed, going back to previous development steps if needed. Evolutionary delivery is essentially a cross between evolutionary prototyping and staged delivery.

Rapid Application Development (RAD) emphasises user involvement, small development teams, and makes heavy use of prototyping and automated tools. In a slight twist on other processes the development time frame is established up front and considered immovable. Then as many features as feasible are incorporated into the design to accommodate the deadline - some features may be sacrificed.

The much-hyped extreme Programming is like some of above, with emphasis on developing tests up front, remaining flexible and nimble, and focusing on the customer. There is a lot of wisdom in placing such emphasis on testing. A lot of its other maxims receive more debate.

If you've read this far and not got bored yet then you're doing well. Finally, and perhaps more importantly, what are the key points to draw out from all this? The 'professional' software developer should have a working understanding of processes and methodologies, but anyone can get this from the right books. How do we apply this to our work to the best of our abilities?

With all these processes there are some common threads that we should observe. The phases described in the Stages of development sidebar are present in each. The processes really only differ in the length and relative positioning of the stages. Each of these activities is vital to the production of good quality software. The better processes ensure that testing is not left as an afterthought at the end of development, but it carried out continuously and monitored throughout the development process.

It's hard to compare or evaluate these processes and methodologies. Which is best? Which will ensure a high quality product is shipped on time and to budget? There is no answer, because those are not the right questions. Which process is suitable depends on the project and the culture of your company. If you have 20 programmers who know nothing of object oriented development and only know C, trying to build an OO C++ product is clearly a stupid idea.

We can see two extremes, the anarchy of the ad hoc method against a strict dictatorship of a rigid process. In the latter any experimentation that could yield a more elegant architecture is discouraged. The user's real requirements may never filter down to a developer since it's lost in a wall of bureaucracy, the programmer just codes to a spec that's passed on to him.

The most flexible approach is somewhere in between. You do need to know the process you're working to and where it's defined. For effective development it is necessary to be ordered and have a process. However, the experienced programmer knows the value of the process, and its faults too. They know how to work with it, and when to step outside it. Good programmers don't just program. They understand their recipes and how to adapt them as appropriate. This is where the science is still part art.

New methodologies spring up (or rather evolve) from time to time. They tend to arrive with a big fanfare and a spurt of fireworks, and they are claimed to be the silver bullet, the panacea that will make development better for our children and our children's children. Sadly, it's never the case. When it comes down to it, no matter what lifecycle you follow, the programming team is only as good as its programmers. If there is no intuition, no class, no experience and no drive present, then no matter what the lifecycle is you won't reliably produce good code. You might be better able to track how far behind schedule you are, though.