It's good to talk...

Consider a typical work day - what interactions do you have? Unless you only ever write software on your own for your own needs exclusively, at some point you will talk to someone. But what sort of interactions are they? In other words, who, what, how, and how often?

For example, here are some of my own recent experiences. First thing on Monday morning I chat with the person at the next desk about what we did over the weekend, which morphs into a discussion about a bug we were tackling at the end of the previous week. After a few code experiments, we update bugzilla with what we have learned. I use skype to check with the support team to find out how important a fix is to the customer, and then talk with the project manager about the approaches we could use and the risks involved. As a treat, I have a quick read of accu-general and the newsgroups, which gets an answer to a tricky template question. After a pub lunch with some new starters, I attend the daily stand-up team meeting to see what everyone is up to, and update the white board with the release schedule on it, after which I update the current branch diagram and publish to a remote monitor so that everyone can see it. While getting a coffee I bump into a firmware developer and check on a detail of a system we've just finished that had been queried. Then it's a scheduled code review, where I go though my comments with the recipient, then email the details to them. After checking my assigned bug list, I study some logs from a customer's machine and fix the bug, commenting in code why there had been a problem. I get a quick review of the change before checking it in and waiting for build server to mail everyone the build report. To round off the day there's a big company meeting involving Seattle ringing in and Palo Alto on videophone to round up what's happening in sales, marketing, engineering, finance and HR.

The range of interactions can be remarkably diverse.

The frequency can go from constantly during an intensive pair programming session, once a day at a team meeting, once a week at a department or company meeting, every few months when people visit from another office, to a one off client meeting.

The media range from a simple chat, a 'second pair of eyes' check of code, a formally arranged meeting, email, newsgroups, blogs, phone call, instant messaging, video conferencing, presentations, bug reports, debug logs, design documents, whiteboards, code comments, check in reports.

The type of person varies too: programmers, architects, project managers, product managers, customer support, customers, external contractors, sales, marketing. The immediate usefulness of the information they have will vary too, so the initial level of detail will often mirror the likelihood of it being of interest, with the option of going into further detail if needed.

And there's a time lag aspect too - a real time conversation is immediate; email has a noticable roundtrip delay; a comment in code is often communicating with a very important person - myself in six months time when I've forgotten the details; and a debug log is a result of the development team sending 'help' messages in a bottle to their future selves trying to diagnose problems.

There's a lot of talk going on.

I think this is for several reasons: almost always writing software is doing something new (at least to you!), which requires generating ideas, followed by implementing, checking, and backtracking as you try them out; you have to find out what to do in the first place, and checking you're still doing the right thing as you learn; and letting people know how things are going so that problems can be spotted and plans adjusted in a timely manner.

Thus developing software is not so much a technical problem as a communication one. To be succesful, all of it has to work well - you can have the best development team in the world, deliver above spec, early, and under budget, and yet it never gets used because despite it doing what was asked for, that turned out to be not useful to the customer in practice.

I'll go further - the best teams are the ones who can move information such that the right level of detail gets to the right people at the right time. This is a tough challenge: collecting the information, knowing who needs it, in what form, and correctly prioritised. One of the real difficulties here is that the answers to these questions will be different for the sender and the recipient. Think of it as an economic problem - how to organise this information-processing system so that the maximum value is extracted.

But things aren't hopeless - we've been solving these sorts of organisation problems since humans got together into groups, so we must have learnt something along the way. Two key ideas are already implicit in my commentary above - specialisation, and teams.

A good all-rounder is very useful, and essential for a successful small company, but if you gather a group of people all of whom are experts in their own area, they can be much more productive than the same number of all-rounders. The downside is that a group of people now have to cooperate to achieve a goal, and that requires coordination of aims and delivery, and that requires communication between the members. But as a group grows, you end up with a combinatorial problem - if everyone talks to everyone else, the number of possible communication channels explodes, so that you end up all your time talking to people and not actually doing anything.

Some solutions are fairly natural: Organise people so they talk to the people they need to easily, and it's harder to talk to the others. The inputs are from looking at the value generated from all possible interactions, and finding which are most beneficial. This can usually be done quite efficiently on an ad hoc basis, but a formal analysis could be an interesting exercise. (I'm reminded of the book Notes On the Synthesis Of Form [Alexander] which gives the mathematics of finding tractible design sub-problems given a network of dependencies. This looks analogous to finding the communication sub-networks of most value.)

For example, I can think of two very different ways that can work: organise similar roles into teams then have representatives do the inter-team communication; or do a vertical slice, where a team has representatives from each role and they work as an almost self-contained unit. Which is more appropriate depends on your situation, and can even be mixed.

Great, we've worked out our team structure, where shall we put them? Think carefully, and compare your answer to your current structure and locations.

I think a good rule of thumb is 'physical proximity should reflect desired communication intensity'. In English: be near the people you need to talk to most. This is similar to Conway's Law [Conway], in that the resulting structure will reflect the relative communication channels, and that is dominated by physical location.

As Churchill remarked, 'First we shape our buildings and afterwards our buildings shape us' [Churchill43]. This was in the context of rebuilding the Houses of Parliament, which had been damaged in a bombing raid. He undestood that the government/opposition design of the chamber and its inability to fit everyone in at once actively shaped the political process, and should be retained.

New technologies warp this a little: from the telegraph to the phone to an instant message chat, we can interact remotely, but it is a different sort of communication to a face to face conversation. An example: in two minutes talking to someone with a customer support issue who'd popped in to talk to someone else, we cleared up more than the previous hour of back and forth email.

Ah, a serendipitous ad hoc meeting! A strikingly effective communication channel for those rare but valuable times you need to talk outside of your normal day-to-day interactions. But these 'chance' meetings need some way of being allowed to happen. A classic solution is a kitchen area with (a good) coffee machine, water coolers, comfy seats, and whiteboards. Steve Jobs apparently understood this, and put the Pixar bathrooms in what initially seemed an awkward location in a central atrium so everyone could bump into each other [Bird]. Stuart Brand also discusses building and office design to make these things happen, and how bad buildings prevent them [Brand94].

All these dynamics need to be understood in a successful organisation and acted upon - the organisation should reflect the desired communication of information value, and physical locations chosen to encourage it. And it should be reviewed and adjusted as the world changes around you - nothing stays the same.

Communication changes

And there are indeed changes happening in how we communicate with you. Up till now CVu and Overload have been produced every two months, and sent to you in one mailing. But from now on they will alternate and you will be sent one magazine each month - the next Overload will be delivered in October as normal, but will be followed by CVu in November, then Overload in December etc. While seemingly a small change, this will mean you'll get a magazine twice as often, but without the information overload [sic] of two magazines to read all at once.

References

[Alexander] Alexander, Christopher (1974) Notes on the Synthesis of Form, Harvard University Press: USA (ISBN 0-674-62751-2).

[Bird] http://gigaom.com/2008/04/17/pixars-brad-bird-on-fostering-innovation/Lesson Six: Steve Jobs Says 'Interaction = Innovation'

[Brand94] Brand, Stewart (1995) How Buildings Learn: What Happens After They're Built, Penguin Books (ISBN 978-0140139969).

[Churchill43] House of Commons (meeting in the House of Lords), 28 October 1943, http://www.winstonchurchill.org/i4a/pages/index.cfm?pageid=388#Shape_our_Buildings

[Conway] 'Any piece of software reflects the organizational structure that produced it.'
It was actually an observation on how committees work: http://www.melconway.com/research/committees.html
And there's some empirical evidence for it: http://www.hbs.edu/research/pdf/08-039.pdf