Title: I_mean_something_to_somebody, Part Two

Words are used both to communicate with other people and for internal thought processes. The culture we are born into provides us with a predefined set of words and a network of meanings associated with them. The use of words in their spoken form to communicate with other people has a cost that speakers attempt to minimise by using them in a way that is consistent with the meaning they believe their listeners will assign to them. A lifetime of realtime feedback from the people spoken to enables users of a language to build a detailed collection of beliefs on the meanings assigned to words by both people in general and some specialist groups of people (e.g., software engineers).

When speaking it is expected that not only will listeners make an effort to comprehend the speakers' thought processes, but that speakers will make an effort to ensure that what they are saying is comprehensible to their listeners. When writing text people must make use of their experience with the spoken form to help ensure that readers will assign a meaning to the words that is consistent with that intended. However, there is no realtime feedback between writer and reader^[1] and experience shows that readers often have to invest significantly more effort to assign a coherent meaning to what they read, compared to the effort needed while listening during a spoken conversation.

Software developers are not usually told which identifiers they should use in a given context and are rarely given rules for creating new identifier names from existing ones^[2].

Experience shows that many developers believe that the names they select for identifiers are 'obvious', 'self-evident', or 'natural'. Studies of people's performance in creating names for objects suggests that this belief is false [Carroll, Furnas-1983, Furnas-1987]. When asked to provide names for various kinds of entities people have been found to select a wide variety of different names, showing that there is nothing 'obvious' about the choice of a name.

One naming study [4, 5] described operations (e.g., hypothetical text editing commands, categories in 'Swap 'n Sale' classified ads, keywords for recipes) to subjects, who were not domain experts, and asked them to suggest a name for each operation. The results showed that the name selected by one subject was, on average, different from the name selected by 80-90% of the other subjects (one experiment included subjects who were domain experts and the results for those subjects were also consistent with this performance). The number of occurrences of different names chosen tended to follow an inverse law with a few words occurring frequently and most only rarely.

Various factors have been found to influence the selection of what is believed to be the appropriate word in a given context. A study by Labov [Labov] showed subjects pictures of individual items that could be classified as either cups or bowls, as shown in Figure 1. These items were presented in one of two contexts; a neutral context in which the pictures were simply presented and a food context (subjects were asked to think of the items as being filled with mashed potatoes).

Figure 1. Cup and bowl like objects of various widths (ratios 1.2, 1.5, 1.9, and 2.5) and heights (ratios 1.2, 1.5, 1.9, and 2.4). From Labov [Labov].

Figure 2. The percentage of subjects who selected the term 'cup' or 'bowl' to describe the object they were shown (the paper did not explain why the figures do not sum to 100%). From Labov [Labov].

The results (Figure 2) showed that as the width of the item seen was increased, an increasing number of subjects classified it as a bowl. By introducing a food context subject responses are shifted towards classifying the item as a bowl at narrower widths.

Human languages have a relatively fixed set of letter sequences that are acknowledge by speakers of a language as being 'root words' (this glosses over the heated discussions that sometimes occur over what letter sequences should be treated as root words). Additional words can be derived from these words using language specific rules (e.g., write -> writes, writing, written; writer could be treated as either a derived or a root word).

Identifiers sometimes contain more than one word. In this case readers need to either use their knowledge of existing words to subdivide an identifier's character sequences, or use deduction based on common naming conventions to extract words (e.g., IsHot is likely to be interpreted as the phrase 'is hot', rather than 'I shot').

Identifiers often have the form of one or more abbreviated words. A study by Ehrenreich and Porcu [Ehrenreich-] found that readers' performance in reconstructing the original word, from an abbreviated form, was significantly better when they knew the rules used to create the abbreviation (81-92% correct), compared to when the abbreviation rules were not known (at best 62% after six exposures to the letter sequences). Given that this experiment was not intended to measure subjects' abbreviation to word reconstruction performance, no rarely occurring abbreviations were used.

While there have been no other published studies of how people assign a meaning to identifiers there have been a few studies of a similar nature for words.

A study by Nickerson and Cartwright [Nickerson-] asked subjects to write down as many different meanings of a word (presented one at a time, in written form, for 30 seconds). Combining the results from all subjects showed that words were often given over 6 and sometimes as many as 20 different meanings. The majority of the responses for a given word were usually contained within one or two meanings.

Word association is an activity that has some similarities to providing a meaning for a word. Studies of word association give subjects a word and ask them to write down the first meaningfully related word that comes to mind. (e.g., doctor ? nurse).

The results of these studies^[3] have found that there is rarely a single answer, a wide range of responses is given, and words given by subjects do not always overlap those of other subjects.

A subject's age has also been found to be a factor in word association performance. A study by Hirsh and Tree [Hirsh-] compared the responses of young (21-30) and older (66-81) adults to 90 stimulus words. The results showed that the same word was produced as the most popular response, for a given age group, in 36 out of 90 cases (when the top three responses were considered the overlap between groups was 57%). They also found that the younger group produced a wider range of responses, and that members of the older group were much more likely to select the most popular response for their group (40%, against 20% for the younger group).

The experiment was performed during two 30 minute sessions on different days of the 2003 ACCU conference held in Oxford, UK. Subjects were given a brief introduction to the experiment, during which they filled out background information about themselves. They then spent 15 minutes working on the identifier list. All subjects volunteered their time and were anonymous.

The first part of this paper describes the background of the subjects and how this information was collected.

Almost any sequence of characters could serve as an identifier. However, the initial list of identifiers considered for use in the experiment were obtained by extracting all identifiers that were common to the source code of a variety of programs. These programs were the Linux kernel, the game Doom, gcc (the GNU compiler collection), Netscape internet browser, PostgresSQL database, AFS (Advanced File System) from IBM, and OpenMotif from the OpenGroup. It was hoped that usage in a wide variety of programs was an indication that an identifier had a significant meaning to a large number of developers. This method also removes experimenter bias from the choice of identifier names (but not from the choice of programs to consider).

The initial list was refined by removing those identifiers that were the names of standard library functions (these might be recognized as such and their library meaning given as a response), or contained rarely occurring abbreviations, or contained a single character. The resulting list of identifiers was randomized and printed one per line on A4 sheets of paper.

All subjects from both groups saw an identical list of identifiers. However, one group was told that the identifiers came from a multiplayer game, while the other that they came from the Linux kernel. The instructions given were:

There are a number of reasons why the responses given in this experiment might not be valid in a source code comprehension context. These include:

A few of the subjects had a different cultural background from the majority of the subjects (i.e., they were not British). It is possible that these subjects made use of different cultural conventions when assigning meaning to identifiers. For instance, in the US politicians run for office, while in Spain and France they walk, and in Britain they stand for office.

It is possible that on the first day I failed to point out during the introduction that the identifiers were extracted from a multiplayer game (I did point out that the identifiers came from the Linux kernel on the second day). This information is given in the instructions, but it is possible that subjects did not read the sentence containing this information.

The 45 subjects produced a total of 1662 responses (34.8% Linux, 65.2% game), and 74 different words were responded to. There were 179 responses (45 different words) where the subject had written "none" (or a question mark, or a dash). The identifiers were printed on both sides of the page and some subjects only gave responses for identifiers appearing on the odd numbered pages. In this case the identifiers appearing on the even numbered pages were not counted as "none". See Table 1 on the next page for a summary of responses.

Each subject's response for each identifier needed to be classified. The following process was intended to ensure that the person doing the classification (your author) was not influenced by information about the subject who gave the response. (i.e., whether the subject belonged to the Linux or games group, and which responses were given by the same subject). Every response was automatically assigned a random number and the resulting list of identifier/response pairs was sorted. This list of randomised responses was the one used for classification.

Certain words and phrases occurred several times in the responses and were assumed to imply a game context, but not a Linux context. These included: player, game, skill level, and shoot. While some words appear to have an obvious games meaning (i.e., kill), if it was possible that they also had a Linux meaning they were not classified as being games related.

Words and phrases that might be claimed to be a strong indication of a Linux context (e.g., Linux, operating system) rarely occurred in the responses. Much of the functionality provided by an operating system (e.g., Linux) might reasonably also be expected to be provided internally within a game. For instance, virtual memory refers to a memory management mechanism used by both operating systems and games (which, for efficiency reasons, might swap unneeded game information out of fast memory). This overlap in functionality, which many subjects are likely to be aware of, makes it difficult to reliably classify any responses as belonging to a Linux context.

A games context was assigned to 134 responses (12.4% of responses made by games subjects) scattered over 33 different words. A Linux context was assigned to 10 responses (1.2% of responses made by Linux subjects) scattered over 6 different words.

The forms of the meanings given were such that it was rarely possible to definitely specify which group a response belonged to. For instance, for the identifier blue_pos many subjects gave a response of the form position of some blue thing. In itself this response is not sufficient to be able to assign a Linux or game context. Additional information such as index into array could apply in either context, while use of the word player would suggest a game context.

In many cases the responses described a possible role that the identifier might fill, e.g., flag, or counter, while in other cases subjects simply expanded an identifier to a non-abbreviated form, e.g., gave page number as the response to pagenum.

The responses contained fewer different meanings per identifier than the Nickerson and Cartwright [9] study. However, this experiment did not explicitly request subjects to list all possible meanings of an identifier.

This study set out to investigate the extent to which knowledge of the applicable application domain affected the meaning assigned to identifier names. A single experiment was performed, resulting in a single data point. More measurements, based on responses for other identifiers and application domains, are needed before it is possible to draw any general conclusions about the interaction between developer knowledge of the application domain and the meaning assigned to identifiers.

However, the 12.4% of game subject responses having a game context is significantly less than 100%. Some of the possible reasons for this include:

A "flag" meaning was given in a surprising number of responses. This may represent a default response, given when subjects could not think of anything else to write, or perhaps the identifier names used in this experiment often have this meaning in source code.

The responses generally involved concepts encountered in software engineering.

As the first of its kind the results of this experiment encountered a number of problems:

In many ways identifiers are metaphors. For a fascinating introduction to metaphors in English see: "Metaphors we live by" by G. Lakoff and M. Johnson.

For an interesting, and readable, discussion of people's performance in answering questions they do not know the answer to see: "Simple heuristics that make us smart" by G. Gigerenzer and P. M. Todd.

The University of South Florida word association norms can be downloaded from: http://w3.usf.edu/FreeAssociation

The responses given in the experiment (stripped of subject background information) can be downloaded from: http://www.knosof.co.uk/cbook/accu2003.html

The author wishes to thank everybody who volunteered their time to take part in the experiment and the ACCU conference organizers for making conference slots available to run it.