Human-computer interaction has spent four decades expanding its account of what happens between people and machines. Artificial intelligence is now making the field's oldest uncertainty harder to ignore. What, exactly, is an interaction?
The question sounds like the kind of thing a mature field should have settled long ago. It has not, and the reason is not negligence. The field advanced precisely by refusing to settle it, admitting more of the phenomenon each time its existing account proved too small. This essay traces that expansion, shows why current AI systems press on its unfinished edge, and explains why the problem is not one more methodology can solve. It ends where a research program begins.
A machine between information and understanding
In July 1945, as the Second World War approached its end, Vannevar Bush published an essay in The Atlantic about a problem victory would not solve. Scientists had become extraordinarily capable of producing knowledge and much less capable of finding their way through what they had produced. Research accumulated faster than any person could absorb it. Specialization made discoveries possible while making their relationship to other discoveries harder to see. The danger, Bush believed, was not simply that information would be lost. It was that the structure of knowledge would become inaccessible beneath the volume of its records.
Bush imagined a machine that might help. He called it the memex, a desk with translucent screens and a large store of documents on microfilm. A person using it could retrieve a book, add an annotation, or connect one document to another. The most important feature was not storage but the associative trail: a durable path through several records that followed the movement of an idea rather than the categories of an index. The memex would not think in place of its owner. It would preserve enough of the owner's intellectual movement that thought could resume where memory failed.
The machine was never built. Its mechanisms belonged to an era of microfilm and photocells that digital computing would soon overtake. But Bush had made a shift that outlived the hardware. He was no longer treating machinery only as a means of calculation. The machine had entered the space between information and understanding, and once it was there, someone would have to describe what it was doing.
Augmentation, and the event that resisted description
Douglas Engelbart encountered Bush's essay while serving as a young naval radar technician in the Philippines. By the late 1950s he was pursuing a question more ambitious than the design of a convenient computer. He wanted to know whether machines could improve the collective ability of human beings to confront difficult problems. At the Stanford Research Institute he built a laboratory around what he called the augmentation of human intellect, joining software, organizational method, and new forms of collaboration, because he did not believe any of them could be redesigned in isolation.
On December 9, 1968, he demonstrated the result before an audience in San Francisco. Seated at a console, Engelbart moved through linked documents, edited text, reorganized an outline, and worked with colleagues connected from Menlo Park. A small wooden device controlled a pointer on the screen. The demonstration is remembered now as a compressed preview of personal computing, but Engelbart's purpose was larger. He was not presenting a collection of interface inventions. He was presenting a system for reorganizing intellectual work.
That ambition complicated the meaning of the machine. If a computer merely calculated a predetermined result, its operation could be understood through engineering and mathematics. Engelbart's system did something harder to isolate. A person acted, the computer responded, and the person interpreted that response before deciding what to do next. The intention that entered the system might not be the intention that emerged from it. Feedback altered the task while the task was being performed.
This was a different kind of computational event from the batch processing that still defined most computing. In a batch system a programmer prepared instructions and submitted them, and the delay between action and consequence could last minutes or hours. Interactive computing shortened that interval until the consequence of one action could influence the next. The change was temporal, but its effects were cognitive. People no longer needed to determine every step in advance. They could work through a problem by acting within it.
The first attempt to make interaction measurable
By the 1970s, Xerox's Palo Alto Research Center had become one of the places where this new relationship was being made visible. The Alto placed a graphical display, keyboard, and mouse at a single workstation. Researchers built editors and overlapping windows and connected computers through a local network. The technical achievements were substantial, and they produced questions that did not fit comfortably inside computer science as it was then constituted. How quickly could a person select a target? How much could be held in memory while navigating a command structure? Why did one editing method feel harder than another when both produced the same result?
Stuart Card arrived at PARC with training in psychology. Thomas Moran had studied cognitive psychology and linguistics. Allen Newell, at Carnegie Mellon, had already helped establish artificial intelligence and cognitive science. Together they tried to describe skilled computer use with enough precision that an interface could be compared before every alternative had been built and tested.
Their 1983 book, The Psychology of Human-Computer Interaction, treated performance as a sequence of perceptual, cognitive, and motor operations. The Model Human Processor offered a simplified account of the systems involved. GOMS described tasks through goals, operators, methods, and the rules by which a person selected among them. A researcher could represent competing methods, estimate their demands, and predict which would be faster. The work did not claim to hold the full meaning of computer use. It established something narrower and more important: that some part of interaction could be formally described rather than left to intuition.
The models fit the systems around which they were built. Many interactions consisted of selecting commands, entering text, moving a pointer, and navigating stable structures. A skilled participant had a goal, the software made certain operations available, and expertise meant learning an efficient route through them. Under those conditions the duration of a keystroke mattered, because small differences accumulated across repeated work.
This was also when the field acquired an institutional identity. In 1982 the Association for Computing Machinery refocused a special interest group and announced the formation of SIGCHI. The following year it co-sponsored the first Conference on Human Factors in Computing Systems. Even the names recorded the field's uncertainty. One placed the computer before the human. The other borrowed the language of human factors, a tradition tied to ergonomics and the effort to fit machinery to human capability.
The field did not emerge because its participants agreed about interaction. It emerged because computing had become impossible to study from a single disciplinary position. Cognitive psychologists measured memory and attention. Engineers studied performance and error. Researchers in artificial intelligence modeled reasoning. Linguists examined commands and dialogue. Ergonomists considered the body at the equipment. Designers attended to form and legibility. Each discipline saw something the others had missed, and SIGCHI gave these approaches a shared home before they shared an object.
The plan that did not determine the action
The disagreement grew more consequential as computers moved out of laboratories and into ordinary organizations. A system that performed well in a controlled test could behave differently in an office. Instructions that looked complete to a designer became insufficient when the person following them was interrupted or hit an exceptional case. The gap was not always poor implementation. Sometimes it revealed that the system's description of human action was too orderly.
Lucy Suchman met this problem at Xerox PARC. Trained as an anthropologist, she studied systems by watching how people used them in real settings. Her best-known example was an office copier with an elaborate help system, designed to guide a person through its operation. In principle a user could identify a goal, follow the prescribed steps, and produce a copy.
People did not proceed that way. They read partial signals from the machine, tried something and watched what followed, spoke to one another, and revised their understanding when the copier behaved unexpectedly. The help system assumed that person and machine shared a complete enough account of the situation for instructions to direct action. The videotapes showed how rarely that shared account existed.
In Plans and Situated Actions, published in 1987, Suchman argued that plans did not determine behavior the way some computational models assumed. People made plans, but the meaning and adequacy of a plan emerged within a particular situation. Action was continually reconstructed through contact with the material and social world. The plan was not a program running inside the person. It was one resource among the circumstances through which the person found a way forward.
The finding changed the meaning of user error. A departure from an expected procedure could no longer be dismissed automatically as noise. It might mean the person had encountered a situation the system did not represent. The machine recognized only the states its designers had anticipated. The person was responding to a larger world.
Cognition outside the head
If action depended on the surrounding situation, then the relevant process could not always be contained within a single individual. Edwin Hutchins reached a similar conclusion from a setting far from office equipment: the navigation team aboard a Navy ship.
Fixing a ship's position required observation and calculation, but no single navigator did the work alone. Crew members took bearings and called out numbers. Others recorded them. Positions were plotted on charts. Instruments preserved some relationships while established procedures carried information from one participant to the next. The ship stayed on course because the organization distributed what had to be perceived, remembered, and computed.
In Cognition in the Wild, published in 1995, Hutchins argued that the navigation team could be understood as a cognitive system. The process extended through people, instruments, spoken reports, written marks, and routines. The chart did not merely display the result of thinking done elsewhere. By preserving spatial relationships in a form that could be inspected, it took part in the thinking.
The computer now appeared in a different role. It was not only an instrument operated by a mind. It could become part of a system through which memory and judgment were distributed. A shared document changed who could contribute and when. A display determined which part of a developing situation could be seen. A notification moved information from one person's attention into another's. Designing software meant reorganizing the cognitive work of the surrounding group.
The distance between a system and its user's understanding
A naval chart and an image editor seem to belong to unrelated activities. Yet both depend on a state that cannot be observed directly in its entirety. The navigator cannot see latitude and longitude beneath the vessel; position must be constructed from bearings, prior locations, and the passage of time. The person editing an image also works through an incomplete presentation of state. A layer may be hidden. A selection may remain active after its boundary is hard to notice. The effect of an operation may depend on a mask or mode that is not visually prominent. In both cases, successful action depends on maintaining a correspondence between what can be seen and what is believed to be true.
Donald Norman spent years on exactly that distance. A cognitive scientist by training, he became a central figure in user-centered design. In The Design of Everyday Things, published in 1988, he began not with a computer but with ordinary objects that made intelligent people feel incompetent. A door that needs pushing but presents a handle that invites pulling. A row of stove controls with no visible correspondence to the burners they govern. Norman argued that these failures belonged to the design, not to the person who hesitated. The object had encouraged an expectation its operation contradicted.
Norman adapted Gibson's concept of affordances to design, and later separated affordances from the perceptible signals that communicate them. He emphasized conceptual models, the accounts people form of how a system works, which need not reproduce the hidden engineering so long as they support accurate enough expectations about what an action will do. This explained how a mechanism could work correctly and still produce repeated mistakes. Failure lived in the relationship between the system's operation and the model its design led a person to build.
The insight put interpretation near the center of usability, and it opened a question layout and feedback could not answer. If people bring models to a system, where did the models come from? Some were learned from earlier technology. The image of a diskette came to mean saving long after the physical object disappeared. Others came from bodily experience, or from culture and institution. The meaning of a red mark depends on whether it appears on a road, a financial statement, or an examination paper. An interface never addressed an abstract processor. It addressed someone who had already learned how objects and signs tend to behave.
Paul Dourish carried this into the era of ubiquitous and embodied computing. By the late 1990s, interaction was leaving the desktop for mobile devices, tangible interfaces, and behavior embedded in rooms and objects. In Where the Action Is, published in 2001, Dourish drew on phenomenology and sociology to argue that meaning arose through embodied participation in a social and material world. Context could not be captured as a set of variables attached to a user. The same message could function as a request, a command, or a joke depending on a relationship the computer did not possess. Meaning did not sit inside the message waiting to be extracted. It was produced through participation.
A field that grew by widening its frame
By the early twenty-first century, human-computer interaction held several accounts of what happens between people and machines. Card, Moran, and Newell had shown that skilled action could be modeled under defined conditions. Suchman had shown that real activity exceeded the execution of prior plans. Hutchins had followed cognition beyond the individual. Norman had made expectation central to design. Dourish had located meaning in embodied practice.
These accounts did not cancel one another. A predictive model stayed useful even though it did not explain a workplace. An ethnographic study revealed social organization without timing a repeated task. The field grew stronger because different conditions required different explanations, and because it resisted defending any one account as complete.
That generosity has a cost, and the cost is the subject of this series. As more phenomena entered the field, the word interaction carried more explanatory weight without acquiring a stable meaning. A researcher could study the usability of a control panel or the politics of an algorithmic platform and place both inside HCI. The breadth was productive. It also made the field's central object hard to isolate.
In practice the uncertainty rarely blocked useful work. A designer could improve a medical device without deciding what interaction was in the abstract. The system and the question supplied a local boundary. The difficulty appears when researchers try to compare interactions across systems. A person may transfer skill from one application to another even when the controls have changed. Two interfaces may look nearly identical yet produce different errors because one conceals a consequential state. A procedure that works for an individual may fail when divided among several people. These differences cannot always be explained by the visible form of the interface, and their similarities cannot be captured by saying that people used computers in both cases.
A scientific comparison needs more than resemblance. It needs a declaration of what is held constant and what may vary. It needs to specify which observations count as evidence, and to preserve the difference between what the system made possible, what the interface presented, what the participant believed, and what the participant did. Mature fields become cumulative when they can define objects and measurements clearly enough that findings from different studies can be compared. That is the capacity HCI has never fully built for interaction itself, not from failure, but because the object kept growing.
Why artificial intelligence forces the question
Artificial intelligence makes the absence visible, because it destabilizes the artifact around which earlier analysis was organized.
A conventional graphical application offers a designed set of possible actions. Its menus may be complex, but their structure was specified in advance. Its behavior may depend on internal state, yet the relationship between actions and outcomes stays bounded by code that designers can, in principle, inspect. A person may misunderstand the system, but the system does not improvise a new response from a learned statistical model.
A language model changes the arrangement. The participant no longer selects among predefined commands. A request may be expressed in countless forms. The system generates a response that was not written in advance and may differ when the request is repeated. The participant learns what the system can do by trying to do it, and each answer becomes evidence from which the person builds a provisional account of the machine.
The interface has not disappeared. There is still a text field and a visible response. But the surface no longer provides a reliable inventory of the system's possibilities. Its apparent simplicity conceals a large and changing space of behavior, and the person must infer capability from interaction itself.
This complicates every account the field spent decades building. It complicates Norman's conceptual model, because what model should a person form of a system whose responses vary and whose internal representations resist ordinary explanation? It complicates Suchman's situated action, because the machine now responds to language in ways that look locally appropriate while remaining insensitive to much of the situation that gives the language meaning. It complicates Hutchins's distributed cognition, because generated output may enter an organization's reasoning before anyone has established where its claims came from.
The old distinctions survive, and their separation becomes essential. A model may be able to produce an answer without being authorized to provide it. An interface may expose a possibility the system cannot execute reliably. A person may believe an action is available because the system signaled it, then discover the data or the authority is absent. An action may execute without achieving its intended outcome. A trace may show what happened without establishing why the participant chose it.
Ordinary product language collapses these differences. A feature is called available when it is merely visible. An action is called successful because it executed without an error. A behavioral trace is read as evidence of intent. A generated answer is treated as a measurement of knowledge. Each compression is convenient until a comparison or a failure requires the distinctions to be recovered.
Why this is not a job for another methodology
The natural response, inside a design field, is to reach for a better method. A new evaluation technique, a richer set of heuristics, a sharper way to run a usability study. Those are valuable, and they are not what this problem needs.
A methodology improves a product. It helps a team find what is wrong with the thing in front of them and decide what to build next. It does not, by itself, let a finding from one system inform a finding from another. Two teams can run impeccable studies and produce results that cannot be compared, because each defined availability, success, and error in its own local terms. The knowledge does not accumulate. It piles up.
The problem is not that HCI lacks methods. It has many strong ones, quantitative and qualitative, and it uses them well. The problem is architectural. There is no shared way to describe an interaction independently of the product in which it occurred, without stripping away the conditions that gave it meaning. Until that description exists, comparison stays local, and a field that cannot compare its findings across cases cannot become cumulative about its central object, however sophisticated its individual studies are.
This is a problem of scientific architecture rather than technique. Physics did not advance by treating every moving object as an unrelated event. Biology did not mature by placing every observed difference into a single category called life. In each case, progress required distinctions that let some relationships be studied without pretending the rest of the world had stopped mattering. The difficulty was never finding the one true description. It was constructing descriptions that made particular explanations possible.
Why I began this work
Over the past several years I have been developing a research program that I call Computational Interaction Science, or CompInt. I do not present it as a new scientific discipline, nor as a replacement for human-computer interaction. It is an attempt to investigate whether computational interaction itself can be represented, observed, measured, compared, and eventually explained with greater precision than existing frameworks presently allow. Whether that attempt succeeds is an empirical question, not a conceptual declaration.
CompInt begins from a simple scientific observation. Mature sciences become cumulative only after they establish stable objects of inquiry, reproducible observations, explicit measurements, principled comparison, and eventually predictive models. The framework asks whether computational interaction has reached a similar point in its own intellectual development, and whether the distinctions the field already knows how to draw in prose can be made explicit enough to compare across systems.
It does that by separating things ordinary language conflates. What a system can do, what it will permit, what its current state allows, what its interface exposes, and what a person believes possible are treated as distinct conditions rather than a single word, available. What a system contains and what its interface projects are held apart. What was observed and what was inferred are kept separate, so that a behavioral trace is read as a record rather than a self-interpreting explanation. And when two things cannot be compared on a common basis, the honest result is that they are incomparable, not that they are merely dissimilar.
I am not going to develop that architecture here. This essay had one job, which was to show why the architecture is needed, and the history is the argument. For four decades the field met something outside its current description and widened the frame to include it. Artificial intelligence is another such encounter, larger than most, because the systems now entering workplaces and homes do not wait for instructions. They propose, summarize, interpret, and sometimes act, and their possibilities cannot be read from the controls on a screen.
A field can keep widening around that, as it has before. What it has not yet done is build the one thing that would let its findings accumulate: a disciplined way to describe an interaction as an object in its own right while keeping it connected to the people, systems, and institutions through which it occurs. Whether such a description is possible, and whether it earns its keep, is what the rest of this work sets out to test.
For now the aim is narrower, and worth stating plainly. Before a field can study a phenomenon, it has to stop confusing the phenomenon with the instruments used to observe it. That is where the next article begins.
Selected references
Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101–108.
Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-computer interaction. Lawrence Erlbaum Associates.
Dourish, P. (2001). Where the action is: The foundations of embodied interaction. MIT Press.
Engelbart, D. C. (1962). Augmenting human intellect: A conceptual framework (SRI Summary Report AFOSR-3223). Stanford Research Institute.
Gibson, J. J. (1979). The ecological approach to visual perception. Houghton Mifflin.
Hutchins, E. (1995). Cognition in the wild. MIT Press.
Norman, D. A. (1988). The psychology of everyday things. Basic Books. (Reissued as The design of everyday things.)
Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge University Press.
Additional context on the models and institutions referenced above: the ACM Special Interest Group on Computer-Human Interaction (SIGCHI), founded 1982; the GOMS and keystroke-level models developed by Card, Moran, and Newell; and, on why contemporary AI systems strain earlier assumptions about interaction, Bommasani et al. (2021), On the opportunities and risks of foundation models (arXiv:2108.07258).
