Systems, manuals, usability and graph theory
SERC Grant No. GR/J43110
Most people shrug off their problems using gadgets like video recorders, fax
machines, photocopiers and computers. Yet closer analysis shows that such systems
have profound engineering problems, indicating that manufacturers add many features
rather than design a coherent system. User manuals are unintelligible, mainly because
they describe unintelligible systems.
A tool was built that can simulate and analyse many interactive systems, and combines
the simulation with the systems' manuals. This ensures the manuals are complete and
correct, and indeed significantly eases the manual writers' task. The tool also
demonstrated a powerful approach to user assistance, which can either be used to help
the designer appreciate the users' potential problems, or which can be used to provide
interactive help for a user -- of an unusually powerful and informative kind.
The tool, and the whole approach, was based on sound engineering principles (graph
theory), and this permits unusually rigorous analysis and design-time evaluation.
Scientifically, we have shown the advantage of a sound theory base, and that one is
applicable to human-computer interaction issues, an area where system evaluation is
normally delayed till (or past) delivery. Practically, we have taken several types of
system (VCR, fax, calculator) and shown they can be engineered to be vastly simpler
and easier to understand, yet can retain the original functionality. Moreover, we can
generate interactive help/training as well as [currently skeleton] conventional manuals
that are guaranteed to be complete and correct.
Several programs for the Apple Macintosh are available. Many offprints
and technical reports are available, both directly generated by the project
and on closely related topics.
Mathematics is what does not change when things change -- a brief way of
summarising Russell or Whitehead on the subject. Take apples or oranges, and it is
obvious that natural numbers have advantages for precise reasoning about what fruit
does, especially when there is a lot of it.
Computer scientists agree that most computer systems, actual or imagined, are too
complex to think clearly about, and that mathematics must be used. Hence 'formal
Interactive systems are also too complex for users to reason about. User manuals are
unintelligible, mainly because they describe unintelligible systems. Like accounting for
apples or understanding program design, we need a different and more fruitful
representation for what is important in user interfaces. (Computer scientists, having a
privileged view of implementation, tend to under-estimate perceived complexity for
The project Systems, manuals, usability and graph theory viewed
interactive systems as finite state machines (specifically, finite synchronous reactive
systems), which view considerably simplifies comprehending them, and leaves their
deep user interface features unaffected. This has the advantage that, so-expressed,
interactive systems' properties are amenable to graph theory, quantitative analysis, AI
techniques, ... and obtains a direct correspondence with hypertext (which we use for
representing conventional and interactive manuals).
If we cannot say anything useful about real user interfaces with our approach, then we
would indeed have discovered something fascinating about the nature of user interfaces
and the inadequacies of formal methods to tackle significant aspects of design. This
project found, to the contrary, that there are huge untapped opportunities for
identifying problems and correcting them.
Overview of the research programme
The project Systems, manuals, usability and graph theory very
successfully showed that the otherwise varied and separate activities of user interface
design, manual writing, user interface analysis, intelligent help, and manual writing
can all be viewed from the same, powerful, perspective. Thus any useful mathematical
properties in one of these areas apply with force in the other areas; since we know a lot
of psychological results about manuals, this applies also to system use or system
design. We built a system, called Hyperdoc, that combined these different views of
interactive systems into a single environment: simultaneously a design
tool, a simulation tool, a manual-writing tool, a simulator with intelligent help, an
analytic tool, and so on.
Hyperdoc enabled us to simulate and analyse several commercial user interfaces (such
as video recorders, an ATM, a fax machine, and a subsystem of the Airbus A320). We
explored various avenues that Hyperdoc's development suggested. Hyperdoc has been
distributed, for example to Georgia Tech, and has been used in graduate labs. We have
had very enthusiastic feedback for our approach, and for Hyperdoc in particular. We
have also published several papers, submitted more, and developed declarative user
interfaces to explore their usability characteristics as alternatives to conventional
imperative (state transition style) interfaces.
Main aims and achievements
In contrast to almost all other work in user interfaces (human computer interaction), in
describing systems as they are, or in finding ad hoc ways to specify complex
interactive features; instead, we wanted a theory. We wanted to understand clearly why
interactive systems have problems.
The overall aim of the project was to contribute to the theory and practice of interactive
systems. We have made significant progress and have laid substantial foundations for
The two main specific aims were: to build a system development environment that
permitted the simulation and analysis of interactive systems; secondly, to explore graph
theoretic issues and relate them to user interfaces. With the proviso that the system we
have developed has a number of minor limitations (it was a one year project), these
two aims were fulfilled. In fact, the productivity of the project has been higher than
anticipated, which we attribute to the clarity of purpose afforded by the plans and
general approach taken, namely of taking a radically abstract view of user interface
The deliverables of the project are the system Hyperdoc (which is available as a self-contained program, together with some Mathematica packages for supporting
analysis), as well as various papers and reports. A World Wide Web document is also
available, including some hypertext user manuals generated directly from
our simulations -- thus emphasising the close connection between system, manual, and
Illustrative technical contributions
The major contribution of the project has been to reassert the role of tractable systems
specification in user interface design. The varied and specific insights gainedconfirm
We used FSMs annotated with user manual text. The FSM can be executed by
Hyperdoc as a conventional system simulator (including appropriate realistic visual
effects, typically scanned in from an actual device), can generate manuals (e.g., in
HTML). Hyperdoc can do simple analysis itself, or generate Mathematica expressions
for detailed analysis, and can provide interactive intelligent adaptive help to the user
during use of the simulation. Statistics of use are also collected, which can be used in
many ways. Because all of these features flow from a single conceptual approach, the
approach is very powerful, and it means that design ideas flow into interaction ideas
which flow into manual ideas, and manual ideas flow into design. The system itself,
Hyperdoc, does not distinguish between any of these 'modes' of use.
We give below a sampler of contributions:
A typical error users make is to press a button once too often, because the system is too
slow confirming it had been pressed the first time. For a certain video recorder, the
average cost of recovering from such an overrun error is larger than the average cost of
moving between any pair of states, and the worst case recovery involves 11 presses.
Given that an 'undo' button would make the cost uniformly one press, this is an
avoidable flaw in the design.
This point is very basic. Why, then, do manufacturers miss the design opportunity? A fundamental
contribution of this project has been to show that a simple, well-founded and easily applied theory gets
useful results. This clarity is directly opposed to conventional approaches to interactive systems
Compare this approach with, say, Brenda Laurel's Computers As
Theatre, which takes the view that interacting with complex
systems is a dramatic experience; compare Donald Norman's
Psychology of Everyday Things, which provides
psychological explanations of breakdown. These (and other) views,
whilst stimulating hindsight, are difficult to apply proactively, let alone
Interactive systems typically have status indicators (e.g., an 'on' light). The simulator
supports indicators, and its help system can conveniently assist the user to find
answers to questions such as "Why isn't the ALERT light off?" -- or show the user
how to get it off, subject to various invariants (e.g., don't make the ALERT light go
out by switching the system off!) or whatever as appropriate ... and the
same mechanism supports the designer in constructing a complete manual,
for the assistant can also answer questions like "Where is there a state that hasn't been
Much conventional HCI is empirical, for instance manuals have been widely and
effectively studied; but this mainly contributes to evaluating system output (manuals
are generally written after a system has been defined). Instead Hyperdoc enables such
HCI insights to now input into system design -- that is, input into the
same system design, not the next.
What is a 'structured program'? A theoretical definition is that it is composed of certain
primitive flowgraphs, corresponding to sequence, if-then-else,
while-do, and certain others (e.g., while with
exit). If a graph is not composed (in a way that is formally defined) of
such primitives, we say it is unstructured. There are standard algorithms
for flowgraph decomposition.
Recall the lore of unstructured programs: It is widely held that structured programming
is better than unstructured, for many good reasons that do not need rehearsing here.
We now identify <programmer, program, computer> with <author, manual,
user> and adopt the arguments in support of structured programming mutatis
We found that every flowgraph that could be constructed from the manual
of a complete and correct model of a video recorder was unstructured. In short, the
system was unstructured. This result implies that the user manual is unlikely to be
correct and unlikely to be correctly understood -- neither by its author(s) nor by its
reader(s). Whether the system was implemented from the manual or vice versa, there is
likely to be little correspondence between the two, for precisely the same psychological
reasons of complexity that an unstructured Pascal program is likely not to do what its
We simulated a BT fax, and quickly found it has features the manual does not define. (A
Mathematica function determines a Chinese Postman tour of the FSM,
which is used to check the accuracy of the simulation vis
à vis the simulated device.) Again, the machine's manual does
not decompose into structured subflowgraphs; and it is plausible that the lack of
correspondence between the manual and the system is attributable to the unstructured
nature of the device. (In the case of the fax there are known reasons for its poor
Had the device been developed within Hyperdoc, this fact (and other properties) could
have been readily determined, and the designer might have chosen alternatives.
Moreover, a correct manual could have been generated automatically,
regardless of the obfuscation in the design!
Summary of basic work
How can manufacturers design such difficult to use and badly documented devices?
The answer is that they don't understand them. They sorely need a well-founded,
simple and reliable design environment, such as Hyperdoc proves can be built.
Hyperdoc was built to explore existing systems, rather than specifically to ease the
constructive specification process. That is the next research project, which
is fully justified now we know how readily and effectively it may be done.
Alternative styles of user interface
Interactive finite state machines are ubiquitous, and evidently need improving. The
project showed this can be done effectively, and from a sound theory base.
For some applications it is possible that very different styles of interaction may be
preferable. It is not always the case that a direct correspondence is required between
the manual and the system it describes (as Hyperdoc directly supports). Sometimes it
may be better to provide rules for the user, rather than recipes. Therefore
we examined an extreme form of manual and system, one which was specified for the
user purely in terms of rules.
The project implemented a purely declarative calculator. The result is perhaps the only non-
trivial declarative user interface (and one that can be used by the public -- school
children to scientists -- for conventional 'calculator' tasks). It certainly shows the
power of the approach, and indeed confirms some of the claims made for the clarity of
In assessing the new calculator design we examined many commercial calculators and
were appalled at their egregious design, lying
manuals, advertising hyperbole, and dismal software engineering. It suffices to
mention that one calculator currently advertised as supporting complex numbers cannot
get the square root of -1.
Manuals as structured programs
Abstract: A user manual may provide instructions that, if the user follows them, achieve any of
certain objectives as determined by the manual designers. A manual may therefore be viewed
rather like a computer program, as pre-planned instructions. Accordingly, software engineering
and its methods may be applied mutatis mutandis to the manual and its design
We consider structured programming methods, and show that some difficulties with user
interfaces may be attributed to manuals being 'unstructured.' Since there are many programming
metrics, and very many styles of manuals for user interfaces, this paper is concerned with
justifying the approach and showing how insightful it is.
Relation to project: a theory contribution from the project. Status: published in
BCS Conference, People and Computers, IX, pp67-79, CUP, 1994 (HCI'94).
Intelligent adaptive assistance and its automatic generation
Abstract: Manuals and interactive help are tedious to provide, difficult to maintain and difficult to
ensure correct, even for simple systems. This causes significant problems for users as well as
long-term product development.
This paper shows that it is possible to put a system specification and its documentation into
exact correspondence, it then follows that much previously manual work can be done
automatically -- and with considerable advantages, including guaranteed correctness and
completeness, as well as supporting powerful new features such as intelligent adaptive assistance.
This paper discusses how assistance can be provided to optimally answer "how to?" and "why
not?" and other questions.
Relation to project: describes the intelligent help in detail. Status: submitted to
Interacting with Computers.
Extract from abstract: HyperDoc is an interactive design/analysis tool for the evolution of well
documented interactive systems. It supports many desirable features and capabilities that are
essential from a development perspective, such as: a rapid prototyping/simulation environment;
interactive help and intelligent assistance (both provided through a hypertext mechanism that is
embodied in the underlying representation, a finite state machine). ...
Relation to project: demonstration of main tool. Status: presented at HCI'94.
Extract from abstract: The writing of manuals is time consuming, difficult and error prone,
particularly when they are developed by humans unaided. As the complexity of devices and hence
their manuals increase, support tools need to be used. We identify particular difficulties that need
to be addressed.
... Further, it is possible to tailor manuals for specific purposes, for example training; in fact
such a system is able to generate many alternative manuals assuming appropriate design
strategies can be devised. In principle we can also generate 'template' manuals suitable for
translation experts; this would help widen the market for the manuals and for the products.
Relation to project: summary of part of project. Status: submitted.
Design for a fax
Abstract: We examine a 1993/4 state of the art domestic phone/fax/answerphone machine as an
example and show that many reasonable alternative designs have advantages over it. We point to
many strange -- obscure, undocumented, non-functional, &c. -- features, and raise questions about
the process that led to the design, and what roles human factors and computer science played.
Relation to project: evaluation of a BT DF200 fax. Status: working paper.
Interactive consumer user interfaces
Abstract: Most interactive consumer products, such as video recorders are very difficult to use. How
could they be designed better? A system has been implemented that can simulate push-button
devices, and can be used as a tool for design and testing. As an interactive system, it directly
combines the user manual with the simulation so that "intelligent" help can be provided. It can
also generate conventional paper manuals, that are not only correct and complete, but which can
be "minimised" on any given criteria.
Relation to project: summary of part of project. Status: published in Proceedings
Norwegian Computer Society Annual Conference, Oslo, pp1-9, 1993.
A new calculator and why it is necessary
Abstract: Conventional calculators are badly designed: they suffer from bad computer science -- they
are difficult to use and buggy. I describe a solution, avoiding the problems caused by
conventional calculators, one that is more powerful and arguably much easier to use. The
solution has been prototyped, and design issues are discussed.
This paper shows an interactive system that is declarative, with the advantages of clarity and
power that entails. It frees people from working out how a calculation should be
expressed to concentrating on what they want solved. An important contribution is
to demonstrate the very serious problems users face when using conventional calculators, and
hence what a freedom a declarative design brings.
Relation to project: the improved interface developed is declarative and this poses a
pleasing balance to the imperative thrust of the rest of the project. Status: revised and
resubmitted to Computer Journal.
'Fifth Generation' user interfaces: The way ahead for complex systems
Abstract: The arguments for 'fifth generation' and declarative programming languages are well
known. If declarative programming is good for programmers, would it not have advantages for
users also? These advantages would be every bit as beneficial -- perhaps more so, in terms of
clarity, reliability, understandability -- for the appropriate sorts of user task.
Relation to project: a description of the new calculator user interface justified in
terms of declarative programming, rather than in terms of the (deplorable) state of the
art. Hence the emphasis is the generality of declarative user interfaces. The relation to
the project hinges on the very different styles of manuals required for declarative as
opposed to imperative interfaces. Status: presented at HCI'94.
A simple algorithm for the optimal Chinese Postman Tour
Abstract: The Chinese Postman Tour problem has many natural applications but standard algorithms
for it are complex. An approach using linear programming is, however, much easier to
understand, easier to prove correct, and more conveniently implemented. This paper, more
generally, addresses software engineering issues: software reuse, reliability and efficiency.
Relation to project: establishing the correctness of a simulation is crucial to any
criticisms of the system being valid. The Chinese Postman is one way to check a
simulation against the simulated machine. Status: working paper.
Users as computers: an approach to VR design and conceptual evaluation.
Abstract: An aim of VR is to immerse users, with a realistic (or metaphorical) experience. The goal
of immersion leads naturally to emphasising the technology to ensure it is capable of delivering
the required performance, and emphasising those presentation schemas (drama, metaphor) to
enhance immersion -- but perhaps at the expense of considering the sometimes difficult, even
impossible demands being placed on the user's computational (i.e., cognitive) resources.
This paper makes the case for considering the user as a computer, so that computational
aspects can be precisely handled in the design stages. This leads naturally to novel and creative
approaches to VR user interface design.
Relation to project: Argues that the techniques of the project generalise powerfully
to manage multimedia and VR issues. Status: submitted.
The main tool product of the project. Runs on Apple Macintosh computers.
Demonstrated at HCI'94.
Analytic framework for files input and output by Hyperdoc.
Proof of concept. Runs on Apple Macintosh. Demonstrated at HCI'94.
Sample hypertext manuals are available on the World Wide Web, generated
automatically from Hyperdoc.
In defence of FSMs
Finite state machines, FSMs, are covered in first year computer books; they can't do
much -- much less than Turing Machines, which are covered in the next chapter, and
Turing Machines aren't of much practical use either, even though more powerful than
finite machines! FSMs have further presumed disadvantages, at least for workers in
human-computer systems: they made a brief but rapidly eclipsed appearance in the
1960s HCI literature, and they are not 'realistic.'
On the other hand, there is a very substantial theory of FSMs, what they can and can't
do, what their properties are, how to optimise them, and so forth. Moreover, FSMs to
user interfaces are like numbers to grocers' accountancy; they are abstractions and can
be used to talk about, reason about and specify, a range of systems.
If a user interface has problems (or 'features') that cannot be expressed abstractly in
terms of FSMs, then one or more of the following may apply:
the problems are superficial (i.e., they can be fixed very late in the design cycle, for
example by modifying screen layout or colour);
the problems are so subtle that the users would be unlikely to be able to understand them.
or, there may be timing problems. This worry relates to the criticism (misguided, in fact)
that reactive systems cannot handle time. Moreover, time (e.g., in 'time outs') causes
users more problems than it solves. It certainly causes users reading manuals problems
when a system changes state before the paragraph explaining the so-called 'current'
state has been understood by the user!
Many authors who criticise FSMs ignore the power of abstraction; for instance, that
humans need never see an explicit FSM even as small as ten states (though
we can draw structured FSMs automatically, so seeing visual structure in them is
anyway not a problem). To criticise FSMs on grounds of size or unstructuredness is as
silly a mistake as criticising numbers on the grounds that 1000 apples is too many to
count, or that someone hasn't said 1000 is composite.
SERC Final Report / Prof. Harold Thimbleby
Bounds Green Road
LONDON, N11 2NQ
December 1, 1994