Systems, manuals, usability and graph theory

SERC Grant No. GR/J43110
Most people shrug off their problems using gadgets like video recorders, fax machines, photocopiers and computers. Yet closer analysis shows that such systems have profound engineering problems, indicating that manufacturers add many features rather than design a coherent system. User manuals are unintelligible, mainly because they describe unintelligible systems.


A tool was built that can simulate and analyse many interactive systems, and combines the simulation with the systems' manuals. This ensures the manuals are complete and correct, and indeed significantly eases the manual writers' task. The tool also demonstrated a powerful approach to user assistance, which can either be used to help the designer appreciate the users' potential problems, or which can be used to provide interactive help for a user -- of an unusually powerful and informative kind.

The tool, and the whole approach, was based on sound engineering principles (graph theory), and this permits unusually rigorous analysis and design-time evaluation.

Main achievements

Scientifically, we have shown the advantage of a sound theory base, and that one is applicable to human-computer interaction issues, an area where system evaluation is normally delayed till (or past) delivery. Practically, we have taken several types of system (VCR, fax, calculator) and shown they can be engineered to be vastly simpler and easier to understand, yet can retain the original functionality. Moreover, we can generate interactive help/training as well as [currently skeleton] conventional manuals that are guaranteed to be complete and correct.


Several programs for the Apple Macintosh are available. Many offprints and technical reports are available, both directly generated by the project and on closely related topics.


Mathematics is what does not change when things change -- a brief way of summarising Russell or Whitehead on the subject. Take apples or oranges, and it is obvious that natural numbers have advantages for precise reasoning about what fruit does, especially when there is a lot of it.

Computer scientists agree that most computer systems, actual or imagined, are too complex to think clearly about, and that mathematics must be used. Hence 'formal methods.'

Interactive systems are also too complex for users to reason about. User manuals are unintelligible, mainly because they describe unintelligible systems. Like accounting for apples or understanding program design, we need a different and more fruitful representation for what is important in user interfaces. (Computer scientists, having a privileged view of implementation, tend to under-estimate perceived complexity for users.)

The project Systems, manuals, usability and graph theory viewed interactive systems as finite state machines (specifically, finite synchronous reactive systems), which view considerably simplifies comprehending them, and leaves their deep user interface features unaffected. This has the advantage that, so-expressed, interactive systems' properties are amenable to graph theory, quantitative analysis, AI techniques, ... and obtains a direct correspondence with hypertext (which we use for representing conventional and interactive manuals).

If we cannot say anything useful about real user interfaces with our approach, then we would indeed have discovered something fascinating about the nature of user interfaces and the inadequacies of formal methods to tackle significant aspects of design. This project found, to the contrary, that there are huge untapped opportunities for identifying problems and correcting them.

Overview of the research programme

The project Systems, manuals, usability and graph theory very successfully showed that the otherwise varied and separate activities of user interface design, manual writing, user interface analysis, intelligent help, and manual writing can all be viewed from the same, powerful, perspective. Thus any useful mathematical properties in one of these areas apply with force in the other areas; since we know a lot of psychological results about manuals, this applies also to system use or system design. We built a system, called Hyperdoc, that combined these different views of interactive systems into a single environment: simultaneously a design tool, a simulation tool, a manual-writing tool, a simulator with intelligent help, an analytic tool, and so on.

Hyperdoc enabled us to simulate and analyse several commercial user interfaces (such as video recorders, an ATM, a fax machine, and a subsystem of the Airbus A320). We explored various avenues that Hyperdoc's development suggested. Hyperdoc has been distributed, for example to Georgia Tech, and has been used in graduate labs. We have had very enthusiastic feedback for our approach, and for Hyperdoc in particular. We have also published several papers, submitted more, and developed declarative user interfaces to explore their usability characteristics as alternatives to conventional imperative (state transition style) interfaces.

Main aims and achievements

In contrast to almost all other work in user interfaces (human computer interaction), in describing systems as they are, or in finding ad hoc ways to specify complex interactive features; instead, we wanted a theory. We wanted to understand clearly why interactive systems have problems.

The overall aim of the project was to contribute to the theory and practice of interactive systems. We have made significant progress and have laid substantial foundations for continued work.

The two main specific aims were: to build a system development environment that permitted the simulation and analysis of interactive systems; secondly, to explore graph theoretic issues and relate them to user interfaces. With the proviso that the system we have developed has a number of minor limitations (it was a one year project), these two aims were fulfilled. In fact, the productivity of the project has been higher than anticipated, which we attribute to the clarity of purpose afforded by the plans and general approach taken, namely of taking a radically abstract view of user interface design.

The deliverables of the project are the system Hyperdoc (which is available as a self-contained program, together with some Mathematica packages for supporting analysis), as well as various papers and reports. A World Wide Web document is also available, including some hypertext user manuals generated directly from our simulations -- thus emphasising the close connection between system, manual, and hypertext.

Illustrative technical contributions

The major contribution of the project has been to reassert the role of tractable systems specification in user interface design. The varied and specific insights gainedconfirm this success.

We used FSMs annotated with user manual text. The FSM can be executed by Hyperdoc as a conventional system simulator (including appropriate realistic visual effects, typically scanned in from an actual device), can generate manuals (e.g., in HTML). Hyperdoc can do simple analysis itself, or generate Mathematica expressions for detailed analysis, and can provide interactive intelligent adaptive help to the user during use of the simulation. Statistics of use are also collected, which can be used in many ways. Because all of these features flow from a single conceptual approach, the approach is very powerful, and it means that design ideas flow into interaction ideas which flow into manual ideas, and manual ideas flow into design. The system itself, Hyperdoc, does not distinguish between any of these 'modes' of use.

We give below a sampler of contributions:

Summary of basic work

How can manufacturers design such difficult to use and badly documented devices? The answer is that they don't understand them. They sorely need a well-founded, simple and reliable design environment, such as Hyperdoc proves can be built.

Hyperdoc was built to explore existing systems, rather than specifically to ease the constructive specification process. That is the next research project, which is fully justified now we know how readily and effectively it may be done.

Alternative styles of user interface

Interactive finite state machines are ubiquitous, and evidently need improving. The project showed this can be done effectively, and from a sound theory base.

For some applications it is possible that very different styles of interaction may be preferable. It is not always the case that a direct correspondence is required between the manual and the system it describes (as Hyperdoc directly supports). Sometimes it may be better to provide rules for the user, rather than recipes. Therefore we examined an extreme form of manual and system, one which was specified for the user purely in terms of rules.

Main publications

Manuals as structured programs

Abstract: A user manual may provide instructions that, if the user follows them, achieve any of certain objectives as determined by the manual designers. A manual may therefore be viewed rather like a computer program, as pre-planned instructions. Accordingly, software engineering and its methods may be applied mutatis mutandis to the manual and its design process.

We consider structured programming methods, and show that some difficulties with user interfaces may be attributed to manuals being 'unstructured.' Since there are many programming metrics, and very many styles of manuals for user interfaces, this paper is concerned with justifying the approach and showing how insightful it is. Relation to project: a theory contribution from the project. Status: published in BCS Conference, People and Computers, IX, pp67-79, CUP, 1994 (HCI'94).

Intelligent adaptive assistance and its automatic generation

Abstract: Manuals and interactive help are tedious to provide, difficult to maintain and difficult to ensure correct, even for simple systems. This causes significant problems for users as well as long-term product development.

This paper shows that it is possible to put a system specification and its documentation into exact correspondence, it then follows that much previously manual work can be done automatically -- and with considerable advantages, including guaranteed correctness and completeness, as well as supporting powerful new features such as intelligent adaptive assistance. This paper discusses how assistance can be provided to optimally answer "how to?" and "why not?" and other questions. Relation to project: describes the intelligent help in detail. Status: submitted to Interacting with Computers.


Extract from abstract: HyperDoc is an interactive design/analysis tool for the evolution of well documented interactive systems. It supports many desirable features and capabilities that are essential from a development perspective, such as: a rapid prototyping/simulation environment; interactive help and intelligent assistance (both provided through a hypertext mechanism that is embodied in the underlying representation, a finite state machine). ... Relation to project: demonstration of main tool. Status: presented at HCI'94.

Electronic manuals

Extract from abstract: The writing of manuals is time consuming, difficult and error prone, particularly when they are developed by humans unaided. As the complexity of devices and hence their manuals increase, support tools need to be used. We identify particular difficulties that need to be addressed.

... Further, it is possible to tailor manuals for specific purposes, for example training; in fact such a system is able to generate many alternative manuals assuming appropriate design strategies can be devised. In principle we can also generate 'template' manuals suitable for translation experts; this would help widen the market for the manuals and for the products. Relation to project: summary of part of project. Status: submitted.

Design for a fax

Abstract: We examine a 1993/4 state of the art domestic phone/fax/answerphone machine as an example and show that many reasonable alternative designs have advantages over it. We point to many strange -- obscure, undocumented, non-functional, &c. -- features, and raise questions about the process that led to the design, and what roles human factors and computer science played. Relation to project: evaluation of a BT DF200 fax. Status: working paper.

Interactive consumer user interfaces

Abstract: Most interactive consumer products, such as video recorders are very difficult to use. How could they be designed better? A system has been implemented that can simulate push-button devices, and can be used as a tool for design and testing. As an interactive system, it directly combines the user manual with the simulation so that "intelligent" help can be provided. It can also generate conventional paper manuals, that are not only correct and complete, but which can be "minimised" on any given criteria. Relation to project: summary of part of project. Status: published in Proceedings Norwegian Computer Society Annual Conference, Oslo, pp1-9, 1993.

A new calculator and why it is necessary

Abstract: Conventional calculators are badly designed: they suffer from bad computer science -- they are difficult to use and buggy. I describe a solution, avoiding the problems caused by conventional calculators, one that is more powerful and arguably much easier to use. The solution has been prototyped, and design issues are discussed.

This paper shows an interactive system that is declarative, with the advantages of clarity and power that entails. It frees people from working out how a calculation should be expressed to concentrating on what they want solved. An important contribution is to demonstrate the very serious problems users face when using conventional calculators, and hence what a freedom a declarative design brings. Relation to project: the improved interface developed is declarative and this poses a pleasing balance to the imperative thrust of the rest of the project. Status: revised and resubmitted to Computer Journal.

'Fifth Generation' user interfaces: The way ahead for complex systems

Abstract: The arguments for 'fifth generation' and declarative programming languages are well known. If declarative programming is good for programmers, would it not have advantages for users also? These advantages would be every bit as beneficial -- perhaps more so, in terms of clarity, reliability, understandability -- for the appropriate sorts of user task. Relation to project: a description of the new calculator user interface justified in terms of declarative programming, rather than in terms of the (deplorable) state of the art. Hence the emphasis is the generality of declarative user interfaces. The relation to the project hinges on the very different styles of manuals required for declarative as opposed to imperative interfaces. Status: presented at HCI'94.

A simple algorithm for the optimal Chinese Postman Tour

Abstract: The Chinese Postman Tour problem has many natural applications but standard algorithms for it are complex. An approach using linear programming is, however, much easier to understand, easier to prove correct, and more conveniently implemented. This paper, more generally, addresses software engineering issues: software reuse, reliability and efficiency. Relation to project: establishing the correctness of a simulation is crucial to any criticisms of the system being valid. The Chinese Postman is one way to check a simulation against the simulated machine. Status: working paper.

Users as computers: an approach to VR design and conceptual evaluation.

Abstract: An aim of VR is to immerse users, with a realistic (or metaphorical) experience. The goal of immersion leads naturally to emphasising the technology to ensure it is capable of delivering the required performance, and emphasising those presentation schemas (drama, metaphor) to enhance immersion -- but perhaps at the expense of considering the sometimes difficult, even impossible demands being placed on the user's computational (i.e., cognitive) resources. This paper makes the case for considering the user as a computer, so that computational aspects can be precisely handled in the design stages. This leads naturally to novel and creative approaches to VR user interface design. Relation to project: Argues that the techniques of the project generalise powerfully to manage multimedia and VR issues. Status: submitted.

Other output


The main tool product of the project. Runs on Apple Macintosh computers. Demonstrated at HCI'94.

Mathematica packages

Analytic framework for files input and output by Hyperdoc.

Declarative calculator

Proof of concept. Runs on Apple Macintosh. Demonstrated at HCI'94.

Hypertext manuals

Sample hypertext manuals are available on the World Wide Web, generated automatically from Hyperdoc.

In defence of FSMs

Finite state machines, FSMs, are covered in first year computer books; they can't do much -- much less than Turing Machines, which are covered in the next chapter, and Turing Machines aren't of much practical use either, even though more powerful than finite machines! FSMs have further presumed disadvantages, at least for workers in human-computer systems: they made a brief but rapidly eclipsed appearance in the 1960s HCI literature, and they are not 'realistic.'

On the other hand, there is a very substantial theory of FSMs, what they can and can't do, what their properties are, how to optimise them, and so forth. Moreover, FSMs to user interfaces are like numbers to grocers' accountancy; they are abstractions and can be used to talk about, reason about and specify, a range of systems. If a user interface has problems (or 'features') that cannot be expressed abstractly in terms of FSMs, then one or more of the following may apply:

Many authors who criticise FSMs ignore the power of abstraction; for instance, that humans need never see an explicit FSM even as small as ten states (though we can draw structured FSMs automatically, so seeing visual structure in them is anyway not a problem). To criticise FSMs on grounds of size or unstructuredness is as silly a mistake as criticising numbers on the grounds that 1000 apples is too many to count, or that someone hasn't said 1000 is composite.

SERC Final Report / Prof. Harold Thimbleby
Computing Science
Middlesex University
Bounds Green Road
December 1, 1994