Figure 2.2. Sutherland's diagram for calculating a square root. Reproduced
from Curry (1978).
The field of visual programming has been active since 1969, and many
different systems have been built to both create and visualize computer
programs. Our goal in this chapter is to identify and examine the most
important of these systems. It will become clear that all systems are drawing
on a common set of visual conventions.
We find that an understanding of diagramming gives insights that will
be useful in later chapters when we discuss the limits of graphic representation.
Therefore this survey includes many examples of diagrams from the computer
science literature, as well as examples of work in the field of visual
programming. We have also defined the set of conventions that recur in
all computer science diagrams.
Shu (1988) defines Visual Programming as:
The use of meaningful graphic representation in the process of programming.
This definition is too general - we suggest:
The use of diagrams in the process of programming
For diagram we start with James Maxwell's definition (Encyclopedia
Brittanica, 11th edition):
a figure drawn in such a manner that the geometrical relations between the parts of the figure illustrate relations between other objects.
To be more accurate, we change the term geometrical relations to
geometrical or topological relations, as in most of the diagrams
shown here, it is connections rather than distances that are significant.
We contrast graphic representation with textual representation, which
may name relations, but does not illustrate them. Other texts make the
point that visual programming makes use of two dimensions, while textual
representation makes use of one. Yet the dimensional distinction does not
communicate what we regard as the main difference - a diagram shows
relations, while a text names them. In a basic sense it is the difference
between a visual mode of expression and a mode of expression that, of all
our senses, is closest to the auditory.
The process of programming includes both the visual creation
of programs and the visualization of programs executing. These two fields
share many of the same problems, such as how to represent data structures.
We believe that the visual creation of programs is the harder problem of
the two, and put much of our concentration on this area.
There are five important source books relating to the topic. The first is Shu (1988), a book-length survey of the field with many diagrams and illustrations. The second book is a Springer-Verlag Lecture Notes in Computer Science, called Visualization in Programming, edited by Gorney (1986). The third is a book edited by Chang (1986). Two collections of papers contain many of the notable articles in the field: Visual Programming Environments: Applications and Issues, and Visual Programming Environments: Paradigms and Systems, both by Ephraim P. Glinert (1990). The Ph.D. theses by Raeder (1984), Glinert (1985), Brown (1987), and Curry (1978) all contain survey chapters. There is a special issue of Computer devoted to the topic (August 1985). The Internet news group comp.lang.visual contains announcements and current debate.In the realm of diagrams, the observations here are influenced by the thinking of Peirce (1934) and Nadin (1984). Also, the work of Tufte (1983) is an important source for clear thinking about graphic representation.
2.1.3. History
Visual programming, like much else in computer science, has a relatively short direct history. Yet the visual part of visual programming owes much to conventions for representing information that go back much longer. As our later discussion will involve issues of representation, we now take a short look at the history of diagrams, then discuss the origination of visual programming.
Flowcharts are the first and best known diagrams of software. Goldstine
(1972: 266) claims he created the first flowchart for computers in 1947,
while he was working with Von Neumann. Yet these early charts were entirely
decoupled from the computer itself. It wasn't until the creation of graphic
display technology in the 1960s that such a coupling became possible.
W. Sutherland, in 1966, created the first interactive visual programming
language:
Figure 2.2. Sutherland's diagram for calculating a square root. Reproduced
from Curry (1978).
As we will see, it is notable how much current visual programming systems
still resemble this first example of Sutherland's. In 1969, researchers
at Rand created a language based on flowcharts (Ellis 1969).
Starting in the early 1970s, researchers at Xerox Parc created the first visual programming environments. Bitmapped graphics, mice, and window systems can be mainly credited to this research laboratory. The culmination of the work came in the form of Smalltalk, an operating system/programming environment/programming language (Goldberg 1983). The present graphic user interfaces differ little in concept from the Xerox Parc vision. In the 1980s, Apple Computer, Sun Microsystems, and M.I.T. (X Windows) spread graphic user interfaces to researchers and consumers; recently, the creation of working windowing systems for PCs (Microsoft, IBM, NeXT) has created a flurry of systems based on graphic user interfaces. In the language realm, a commercial version of PROGRAPH exists on the Macintosh platform.
When many think of visual programming languages, the immediate association
is to Visual Basic, a Microsoft product. By our definition, it is not a
purely visual language. Instead of being based on diagrammatic representation,
its underlying language is an enhanced textual version of the Basic language.
In front of this textual language is a well-thought-out graphic user interface,
which allows the programmer to construct windows and all their corresponding
components such as buttons, slider bars, and menus, by selecting graphic
icons and dragging them onto a graphic representation of a window. The
programmer then writes textual source code fragments that are essentially
event handlers for the different possible mouse and keyboard events. This
code is linked to the graphic representation of the window, so that instead
of scrolling through long files of source, a programmer can access relevant
code by clicking on a physical location. In other words, the interface
provides, first of all, a way of constructing the framework of a user interface
by manipulating graphic objects, and, second of all, a way of spatializing
access to the textual code that needs to be written. Part of the success
of Visual Basic is the flatness of the language - many verbs are provided,
and many software vendors have been encouraged to create modules that add
more verbs. As we will discuss in chapter 7, the hybrid nature of Visual
Basic, along with the flatness of the language, are good design decisions
that allow graphic representation to be used in a productive way. But what
we are searching for here is whether a more extreme language that is purely
diagrammatic is possible.
Visual basic is one among many emerging graphic user interfaces oriented
toward programmers. Microsoft makes Visual C++, a graphical user interface
closely coupled with a C++ compiler. Borland and PowerSoft also make products
which use visual conventions in a way similar to Visual Basic. Computer-Aided
Software Engineering (CASE) companies such as Cadre and IDE have been making
visual tools for many years, and are now taking advantage of the current
interest in object-oriented notations to widen the availability of their
products.
In this section we discuss the main type of software diagrams. These
types form the basis for all visual programming systems.
In the course of this chapter identified a way of speaking about diagrams
in terms of metric, topological, and symbolic realms. Within the topological
realm we identified three main conventions, those of adjoinment, linking,
and containment, that when combined can be used to describe all topological
diagrams. Using these distinctions, we covered the field of software representation,
drawing examples from textbooks and journals on computer science. We identified
and described flow charts, data flow diagrams, signal processing diagrams,
layer charts, and a variety of other diagrams in formal visual terms.
Also discussed were a variety of systems built to develop and visualize
computer programs. Each of these systems made use of the diagrammatic conventions
we identified. Most of the systems were based on functional languages,
which are more suited to visualization because of their simpler parameter
schemes, their concept of single-assignment, and their data driven nature.
It is from the functional side that we begin in creating some alternate
visual languages.
We noted throughout the discussion some inherent limits in the visual realm. We discussed the representation of recursion, and pointed out that the visual cannot express this symbolic concept, and that this is a warning sign about the limits of the visual. We will return to these limits in Chapter 7.