The field of visual programming has been active since 1969, and many different systems have been built to both create and visualize computer programs. Our goal in this chapter is to identify and examine the most important of these systems. It will become clear that all systems are drawing on a common set of visual conventions.

We find that an understanding of diagramming gives insights that will be useful in later chapters when we discuss the limits of graphic representation. Therefore this survey includes many examples of diagrams from the computer science literature, as well as examples of work in the field of visual programming. We have also defined the set of conventions that recur in all computer science diagrams.

2.1.1. Terminology

Shu (1988) defines Visual Programming as:

This definition is too general - we suggest:

For diagram we start with James Maxwell's definition (Encyclopedia Brittanica, 11th edition):

To be more accurate, we change the term geometrical relations to geometrical or topological relations, as in most of the diagrams shown here, it is connections rather than distances that are significant.

We contrast graphic representation with textual representation, which may name relations, but does not illustrate them. Other texts make the point that visual programming makes use of two dimensions, while textual representation makes use of one. Yet the dimensional distinction does not communicate what we regard as the main difference - a diagram shows relations, while a text names them. In a basic sense it is the difference between a visual mode of expression and a mode of expression that, of all our senses, is closest to the auditory.

The process of programming includes both the visual creation of programs and the visualization of programs executing. These two fields share many of the same problems, such as how to represent data structures. We believe that the visual creation of programs is the harder problem of the two, and put much of our concentration on this area.

2.1.2. Major Sources

There are five important source books relating to the topic. The first is Shu (1988), a book-length survey of the field with many diagrams and illustrations. The second book is a Springer-Verlag Lecture Notes in Computer Science, called Visualization in Programming, edited by Gorney (1986). The third is a book edited by Chang (1986). Two collections of papers contain many of the notable articles in the field: Visual Programming Environments: Applications and Issues, and Visual Programming Environments: Paradigms and Systems, both by Ephraim P. Glinert (1990). The Ph.D. theses by Raeder (1984), Glinert (1985), Brown (1987), and Curry (1978) all contain survey chapters. There is a special issue of Computer devoted to the topic (August 1985). The Internet news group comp.lang.visual contains announcements and current debate.In the realm of diagrams, the observations here are influenced by the thinking of Peirce (1934) and Nadin (1984). Also, the work of Tufte (1983) is an important source for clear thinking about graphic representation.

2.1.3. History

Visual programming, like much else in computer science, has a relatively short direct history. Yet the visual part of visual programming owes much to conventions for representing information that go back much longer. As our later discussion will involve issues of representation, we now take a short look at the history of diagrams, then discuss the origination of visual programming.



Flowcharts are the first and best known diagrams of software. Goldstine (1972: 266) claims he created the first flowchart for computers in 1947, while he was working with Von Neumann. Yet these early charts were entirely decoupled from the computer itself. It wasn't until the creation of graphic display technology in the 1960s that such a coupling became possible.
W. Sutherland, in 1966, created the first interactive visual programming language:

As we will see, it is notable how much current visual programming systems still resemble this first example of Sutherland's. In 1969, researchers at Rand created a language based on flowcharts (Ellis 1969).

Starting in the early 1970s, researchers at Xerox Parc created the first visual programming environments. Bitmapped graphics, mice, and window systems can be mainly credited to this research laboratory. The culmination of the work came in the form of Smalltalk, an operating system/programming environment/programming language (Goldberg 1983). The present graphic user interfaces differ little in concept from the Xerox Parc vision. In the 1980s, Apple Computer, Sun Microsystems, and M.I.T. (X Windows) spread graphic user interfaces to researchers and consumers; recently, the creation of working windowing systems for PCs (Microsoft, IBM, NeXT) has created a flurry of systems based on graphic user interfaces. In the language realm, a commercial version of PROGRAPH exists on the Macintosh platform.


When many think of visual programming languages, the immediate association is to Visual Basic, a Microsoft product. By our definition, it is not a purely visual language. Instead of being based on diagrammatic representation, its underlying language is an enhanced textual version of the Basic language. In front of this textual language is a well-thought-out graphic user interface, which allows the programmer to construct windows and all their corresponding components such as buttons, slider bars, and menus, by selecting graphic icons and dragging them onto a graphic representation of a window. The programmer then writes textual source code fragments that are essentially event handlers for the different possible mouse and keyboard events. This code is linked to the graphic representation of the window, so that instead of scrolling through long files of source, a programmer can access relevant code by clicking on a physical location. In other words, the interface provides, first of all, a way of constructing the framework of a user interface by manipulating graphic objects, and, second of all, a way of spatializing access to the textual code that needs to be written. Part of the success of Visual Basic is the flatness of the language - many verbs are provided, and many software vendors have been encouraged to create modules that add more verbs. As we will discuss in chapter 7, the hybrid nature of Visual Basic, along with the flatness of the language, are good design decisions that allow graphic representation to be used in a productive way. But what we are searching for here is whether a more extreme language that is purely diagrammatic is possible.

Visual basic is one among many emerging graphic user interfaces oriented toward programmers. Microsoft makes Visual C++, a graphical user interface closely coupled with a C++ compiler. Borland and PowerSoft also make products which use visual conventions in a way similar to Visual Basic. Computer-Aided Software Engineering (CASE) companies such as Cadre and IDE have been making visual tools for many years, and are now taking advantage of the current interest in object-oriented notations to widen the availability of their products.


In this section we discuss the main type of software diagrams. These types form the basis for all visual programming systems.






In the course of this chapter identified a way of speaking about diagrams in terms of metric, topological, and symbolic realms. Within the topological realm we identified three main conventions, those of adjoinment, linking, and containment, that when combined can be used to describe all topological diagrams. Using these distinctions, we covered the field of software representation, drawing examples from textbooks and journals on computer science. We identified and described flow charts, data flow diagrams, signal processing diagrams, layer charts, and a variety of other diagrams in formal visual terms.

Also discussed were a variety of systems built to develop and visualize computer programs. Each of these systems made use of the diagrammatic conventions we identified. Most of the systems were based on functional languages, which are more suited to visualization because of their simpler parameter schemes, their concept of single-assignment, and their data driven nature. It is from the functional side that we begin in creating some alternate visual languages.

We noted throughout the discussion some inherent limits in the visual realm. We discussed the representation of recursion, and pointed out that the visual cannot express this symbolic concept, and that this is a warning sign about the limits of the visual. We will return to these limits in Chapter 7.

Return to Visual Programming Index Page