Database as a symbolic form

  [ Lev Manovich ] *

  The semiotics of database

The dynamics which exist between database and narrative are not unique in new media. The relation between the structure of a digital image and the languages of contemporary visual culture is characterized by the same dynamics. As defined by all computer software, a digital image consists of a number of separate layers, each layer containing particular visual elements. Throughout the production process, artists and designers manipulate each layer separately; they also delete layers and add new ones. Keeping each element as a separate layer allows the content and the composition of an image to be changed at any point: deleting a background, substituting one person for another, moving two people closer together, blurring an object, and so on. What would a typical image look like if the layers were merged together? The elements contained on different layers will become juxtaposed resulting in a montage look. Montage is the default visual language of composite organization of an image. However, just as database supports both the database form and its opposite — narrative, a composite organization of an image on the material level supports two opposing visual languages. One is modernist-MTV montage — two-dimensional juxtaposition of visual elements designed to shock due to its impossibility in reality. The other is the representation of familiar reality as seen by a photo of film camera (or its computer simulation, in the case of 3-D graphics). During the 1980s and 1990s all image making technologies became computer-based thus turning all images into composites. In parallel, a Renaissance of montage took place in visual culture, in print, broadcast design and new media. This is not unexpected — after all, this is the visual language dictated by the composite organization. What needs to be explained is why photorealist images continue to occupy such a significant space in our computer-based visual culture.

It would be surprising, of course, if photorealist images suddenly disappeared completely. The history of culture does not contain such sudden breaks. Similarly, we should not expect that new media would completely substitute narrative by database. New media does not radically break with the past; rather, it distributes weight differently between the categories which hold culture together, foregrounding what was in the background, and vice versa. As Frederick Jameson writes in his analysis of another shift, in this case from modernism to post-modernism: «Radical breaks between periods do not generally involve complete changes but rather the restructuration of a certain number of elements already given: features that in an earlier period of system were subordinate became dominant, and features that had been dominant again become secondary.»(16)

Database — narrative opposition is the case in point. To further understand how computer culture redistributes weight between the two terms of opposition in computer culture I will bring in a semiological theory of syntagm and paradigm. According to this model, originally formulated by Ferdinand de Saussure to describe natural languages such as English and later expanded by Roland Barthes and others to apply to other sign systems (narrative, fashion, food, etc.), the elements of a system can be related on two dimensions: syntagmatic and paradigmatic.(17) As defined by Barthes, «the syntagm is a combination of signs, which has space as a support.» To use the example of natural language, the speaker produces an utterance by stringing together the elements, one after another, in a linear sequence. This is the syntagmatic dimension. Now, lets look at the paradigm. To continue with an example of a langauge user, each new element is chosen from a set of other related elements. For instance, all nouns form a set; all synonyms of a particular word form another set. In the original formulation of Saussure, «the units which have something in common are associated in theory and thus form groups within which various relationships can be found.»(18) This is the paradigmatic dimension.

The elements on a syntagmatic dimension are related <i>in praesentia</i>, while the elements on a paradigmatic dimension are related <i>in absentia</i>. For instance, in the case of a written sentence, the words which comprise it materially exist on a piece of paper, while the paradigmatic sets to which these words belong only exist in writer's and reader's minds. Similarly, in the case of a fashion outfit, the elements which make it, such as a skirt, a blouse, and a jacket, are present in reality, while pieces of clothing which could have been present instead — different skirt, different blouse, different jacket — only exist in the viewer's imagination. Thus, syntagm is explicit and paradigm is implicit; one is real and the other is imagined.

Literary and cinematic narratives work in the same way. Particular words, sentences, shots, scenes which make up a narrative have a material existence; other elements which form an imaginary world of an author or a particular literary or cinematic style and which could have appeared instead exist only virtually. Put differently, the database of choices from which narrative is constructed (the paradigm) is implicit; while the actual narrative (the syntagm) is explicit.

New media reverses this relationship. Database (the paradigm) is given material existence, while narrative (the syntagm) is de-materialised. Paradigm is privileged, syntagm is downplayed. Paradigm is real, syntagm is virtual. To see this, consider the new media design process. The design of any new media object begins with assembling a database of possible elements to be used. (Macromedia Director calls this database «cast,» Adobe Premiere calls it «project», ProTools calls it a «session,» but the principle is the same.) This database is the center of the design process. It typically consists from a combination of original and stock material distributed such as buttons, images, video and audio sequences; 3-D objects; behaviors and so on. Throughout the design process new elements are added to the database; existing elements are modified. The narrative is constructed by linking elements of this database in a particular order, i.e. designing a trajectory leading from one element to another. On the material level, a narrative is just a set of links; the elements themselves remain stored in the database. Thus the narrative is more virtual than the database itself. (Since all data is stored as electronic signals, the word «material» seem to be no longer appropriate. Instead we should talk about different degrees of virtuality.)

The paradigm is privileged over syntagm in yet another way in interactive objects presenting the user with a number of choices at the same time — which is what typical interactive interfaces do. For instance, a screen may contain a few icons; clicking on each icon leads the user to a different screen. On the level of an individual screen, these choices form a paradigm of their own which is explicitly presented to the user. On the level of the whole object, the user is made aware that she is following one possible trajectory among many others. In other words, she is selecting one trajectory from the paradigm of all trajectories which are defined.

Other types of interactive interfaces make the paradigm even more explicit by presenting the user with an explicit menu of all available choices. In such interfaces, all of the categories are always available, just a mouse click away. The complete paradigm is present before the user, its elements neatly arranged in a menu. This is another example of how new media makes explicit the psychological processes involved in cultural communication. Other examples include the already discussed shift from creation to selection, which externalizes and codifies the database of cultural elements existing in the creator's mind; as well as the very phenomena of interactive links. New media takes «interaction» literally, equating it with a strictly physical interaction between a user and a screen (by pressing a button), at the sake of psychological interaction. The psychological processes of filling-in, hypothesis forming, recall and identification — which are required for us to comprehend any text or image at all — are erroneously equated with an objectively existing structure of interactive links.

Interactive interfaces foreground the paradigmatic dimension and often make explicit paradigmatic sets. Yet, they are still organized along the syntagmatic dimension. Although the user is making choices at each new screen, the end result is a linear sequence of screens which she follows. This is the classical syntagmatic experience. In fact, it can be compared to constructing a sentence in a natural language. Just as a language user constructs a sentence by choosing each successive word from a paradigm of other possible words, a new media user creates a sequence of screens by clicking on this or that icon at each screen. Obviously, there are many important differences between these two situations. For instance, in the case of a typical interactive interface, there is no grammar and paradigms are much smaller. Yet, the similarity of basic experience in both cases is quite interesting; in both cases, it unfolds along a syntagmatic dimension.

Why does new media insist on this language-like sequencing? My hypothesis is that it follows the dominant semiological order of the twentieth century — that of cinema. Cinema replaced all other modes of narration with a sequential narrative, an assembly line of shots which appear on the screen one at a time. For centuries, a spatialized narrative where all images appear simultaneously dominated European visual culture; then it was delegated to «minor» cultural forms as comics or technical illustrations. «Real» culture of the twentieth century came to speak in linear chains, aligning itself with the assembly line of an industrial society and the Turing machine of a post-industrial era. New media continues this mode, giving the user information one screen at a time. At least, this is the case when it tries to become «real» culture (interactive narratives, games); when it simply functions as an interface to information, it is not ashamed to present much more information on the screen at once, be it in the form of tables, normal or pull-down menus, or lists. In particular, the experience of a user filling in an on-line form can be compared to pre-cinematic spatialised narrative: in both cases, the user is following a sequence of elements which are presented simultaneously.

(16) Fredric Jameson, «Postmodernism and Consumer Society,» in The Anti-Aesthetic. Essays on Postmodern Culture, ed. Hal Foster (Seattle: Bay Press, 1983), 123.

(17) Roland Barthes, The Elements of Semiology (New York: Hill and Wang, 1968), 58.

(18) Qtd. in ibid., 58.