Parsing the Languages of the New Media

“We begin to see that the printed word and cinema also can be thought of as interfaces, even though historically they have been tied to particular kinds of data.  Each offers its own grammar of actions, each comes with its own metaphors, each offers a particular physical interface.”

Treating the notion of “interface” both as noun and verb — as a cultural practice as well as a cultural product — certainly has advantages when it comes to a discussion of something so broad as “media.”  The objective here is to ground the conventions of new media in pre-existing forms, and to thereby establish the validity of Manovich’s attempt to interpret the emerging conventions of new media practices in terms of an existing body of criticism.  The danger of such expansive definitions, however, is that if a concept is defined too broadly, it becomes difficult to discern its proper applicability.  Such expansive definitions, therefore, must rely on a precise delineation of terminology if they are to be successful.  In this case, Manovich’s use of the term “interface” suffers in part from how the term is defined, and in part from how the term is applied.

While it may seem sensible to understand an interface in terms of a particular way of “organizing information, presenting it to the user, correlating space and time, and structuring human experience,” there is little about this list of features that uniquely signifies the notion of “interface.”  A narrative and a traffic sign both accomplish these goals.  And while there may be some novelty — and perhaps even some insight — in an expansive definition of the “interface” which suggests that traffic lights are a way of “interfacing” with a city, such an expansive definition becomes problematic when one considers that traffic lights were developed to manage the behavior of many people choosing to “interface” with cities by automobile.  By this understanding, then, “interfaces” are developed for “interfaces” to “interface” with “interfaces” — which not only reduces the term to absurdity, but provides few clues about the cultural practices that compel such developments.

While such an expansive definition may provide some insight when treated in a strictly metaphorical sense, a literal or material applications lead to undesirable consequences.  While a narrative clearly meets Manovich’s definition of an “interface,” it is worth noting that a narrative is not tied to any particular media form: narrative appears in fiction, poetry, cinema, drama, and ordinary conversation.  Thus, in terms of Manovich’s exposition of the “interface,” it is unclear why print and cinema should be more valuable to the discussion than narrative or traffic regulation.  And while it may be argued that print and cinema both refer to specific media types in virtue of their material instantiation, a book is every bit as material as a city; furthermore, such an appeal to the material instantiation of various media types provides no recourse here, since it is precisely the notion of a media types specific materiality that Manovich seeks to undermine in his discussion of new media.

In terms of how Manovich employs his expansive definition of the “interface,” the discussion suffers from serious ambiguities.  For example, when Manovich asserts that the printed word is an interface that has been historically tied to a particular kind of data, we must ask just what sort of data it is to which the printed word provides an “interface.”  If we understand this assertion to mean that the printed word has, historically, only been used to “interface” with text, we are left with something quite close to a tautology, which here provides little or no insight.  If we insist that print is a unique instance of text, we must admit that print is found in many places other than books — thus raising questions about Manovich’s discussion of the “page” as a fundamental interface metaphor.  If we examine how the printed word has been used historically, we can see quite clearly that print is not by any means bound to a particular type of data: poetry relates to feelings, an anthropological study conveys facts, a legal document describes laws, a geometrical treatise presents certain logical forms, and so on.

Likewise, cinema can be used to convey facts for the purpose of instruction, rhetoric for the purpose of indoctrination, propaganda for the purpose of persuasion, or abstract visual forms either to provide aesthetic pleasure or to illustrate certain critical theses.  Just what “data” is, and how the term ought to be applied, becomes as important here as what an “interface” is.

Manovich’s discussion of the “interface” here serves to foreground a perception that traditional media types are becoming unhinged from their material instantiation, and insofar as they are then incorporated into computer-mediated aesthetics, they become metaphorical as a result.  That is, print is no longer just print, but a signifier that becomes a source of raw material for the new media.  Although there may be some truth in this perception, to then suggest that such a trend finds either its apex or an initial catalyst in the new media is deeply problematic.  It is not a new development that printed words may be enjoyed either as a private experience between a reader and a book, or read aloud in a performance.  A book and a theatrical production each provide a distinct way of “interfacing” with the printed word.  In the case of print especially, letters themselves may also be appreciated on the basis of visual aesthetics, either in terms of a particular typographer’s skill, or in terms of the antiquarian quality of a rare historical artifact.  One can find an illuminated manuscript visually beautiful even if one can’t read Latin.

And as for the metaphorical value of print, we find in the Gospel of John the opening statement: “In the beginning was the Word.”  Even if we accept this particular phraseology as tied to the King James English, it provides a noteworthy corollary to the Shakespearean dictum that “all the world’s a stage.”

“The shift to digital media affects not just Hollywood, but filmmaking as a whole.  As traditional film technology is universally being replaced by digital technology, the logic of the filmmaking process is being redefined.”

While this observation certainly alludes to a process which many critics have observed — that digital technology exerts an influence on how films are produced by filmmakers and understood by audiences — the formulation and exposition of this observation here relies on generalities that omit many practical considerations of great importance.

The difficulty is a product of a methodological problem with Manovich’s text — that is, he begins with a reductionist approach to analyzing new media, then reasons through the potential consequences of his premises.  The problem with such an approach is that it relies entirely on the validity of the premises.  In this case, the premises are not only flawed, but in their focus on the purported “concrete” factors that distinguish new media objects from traditional media, they gloss over many cultural continuities.

The result of this difficulty is a series of observations that are equally disconnected from the history and the present reality of filmmaking.

Take, for example, Manovich’s assertion that the result of 3D computer animation is that “live-action footage is displaced from its role as the only possible material from which a film can be constructed.”  The thrust of Manovich’s assertion here is to emphasize the “newness” and the “otherness” of digital cinema; yet he does so as the expense of accuracy.

The history of cinema provides numerous examples that demonstrate the utter falsity of this claim.  Early in the history of cinema — before the Hollywood studio system came to exert a dominant influence on the aesthetics of film — we can see filmmakers such as Man Ray conducting experiments to produce moving images on celluloid without relying upon the photographic apparatus of the camera.  Man Ray’s 1926 film Emak Bakia contains several sequences produced by placing various objects directly on a strip of film and then exposing the film to light.  Aside from Hollywood’s rich history of cell-based animation (which clearly does not rely on live-action footage) we find contemporary experimental filmmakers such as Stan Brakhage — who made numerous films by directly applying pigment to clear leader, or scratching the emulsion off of black slug — continuing the investigations begun by Man Ray.  Furthermore, it is worth considering that scientific time-lapse footage, such as of microbes or phototropism, do not qualify as live-action imagery (although they are examples of photographic motion pictures).

The Fifth Principle of New Media describes how:

“the logic of a computer can be expected to significantly influence the traditional cultural logic of media; that is, we may expect that the computer layer will affect the cultural layer.”

This Principle of New Media is the least well-defined, in part due to the unusual technical term used to name it, and in part for how it draws very general cultural considerations into what is otherwise primarily a discourse about the mechanical features of computers.

Transcoding is a technical term in computer science that relates, as Manovich notes, to the translation of information from one format to another; it is an important feature of this technical term, however, that the translation occurs within a computer system. Transcoding is the translation of information from one digital format to another; thus, printing a digital photograph onto paper does not qualify. The use of the word “transcoding” is unfortunate because it deprives readers of linguistic intuitions that might be derived from a more familiar term; the use of the word as a metaphor is also problematic, because culture is neither a “format” nor a product of the types of formal relationships that govern computer formats.

While it might be Manovich’s intent in this case to argue that our experience with computers colors how we view cultural activity — that computers make us see cultural activity in a more “computerized” sense — it is important to understand Manovich’s treatment of this term as an analogy, rather than a statement about formal equivalency, or one that implies a strong causal relationship. Similar analogies have arose in the past: after the invention of the mechanical clock, for example, it became popular in Western science to approach cosmology as though one were studying a clock-like mechanical device.

Although it is undoubtedly the case that computers have had some impact on culture, just how this effect is to be understood as substantially different from the technological impact of more traditional media is unclear. In terms of the linguistic consequences of media on culture, it is worth noting that following the widespread cultural acceptance of television and radio, for example, the English language gained a new colloquialism: “to tune out” what one finds uninteresting. The Sapir-Whorf Hypothesis in linguistics, which asserts that language plays a central role in what features of the world we can readily perceive, suggests that the emergence of such colloquialisms as “tuning out” might have consequences more profound than simply the availability of particular informal expressions. Even outside of a discussion about recording media, one can find in Christianity or Islam a mystical, cosmological significance attributed to the word.

Described as “a blend of human and computer meanings, of traditional ways in which human culture modeled the world and the computer’s own means of representing it,” the cultural effect of “transcoding” is understood as affecting “all cultural categories and concepts.”

Given that computers were designed under considerations of precisely the reflexive relationship Manovich here identifies, it should come as no surprise to discover such a relationship present in new media. Manovich’s description of transcoding, however, privileges the relationship as proceeding from computers to culture, and largely ignores the impetus behind the trend in the opposite direction. The discussion of “transcoding” is problematic insofar as it is held within the context of an analysis philosophically-grounded as though practical computers, in their design and use, could be meaningfully understood apart from the cultural attitudes, beliefs, goals, and habits that produced computers and made their presence commonplace.

In many important respects, computers are modeled on human physiology and the ways our physiology allow us to perceive the world. The RBG color model used by computers to represent images, for example, is successful at reproducing the colors we see in the world because it is modeled on how our physiology recognizes color. Similarly, much of the work that went into designing computers as formal systems derives from Gottlob Frege‘s study of natural language.

The distinction Manovich draws between “the computer layer” and “the cultural layer” may be part of an attempt to structure a dialectic relationship between the mechanical behavior of computers and the cultural uses for computers, wherein “new media” becomes a synthesis of “computers” and “culture.” In such a case, culture would seem to carry the connotation of something organic, while computers would carry the connotation of something artificial; such a dialectic, however, would presuppose an opposing relationship between computers and culture that really does not exist as presupposed.

It is generally assumed that because computers are human inventions governed by well-defined mechanical relationships, they can therefore be more fully understood than something like culture, in which we participate, but never deliberately invented. Despite the well-defined nature of practical computers, there are a number of programmatic difficulties in attempting to formulate a comprehensive theory of computation. The way Manovich relies upon concepts drawn from computer science involves many of these difficulties.

Brian Cantwell Smith, in his essay “The Foundations of Computing” wrote:

“What has been (indeed, by most people still is) called a ‘Theory of Computation’ is in fact a general theory of the physical world — specifically, a theory of how hard it is, and what is required, for patches of the world in one physical configuration to change into another physical configuration. It applies to all physical entities, not just to computers.

“Not only must an adequate account of computation include a theory of semantics; it must also include a theory of ontology… Computers turn out in the end to be rather like cars: objects of inestimable social and political importance, but not in and of themselves, qua themselves, the focus of enduring scientific or intellectual inquiry — not, as philosophers would say, natural kinds.

“It is not just that a theory of computation will not supply a theory of semantics… or that it will not replace a theory of semantics; or even that it will depend or rest on a theory of semantics… computers per se, as I have said, do not constitute a distinct, delineated subject matter.”

The main thrust of Smith’s argument is that the idea of an all-encompasing theory of computation may be as incoherent as an attempt to formulate an all-encompasing “theory of walking.” For Manovich then to ground his theory of new media in terminology from computer science, without carefully delineating in what possible domains his assertions are applicable, presents very fundamental difficulties to the use of The Language of New Media for making valid inferences about individual new media objects.