Parsing the Languages of the New Media

“The most likely reason modern media has discrete levels is because it emerged during the Industrial Revolution… Not surprisingly, modern media follows the logic of the factory.”

The argument here suggests that the way new media objects implement computer code is a product of the industrial mindset, with the implication that the values of industrial division of labor, specialization, and standardization led to the modern computer. This suggestion involves a complex set of interrelations between the thought processes introduced by industrialization, the structure of computers, and how these thought processes interact with the structure of computers when people create new media objects.

New media objects are conceived of as collections of discrete, indivisible units, such as pixels; and this conception presupposes a contradistinction to traditional media — such as sculpture or chemical photography — where surface properties vary with continuous and arbitrary degrees of detail.

The use of “discrete” here connotes precision, while “continuous” connotes imprecision: however accurately one attempts to measure the height of a bronze sculpture, for example, changes in temperature will cause the metal to expand or contract slightly on different days, contributing to an inherent imprecision in one’s measurement; a digital picture file, however, will always have the same number of pixels no matter on what day one decides to make a tally.

As it is a central feature of industrial mass production that one be able to manufacture large numbers of precisely identical objects, there are a number of superficial reasons why computers might seem to be the product of an industrial mindset: industrial fabrication techniques facilitated computers coming into widespread use, the individual components of computer hardware are in many respects both standardized in their construction and specialized in their function, and the binary code used by computers very much resembles an idealization of industrial order and production.

These congruences aside, however, the aforementioned argument as presented in The Language of New Media involves a number of substantial problems. Most obviously, the written alphabet is a system of discrete symbols: letters came into use long before industrialization, are just as indivisible as pixels in a digital image, and type set in a monospaced font falls into a grid not unlike the arrangement of pixels on a computer screen. Moreover, letters can be assigned numerical meanings: Hebrew is one example of an alphabet that does this.

There are also historical problems with attributing the discrete operations performed by computers to an industrial mindset. The history of computing machines reaches back to antiquity, and its early history can be found in such relics as the Antikythera mechanism. It could be argued that it was “the logic of the factory” that spawned the invention and design of digital computing machines, but it was, rather, a theological motivation that compelled Gottfried Leibniz in the late 1600′s to formalize the system of binary code used by today’s computers; Leibniz furthermore envisaged machines that would perform calculations using his binary system. Although it may be the case that industrialization substantially helped such computing machines in becoming a material reality, their conception lies very much apart from the industrial mindset.

While consumer use of computerized media might in many respects seem to follow “the logic of the factory” — especially as numerous commercial websites profit from user-generated content, which transforms the consumer into a type of specialized producer — the formal and material qualities of modern computerized media follow from a quite different logic.

In discussing new media’s use of spatialized visualizations for representing computer data, Lev Manovich observes that:

“The very first coin-op arcade game was called Computer Space. The game simulated a dogfight between a spaceship and a flying saucer. Released in 1971, it was a remake of the first computer game, Spacewar, programmed on a PDP-1 at MIT in 1962. Both of these legendary games included the word space in their titles; and appropriately, space was one of the main characters in each of them.”

Although the discussion prior to this passage uses the word “space” in a topological sense, as in “that which might be filled with objects,” the word “space” is here used to denote “that region beyond the Earth’s atmosphere.” The word “space” is spelled the same in each context, but makes reference to distinct concepts in each context. This conflation of otherwise distinct conceptual vocabularies makes two unrelated discussions appear to be connected in a way they really are not.

The argument that follows from the discussion of “space” is similarly unclear. Manovich observes that “in the original Spacewar, the players navigated two spaceships around the screen” while noting later in the same paragraph that “the space of Spacewar and Computer Space was not navigable.”

It might be supposed that the discrepency surrounding the use of “navigation” in these two cases is meant to point out that it is the spaceship rather than the environment that one manipulates while navigating; but to privilege space in the second instance as that which is manipulated while navigating would be at odds with the conventional usage of the word. To suppose then that the intent is to draw attention to new media’s potential to offer new conceptions of one’s relationship to space, it would be difficult to justify the use of these particular video games as new media objects while elucidating the properties of new media.

What conclusions a reader might draw from this argument is as unclear as the use of terminology, or furthermore how the discussion of “navigable space” relates to the Five Principles of New Media. The problem in part has to do with confusions that arise from an imprecise delineation of context, but also involves matters of interpretation.

If Spacewar was programmed in 1962, the prominence of “space” as a theme in the game might relate in some way to cultural attitudes influenced by the launch of Sputnik and the start of the “space race” five years earlier. As more computers came into use at the academies, their novelty was surely appreciated by those studying them. “Space” in this context might be understood as signifying novelty and to invoke an aestheticized vision of the future. To then contextualize this video game in a more general context of art history, one might appeal to novelty as a central feature of Modernism.

The First Principle of New Media identified in The Language of New Media is that the resulting objects of new media practices “are composed of digital code; they are numerical representations.”

When we look at a picture of the Mona Lisa on a computer screen, we do not consciously perceive digital code or numerical representations; there is an implicit phenomenological distinction being drawn between two descriptive categories: the operation of the computer (how information behaves within a computer as a formal system), and the conscious perceptions of a person interacting with a new media object through a computer interface (we see “new media objects” rather than digital code or numerical representations).

While these two descriptive categories refer to the same physical behavior (what, in a given instance, a person might be doing with a computer), they also require different vocabularies to adequately describe precisely what physical processes are involved and how these processes affect us. In the case of a person doing something with a computer, these vocabularies differ in terms of “how” something might be happening, agree on “that” which might be happening, and converge on “why” certain events result.

The First Principle of New Media elects to analyze the resulting objects of new media practices in terms of how information behaves within a computer as a mechanistic formal system. The assumption here is that analyzing new media in such a way will lead to a more fundamental or profound account of new media, compared to a study of what people do with new media; the desire behind such an assumption is often a belief that a more fundamental account of a phenomenon might be understood as objectively grounding subsequent inferences about observations of relevant phenomena — in this case, that phenomenon being new media objects.

Contemporary computers are formal machines designed to provide a syntactical specification for a set of formal symbols, the rule-governed means to manipulate these symbols, and the physical means to express these symbols using arbitrarily distinct formal systems.

The information stored on a computer is stored in a symbolic form, and numbers are just one possible way to represent these symbols. A loose analogy can be drawn with natural languages: when we have something in mind that we would like to express, we may write it down on paper so that it is expressed as chemical pigments and light waves, or we may speak it aloud so that it is expressed as air molecules and sound waves. If we say something aloud, it would not be correct to assume that the spoken expression is more or less fundamental than its written equivalent; if we want to understand natural language communication, it would not be correct to privilege the spoken word over the written word, or to focus strictly on how words exist in minds before they are communicated to others.

When we talk about information stored in a computer, we frequently talk about binary code being composed of 1′s and 0′s; the fact that we use the Arabic numerals “1″ and “0″ to represent this code is simply a matter of convention. TRUE and FALSE or ON and OFF work just as well as 1 and 0; the convention we use has little effect on how the stored information is structured so long as the convention allows exactly two states in binary opposition.

It is certainly possible to treat all the 1′s and 0′s involved in storing a software application on a computer (along with all the plain-language text unique to that software application, such as online documentation) as a single integer, represented by a very large base-2 number. However, the same 1′s and 0′s can also be viewed equivalently as instructing the computer hardware to move bits of information from one place to another; or, those 1′s and 0′s can be viewed as truth-value assertions in a long expression of Boolean logic.

In his essay, “The Practical Logic of Computer Work,” Philip E. Agre asserts that:

“The design of a computer begins with formalization — an intensive and highly skilled type of work on language. A computer is, in an important sense, a machine that we build so that we can talk about it; it is successful if its operation can be narrated using a certain vocabulary. As the machine is set to running, we too easily overlook the complicated reflexive relationship between the designer and user on which the narratibility of the machine’s operation depends. The corpus of language that was transformed in producing the machine, like any discourse or text, is necessarily embedded in an institutional and historical context, and the machine itself must therefore be understood as being, in some sense, a text.”

To describe a computer program as a long expression in Boolean logic shouldn’t be counter-intuitive: computer programs are, after all, written in artifical “programming languages.”

This means that computers are just as much logic machines as they are arithmetic machines. That the information computers store can be represented numerically is very much incidental to how the stored information is structured, and is neither a fundamental nor an axiomatic observation about how computers behave.

Why we so often talk about computers as though they are arithmetic machines (or fancy calculators) is in large part a matter of what sorts of patterns we are culturally taught to be sensitive to. A modern city-dweller well-acquainted with the cycles of the business day and traffic laws might easily see patterns in a contemporary cityscape which a farmer from 1800 might have to struggle to perceive. We are taught a good deal of arithmetic in public schools, but very little formal logic; we are furthermore taught science in the tradition of a Platonic-Pythagorean conception of nature, which holds abstract constructs to be the most fundamental description of the world. Since we know science gave us computers, why wouldn’t arithmetic therefore appear to be essential to the most fundamental description of what computers do?

Given, however, that computers are just as much logic machines as arithmetic machines, it is not at all self-evident why the numerical features of computerized information storage should be privileged in an analysis of how practical computers behave.

Furthermore, it is not unique to new media objects that they originate in a numerical form: musical notation, too, can be viewed mathematically, as can the design of architecture (or the motion of planets for that matter). In fact, the information contents of practical computers are hardly unique in their ability to be described mathematically, and their ability to be described mathematically is hardly fundamental to their operation. It is therefore just as problematic to describe computers as arithmetic machines as it is to describe the arithmetic features of new media objects as characteristic of new media.

It is perhaps more relevant to discuss contemporary computers in terms of the relative convenience with which they allow us to perform sophisticated manipulations of symbolic information.