This article appeared in Theatre Design and Technology, Winter, Vol. 32, No.1, 1996, pp. 36-43.

A recent experimental production of Elmer Rice's The Adding Machine at the University of Kansas featured the premier use of virtual reality in a fully mounted theatrical production. In the process of merging the art of live performance and the technology of real-time computer generated graphics (virtual reality or VR) we looked for answers to many questions. Could this in fact be done, and how? Could VR technologies could help stage dramas with unusual or difficult scenographic problems? How can this new technological method of presentation help a modern audience gain new insights into a drama? Would the combination of these two different but very similar fields of endeavor serve to illuminate each other? How would an examination of VR lead to a better understanding of the theatrical art and how could a comparison to theatre practices allow for better design of virtual-worlds? How would the two practices have to be modified from their normal application to fit one to the other?

During the process of planning and mounting the production we faced several challenges, some practical, others artistic. The focus of this article is to outline some of the unique practical or technical aspects of the production in order that subsequent discussions of artistic and philosophical discoveries will be made clearer.

The over-arching technical consideration in the planning of this production was the need to devise a practical method of presenting virtual environments to a theatre audience. In order to find a suitable audience/actor/virtual-world interface, traditional methods of staging plays and presenting virtual realities had to be examined for any similarities that could be exploited and any conflicting differences for which compromises would have to be found.

"Virtual reality" is a trendy phrase, and the subject of numerous interpretations. Common to most definitions is that VR is a computer simulation of real or imaginary environments and that these simulations are "interactive", capable of being navigated or manipulated in real-time rather than being pre-recorded. Some definitions also include the idea of immersion, in which the images of the virtual environments are presented to the user in such an intimate manner that the real world can be disregarded.

The main similarity between VR and theatre , and the one that first suggested making the link between the two, is that both are dependent on this live, or real-time action. For theatre to remain true to it's form and not wander into the realms of television or film, it must be played live, not pre-recorded, with an immediate relationship between actor and audience. Likewise, VR must also be experienced in real-time, as it is generated, or it loses it's unique quality and becomes a member of the pre-recorded computer animation genre.

Another common characteristic of VR and theatre is that both occupy 3-dimensional space and more often than not, rely on some form of illusion to suggest the form of that space. Here there a great deal of leeway to find similar uses of real and illusory space. Just as theatre is comprised of wildly different styles of production and mechanisms of presentation, VR also has it's different styles. The best known VR interface, the head-mounted display, relies almost totally on presenting an illusion of an environment to a user's eyes. Other interfaces however, more closely resemble theatre in that they present a combination of real and fictive space to the user.

These interfaces such as the heads-up display, in which electronically generated images are over-laid onto views of reality, are often put in a sub-category of VR referred to as augmented reality or AR. CAVE environments are rooms in which the walls are made of rear-projection panels. Users can enter alone or in groups and view different computer generated vistas projected onto the walls. "Environmental" theatre productions seek to create the sensation of immersion in a similar manner. Cockpit interfaces, such as those used in flight or automobile simulators, place the user within a tangible space, or cockpit, from which an illusion of the outside world is viewed through windows. Compare this to a set seen through the portal of the procenium or to the illusionistic painted drops revealed behind the set.

A useful VR-theatre interface then must capitalize on these mutually defining characteristics by presenting live actors within 3-dimensional virtual worlds. These worlds must have the ability to be navigated in real-time as the drama progresses. This hybrid virtual performance space can be a combination of actual or illusory space and still remain faithful to the tenets of both theatre and VR just so long as it is presented in a live format.

Key differences in VR and traditional theatre practice proved to be the cause of some concern during the design of the interface. Theatre has typically been a communal experience. Its audience is a large group of people gathered together to share an experience, and through this sharing, the experience is heightened. Much of our present thinking about the nature of theatre assumes this to be true without being questioned. On the other hand, virtual reality most often is experienced by a single user, or a group of two or three. This is due mostly to the desire that VR systems be "immersive", giving users the illusion that they are within the virtual world. The head-mounted display is a good example of immersive technology that limits the system to a single user at one time.

Another contrasting element in VR and theatre is the matter of control. In the theatre it is usually assumed that the control of a performance rests with the performers, including the stage manager and crew members. The audience acts as relatively passive observers whose control is indirect, urging the performers one way or the other through their responses to the action before them. The interactive nature of VR, however, usually requires that the user who is viewing the scene be in direct control of the action. This interactivity is derived by navigating through the virtual world and often by manipulating objects within that world through special input devices such as joysticks or electronic gloves. But in a VR-theatre interface, who is the user? Is it the audience who views the results or the performers who are using the technology for their own ends?

It was clear that some compromises between the typical theatre production and the typically application of VR technology would have to be made to settle these differences. And, since the focus of the project was to create a theatre piece enhanced by VR and not to produce a VR artwork using theatrical methods, it was decided that in most cases the theatre conventions would take precedence over VR practices. Therefore, the interface for the VR enhanced production of The Adding Machine would have live actors. It would incorporate 3-dimensional spaces, both real and illusory. It would have a larger audience while retaining the illusion of immersion as much as possible. And, it would be under the control of the production personnel.

The Stage

The final design required a major change from the 1800 seat, orchestra and balcony, 40 foot proscenium configuration of the Crafton-Preyer Theatre on the University of Kansas campus (Figures 1&2). In order to put the audience in close proximity to actors and projection screens, the audience was turned 180 degrees and moved to the rear of the stage house. To enhance sightlines, seating platforms were built at a steeper rake, rising 1 foot every 3 feet horizontally. Original plans called for an audience of 150, but later revisions increased the number to 190. Black velour stage drapery was hung at the stage's cyclorama positions. This drapery then surrounded the audience behind and on either side, eliminating views of the stage house, focusing the audience on the playing area and enhancing the illusion of immersion within the virtual world.

The playing area was a platform 2 feet in height, positioned 4 feet from the first row seating. It was 12 feet deep, 20 feet wide at the rear, and tapered to 14 feet wide at the down stage edge. It was covered in black carpet to create a quiet, dark neutral floor and to minimize the bounce of lights onto the projection screen.

Directly behind the platform was a black rear-projection screen. As the sceme required that the light from the projectors to be polarized, it was necessary to use a rear projection screen. Light reflected from a front-projection screen will not remain polarized. Only more expensive "metalized" or "aluminized" front-projection screens are capable of properly reflecting polarized light. This 40 foot wide RP screen extended 10 feet on either side of the 20 foot wide platform. This extra screen space was angled toward the audience to somewhat surround the actors with the screen. Although the stock RP screen used was 25 feet tall, the area actually used for projection was 14 feet tall.

On each down-stage corner of the screen was a follow spot mounted atop a 12 foot tall platform. The platform was masked on its down stage side and remained open beneath, providing the main actor's entrances to the stage. The follow spots provided a major source of illumination during the production. Standing on either side of the acting area, they were capable of lighting the actors with little bounce onto the projection screen.

Mission Control

Behind the projection screen could be found a bewildering array of equipment (Figure 5). Early in tech rehearsals, one crew member dubbed the bank of computers, mixers, projectors, and cameras "Mission Control".

At the heart of the formation (Figure 4) were two Dukane overhead projectors, each equipped with a NView LCD projection pad. Two projectors were needed in order to project graphics with an illusion of three-dimensionality. One projector was assigned to right eye views only, and the other to left eye views. Each was then polarized with the "direction of polarization" in opposition to the other. These opposite polarizations were done in such a manner as to match the polarization of the special glasses given to each audience member. Using the glasses, the audience could then see the projection from each projector only with the appropriate eye and the 3-D illusion was formed.

In fact, the operating principle of LCD projection pads decrees that they are always polarized. So, the task of correctly polarizing them was accomplished not by adding additional polarized filters. Instead, an optic material known as a 1/2 wave plate or retarder was used on one of the LCD pads to "turn" the polarization 90 degrees, then being opposite to its mate. Finally, because we were unable to move the projectors far enough from the RP screen, the image was enlarged to the desired dimensions by adding plastic fresnel magnifiers a few inches above the LCD pads.

Because the material we were to project was to come from multiple sources, the LCD pads were connected to a pair of digital video mixers. Each of these Videonics MX-1 mixers were capable of switching between four video sources and offered a vast selection of fades and wipes. The most important feature of this mixer was its ability to super-impose one scene over the other. Each mixer required its own video monitor to set-up and preview scene changes and special effects.

The essential graphics that were to be projected onto the RP screen were the virtual-worlds. Each scene had it's own computer modeled environment that could be traversed in real-time in correspondence with the actor's movements. These worlds were created and run on a PowerMac 7100/AV with special prototype software loaned by the Virtus Corporation of Cary NC. This software is identical to Virtus WalkThrough Pro but has the added feature of stereographic screen display. This added feature, which will soon be available on commercial releases of Virtus software, renders right and left eye stereo views in several formats including side by side pictures.

Because the AV equipped Mac is capable of running two separate monitors with it's two video cards, it was possible to enlarge the side by side stereo pictures in such a manner that each picture filled one of the monitors. Then the two seperate monitor outputs were converted to standard NTSC video signals and sent to the two video mixers. The signal from the Mac AV card was converted to video internally, while the signal from the standard video card required an external video encoder, in this case a Focus L-TV Portable Pro.

A second Power Mac computer was also connected to the video mixers via a video encoder. This computer was used to generate titles at the top of the show, to create a few special graphics used during segues and to run a computer generated version of the "Head" character in the graveyard of scene 6. This disembodied head has only three lines and seemed a likely vehicle for testing out the idea of a "virtual agent" within the production. The head was animated using the Interface application which runs as an auxiliary to HyperCard. The head was rendered on a green background which allowed it to be superimposed over the graveyard virtual-world by the video mixers' chroma-key features. These graphics were not rendered in stereo but simply split and sent to the two mixers. However, when super-imposed in front of the 3-D backgrounds the graphics seemed to float over the scene.

Chroma-key special effects were used again to add live actors directly into the virtual worlds. Off-stage and behind the black velour surround, a small TV studio was erected (Figure 3). Actors worked in front of a green chroma-key screen, listening for their cues from the stage. Necessary lighting was supplied on cue and a video camera captured the actors and sent a signal to the video mixers to be chroma-keyed into the virtual worlds. In this manner, characters appeared not only in front of the scene but often within it. At the end of scene 2, Zero's Boss appears in this manner. By zooming the camera, the boss was be made to "grow" during the scene, his face finally filling the screen. The party guests of scene 3 also appeared via the camera, their images bouncing around the screen in time with the hectic party conversation. The jury of scene 4 was also superimposed on the scene. Actually played by only two actors, the jury's twelve large faces were generated through the use of a novelty multi-image lens positioned in front of the camera.

In order to fill the visual field and increase the sense of immersion within our VR stage, it was necessary to utilize the smaller, 10 foot wide, sections of projection screen flanking the platform on both sides. 3-Dimensional slides were used for this purpose. Side by side stereo graphics were captured onto 35 mm slides. While the graphics were created with the same software that was used to create the virtual worlds, the subject matter of the slides varied. In some scenes, the slides presented a continuation of the virtual worlds. In others however, an alternate scene was depicted; one that somehow commented on, or indirectly added to the underlying theme of the virtual-worlds. In the disjointed party scene for example, rather than show a continuation of Zero's expressionistic living room with it's floating furniture, one side screens showed a close-up of a champagne bottle and the other an odd Magritte-like figure with no head under his hat.

Due to the availability of only one high powered slide projector for each side screen, the 3-D effect of the slides was achieved in a rather unorthodox manner. For each projector, a pair of horizontally mounted mirrors was used to intercept one half of the projected stereographic image (one eye's view) and bend it inward, overlapping the other half. When each half of the stereo image was polarized appropriately, the audience saw a 3-D image.

Sound was also run with computers located in "Mission Control". While the show's relatively simple sound plot could have been run from the theatre's existing sound facilities, it was decided to remain faithful to the computer generated nature of the virtual environments. A Mac LC was used to run background music from a digital CD-Rom, and a Mac PowerBook was used to play the few spot sound effects used in the play. Microphones in the offstage TV studio and sound outputs from the computers were connected to a small mixer which was connected in turn to the house sound system. Special speakers were mounted in the front of the playing platform at floor level.

Virtual Theatricality

The audience, wearing polarized glasses, was seated in the darkened stage house and surrounded on three sides by black velour. Before them stood a platform, covered with a black carpet. Immediately behind the platform, a black rear-projection screen hung, flat in the center, but angled toward the audience as it extended from either side of the platform.

As the action of the play began, the screen seemed to disappear and was replaced by a series of virtual worlds. Eight different scenes including a bedroom, an office and a graveyard, appeared as seeming 3-dimensional spaces behind the actors. In some instances, objects seemed to emerge from the screen into the playing area. Often, live actors on-stage conversed with images of other actors who were working live in front of a camera backstage. An eerie soundtrack played under the action throughout.

Behind the screen, the crew watched the action on a video monitor. As the actors moved about the stage, an operator known as the Virtual Environment Driver, or VED, steered the observation point within the virtual worlds. Thus the actors appeared to "travel" within the world, or the audience's vantage point was changed. As Mr. and Mrs. Zero enter and move about the bedroom in scene 1, the scenery moves with them, moving to the right and left, closer and further away and panning around the room. When Zero climbs into bed, the audience's view point was raised to the ceiling where they could look down on the bed. The actor playing Zero then stood in front of the 3-D image of the bed and seemed to be lying upon it, sleeping soundly.

This movement within the virtual worlds took many forms. While it sometimes represented physical movement of the characters through the setting such as Zero and Shrdlu rising out of the ground in the graveyard, more often it reflected Zero's changing state of mind. In scene three, when a dazed Zero returns home after killing his boss, the furniture is floating in mid-air and the walls of his living room are spinning and tilting wildly. In jail, Zero daydreams of past excursions to the beach and the audience travels there with him, only to be wrenched back to his cell as the daydream ends.

Conclusions

At its conclusion, I considered the project a success. Just as in any production, there were moments that were effective and others that were not. But as is usually the case in experimental theatre, we learned as went through the process. Many of the carefully planned uses of the new technology failed to achieve the effects we desired while other uses that occurred to us in the midst of the work achieved the project goals admirably. Certainly, the physical structure of the actor/audience/virtual-world interface was successful if not perfect.

Future computers will be faster and therefore the virtual-worlds will be rendered with more detail. Better projectors will make the images brighter and more vibrant. New technologies will generate 3-D views without the need for special glasses. Nevertheless, in this production the interface did achieve the goals set out for it. VR was used not for its own sake, but in service of the play. Through it, we were able to execute scene changes without a single pause in the action of the play. We were able to stage complex scenes and special effects that would have been extremely difficult and expensive otherwise. Perhaps most importantly, we used this new technology to present a work of expressionist theatre by depicting the world as seen through the mind's eye of the main character. Our scenery was fluid, metamorphasizing with Zero as he endured his many trials, denials, enlightenments and his final doom.

It is this perspective, the inner reality of the characters, that is so vital to expressionist work and is so often lacking in productions limited to rigid conventional scenic practices. Virtual reality can unlock many scripts, realizing potentials that have been thwarted by production techniques that, being bound by muslin, wood and steel, can not keep pace with the imagination of playwrights. Furniture can float, walls can spin, the audience can look down from the ceiling, dancers can fly and people can walk through walls. Elmer Rice described such dramatic devices in an interview in the New York Times of April 1, 1923, I, 2.

"The author attempts not so much to depict events faithfully as to convey to the spectator what seems to be their inner significance. To achieve this end the dramatist often finds it expedient to depart entirely from objective reality and to employ symbols, condensations, and a dozen devices which, to the conservative, must seem arbitrarily fanatastic."

It is not to be suggested that this interface is the only, or the best, possible way to use virtual reality in a theatre setting. Only that it was the one that seemed most effective for the production of The Adding Machine that we had envisioned. The technologies known as VR are versatile enough to adapt to the inevitable new production concepts that will follow. Already, I have started work on a production that will incorporate more advanced VR technology and will likely bear little resemblance to The Adding Machine. Contemplating the possiblilties is thrilling.

For more on The Adding Machine: A Virtual Reality Project, see the WWW site, URL http://ukanaix.cc.ku.edu:80/~mreaney

This investigation was supported by University of Kansas General Research allocation #3963-60-0039, The Virtus Corporation, Bank IV of Lawrence and the USITT Heart of America Section.