Facebook’s recent purchase of Oculus VR–Palmer Luckey’s interactive virtual reality (VR) company and its head-mounted display (HMD)–has stimulated speculation about the next generation of consumer electronic entertainment. From the publicity, one might assume that this is a revolutionary advance. Interacting with electronic displays runs back, however, to technicians and operators adjusting equipment in responses to oscilloscope and radar displays up to a century ago, and to government and academic researchers researching and developing specialized interactive applications. During World War II, nearly one million American and British soldiers trained on Waller Gunnery Trainers using five 35mm projectors to create an immersive display.
In terms of innovating real-time, interactive and immersive audio-video environments for a mass market, however, the David Sarnoff Research Center (DSRC) in Princeton, N.J., along with many collaborators, pioneered developments in the 1980s and early 1990s. Since the DSRC and its predecessors at RCA Laboratories spearheaded the innovation of electronic television systems and CMOS microprocessors, this is not surprising. The ironic aspect rests in consumer interactivity’s beginnings in two other innovations, one of whose commercial failure helped sink RCA and one whose technical proponents had to overcome the indifference of two major corporations. The success of the engineers laid the foundations for DSRC’s work in virtual reality for consumers, which will be covered in the next issue.
Interactive VideoDiscs
Interactive video originated in two projects at the DSRC. First was the Capacitance Electronic Disc (CED), or VideoDisc, which debuted as a prerecorded playback medium for consumers in 1981; the second was a last attempt at developing a home computer before General Electric bought DSRC’s parent, RCA, in 1986. VideoDisc’s format was driven by RCA’s long experience with disc records for sound reproduction; their low cost of manufacture compared to magnetic or laser-read media; and the conviction that researchers could achieve practical reproduction of video as well as audio signals for the home consumer. The 12-inch, carbon-loaded, vinyl disc spun a spiral groove 450 rpm under a titanium-tipped diamond stylus; the changing distance between the depth of the groove and the stylus tip gave a variable capacitance that represented the recorded NTSC audio/video signals. Each rotation provided four frames of 3 MHz video in two interlaced fields, providing about as much resolution as VHS magnetic tape. The players debuted at $499, declining in cost to $149 by 1984; the videodiscs retailed for about $20 each.
The CED competed not only with the established VHS videocassette, but also with the more expensive MCA/Pioneer LaserDisc. Therefore RCA wanted to show that it was not only cheaper and longer playing, but that it was as durable as the other media and could also provide random access to its data. The vertical blanking interval between fields offered space for recording “digital auxiliary information” or DAXI code, and the potential for what a former DSRC researcher calls a “primitive type of CD-ROM.” After the DSRC’s Charles Wine showed that fields on the disc could be accessed through DAXI code near the end of 1980, staff at the Princeton labs and RCA’s Consumer Electronics Division in Indianapolis worked on storage and interactive techniques before the technology’s commercial introduction. In the early 1980s, Charles Dieterich, Todd Christopher, James Rustman, Michael Mindel, Rebecca Mercuri, and others found that, even after quadrupling redundancy to stabilize an analog, electromechanical format for digital readout, the CED DAXI spaces provided about 90 megabytes of data storage to accompany the video. This was huge and cheap compared to the 4K and 8K ROM available on chips or the megabyte-plus storage of 8-inch and 3.5-inch floppy disks. DAXI code provided information and instructions for the player’s microprocessor, including “freezing” a frame by recycling a spiral rotation and “kicking” the stylus back a groove to repeat. In 1981-82 Mercuri demonstrated the possibilities by operating an SFT-100 player and the standard videodisc Tut: The Boy King/The Louvre (RCA 01202) with a Z80 microprocessor, CP/M operating system, and an interactive program written in Basic.
A year later the DSRC and the Indianapolis factory combined video sequences with interactive choices in a hybrid analog-digital system that cost $88 in parts and fit on top of a television. Like the larger and more expensive systems experimented with at MIT’s Architecture Machine Group and Media Lab, it switched video sequences based on a player’s input, and overlaid data or graphics over video frames. In the fall of 1983 RCA began selling the $500 SelectaVision SJT 400 interactive player with its 32-button remote, and the first of seven interactive discs. Legendary jockey Willie Shoemaker hosted A Week at the Races (RCA 25001), where players picked horses and the machine randomly generated a winner from its assortment of race clips, while The Life and Work of Michelangelo (RCA 25003) was modeled on Mercuri’s demo and proposal.
The discs included a collaboration with Walt Disney Home Video, the DisneyDic of Mystery and Magic, “the first fully-interactive Disney VideoDisc for children!” The content is rightly characterized as “bizarre” by one DSRC engineer, and not just because the marketers and scriptwriters showed little understanding of the esthetic of the British murder mystery: Agatha Christie it’s not. With the crash of the videogame industry, however, RCA and Disney wanted to expand the audience and exploit the greater realism of recorded video on the CED. Middle-aged marketers guessed at how the Middle American audience they knew would understand interactive CEDs, and included instructions for their use within the presentation of the murder mystery. The Consumer Electronics Division and Disney apparently hoped that families would gather to play interactive discs much as they gathered to play board games.
A more promising approach appeared in the interactive CED demonstration, Hoyle’s Last Resort. Based on the film noir Lady in the Lake (1947), notable for being filmed almost entirely from the detective’s viewpoint, the player is the detective, examining the crime scene and deciding which witnesses and suspects to interview. DSRC’s Scott Marshall “developed a . . . language that allowed one to interact with a script on the computer as if playing the game with video” with all choices navigated by joy stick and a friendly GUI. Marshall recalled it being “a major production with a full Hollywood-style film crew, professional writer and actors, and filmed in 35 mm. We completed the disk and software for a 20-minute pilot which was demonstrated in many settings to RCA brass. Even Grant Tinker, then head of NBC, got to see it.”
In December 1983, RCA contracted with the country’s largest arcade company, Bally Midway Manufacturing, to provide “several thousand random access” videodisc players. Following the licensing and editing of game film from the National Football League, RCA issued a disc featuring the Oakland Raiders and San Diego Chargers, followed by the more popular rivals, the Dallas Cowboys and Washington Redskins. Two players chose one of eight defensive or eight offensive options from an on-screen playbook, after which the videodisc and display showed the outcome of the selections, complete with an appropriate soundtrack and animated graphics. Requiring no skills and needing repetitions to play a full game, the cabinet was Bally’s first to accept one- and five-dollar bills.
The interactive VideoDisc was successful enough that RCA incorporated it into its $5,000 computerized home entertainment center branded as “Dimensia,” a term for which Google asks, “Did you mean dementia?” An RCA-type “Control” jack on the back of late-model CED players enabled two-way communication with a microprocessor, as in the Bally system, Dimensia’s central control board, or, potentially, Coleco’s Adam computer. Despite positive reviews and responses from dealers and consumers, however, the interactive CED was not popular enough to save the VideoDisc system. RCA made the last of its 550,000 CED players in April 1984, six months before Dimensia’s introduction, and its 16,400,000th VideoDisc in June 1986, thereby closing out a $500 million investment in the future of home video.
Digital Video Interactive (DVI)
At the same time that RCA was winding down the VideoDisc, it was also closing out its last effort at innovating a home computer (HC). Initiated in 1982 by Art Kaiman of DSRC’s Digital Products Research Laboratory and the marketing director at RCA’s New Products Division, researchers explored ways that RCA’s expertise with color video could offer a competitive advantage for a tardy rival. At the time IBM and Commodore competed for supremacy in an exploding market. As a consultant later recounted, the “little HC was being pushed in many directions at once. . . . While each of those directions had merit, not even RCA had the resources to develop them all within the scope of a single product.” With the withdrawal of New Products’ support the following year, however, Kaiman’s researchers, led by Larry Ryan, began focusing on the processing of real-time, full-screen video from a CD-ROM at a broadcast standard of 30 frames per second along with stereo audio to a personal computer. In a world of simplified color graphics on the most expensive videogames and consumer computers, this would be a revolutionary advance.
The challenge was how to achieve it. The CD-ROM could stream 1.5 megabits per second. This rate was about 150 times slower than that required for streaming broadcast video and audio; even if the DSRC team compressed the data effectively, new processors were required to decompress and play it back on an IBM PC/AT. Michael Keith described finding an optimal solution through microcoding on the video display processor to produce quality video as “a team effort between the designers of the decoding chip, the engineers developing the compression methods and software, and the engineer writing the software for decompression (me). The way we all worked together creatively, not as three separate entities but all contributing ideas in all three of those domains, was one of the most satisfying and enjoyable aspects of the project.”
In the first quarter of 1984, as Arch Luther joined Kaiman in managing the project and Apple introduced its Macintosh, forty people in three teams developed the “Galactic Challenge” demo to show DSRC and RCA management a profitable future for their investment in digital home video. Using two Ikonas Graphics Systems to digitize and edit 900 frames of imagery and a DEC VAX computer to compress and uncompress them, the software group also applied a YIQ color pyramid compression algorithm to make almost seamless 360-degree landscapes. Using the VAX to run the demo as well, the group developed content including a princess and spacecraft to impress outsiders with the power and possibilities of digital, interactive video where one could pan and travel about realistic imagery and interact with the onscreen environment.
Having gained upper management approval, the team, whose technology did not gain its DVI name until ten days before its introduction in 1987, proceeded to develop the chip sets for real-time audio and video processing in tandem with an Intel 286 microprocessor. Other members wrote sophisticated functionality demos. These included the fast drawing of solid and textured polygons, manipulating video in a frame buffer, and a warping algorithm that enabled transformations and texture-mapping of images onto three-dimensional objects. The programmers worked with video producers and instructional designers to prototype programs, including auto repair, real estate sales, and interior design.
These were all mass-market industrial applications, however; what would people use interactive video for at home? The answer was not obvious beyond videogames, where Scott Marshall and others had switched from interactive CEDs to CD-ROMs. They collaborated with Activision and Gary Kitchen to digitize the sounds and cockpit of a Spitfire fighter plane and give players a tour over a computer-generated, 1930s English landscape. “The flight simulator was really just a copy of Microsoft’s . . . with war game challenges. . . . The first-person flying screen consisted of three layers: the world view outside the window, the cockpit window frame and instrument panel, and computer animated dials and indicators on the panel.”
This market was largely comprised of males aged twelve to 25, however, and cycled in a boom and bust pattern that concerned a corporation already cycling in the broadcasting and semiconductor industries. People who had grown up with television understood it and its cathode-ray tube display as a passive device before which people consumed programs. Would adult consumers embrace a cultural shift in relation to the electronic boxes in their homes, and interact with their monitors?
This was the unanswered question behind interactivity. To help answer it, the DSRC contracted with Lucasfilm’s subsidiary Lucasfilm Games to come up with a game that exploited the technology’s advantages. Perhaps Douglas Crockford was not the best choice for the job. He thought the videogame industry, which was in its third year of recession, “pathetic” and spent ten months wrestling with existential issues of home entertainment. What is interactivity? Crockford spoke of the paradox he saw in videogames: a joystick and a screen seem to provide unlimited possibilities, but practical limits keep players on one thinly disguised track or another. He argued that interactivity should create an emotional experience within the player, plausibly by repeating an earlier enjoyable routine, and called for an interactive experience that “Mom”–who was an archetype, not a stereotype, in opposition to young male gamers–could enjoy.
Crockford did not succeed in his quest to redefine interactive video at RCA. It did not help to initially nickname the engineers’ work “Compressovision,” criticize Luther’s audio compression, and propose, as one Sarnoff engineer described it, “a pleasant, flying dream-like application with no goal or challenge for the player. We were befuddled but wanted to put faith in anything associated with Lucas.” Nor did his final report’s mix of myth, philosophy, his five-year-old daughter’s interactive preferences, and whimsy prove persuasive.
Instead the DSRC found funding for production through a traditional venue: children’s education. With Children’s Television Workshop, the creators of Sesame Street, it developed “Words in the Neighborhood,” where video sequences coordinated with graphical overlays and synchronized audio allowed young children using a joystick to learn the similarities in words beginning with the same letter. More ambitious was “Palenque,” a Bank Street College of Education production based on the PBS series, The Voyage of the Mimi, and featuring the same young Ben Affleck. In this case he journeyed to the Mayan city and surrounding jungle to learn about archaeology and ecology. This required location filming and recording, taking one photo per step in what researcher Jeremy Pollack called a proto-Google Street View, and sometimes using a fisheye lens to generate seamless 360-degree views of some scenes. Graphical, textual, audio, and additional video overlays provided more information as the user, climbing the towers and walking through jungle, desired. It was a tour de force of what one could do in the educational market with interactive video on a PC, and academics investigating approaches to computerized education cited it frequently.
In the middle of these successes, General Electric Company (GE) bought RCA for the value of its assets. Neither the DSRC nor the interactive project was considered economically useful, and Kaiman, Luther, and their superiors argued vigorously for an opportunity to show off their technology to the computer industry before closing it out. In a decision as narrow as the one to preserve the labs, GE approved a presentation at Microsoft’s CD-ROM conference in March 1987. The auditorium was packed with over a thousand onlookers by Bill Gates’s introduction, word of mouth, suggestive marketing, and RCA’s reputation. Kaiman, Digital Systems Group leader Larry Ryan, and Luther projected and explained the DVI clips from the array of applications of full-motion, compressed video while a stereo classical soundtrack filled the air. Having concluded the demo of what Kaiman called a “new technology,” he called on the crowd to join them in creating a “new industry.” David Sarnoff, speaking to a smaller crowd in 1939 on RCA’s introduction of scheduled electronic television, would have been pleased. As all three men stood, they were greeted with thunderous applause that grew into a standing ovation.
Requests for more information poured in, the DVI team began releasing data file formats and compression techniques, and GE found a buyer for the technology in Intel. The technology was still some distance from commercialization with its three custom boards and VDP chips sticking out of the PC/AT; the semiconductor company saw DVI as a means of establishing a technical standard in a new field as it led the microprocessor industry in fulfilling Moore’s Law. By the turn of the millennium, Intel’s chips would have 80 times more transistors and DVI would be an integral component to fulfilling a dream of getting “a Pixar machine down into a board.” In October 1988, Intel bought GE’s intellectual property and 35 of its developers for $40 million, and moved them to its Intel Princeton Operation a mile up Route 1 in Plainsboro.
Nine months later, the group shipped its first DVI product, the $25,000, seven-board Pro 750 Application Development Program for PCs. Over the next five years, working closely with IBM and Microsoft, the group expanded and rolled out more sophisticated algorithms and software and won two Comdex awards for IBM’s ActionMedia II board set and i750 PB/DB chips. The Wintel dominance of personal computing did not, however, clear the field of digital multimedia competitors. DSRC’s success with DVI and Philips’ competitive work with CD-i (demonstrated in June 1987) led Leonardo Chiariglione of CSELT to start the ISO/IEC MPEG standards committee in 1988. Other related standards followed from Apple, IEEE, and consumer electronics manufacturers. Intel closed the Princeton DVI office in the summer of 1993, relocating willing staff to Intel groups in Arizona and Oregon. While those teams formed Indeo and its iterations of real-time AV codecs, the last of which were offered through Ligos for Windows XP and NT/2000, some veterans returned to the David Sarnoff Research Center in the fall of 1993. There they soon joined in the effort to develop the first virtual reality system for consumers: Hasbro’s Sliced Bread Project, or Toaster.
To be continued. . .
Acknowledgments
I am most grateful to Doug Dixon, Michael Keith, Arch Luther, Scott Marshall, Rebecca Mercuri, and Jeremy Pollack for their extensive recollections of the events described here. Any errors in this account are the author’s and corrections and documentation are welcomed.
References
Todd J. Christopher, U. S. Patent No. 4,309,721, “Error Coding for Video Disc System,” filed 12 October 1979, published 5 January 1982.
Todd J. Christopher and Charles B. Dieterich, U. S. Patent No. 4,419,699, “Digital Video on Recording System and Playback,” filed 16 November 1981, published 6 December 1983, priority 12 October 1979.
Douglas Crockford, “Quest into the Unknown,” DVI/LucasFilm 1985-86 notebooks transcribed on crockford.com.
“Design and Decorate: DVI Pilot Application” (Design & Decorate Inc., 1988), YouTube/Alexander Benenson.
Charles B. Dieterich, U. S. Patent No. 4,308,557, “Video Disc System,” filed 12 October 1979, published 29 December 1981.
“Digital Video Interactive Technology” (GE/RCA, 1987), YouTube/Alexander Benenson.
Douglas Dixon, Manifest Technologies/”DVI Technology, 1982-1992.”
“The DVI Story: Making it Happen in Seattle” (David Sarnoff Research Center, 1987), YouTube/Alexander Benenson.
William J. Hawkins, “Video Teacher,” Popular Science, February 1984, p. 110-4.
Tom Howe, CED Magic/Capacitance Electronic Disks.
“Intel DVI File Format Summary” (Encyclopedia of Graphics File Formats, 2nd ed., 1996).
Arch C. Luther, Digital Video in the PC Environment (Intertext Publications, 1989).
Arch C. Luther, Many Threads: The Saga of an Electronics Engineer (Luther, 2008).
Rebecca T. Mercuri, Computer-Interactive Modifications for the RCA VideoDisc Player, PRRL-82-TR-099, June 1982, David Sarnoff Library Collection, Hagley Library.
Michael J. Mindel and James C. Rustman, U.S. Patent 4,313,134, “Track Error Correction System as for Video Disc Player,” filed 12 December 1979, published 22 December 1981.
Kathleen S. Wilson, “Palenque: An Interactive Audio/Video Research Prototype,” in Colin Harrison, ed. Interactive Learning and the New Technologies (Berwyn, PA: Swets North America, 1988), p. 172-8.
