Your Engineering Heritage: The Long Road to Consumer Virtual Reality, Part II

In 2004, the Tuck School of Business at Dartmouth College published a case study detailing Hasbro’s innovation of electronic games. Through internal documents and executive interviews, Tuck’s team explained how the world’s largest toy company successfully created a division of virtual entertainment to complement its offerings of toys and dolls. Before Hasbro Interactive, however, there was the Sliced Bread Project (SBP), an ambitious gamble on consumer virtual reality games in the early 1990s. The SBP is a more significant initiative because Hasbro’s subcontracts with leading game developers and its abandonment of relevant patents and applications seeded the VR environment that made possible Oculus VR’s latest iteration of consumer VR technology some twenty years later.

The story begins not with Hasbro, however, but a tiny company in New York City, Abrams/Gentile Entertainment. AGE began as a marketing and design company known for its Rambo movie posters when it began expanding into Rambo toys in 1986. Nuclear engineer Chris Gentile joined his twin brothers to handle the engineering issues in manufacturing figurines; he soon turned their focus to reviving the videogame industry. With game sales at three percent of their peak, Gentile argued that AGE could take gaming into three dimensions with its own system. After licensing the entertainment rights to Jaron Lanier’s $8,800 VPL DataGlove II, based on Thomas Zimmerman’s flex-sensor patent, Gentile worked with VPL’s engineers to simplify its operation through conductive ink and ultrasonic sensors. The new glove was connected to a computer console mixing live and digital graphics, and LCD goggles and stereo surround sound to create the 3-E, “the next thing after 3-D entertainment.”

With the success of the Nintendo Entertainment System in 1986-87, however, AGE could not persuade American toy makers to risk innovating an incompatible system. Instead Mattel offered to license the glove and worked with Gentile and AGE’s contract engineers to turn the glove into an NES-compatible product that could be sold for $88. Mattel sold 654,000 of the PowerGloves in the six weeks before Christmas 1989 and another 350,000 before its Japanese partner went bankrupt, the lack of programs exhausted its value, and users discovered its inconsistencies.

Nonetheless, AGE now had an entrée to developing Gentile’s vision of “Total Virtual Reality” for consumers. In 1990 he began explaining how architects could immerse clients in their designs using a head-mounted display connected to an improved PowerGlove: “We went directly from research to the toy industry and now we’re working our way back to practical applications.”

When the costs of production did not fit with what architects were willing to pay, Gentile returned to the consumer market. It was an audacious shift when VPL’s “Eyephone” cost over $10,000 and Provision’s Vision 100 VR work station cost $70,000. Yet the PowerGlove’s success led to a partnership with Steve E. Tice, who had worked in military and NASA 3D. Together they built a fixed, desktop VR console system with a dot-matrix display that attracted interest by Texas Instruments in developing a processor, if a proper LCD and game manufacturer could be found.

In 1992 Gentile approached the renowned president of Hasbro’s Toy Division, Larry Bernstein, backed by the reputation of his PowerGlove, Texas Instruments’ standing in electronics, Tice’s experience in VR, and the prospect of working with the David Sarnoff Research Center (DSRC), the birthplace of LCDs and color TV. Hasbro’s reputation rested on G.I. Joe and board games, but Bernstein and CFO Al Verrecchia were acutely aware of the rise of videogames as competitors for children’s attention and parents’ budgets. The company had spent $20 million developing an electronic interactive game, Control-Vision, that never went to market in the mid-1980s. That experience and unfamiliarity with the appeal of digital entertainment left some in Hasbro reluctant to try again. Commercial pressure from within the industry by rising rival Mattel, and from outside the company by the allure of videogame profits and a survey showing a market against Nintendo and Sega, overcame the resistance.

Gentile’s confidence in the future of consumer VR matched Bernstein’s spirit, and so began Hasbro’s Sliced Bread Project. With Bernstein’s approval, Gentile visited the David Sarnoff Research Center in Princeton early in 1993 in search of a cheap LCD and, in project manager Norm Goldsmith’s words, “discovered that all of the technology he ever dreamed of was gathered in one place.” The DSRC was extremely receptive to the prospect of working for Hasbro. Its five years of guaranteed contracts from General Electric Company with other former RCA divisions had expired; the more it could offer Gentile and Hasbro in developing the VR system, the more technical staff time would be underwritten. By the time DSRC began Phase II that December of what it called Project Toaster, it had become lead contractor and systems integrator for hardware, processor, and software development. Toaster would deliver to Hasbro a complete consumer VR system ready for manufacture, to retail at $199, in less than two years.

Creating a virtual reality system for consumers on such a schedule created challenges similar to those in creating Digital Video Interactive, or interactive VideoDiscs. Engineers would have to generate stereo, wide-angle images of the highest resolution possible for the head-mounted display in real-time. What applied to vision applied to sound. The images had to synchronize with the accurately tracked movements, in any direction, of the head and equipped hand in real-time as well. The closeness of the eyes to the displays required special optics to reduce the perception of proximity and replicate the perception of depth. The lenses, displays, and related electronics should not weigh much to keep the HMD comfortable to wear.

Finally, the presumed lack of limits on a player’s movements demanded an exceedingly powerful graphics processor that could react almost instantly to sudden changes in the player’s movement or view, and handle the highest resolution possible from the displays. Too much delay in the system’s response to head or hand movement would spoil the effect and generate dizziness or even nausea in the user. To deliver this in games offering some sense of virtual immersion for under $200 was hardly realistic, but the DSRC and its partners came closer than anyone else for the next twenty years.

Having gained permission after the fact from Thomson, DSRC’s consumer electronics client, to contract with Hasbro, CEO James Carnes put Arthur Firester in charge of Toaster. He was given the secluded basement of Building 3 to work in, not knowing that RCA’s first computer, Typhoon, was built there after World War II. Firester also had with the power to recruit staff from anywhere in the company. Firester recruited Goldsmith, a semiconductor engineer with a talent for complex project management, to organize the project and budgets. Three weeks before Christmas 1993 Goldsmith delivered a detailed PERT chart outlining how the DSRC would run the project as prime contractor. He and Gentile got along well in part because Gentile “was one of two or three people I’ve ever worked with who understood that the PERT chart (actually the activity diagram) was the only way to manage a complex project.”

Goldsmith broke the project into hardware, electronics, and software groups. Douglas Dixon ran the latter, newly returned with Michael Tinker and others from Intel’s shuttered Princeton DVI operation. After ten years’ experience in computer video R&D, Dixon managed the smoothest part of the project. His group established flow charts for the artists’ work on images, 3D models, audio, and bitmap fonts; their conversion into Toaster formats; and their linkage through GNU to the programmers’ libraries, codes, and programs. It also organized and standardized the application program, authoring templates, authoring system libraries, and platform system libraries. Charles Wine designed “an elegant and efficient operating system.”

Meanwhile Tice, Gentile, and Turner Whitted recruited game programmers–notably Electronic Arts–to develop twelve prototypes, which involved training them to think beyond Nintendo’s side-scrolling to exploit 3D interactivity. The appeal of getting a million dollar, Silicon Graphics RealityEngine platform helped; the SG salesman retired on his commission. By the summer of 1995, five games were demonstrated on the complete system of hardware and software, and tested by the 16-year-old son of one of the group leaders.

The hardware challenge lay in the displays. In the early 1990s, color LCDs that fit inside a helmet were expensive. To provide a 320 x 200 image in stereo would take up a third to half the cost of the entire system. In addition the drivers for the head-mounted display (HMD) required a counterweight on the back of the helmet. Only in June 1995 did Dietrich Meyerhofer and Herschel Burstyn file for a patent on a “brilliant optics design” to split the video displayed by one LCD into two images through several injection-molded lenses. With distortion cancelled out by pre-distorting the lenses, this innovation promised to save on weight, the price of an LCD, and the cost of manufacturing the HMD. In addition, a subcontractor integrated spatialized audio with the video, which allowed different points of view; the handsets were designed to be touch-friendly when out of view; and serialized video through DSRC-designed and -built integrated circuits meant that only two wires were needed to bring signals to the headset. Taking the low-cost dictum too far when selecting the requisite piece of ferrite, however, made the calibration of the head tracker difficult.

Bruce Anderson led the electronics group in an endless search for an affordable graphics processor (“Curly”), other ASICs (“Moe” and “Larry”), and DRAM. The processor was to handle 500,000 textured polygons per second, when SGI’s RealityEngine processed only 300,000. After Texas Instruments stood its ground on providing the necessary chips in return for production guarantees, Hasbro contracted with Jez San’s Argonaut Games to optimize an ARM core processor for VR. Perhaps encumbered with Nintendo work, Argonaut in turn subcontracted it to Ben Cheese Design’s Rob MacAulay, who had led the creation of Nintendo’s Super FX processor for 3D games. The challenges were such that Anderson brought in Phoenix VLSI Consulting and visited both groups twice a month, often accompanied by Whitted. For memory, Rambus’s new RDRAM proved significantly superior to more expensive SDRAM in transfer rates, a crucial factor in processing polygons. To exploit this speed, DSRC pioneered in pipelining a bidirectional, multiplexed, high-speed bus. Creating a pipeline with a flow matched to the Rambus memory was the key to reducing latency between physical motion and electronic response. With the end of the project in July 1995, however, the patent applications related to this problem were abandoned.

What happened? After all, Hasbro had hired Frog Design, known for its work with Apple, to design the production-model headset, and placed an order for the molds to be fabricated in Portugal. It negotiated RDRAM production prices with Hitachi and was in negotiations with Panasonic and Matsushita for additional production as OEMs. As importantly it had already released an ad for what was now the xScape VR system and its five games; a shorter, edgier video with an electrically demonic icon alerted young males to the next generation of gaming now called “Rush.”

People outside Hasbro portray different parts of an elephant in searching for an answer. Yes, the price had crept up–$299, $399, topping out at $499–despite the monocular LCD breakthrough, with the introduction delayed to the holiday season of 1996 at earliest. Focus groups indicated that Hasbro’s VR system was worth the higher price. One LCD or two, Toaster included the cost of a TV set in its 3D VR system against Nintendo, Sony, and Sega’s increasingly powerful consoles, and in Firester’s words, “Sony unambiguously scared them.” If only the Hasbro executives had understood that the profits in the videogame industry came from the games, not the hardware, or the difference between the annual drop in electronics costs compared to the inflation rate for plastic . . ..

The decision was not an easy one for CEO Alan Hassenfeld. He succeeded his late brother Stephen in running the family-owned company and was well aware of Stephen’s interest in successfully expanding into electronic games. His number two, CFO Al Verrecchia, despite his remorseless reliance on numbers and Sliced Bread Project’s slim projected profit margins, continued to support it. So too did Sandra Schneider, whom few at the DSRC respected despite her long experience in gaming software and marketing pop culture. She had succeeded Bernstein in running the SBP and contributed to its delayed introduction by changing the software format from cartridges to CD-ROMs.

At the executive council, however, they were the only supporters. The business plan anticipated selling two million units in five years, with twelve percent profit only by Year 4. Hasbro had spent $45 million in three years and was due to spend another $22 million on advertising Rush. Tens of millions more would go toward further innovation and inventory. Keeping it or killing it would cost money that would displease the board of directors. Hassenfeld had played the latest games and could see Hasbro gain in prestige and talent with this cutting-edge system. After polling his inner circle, however, Hassenfeld said “I’m killing it.” He had already made up his mind to move into digital electronics with a safer bet: creating the Hasbro Interactive division to centralize efforts and focus on adapting Hasbro’s popular games for the small screen.

From Hasbro’s perspective, and perhaps the wider world’s, the CEO was correct. The company profited handsomely from its shift, and neither rival Mattel nor the videogame industry followed on with a successful VR system for consumers. . . . Until next year, when many expect Oculus VR to start selling its Rift system to the public. If it succeeds the way its backers hope, they can thank Hasbro, the David Sarnoff Research Center, and the many individual participants who spread what they learned from Toaster about Virtual Reality hardware, software, and the uncertain interface between them in that black, head-mounted display.

Acknowledgments

I am most grateful to Bruce Anderson, Jim Carnes, Doug Dixon, Art Firester, Chris Gentile, Norm Goldsmith, Jeremy Pollack, Steve E. Tice, and one who wishes to remain anonymous for their extensive recollections of the events described here. Any errors in this account are the author’s and corrections and documentation are welcomed.