US Army, University of Southern California ink pact for research center
This is NOT a Game
By Ben Delaney © 2000

This article originally appeared in IEEE Computer Graphics and Applications January/February © 2000.

US Army PFCs Beth Shapiro and Jason Washington are laying low. Hidden in hip-deep grass 100 meters away from the enemy’s HQ, they are in big trouble. On a reconnaissance mission, they have gotten separated from their squad. They shouldn’t be so close to the enemy, they shouldn’t be alone, and they should their butts get out of there fast.

As they start to creep backwards, they hear a twig snap, and turning to face the sound, find themselves looking down the muzzles of several Kalishnikovs, and behind those muzzles, the angry faces of the enemy. Captured, they are marched hands-up towards the nearby building as enemy troops appear all around them. They are in deep trouble. As they turn to each other with dismayed, fearful glances, the lights go out and they hear a voice tell them, in English, "OK. Washington and Shapiro, you’re dead. Take off your gear and wait in the ready room."

Whether being captured on their training mission is worse than being captured in the field is just one of the thoughts going through their minds at that moment. But at a deeper, gut level, they are having much the same reactions that they would have if they had really been found by enemy troops while on a scouting mission. They’re scared, they are sweating, their pulse and respiration rates are elevated, their guts feel loose. About the only physiological difference is that they don’t have any wounds, and they won’t be tortured by the enemy.

Training with realism
Training exercises with this degree of realism have always been the goal of the armed forces. To a large extent, the US military has succeeded in creating the training simulations needed to turn out the world’s most successful fighting forces. But the cost has been staggering. And with weapons and tactics changing faster than the weather, and budgets shrinking, yesterday’s planning, acquisition, and design procedures have been increasingly less able to provide what is needed when it is needed.
ApacheApacheThese two images was captured from the USC Information Sciences Institute "Helicopter" demonstration that pits helicopters against tanks. The simulation allows a participant to navigate a helicopter in three dimensions over desert terrain similar to Desert Storm. Other helicopters and the tanks are not scripted but act independently in response to the participant's actions and changing conditions. Photo courtesy of USC.

The US Department of Defense (DoD) has been aware of this issue for several years, and has been actively looking at new ways to do things. The Army Research Lab (ARL), the Defense Modeling and Simulation Organization (DMSO), and the Simulation Training and Instrumentation Command (STRICOM), among other organizations, have been looking into simulation technology for years, and have sponsored several noteworthy projects, such as the award-winning Omnidirectional Treadmill developed under STRICOM sponsorship by Virtual Space Devices (Minneapolis, MN), and the important work done on simulation sickness by ARL.

But DoD is not satisfied. They spend a lot of money on simulation for training, but it just doesn’t seem quite right. And when the DOD specialists see the games their kids are playing at home, a little nagging voice keeps asking them why their multi-million dollar simulator is less engrossing than the $39.95 Nintendo game in their living room.

At the Modeling and Simulation: Linking Entertainment and Defense workshop in October, 1996, the DOD made an effort to understand and connect with the gaming and film businesses. At this meeting, which I attended along with more than 50 other people from the military, game, VR, film, and location-based entertainment (LBE) sectors, the discussions focused on identifying ways that the entertainment technologies could be put to use by the military. The overriding assumption was that if Electronics Arts (for example) can produce a multi-player, networked battlefield simulation game that they can sell for less than $50, they must know something that the DOD has yet to discover. DOD might spend millions to arrive at a similar level of engagement and value. The entire meeting had an air of twenty questions. The DOD folks seemed sure that the entertainment folks were holding out on them, while the entertainment people couldn’t understand why the military establishment couldn’t see the obvious. (By the way, their secret is: the game people cheat.)

Creating Show Tech
Flash forward to August 17, 1999, Los Angeles. In the main ballroom of the University of Southern California’s (USC) Davidson Conference Center a hundred journalists, 10 TV crews, a phalanx of Army Brass, headed by the Secretary of the Army, Louis Caldera, the President of the University, Steven B. Sample, representatives of Hollywood headed by Jack Valenti, President and CEO of the Motion Picture Association of America (the people who rate all the movies), the deputy Mayor of LA, the (now departed) CEO of SGI, and, by live video link, the Governor of California, are gathered to hear the announcement of a $45 million dollar contract between the US Army and USC. What is going to be done with that money? Just the foundation of a center, the Institute for Creative Technologies (ICT), to encourage film, game and other entertainment types to share their secrets with the Army. On that warm summer afternoon in LA, the future of simulation-based training was probably changed forever.

establishing the USC Institute for Creative TechnologiesSecretary of the Army Louis Caldera, on the left, and USC President Steven B. Sample sign the contract establishing the USC Institute for Creative Technologies. Looking on are Jack Valenti, president and CEO of the Motion Picture Association of America; Rick Belluzzo, former chairman and CEO of SGI; Los Angeles Deputy Mayor Rockard "Rocky" Delgadillo and Lon S. Hatamiya, secretary of the California Trade and Commerce Agency. Photo courtesy of CyberEdge Information Services. This "historic partnership between the US Army and California high-tech and entertainment industries," as California governor Gray Davis put it, "will not only strengthen military readiness, but will add to entertainment technology." It is, he added, a "landmark venture."

Secretary of the Army Caldera provided some of the bottom-line justification. "The cost of a live fire exercise for a single armored Bradley Fighting Vehicle is a little under five thousand dollars. In a simulator, it is eleven dollars." But he sees more than just financial benefits. "For the Army, [entertainment technology] is not only a medium that can help tell the story of the American soldier, as in Saving Private Ryan, but a medium whose technology applications can help revolutionize the way we train, equip and prepare the American soldier. And that’s why we’re so excited about this venture."

"Traditional Army simulations were boring and not quite engaging. The Army wants enlivening and energizing simulations to train their military. They want the Hollywood kind of story-telling capabilities in their simulations," says Jim Korris, recently named Creative Director of the new Center.

But there are significant challenges to overcome before the ICT becomes a productive contributor to Army training programs. Chief among them is the issue of cultural disparity. Melding the Army culture – spit and polish, precision and accuracy, and high costs – with Hollywood’s good-time, high-stakes, profit-driven attitude may be tough.

Richard Lindheim is the newly-appointed Director of the ICT, and until recently was Paramount’s Executive Vice President for Television, where he was responsible for their Star Trek franchise. He talked to me about this issue. "Yes, you’re absolutely right [about the cultural differences]. I had this experience at Paramount. You go through a learning curve and your objectives are often different. Often the creative person has an idea and a scientist turns it into reality. We want to encourage that collaboration. The purpose of the Institute is to encourage that by working on specific projects." He added, "We have the opportunity to be in contact with the best and brightest in entertainment and put them together contact with the best and brightest in computer technology." That, he said, is going to result in some exciting and important developments in what he calls "Show Tech", which he defines as the connection of entertainment and technology that is analogous to the connection between entertainment and business that we call show business.

ICT Technical Director, William Swartout, offered more on this question. "We have two groups that come at things from very different points of view. There is a possibility of conflicts, but also a great possibility for synergy."

He added, "Hollywood is all about creating a reality that’s as you expect it to be, as opposed to the way it really is. By and large, the engineering approach has been to try to recreate things as faithfully as possible. From the entertainment side, we create things that are perceived as reality." The Army needs what Hollywood has to offer, he explained, to make their simulations more believable. Because, when the student believes what is happening is real, when that student is affected at a gut level, deep learning takes place. Combining the two cultures may make this happen. Mistakes are expected, because knowledge comes from error. Swartout’s mission is to conceive new ways to do things. "They really want us to be thinking outside the box," he said, "some things will work, some won’t."

Jack Valenti seems to get it. A decorated combat veteran, he said that what he called "the binary era" is "as important as Gutenberg". "This will result in a new heightening of training for our brave young men and women," he proclaimed.

Jack ValentiJack Valenti gets it. Here he discusses a flight simulator with a couple of pilots.
Photo courtesy CyberEdge Information Services.

Caldera concurs. "The Army will benefit from this research in a number of ways. It will enhance the realism and thus the value of the individual, crew-served and networked training simulators that we use to train our soldiers. It will permit our soldiers to do enroute mission rehearsals immersed in high fidelity imagery of the actual terrain to which they are deploying with very real shape and character content that will help them accomplish the mission. It will significantly enhance our leader-development processes in which we use complex, interactive simulation for large-scale war-fighting exercises. It will also enhance our weapons acquisition process by allowing us to test new doctrines and advanced weapons concepts and prototypes in synthetic environments that are populated with intelligent agents and future threat challenges. It will even help our recruiting, as the Army goes more hi-tech, giving young Americans a chance to learn by using technology they are attuned to and leaving them with skills they value."

Open to all
The ICT’s programs will not be under classified status, so the projects and progress will be open for all to see. Some of the possible projects include:

  • Creating new, high resolution/high speed digital cameras. The Army wants to photograph things on test ranges, like missiles in flight. Now they use CCD cameras and conventional film systems. The issue here is speed. There is a need for faster film, faster sensors, and high speed memory. Plus, they need a huge dynamic range. Of course, film makers could use this same technology for low light and other unusual situations.
  • Development of future combat systems. These would be very different from current systems, and most likely would combine robot and manned vehicles in ensemble. One problem is simply visualizing such a system. What would it look like to a soldier in the field, and to her commander? How will the human and artificial intelligences work together?
  • Improving game technology. How do you make games truly interactive and still provide the didactic message? The question is not how to make people shoot better. We know how to build shooting games and train people how to shoot. We need techniques for training people how to think in complex, dynamic, and foreign environments. Training for missions where everyone looks unusual to the soldiers, and where the language is unintelligible, is an increasingly important task, and a very difficult one. The Army now hires more than 200 actors used to provide "local" populations for training exercises. The actors play local villagers, partisans, reporters. The Army would like to replace them with intelligent avatars.
  • Task training with intelligent, artificial instructors. At the news conference announcing the Institute we saw a number of demos of potential projects. One showed an intelligent avatar, called STEVE, which is described as a "pedagogical agent for Virtual Reality", by its creators in the computer science department at USC. STEVE helps students learn complicated tasks by demonstrating them in a virtual environment, asking questions, and providing information. The system is a prototype of the sort of training the Army, and industrial companies around the world, expect to be using regularly in the next decade.
Steve: a softbotSteve: a softbotCreated by the USC's Information Sciences Institute, Steve is a completely autonomous softbot - a robot made of software - who lives in a fully 3-D virtual engine room. He provides explanations, answers questions, gives demonstrations, offers hints when you're stumped, never makes a mistake and never tires.

Photo courtesy of USC.

ICT has picked out a home, an existing building in Marina del Rey, at the beach not far from USC’s downtown LA campus. Staffing is expected to top off at about 50 people in a year or two. They will occupy three floors of offices, labs, and a dramatic demo center, which is being designed by the Star Trek Art Director, Hermann Zimmerman.

ICT has an interesting and challenging mission. As Michael Macedonia, Chief Scientist and Technical Director at STRICOM put it, "the Army is interested in training and the technology it requires. Story telling is essential to training. The best story teller are in Hollywood. Think about the first 20 minutes of Speilberg’s Saving Private Ryan. That was magic. We don’t know how to do that."

With the resources the Army and USC are committing to the Institute for Creative Technologies, I suspect they’ll figure out how to recreate that magic.

_back to top_