Forza Motorsport 2: Artificially Intelligent
An In-depth Discussion on In-game AI with Turn 10 Game Director Dan Greenawalt.
03.02.07 // 16:00
Every racing game talks about Artificial Intelligence (AI) innovation. But when you look at the evolution of racing game AI over the past decade, AI innovation really hasn’t kept pace with advances in graphics and audio. That shouldn’t be a revelation to anyone. The truth is it shouldn’t be a surprise either. Graphics and audio get immediate benefits from more powerful hardware and optimization tricks – and we get more powerful hardware all the time. AI really isn’t about the hardware as much as it is about the approach. The best AI work usually takes place in different technology arenas far away from games -- such as robotics and search engines. In many ways, AI tends to be more of an academic field rather than an entertainment one. In this enlightening Q&A with Turn 10's game director Dan Greenawalt, we go deep into how Forza 2's AI behaves, and why it'll set the bar on racing game AI this generation.
Q: In Forza Motorsport 1 on the Xbox, you guys had this "Drivatar" technology that seemed pretty ambitious at the time, and certainly on paper, but in practice, it had shortcomings in the gameplay department...
Dan Greenawalt: I’ve worked on a lot of racing games over the years with several world-class developers all over the world. I can honestly say that the original Forza Motorsport featured a revolutionary approach to AI. That’s not to say that it was perfect by any means. Our AI limitations were not caused by an outdated approach. Rather, most of our AI issues are the result having such a new approach that we struggled to get it under control. On the up side, because of this approach, our AI has room to grow where other systems will be forever capped.
Q: So how does racing game AI usually work, in a more conservative game?
DG: In many racing games, the AI doesn’t use the same physics parameters as the player. The AI opponents are more like “rocket powered shopping carts”, blindly following a line on the track. The AI cars don’t have the same friction values or moment of inertia as the player. The power is applied to the body or the tires as an impulse that is scaled by the gear ratios. Suspension animation and body roll is handled by the same physics engine as the player, but doesn’t necessarily apply to the vehicles' friction. This approach has its benefits -- it’s very lightweight on the CPU, it’s fairly predictable and easy to tune for gameplay, and it’s very easy to “rubber-band” to the player. This approach has got some drawbacks too -- the AI may not be able to drift or yaw naturally and the AI can spin out the player, but can’t be spun out itself, the AI is fast where it should be slow and slow where it should be fast.
Q: So what was the new approach you guys took with Forza 1 to curb the status quo on racing game AI?
DG: In Forza Motorsport, we use Drivatar technology to drive our AI opponents using the same physics system driven by the player -- it doesn’t cheat. The Drivatar technology was developed by Microsoft’s Research department in Cambridge England. There are some definite benefits to working in such a large and diverse company -- resources like this just aren’t available for other game developers. They didn’t start out developing it for Forza -- far from it. At first it was more of an academic AI project for a group of robotics and search researchers. They chose to develop it for a racing application because of their passion for Formula One racing. Turn 10 was originally contacted as more of a test bed for their research. We had an internally developed simulation engine and a graphics renderer. Also, we had already hired a PhD robotics researcher onto our own dev team to do our in-game AI. He worked very closely with the research team in Cambridge to get their Drivatar technology integrated into the game. As we got their code better integrated into our game, it became clear that they were onto something cool, new and sorta scary.
Q: Scary?
DG: Drivatar technology is a Bayesian learning system. It observes and remembers racing techniques and generalizes what it’s learned across different cars and tracks. As it gets more training, it generalizes less and optimizes more. This means it can be trained by observing humans, through drawn splines and overrides or by automating laps with random variants. However, it can also be unpredictable and learn bad habits. It can learn to cut corners, early or late apex different areas of the track and even check-brake. While this is cool and interesting, it can be a game designer’s worst nightmare. In many ways, it’s more like a child then a traditional AI system. You teach it more than you script it.
Q: So how much "schooling" did the AI have in Forza 1?
DG: In the original Forza Motorsport, we didn’t get the system integrated early enough to realize it’s true potential. We had barely gotten our heads wrapped around its capabilities when we released it out into the world. As a result, I would say it was trained up to an adolescent stage of development. It was still doing things that we had to scratch our heads at -- "why did it do that?" Sometimes it was a bully. Sometimes it was tentative. We could make it drive really fast, but we knew it could drive faster with better training.
We’ve often discussed allowing it to continue to learn after it’s been released -- basically using the player community to make the AI smarter. There are a lot of cool gameplay possibilities there. But truthfully, I’m still too scared of the system to unleash it like that. I worry that it would learn some deviant behavior or just get too fast to compete against. It’s always going to be in the back of my mind, but I think we have to wait until we understand the technology and all its capabilities better.
Q: So "les enfant terrible" returns for Forza Motorsport 2. What have you guys done to leverage the potential of Drivatar and still reconcile it with game design and balance?
DG: For Forza Motorsport 2, we assembled a really strong internal team to improve and rebuild our AI for the Xbox 360. We’ve now had a full product cycle to play with the system and wrestle the Drivatar technology into submission. In the past year in particular we’ve learned a lot, fixed a bunch of bugs and done a ton of training. That said it still does things we don’t expect. It’s still learning. It’s still growing. At this point, I would say it’s in its late teens. It’s faster. It uses the car better. It knows the tracks better. It wheel-to-wheel races much better. It doesn’t cheat and it drives the same complex physics engine as the player.
Q: What are some specific improvements you guys have made to the AI in Forza 2?
DG: There are many, but here are some highlights...
- The AI takes a simulated test drive in every car before it races it to better learn the car’s capabilities. This allows it to understand how to get the most out of each individual car. It does NOT drive a Civic the same way it drives a Ferrari. This is at the heart of our ability to allow the player to hire AI drivers to drive the player’s custom tuned car. Even with upgrades, the AI will quickly learn the car’s quirks and drive its wheels off.
- The AI has a more accurate understanding of the physics engine than in the original Forza Motorsport. It is much better at estimating how much grip it will have available for turning, braking, accelerating at all times, and is better at using this to make a fast run through each turn. This may be one of its strongest aptitudes right now -- it has a very accurate prediction of its own performance.
- In the original Forza Motorsport, the AI had a very rigid understanding of the shape of each track, that was technically accurate, but not necessarily the way a race car driver would see it. A chicane, for example, was often seen as a rapid series of alternating corners. But a real driver can recognize that there’s a much smoother line through there. For Forza Motorsport 2, the AI has a better understanding of the driving line within the literal track edge.
- It’s much more cognizant of the cars around it and openings for it to pass. There are cases where the AI in the original just didn’t recognize it when another driver made a mistake. It has evolved to take better advantage of these kinds of opportunities. Moreover, it’s become really fun to wheel-to-wheel race against. This is one of the changes in the AI I’m most happy about.
Q: You mention hiring AI to drive for the player in single-player races. Does that mean each one will have unique characteristics?
DG: Yes. One of the coolest new features of our AI system is Drivatar personalities. We have persistent AI drivers that move through career with you. Each one of these drivers has a unique skill level, aggression tendencies and risk tolerance. The top drivers carve a consistently fast line around the track, while the lesser drivers are less consistent and don’t always use the whole car. Some of the drivers are extremely anxious to pass, darting inside during braking zones when there’s the smallest of available windows (Juan Pablo Montoya, anyone), while others are content to wait for a clear opening. This creates a lot of interesting situations during a race, as fast but tentative drivers may get bottled up behind slower cars, only to be passed by less-skilled, more aggressive drivers. We also tried to train tendencies for the real-world race car drivers we feature in the game. Without pulling up debug info or watching a replay, it’s much harder to see the system at work with the real-world drivers. Since they are modeled on professionals, there is much less observable variance -- they’re just fast.
Q: We're obviously still tuning AI for the game in the final days before ship. Anything else you'd like to add for our readers?
DG: Currently, our biggest challenges come in the area of player prediction. The Drivatars see the player, but they aren’t very good at predicting what the player is going to do. This leads to some pretty humorous situations. At times, the AI looks like it’s getting frustrated with the player. Logically, I know that’s not the case, but it’s hard not to interpret its behavior that way. I find the whole system sort of hypnotic to watch. We have debug hooks that allow you to see what the AI is thinking as it drives. I can’t escape the whole creepy HAL thing -- I know I’m working too much when I think the AI is watching my back. But in a way, our AI is watching… and learning…
// Comment on Forza Motorsport 2's Artificial Intelligence //