Creating Engaging AI
by Brian, UpRoom Games Founder
Like most programmers, my intuition when approaching a problem is to find the “best” solution. This is usually pretty obvious with a lot of typical programming tasks. When sorting a list for example, the best solution is one that actually sorts the list and sorts it as fast as possible. I think this intuition, while usually admirable, is actually what causes a lot of problems when it comes to programming game AI, since the “best” solution for developing an engaging artificial intelligence is anything but obvious. Does “best” mean lifelike? Smart? Predictable? Unpredictable? Unfortunately, the answer to all of these questions is both yes and no depending on the context. If you are working on an RPG, you probably want your NPCs to act intelligently, to have behavior patterns that are both predictable and unpredictable and to engage with the player in a human-like way. If you’re making a strategy game on the other hand, you probably aren’t too concerned with human-like behavior, but you are probably concerned with providing a sufficiently intelligent challenge. However, neither of these solutions are actually being applied to the right problem. We are trying to solve the problem of the “best” AI, but we really should be trying to solve the problem of creating the most engaging AI, which is counterintuitively often not the same thing.
For example, when you play a game of chess against a computer, there always seems to be something lacking. When you win, you’re often left with the feeling that if the difficulty had been turned up a bit higher maybe you would have lost, but such an arbitrary switch can suck the joy out of victory. When you lose, it feels like inevitability. You couldn’t process as much information as the computer so what chance did you have? I think this issue is the reason why most chess computers these days are marketed as being training programs for actually playing against a human. If you lose to a training computer, well, you should learn something! If you win it means that training must be paying off! With this mindset, the different levels of difficulty represent your progress, rather than arbitrarily limiting the capacity of the program you’re against. In this case, the most engaging AI is clearly not one that always wins, no one would continue against a “training program” that they could never beat. The most engaging AI in this case is one that feels like it’s teaching you something, one which you can replicate the moves of in new scenarios and be rewarded for it. Modern chess programs are quite good at this, but it took a surprisingly long time to get where we are.
The discrepancy between “best” and engaging can be seen in a lot of AAA RPG and shooter games. In these games, combat often feels quite fun, challenging and like a skill you can develop. If you really take a look at these games, however, you’ll notice that your enemies often behave quite stupidly. Often, hordes of baddies will line up Hollywood style in order to be slapped down by the player in a timely fashion. This obviously makes no sense from the standpoint of actually creating intelligent opponents, but makes a lot of sense from a player engagement point of view. Being overwhelmed by lots of enemies all attacking at once isn’t very fun. Having the illusion of being overrun while somehow managing to come out on top, on the other hand, is extremely fun and feels very satisfying.
In both of these cases, the most engaging AI is, objectively, not the “best” at utilizing the mechanics of the game it is in. For me at least, this idea can be difficult to grasp. Not from a comprehension standpoint, I think it makes a lot of sense, but when I’m wading through a sea of code trying to program AI for a game it’s pretty easy to lose sight of the difference between engagement and proficiency when the programmer brain wrests control from the designer brain.
So how do we figure out what it means for our AI to be engaging? This obviously is game dependent but I think there are a few questions we can ask to figure out an overall strategy for developing an interesting AI in most cases.
What do we want our player to spend their time on (the core gameplay)?
How do we want our players to feel while engaging in the core loop?
What can our AI do to reinforce our core gameplay and feel?
What can our AI do to break our core gameplay and/or feel and how do we avoid it?
Let’s use a hypothetical single player FPS as an example for applying these questions. For our purposes, imagine we’re making a simple single player FPS where the player shoots baddies in order to get better guns and then shoots more baddies, progressing through some sort of narrative.
-What do we want our players to spend their time on?
We want them to be shooting baddies and moving forward through our story / level. They will ideally get better at shooting baddies and should be getting new things that let them shoot baddies better regardless.
-How do we want our players to feel while engaging in the core loop?
A bit open ended, but in general we want our players to feel challenged and like they are improving as well as moving forward.
-What can our AI do to reinforce the core gameplay and feel?
Get shot. As counterintuitive as it may sound, our core gameplay loop doesn’t really involve the player dying. It’s a consequence of not getting better at the baddie shooting, but in this hypothetical game dying isn’t really part of our loop. However, in order to reinforce the feeling of challenge it needs to be somewhat difficult for the baddies to get shot.
-What can our AI do to break our core gameplay and how do we avoid it?
Not get shot or, alternatively, get shot too easily. How do we avoid it? We need to make sure the baddies are shootable, but we want our player to work for it. Baddies should engage the player occasionally but not all at once, and provide ample time for the player to shoot them.
With these questions answered, we can design a rough draft of our AI. We want baddies that are good at getting shot, but that sometimes try to not get shot. In an effort to not get shot, our baddies will probably try to attack the player to force them to adapt, but not so much that they won’t get shot relatively soon. For most people this is a really counter intuitive AI, since the entire design is focused on allowing the player to shoot the AI agent, even though ostensibly in the scenario our game is portraying the baddies would be trying not to get shot. Obviously this is a bit of an oversimplification since there can be a lot of nuance here, but it’s a good starting point for really thinking about what the purpose of your AI really is. Often, I find that what I initially think the AI should do and what I realize the AI should really do after asking myself these questions wind up being very different. This is just the tip of the iceberg for game AI, since there are many more complex considerations that need to be taken into account and then there’s the not so small job of actually programming the thing. I wanted to conclude with a quick case study of our upcoming game: A Token War. The AI in A Token War is one of the most important components of the game and really made me confront the problem of creating the “best” AI vs. the most engaging AI. I hope this post provides some insight into high level AI design for games. I would go into more detail about the low level implementations but I’m really not a very good AI programmer, so I’ll just stick with my strengths.
A Token War, Case Study (because of course)
For an overview of A Token War, check out our intro video. Also feel free to Wishlist A Token War on Steam!
When designing the AI for A Token War, I really tried to craft an AI that felt both challenging and fun, whether or not I succeeded is up to you to judge. I started by answering the questions I laid out previously. Let’s see what sort of skeleton we can come up with for an AI by answering these questions.
-What do we want our player to spend their time on (the core gameplay)?
We want the player to spend their time battling against the AI (and winning) in a sequence of progressively more difficult battles. The player should get rewards that enable them to perform better in future battles.
-How do we want our players to feel while engaging in the core loop?
We want our players to feel like their victory (or defeat) is a direct consequence of their choices, both tactically and based on how they built their Collection.
-What can our AI do to reinforce our core gameplay and feel?
Our AI needs to behave in a manner that feels consistent. Erratic behavior means that player strategy can be thrown out of balance, which we want to avoid. The AI needs to be active and moving towards an obvious goal, allowing our player to anticipate moves and craft their own defense / offense. Lastly our AI needs to be reasonably beatable.
-What can our AI do to break our core gameplay and/or feel and how do we avoid it?
If our AI rolls over and dies or just smashes the player our game-feel is completely broken. Also, if the AI never uses new Units or tactics the core loop will get stale. We can avoid these by striking a balance between proper tactics and more aggressive play, while continually giving our AI new tools to use in order to vary the tactics presented to the player.
With the all important questions answered, we can see we need an AI that is competent but not oppressive, gets stronger as our player progresses, prioritizes aggressive tactics over passive behavior and can be, to an extent, anticipated. Breaking things down even further, we wind up with the following design pillars for the AI in A Token War. These are the foundation on which the battle experience within the game is built.
Enemy Tokens prioritize attacking over positioning.
Being more aggressive often feels more active. When the AI plays patiently, even if it is often more advantageous, players tend to feel as if they are alone in the game and the AI is “being dumb.” Through testing it’s clear that patience is often best for the AI, since by raw power it usually has an advantage. However the more aggressive approach engages players more since they feel put under pressure.
The AI only looks a few moves ahead
Sometimes it feels like the AI has a grand plan that is coming to fruition. In fact, it’s exactly the opposite. When the AI looks too far ahead it feels oppressive rather than deliberate. Only looking a few moves ahead but acting consistently often feels like a plan to the player, without the oppression of actually enumerating many many moves into the future.
Player stronghold position influences aggression. A central placement makes the AI more aggressive towards the Stronghold while a more defensive placement makes the AI prioritize destroying Units and clearing a path.
Placing your stronghold centrally is an aggressive move and the AI reacts to this. Statistically, the AI would get more wins by playing even more patiently against a central Stronghold since the human player is the one that has to move to achieve victory. However, when the enemy reacts strongly against an aggressive opening the battle feels more scrappy and feels more on a knife edge, which is generally good for player engagement.
The AI puts heavy priority on summoning more Units.
Giving the player more things to smash is usually a good idea, since smashing things is fun. Hey, whatever works.
AI aggression is resource based
When the AI has more resources it’s generally more aggressive. This means that over time as the opponents forces grow the AI will feel more active and engaging.
Putting this together, have we avoided the Chess computer problem? To answer this, there are two main things we want to consider.
If the player loses, it should feel like they had the agency in the situation and it was not inevitable, or it should feel like a learning experience.
If the player wins, it should feel like a result of their choices and not arbitrary computer limitation. They should also have a sense of improvement
Starting with the first point, does the player (generally) feel as though a loss was through player agency and not through silicon inevitability? I think the deckbuilding nature of the game coupled with the setup phase gives us a good head start in that direction. As far as the AI is concerned, coupling significant aggression with only looking a few moves into the future makes loss a definite possibility, but not an inevitability. The second point I think is a little trickier. Since the computer is not always playing optimally there is a risk that victories don’t feel as earned (although it should be said that even though it isn’t playing optimally, it’s still playing pretty well.) To help combat this the AI usually has more raw power available than the player. This is sort of a quick trick to boost apparent AI strength, but I think it works reasonably well in this case. Furthermore, the player can be set fairly far back if they lose too often in A Token War. This added consequence to loss can actually have a significant positive effect on the feeling of victory. Even if a battle doesn’t feel as hard as it possibly could be, the fact that not losing preserves your momentum and carefully cultivated collection can add a lot of weight to victories. While this isn’t strictly an AI design choice, it is very closely related and is an example of the external factors which can heavily influence AI design choices.