Category: StateMachine

April 6, 2020

AI For Beginners

Finite State Machines

Finite State Machines

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Finite State Machine Challenge

This tutorial provided a challenge to complete and then provided a solution. The challenge was to create a state where the npc would retreat to an object tagged as “Safe” when the player approached the NPC closely from behind.

My Approach

Since they had a State enum named Sleep already that we had not worked with yet, I used that as the name of this new state (I started with Retreat, but then found the extra Sleep enum so I changed to that since I assumed it would be more consistent with the tutorial). Similar to the CanSeePlayer bool method added to the base State class for detecting when the player is in front of the NPC, I added an IsFlanked bool method here that worked similarly, but just detected if the player was very close behind instead of in front. I used this check in the Patrol and Idle state Update methods to determine if the agent should be sent into the new Sleep state.

In the Sleep state itself I used similar logic from the Pursue state for the constructor, so I set the speed to a higher value and isStopped to false so the NPC would start quickly moving to the safe location. In the Enter stage I found the GameObject with tag “Safe” (since this was set in the conditions for the challenge to begin with) and used SetDestination with that object’s transform.position.

The Update stage simply checked if the npc got close to the safe object with a continuous magnitude vector check, and once it got close enough, it would set the nextState to Idle before exiting (since Idle quickly goes back to Patrol in this tutorial anyway, this is the only option really needed).

Finally, the Exit stage just performs ResetTrigger for isRunning to reset the animator and moves on to the next State (which is only Idle as an option at this time).

Their Approach:

Most of what they did was very similar, although they did make a new State named RunAway instead of the extra Sleep State, so I could have stuck with Retreat and been fine.

Notable differences were that they checked if the player was behind them by changing the order of subtraction when performing the direction check (changed player – npc to npc – player) where I just had the angle check use the negative forward vector of the npc instead of the positive vector. These give effectively the same results, but I liked my approach better since it matched up with what was actually being checked better.

They also set the safe gameObject immediately in the constructor, where I was setting this in the Enter stage of the State. Again, this basically gives the same results in most cases, but I think their approach was better here just because the sooner you perform and set that FindGameObjectWithTag the better I think just to make sure it has access when it does need it.

Finally, for their distance check to see if they had arrived at the safe zone, they used a native NavMeshAgent value, remainingDistance. I used the standard distance check of subtracting the vectors and checking the magnitude, so these again both give similar results. Mine is more explicit in how it is checking, and the NavMeshAgent value is just cleaner, so these both had pros and cons.

Summary

This was a nice challenge just to work with a simple existing finite state machine. Similar to what they mentioned in the tutorial, I think setting the safe object in the static GameEnvironment script and just pulling from that (instead of using FindGameObjectWithTag every time the NPC enters the Sleep/RunAway State) would be much more efficient. Also just to help with checking states and debugging, I added a Debug.Log for the base State Enter stage method that just returned the name of the current State as soon as it was entered each time. This let me know which State was entered immediately when it was entered, so this also helped show me when the states were entered, so this was a very nice state machine check that only required a single line of code.

April 1, 2020

AI For Beginners

Finite State Machines

Finite State Machines

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Finite State Machines

Finite State Machine (FSM): conceptual machine that can be in exactly one of any number of states at any given time.
Represented by a graph where nodes are the states, and the paths between them are the transitions from one state to another. An NPC will stay in one state until a condition is met which changes it to another state.

Each state has 3 core methods: Enter, Update, Exit

• Enter: runs as soon as the state is transitioned to
• Update: is the continuous logic run while in this state
• Exit: run at the moment before the NPC moves on to the next state

State Machine Template Code (Can use core of this for each individual state):

public class State
{
public enum STATE
{
IDLE, PATROL, PURSUE, ATTACK, SLEEP
};

public enum EVENT
{
ENTER, UPDATE, EXIT
};

public STATE name;
protected EVENT stage;

public STATE()
{ stage = EVENT.ENTER; }

public virtual void Enter() { stage = EVENT.UPDATE; }
public virtual void Update() { stage = EVENT.UPDATE; }
public virtual void Exit() { stage = EVENT.EXIT; }

public State Process()
{
if (stage == EVENT.ENTER) Enter();
if (stage == EVENT.UPDATE) Update();
if (stage == EVENT.EXIT)
{
Exit();
return nextState;
}
return this;
}
}

Creating and Using A State Class

State class template (similar but slightly different from last tutorial, with some comments):

public class State
{
public enum STATE
{
IDLE, PATROL, PURSUE, ATTACK, SLEEP
};

public enum EVENT
{
ENTER, UPDATE, EXIT
};

// Core state identifiers
public STATE name;
protected EVENT stage;

// Data to set for each NPC
protected GameObject npc;
protected Animator anim;
protected Transform player;
protected State nextState;
protected NavMeshAgent agent;

// Parameters for NPC utilizing states
float visionDistance = 10.0f;
float visionAngle = 30.0f;
float shootDistance = 7.0f;

public State(GameObject _npc, NavMeshAgent _agent, Animator _anim, Transform _player)
{
npc = _npc;
agent = _agent;
anim = _anim;
stage = EVENT.ENTER;
player = _player;
}

public virtual void Enter() { stage = EVENT.UPDATE; }
public virtual void Update() { stage = EVENT.UPDATE; }
public virtual void Exit() { stage = EVENT.EXIT; }

public State Process()
{
if (stage == EVENT.ENTER)
Enter();
if (stage == EVENT.UPDATE)
Update();
if(stage == EVENT.EXIT)
{
Exit();
return nextState;
}
return this;
}
}

Notice that the public virtual methods for the various stages of a state look a bit awkward. Both Enter and Update set the stage to EVENT.UPDATE because you want them to be setting stage equal to the next process called, and both of them would look to move that to Update. After entering, you move to Update, and each Update wants to move to Update again until it is told to do something else to Exit.

They also started to make actual State scripts, which create new classes that inherit from this base State class. The first example was an Idle state with little going on to get a base understanding. Each of the stage methods (Enter, Update, Exit) used the base versions of the methods from the base class as well as their own unique logic particular to that state. Adding the base methods within just ensured the next stage is set properly and uniformly.

Patrolling the Perimeter

This tutorial adds the next State for the State Machine, Patrol. This gets the NPC moving around the waypoints designated in the scene using a NavMesh.

They then create the AI class, which is the foundational logic for the NPC that will actually be utilizing the states. This is a fairly simple script in that the Update simply runs the line:

currentState = currentState.Process();

This handles properly setting the next State with each Update, as well as deciding which state to use and which stage of that state to run.

It turns out running the base Update method at the end of all the individual State classes’ Update methods was overwriting their ability to set the stage to Exit, so they could never leave the Update stage. They fixed this by simply removing those base method calls.

Summary

Using Finite State Machines is a clean and organized way to give NPCs various types of behaviors. It keeps the code clean by organizing each state of behaviors into its own class and using a central manager AI for each NPC to move between the states. This also helps ensure an NPC is only in a single state at any given time, to reduce errors and debugging.

This setup is similar to other Finite State Machine implementations I have run into in Unity. The Enter, Update, and Exit methods are core in any basic implementation.

Novemeber 18, 2019

Beginner Programming: Unity Game Dev Courses

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – Beginner Programming

Finite State Machines

Building the Machine

This part of the tutorial has a lot more hands on parts, so I skip some of the sections for note purposes when there is not much substance to them other than following inputs.

A key in finite state machines is that an object can only ever be in exactly one state at a time. This helps ensure each state be completely self contained.

Elements of a Finite State Machine:
Context: maintains an instance of a concrete state as the current state
Abstract State: defines an interface which encapsulates behaviors common to all concrete states
Concrete State: implements behaviors specific to a particular state of context

To get started in the tutorial, they created a public abstract class PlayerBaseState which will be the abstract state for this example. PlayerController_FSM is the context. They note that while in this case all the abstract state methods take the PlayerController_FSM (the context) in as a parameter in this case, that does not necessarily have to be the case for the general FSM pattern.

Concrete States

It is noted that the context in a FSM needs to hold a reference to a concrete state as the current state. This is done in the example by creating a variable which holds that of the type that is our abstract state, which is PlayerBaseState in this case. They then create a method called TransitionToState which takes a PlayerBaseState in as a parameter. It then sets the currentState to that parameter state, and then calls the new state’s EnterState method (all states have this method as it is dictated by the abstract class they all implement). This determines what actions should be done immediately upon entering this new state.

Example:

public void TransitionToState(PlayerBaseState state)
{
currentState = state;
currentState.EnterState(this);
}

The tutorial also shows a way to take control of the context’s general Unity methods and pass the work on to the concrete states instead. This example did this with Update and OnCollisionEnter. The abstract state, and in turn, all of the concrete states, have their own Update and OnCollisionEnter method. The context, PlayerController_FSM, then simply calls currentState.Update(this) in its Update method, and currentState.OnCollisionEnter(this) in its OnCollisionEnter method, so that the current concrete state’s logic for these methods are used without flooding the context itself with any more code.

Since it is necessary that your context has some initial state, they do this by simply calling the TransitionToState method within the Start method and entering the IdleState. IdleState is the initial state for this case.

Beginning the Implementation

Important benefits seen using this system:
While working on the concrete classes themselves, we never needed to go back to the PlayerController_FSM class (the context) to modify any code there. The entire behvior is handled within the concrete states and is abstracted from the character controller (the context). Setting expressions was much easier as no checks are needed and this can just be set in the EnterState method of each concrete state.

It is already clear that this method removes a lot of boolean checks from the overall code, and helps organize the code by ensuring any logic about a state is contained within the class for that state itself (with less bleeding into the code of other states).

Continuing the Implementation

It is worth noting that PlayerController_FSM holds a reference to every concrete state except the spinning state. This was done because they actually have the jumping state create a new spinning state on transition each time it is invoked. They apparently do this so that the local field for rotation within the spinning state is reset to 0 each time it is called, but it seems like there would be other ways to do this that seem less wasteful (such as resetting it to 0 when exiting the state). I am also not sure if this is intended behavior, but the spin also immediately cancels upon contacting the ground (resetting the player rotation to 0) with this setup, where as in the previous behavioral setup the spin completed even if the player contacted the ground.

Module Conclusion

Benefits of FSM:

• More modular
• Easier to read and maintain
• Less difficult to debug
• More extensible

Cons of FSM:

• Take time to setup initially
• More moving parts
• Potentially less performant

Just something very notable with this approach, it seems much harder for me to break than the naive implementation. If I spammed key presses (like pressing jump and duck a lot) with the naive approach, sometimes I could break the system and have the player stuck in the duck position or be ducking while jumping. I have not been able to break it at all with the full FSM setup, which makes sense since transition behaviors solely exist within the states themselves so these inputs cannot be jumbled in any way.

SUMMARY

Using state machine systems appear way easier to use and build on than the “naive” approach of basic boolean behaviors (with lots of if statements and boolean checks). Not only was I very excited about how much easier this appears to work as a scalable option, it also just worked better and more cleanly when it was all put together.

The other version had small bugs that would pop up if you spammed all the different action keys (such as getting stuck ducking or being ducked in a jump), which were possible just because the key presses would get recorded before reaching the bools or if statements that should be telling them that they are not proper options. These very separated states make that type of error impossible as it is only concerned with a single state at a time.

This type of system just seems much cleaner, more organized, and less error prone than what I have done before and I am very excited to try and build a system like this for my own project (for both players and enemy AI).