Category: GOAP

August 19, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Parts 14 and 15

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Adding Goal Challenge

Adding Goal Challenge

My Approach

This tutorial starts with another challenge to implement a new GAction. This time it is the GoHome GAction for the patient to return to a transform tagged “Home” after they receive treatment. This was rather straight forward as I created this new GAction that just had a Preperform method setting the target to the tagged gameobject “Home” and putting in the Postperform method that the TreatingPatient world state should be reduced by 1 (since they have finished being treated).

Once this GAction was applied to the patient, it needed to connect to the previous action by having a precondition of “isTreated” and I added an after effect of “isCured”. The “isCured” state was also added as the new secondary subgoal of the patient GAgent to ensure they would move entirely through the chain of actions. I did initially just add “isCured” as a 3rd goal (keeping the 2nd goal of “isTreated”) but this created a strange behavior where the agent would take one extra step of actions and return to the hospital entrance after returning home.

Their Approach

They named their new subgoal “isHome”, which I think is important to note because I think with a system like GOAP you should keep in mind that the actions should be somewhat isolated for their specific purpose and you should at least be cognizant of chaining the actions together too much for what you think their intended purpose may be and recognizing when to keep them more encapsulated as their own action. It makes sense that the patient’s true goal is just “to get home” and getting cured of their ailments is just an obvious way to us to achieve that goal given the settings, but being cured is just a major step towards simply getting home.

They also added an extra step of states for the patient agent. I simply connected the previous action to the new action by matching the GoHome action’s precondition with the GetTreated after effects (“isTreated”), but they actually had the GetTreated action modify the state of the agent to add “isCured” and used this (“isCured”) as the precondition for their GoHome action.

While this does not seem necessary in the current state of the tutorial, I think this makes sense to add this step when developing a real GOAP system because there could be several ways for a patient to be cured (such as simply waiting it out if it is a minor ailment like a cold). So this approach may make more sense when building a proper system, which again, is good to note for the future if building a system like this for your own project.

I ran into an issue where the patient agent was not leaving the waiting room, and it turned out this was because I still had the “isTreated” intermediate subgoal commented out because it was necessary for my approach to work. Adding this subgoal back in (along with the additional “isHome” goal) allowed the agent to work properly again.

Errors

They ran into a problem with the tutorial (that I also replicated) when they turned the spawner back on to add many patients to the system. Many of them began congregating at the hospital entrance, which was due to the fact that “isTreated” was one of their subgoals and it was being immediately met because the first patient treated added “isTreated” to the entire world state so every patient thought it was already treated.

This led to them pointing out a rule of thumb where you do not want any of the subgoals of the agents to match a state that is directly added to the world state or the agent’s beliefs. So to follow this up, I changed the world state modification within the GetTreated action from “isTreated” to just “Treated” (this appears to be what they did in the tutorial but it is not clearly stated).

While fixing this, they also note a possible issue with the NavMesh system in Unity. Something about the NavMesh that even when it gets a path it thinks the destination was the previous destination, so it returns that the path was “completed” before the agent actually gets there. This can cause agents to start on the next action since many are location based. This was clarified to possibly be a result of their large update method within GAgent that is using NavMesh’s SetDestination method. They suggest that the loop may be processing through again before the SetDestination has been fulfilled which can cause issues.

Distance Check

To solve this issue they decide to just have their own distance check for destinations. For this they just did a basic distance check between the agent’s position and its destination position.

Speeding Up the System

To help with the debugging process they simply sped up the entire game system. This can be done very easily by just setting Time.timescale to a value greater than 1.0 anywhere in your scripts. This is actually very handy to know and would be an insanely useful tool moving forward to add to a major static debugging script for any project that could be toggled and edited to help with various debugging processes (i.e. they just added that line to the GWorld class since it’s a systematic static class).

With all of these fixes and modifications in place, the system now appears to run rather smoothly and appropriately. It was working so well I tested adding another nurse agent to see how it would adapt and it continued to work flawlessly. The new nurse would properly grab patients that the other nurse was not treating and take them to a different cubicle to treat them, so there was no strange noticeable overlap of actions.

Taking a Break

Taking a Break

This tutorial looks at adding a Rest action to the nurses so they will sometimes take a break and move to the lounge area to regain energy to continue working. To begin, they added a small method within the nurse GAgent class to modify their beliefs every so often to add the exhausted state to them. This gets the exhausted state into the overall system to help start allowing it to control their actions.

They then added the new action named Rest which simply has a Postperform method for removing the exhaust state. This paired with adding a duration to this action is all that is necessary to provide a timed cool down for the nurse to remain out of commission for a time before coming back to action rested.

How they set it up the nurse agent’s are only able to rest if there is time (so either there are no patients or no available cubicles). So when there is only a single nurse and a lot of patients, the rest action is never seen. To show it working, they added several nurses (about 5) to the system. Once this was done, it could be seen that occasionally when a nurse was finished treating a patient they would go to the lounge area instead of directly to another patient. They would then come out of the lounge to grab a patient after being rested and having another action to perform.

Finally, they added the removal of the patient agents once they returned home just to keep the system cleaner. This was very simple as they just added Destroy(this.gameobject) to the Postperform method of the GoHome action.

The following is a quick link showing my finished GOAP tutorial project in action
Link to Video of My Completed GOAP Tutorial in Action

Summary

This overall tutorial was amazing for introducing me to a new AI system. I still need to look over some of the foundational interactions of the major classes, but I understand the system from a conceptual view pretty well now and it seems very useful as another AI option depending on your case.

The next to last class (Adding Goal Challenge) was very useful for understanding some of the issues you can run into and how to ensure the system runs smoothly and controlled. Understanding keeping the states very separated between the different actions/agents and the overall world state is crucial to make sure you do not get some nasty bugs. Then the speed up feature available in Unity is definitely something I need to take more advantage of when debugging in particular.

July 30, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Parts 9 and 10

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Creating a Multi-Step Plan

Creating a Multi-Step Plan

This tutorial begins linking multiple actions together to create more intricate action plans. To do so, they created a second GAction, GoToWaitingRoom. GoToHospital action was given an after effect of hasArrived and GoToWaitingRoom was given a precondition of hasArrived to give them a point to link together. GoToWaitingRoom was then given the after effect isWaiting, so that the goal given would match up with this second action’s outcome. This let’s the system take an input goal of isWaiting and deduce the plan of action to get there through these two actions.

Just to better visualize this demonstration, they created a basic spawner to consistently spawn in new patients to show that the newly created objects can follow the plans created properly as well.

Plans that Require Multiple Agents

Plans that Require Multiple Agents

I immediately had issues with this tutorial because they give a couple scripts directly to you for use in these next steps and they had compile errors in them that stopped me from progressing. These are the GAgentVisual and GAgentEditor classes. One of the first steps requires you to add the GAgentVisual component to the Patient game objects but I could not because of errors in the GAgentEditor class.

The issue was that it did not recognize the lists of preconditions and after effects within the GActions as KeyValuePairs. To resolve this, I modified them to go through the lists as WorldStates and just use those key values instead of the very direct Key values of a KeyValuePair. This at least removed the compile errors initially so I could progress.

This visualizer however is a nice addition. It lays out the individual agent’s plan as well as the preconditions and after effects throughout the process to help the designer keep track of what is going on behind the scenes. This is just generally helpful as well as a useful tool for debugging.

This tutorial begins to work with the nurse agents in the scene. Their first action is GetPatient, but it requires a precondition that is triggered by a change in the world state (beginning the idea of having plans involving multiple types of agents). This is done by adding the following line to the GoToWaitingRoom class:

GWorld.Instance.GetWorld().ModifyState(“Waiting”, 1);

To aid the nurses in keeping track of the patients, a queue of gameobjects holding the patient references was added to the overarching GWorld class. This gives a world state parameter to keep track of patients that the nurses can reference.

Getting the world states involved in the planning process has finally necessitated adding some logic to the Preperform and Postperform methods of the GActions. The patients GAction GoToWaitingRoom has a Postperform which informs the world state that more patients are waiting and adds them to the queue, while the nurses have a GAction GetPatient with a Preperform which checks if there are any patients in queue and sets their course for that target only if there is anyone waiting. This also begins the option of having plans “fail” because the goal cannot be met at this time.

Tutorial Progress Showing Patient and Nurse Agents in Action

Summary

These two tutorials have finally started showing the actual effects of this GOAP system. It shows how creating simple lists of actions can be done easily by the designer, as well as how much easier implementing world state AI is with this system. All of the data is nicely contained and distributed throughout the GActions, the GPlanner, GAgents, and the GWorld. I am excited to see how adding values to the different actions works out and how easily it is implemented on the design side.

July 21, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Part 8

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Executing a Simple Plan

Executing a Simple Plan

This tutorial starts by actually implementing the newly created planner setup within the agents themselves. This involves creating a rather bulky LateUpdate method within the GAgents class which tries to handle all the possible cases of different states of having a planner and having actions to perform.

This LateUpdate can be broken down into four main sections:

1. Continues running an action that is in progress
2. Generates a plan and possibly an action queue if they are not already present in agent
3. Removes goal and planner once entire action queue is used up
4. Selects the next action in queue to start performing

They then created a new Patient class (which inherited from GAgent) to place on the patient agents and they created a new GoToHospital class (which inherited from GAction) to give this agent something to use with the planner. This was done to simply show the interaction of all these class types, the GAgent, GAction and GPlanner.

Summary

While brief, there is a lot going on in the GAgent class with this tutorial. I feel like having such a heavy Update type method (LateUpdate in this case) is not normally ideal, so there may be better ways to condense that section into separate methods or have other parts of the system help make some checks to support the GAgents class.

July 21, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Part 7

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

The GOAP Planner

GOAP Planner – Part 1

Planner starts by looking through all the achievable actions and finding which can be performed given the starting states from the first node. This can create the branches for the action network which will then continue to have actions connected to them (that satisfy the state conditions) until a chain is found that has a result which matches the initial goal.

GPlanner Class

The first major step of this class is creating the Plan method which returns a Queue of GActions (this will basically be the full list of actions that will be chosen to actual be acted out). This starts by making a List of usable GActions (those which satisfy the IsAhchieavable condition).

They then start a List of the newly create Node class objects, which acts as a more encompassing container for a GAction object. The most important data of which is the running cost of the entire branch of Node objects. This allows for the next step where they go through the list of Nodes (named leaves) to find the cheapest Node (which in turn is the cheapest path). The running cost is an assumption of the capabilities of the BuildGraph method they added in the middle of this process they have not yet defined.

Finally another List of GActions is created. This time the list is simply created by starting with the cheapest Node and following it back to the original Node through their parents contained within the Node class (this is basically a linked list, and also works similarly to the basic A* node/waypoint reading method I have worked with).

C# Note

When creating the Node helper class they made sure to create a new Dictionary and pass it the values from the other Dictionary since they wanted a new copy of that Dictionary. It is important to note that if you simply set the Dictionary equal to the other Dictionary that it becomes a pointer to that Dictionary as opposed to its own separate new Dictionary.

GOAP Planner – Part 2

This tutorial focuses on building the BuildGraph method for the GPlanner class. It is noted ahead of time that this method will strongly rely on recursion.

The BuildGraph method returns a bool value and has input parameters: Node parent, List leaves, List useableActions, and Dictionary goal. It begins by searching through the List of useableActions to determine which are achievable given the parent Node’s state.

For each achievable action, it creates a replica state of the parent Node’s state, searches through all the effects of that action (resulting states from performing the action), and adds any that are not already found in the current state to that replica state. This setup allows the method to predict what the chain of states would look like if it were to progress through these actions so it can determine a plausible entire chain of actions.

Using all of this information, a new Node object is created which receives the parent Node as its parent, the parent cost combined with the current action’s cost as its cost, the modified parent state as its currentState, and the action itself for its action.

Finally the recursion comes in to finish out the method. If the goal is achieved as indicated by the modified currentState outcome, the newly create Node is added to leaves, the true list of actions to perform. Otherwise, a new subset of actions is created from the full list of usableActions and passed into the BuildGraph method recursively (this subset simply removes the action it just tried since it was deemed ineffective). This continually shrinks down the list of actions to create the terminal condition for this recursive method. GoalAchieved and ActionSubset are two more methods used within this method to help with this final recursive step.

Summary

This section was very coding heavy and got deep into the core of creating the GOAP AI system. The GPlanner class is doing the heavy lifting of finding which actions are viable for an agent to perform and determining what chain of actions can get the agent from its initial world state to its final goal state it is aiming to achieve.

June 11, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Parts 4, 5, and 6

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Intro

This part of the course gets into actually programming the GOAP system and using it within their example Unity project. These tutorials focused on setting up the foundational classes: GAgent, GAction, GWorld, and WorldStates. These are used as the core of the system as they either hold all the information to pass around through the system, or serve as the base classes to work with for creating agents within the system.

The World States

The World States

The main scripts created here are:
• GAgent
• GAction
• GWorld
• WorldStates

WorldStates:

They added another class within this script named WorldState. This WorldState class just holds a string named key, and an int named value. These will be used for a dictionary setup, hence the key and value naming setup. The rest of the WorldStates class itself deals with creating and modifying said dictionary of WorldState objects. It has several dictionary management methods such as: HasState, AddState, ModifyState, RemoveState, SetState, and finally GetStates (to return all states when needed).

GWorld:

This was set as a public sealed class, because this will apparently help with setting up queues in the future. It also makes use of a singleton pattern to create a single location to hold all the world data that can be accessed by many objects.

Actions

Actions

GAction class:

This class is an abstract class, so it will be used as the overall parent class from which all actions for the GOAP system will be derived. Therefore it will serve as a framework for those classes.

Important Fields within this Class:

• float cost – assigns value to an action (or chain of actions) to help determine between various plans
• GameObject target – location where action will take place
• float duration – how long the action will take to perform
• WorldState[] preconditions – group of preconditions which the action requires to be met in order to perform said action
• WorldState[] afterEffects – state results from performing this action
• WorldStates agentBeliefs – states within the agent necessary to monitor with this action

The constructor for the GAction class just initializes a new dictionary for preconditions and effects (preconditionsDict and effectsDict). There is then a method named IsAchievable to determine if a particular goal can be achieved (which just returns true currently and that is it). Then there is an IsAchievableGiven method, which requires an input parameter of a Dictionary of conditions to see if they are present in the preconditionsDict within this action (so it basically returns whether the preconditions of an action are met or not).

Finally, there were two abstract methods added to use in derived actions. These were Preperform() and Postperform(). Preperform provides extra logic to check for before performing an action (i.e. checking what resources or agents are available), and Postperform will hold extra logic for how the world or agent states are impacted after the action is performed.

Agents

Agents

GAgent Class:

This class will use the System.Linq namespace to help sort and organize objects. They also added a SubGoal class within the GAgent script. This helps hold goals within the overall goal that can be added and removed as they are completed. This tutorial sets up the GAgent class so that it can be populated with GAction objects that are available for that specific GAgent.

General Unity Note – GetComponents Method

They use this.GetComponents (plural) to fill an array with all the components located on this game object. That is a good way to quickly populate an array with a bunch of similar components on a single game object I hadn’t used before.

Summary

It was nice to finally get into actually programming the GOAP setup after following all the conceptual work. I really enjoy setting up base classes for bigger systems and getting to see the foundation of the GAction class specifically is interesting and I am excited to see how it ends up working out. I can already start to see how the flexibility of this system works through the GAction class specifically.

It feels like a lot of information needs to be passed around, and there are going to be some large classes holding a lot of information at one time, such as those holding information about the entire world state. I am interested to see how they approach these two issues while keeping everything manageable on the programming side while also keeping it running efficiently enough to run several agents (and different types of agents).

June 10, 2020

AI For Beginners

Goal Orientated Action Planning (GOAP)

Parts 1, 2, and 3

Beginner Programming: Unity Game Dev Courses

Unity Learn Course – AI For Beginners

Intro

This course covers goal orientated action planning (GOAP) as another way to set up flexible systems of AI behavior. This blog covers the first 3 tutorials of the course which introduce the general GOAP concept by defining its parts and explaining the planning process going in to creating a GOAP system.

An Introduction to GOAP

GOAP Introduction

Goal Orientated Action Planning (GOAP):
• has all the elements of a finite state machine, but uses them differently
• uses graphs for processing
• GOAPs actions and goals are decoupled
• actions are free elements in the system that are mixed/matched to meet goals
• instead of having a list of actions needed to meet a goal, GOAP allows for multiple solutions to be chosen from

Actions

Components: Precondition, Effect
Precondition: state that must be met before the action can take place
Effect: how the action leaves the state of the agent (or world) after the action has occurred
Actions are connected similarly to dominoes where the effects of actions are matched with the preconditions of other actions to create action chains.

Goal: the end state of the agent

Creating a plan comes from understanding the current state of the agent itself and the world around it. The planning stage of GOAP works backwards to see if it is achievable from the currents states (again, of the agent and the world it understands). Since this approach can lead to multiple action chains that lead to the same goal, costs are implemented to help the agent decided which action chain to choose (and in the case of a tie, one can be selected at random).

A general GOAP system follows this structure where actions, goals, and the world state data are fed into a planner. The planner chains actions together according to the goals and starting states to see which plans are achievable. The planner uses the A* algorithm (exactly the same as some pathfinding algorithms) to determine the “best plan”. Once a plan is generated, the agent goes to achieve it with a simple state machine. This state machine looks to simply move the agent where it needs to be to perform the actions necessary, and perform said actions until a goal is achieved. Before each action is performed, it is checked to see if it is valid. If it is not valid anymore, the entire plan is abandoned, and a new plan is generated.

One of the biggest benefits of the GOAP approach is that new actions can continually be added to the pool of available actions for the agent. These actions will then always be picked up by the planner as another possible action to create an action chain or plan to solve a goal. The planner effectively creates the total graphs of actions for the designer, so it is also just easier to program additional factors as complete graphs do not need to be directly programmed as is more common with complete finite state machines.

Setting Up A GOAP Environment

Setting Up A GOAP Environment

This first major tutorials starts the setup for a basic GOAP project. It is set in a small hospital with patient agents and nurse agents.

Patient Agents

They come into the hospital and get registered at the recption desk. They then move to the waiting room until it is their time to be received by the hospital where they are then brought to a cubicle to be checked on before leaving the hospital.

Nurse Agents

They operate the reception desk receiving incoming patient agents, they come and grab patients from the waiting room to take them to a cubicle to check on them, and they can also rest in the staffing area if they need a break.

Since a lot of the actions and goals of these two agents are heavily location based, they implemented an underlying waypoint system to the entire setup. Empty gameobjects are set at all the critical locations to be used as waypoints for guiding the agents moving forward. All the waypoints have been tagged with individual unique tags to help the agents locate them immediately upon instantiation.

Pre-Planning the Agent Actions

Pre-Planning the Agent Actions

The foundation of GOAP is that there are: goals, actions, and states. These must all be taken into consideration when planning those which to include and which should be able to lead into others when graphing out plans. An agent has a list of goals it can achieve. It also has a pool of actions it can use to achieve these goals. The actions may also require other actions before being able to be performed. Some actions may also require other agents or some other outside resource.

Patient Actions:

• Come into hospital
• Register at the front desk
• Go to waiting room
• Get treated when nurse and cubical are available
• Go home

Nurse Actions:

• Get Patient
• Go to Cubicle
• Rest

The agent must draw information from the world states to help determine what goals are achievable. It will also combine this information with its own state (data it has within itself) to help it determine what is possible. This world state information and the agent state information is passed into the planner, along with all the goal and action options of the agent, so the planner can generate a plan for the agent.

Summary

Overall this felt like a good introduction to the GOAP AI system approach. While I don’t think I could build my own systems quite yet using it, I have a much better understanding of the overall concept and how it can be used to create a flexible set of options for actions for AI’s to use to guide their behavior. The ability to add actions to this pool with relatively little cost is very nice for creating a scalable system with a lot of variability. I could also see this leading to a lot of interesting emergent behavior with large pools of agents, goals, and actions. I look forward to the next parts of the course where they get more into how to program such a system.