Entity AI

Entities need to perform complex behaviors on their own (or even when controlled by the player). Examples of such behaviors are targeting the closest enemy when figthing, detecting enemies in radius, avoiding buildings and other units whilst moving, getting back up and return to formation position when knocked back, or moving to a new position when changing formation.

In order to implement those behaviors in a clean, flexible, maintainable and extendable way, we decided to use Unreal's new State Trees, which provide a performant and convenient Finite State Machine with a graphical editor, conditions, tasks and transitions.

In the RTS Framework, AI works at two levels:

Battalion AI: Handles the high level behaviours of the battalions and associated entity handles. For example, when a player issues a move order, the battalion will set a target and try to move towards it.
Unit AI: Handles lower level behaviors at the unit level, such as attacking another entity, dying, moving to their formation slot, etc.

Let's examine how both are setup:

1. Battalion AI

The battalion AI is fundamentally driven by the order queue present in the entity handle. The order queue contains all actions input by players such as moving, attacking, holding position, etc. Therefore, there is a strong parallel between the order types and the high level tasks that the battalion performs.

The battalion state tree will also be the responsible to manage orders being performed, and removing them from the queue upon task completion.

Every time a new order is ready to be performed (either because the player issued an immediate order Perform_Now or Perform_Before, or because the next order in the queue is available) the entity handle will notify the state tree via an event ("StateTreeEvent.Battalion.NewOrder").

The state tree will then abort the current task and reevaluate the tree. Each state has an associated condition (derived from FStateTreeConditionCommonBase) which will check if the current order exists and is of a specific type. For example the AttackCondition only returns true when the first order on the queue is of type attack:

bool FRTSStateTreeBattalionAttackCondition::TestCondition(FStateTreeExecutionContext& Context) const
{
	auto EntityHandle = Cast<ARTSEntityHandle>(Context.GetOwner());
	checkf(EntityHandle, TEXT("FRTSStateTreeBattalionAttackCondition is being used in a State Tree which is not owned by an EntityHandle actor"))

	if (EntityHandle->OrdersQueue.IsEmpty())
	{
		return false;
	}

	if (EntityHandle->OrdersQueue[0].OrderType != ERTSOrderType::ORDER_Attack)
	{
		return false;
	}

	return true;
}

If the condition is met, then the state tree will enter that state (and potentially any substate if present) and execute the associated task(s).

Each state tree task has usually three main logic entry points. Note that the code snippets are slightly edited from source code to simplify the examples:

EnterTask: This is executed once when the state is entered, and should be used to do any initial setup, for example in the attack task, we do set some initial data on the task instance:

EStateTreeRunStatus FRTSStateTreeBattalionAttackTask::EnterState(FStateTreeExecutionContext& Context,
                                                                 const FStateTreeTransitionResult& Transition) const
{
	FInstanceDataType& InstanceData = Context.GetInstanceData(*this);
	InstanceData.TargetHandle = CurrentOrder.TargetHandle;
	InstanceData.OptionalTargetEntity = Cast<ARTSEntity>(CurrentOrder.OptionalTargetActor);
	InstanceData.bIsMovingTowardsTarget = true;

	return EStateTreeRunStatus::Running;
}

Tick: If the task needs to perform logic every tick, this is the place. For example, the attack task checks if the target is too far, and checks when the enemy target is dead in order to consider the task succeeded.

EStateTreeRunStatus FRTSStateTreeBattalionAttackTask::Tick(FStateTreeExecutionContext& Context,
	const float DeltaTime) const
{
	const auto EntityHandle = Cast<ARTSEntityHandle>(Context.GetOwner());
	checkf(EntityHandle, TEXT("FRTSStateTreeBattalionAttackTask is being used in a State Tree which is not owned by an EntityHandle actor"))

	// If the current move order is removed for whatever reason, such as another order being issued
	if (EntityHandle->OrdersQueue.IsEmpty() || EntityHandle->OrdersQueue[0].OrderType != ERTSOrderType::ORDER_Attack)
	{
		return EStateTreeRunStatus::Failed;
	}

	FInstanceDataType& InstanceData = Context.GetInstanceData(*this);

	// If a new attack order has been given (targets have changed) abort and we'll restart the task
	if (HasTargetChanged(EntityHandle, InstanceData))
	{
		return EStateTreeRunStatus::Succeeded;
	}

	if (!InstanceData.TargetHandle.IsValid() || InstanceData.TargetHandle->IsDead())
	{
		// remove current completed order and notify that the task has succeeded
		EntityHandle->OrdersQueue.RemoveAt(0);
		EntityHandle->Battalion->Path.Empty();
		return EStateTreeRunStatus::Succeeded;
	}

	CheckTargetStatus(EntityHandle, InstanceData);

	return EStateTreeRunStatus::Running;
}

ExitState: Is called when the task has succeeded or failed, can be used to do cleanup on the data. For example on the exit building task, we re-enable unit controllable state and stop march sound.

void FRTSStateTreeBattalionExitingBuildingTask::ExitState(FStateTreeExecutionContext& Context,
	const FStateTreeTransitionResult& Transition) const
{
	auto EntityHandle = Cast<ARTSEntityHandle>(Context.GetOwner());
	if (!IsValid(EntityHandle))
	{
		return;
	}
	
	// failsafe, in case anything happens, we still want to enable units to be controllable
	if (IsValid(EntityHandle->Battalion))
	{
		EntityHandle->EnableSelectivity();
		EntityHandle->SetUnitCollision(true);
		EntityHandle->SetBuildingCollision(true);
		EntityHandle->SetBuildingAvoidance(true);
	}

	if (IsValid(EntityHandle->HandleAudioComponent))
	{
		const auto EntityConfig = EntityHandle->GetEntityConfig();
		if (const auto UnitConfig = Cast<URTSUnitConfig>(EntityConfig))
		{
			EntityHandle->HandleAudioComponent->SetTriggerParameter(UnitConfig->MarchSoundStopTrigger);
		}
	}
}

1.1. Reference of Battalion AI states

List of current AI states for battalion. The names are self-explanatory:

Idle
Move
Stop/Hold Position
Patrol
Exit Building
Attack
Attack-move
Construct Building

2. Unit AI

Unit AI is mainly driven by the parent battalion AI (see previous section). However, there are some exceptions where the AI is driven by other gameplay logic: For example, when units die they will stop listening to any order from the handle (dead units are not controllable for obvious reasons) and just progress through the death states. Another example will be when units are knocked back (from spell, hit by a troll or trampled by cavalry). In that case, the entity is not responsive while on the floor and will take a bit of time to get back up and behave normally.

Contrary to entity handles, units don't have an order queue and just are "told" to switch states by external logic. For example to notify a unit that they need to switch to attack state, we set the enemy target data and send a new event to the state tree so that units switch the state.

void ARTSUnit::NotifyAttackState(ARTSEntityHandle* EnemyHandle, ARTSEntity* EnemyEntity)
{
	EnemyEntityHandle = EnemyHandle;
	OptionalEnemyEntity = EnemyEntity;

	NotifyStateToStateTree("StateTreeEvent.Unit.BattalionAttack");
}

The state tree tasks are setup in a similar way to battalion tasks (enter task, tick, exit task) so we will refer the reader to visit the above section instead of repeating ourselves.

2.1. Reference Unit AI states

List of all the possible unit AI states and theier subtasks:

Idle: When doing nothing. The unit will still attack enemies in range and move to keep formation position if displaced.
Move: When moving towards a destination.
Attack: When ordered to attack. Units won't be in attack state until the battalion has arrived very close to the enemy target and entered the "narrow phase combat radius".
- Waiting for target: Even if the battalion has been ordered to attack, the unit will get a concrete target from the enemy handle after a potentially async request. The reason is that in multiplayer only the server calculates targets and we need to wait for the results back to the client.
- Moving towards target: Once a target has been assigned, the unit will try to move towards it. If not possible (out of range or the unit is somehow blocked) we will request a new target after a delay.
- Attacking: Once the target has been reached, we will start performing the "attack loop".
  - Pre-attack: Initial delay until the time to hit and attack animations are calculated.
  - Attack duration: Time in which unit is actually dealing damage and playing attack animations. During this task the state tree is blocked from external input and the attack will always complete.
  - Reload: Delay until the next attack (pre-attack) is ready. Will automatically transition to Pre-attack state to repeat the loop. The state tree remains locked in this task to avoid cheating (clicking multiple times on target to attack faster).
Stop: When ordered to hold position
Exit building: When spawned and exiting building, exiting a garrison building, or builder after construction finished.
Knockback: When being knocked back, the unit will be in this state until it gets back up. State tree is blocked as well during this time.
Dying: After a complete depletion of their health, the unit will start dying. This includes playing death animation, potentially rag-dolling, etc.
Dead: After playing the death animation, the unit corpse will remain on the ground for a while, and sink slowly. This task will run until the entity actor is completely destroyed and cleaned up.

RTS Framework Docs

Entity AI

1. Battalion AI

1.1. Reference of Battalion AI states

2. Unit AI

2.1. Reference Unit AI states