Software Documentation

Installation

Install Terra Nova from the official repository:

Quick Start

Get started with Terra Nova in just a few lines of code:

import argparse
import os
import pickle
import jax

from sim.build import build_simulator


parser = argparse.ArgumentParser()
parser.add_argument("--seed", type=int)
parser.add_argument("--num_steps", type=int, default=300)
parser.add_argument("--map_folder", type=str)
parser.add_argument("--distributed_strategy", type=str)
parser.add_argument("--memory_length", type=int, default=1)
args = parser.parse_args()

all_maps = os.listdir(args.map_folder)

games = []

for game in all_maps:
    if ".gamestate" not in game:
        continue
    with open(f"{args.map_folder}/{game}", "rb") as f:
        gamestate = pickle.load(f)
    games.append(gamestate)

(
  env_step_fn,
  games,
  obs_spaces,
  episode_metrics,
  players_turn_id,
  obs,
  GLOBAL_MESH,
  sharding
) = build_simulator(
    games,
    args.distributed_strategy,
    jax.random.PRNGKey(args.seed),
)

for game_step in range(args.num_steps):
    for agent_step in range(6):
        actions = ...
        (
          games,
          obs_spaces,
          episode_metrics,
          new_players_turn_id,
          next_obs,
          rewards,
          done_flags,
          selected_actions
        ) = env_step_fn(
            games, actions, obs_spaces, episode_metrics, players_turn_id
        )

        players_turn_id = new_players_turn_id
        obs = next_obs

For more detail, see the file training_demo.py.

Action Space

Summary

Terra Nova's action space is made of several sub-action spaces that each control a different mechanic in the environment. Below is a high-level summary of the array shapes the environment expects to receive from each agent at each timestep. These actions should be given to the environment in the form of a nested tuple in the following order:

Below is further detail on the endpoint being controlled by each sub-action space. Each each given to the environment should have two leading axes. The first is the number of devices (e.g., GPUs) the simulator is distributed over (num_devices). The second is the number of parallel games per device (num_games).

These actions control the process through which agents may trade resources, embassies, etc. Only one trade deal can be offered per agent per turn.

The actions controlling social policy selection will only be executed if the agent has generated enough culture to allow for a new selection.

Similar to social policies, the actions controlling religious tenet selection will only be executed if the agent has generated enough faith to allow for a new selection.

Similar to religion, the actions controlling technology selection will only be executed if the agent has generated enough science to allow for a new selection. Once a technology has been selected, research towards that technology cannot be stopped.

The actions for units control both the type of action and where on the map the selected action type should occur. A unit's type (e.g., worker versus trade caravan) determines which action categories are valid. The tiles upon which a given action category can be executed depends on how many action points a given unit type has, the action category itself, the map's terrain, and several other factors.

The actions for cities control what is being made in each city as well as which tiles are being worked within the owned tiles of each city. Once a construction target has been selected, the thing being constructed in each city cannot be changed.

Detail

The actions in Terra Nova represent a preference order over outcomes. As several mechanics require sequential resolution, the most preferred outcome is not always a valid, executable action. For example, consider the following scenario:

A given agent has two cities and inputs a preference order over things to build in those two cities like:

Swordsman require a connected iron resource to build, and the Great Library is a World Wonder (i.e., only one of a given agent's cities may attempt to build this at a time). If the agent has iron connected, City 1 will begin a swordsman and City 2 will begin the Great Library. However, if the agent does not have iron connected, City 1 will begin the Great Library and City 2 will begin a worker. It is due to the conditional and sequential nature of action resolution that the environment requires the full distribution over all possible actions for all controllable endpoints at each timestep.

Because the environment automatically handles masking and ultimately the selection of specific actions, the user will not know the exact actions that will be executed ahead of time (without an accurate world model). To help with training, the environment will return a summary of which index was ultimately selected for each of the sub-action categories.

Executed Actions

As described in the previous step, agent must submit their actions over all endpoints as a preference order over all possible actions. To help with training, the environment will return the actions that were ultimately exectuted in the environment. The sub-action spaces will be in the same order on the way out as they are on the way in.

Similar to the observation space, when information is either missing or non-applicable, values will be -1. For example, if an agent has 10 of the maximum 30 units, selected-action information for the final 20 indices of unit-related arrays will be filled with -1.

Certain actions in the game grant an agent a free social policy on occasion. Each game returns a vector of integers of shape (2,) representing the (non-free, free) selections, respectively.

When founding a religion, agents must select three tenets at once. To accomidate for this information, all turns will return a vector of integers of shape (3,). During religion founding, the indices correspond to Founder tenet, Follower tenet, Follower tenet. When other types of tenets are selected, only the first element will correspond to a tenet integer.

Certain actions in the game grant an agent a free technologies on occasion. Each game returns a vector of integers of shape (2,) representing the (non-free, free) selections, respectively.

Observation Space

The observation space in Terra Nova is large, but organized and hierarchical. Some components are in the shape of the map (i.e., a (..., 42, 66, ...) array), some are arrays, some are vectors, and some are scalars. The information also spans numerical type: some are continuous values and some are non-ordinal integers. Also, due to partial observability, any of the information could be "unknown", which is always signified with a value of -1. Here we first describe the entire layout of a Terra Nova observation (along with the data shapes). Then we detail the contents of each item in an observation.

All fields that begin with 6 (the number of players) are revealed on a full-visibility basis only. That is, when a given player can see another player's units, the information about that unit is revealed. Likewise, when the seen units dieappear from full visibility, that information is removed from the observation.

Fields beginning with 6 (the number of agents in the game), are discoverable by all agents in the game.

from game.recorder import GameStateRecorder

...

# To initialize the recorder, we need to extract a single game from the bundle.
# This requires indexing within *both* the device and
# game axis => games[device_idx][games_idx]
# This exact process will need to be repeated after each player
# takes its step within each game turn.
gamestate = jax.tree_map(lambda x: x[args.device_idx][args.games_idx], games)
recorder = GameStateRecorder.create(reference_gamestate=gamestate, num_steps=args.num_steps)
recorder = recorder.record(gamestate)


for recording_int in range(args.num_steps):
    for agent_step in range(6):
        actions = ...
        (
          games,
          obs_spaces,
          episode_metrics,
          players_turn_id,
          obs,
          rewards,
          done_flags,
          executed_actions
        ) = env_step_fn(
            games, random_actions, obs_spaces, episode_metrics, players_turn_id
        )

    gamestate = jax.tree_map(lambda x: x[args.device_idx][args.games_idx], games)
    recorder = recorder.record(gamestate)

recorder.save_replay(f"./renderer/saved_games/{args.save_filename}")

bash ./run_viewer.sh