Guide: How do we run your submitted agents?

What is this guide about and who is it for?

This guide describes how we run and evaluate the agents you submit to the Geometry Friends Game AI competitions.

If you are interested in knowing how to participate, please check out our guide on how to participate.

This guide is for those who:

• got an error (e.g. extraction error, compile error...) on their submission and would like to understand or even replicate the server environment in order to figure out what could be wrong
• want to know what kind of fairness measures are being taken to ensure equal treatment for each submission
• are curious about what's happening 'under the hood'

What machines are being used to run the agents?

Participants are allowed to build their agents with the aid of third party libraries. However, these may be platform specific, which is why we have two different virtual machines available:

• Ubuntu Server 22.04. Uses Mono 6.12.0.182 and MSBuild 16.10.1.31701

• Windows Server 2019. Uses MSBuild 15.6.85.37198 (unavailable)

This is why you need to select the platform when submitting your agent. If you don't use platform specific libraries, then it shouldn't matter which platform you choose.

Moreover, they are both configured to use the following resources:

• 2 cores
• 4 GB of RAM

Is the system fair?

The virtual machine based system offers great benefits in terms of fairness, such as:

• unchanging machine resources; the VM's are always configured to use the same resources mentioned above
• constant machine state, i.e. the state of the machine is guaranteed not to be affected by previous submissions through the use of VM snapshots

Extraction

After sending your submission zip file to the appropriate machine, we extract its contents using:

• Ubuntu Server 22.04: the 'unzip' command line utility

• Windows Server 2019: the 'Expand-Archive' powershell utility (unavailable)

Afterwards, we clean up part of your submission in case you included your copy of the game. This is so we can then use our own up-to-date and untampered version of the game. This is also why we advise you not to place any additional files in game folders like "Content', "Levels" and "Results".

Compilation

At this point, it is assumed that there is a ".sln" and/or a ".csproj" file in the same directory as the zip file.

To compile, 'msbuild' is called in the mentioned directory.

Simulation

If nothing failed so far, a '.dll' should have been generated from the previous step, and placed automatically in the Agents folder.

Then we move to the GeometryFriends/Release directory, which will be the simulation's working directory. If you use auxiliary files and which to specify their relative path, their path should be relative to the GeometryFriends/Release directory.

To actually run your agent we use the command:

./GeometryFriends.exe -st 0 'level_number' -a 'path_to_generated_dll'

To know more about how the Geometry Friends command line utility works, head over to our guide on our parent site.

Short note: we can always use '0' as the world identifier on our machines because there will only be the world of the competition you submitted to.

Scoring

After extracting the results from the machine, we compute your score using a formula described in here.

We update our database after evaluating your agent on each level, which means that you may observe your score going up (hopefully) if you manage to catch your submission in the 'Processing...' status. Just spam that refresh button!