ΕΝ

Pyrsos.dev

"Passing on the flame"

In the Age of Prostagma and Hacking

Vangelis Garaganis - 20 September 2024

Here is a short story of Age of Mythology, hacking and pure nostalgia that doesn't take place in the distant 2002, but some while ago, before Retold made it's first appearance.

To begin with, meet Skelo. Skelo is my arch enemy for the past years in Age of Mythology. Me and Skelo have fought some epic games in the past that pushed both of us in our very limits. In the last day of 2023 the twelve AOM gods blessed us with what I reckon to be our best game, so far, that resulted in a bitter defeat. While defeated, what ached me more was that all of those epic games were not captured for the rest of the community to witness. At least that was what I initially thought.

After some ranting with Skelo, about how it was a shame for all those games to be lost, Skelo revealed to me that he had all of his recorded games. The problem was that he had 3000 game recordings, with no means to search them through and filter our games. The fact though, that there was a slight chance of retrieving them lit a glimpse of hope in me, so I asked him to sent me the directory with his recordings.

As expected, the odds were pretty dissapointing. Just a list of 3000 obscure binary .rcx files. Each file was a recording that could be opened only by the Age of Mythology's embedded recordings player. I hoped that those files could contain some sort of metadata in them, for example the players playing in the current recording - you bet. But since those files were binary and there was little information about them in the internet, it seemed that there wasn't any feasible way to retrieve anything useful from them. That meant that, unless you provide me some better options, there was only a solution. Watch and archive every single one of his 3000 games, until I could find our games.

While I do love Age of Mythology, archiving 3000 games isn't a task that any human in the right mind would willingly do. But what if it wasn't a human? What if it was a robot?

It seemed like the perfect automation task. First, I had to break things down and invistigate what the robot had to do. And that was, theoreticaly, simple. Just like always, the robot must immitate exactly what a human would do and be able to:

  1. Move mouse and type, so that it can open windows and type the recording names.
  2. Be able to watch the game.
  3. Be able to read and identify the player's usernames, so that it can then archive the recording.

Since those days I had been writing some JS scripts, I thought about creating my robot with JS, too. All that needed was the proper libraries to support the afforementioned robot capabilities.

  1. The capability to move mouse and type, so that it can open windows and type the recording names will be granted from RobotJS. RobotJS is a cross platform NodeJS desktop automation library that would act as robot's arms.

  2. The capability to watch the game will be granted from Screenshot-desktop and Sharp. Those two libraries would be robot's eyes. The screenshot-desktop library is the one that would take screenshot of the recording and let the sharp library crop the usernames part, allowing robot to focus on the crucial part of the archiving process.

  3. The capability to read and identify the player's usernames, so that it can then archive the recording, will be granted from Tesseractjs. Tesseract is a OCR libarry, allowing to extract text from images.

With that being said, I think that the idea has been kinda clear by now. Just create a robot that would pretty much immitidate my agonizing efforts to find my games. Let's move now to the coding part and how the automation process would work. All of the article's code is available here, along with the small dataset containing 3 rcx files.

async function automateRecordingsArchivalProcess() {
   const folderPath = "./haki_games_small_subset";

   const files = await fs.promises.readdir(
      folderPath,
      (err) => err && console.error(err)
   );

   for (const fileName of files) {
      console.log(`🔴 Archiving recording with name ${fileName}`);
      await automateRecordingArchivalProcess(fileName);
   }
}

automateRecordingArchivalProcess();

As you can see we run the automateRecordingArchivalProcess which performs the archival of a game to its specific folder.

async function automateRecordingArchivalProcess(recordingName) {
   moveCursorToTextInputAndTypeRecordingName(recordingName);
   openTheRecording();
   openTheDropdownMenuWithUsernames();
   await takeScreenshotAndCropUsernamesPart();
   const usernamesRecognized = await tesseractRecognizeUsernamesInScreenshot();
   moveRecordingToFolderForThoseUserMatches(usernamesRecognized);
   openWindowToWatchAnotherRecording();
}

The robot starts of by typing the recording's name and opening the recording.

function moveCursorToTextInputAndTypeRecordingName(recordingName) {
   console.log("...Moving cursor to text input and typing recording name...");

   const TEXT_INPUT_LOCATION = [816, 699];

   robot.moveMouse(TEXT_INPUT_LOCATION[0], TEXT_INPUT_LOCATION[1]);
   robot.mouseClick();

   const dateTimeComponents = extractDateTimeComponents(recordingName);

   robot.keyToggle("shift", "down");
   robot.keyTap("R");
   robot.keyToggle("shift", "up");
   robot.keyTap("e");
   robot.keyTap("p");
   robot.keyTap("l");
   // ...
   robot.keyTap(dateTimeComponents.year[0]);
   robot.keyTap(dateTimeComponents.year[1]);
   robot.keyTap(dateTimeComponents.year[2]);
   robot.keyTap(dateTimeComponents.year[3]);
   robot.keyTap(".");
   robot.keyTap(dateTimeComponents.month[0]);
   robot.keyTap(dateTimeComponents.month[1]);
   robot.keyTap(".");
   robot.keyTap(dateTimeComponents.day[0]);
   // ...
   robot.keyTap(dateTimeComponents.rest[5]);
   robot.keyTap(".");
   robot.keyTap("r");
   robot.keyTap("c");
   robot.keyTap("x");
}

function openTheRecording() {
   console.log("...Opening the recording...");

   const OPEN_RECORDING_BUTTON_LOCATION = [906, 753];

   robot.moveMouse(
      OPEN_RECORDING_BUTTON_LOCATION[0],
      OPEN_RECORDING_BUTTON_LOCATION[1]
   );
   robot.mouseClick();
}

Now that the recording has been opened, we want the robot to open the players dropdown and focus in this part of the screen:

And here is the code of how our robot does it:

function openTheDropdownMenuWithUsernames() {
   console.log("...Opening the dropdown...");

   const DROPDOWN_MENU_LOCATION = [622, 908];

   robot.moveMouse(DROPDOWN_MENU_LOCATION[0], DROPDOWN_MENU_LOCATION[1]);
   robot.mouseClick();
}

async function takeScreenshotAndCropUsernamesPart() {
   console.log("...Taking a screenshot and cropping usernames part...");

   try {
      await screenshot({ filename: "shot.jpg" });
      await cropScreenshotTaken();
   } catch (error) {
      console.log("Error in taking screenshot", error);
   }
}

async function cropScreenshotTaken() {
   const SCREENSHOT_LOCATION = [622, 908];
   const SCREENSHOT_SIZE = [180, 80];

   try {
      await sharp("shot.jpg")
         .extract({
            left: SCREENSHOT_LOCATION[0],
            top: SCREENSHOT_LOCATION[1],
            width: SCREENSHOT_SIZE[0],
            height: SCREENSHOT_SIZE[1],
         })
         .toFile("shot2.jpg");

      console.log("...Image cropped and saved successfully.");
   } catch (error) {
      console.error("Error in cropping screenshot taken:", error);
   }
}

Here is the cropped screenshot taken from our robot:

The last part, before moving the recordring to its corresponding folder, would be for the robot to identify the usernames from the screenshot taken. And here is where Tesseract's OCR library comes into play:

async function tesseractRecognizeUsernamesInScreenshot() {
   console.log(
      "...Waiting for tesseract to recognize usernames in screenshot..."
   );

   try {
      const tesseractResponse = await tesseract.recognize("shot2.jpg", "eng");
      return tesseractResponse.data.text;
   } catch (error) {
      console.error("Error in tesseract usernames recognition:", error);
   }
}

The final step for the archiving process to be complete, is to process Tesseract's response, move recording to its corresponding folder (the archiving policy is yet to be defined and for the shake of the article is ommited) and open the recordings list again to repeat the process:

function moveRecordingToFolderForThoseUserMatches(
   recordingName,
   usernamesRecognized
) {
   // It gets text input like this 'Haki Terror\nSkelo\n' and returns this 'Haki Terror-Skelo'
   const usernames = usernamesRecognized.split("\n");
   const matchesFolderName = usernames[0] + "-" + usernames[1];

   console.log(
      `...Moving ${recordingName} to folder: ${matchesFolderName} 🟢 `
   );
   // TODO, archiving policy yet to be defined
}

function openWindowToWatchAnotherRecording() {
   console.log("...Opening new window to watch another recording...\n");

   const MENU_LOCATION = [1533, 138];
   const RECORDED_GAMES_BUTTON_LOCATION = [895, 561];

   robot.moveMouse(MENU_LOCATION[0], MENU_LOCATION[1]);
   robot.mouseClick();

   robot.moveMouse(
      RECORDED_GAMES_BUTTON_LOCATION[0],
      RECORDED_GAMES_BUTTON_LOCATION[1]
   );
   robot.mouseClick();
}

With that being said, here is the actual footage of the robot automating our task:

And the plan worked! I can now let the robot watch all of the recorded games and find the ones played against Skelo.

Perfomance-wise, I set the keyboard and mouse delay be kinda long for the video's shake.

robot.setKeyboardDelay(20);
robot.setMouseDelay(2000);

We can decrease the mouse and typing speed delay to allow watch the recordings even faster.

Final thoughts - Part 1

Meme-wise, Prostagma means to execute an atomic command. And just like our AoM units of work execute our commands to win our wars, our robots perform no less. Fast forward to today, the spirit of Prostagma lives on, but with a modern twist: automation. Given the prevalence of automation in recent years, I think it is prominent to recognize its potential ahead of us, especially with how rapid and easy it has become even in the most surreal scenarios. The tools we have today are the harbingers of this new era. As developers and engineers, we stand at the forefront, wielding automation to build solutions that were once the stuff of imagination. The question now isn't whether automation will transform our world but how quickly we can embrace this shift from the Age of Mythology towards an Age of Robots and Automation, or perhaps more fittingly, an Age of Prostagma.

Thanks for the reads.

Bonus solution 1

Do you know what is even better than finding my games against Skelo? Me finding all of my 2500 game recordings, as well. After some investigation, I found out that Steam automatically saves all of the game recordings in a specific folder in the disk. And now that I had both folders of rcx games, I wondered if there was another way to find my games against Skelo. And there was!

Both of the 2 recordings that me and Skelo had in our recordings directory for our last match, had something in common. And that was the last modified time.

Those two games had exactly 1 hour time difference and that's logical given that me and Skelo live in countries with different timezone. So, I created a bash script that finds files between folders that their modification time differs exactly one hour.

#!/bin/bash

if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <directory1> <directory2>"
    exit 1
fi

dir1="$1"
dir2="$2"

# Find all files in both directories and store them in arrays
mapfile -t files_dir1 < <(find "$dir1" -type f)
mapfile -t files_dir2 < <(find "$dir2" -type f)

# Iterate through each file in the first directory
for file1 in "${files_dir1[@]}"; do
    echo $file1

    # Iterate through each file in the second directory
    for file2 in "${files_dir2[@]}"; do
        # echo "Comparing times between \"$file1\" and \"$file2\""

        # Get modification times in seconds since epoch
        mtime1=$(stat -c %Y "$file1")
        mtime2=$(stat -c %Y "$file2")

        # Calculate the time difference in seconds
        time_diff=$((mtime2 - mtime1))
        abs_time_diff=${time_diff#-}
        # echo "Time difference: $abs_time_diff seconds"

        # Check if the time difference is exactly 1 hour (3600 seconds)
        if [ "$abs_time_diff" -eq 3600 ]; then
            echo $file1
            echo $file2
            echo "The modification times are exactly 1 hour apart."
        fi
        # else
        #     echo "The modification times are not exactly 1 hour apart."
        # fi

    done
done

While the script is far from optimal, it seems to work just fine because it returned me all the of the games between me and Skelo:

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.12.31 210150.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.12.31 200150.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.12.02 011802.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.12.02 001801.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.12.06 003856.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.12.05 233855.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.12.05 225003.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.12.05 215003.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.12.18 003154.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.12.17 233411.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.07.13 171923.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.07.13 184351.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.09.13 004102.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.09.12 234102.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.09.13 010419.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.09.13 000419.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.09.13 005359.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.09.12 235359.rcx

./Desktop/savegame_haki_og/rcx/Replay v2.8 @2023.10.04 002937.rcx
./Desktop/savegame_skelo_og/rcx/Replay v2.8 @2023.10.03 232937.rcx

Bonus solution 2

Apparently, there was a way to extract some useful metadata from the RCX files. After sharing my experience with my friend and Pyrsos co-writer, Spyros, he pinpointed a repo or two that could be of use:

  1. AOM RCX tools
  2. AOM Recparser

Both of these repos were able to decompress the contents of the RCX files and retrieve an XML file containing all usefull information about the game. I don't know how both authors managed to pull-out the decompressing procedure, if they reverse-engineered that or if they better searched the web for that, but either way huge props to them. Based on their work, I managed to modify stnevans work and create a recording information extractor python script that could iterate recordings within a folder and identify games played against specific opponent (opening all kind of possibilities to gain more powerful search capabilites, like elo filtering and more):

import zlib
import struct
import argparse
import xml.etree.ElementTree as ET
import os

class RecordingsFacade:

    def __init__(self, filepath, is_ee):
        self.filepath = filepath

        with open(filepath, "rb") as f:
            all = f.read()

        # Check the header magic
        magic = all[:4]
        if magic != b"l33t":
            raise ValueError("Bad magic value")

        size = struct.unpack("<I", all[4:8])[0]
        rest = all[8:]
        decomper = zlib.decompressobj()

        # Decompress data
        try:
            decomp = decomper.decompress(rest)
        except zlib.error as e:
            raise ValueError("Recording corrupt")

        # Sanity check size
        if size != len(decomp):
            raise ValueError("Error in decompression. File might be corrupted")

        self.decomp = decomp
        self.seek = 1474 if is_ee else 1466

        self.gameSettingsXml = self.read_file()

        decodedXML = self.gameSettingsXml.decode("utf-16")
        #print("Decoding game..." + filepath)
        #print(decodedXML)

    def is_game_against(self, opponent, expectedNoPlayers):
        root = ET.fromstring(self.gameSettingsXml)
        players = list(map(lambda x: x.find("Name").text, root.findall("Player")))
        noPlayers = root.find("NumPlayers").text
        if(self.any_equals(players, opponent) and noPlayers == str(expectedNoPlayers)):
            print(players[0] + "-" + players[1] + ": " + self.filepath)

    def any_equals(self, array, string):
        return any(item == string for item in array)

    def read_section(self, totalSize):
        read = b""
        while totalSize > 0:
            blockSize = self.read_four()
            if blockSize == 0:
                raise ValueError("Zero block size")
            toRead = min(totalSize, blockSize)
            read += self.read_n(toRead)
            totalSize -= toRead
        return read

    def read_file(self):
        totalSize = self.read_four()
        return self.read_section(totalSize)

    def read_four(self):
        data = struct.unpack("<I", self.decomp[self.seek:self.seek+4])[0]
        self.seek += 4
        return data

    def read_one(self):
        data = struct.unpack("B", self.decomp[self.seek:self.seek+1])[0]
        self.seek += 1
        return data

    def read_two(self):
        data = struct.unpack("H", self.decomp[self.seek:self.seek+2])[0]
        self.seek += 2
        return data

    def read_n(self,n):
        data = self.decomp[self.seek:self.seek+n]
        self.seek += n
        return data

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('dirpath')
    args = parser.parse_args()

    for filename in os.listdir(args.dirpath):
        filepath = os.path.join(args.dirpath, filename)
        try:
            if os.path.isfile(filepath) and filename.endswith(".rcx"):
                RecordingsFacade(filepath, is_ee=True).is_game_against("Skelo", 2)
        except Exception as e:
            e
            #print(e)
            #print("Something went wrong...Possibly a corrupted recording")
            # Do nothing

if __name__ == "__main__":
    main()

Paying a closer look to the game details retrieval, while the decomporession process is simple, internals like where to seek for the actual content or how the f did they managed to find that the magic value of the file header is inspired by the l33t culture remains a mystery for me.

    def __init__(self, filepath, is_ee):
        self.filepath = filepath

        with open(filepath, "rb") as f:
            all = f.read()

        # Check the header magic
        magic = all[:4]
        if magic != b"l33t":
            raise ValueError("Bad magic value")

        size = struct.unpack("<I", all[4:8])[0]
        rest = all[8:]
        decomper = zlib.decompressobj()

        # Decompress data
        try:
            decomp = decomper.decompress(rest)
        except zlib.error as e:
            raise ValueError("Recording corrupt")

        # Sanity check size
        if size != len(decomp):
            raise ValueError("Error in decompression. File might be corrupted")

        self.decomp = decomp
        self.seek = 1474 if is_ee else 1466

        self.gameSettingsXml = self.read_file()

        decodedXML = self.gameSettingsXml.decode("utf-16")
        print("Decoding game..." + filepath)
        print(decodedXML)

The above function would give us the following information about the recording Replay v2.8 @2023.12.31 210150.rcx:

<GameSettings>
	<GameType>2</GameType>
	<Filename>Anatolia</Filename>
	<CurrentPlayer>1</CurrentPlayer>
	<ScenarioFilename></ScenarioFilename>
	<FilenameCRC>0</FilenameCRC>
	<GameStartTime>1704049304</GameStartTime>
	<MapVisibility>0</MapVisibility>
	<WorldResources>1</WorldResources>
	<MapSize>0</MapSize>
	<RestrictPauses>5</RestrictPauses>
	<GameFlags LockedTeams="" />
	<TitanMode>0</TitanMode>
	<TreatyLength>0</TreatyLength>
	<DayNightCycle>0</DayNightCycle>
	<HourOfAlpaca>0</HourOfAlpaca>
	<GameMode>0</GameMode>
	<HandicapMode>0</HandicapMode>
	<MapSeed>9747</MapSeed>
	<Difficulty>0</Difficulty>
	<NumPlayers>2</NumPlayers>
	<Player ClientIndex="0" ControlledPlayer="1" >
		<Name>Haki Terror</Name>
		<Rating>1925.000000</Rating>
		<Type>0</Type>
		<TransformColor1>0</TransformColor1>
		<TransformColor2>0</TransformColor2>
		<Team>255</Team>
		<Civilization>7</Civilization>
		<AIPersonality></AIPersonality>
	</Player>
	<Player ClientIndex="1" ControlledPlayer="2" >
		<Name>Skelo</Name>
		<Rating>1824.000000</Rating>
		<Type>0</Type>
		<TransformColor1>0</TransformColor1>
		<TransformColor2>0</TransformColor2>
		<Team>255</Team>
		<Civilization>4</Civilization>
		<AIPersonality></AIPersonality>
	</Player>
</GameSettings>

Which is more than enough to find my games.

So running on the recordings folder:

$ python3 extract_recording_info.py ~/Downloads/savegame

For once again returns my games against Skelo (and actually returns more):

I don't know if my enthusiasm to write a human-like recordings archiver blinded me, but the fact that I understestimated that any other sane person out there would create such tools, was wrong. So, although it's common sense, you can always search a little bit more before proceeding to your hacks.

Final thoughts - part 2

Well, since we managed to reach 3 completely different approaches to our problem, I thought about giving a second part of final thoughts. Each time you step on a problem, it's most likely you have several ways to approach it. In our case, I started unorthodoxically. I initially did an "AI" hacky approach, then a forensic-like solution and finally, what should I have been first, a traditional heads on tackle on the problem. Had plenty of fun solving that seemingly trivial task, but it think that's what our art is all about, so yeah, peace.

Enjoy our games

With that being said, enjoy our games: