066 - Integration Testing Infrastructure

May 10, 2020

It's not a coincidence that many of my journal entries mention testing.

Testing is something that I think is important for ~any code base, but especially so when you're a team of one.

Time spent searching for the root cause of an issue is time that new features and adventures aren't being added to the game.

When I had a full time job, debugging was normal and expected.

If three hours went to tracking something down, that was par for the course.

As a solo developer it becomes painfully obvious that burning time trying to fix something that you haven't thought about in months is a cost that you simply can not afford.

Consistent progress is a key ingredient to staying sane and feeling productive. Anything that takes away from that is the enemy and should be treated as a plague.

It needs to be easier to add things to the game over time, not harder.

Test driven development isn't just something that helps with that. In my personal experience, it's a requirement for creating a codebase that becomes easier to work with over time.

When you are ~always only writing code that passes tests then you are ~never debugging.

Naturally there are exceptions to the rule.

For example, when working on something so completely out of your comfort zone that you don't have the mental stamina to TDD and need to instead just throw code at the wall.

I've found that as months of strict TDD go by these deviations from the practice become more and more rare and you learn to test your way through even completely new concepts and ideas (typically after bulleting out some thoughts and ideas on paper first).

As I've written more tests my understanding of what my test suite needs to look and feel like has evolved.

I'm still making improvements to my underlying testing infrastructure and the style with which I write tests.

Just a week or two ago I introduced the struct TestWorld(specs::World), a new type wrapper around my ECS (entity component system) World where I've started to organize helper functions that I can use across tests.

Previously when a helper function was small I would inline it in the test module that I was working on, but over months and years this means that you end up writing the same 5 line helper function multiple times.

All fine and dandy in a small codebase, but as the codebase grows the time and focus and flow saved by re-using an existing three line method instead of re-imagining it begins to become more and more noticable.

A good example of eliminating a small source of testing friction this is with accessing resources.

I recently introduced this macro.

// impl TestWorld {
//     pub fn mesh_job_descriptors(&self) -> Fetch<MeshJobDescriptors> {
//         self.fetch()
//     }
//
//     pub fn mesh_job_descriptors_mut(&self) -> FetchMut<MeshJobDescriptors> {
//         self.fetch_mut()
//     }
// }
macro_rules! resource_accessors {
    ($($res_type:ty , $fn_name:tt, $fn_name_mut:tt)*) => ($(
        #[allow(missing_docs, unused)]
        impl TestWorld {
            pub fn $fn_name (&self) -> Fetch<$res_type>{
                self.fetch()
            }

            pub fn $fn_name_mut (&self) -> FetchMut<$res_type>{
                self.fetch_mut()
            }
        }
    )*);
}

resource_accessors! {
    Arc<Mutex<GameServerReceivedMessages>>, game_server_received_messages, game_server_received_messages_mut
    ClientWorldState, client_world_state, client_world_state_mut
    GameClock, game_clock, game_clock_mut
    GpuBufferedDataResource, gpu_buffered_data_resource, gpu_buffered_data_resource_mut
    MeshJobDescriptors, mesh_job_descriptors, mesh_job_descriptors_mut
    PendingClientRequestBatch, pending_client_request_batch, pending_client_request_batch_mut
    RenderableIdData, renderable_id_data, renderable_id_data_mut
    ServerEidToLocalEntity, server_eid_to_local_entity, server_eid_to_local_entity_mut
    TextureNameMapResource, texture_name_map, texture_name_map_mut
    UserInterface, user_interface, user_interface_mut
}

All it does is let allow my tests to do this let tnm = world.texture_name_map(); instead of let tnm = world.fetch::<TextureNameMapResource>();.

At first glance one might think -

Why in the world would you write a macro to turn one one liner into another one liner?

The difference between the two invocations is subtle, but important.

The macro generated function auto completes somewhere around world.te.

Where as in the old way I had to type world.fe to autocomplete fetch, then ::<Te to autocomplete TextureNameMapResource.

Sounds small, but when you're spending the majority of time writing lots of small test cases the difference between the two becomes very claerly felt.

Integration Testing

Over a year ago in #029 I introduced the first integration test with a screen shot.

That integration test played through the game's first quest (that I've since removed) in much the same way that a player would.

It connected to the game server (over a crossbeam-channel) and sent the game server requests for things that it wanted to do.

It asserted against the state that the game server updated it with to make sure that what ia was trying to do happened.

For example, if we wanted to pick up an item we might issue a request to do so and then assert that the item was present in the inventory section of our client state that the server sent us.

This was not a replacement for unit tests. All of the underlying functionality such as picking up items had dedicated unit tests.

This was more so meant for playing through real player experiences and making sure that they could be completed from start to finish.

As with any first pass at something, there was quite a bit lacking.

The GameThread struct (that as the name suggests starts a running game-server in a thread) connected a player to the game and returned it, so you could only write integration tests that had one player connected to the world.

You also couldn't control the components that the player had, it was whatever the default components were when the player connected to the game server.

This meant that I could really only write tests for one new human player. If I wanted to connect to the game pretending to be an arbitrary entity or multiple arbitrary entities I couldn't

Since then I've written other integration tests such as the chase_entity_while_attacking test that led to the development of ways to connect and control arbitrary entities. In that case I connected an entity that could attacked and one that could be attacked and proceeded to assert against different expected behavior of two entities in combat.

// A snippet of how I initialize the components for entities that I'm controlling in an
// integration test.
// The game server allows clients to be sourced from the database or from a
// `ConnectedClientComponentsSource::Database` or
// `ConnectedClientComponentsSource::Provided`. In most integration tests we use the `Provided`
// varient in order to easily control the components of our test entities.

fn client_components_source() -> Arc<Mutex<ConnectedClientComponentsSource>> {
    let mut client_components_source = HashMap::new();

    client_components_source.insert(ATTACKER_USER_ID, Arc::new(attacker_components()));
    client_components_source.insert(TARGET_USER_ID, Arc::new(target_components()));

    let client_components_source =
        ConnectedClientComponentsSource::Provided(client_components_source);
    Arc::new(Mutex::new(client_components_source))
}

fn attacker_components() -> ComponentsToSpawnEntity {
    ComponentsToSpawnEntity {
        attacker: Some(AttackerComp::new()),
        main_action: Some(MainActionComp::new()),
        movement: Some(MovementComp::new()),
        tile_position: Some(TilePositionComp::new_1x1(5, 5)),
        trigger: Some(TriggerComp::new()),
        ..ComponentsToSpawnEntity::default()
    }
}

fn target_components() -> ComponentsToSpawnEntity {
    ComponentsToSpawnEntity {
        attackable: Some(AttackableComp::new()),
        hitpoints: Some(HitpointsComp::new(1_000_000, 10_000)),
        main_action: Some(MainActionComp::new()),
        movement: Some(MovementComp::new()),
        tile_position: Some(TilePositionComp::new_1x1(5, 5)),
        trigger: Some(TriggerComp::new()),
        ..ComponentsToSpawnEntity::default()
    }
}

Testing Quests

While working on the Tutor of War integration tests I've been adding new helper functions and assertions to the struct ConnectedPlayer that is used to connect to the GameThread.

Examples are a method to start a conversation with another entity and one to advance through that conversation.

I'm excited for five years from now when the testing infrastructure has been shaped by dozens of use cases. I already find it beautiful, albeit still nascent.

So I can't wait to see what a mature set of integration testing tools looks like for the game.

/// When you first play the game you start on Jaw Jaw island.
///
/// On this island are four tutors, each of which gives and/or teaches you something
/// that you'll need along your journey.
///
/// One of these is the Tutor of War.
///
/// Here we verify that the Tutor of War quest works as expected by playing through it.
#[test]
pub fn tutor_of_war_quest() -> Result<(), anyhow::Error> {
    let game = GameThread::new(client_components());
    let mut player = game.connect_player_tick_until_in_world(USER_ID)?;

    start_quest(&game, &mut player)?;

    unimplemented!(
        r#"
    Add tests that the cut scene happens. Not sure what this should mean yet so just add some
    assertions and feel it out.
    "#
    );

    Ok(())
}

/// Talking to the tutor of war
fn start_quest(game: &GameThread, player: &mut ConnectedPlayer) -> Result<(), anyhow::Error> {
    player.start_dialogue_with_display_name(TUTOR_OF_WAR_DISPLAY_NAME)?;
    game.tick_until(|| player.is_in_dialogue(), 5);

    player.assert_current_dialogue_node(TUTOR_OF_WAR_FIRST_INTRO_NPC);

    player.advance_dialogue(
        &[
            TUTOR_OF_WAR_CONFUSED_ABOUT_WELCOME_TO_THIS_WORLD_PLAYER,
            TUTOR_OF_WAR_I_SUPPOSE_HE_SENT_YER_FOR_TRAINING_NPC,
            TUTOR_OF_WAR_WHAT_TRAINING_PLAYER,
            TUTOR_OF_WAR_LETS_GET_STARTED_WITH_TRAINING_1_NPC,
            TUTOR_OF_WAR_LETS_GET_STARTED_WITH_TRAINING_2_NPC,
        ],
        &game,
    );

    // TODO: Define constants for each quest step to make this easier to read. Either by hand
    //  or auto generated. Need to think through what makes more sense
    player.assert_quest_step(QuestId::TutorOfWar, 10);

    player.advance_dialogue(&[TUTOR_OF_WAR_CONFUSED_ABOUT_REQUEST_TO_HIT_PLAYER], &game);

    Ok(())
}

Art Practice

I'm three weeks into practicing art every morning and I'm getting stronger each day.

At first my practice involved following tutorials, but these days I'm spending more of my practice time by modeling things from the physical world using only my eyes for reference.

Going forwards I'd like to start my day with two practice sessions. One where I model something on my own using inspiration and reference from the physical world, then a second where I follow a tutorial.

I'd like to strike a nice balance between building my confidence and artistic critical thinking skills by creating 3D models on my own (which is far more of a problem solving endeavor that I had initially realized) while still getting exposure to expert approaches by following tutorials.

This week I finished the elephant that I started in #065.

Felt like moving on before I made it to the trunk and tail. Even still, my stamina is improving and I'm gaining the ability to remain focused for longer while modeling. I'm still lacking in technique but that will come with practice.

My sense of good vs. bad topology is improving!

I'm now better able to instinctively recognize bad topology and I am developing an intuition for why it's bad.

This is evolving as I run into more problems that are caused by bad topology.

One example of this is when I want to widen something but I can't select a single edge loop since my topology is poor and I instead need to make a bunch of clicks to select and modify my topology.

Hey, miles of room for improvement, but trending upwards!

Right now I'm still at the point where everything that I make is very obviously bad, but at at this point I'm fully confident that with continued practice I'll be able to hit the point where what I make looks good to me and I need to get feedback from others in order to see all of the places that it's bad.

I'm excited to hit that milstonee - hopefully over the next few months of daily practice!

Other Notes and Progress

• One area that I want the game to shine is in the story and dialogue. I'm currently planning and writing the code to power cut scenes as there will be a short one included in the Tutor of War's dialogue.

• Wrote the first draft of the Tutor of War's dialogue. I'll want to edit and tweak it a bit but I'm happy with the direction of the character.

Next Week

I'm ready to take off the artistic training wheels.

Am I all of a sudden a grade A artist? Of course not.

But my deliberate practice has given me the confidence to know that it doesn't matter. Just keep making and keep improving.

So this week I'll be starting on creating real assets for the game.

I'll have my first art session in the morning be for real assets that I need. Then my second practice session will be something throwaway such as modeling something around my apartment or following a tutorial.

On the code side I'll be implementing cut scenes. It's the type of implementation where I'm not entirely sure of where it best fits, so I just have to pick a direction knowing that as I implement the best approach will begin to reveal itself.

So this week we're rolling up our sleeves and diving into that - learning as we go.

Lastly I want to start working on tooling at night more consistently. During the first couple of weeks of practicing art in the mornings my brain was so drained from exercising it in all of these new ways that by the time night came I had no mental energy to advance the game's tooling infrastructure.

That was the entire strategy though. Do art and gameplay first because those are the most important right now. Save tooling for last in the day because it's my favorite thing to work on so I'll have that carrot in front of me to motivate me to find a way to do it.

As art becomes less of a mental burden I'll be able to do more tooling in the evenings. I think this week is the week that I start dipping my toes into that part of the schedule.

Morning art, afternoon gameplay, evening tooling, and then on days that I'm doing consulting work I'll do it between the gameplay and tooling sessions.

In bed by 20:00. Or 21:00 if I'm in a flow state and need a bit more time. Then rinse and repeat the next day.

Cya next time!

- CFN