Some Testing Theory

May 14th, 2020

TL;DR - Focus on testing behaviours instead of testing code

(If you like the work I have done with testing PHP code and OpenCFP, please consider sponsoring me at https://github.com/sponsors/chartjes/)

Some Testing Theory

While doomscrolling on Twitter, I saw internet-friend Snipe ask the following question:

When writing unit tests, do you typically write a test to check if the model is saved (i.e. create it via factory, check it passes built-in validation)? Feels a bit too much like testing the framework (or the factory) to me

To which I responded

Personally I am writing unit tests to verify behaviour, and if it requires making sure the model was saved then I will check

Snipe followed-up-with:

In acceptance or functional tests, sure - just seems weird to have them in unit tests.

Her response got me to thinking about some testing theory ideas that have changed how I approach the tests I write and how I categorize a particular test. As always, there are multiple approaches to solving these problems -- "trust, but verify" is a good practice.

Test Behaviours, Not Code

No matter what type of test I am writing (more on that later) I always ask myself "what test will prove this code is behaving as expected?". This is a different approach from "what parts of the code am I going to test". My experience has been that when you focus on testing behaviours, you end up writing fewer tests but with the same level of coverage of the code under test.

Focussing on behaviour also means you do not have to make context switches when you start thinking about tests of different types.

Test Types

Commonly these have been referred to as unit vs integration vs acceptance. Labels can change over time and right now I have settled on three types of tests:

Microtests

Microtests are tests that are verifying the behaviour of a single object in isolation. Calling them unit tests works here too. The additional pressure being applied here is whether or not you will use real versions of the dependencies the code you are testing requires or if you will create test doubles.

Both approaches have benefits and drawbacks. Tests with doubles tend to run quickly but have the maintenance overhead of needing to be updated if the behaviour of the doubles no longer matches the dependency. Tests what use real dependencies are slower and can have the maintenance overhead of needing databases or services to be made available and updated on a regular basis.

These are typically written a testing framework and can be automated via CLI tools.

Integration Tests

Integration tests are tests that verify that the behaviour of two objects, when interacting with each other, is as we expect. These tests should almost always use real dependencies unless there is a really good reason not too. Maybe something like an API with sandbox access because the API you are using in production is rate limited or charges per use. These things should be exceptions rather than a common practice.

These are also typically written with a testing framework and can be automated via CLI tools. The goal of this level of tests is to act as a filter that catches any bugs that your first layer of microtests missed.

Acceptance Tests

Acceptance tests are tests that verify that the behaviour of the application is correct, meaning that multiple objects will be interacting with each other using real dependencies. These sort of tests are usually conducted manually or built using some kind of automation framework that can drive a client application (usually a web browser).

Just like the integration tests, this layer should be catching any bugs your microtests and integration tests didn't find. Tests are usually written by humans, so there are some scenarios and edge cases that were not considered when the tests were written. All you can do is write code as defensively as possible, carefully consider your testing scenarios, and hope that nothing goes horribly wrong in production.

Back to Snipe's Question

So the original question is "should my unit tests be checking that data is saved?". The answer, in my mind, is that if the behaviour you are testing requires you to verify that data that was just created is saved and contains data you expect, then you will need to use real models with a real database connection.

I emphasize there is no wrong answer to Snipe's question! It is a matter of deciding on an approach and dealing with the associated technical debt.

So you find yourself working from home...

March 23rd, 2020

So, you've found yourself suddenly working from home as part of the massive upheaval CORVID-19 has caused worldwide. I have worked from home for the past 13 years for a variety of companies so I feel like I have the "work from home" process working correctly.

Even long-time remote workers like myself are struggling to stay on track and focussed on work. It's okay to feel overwhelmed. This is not a normal situation. As always, take care of your mental health and physical health.

Before I go any further, I want to acknowledge that what I am going to talk about works for me under my particular set of circumstances. Lots of this advice will likely work for you too.

Location

A dedicated work space is the best thing you can do to build what I think is the tripod that will support you -- comfort, convenience, and discipline. When I first started working from home, I worked from a desk in my unfinished basement. Once I got tired of being cold all the time, I relocated to my dining room table. We then got our basement renovated to include a home office, and I worked from there for 10 years until we moved to our current location. Where I live right now I have a home office on the main floor.

Now, I realize that this is not viable for everyone. You probably lack the space for a separate room. If you can't get dedicated space with a door you can close when you need to concentrate, I suggest picking a spot where you living and make that where you will work. When not working, try not to be in that spot unless you have to. Harder to do if you are not using a laptop, I know.

One of the key coping mechanisms is making sure you can put a clear separation between work life and home life. Being at home constantly can make it feel like you never get a break from work, especially if you are prone to overworking/workaholic tendencies.

Equipment

There is no special set of equipment that will make you more productive when working from home. I mean, here is my list of what I use every day:

  • Late 2016 13" MacBook Pro (run closed and connected to display)
  • Logitech MX Master 2S (with an Apple Magic Mouse as backup)
  • Advantage2 LF Kinesis keyboard (to stay out in front of RSI)
  • Dell P2715Q display
  • Logitech web cam
  • Blue Yeti Mic
  • Senheiser HD 558 headphones

I think the bare minimum you need for working from home is:

  • web cam (built-in is good enough, don't kid yourself)
  • headphones

Everything else depends on budget and the context of where you are working.

I work from an IKEA desk using a Herman Miller Aeron chair for Large Humans that I bought in 2014, both of them adjusted at the correct height for my particular ergonomics. I am an Old Man Using A Computer (I turn 49 tomorrow) so I also have fairly large font sized and the monitor at a height to not make me strain my neck or my back.

I am one of those people who have never been able to get comfortable working on a couch or laying in bed. I encourage you to make a choice that is both comfortable and does not lead to long-term injuries due to poor posture and having your wrists at an angle that is not good for it.

It took me maybe six months to get back to my normal typing speed when I got my ergonomic keyboard.

Communication

This is the hardest part of working remotely. I have worked at places that have done this well (Mozilla is the best so far) and lots of places where it was not done well. In my experience, the key to good communication with a remote workforce is that everyone uses the same communication channels. This means, at minimum

  • email
  • text-based chat
  • video conferencing

Effort also needs to be made to document what was discussed so anyone who could not be present for anything in those three communication channels can be brought up to speed.

Every place that I have worked where the decisions made by people caused lots of friction where because they were made outside of the "company" channels. Making key decisions during cigarette breaks or after-work dinners is a sure way to make a large number of your employees angry at your choices.

Discipline

To be very blunt -- people can find just as many ways to screw around and waste time in a centrally-located work environment as they can working remotely. If you screw around long enough in any work environment, you are likely to get fired.

My suggestion is to try and stay focussed on the tasks at hand, and allow yourself to blow off steam whenever you have completed one of those tasks. Learning how to decide what needs to be done for any task is a critical skill you will need, since remote work is often asynchronous and lacking in real-time answers.

Again, there are no silver bullets or special tools to handle this. Over the years I have found the book "The Mikado Method" to be very helpful.

Social Interaction

Again, to be blunt, you are as socially isolated as you want to be when working remotely. Under normal circumstances you can always go out and maybe work from a coffee shop or go to a co-working space. In our current COVID-19 situation, those are not options that you can choose. I stay in touch with my friends via text chats (both group ones online and individual ones) and video chats. I deliberately choose hobbies that have a very large social part to them. All of this is to ensure I do not end up feeling isolated and lacking in human contact. Working from home also allows me to spend time with my family, and that contact goes a long way towards making sure my mind is focussed on the important things in my life.

In other words, stay in touch with the people in your life who are important to you. Even a phone call to a friend can do wonders for your mood at times where you feel isolated and ignored.

Make The Right Choices

Obviously during this time you really have no choice -- it's work from home or don't work at all. Working remotely is not for everyone but it has been a great fit for me.

Engineering Diary -- Building an API using TDD

February 22nd, 2020

TL;DR - using Test-Driven Development to build an API

(If you like the work I do on OpenCFP, please consider sponsoring me at https://github.com/sponsors/chartjes/)

OpenCFP Central is a companion web application that I run that works with OpenCFP. Right now people can sign up for an account at OpenCFP Central and get an key that can be used to allow OpenCFP to use OpenCFP Central accounts as authentication. This feature was one that was requested for a long time, and I'm happy I can offer it to folks using a more recent versions of OpenCFP.

The next feature I am wanting to build is to add an API to OpenCFP Central that allows talks to be stored there and then can be pulled and submitted to any OpenCFP application that has been updated to include that ability. I'm already using OAuth so I can make the API use that.

OpenCFP Central is currently a Laravel 6 application. I'm using Passport for my OAuth needs. My initial scan of the documentation seems to indicate that is straightforward (but not necessarily easy) to do what I am trying to do.

Models and Resources

This being the first time I've built anything related to an API with Laravel, it looks like the path-of-least-resistance is to use as much of the built-in tools that Laravel gives me. I settled on sending things back and forth as JSON. Looking at the documentation it appears I should be using resources and resource collections. They will in turn become JSON responses via some code behind the scenes.

What makes this appealing to me is that given the lack of time (basically one day a week) I can give my side projects, I can have a lot of the plumbing for these API calls already done.

So, I can use my existing User model and just add a relationship to say that "a User can have one or more Talks"

/**
* @return HasMany
*/
public function talks(): HasMany
{
    return $this->hasMany(Talk::class);
}

Next I have to go back and create my Talk model. But before that I need to create a migration to add it to the database. I decided to keep things simple:

  • an ID
  • a UUID I will use for display purposes
  • the ID of the user the talk belongs to
  • the details of the talk stored as JSON

I am using PostgreSQL as my database and it has a "JSONB" field type.

<?php
declare(strict_types=1);

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class NewTalksTableAsJsonb extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('talks', function (Blueprint $table) {
            $table->increments('id');
            $table->uuid('uuid');
            $table->integer('user_id');
            $table->jsonb('details');
            $table->datetime('created_at');
            $table->datetime('updated_at');
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('talks');
    }
}

The details field needs to contain whatever is the data for an OpenCFP talk that is not configurable by whomever is running it. I believe this to be the following:

  • title of the talk
  • description of the talk
  • and then any other details the speaker wishes to share

So I think a typical JSON payload for an individual talk will look something like this:

{
  "user_id": 1,
  "uuid": "someuuidweusetodisplay",
  "title": "Test Title",
  "description": "Test Description",
  "other": "Things the speaker might want the organizers to know about"
}

With that design in place, time to write my first test. I am going to start with "get one talk for a user. As I always do, I start off with a skeleton of a test with just enough code in it to run and generate a failing response.

<?php
declare(strict_types=1);

use App\Talk;
use App\User;
use Tests\TestCase;

class GetTalkViaApiTest extends TestCase
{
    /**
     * @test
     */
    public function correctlyRetrievesTestAssociatedWithUser(): void
    {
        /**
         * Create a test user
         * Create a talk associated with that user
         * Save the talk
         */

        /**
         * Make an API call to /api/talk/user/{id}
         */

        /**
         * Assert that the JSON response matches what was created
         */
        $this->assertTrue(false);
    }
}

Those three comment blocks represent the three parts of the Arrange Act Assert pattern that I like to use to organize my tests. As always with my tests, I did some prototyping to figure out the best way to do the assertions of the JSON data. Once I figured out the approach I wanted to use, I was ready to start writing the test

So now I need to create a user for the test. I have another test that creates users for testing purposes, so I will use that.

$user = factory(User::class)->create();

Then I create a talk associated with that test and save it to the database

$talkDetails = [
    'title' => 'Test title',
    'description' => 'Test description',
    'other' => 'Test other'
];
$talk = new Talk(); 
$talk->user_id = $talkDetails['user_id'];
$talk->uuid = Uuid::uuid5(Uuid::NAMESPACE_DNS, 'php.net');
$talk->details = json_encode($talkDetails, JSON_THROW_ON_ERROR);
$talk->save();

Next, I have to make a call to my API end point as a user who is authenticated, using a Laravel helper

/**
 * Make an API call to /api/talk/{id}
 */
$response = $this->actingAs($user, 'api')->get('/api/talk/' . $talk->id);

Without adding any more assertions, I run the test to verify that just the assertion I have there is not passing. I am seeing a failure because I have not configured the API route that I need.

According to the documentation, I can set up the route and do the work inside the closure. In this case I want to make sure my query makes sure to return a response only if the user associated with the talk is correct.

This is what my route looks like:

Route::middleware('auth:api')->get('/talk/{id}', function (Request $request, $id) {
    return \App\Talk::find(['id' => $id, 'user_id' => $request->user()->id]);
});

With that done, I now can add in the code I am using for my assertions.

/**
 * Assert that the JSON response matches what was created
 */
$response->assertOk();
$talkViaApi = json_decode($response->content(), true, 512, JSON_THROW_ON_ERROR);
$this->assertEquals($expected['id'], $talkViaApi[0]['id']);
$this->assertEquals($expected['user_id'], $talkViaApi[0]['user_id']);
$this->assertEquals($expected['uuid'], $talkViaApi[0]['uuid']);

/**
 * Because we cannot guarantee the order of things in JSON, we have to look
 * inside the details field a little differently
 */
$talkDetailsFromApi = json_decode($talkViaApi[0]['details'], true, 512, JSON_THROW_ON_ERROR);
foreach ($talkDetails as $field => $value) {
    $this->assertArrayHasKey($field, $talkDetailsFromApi);
    $this->assertEquals($talkDetailsFromApi[$field], $value);
}

This test passes! Next test I need is one where I create talk associated with one user but pass in the wrong user id. It should return a 404 to let me know that it could not find it.

The test initially looks like this:

    public function cannotAccessATalkNotBelongingToADifferentUser(): void
    {
        /**
         * Create a test user
         * Create a talk associated with that user
         * Save the talk
         */
        $user = factory(User::class)->create();
        $this->be($user);

        $talkDetails = [
            'title' => 'Test title',
            'description' => 'Test description',
            'other' => 'Test other'
        ];
        $talk = new Talk();
        $talk->user_id = $user->id - 1;
        $talk->uuid = Uuid::uuid5(Uuid::NAMESPACE_DNS, 'php.net');
        $talk->details = json_encode($talkDetails, JSON_THROW_ON_ERROR);
        $talk->save();

        /**
         * Make an API call to /api/talk/1
         */
        $response = $this->actingAs($user, 'api')->get('/api/talk/' . $talk->id);

        /**
         * Assert that we are getting back a 404
         */
        $response->assertNotFound();
    }

The test fails, telling me I am getting a 200 instead of a 404. Time to back to our route and write some logic to generate the 404. So the code inside the closure now needs to look like this:

Route::middleware('auth:api')->get('/talk/{id}', function (Request $request, $id) {
    $talk = Talk::find($id);

    if ((int)$talk->user_id === (int)$request->user()->id) {
        return $talk;
    }

    abort(404);
});

I run the tests...and it fails, complaining that it can't find things at index 0 in the array that has my results in it. I see that just returning the talk means I get one record instead of an array that contains one record. I fix the test to reflect that change and now I have a completely passing test.

Here is the entire test and I hope this blog post helps you get better at testing the API's you are building.

<?php
declare(strict_types=1);

use App\Talk;
use App\User;
use Ramsey\Uuid\Uuid;
use Tests\TestCase;

class GetTalkViaApiTest extends TestCase
{
    /**
     * @test
     */
    public function correctlyRetrievesTestAssociatedWithUser(): void
    {
        /**
         * Create a test user
         * Create a talk associated with that user
         * Save the talk
         */
        $user = factory(User::class)->create();
        $this->be($user);

        $talkDetails = [
            'title' => 'Test title',
            'description' => 'Test description',
            'other' => 'Test other'
        ];
        $talk = new Talk();
        $talk->user_id = $user->id;
        $talk->uuid = Uuid::uuid5(Uuid::NAMESPACE_DNS, 'php.net');
        $talk->details = json_encode($talkDetails, JSON_THROW_ON_ERROR);
        $talk->save();

        $expected = [
            'id' => $talk->id,
            'user_id' => $user->id,
            'uuid' => $talk->uuid->toString(),
            'details' => $talk->details,
        ];

        /**
         * Make an API call to /api/talk/{id}
         */
        $response = $this->actingAs($user, 'api')->get('/api/talk/' . $talk->id);

        /**
         * Assert that the JSON response matches what was created
         */
        $response->assertOk();
        $talkViaApi = json_decode($response->content(), true, 512, JSON_THROW_ON_ERROR);

        $this->assertEquals($expected['id'], $talkViaApi['id']);
        $this->assertEquals($expected['user_id'], $talkViaApi['user_id']);
        $this->assertEquals($expected['uuid'], $talkViaApi['uuid']);

        /**
         * Because we cannot guarantee the order of things in JSON, we have to look
         * inside the details field a little differently
         */
        $talkDetailsFromApi = json_decode($talkViaApi['details'], true, 512, JSON_THROW_ON_ERROR);
        foreach ($talkDetails as $field => $value) {
            $this->assertArrayHasKey($field, $talkDetailsFromApi);
            $this->assertEquals($talkDetailsFromApi[$field], $value);
        }
    }

    /**
     * @test
     */
    public function cannotAccessATalkNotBelongingToADifferentUser(): void
    {
        /**
         * Create a test user
         * Create a talk associated with that user
         * Save the talk
         */
        $user = factory(User::class)->create();
        $this->be($user);

        $talkDetails = [
            'title' => 'Test title',
            'description' => 'Test description',
            'other' => 'Test other'
        ];
        $talk = new Talk();
        $talk->user_id = $user->id - 1;
        $talk->uuid = Uuid::uuid5(Uuid::NAMESPACE_DNS, 'php.net');
        $talk->details = json_encode($talkDetails, JSON_THROW_ON_ERROR);
        $talk->save();

        /**
         * Make an API call to /api/talk/1
         */
        $response = $this->actingAs($user, 'api')->get('/api/talk/' . $talk->id);

        /**
         * Assert that we are getting back a 404
         */
        $response->assertNotFound();
    }
}