Learn To Test Like A Grumpy Programmer - Part 2

January 16th, 2019

(If you use PHP, you can learn how to write automated tests for your code via my "Learn To Test Like A Grumpy Programmer" course over at LeanPub)

In this blog post I wanted to talk about some of the lessons I've learned at the day job about how to use tools and techniques we are familiar with in the developer world to make our job of testing things easier.

Mozilla's Push Notification Service

One of the projects I do QA work for is the push notification service that Mozilla runs. Yes, I know a lot of people get supremely annoyed by push notifications. Yes, they are heavily abused by people. Mozilla tries to use them in a way not designed to annoy you. But this is the Internet after all, where everything annoys somebody somehow.

My work for that team involves doing testing of the service whenever a new version is ready to be released. The process goes like this:

  • the development team tags a new release along with a changelog
  • the operations team deploys this new release to the staging environment
  • I create a BugZilla ticket to track the deployment and testing results
  • I run a series of tests against the service on staging, recording my progress there

If all my tests pass on staging, I give approval to deploy that version of the service to production. The process for that is:

  • the operations team deploys the new release to production
  • I create another ticket to track the deployment process and testing results
  • I run the same tests and add an additional set of load tests to make sure the service is responsive

So what tests do I run?

API Contract Tests

I wrote some tests using pytest that make API calls using known values and verify that we are getting the responses that we expect.

These tests usually are not difficult to write and I did experiment with making them asynchronous using pytest-asyncio so I could learn how asynchronous code works in Python. If you had a very large test suite, I could see it being useful to speed up the process. This particular test suite is not big enough to warrant that.

For my fellow PHP folks, there are some plugins for PHPUnit that can run your tests in parallel. For one example check out PHPChunkit

Load Tests

I only run the load tests on the production version of the service because the number of nodes that handle push notification requests is higher in production. These tests are designed to put some non-trivial load onto the system and examine the output from the nodes for any error messages.

VAPID Testing

I use two different Android devices for these tests. Each device has latest stable and nightly releases of Firefox on them. One device is configured to point at staging, the other at production.

A web page is loaded that uses Javascript to generate VAPID-based notifications.

In the summer of 2018 I did experiment with seeing if I could automate these tests so I could use an online service offering cloud-accessible Android images. I did make some progress but it seems like some of the pieces needed to do things like make sure I don't have to click on the browser to accept notifications are either really brittle or don't exist, depending on what automation tools you are using.

Desktop Notification Tests

The person who did this testing work before me created a Nodejs app that serves up an web page with some forms and buttons on them. You fill in some values and it creates a series of push notifications that you visually verify work. He then put it inside a Docker container to make it easier for others to run those tests.

This test used to be on a personal web page of a (now-former) Mozilla employee.

Desktop WebPush Tests

These tests are to make sure that a feature of the service where you can group WebPush messages together by topic, only displaying the last one, works correctly.

The process for these tests are:

  • checkout a GitHub repo that contains the test
  • configure everything according to the documenation
  • start up the "topic server" via the CLI, which is serving up a one-page Cyclone web application
  • click on a button fire up a service worker that receives requests to send a WebPush notification
  • then use another CLI tool to fire off notifications (with or without a topic)
  • visually verify that the notifications appeared

Given that I already had a Docker-based solution, I decided it was time to turn this test into something where I only have to click a few buttons instead of running things via the CLI.

The first step was to update the existing one page app to have two more buttons. So I edited the HTML to add some buttons and then added some JavaScript that made calls to two new URL's for the app.

I then added those two URL's to the Cyclone app, porting over code from the CLI "topic pusher" tool to generate the notifications.

There was already a Dockerfile for the application, so I used that and then spent some time building and rebuilding and debugging the application to make sure it did actually make it so all I had to do was the following:

  • download the Docker image
  • start running the image (making sure to tell Docker to forward the correct port)
  • click on the Subscribe button and see the message that the service worker is running and the browser asks to accept notifications
  • click on the button that sends a single notification and visually verify output
  • click on the button that sends multiple notifications grouped by topic and visually verify output

So now I have a test that I can point other folks to that they can use without having to install any dependencies in their local environment other than a web browser and Docker.

Programming Skills + QA work == Solving Interesting Problems

I am far from the only person who has this skill set. But having some ability to create your own purpose-specific tools means that the people around you get to benefit.

Often the tools (and automation solutions) that folks use to test things are proprietary and not open to be modified. So you expend a lot of energy trying to bend a tool towards a new purpose.

There are other folks out there like me who busy creating wrappers around hard-to-use tools or creating new solutions with the goal of making what used to be difficult a lot easier. Encourage those people and promote what they are doing!

Learn To Test Like A Grumpy Programmer - Part 1

December 18th, 2018

(You can see more stuff like this blog post if you buy my "Learn To Test Like A Grumpy Programmer" course over at LeanPub)

I wanted to show folks how someone who has been writing tests for PHP in anger since 2003 actually implements new features for a web site, driving them with tests.

More importantly, I want you to focus on how deliberate and methodical I am in doing this work. I know it's an overused word, but discipline is a key factor in successfully using a TDD approach for your tests. Skipping steps or taking short cuts is a great way to end up with weird errors that you spend a very long time debugging.

I am building [OpenCFP Central])(https://opencfpcentral.com), which is a companion web application for OpenCFP. OpenCFP is a PHP web application that conference organizers can install and use to collect talk submissions from prospective speakers.

OpenCFP Central is a 100% free service.

Right now you can register for an account with OpenCFP Central. For conference organizers, you can obtain OAuth tokens so that people can use their OpenCFP Central account to login and submit talk ideas to OpenCFP. This helps to reduce one of the biggest complaints, which was "why do I have to keep creating OpenCFP accounts?!?"

The next feature I am building is allowing OpenCFP Central users to create talks and store them there. Once I've built that out, then I will add functionality to OpenCFP itself to retrieve a list of talks for a user who authenticated using OpenCFP Central and allow one-click submission of the talk to that OpenCFP instance.

OpenCFP Central is being built using Laravel starting with version 5.7. This is my first Laravel app but so many of the concepts it presents are not revolutionary or even unexpected for folks who have been doing web development with these "front controller style" web application frameworks.

So, with all that out of the way, let me show you how I implemented the first bit of functionality of the new feature: displaying an empty talk creation form for an authenticated user.

I started with a test that assumed everything was working:

namespace Tests\Feature\Feature;

use App\User;
use Tests\TestCase;

class TalkPage extends TestCase
{
    /**
     * @test
     */
    public function it_displays_the_talk_form()
    {
        /**
         * As a logged-in user
         * When I follow the route for "click here to create a talk"
         * I should see the talk form being displayed
         */
        $user = Factory(User::class)->create();
        $response = $this->actingAs($user)
            ->get(route('create_talk'));
        $response->assertStatus(200);
        $response->assertSeeText('Create A Talk');
        $response->assertSeeText('Title');
        $response->assertSeeText('Description');
        $response->assertSeeText('Other Details');
    }
}

Of course, it failed because I did not have the route created. So I went and added this route to routes/web.php

Route::get('/talk/create', 'TalkController@create')->name('create_talk');

Now the test fails because I don't have a TalkController. I used the artisan CLI tool to create an empty TalkController and modified it to use the auth middleware.

namespace App\Http\Controllers;

use Illuminate\Support\Facades\Auth;
use Illuminate\View\View;

class TalkController extends Controller
{
    /**
     * Create a new controller instance.
     *
     * @return void
     */
    public function __construct()
    {
        $this->middleware('auth');
    }
}

The test fails because I do not have the expected method for TalkController created yet. Let's add that in:

namespace App\Http\Controllers;

use Illuminate\Support\Facades\Auth;
use Illuminate\View\View;

class TalkController extends Controller
{
    /**
     * Create a new controller instance.
     *
     * @return void
     */
    public function __construct()
    {
        $this->middleware('auth');
    }

    /**
     * Show the application dashboard.
     *
     * @return \Illuminate\View\View
     */
    public function create(): View
    {
    }
}

Test still failing (as expected) because it's not doing anything. I'll add some code where I grab our authenticated user and ask it to display a specific view.

<?php

namespace App\Http\Controllers;

use Illuminate\Support\Facades\Auth;
use Illuminate\View\View;

class TalkController extends Controller
{
    /**
     * Create a new controller instance.
     *
     * @return void
     */
    public function __construct()
    {
        $this->middleware('auth');
    }

    /**
     * Show the application dashboard.
     *
     * @return \Illuminate\View\View
     */
    public function create(): View
    {
        $user = Auth::user();
        return view('talk.create')
            ->with('user', Auth::user());
    }
}

Now the test fails because that view does not exist. I need to create the appropriate directory structure to group views related to talks and create a blank one that uses our existing layout.

@extends('layouts.app')

@section('content')
    <div class="container">
        <div class="row justify-content-center">
            <div class="col-md-8">
                <div class="card">
                    <div class="card-header">Create A Talk</div>
                    <div class="card-body">
                        <p>
                            Use the form below to create a talk that can be submitted to OpenCFP instances that are
                            using OpenCFP Central
                        </p>
                    </div>
                </div>
            </div>
        </div>
    </div>
@endsection

Test still fails because there is no form information in there. So I created a partial view that contains the form fields I want. I did it without the opening and closing form tags because I plan on reusing this form snippet elsewhere. As part of a future test I need to modify this snippet to set default values as part of verifying you can actually succesfully create or update a test. Sometimes it's okay to think a little bit ahead. This strikes me as a very minor detail.

<div class="form-group">
    <label for="talk_title">Title</label>
    <input type="text" class="form-control" name="talk_title" />
</div>
<div class="form-group">
    <label for="description">Description</label>
    <textarea class="form-control" name="talk_description"></textarea>
</div>
<div class="form-group">
    <label for="other">Other Details</label>
    <textarea class="form-control" name="talk_other"></textarea>
</div>

Next I updated the main "create talk" view to use this form and wrapped it form tags that POST the results to another method.

@extends('layouts.app')

@section('content')
    <div class="container">
        <div class="row justify-content-center">
            <div class="col-md-8">
                <div class="card">
                    <div class="card-header">Create A Talk</div>
                    <div class="card-body">
                        <p>
                            Use the form below to create a talk that can be submitted to OpenCFP instances that are
                            using OpenCFP Central
                        </p>
                            <form method="post" action="/talk/create">
                                @csrf
                                @include('talk.form')
                                <button type="submit" class="btn btn-primary">Create Talk</button>
                            </form>
                    </div>
                </div>
            </div>
        </div>
    </div>
@endsection

Now the test passes!

I hope this blog post helps you understand what a TDD flow with an app under real conditions looks like.

Why GrumpyConf Has No Sponsors

November 8th, 2018

GrumpyConf 2019 will mark the 7th conference I've been involved in running:

  • CakeFest in Orlando, Florida, USA in 2008
  • TrueNorthPHP in Mississauga, Ontario, Canada from 2012 until 2016
  • GrumpyConf in Ingersoll, Ontario, Canada starting in 2018

As I helped organize these events, I quickly came to understand that the role of sponsors is to help offset the costs of running these events. In exchange, you promise to give them access to your attendees in a number of ways:

  • keynotes/talks/hackathons related to their products
  • booths or tables for outreach
  • paying for meals and/or snacks
  • swag to give to attendees and sometimes speakers

I have never been shy about asking for sponsorship of my events because they are expensive to run and I have certain ideas and standards I want to stick to.

But what do you do when the normal ways you have to reward sponsors aren't available? This is the dilemma I face with GrumpyConf.

The idea of GrumpyConf is to have a small event at a somewhat isolated location and set up conditions where the attendees mingle with speakers who have experience solving tough problems and have great personal skils. If you're looking for an event that will show you a path to "levelling up" your skills to further your career, this is the one for you.

The event is not cheap because it's at a nice resort and the ticket includes three nights' stay and meals. Plus I pay my speakers' travel expenses and hotel. So I set the prices to make sure I could break even if the same number of folks come who did last year. It's entirely funded out of my pocket through ticket sales and whatever other expenses I need to cover. As long as I don't have to put a lot of my own money into the event, I can continue to run it. I am not a charity and will not run an event at any kind of substantial loss.

I would love sponsorship of my event to defray the costs of my speakers. But I am at a lost to figure out what I can give these generous sponsors in exchange for their money.

If you are interested in sponsoring GrumpyConf 2019, please get in touch with me. My contact information is in the sidebar (on desktop) or all the way at the bottom in mobile versions.