A Grumpy Programmer's Neovim Setup

November 12th, 2021

(Almost a year without posting! I suggest following me on Twitter for more content)

A Grumpy Programmer's Neovim Setup and "Tooling FOMO"

I have been a Vim user for many years. I learned the basics a long time ago, wrote all my books in Vim, and did development work for many years just inside the One True Editor. I learned to appreciate using a modal editor. I grew to like the idea of using my mouse as little as possible.

But the biggest problem was always the same one: tooling FOMO (Fear of Missing Out).

Vim does has a very rich plugins ecosystem designed to extend what that editor to do and give you an experience closer to what you might get from an IDE (Integrated Development Environment). Out of the box you don't get much with Vim if you're a programmer -- syntax highlighting for some common languages and that's about it.

So despite having invested many years into creating muscle memory for moving around inside Vim, I always ended up trying out other editors (using PHP-centric ones) because they had a lot of additional features that either not available via plugins or required heavy customization to get them to work.

Now, I have a PhpStorm subscription and do use it quite a bit. The associated tooling for it is extremely useful -- automated linting, static code analysis, easy searching for code, jumping to definitions, one-click running of tests. These are all things that help reduce the day-to-day friction of doing development work.

I've also dabbled in using Visual Studio Code which has great integration with Dockerized development environments and other tools (like GitKraken, the Git GUI I like to use) to, again, reduce friction.

Both editors have "Vim support" which allows me to use various keyboard shortcuts that seem to be permanently set in my mind. But at the end of the day, I find myself asking "why aren't I just using Vim if I can get 99% of what I want in an editor?".

Some time in the last few years I came across NeoVim, which had been marketing itself as a replacement for Vim and promoting that you could use something other than VimScript to create plugins. I'm not a plugin writer but I did recognize that they were trying to rewrite a tool while remaining as close in behaviour and functionality with Vim.

I installed it on my Mac and have been using it ever since.

Recently I watched recordings of content from an online Vim conference and what I saw in them made me decide to take another look at my NeoVim setup and give it another chance to become the daily driver.

So, here is how I have things setup.

Where's My NeoVim Stuff?

I chose to go with the configuration of having everything in ~/.config/nvim.

So the first file I want to share is my init.vim and I have interspersed some comments throughout it.

"
" MAIN CUSTOMIZATION FILE
"
"
set nocompatible
syntax on 
set encoding=utf8
filetype off

" Load our plugins
call plug#begin()
    source ~/.config/nvim/plugins.vim
call plug#end()

I use the built-in plugin management system. I have seen other ones being promoted, haven't seen a reason to switch.

filetype plugin indent on

" Do smart autoindenting
set smartindent
set autoindent

" I like linenumbers, thanks
set number

" Save temporary/backup files not in the local directory, but in your ~/.vim
" directory, to keep them out of git repos.
" But first mkdir backup, swap, and undo first to make this work
call system('mkdir ~/.vim')
call system('mkdir ~/.vim/backup')
call system('mkdir ~/.vim/swap')
set backupdir=~/.vim/backup//
set directory=~/.vim/swap//

" Keep undo history across sessions by storing it in a file
if has('persistent_undo')
   call system('mkdir ~/.vim/undo')
    set undodir=~/.vim/undo//
    set undofile
    set undolevels=1000
    set undoreload=10000
endif

" set search case to a good configuration http://vim.wikia.com/wiki/Searching
set ignorecase
set smartcase

These are just settings I found elsewhere to accomplish a series of housekeeping tasks. They don't seem to impact my day-to-day usage much.

" I like pretty colours in my terminal
set t_Co=256


" Let's get some good colours in our terminal
let $NVM_TUI_ENABLE_TRUE_COLOR=1
set termguicolors
color dracula 

" We want to use ripgrep for any grep commands
set grepprg='rg'

" Basic configuration options
set tabstop=4
set shiftwidth=4
set softtabstop=0
set smarttab
set expandtab
set wildmenu
set wildmode=list:longest,full
set ttyfast
set showmatch
set hlsearch
set incsearch
set backspace=indent,eol,start

" We always want to use UTF-8
set encoding=UTF-8
set fileencoding=UTF-8

Again, these are more boilerplate for setting what I want the default settings for things to be.

lua require('lsp-config')
lua require('formatting')
lua require('treesitter')
lua require('telescope-config')
lua require('nvm-cmp')

NeoVim introducted the ability to use Lua to extend functionality and create plugins. In my opinion this is a giant leap forward.

" Look for project-specific files
if filereadable("project.vim")
    source project.vim
endif

Finally, I have to ability to define some things that NeoVim should care about on a project-by-project basis.

Plugins

Ah yes, what most of you have been probably waiting for.

First, I use a set of plugins for general tasks:

  • color scheme
  • LSP (Language Server Protocol) support
  • tree-sitter support for use by parsers
  • code snippets with autocompletion
  • nice icons during tab-completion
Plug 'dracula/vim'
Plug 'junegunn/vim-easy-align'
Plug 'neovim/nvim-lspconfig'
Plug 'jose-elias-alvarez/null-ls.nvim'
Plug 'nvim-treesitter/nvim-treesitter', {'do': ':TSUupdate'}
Plug 'hrsh7th/cmp-vsnip'
Plug 'hrsh7th/vim-vsnip'
Plug 'onsails/lspkind-nvim'

I like to use FZF as a fuzzy finder inside NeoVim

Plug 'junegunn/fzf', { 'do': { -> fzf#install() } }
Plug 'junegunn/fzf.vim'
Plug '/usr/local/opt/fzf'

I like a very simple Git status indicator

Plug 'airblade/vim-gitgutter'

I then have some PHP-specific plugins

Plug 'tpope/vim-dispatch'
Plug 'StanAngeloff/php.vim'
Plug 'stephpy/vim-php-cs-fixer'
Plug 'jwalton512/vim-blade'
Plug 'noahfrederick/vim-laravel'

" Help for vim-laravel
Plug 'tpope/vim-projectionist'
Plug 'noahfrederick/vim-composer'

I also like to pick up on EditorConfig settings for a project.

Plug 'editorconfig/editorconfig-vim'

I also recently discovered how useful Telescope is for Nvim, especially when I can use FZF with it.

" Telescope support
Plug 'nvim-lua/plenary.nvim'
Plug 'nvim-telescope/telescope.nvim'
Plug 'sharkdp/fd'
Plug 'nvim-telescope/telescope-fzf-native.nvim', { 'do': 'make' }

I am also doing some TypeScript work right now.

" LSP support for TypeScript
Plug 'jose-elias-alvarez/nvim-lsp-ts-utils'

Finally, I have a bunch of plugins to make autocompletion work a lot better.

" nvim-cmp support
Plug 'neovim/nvim-lspconfig'
Plug 'hrsh7th/cmp-nvim-lsp'
Plug 'hrsh7th/cmp-buffer'
Plug 'hrsh7th/cmp-path'
Plug 'hrsh7th/cmp-cmdline'
Plug 'hrsh7th/nvim-cmp'
Plug 'hrsh7th/cmp-vsnip'

LSP support

For me, LSP is the big thing that suddenly made NeoVim so much more useful to me. I purchased a license for Intelephense so I had all my PHP needs covered and have found other plugins for other languages do a better job than my previous setup.

Here is the Lua file I am currently using for LSP support, most of which I simply copied from other people who seemed to know what they are doing.

require("null-ls").config {}

--- Configuration for LSP, formatters, and linters.
local nvim_lsp = require("lspconfig")

nvim_lsp["null-ls"].setup({ on_attach = on_attach })

-- short cut methods.
local t = function(str)
  return vim.api.nvim_replace_termcodes(str, true, true, true)
end

local on_attach = function(client, bufnr)
  local function buf_set_keymap(...) vim.api.nvim_buf_set_keymap(bufnr, ...) end
  local function buf_set_option(...) vim.api.nvim_buf_set_option(bufnr, ...) end

  -- Enable completion triggered by <c-x><c-o>
  buf_set_option('omnifunc', 'v:lua.vim.lsp.omnifunc')

  -- Mappings.
  local opts = { noremap=true, silent=true }

  -- See `:help vim.lsp.*` for documentation on any of the below functions
  buf_set_keymap('n', 'gD', '<cmd>lua vim.lsp.buf.declaration()<CR>', opts)
  buf_set_keymap('n', 'gd', '<cmd>lua vim.lsp.buf.definition()<CR>', opts)
  buf_set_keymap('n', 'K', '<cmd>lua vim.lsp.buf.hover()<CR>', opts)
  buf_set_keymap('n', 'gi', '<cmd>lua vim.lsp.buf.implementation()<CR>', opts)
  buf_set_keymap('n', '<C-k>', '<cmd>lua vim.lsp.buf.signature_help()<CR>', opts)
  buf_set_keymap('n', '<space>wa', '<cmd>lua vim.lsp.buf.add_workspace_folder()<CR>', opts)
  buf_set_keymap('n', '<space>wr', '<cmd>lua vim.lsp.buf.remove_workspace_folder()<CR>', opts)
  buf_set_keymap('n', '<space>wl', '<cmd>lua print(vim.inspect(vim.lsp.buf.list_workspace_folders()))<CR>', opts)
  buf_set_keymap('n', '<space>D', '<cmd>lua vim.lsp.buf.type_definition()<CR>', opts)
  buf_set_keymap('n', '<space>rn', '<cmd>lua vim.lsp.buf.rename()<CR>', opts)
  buf_set_keymap('n', '<space>ca', '<cmd>lua vim.lsp.buf.code_action()<CR>', opts)
  buf_set_keymap('n', 'gr', '<cmd>lua vim.lsp.buf.references()<CR>', opts)
  buf_set_keymap('n', '<space>e', '<cmd>lua vim.lsp.diagnostic.show_line_diagnostics()<CR>', opts)
  buf_set_keymap('n', '[d', '<cmd>lua vim.lsp.diagnostic.goto_prev()<CR>', opts)
  buf_set_keymap('n', ']d', '<cmd>lua vim.lsp.diagnostic.goto_next()<CR>', opts)
  buf_set_keymap('n', '<space>q', '<cmd>lua vim.lsp.diagnostic.set_loclist()<CR>', opts)
  buf_set_keymap('n', '<space>f', '<cmd>lua vim.lsp.buf.formatting()<CR>', opts)

end

-- PHP
nvim_lsp.intelephense.setup {
    cmd = { "intelephense", "--stdio" },
    filetypes = { "php" }
}

--- Linter setup
local filetypes = {
  typescript = "eslint",
  typescriptreact = "eslint",
  php = {"phpcs", "psalm"},
}

local linters = {
  phpcs = {
    command = "vendor/bin/phpcs",
    sourceName = "phpcs",
    debounce = 300,
    rootPatterns = {"composer.lock", "vendor", ".git"},
    args = {"--report=emacs", "-s", "-"},
    offsetLine = 0,
    offsetColumn = 0,
    sourceName = "phpcs",
    formatLines = 1,
    formatPattern = {
      "^.*:(\\d+):(\\d+):\\s+(.*)\\s+-\\s+(.*)(\\r|\\n)*$",
      {
        line = 1,
        column = 2,
        message = 4,
        security = 3
      }
    },
    securities = {
      error = "error",
      warning = "warning",
    },
    requiredFiles = {"vendor/bin/phpcs"}
  },
  psalm = {
    command = "./vendor/bin/psalm",
    sourceName = "psalm",
    debounce = 100,
    rootPatterns = {"composer.lock", "vendor", ".git"},
    args = {"--output-format=emacs", "--no-progress"},
    offsetLine = 0,
    offsetColumn = 0,
    sourceName = "psalm",
    formatLines = 1,
    formatPattern = {
      "^[^ =]+ =(\\d+) =(\\d+) =(.*)\\s-\\s(.*)(\\r|\\n)*$",
      {
        line = 1,
        column = 2,
        message = 4,
        security = 3
      }
    },
    securities = {
      error = "error",
      warning = "warning"
    },
    requiredFiles = {"vendor/bin/psalm"}
  }
}

nvim_lsp.diagnosticls.setup {
  on_attach = on_attach,
  filetypes = vim.tbl_keys(filetypes),
  init_options = {
    filetypes = filetypes,
    linters = linters,
  },
}

I have a separate configuration file for TypeScript:

local lspconfig = require("lspconfig")
local buf_map = function(bufnr, mode, lhs, rhs, opts)
    vim.api.nvim_buf_set_keymap(bufnr, mode, lhs, rhs, opts or {
        silent = true,
    })
end

local on_attach = function(client, bufnr)
    vim.cmd("command! LspDef lua vim.lsp.buf.definition()")
    vim.cmd("command! LspFormatting lua vim.lsp.buf.formatting()")
    vim.cmd("command! LspCodeAction lua vim.lsp.buf.code_action()")
    vim.cmd("command! LspHover lua vim.lsp.buf.hover()")
    vim.cmd("command! LspRename lua vim.lsp.buf.rename()")
    vim.cmd("command! LspRefs lua vim.lsp.buf.references()")
    vim.cmd("command! LspTypeDef lua vim.lsp.buf.type_definition()")
    vim.cmd("command! LspImplementation lua vim.lsp.buf.implementation()")
    vim.cmd("command! LspDiagPrev lua vim.lsp.diagnostic.goto_prev()")
    vim.cmd("command! LspDiagNext lua vim.lsp.diagnostic.goto_next()")
    vim.cmd("command! LspDiagLine lua vim.lsp.diagnostic.show_line_diagnostics()")
    vim.cmd("command! LspSignatureHelp lua vim.lsp.buf.signature_help()")    buf_map(bufnr, "n", "gd", ":LspDef<CR>")
    buf_map(bufnr, "n", "gr", ":LspRename<CR>")
    buf_map(bufnr, "n", "gy", ":LspTypeDef<CR>")
    buf_map(bufnr, "n", "K", ":LspHover<CR>")
    buf_map(bufnr, "n", "[a", ":LspDiagPrev<CR>")
    buf_map(bufnr, "n", "]a", ":LspDiagNext<CR>")
    buf_map(bufnr, "n", "ga", ":LspCodeAction<CR>")
    buf_map(bufnr, "n", "<Leader>a", ":LspDiagLine<CR>")
    buf_map(bufnr, "i", "<C-x><C-x>", "<cmd> LspSignatureHelp<CR>")

    if client.resolved_capabilities.document_formatting then
        vim.cmd("autocmd BufWritePre <buffer> lua vim.lsp.buf.formatting_sync()")
    end
end

lspconfig.tsserver.setup({
    on_attach = function(client, bufnr)
        client.resolved_capabilities.document_formatting = false
        client.resolved_capabilities.document_range_formatting = false        local ts_utils = require("nvim-lsp-ts-utils")
        ts_utils.setup({
            eslint_bin = "eslint_d",
            eslint_enable_diagnostics = true,
            eslint_enable_code_actions = true,
            enable_formatting = true,
            formatter = "prettier",
        })
        ts_utils.setup_client(client)        buf_map(bufnr, "n", "gs", ":TSLspOrganize<CR>")
        buf_map(bufnr, "n", "gi", ":TSLspRenameFile<CR>")
        buf_map(bufnr, "n", "go", ":TSLspImportAll<CR>")        on_attach(client, bufnr)
    end,
})

require("null-ls").config({})
lspconfig["null-ls"].setup({ on_attach = on_attach })

NeoVim completion

Here's the lua file I use to tweak all that

local cmp = require'cmp'
local lspkind = require('lspkind')

cmp.setup({
    formatting = {
        format = lspkind.cmp_format({with_text = false, maxwidth = 50})
    },
    snippet = {
        -- REQUIRED - you must specify a snippet engine
        expand = function(args)
            vim.fn["vsnip#anonymous"](args.body) -- For `vsnip` users.
        end,
    },
    mapping = {
        ['<C-d>'] = cmp.mapping(cmp.mapping.scroll_docs(-4), { 'i', 'c' }),
        ['<C-f>'] = cmp.mapping(cmp.mapping.scroll_docs(4), { 'i', 'c' }),
        ['<C-Space>'] = cmp.mapping(cmp.mapping.complete(), { 'i', 'c' }),
        ['<C-y>'] = cmp.config.disable, -- Specify `cmp.config.disable` if you want to remove the default `<C-y>` mapping.
        ['<C-e>'] = cmp.mapping({
            i = cmp.mapping.abort(),
            c = cmp.mapping.close(),
        }),
        ['<CR>'] = cmp.mapping.confirm({ select = true }),
    },
    sources = cmp.config.sources({
        { name = 'nvim_lsp' },
        { name = 'vsnip' }, -- For vsnip users.
    }, {
            { name = 'buffer' },
        })
})

-- Use buffer source for `/` (if you enabled `native_menu`, this won't work anymore).
cmp.setup.cmdline('/', {
    sources = {
        { name = 'buffer' }
    }
})

-- Use cmdline & path source for ':' (if you enabled `native_menu`, this won't work anymore).
cmp.setup.cmdline(':', {
    sources = cmp.config.sources({
        { name = 'path' }
    }, {
            { name = 'cmdline' }
        })
})

-- Setup lspconfig.
local capabilities = require('cmp_nvim_lsp').update_capabilities(vim.lsp.protocol.make_client_capabilities())
-- Replace <YOUR_LSP_SERVER> with each lsp server you've enabled.
require('lspconfig')['intelephense'].setup {
    capabilities = capabilities
}
require('lspconfig')['tsserver'].setup {
    capabilities = capabilities
}

Telescope

More Lua wizardry I stole from smarter people to make it work with FZF.

require('telescope').setup {
    fzf = {
        fuzzy = true,
        override_generic_sorter = true,
        override_file_sorter = true,
        case_mode = "ignore_case",
    }
}

require('telescope').load_extension('fzf')

Treesitter

This tells NeoVim to use all the currently maintained language parsers that is has available. Saves me time remembering to download ones for one-off file editing.

require'nvim-treesitter.configs'.setup {
    ensure_installed = "maintained",
    highlight = {
        enable = true,
        additional_vim_regex_highlighting=false,
    },
    indent = {
        enable = true
    },
}

Key bindings

I don't use too many as I find my grumpy brain requires a lot of repetition for the new habits to be build. They are mostly related to using Telescope to better navigate through files.

" easily switch between vsplit windows
map <Leader>j <C-w>j
map <Leader>k <C-w>k
map <Leader>h <c-w>h
map <Leader>l <c-w>l

" Remove highlighing of search terms
nnoremap <leader><space> :nohlsearch<CR>

" Mappings for EasyAlign
xmap ga <Plug>(EasyAlign)
nmap ga <Plug>(EasyAlign)

" Use <Tab> and <S-Tab> to navigate through popup menu
inoremap <expr> <Tab>   pumvisible() ? "\<C-n>" : "\<Tab>"
inoremap <expr> <S-Tab> pumvisible() ? "\<C-p>" : "\<S-Tab>"

" Telescope Lua mappings
nnoremap <leader>ff <cmd>lua require('telescope.builtin').find_files()<cr>
nnoremap <leader>fg <cmd>lua require('telescope.builtin').live_grep()<cr>
nnoremap <leader>fb <cmd>lua require('telescope.builtin').buffers()<cr>
nnoremap <leader>fh <cmd>lua require('telescope.builtin').help_tags()<cr>
nnoremap <leader>fr <cmd>lua require('telescope.builtin').lsp_references()<cr>
nnoremap <leader>fd <cmd>lua require('telescope.builtin').lsp_definitions()<cr>
nnoremap <leader>ft <cmd>lua require('telescope.builtin').lsp_type_definitions()<cr>

Final Thoughts

So, I think it is clear that you can get Vim/NeoVim to do many, many things that other popular IDE's can do. But you can also see that you will likely have to do a lot of digging around and copying other people's work to make it happen.

In my case, I realized I had been spending a lot of time fighting with other editors because I had gotten used to how Vim did things. Yes, Vim mode in other editors is actually helpful but it is not as good as using Vim itself.

So, I am going to make a commitment to use NeoVim with all the stuff above I have shared as my daily editor for the near future. Personally, I encourage you to find tools that fit the way you want to work and learn them as well as you can.

Testing Laminas API end points

December 11th, 2020

Testing Laminas API end points

As I write this, I have been working for Unnamed Financial Services Company for 3 months. It has been a good exercise in figuring out how to do a slow-but-steady migration of a CodeIgniter 3.x application to a macro-service where we are moving some things to an API and will move the other functionality (which is mostly used by the customer service folks at UFSC) into a different application.

For the API side of things I decided we should go with Laminas API tools due to tooling and the flexibility that we can get from writing our own glue code to solve particular problems. As much as I am an old veteran of full-stack PHP frameworks, our architectural plans leave me worried that I would end up fighting with the conventions of one of those types of frameworks too much.

So, having used the admin UI to create an API end point, and organizing our code that interacts with the database into some abstractions that I think make sense and provide us with some much-needed structure, I have a single-action controller that handles the API call:

<?php
declare(strict_types=1);

namespace Lead\V1\Rpc\FindById;

use Application\Repository\DoctrineLeadRepository;
use Doctrine\ORM\EntityManagerInterface;
use Laminas\Mvc\Controller\AbstractActionController;
use Laminas\ServiceManager\ServiceLocatorInterface;

final class FindByIdController extends AbstractActionController
{
    private EntityManagerInterface $entityManager;

    public function __construct(ServiceLocatorInterface $serviceLocator)
    {
        $this->entityManager = $serviceLocator->get('doctrine.entitymanager.orm_default');
    }

    public function findByIdAction(): array
    {
        $leadId = (int) $this->getRequest()->getQuery('id');
        $lead = (new DoctrineLeadRepository($this->entityManager))->findById($leadId);

        if ($lead !== null) {
            return $lead->toArray();
        }

        return [];
    }
}

Now the discussion worth having: how do we test this thing?

For pragmatic reasons I decided that I would not go the path of creating test doubles for everything and then using the Laminas service manager to replace the existing dependencies with doubles. I'll just use the real database that the DoctrineLeadRepository will be talking to and get on with a test.

So, we have two scenarios that we need to test:

  • Does it behave correctly if it cannot find a lead in the database
  • Does it behave correctly if it finds a lead and returns some response

Okay, so I get started with my test. The first step is creating a skeleton that reads in the Laminas-specific configuration options for the main Application module and then the module that contains our API controller action.

<?php

namespace ApplicationTest\Lead\V1\Rpc\FindById;

use Laminas\Stdlib\ArrayUtils;
use Laminas\Test\PHPUnit\Controller\AbstractHttpControllerTestCase;

final class FindByIdControllerTest extends AbstractHttpControllerTestCase
{
    private \Laminas\ServiceManager\ServiceManager $serviceLocator;

    protected function setUp(): void
    {
        $this->setApplicationConfig(ArrayUtils::merge(
            include __DIR__ . '/../../../../../../../config/application.config.php',
            include __DIR__ . '/../../../../../../../module/Lead/config/module.config.php',
        ));
        parent::setUp();
        $this->serviceLocator = $this->getApplicationServiceLocator();
    }
}

Okay, now a test for the first scenario:

<?php

    // Addtional dependency...
    use Lead\V1\Rpc\FindById\FindByIdController;


    /** @test  */
    public function it_handles_missing_lead_correctly(): void
    {
        $response = (new FindByIdController($this->serviceLocator))->findByIdAction();
        self::assertEquals([], $response);
    }

This one passes since the code in the controller action behaves as follows:

  • there is no value in the query for 'id'
  • so when it tries to retrieve a Lead it will get back null
  • so it returns an empty array

The next scenario was trickier. It was not obvious to me how I inject a query parameter into the request. I was used to other frameworks where I could add a parameter to the action of the controller and the framework would automagically inject that corresponding HTTP query parameter.

Using an old tactic, I started digging around in the tests for the Laminas MVC package to see how they were testing things. It took a while and some trial and error, but I did figure out.


// New dependencies added use Laminas\Http\Request; use Laminas\Mvc\MvcEvent; use Laminas\Router\RouteMatch; use Laminas\Stdlib\Parameters; /** @test */ public function it_finds_something_that_looks_like_a_lead(): void { $controller = new FindByIdController($this->serviceLocator); // Create what route we want to execute $routeMatchParams = [ 'controller' => 'Lead\\V1\\Rpc\\FindById\\Controller', 'action' => 'findById' ]; $routeMatch = new RouteMatch($routeMatchParams); // Build up the request that contains our lead ID $request = new Request(); $request->setQuery(new Parameters(['id' => 2])); $request->setMethod('GET'); $request->setUri('/lead/find'); // Create an event that the app is listening for // and tell the controller to use it $event = new MvcEvent(); $event->setRouteMatch($routeMatch); $event->setRequest($request); $controller->setEvent($event); // Get our response $result = $controller->dispatch($request); // Make sure that it is actually a lead self::assertEquals(2, $result['id']); }

While the test passes, in the next version I want to create a Lead as part of the test, store it in the database, and then make sure I retrieve the one I expect. Hard-coding is okay for the first pass but should not be in the final version.

Hopefully this blog post helps you solve your own Laminas-related problems faster than I did. ;)

Testing Legacy Apps - Episode 1

June 19th, 2020

(If you like the work I have done with testing PHP code and OpenCFP, please consider sponsoring me at https://github.com/sponsors/chartjes/)

Testing Legacy Apps - Episode 1

I have an application that has been in use by the people who participate in my longest-running hobby for more than a decade. It has no tests. I have no excuses other than laziness. Time to change that.

In this continuing series of blog posts I am going to show you how to start from having no tests and ending up with a test suite that covers the behaviour of your application that matters the most. Along the way I am going to teach you what I feel is a repeatable framework for approaching testing existing code.

I also want to emphasize that my approach doesn't depend on the framework you are using. Most PHP web application frameworks include helpers to make testing easier, and this is a good thing! But there is probably more PHP code out there either built without a web application framework or one without helpers for testing that we need to learn some new techniques for writing tests.

For this whole series there are some constraints we are going to be dealing with:

  • I'll be using PHPUnit to write the tests
  • I'll be refactoring code to make it easier to test
  • I won't be adding any new features

With those three conditions in place, let's pick something and get started.

What Is The Application's Domain

The application we are writing tests for is for handling transactions and managing rosters for a tabletop baseball simulation league. The most popular game of this genre is Strat-O-Matic Baseball and the league I am in uses a game we created ourselves.

I'm sorry if some of the terms I end up using are ones you don't understand as some domain knowledge is required to figure out tests.

I created a web application to manage all the roster stuff more than a decade ago, and slapped it together quickly because we needed something. Over the years it's been tweaked but I have been lazy and justified not having any tests for it because "I understand the domain well enough to manually test it". Shockingly, I still manage to break things.

As with any long journey, it begins with a step. I'm going to pick something that I constantly break and wrap some tests around it.

What is our first testing scenario?

As part of this whole series, I also want to emphasize that I will be focussing on not testing the code but coming up with tests for how the application is supposed to behave. To figure this out, I need to identify what parts of the application need to work all the time.

  • making trades should not break
  • signing free agents should not break
  • entries in the transaction log should always be correct

Those are the high level tests that need to be written. There are also some tests at a lower level that need to happen to satisfy the conditions above. I keep finding ways to break some functionality that deals with indicating whether or not a player had a "card" in the game (basically, did we print a card for that player for a specific year). Why does it keep breaking? Because I did not create one centralized location that is the source of truth for a player.

I've written tests for this functionality before as examples for my books and presentations but I feel like it is time to take a different approach and build something easier to work with.

When I am creating testing scenarios I like to use the language that people who practice Behavior-Driven Development tend to use. So here is a great testing scenario to start with:

Given I am a Player When I have no card for the current season Then indicate my uncarded status And indicate the season I was uncarded for

Now, when I look at the code I have already I am doing this...but I don't like it. I have created the idea of a Roster and created a 'model' for it and I just deleted some tests I had for it. Instead, I think I want to drill a little deeper into this problem and come up with a better fix.

A Roster is a collection consisting of one or more objects. They are "batters", "pitchers", and "draft picks". Right now I am not making that distinction. Here's the code that I wrote that looks at a player's "uncarded status" and figures out what additional information needs to be displayed next to the player's name:

    public function getBattersByIblTeam($ibl_team): array
    {
        $sql = "SELECT * FROM rosters r WHERE r.ibl_team = ? AND r.item_type = 2 ORDER BY r.tig_name";
        $stmt = $this->pdo->prepare($sql);
        $stmt->execute([$ibl_team]);
        $results = $stmt->fetchAll(PDO::FETCH_ASSOC);

        if (!$results) {
            return [];
        }

        $roster = [];

        foreach ($results as $row) {
            $displayName = trim($row['tig_name']);

            if ($row['uncarded'] === $this->previous_season || $row['uncarded'] === $this->current_season) {
                $displayName .= ' [UC' . $row['uncarded'] . ']';
            }

            $row['display_name'] = $displayName;
            $roster[] = $row;
        }

        return $roster;
    }

This is not bad code by any means -- it works. From a testing perspective there are all sorts of problems with it due t