Letting Emacs into your grumpy heart

May 26th, 2017

Yes, the rumours are true. Your grumpy blogger, a long-time and generally satisfied Vim user has decided to embrace some fear, turn on EVIL mode and give Emacs a fair evaluation.

To be truthful, I had used Emacs before. Back in 2002 I was using Linux as my working environment and used it because the guy sitting next to me (hi Kemo, wherever you are!) was using it and showed me the basics. All I retained over the years was C-x C-f to load files and C-x C-c to quit.

Over the years I tried all sorts of editors -- Eclipse, JEdit, TextMate, Sublime Text -- before finally settling on Vim and builidng the request muscle memory along with the proper superior attitude.

I also have used PyCharm and PhpStorm because sometimes you need an IDE to help you when you are learning new languages beyond the beginner stage or dealing with a really complicated code base you let spin out of control. Right tool for the right job is a real thing, not just advice that developers who are bitter you don't like the tool they think is "right" give.

Vim is a great modal editor -- really powerful, and has a ridiculous amount of plugins that can extend Vim to make it even more useful. I've been able to get quite productive with it. At the same time, I can also get frustrated with how some stuff seemed really hard to get just right with plugins and how weird Vimscript is.

All the while I would see Emacs stuff bubble up to the surface on Twitter or YouTube would recommend that I watch a video about it. I came to realize that Emacs is very powerful and can do pretty much everything that Vim can do. Plus there is EVIL mode, which allows me to use Emacs with all the HJKL goodness I have come to expect from Vim.

I thought it might be constructive for me to go over my .emacs file as it currently stands and explain my choices. Honestly, it hasn't been that hard a transition using Emacs at this point. Retraining my brain to remember new key combinations is the harder part.

;;; Code:
(package-initialize)

(setq package-archives '(("gnu" . "http://elpa.gnu.org/packages/")
                         ("marmalade" . "https://marmalade-repo.org/packages/")
             ("melpa-stable" . "https://stable.melpa.org/packages/")
                       ("melpa" . "http://melpa.org/packages/")))

All I am gonna say here is that Emacs packaging is interesting.

;;; EVIL mode to help me transition from Vim to Emacs
(require 'evil)
(evil-mode 1)

EVIL mode is a really good Vim emulation mode for Emacs. It allows me to use most of the keystrokes I am comfortable with to move around within Emacs buffers.

;;; Some general settings
(setq make-backup-files nil)
(define-coding-system-alias 'UTF-8 'utf-8)
(setq inhibit-startup-message t)
(set-language-environment 'utf-8)
(set-default-coding-systems 'utf-8)
(set-selection-coding-system 'utf-8)
(set-locale-environment "en.UTF-8")
(prefer-coding-system 'utf-8)

I took these from Some Random Person's Emacs Settings. Mostly it's to make sure we do everything in UTF-8 and to not have my filesystem littered with all sorts of back-up files.

;;; Set up autocomplete
(require 'auto-complete-config)
(setq-default ac-sources (add-to-list 'ac-sources 'ac-source-dictionary))
(ac-config-default)
(global-auto-complete-mode t)

Autocompletition that works! Before you get all mad, I have used autocompletion in Vim for a long time but sometimes it wouldn't quite work properly. I have found Emacs autocomplete to be programming language sensitive, something I did not notice happening in Vim.

;;; YaSnippet
(yas-global-mode 1)

YASnippet is an Emacs mode that allows you to create language-specific templates to allow you to quickly generate code for things like for loops or if-then statements.

;;; PHP settings
(require 'php-mode)
(require 'php-auto-yasnippets)
(define-key php-mode-map (kbd "C-c C-y") 'yas/create-php-snippet)
(payas/ac-setup)
(setq php-auto-yasnippet-php-program "/Users/chartjes/.emacs.d/Create-PHP-YASnippet.php")

Still gotta do PHP work despite all my Python QA work! The PHP mode is pretty solid, and I like being able to generate snippets for my work. Remember kids, we only have so many keystrokes in our fingers.

;;; Some file mappings
(add-to-list 'auto-mode-alist '("\\.md\\'" . markdown-mode))
(add-to-list 'auto-mode-alist '("\\.py\\'" . python-mode))

Just telling Emacs what file endings map to what languages. Some programming-language modes can figure this out, Emacs wanted me to be specific about Markdown and Python

;;; Modeline
(defvar sml/theme)
(setq sml/theme 'powerline)
(sml/setup)

If you're a Vim user who liked a cool status line? Check out smart-mode-line

;;; Flycheck
(require 'flycheck)
(global-flycheck-mode t)
(setq flycheck-phpcs-standard "psr2")
(add-to-list 'flycheck-disabled-checkers 'python-pylint)
(add-hook 'python-mode-hook #'flycheck-mode)
(add-hook 'python-mode-hook
      (lambda () (flycheck-select-checker 'python-flake8)))
(require 'flycheck-color-mode-line)
(eval-after-load "flycheck"
  '(add-hook 'flycheck-mode-hook 'flycheck-color-mode-line-mode))

Flycheck will do on-the-fly syntax checking of your code. Just hook into the proper modes and you'll be good to go. Works great with PHP code too.

;;; JEDI autocompletion for Python
(add-hook 'python-mode-hook 'jedi:setup)
(defvar jedi:complete-on-dot)
(setq jedi:complete-on-dot t)

JEDI is a cool tool for doing Python-specific static analysis and syntax checking for your Python code. Highly recommend it.

;;; Mageit for Git
(require 'magit)
(global-set-key (kbd "C-x g") 'magit-status)

The only Emacs mode you will ever need to do work with Git

;;; Elm-specific settings
(require 'elm-mode)

Maybe one day I can do Elm for money.

;;; We will also need web mode stuff
(require 'web-mode)
(setq web-mode-ac-sources-alist
  '(("php" . (ac-source-yasnippet ac-source-php-auto-yasnippets))
    ("html" . (ac-source-emmet-html-aliases ac-source-emmet-html-snippets))
    ("css" . (ac-source-css-property ac-source-emmet-css-snippets))))
(add-hook 'web-mode-before-auto-complete-hooks
          '(lambda ()
             (let ((web-mode-cur-language
                    (web-mode-language-at-pos)))
               (if (string= web-mode-cur-language "php")
                   (yas-activate-extra-mode 'php-mode)
                 (yas-deactivate-extra-mode 'php-mode))
               (if (string= web-mode-cur-language "css")
                   (setq emmet-use-css-transform t)
                 (setq emmet-use-css-transform nil)))))

Web mode is intended to make working with web-centric languages easier in that it can recoginze when you have HTML and CSS code mixed in with your scripting language of choice, applying different syntax highlighting and styling rules to them. Really neat stuff

;;; Python settings
(require 'virtualenvwrapper)
(venv-initialize-interactive-shells)
(setq venv-location '("/Users/chartjes/Kinto/kinto-integration-tests/venv-kit"))

Did I mention that you can make Emacs aware of virtual Python environments?

;;; Settings for Corral (surrounding text with stuff)
(global-set-key (kbd "M-9") 'corral-parentheses-backward)
(global-set-key (kbd "M-0") 'corral-parentheses-forward)
(global-set-key (kbd "M-[") 'corral-brackets-backward)
(global-set-key (kbd "M-]") 'corral-brackets-forward)
(global-set-key (kbd "M-{") 'corral-braces-backward)
(global-set-key (kbd "M-}") 'corral-braces-forward)
(global-set-key (kbd "M-\"") 'corral-double-quotes-forward)

I have relied heavily on surround.vim in the past, Corral seems to be a great solution to do the same thing in Emacs.

So there you have it! My early experiences with Emacs are good, I am slowly learning new keystrokes to do new things and trying to get Emacs to intelligently help me be a better developer.

Async API testing in Python

April 26th, 2017

I've started liking the idea of creating contract-style tests for the APIs that power services at Mozilla. They are simply another line of defense against regressions and the accidental release of new functionality. As with all my work at Mozilla, it's being done in Python using pytest but thanks to some work from my co-worker Tarek I have some new tools at my disposal.

First, we've started making use of Swagger to get the developers to document their APIs. There is a wide variety of tools that work with Swagger API specs, but Tarek went a little further and put together Smwogger to make this type of testing even easier.

He describes it as "a smoke test tool for services described through Swagger" while I describe it as "a way for me to quickly write contract tests against an API spec". I'm pretty sure we're both right.

While you can use it as a CLI tool (the docs are geared towards that) he also thoughtfully added support for me to use the library from inside pytest. The only catch is that the tool uses asyncio, which is what Python 3 uses to support writing concurrent code.

I thought it would be way more difficult than it was but Tarek guided me through the extreme basics of concurrent Python code and after that I was able to write the tests I wanted. So here's some code for you to see:

import asyncio
import configparser
import pytest
from smwogger import API

from fxtesteng.helpers import aslist

@pytest.fixture(scope="module")
def conf():
    config = configparser.ConfigParser()
    config.read('manifest.ini')
    return config


@pytest.fixture(scope="module")
def event_loop():
    return asyncio.get_event_loop()


@pytest.fixture(scope="module")
def api(event_loop, conf, env):
    return API(conf.get(env, 'api_definition'), loop=event_loop)


@pytest.mark.asyncio
async def test_version(api):
    res = await api.__version__()
    data = await res.json()
    expected_fields = aslist(conf.get(env, 'version_fields'))

    # First, make sure that data only contains fields we expect
    for key in data:
        assert key in expected_fields

    # Then make the we only have the expected fields in the data
    for field in expected_fields:
        assert field in data


@pytest.mark.asyncio
async def test_heartbeat(api):
    res = await api.__heartbeat__()
    data = await res.json()
    expected_fields = aslist(conf.get(env, 'heartbeat_fields'))

    # First, make sure that data only contains fields we expect
    for key in data:
        assert key in expected_fields

    # Then make the we only have the expected fields in the data
    for field in expected_fields:
        assert field in data

I hope the code is clear (except for the async parts) but the idea is to use the API object from smwogger to make calls, using the ID's that the developer has assigned to that API call. I then loop through the expected fields for that call (they are stored in that manifest.ini file mentioned in the conf() fixture) and make sure everything matches our expectations.

The next bit of work I need to do is to use the Swagger spec some more and check the fields the API returns against what is actually in the spec instead of using pre-defined values in that manifest file. Making sure the developers follow their own specs is part of regression testing too!

A Different Twitter Experience

March 27th, 2017

It's no secret that I am on Twitter a lot. I mean, really a lot. Almost 100k worth of posts a lot. But as I use it more and more I find I was spending way more time treating Twitter as read-only, when what I wanted was to actually interact with people! Maybe that is a naive position to take, but it is what it is.

Then I stumbled upon this blog post where someone described an experiment they were trying out. The code for it was written in Node and I couldn't get it to work so like any pragmatic programmer I used the tools I am most comfortable with and wrote a version in PHP.

So every morning a cron job kicks off on a server and looks at the last 30 days of my Twitter timeline. It looks for users who I have "interacted" with during that time frame. What is "interaction"? It's defined as:

  • you replied to, favourited, or retweeted a tweet of mine
  • you favourited or retweeted a tweet I was mentioned in
  • I mentioned you in a tweet
  • I replied to or favourited a tweet of yours

The results so far have been interesting. I definitely see a different group of people on my feed than I have expected. Which is entirely the point! It has made me use Twitter differently...which again is the point. I'm far freer with my favourites and retweets than I was before. I'm also slowing down to actually read my timeline rather than just scroll through it super quickly like a trained monkey or something.

It's also making me be a lot less judgemental of the people who do end up on this list. While some of the people I follow are definitely blasting out stuff I am not all that interested in, I'm willing to take a chance and not just mute them as soon as I don't like the minutiae of their life. Diversity is a good thing and filter bubbles are not.

Another side effect of this has been some people proclaiming "I didn't know you were following me!" when I say something to them. Hate to burst your bubble but I'm not following you -- you are on my 'cycle' list and I can see your Tweets.

I know some folks follow accounts that send out emergency information and things like that -- I work from home so things like that are not of great use to me. YMMV etc etc.

If you're looking to have a different experience with Twitter, try my code out and see what kind of list it can generate for you. I'm going to keep using this for the near future because I feel like it's a worthy experiment. In fact, some Twitter accounts I felt were essential to follow have just slid off the list and I haven't missed them.