Ali FarooqBook a call

Case study · Platform and APIs · 2019–2020

Mattermost

Plugin signing, the plugin Marketplace and cluster-wide plugin sync for Mattermost’s open-source messaging platform, then the first pull request, REST API design, OpenAPI docs and Go client for Incident Response, now Mattermost Playbooks.

Mattermost Playbooks on a laptop and a phone, showing an incident channel and its checklist, under the Mattermost logo
Mattermost Playbooks today, on desktop and mobile. Image from mattermost.com.
Company
Mattermost
Product
Open-source team messaging
Timeline
Jun 2019 – Oct 2020
My role
Senior software design engineer
Stack
Go · React · TypeScript · OpenAPI
Outcome
Playbooks, used to handle incidents
On this page
  1. ▶See it in action
  2. 01Challenge
  3. 02What I owned
  4. 03Approach
  5. 04Results
  6. 05Skills this shows
  7. 06For your project
  8. 07On the record
145
merged pull requests
10
repositories
PR #1
the first pull request in Playbooks
48
merged PRs on Incident Response

See it in action

The product as it ships today.

Mattermost’s own walkthrough of an incident response playbook. Playbooks grew out of Incident Response, the plugin I started in March 2020.

Mattermost demo library · 4 min

Challenge

Mattermost is an open-source, self-hosted alternative to Slack, built in Go and React for teams that need control of their own data. In 2019 its plugin system was growing fast, and three gaps stood out. A server would install any plugin bundle it was given, with no way to prove the bundle came from Mattermost or hadn’t been changed. Admins had no central place to find plugins. And in a high-availability cluster, a plugin installed on one node didn’t reach the others.

In 2020 Mattermost started Incident Response, its first product built as a plugin, to help teams coordinate and measure how they handle incidents. It needed a foundation the team could build on quickly, and an API other teams and clients could rely on.

What I owned

I joined the Toolkit and Workflows team in June 2019, working across plugins, the server and clustering, and moved to the Cloud Features team in August 2020.

  • Plugin signing. Designed and built signature verification for plugins across the server, web app and Redux, with a setting to require signatures and GPG public keys the server trusts.
  • Signed releases. Taught the release bot to split plugin binaries by platform, sign them and upload them (matterbuild #17), and made server packaging fail if a bundled plugin’s signature didn’t verify (#13629).
  • Plugin Marketplace. The server API that proxies the upstream marketplace and merges in local plugins (#11977), listings limited to the versions a server supports (marketplace #20), and a local-only fallback when the upstream is off or unreachable (#13250).
  • Clusters and the plugin API. Cluster events that sync plugin installs and removals across nodes (#11611), Prometheus metrics for plugin API calls and hooks (#13825), and direct database access for plugins (plugin-api #46).
  • Incident Response, now Playbooks. Built the first version, from PR #1, which set up the plugin’s API handler, store and models, to the slash command that starts an incident, the incident details panel, team scoping and playbook templates.
  • REST API, docs and Go client. Wrote the REST API section of the technical spec, then the OpenAPI spec and the official Go client, and brought the endpoints in line with them.

On Cloud Features I shipped features for Mattermost Cloud, and throughout I mentored community contributors and junior engineers.

Mattermost Playbooks showing a run as collapsible checklist sections, each with tasks and due dates

Playbooks today. Each run is a set of checklists with owners and due dates, grown from the incident checklist in the first versions. Image from mattermost.com.

Approach

“A mark of a good API design is that it is easy to use and hard to misuse.” That was the first line of the design discussion in my Go client proposal, and it shaped everything else.

Agree on the contract first. The REST API section of the spec set the rules before any endpoint moved. POST creates, PUT replaces and PATCH updates part of a resource, and everything but POST is idempotent. Every list returns the same paginated envelope, and an empty list is an empty array, never null. Endpoints that only serve the plugin moved under /internal, so clients see only the API they can rely on.

Then make the contract executable. The OpenAPI file describes every endpoint with curl examples, served with Redoc. The Go client follows the conventions of Stripe’s and GitHub’s Go clients. The endpoints were then brought in line, with standard error responses and paginated lists.

The workflows REST API docs served with Redoc, listing endpoints beside request and response samples A curl request sample for creating an incident, from the OpenAPI docs

The OpenAPI docs from PR #177, served with Redoc, and one of their curl samples. Screenshots from the pull request.

For signing, trust nothing by default. Marketplace plugins are always verified, and once signatures are required, only signed Marketplace plugins install at all. Signing happens in the release tooling, so no one signs by hand, and packaging the server fails if a bundled plugin doesn’t verify.

Results

  • Plugin signing shipped in the Mattermost server, and it is still how admins verify plugins, as Mattermost’s docs describe.
  • Incident Response, the plugin I started in March 2020, grew into Mattermost Playbooks, which teams use to handle incidents. Mattermost features it in its demo library.
  • The REST API conventions became a template other plugin teams followed.
  • 145 merged pull requests across 10 repositories, all reviewed in the open.

Skills this shows

If your role needs Evidence in this project
Platform and plugin systems Plugin signing, the Marketplace API and cluster-wide plugin sync in Mattermost’s server.
API design REST conventions, an OpenAPI spec and a Go client, designed together.
Security Signature verification for third-party code, from release tooling to install.
Starting from zero The first pull request of a product that now ships as Mattermost Playbooks.
Open source 145 merged pull requests, reviewed in public.

What this means for your project

  • Building a platform other teams extend? I design the extension points, the docs and the client together, so other teams can build on it without waiting on me.
  • Running code you didn’t write? I’ve built signing and verification from release tooling to install time, so a server only runs what it can prove is genuine.
  • Starting a new product inside a large codebase? I’ve built one from its first pull request and set the API conventions it grew on.

On the record