TL;DR
Twirp is a awesome new go RPC framework written by the folks at Twitch. Its powers, combined with an AWS serverless architecture will allow you to write performant web APIs very quickly, with high quality in a manor that can be used in production by a team of developers right away. Check out my boilerplate/example project Twirpl to get started.
Why I love Twirp
Make sure to check out the Twirp release blog post, however in short here is why I LOVE it:
- Simple and convention based: Everything is an HTTP POST. 0 cycles are spent determining if a new endpoint should be PUT/PATCH/POST/GET/DELETE. Endpoint URLs are convention based and clear:
/twirp/com.rynop.twirpl.blog.Blog/Subscribe
- Flexible: works on HTTP 1.1 and supports both JSON and Protobuf
- Generates server AND client code: Just supply a standard protobuf document describing your API. Currently it will only generate a golang client, but hopefully more languages will be supported soon.
- Performant: I like go because the development spend to performance ratio is great.
- Self contained routing: Because go is fast, I don’t mind that the framework handles request routing. Self contained routing is big IMO, more on this below.
Making Twirp serverless
Twirp is used by Twitch for communication between backend servers (aka service-to-service).
I wanted to open this up for communication from client (ex: JS webapp) AND make the backend completely serverless using AWS Lambda and API Gateway (APIG).
Announcing Twirpl
I have created an example/boilerplate for getting all this running. Check it out at https://github.com/rynop/twirpl
This project shows you how to create a Twrip API works with both JSON and binary Protobuf payloads. It uses a simple APIG lambda proxy ({proxy+}) to send all requests to a single go binary (running natively in Lambda) that handles routing.
Preparing for future scale
I’m a big fan of a single binary with self contained routing. One of the many benefits: when Lambda gets too expensive it is easy to migrate. Lambda at scale is expensive. Understandably so – you are paying a premium for not having to manage server resources.
ECS is a cost savings/lower latency step but there are still containers and orchestration to manage. With the recent announcement of AWS Fargate much of this goes away (note: I have not yet used Fargate).
If you put CloudFront in front of APIG, once you outgrow Lamba, you can simply point it to a Fargate cluster running the same Twirp codebase – all you need to do is replace the gateway.ListenAndServe() with http.ListenAndServe().
What’s next
Remove API Gateway
I’m not a huge fan of APIG because it adds cost, latency, complexity, an additional service that could go down, and another attack vector. I would like to completely remove it from the equation and have the twrip based go lambda run at the edge. Bummer is Lambda@Edge only supports Node.js 6.10 at the moment 😦
Create a Javascript client generator
Leveraging something like decodeIO/ProtoBuf.js. Adding Typescript support would be a big bonus (for example, to use in Angular apps).
Get involved
Join the #twirp channel on gophers slack and start submitting those PRs!
Discuss Twirpl on Hacker News
Ryan Pendergast 