A little while back, the web team at SoundCloud got an urgent report that our upload page looked weird in the US. Web engineering is based in Berlin, and it looked fine outside the US. However, we quickly identified that a change we’d made recently was conflicting with some custom visuals that were only being run for US visitors.
Rolling back the change was the easy part, but we needed to emulate the API response in Berlin in order to replicate and fix the issue properly. We develop against a single HTTPS test instance of our BFF, and it’s not trivial to add extra mocking directly into that BFF code (it’s the same code that gets deployed to production). We ended up adding a breakpoint in the browser dev tools where the response was received, and we were able to manually inject the US version of the response. It wasn’t pretty and we weren’t proud, but we were able to replicate and fix the issue.
Although we’d resolved the issue, I still thought: “There has to be an easier way. After all, we just need to override one endpoint and proxy the rest to the real server, so how hard can it be?”
I looked into a few existing solutions, but none were as all-in-one as I’d have liked. I wanted to be able to start a command locally and override specific endpoints and have the HTTPS certificates generated and trusted automatically. I also wanted overriding and editing responses to be possible in languages frontend engineers are familiar with, and I wanted that this configuration could be checked in to the repository to be shared with other engineers.
And so intervene was born.
intervene is a node.js process that works like a man-in-the-middle proxy (except it’s not an HTTP proxy, as it only proxies to a single host and appears to be a single HTTP server). It enables the altering of requests and responses for mocking purposes. The configuration is written in TypeScript, so that frontend engineers are using a familiar language (as TypeScript is a superset of JavaScript, using plain JavaScript is fine too).
It automatically generates and trusts self-signed certificates for HTTPS, so in every scenario, only a single command is necessary to create and start a proxy, with no extra steps to perform manually.
At SoundCloud, we used it to build the frontend for a major creator feature before the backend was ready. We added full typing support to intervene in order to use it to discuss the APIs as they were being built. Our mock became the documentation around which discussions were had on what the API should look like. We were able to slowly turn the mocking off as the backend endpoints started to work, and we were even able to patch up responses when the backend implementation wasn’t finished and was only partially working. It’s now become a key piece of our development tooling, with many mocks checked in to frontend repos so that engineers can quickly turn on mocking for common scenarios.
We’re really excited to be able to open source this tool today. It’s been invaluable not only in development, but also in recreating edge cases and a whole bunch of other scenarios where we just need to alter a header, request, or response to quickly try something out.
To install globally (so you can run intervene from anywhere):
npm install -g intervene
You can also install it as a devDependency of your project:
npm install --save-dev intervene
A configuration can be automatically generated using intervene create https://api.mycompany.com (or whatever host you want to create a configuration for). In this example, it generates a file called api.mycompany.com.ts.
So what does a minimal configuration look like?
import { ProxyConfig } from 'intervene';
const config: ProxyConfig = {
target: 'https://api.mycompany.com'
};
export default config;Let’s add a static JSON response to an endpoint:
import { ProxyConfig } from 'intervene';
const config: ProxyConfig = {
target: 'https://api.mycompany.com',
routes: {
'GET /cats': {
collection: [{ name: 'Charlie' }, { name: 'Whiskers' }]
}
}
};
export default config;Saving the file automatically reloads the configuration, so there’s no need to restart after saving. GET in the route name is the default, so it can be excluded for the sake of brevity.
Now if we make a GET request to https://api.mycompany.com/cats, we’ll get some static JSON back:
$ curl -k https://api.mycompany.com/cats
{"collection":[{"name":"Charlie"},{"name":"Whiskers"}]}The -k is needed in the curl request, because otherwise curl will complain that the certificate is self-signed and not trusted. However, if the URL is opened in Chrome, it will Just Work(™), because intervene has already added the certificate to the operating system’s list of trusted certificates (this list isn’t used by curl).
Note that it doesn’t matter if there was an existing /cats endpoint — either way, the JSON specified in the configuration is what is returned and the real server is not called. CORS is automatically enabled for all endpoints from all origins. This can be configured if more specific CORS rules are required.
So intervene can easily return static JSON, but what about if we want to modify the response from the server? Let’s imagine there’s a GET /dogs endpoint that returns the names of some dogs in the same format as the /cats endpoint we mocked earlier:
const config: ProxyConfig = {
target: 'https://api.mycompany.com',
routes: {
'/dogs': async (req, h, proxy) => {
// Get the response from the real server.
const response = await proxy();
// Mutate the response (add an `age` property to each dog).
response.body.collection.forEach(dog => {
dog.age = Math.ceil(Math.random() * 18);
});
// Return the response.
return response;
}
}
};If a function is used as a route handler, it gets called with the request details, an instance of the hapi.js response toolkit, and a function that returns a promise of the response from the server.
Changing the req object before calling the proxy() function will change the request that gets made to the server. For instance, changing req.url.pathname to /cats before calling the function will mean that the proxy() function will actually call the /cats endpoint on the real server when a request is made to /dogs.
Using the hapi.js response toolkit, it’s possible to create custom responses. We have used this extensively at SoundCloud to develop error handling routines by mocking an endpoint to return an error.
For example:
const config: ProxyConfig = {
target: 'https://api.mycompany.com',
routes: {
'/whoops': (req, h, proxy) => {
// Create a response.
const response = h.response({ error: “whoops” });
// Set the status code.
response.code(500);
// Add a custom header.
response.header(‘x-error-id’, ‘123456’);
// Return the response.
return response;
}
}
};intervene initially writes an entry to /etc/hosts, adding the address of the server to mock as 127.0.0.1 (localhost). This means requests to that server now get routed to 127.0.0.1. If the server is HTTPS, it generates a self-signed certificate and calls operating system commands to add it to the set of trusted certificates. It then starts an HTTP(s) server (based on the fantastic hapi framework). Routes from the configuration file are added, and then default routes are added to proxy all remaining requests to the real server.
In order to proxy requests, when a request comes in, intervene performs a real DNS lookup to ensure it ignores the entry in /etc/hosts. These lookups are cached and use the TTLs returned in the DNS response.
There’s a lot more documentation on the intervene.dev site. Additionally, feel free to raise an issue or send a PR to the GitHub project if you have questions or suggestions.