Backstage Blog

• Engineering
• Finagle
• Twirp
• Scala
• Open Source
• Announcements

This article announces the release of Twinagle, an open source implementation of the Twirp protocol for Scala/Finagle.

• Microservices
• Finagle
• Scala
• Functional Programming

If you’re a regular visitor to this blog, you might be aware that we have been transitioning to a microservices based architecture over the past four to five years, as we have shared insights into the process and the related challenges on multiple occasions. To recap, adopting a microservices architecture has allowed us to regain team autonomy by breaking up our monolithic backend into dozens of decoupled services, each encapsulating a well defined portion of our product domain. Every service is built and deployed individually, communicating with other services over the network via light-weight data interchange formats such as JSON or Thrift. What we haven’t touched on so far is how a microservice at SoundCloud looks backstage.

• Microservices
• Architecture
• Finagle

Building and operating services distributed across a network is hard. Failures are inevitable. The way forward is having resiliency as a key part of design decisions.

This post talks about two key aspects of resiliency when doing RPC at scale - the circuit breaker pattern, and its power combined with client-side load balancing.

• Microservices
• Finagle
• Architecture

Since we started breaking our monolith and introduced a microservices architecture we rely a lot on synchronous request-response style communication. In this blog post we’ll go over our current status and some of the lessons we learned.

• Scala
• Finagle
• Ruby
• Architecture

In the first two parts of this series, we talked about how SoundCloud started breaking away from a monolithic Ruby on Rails application into a microservices architecture. In this part we will talk a bit more about the platforms and languages in which we tend to write these microservices.

At the same time that we started the process of building systems outside the Mothership (our Rails monolith) we started breaking our large team of engineers into smaller teams that focused on one specific area…

• Scala
• Finagle
• Ruby
• Architecture

In the previous post, we talked about how we enabled our teams to build microservices in Scala, Clojure, and JRuby without coupling them with our legacy monolithic Rails system. After the architecture changes were made, our teams were free to build their new features and enhancements in a much more flexible environment. An important question remained, though: how do we extract the features from the monolithic Rails application called Mothership?

Splitting a legacy application is never easy, but…

• Scala
• Finagle
• Ruby
• Architecture

Most of SoundCloud’s products are written in Scala, Clojure, or JRuby. This wasn’t always the case. Like other start-ups, SoundCloud was created as a single, monolithic Ruby on Rails application running on the MRI, Ruby’s official interpreter, and backed by memcached and MySQL.

We affectionately call this system Mothership. Its architecture was a good solution for a new product used by several hundreds of thousands of artists to share their work, collaborate on tracks, and be discovered by the…