Discover the powerful design of Apache Kafka and how SoundCloud consumes events from Kafka.
Discover how SoundCloud extended the concept of Value-Added Services to Domain Gateways.
Learn about the evolution of architectural patterns at SoundCloud — from BFFs to Value-Added Services.
Discover how service architecture has evolved at Soundcloud over the past few years...
Two years ago, if an engineering team at SoundCloud needed data produced by another team, in many cases, that team would have had to start by building a dataset from the raw data. That meant there was no standard way to structure data into a dataset, there were errors in understanding and interpreting raw data, and there was no reuse of datasets across teams. Ownership of datasets was with the teams consuming data, even when the data itself was owned and best understood by someone else. By agreeing on an ownership strategy, a common shape for datasets, and standard tooling, SoundCloud was able to streamline these problems away. Now the data and datasets are provided by those who know best about their data, along with the help of tooling built by those who know best about building tools. Read on to learn what datasets look like at SoundCloud and how they are built.
In 2017, our team of six engineers wanted to try out a clean architectural pattern and decided to use VIPER. In the text below, I’ll cover how the team worked on this.
Track play counts are essential for providing a good creator experience on the SoundCloud platform. They not only help creators keep track of their most popular songs, but they also give creators a better understanding of their fanbase and global impact. This post is a continuation of an earlier post that discussed what we do at SoundCloud to ensure creators get their play stats (along with their other stats), both reliably and in real time.
A little less than two years ago, SoundCloud began the journey of replacing our homegrown deployment platform, Bazooka, with Kubernetes. Kubernetes automates deployment, scaling, and management of containerized applications.
Growth in code bases come with exciting scalability challenges. As the size of our iOS codebase and team at SoundCloud grew, we faced challenges: long compile times and conflicts. Our productivity started to suffer as a result. We took inspiration from the work done in the backend (Building Products at SoundCloud) and applied it to mobile development. The main goal was to get back to a state where development is fun, fast, and would scale as the number of contributors grew. We modularized our iOS project by splitting it up into modules with well-defined responsibilities and public interfaces that interconnect them.
Once upon a time, we had a single monolith of software, one mothership running everything. At SoundCloud, the proliferation of microservices came from moving functionality out of the mothership. There are plenty of benefits to splitting up features in this way. We want the same benefits for our data as well, by defining ownership of datasets and ensuring that the right teams own the right datasets.