Backstage Blog

RSS logo

You're browsing posts of the category Go

Learning Scala at SoundCloud

April 19th, 2022 by Bhuvaneshwari Chinnadurai

This is the story of how I joined SoundCloud and learned Scala as a Golang developer with no prior knowledge of Scala.

Read more…

What Is New with Periskop in 2022

February 1st, 2022 by Marc Tuduri

We made some updates to Periskop, our internal pull-based exception monitoring service. To learn about what is new in 2022, read on.

Read more…

Periskop: Exception Monitoring Service

March 16th, 2020 by Jorge Creixell

A new approach to exception monitoring, designed for high scalability...

Read more…

Prometheus has come of age – a reflection on the development of an open-source project

July 19th, 2016 by Bjorn "Beorn" Rabenstein

On Monday this week, the Prometheus authors have released version 1.0.0 of the central component of the Prometheus monitoring and alerting system, the Prometheus server. (Other components will follow suit over the next months.) This is a major milestone for the project. Read more about it on the Prometheus blog, and check out the announcement of the CNCF, which has recently accepted Prometheus as a hosted project.

Read more…

Prometheus: Monitoring at SoundCloud

January 26th, 2015 by Julius Volz, Björn Rabenstein

In previous blog posts, we discussed how SoundCloud has been moving towards a microservice architecture. Soon we had hundreds of services, with many thousand instances running and changing at the same time. With our existing monitoring set-up, mostly based on StatsD and Graphite, we ran into a number of serious limitations. What we really needed was a system with the following features:

  • A multi-dimensional data model, so that data can be sliced and diced at will, along dimensions like instance, service, endpoint, and method.
  • Operational simplicity, so that you can spin up a monitoring server where and when you want, even on your local workstation, without setting up a distributed storage backend or reconfiguring the world.
  • Scalable data collection and decentralized architecture, so that you can reliably monitor the many instances of your services, and independent teams can set up independent monitoring servers.
  • Finally, a powerful query language that leverages the data model for meaningful alerting (including easy silencing) and graphing (for dashboards and for ad-hoc exploration).

All of these features existed in various systems. However, we could not identify a system that combined them all until a colleague started an ambitious pet project in 2012 that aimed to do so. Shortly thereafter, we decided to develop it into SoundCloud’s monitoring system: Prometheus was born.

Read more…

Roshi: a CRDT system for timestamped events

May 9th, 2014 by Peter Bourgon

Let’s talk about the stream.

The SoundCloud stream represents stuff that’s relevant to you primarily via your social graph, arranged in time order, newest-first. The atom of that data model, an event, is a simple enough thing.

  • Timestamp
  • User who did the thing
  • Identifier of the thing that was done

For example,

If you followed A-Trak, you’d want to see that repost event in your stream. Easy. The difficult…

Read more…

Go at SoundCloud

July 24th, 2012 by Peter Bourgon

SoundCloud is a polyglot company, and while we’ve always operated with Ruby on Rails at the top of our stack, we’ve got quite a wide variety of languages represented in our backend. I’d like to describe a bit about how—and why—we use Go, an open-source language that recently hit version 1.

It’s in our company DNA that our engineers are generalists, rather than specialists. We hope that everyone will be at least conversant about every part of our infrastructure. Even more, we encourage engineers…

Read more…