Building the SoundCloud mobile site using backbone.js

Until early this year, there was a gap. A gap between the desktop-targeted main SoundCloud site, what we call the ‘mothership’, and the native iOS (iPhone, iPod touch) and Android applications. A common and frustrating use-case was mobile Twitter: Someone would share a new favorite or upload on Twitter, you tap on it, and it tried to load the regular site on your tiny smartphone screen. Pushing the whole desktop site over a mobile connection would be a waste of precious bandwidth, if you only want to check out a track. Alternatively we could try to redirect to our native apps, but there’s no guarantee that the user has it installed and the mobile vendors don’t offer any APIs for verifying that in advance.

With that in mind, back in December 2010, we set off to build SoundCloud Mobile, targeting the mobile browsers of iOS and Android. The analytics of the existing site told us that these two platforms make up the overwhelming majority of our users, so we started there. As a mid-term goal, we decided to expand our support to devices, as long as they have a browser capable of streaming audio.

For the architecture of the site we decided to make it a SoundCloud API client, eating our own dogfood just like the native iOS and Android apps already do. With that in mind, we considered the option of building a single-page web application (vs classic serverside rendered pages). To figure out how viable that option is, we spent a week building a prototype based on jQuery Mobile. The prototype included a start page with hot tracks, a basic search, people and track pages and basic audio streaming. The lists used the theme provided by jQuery Mobile, everything else was barely styled HTML. This prototype helped a lot in making several important decisions:

• Building a single-page app was feasible, with the client side application as the direct API client. Later we had to back away a bit from that, introducing a proxy to decorate the API (and work around WebKit bugs), but overall most of the action is still happening on the client.
• jQuery Mobile works great for a fixed number of preloaded and infinite number of server-generated pages, but not for our usecase of generating all pages on the fly based on API results. We needed much more flexible routing with HTML5 history.pushState support, so that we could support the theme URL sets as the main site.
• On a similar note, jQuery Mobile’s theming system allowed us to build a pretty prototype in no time, but wasn’t a good fit for the completely customized UI that we wanted.
• Audio streaming on mobile is still very immature. Even with support for only iOS and Android, plenty of workarounds are required for a somewhat consistent experience.

After throwing away the first prototype, we moved on to create our own basic framework. It described the domain classes like ‘track’ and ‘user’ as global singleton objects. Our ‘router’ object was responsible of passing on the model data onto the responsible controller method. Soon we could see that the approach wouldn’t scale that well, especially when simultaneous instances of a class were required on the same page.

After dismissing a few bigger client-side MVC frameworks, we’ve stumbled upon Backbone.js, which was compact, easily extendable and depended only on Underscore.js. Backbone sets up only the application structure plus it offers a multitude of convenient methods that can be used while building your app. It doesn’t dictate how the application UX works nor describes how the templates have to be structured. While that still left a lot of open questions for us to answer, it also didn’t impose too much unwanted structure.

Backbone.js let’s you choose your own templating engine, and we went with the jquery-tmpl plugin. We restricted our template usage to output and iteration within the template, both to give us the option of switching to another template engine (e.g. handlebars.js) and to keep our sanity. To implement the remaining presentation logic, we used the route suggested by Backbone.js, preparing the data for output in the Model’s toJSON method. This also has the advantage of keeping the model itself clean, making it easy to update the model and send it back to the server. In addition to that we added a decoration step, modifying the template output before inserting it into the DOM. This includes adding additional classes or removing empty nodes.

When we started using Backbone.js, it supported only hash-based history (what Twitter does today when it redirects twitter.com/ericw to twitter.com/#!/ericw). We wanted support for history.pushState to map URLs from soundcloud.com to m.soundcloud.com by only prepending the ‘m.’. We extended Backbone.history for that, while also triggering a custom event. The latter can be used by the Google Analytics tracker or any other component that has to get an update on the current page state.

We also extended regular Backbone.sync method, used by all Models and Collections to exchange data with the server, to add a client side cache, backed by the HTML5 sessionStorage. That way we didn’t have to keep any pages in memory, but can instead rerender them from scratch in milliseconds, as the underlying data is still available in the cache.

With those components in place, a click (or rather, tap) on any internal link caused the following actions:

• Handling the click/tap event, preventing the default browser action, and using history.pushState instead to update the current address. At some point telling the Backbone.router that the page changed.
• Backbone.router maps the URL to a controller method, which creates the model for that URL, e.g. initializing the User model with the username parsed from the URL. It then creates the view and passes the model to that view.
• The view tells the model to fetch its data. Once done, with data loaded from the server or from the client side cache, it passes the model to a template, decorates the result and inserts it into the DOM.
• The view also initializes event handlers (via event delegation) to handle all interactions within that view, e.g. a click event on the ‘Play’ button to start streaming audio.

This turned out to be a very solid application architecture which we continued to fine-tune after the first public launch of the mobile site in March, when we redirected iOS and Android traffic from the main site. Since then we continued to add features and improve the site, watching the traffic almost doubling every month.

Along with this new client side architecture we also experimented with alternatives for development and production. The node.js-based development and production server, including the API-proxy is covered in detail by our node ninja Alexander Simmerl. In the upcoming post we’ll also talk about our approach to testing with QUnit and PhantomJS.