Building the new SoundCloud iOS application — Part I: The reactive paradigm

Recently, SoundCloud launched the new iOS application which was a complete rewrite of the existing iOS application. The Mobile engineering team saw this as an opportunity to build a solid foundation for the future of SoundCloud on iOS and to experiment with new technologies and processes at the same time.

In the world of mobile, you deal with data, errors, threads and concurrency a lot. The common scenario starts with a user tapping on the screen. The application jumps off of the main UI thread, does some I/O related work, some database-related operations along with some transformations to the data. The application then jumps back to the UI thread to render some new information onto the screen.

Both the Android and iOS platforms provide some tools to deal with the different aspects of this scenario yet they are far from ideal. Some of them do not provide much in terms of error handling, which forces you to write boiler-plate code while some of them force you to deal with low-level concurrency primitives. You might have to add some additional libraries to your project so that you do not have to write filtering and sorting predicate code.

We knew early on we wanted to avoid these issues and thus came into the picture the functional reactive paradigm and Reactive Cocoa.

In short, Reactive Cocoa allows you to create composable, event-driven (finite or infinite) streams of data while making use of functional composition to perform transformations on those streams. Erik Meijer is best known in this space with his Reactive Extensions on the .NET platform which also spawned the JVM based implementation RxJava. By adopting this paradigm, we now have a uniform way of dealing with data models and operators that can apply transformations on those data models while taking care of low level concurrency primitives so that one does not have to be concerned about threads or the difficult task of concurrent programming.

Let’s take an example

Like most mobile applications, the SoundCloud iOS application is a typical case of an API client with local storage. It fetches JSON data from the API via HTTP and parses it into API model objects. The persistent store technology we use is Core Data. We decided early on that we wanted to isolate the API from our storage representation so there is a final mapping step involved where we convert API models to Core Data models.

We break this down into smaller units of work: we have

1. Execute a network request. Parse the JSON response.
2. Transform JSON objects into API model objects.
3. Transform API model objects into Core Data models.

1. Executing the network request

For simplicity, assume that the network-access layer implements the following method:

- (RACSignal *)executeRequest:(NSURL *)url;

We do not pass in any delegates or callback blocks, we just give it an NSURL and get back a RACSignal representing a possible asynchronous operation, or future. To obtain the data from that operation, we can subscribe to the signal using subscribeNext:error:completed:

- (RACDisposable *)subscribeNext:(void (^) (id result))nextBlock
                           error:(void (^) (NSError *error))errorBlock
                       completed:(void (^) (void))completedBlock

You might recognize the familiar-looking error and success-callback blocks from other asynchronous APIs. This is where some of Reactive Cocoa’s and FRP’s strengths lie as we shall see later.

2. Parsing the JSON response

After the network request has been made, the JSON response needs to be parsed into an API model representation. For this we use a thin layer around Github’s Mantle library, which wraps parsing in a RACSignal and pushes the result (or error) to the subscriber:

- (RACSignal *)parseResponse:(id)data
{
  return [RACSignal createSignal:^(id<RACSubscriber> subscriber) {
    NSError *error = nil;
    id apiModel = [MTLJSONAdapter modelOfClass:ApiTrack.class
                            fromJSONDictionary:data
                                         error:&error];
    if (error) {
      [subscriber sendError:error];
    } else {
      [subscriber sendNext:apiModel];
      [subscriber sendCompleted];
    }
  }
}

To achieve the composition of operations that we have mentioned earlier, we wrapped the functionality of existing libraries with signals, where appropriate.

3. Persisting the API model with Core Data

In our architecture, the database represents the single source of truth. Therefore to show tracks to the user we first need to store them as Core Data objects. We have a collection of adapter classes that are responsible for mapping API model objects to Core Data model objects. An ApiTrackAdapter might look as follows:

- (RACSignal *)adaptObject:(ApiTrack *)apiTrack
{
  return [[self findOrBuildTrackWithUrn:apiTrack.urn]
                           map:^(CoreDataTrack *coreDataTrack) {
      coreDataTrack.title = apiTrack.title;
      // set other properties
      return coreDataTrack;
  }];
}

Putting it all together

We now have the building blocks to issue a network request, parse the JSON, and store it as a Core Data object. RAC makes it very easy to compose the individual methods functionally by feeding the output of each operation as an input to the next one. The following example uses flattenMap:

-(RACSignal *)loadAndStoreTrack:(NSURL *)url
{
  return [[requestHandler executeRequest:url] flattenMap:^(id json) {
    return [[parser parseResponse:json] flattenMap:^(ApiTrack *track) {
      return [adapter adaptObject:track];
    }];
  }];
}

The flattenMap: method maps or transforms values emitted by a signal and produces a new signal as a result. In this example, the newly created signal returned by loadAndStoreTrack: would either return the adapted Core Data track object or error if any of the operations failed. In addition to flattenMap:, there is a whole range of predefined functional operators like filter:or reduce: that can be applied to signals.

RAC Schedulers

We left out one powerful feature of RAC which is the ability to parametrize concurrency. To ensure that the application stays responsive, we want to perform the network I/O and model parsing in a background queue.

Core Data operations are different, we do not have a choice there. They have to be executed on a predefined private queue, otherwise we risk creating deadlocks in our application.

With the help of RACScheduler we can easily control where the side-effects of a signal are performed by simply calling subscribeOn: on it with a custom scheduler implementation:

-(RACSignal *)loadAndStoreTrack:(NSURL *)url
{
  return [[requestHandler executeRequest:url] flattenMap:^(id json) {
    return [[parser parseResponse:json] flattenMap:^(ApiTrack *track) {
      return [[adapter adaptObject:track]
                       subscribeOn:CoreDataScheduler.instance];
    }];
  }];
}

Here, we use a scheduler that is aware of the current Core Data context to ensure that adaptObject: is executed on the right queue by wrapping everything internally with performBlock:.

If we want to update our UI with the title of the track we just fetched, we could do something like the following:

[[trackService loadAndStoreTrack:trackUrl] subscribeNext:^(Track *track) {
   self.trackView.text = track.title;
} error:^(NSError *error) {
  // handle errors
}];

To ensure that this final update happens on the UI thread we can tell RAC to deliver us the information back on the main thread by using the deliverOn: method:

[[[trackService loadAndStoreTrack:trackUrl]
      deliverOn:RACScheduler.mainThreadScheduler]
  subscribeNext:^(Track *track) {
   self.trackView.text = track.title;
} error:^(NSError *error) {
  // handle errors
}];

By breaking down each operation within this common scenario into isolated units of work, it becomes easier to perform the operations we need on the desired threads by taking advantage of Reactive Cocoa’s scheduling abilities. The functional reactive paradigm has also helped us to compose these independent operations one after another by using operators such as flattenMap. Although adopting FRP and ReactiveCocoa has had its difficulties, we have learned many lessons along the way.

Steep learning curve

Adopting FRP requires a change of perspective, especially for developers who are not used to a functional programming style. Methods do not return values directly, they return intermediate objects (signals) that take callbacks. This can lead to more verbose code, especially when the code is heavily nested which is common when doing more complex things with RAC.

Therefore, it is important to have short and well-named methods, for example a method signature like -(RACSignal *)signal does not communicate anything about the type of values the caller is going to receive.

Another problem is the sheer number of methods or operators defined on a base classes like RACStream / RACSignal. In practice only a few (like flattenMap: or filter:) are used on a regular basis, but the remaining 80% tend to confuse developers who are new to the framework.

Memory management

Memory management can be problematic because of RAC’s heavy use of blocks which can easily lead to retain cycles. They can be avoided by breaking the cycle with weak references to self (@weakify / @strongify macros).

One of RAC’s promises is to reduce the amount of state you need to keep around in your code. This is true but you still need to manage the state introduced by the framework itself, which comes in the form of RACDisposable, an object returned as a result of signal subscription. A common pattern we introduced is to bind the lifetime of the subscription to the lifetime of the object with asScopedDisposable:

self.disposable = [[signal subscribeNext:^{ /* stuff */ }] asScopedDisposable];

Overdoing it

It is easy to fall into the trap of trying to apply FRP to every single problem one encounters (also known as the golden hammer syndrome), thereby unnecessarily complicating the code. Defining clear boundaries and rules between the reactive and non-reactive parts of the code base is important to minimize verbosity and to use the power of FRP And Reactive Cocoa where appropriate.

Performance

There are inherent performance problems within RAC. For example, a simple imperative for loop is guaranteed to execute much faster than flattenMap: which introduces a lot of internal method dispatching, object allocation, and state handling.

In most cases this overhead is not noticeable, especially when I/O latency is involved, as in the preceding examples.

However in situations where performance really matters, such as fast UI rendering, it makes sense to avoid RAC completely.

Debugging

We found this to be a non-issue if your application components are well designed and individually tested. Backtraces tend to get longer but this can be alleviated with some extra tooling like custom LLDB filters. A healthy amount of debug logging across critical components also does not hurt.

Testing

Testing a method that returns a RACSignal is more complicated than testing code that returns plain value objects, but it can be made less painful with a testing library that supports custom matchers. We have created a collection of matchers for expecta that lets us write concise tests. For example:

RACSignal *signal = [subject executeRequest:url];
expect(signal).to.sendSingle(@{ @"track": @{ @"title": @"foo" } });

We found that adopting FRP tends to produce easily testable components because they are generally designed to perform one single task, which is to produce an output given a specific input.

It took a while for the team to get up to speed with FRP and Reactive Cocoa and to learn for which parts of the application it can be used most effectively. Right now it has become an indispensable part of our mobile development efforts, both on Android and iOS. The functional reactive approach has made it easier to build complex functionality out of smaller pieces whilst simplifying concurrency and error handling.