We felt this pain as well and experimented with different solutions over the past months on the SoundCloud Mobile site.
The first approach we had was to track errors with Google Analytics. Their library permits to fire custom events and whenever an ajax error would occur, we would log it.
The biggest benefit of this tool is to monitor the stability of the site and its evolution in longer periods as you can easily go back a few weeks or months to see which events were triggered. Also, it is easy to implement – almost a one-liner!
The drawback, at least for Google Analytics, is that this tool is not meant to track bugs. There is no way to add custom data to these events to get more insight about why and how an error happened, it also doesn’t work in real-time, and you obviously want that when you debug.
So we kept Analytics in place for a long-term view, but took a look at other options for real-time and in-depth tracking.
In our pursuit of getting more insight, we decided to take a look at Airbrake because we were already using it to track back-end errors on our main site.
Our mobile site runs on Node.js, the first thing we did was to integrate an existing plugin for it to handle error tracking on the back-end as well.
Looking a little further we found a front-end notifier, which would catch errors that would fire on window.onerror, but there was no way to report any custom errors.
We decided to take a day to hack this on our own since their API is public and easy to implement.
The benefits of Airbrake were instant. We could see what triggered which error, how, why, in which context, which browser, etc… in real-time!
It also counts errors, which can help you prioritize and include fixes in your roadmap.
However, the lack of filtering, grouping and custom sorting made it difficult to work with. There was also no sense of time or progress, as everything just gets dumped into a single list ordered by time. We needed something a little better than that.
That’s when our Android team showed us their BugSense implementation. BugSense seemed to address all of these issues we had with Airbrake: grouping is more effective, searching and filtering is possible, charts of errors are drawn as well.
There is one more benefit over Airbrake… JSON. No need to convert objects to XML strings anymore!
If you are interested in our BugSense notifier you can find the source on github.
There is still a lot of work needed to make front-end JS debugging as easy as it is for regular back-end environments. For example, stack traces today aren’t that useful, because of anonymous objects and minified code, but hopefully browser vendors will tackle these issues soon. Maybe Source Maps could be the first milestone in this quest.
At SoundCloud, we will continue to use a combination of these tools because of the different strengths outlined above, but there are also other tools we didn’t try out yet like getexceptional or errorception. If you have tried these, or if you have any suggestion on this subject we’d like to get your feedback in the comments below.