SoundCloud is a platform consisting of hundreds of millions of tracks, people, albums, and playlists. Navigating this vast collection of music and personalities poses a large challenge, particularly when it comes to music with so many covers, remixes and original works all in one place. For search on SoundCloud, one of the ways we approach this problem is by using the PageRank algorithm, with our version affectionately known as DiscoRank (Get it? Disco as in discovery and Saturday Night Fever?!). The job of PageRank is to help rank search results from queries such as, finding all Go+ tracks called "royals". At first glance, this task might seem trivial. The first result should indeed be Lorde's original Royals, but of all the covers and remixes, which should we show at the top and in which order? What about other tracks in our catalog that have the word "royals" in them? Where should they be in our search results list?
Backstage Blog RSS
You're browsing posts of the category Search
January 24th, 2018 Search Data Science Machine Learning Analytics Data PageRank in Spark By Josh Devins