How Spotify Creates Your Playlist

The growing popularity of audio streaming services around the world has led to a major disruption in the music industry over the last decade. Spotify, which launched in India last year, is without doubt the dominant player with an estimated 36% market share and over 305 million subscriptions globally. Apple Music is the next with an 18% share. Spotify has already gained a 15% share of the Indian audio streaming market, which is currently dominated by Gaana at 30%, followed by JioSaavn with 24% and Wynk with 15%.

Spotify and other streaming services are really the progeny of the pioneering peer-to-peer file sharing service called Napster, which was set up in 1999 by Sean Parker and Shawn Fanning, two young students who were barely 20.  Its technology allowed people to easily share their MP3 files with other users. At its peak, Napster had 80 Million registered users. The website transformed music into a public good for a brief period of time, making it relatively easy for music enthusiasts to download copies of songs that were otherwise difficult to obtain. Napster was shut by court order in 2001 after the likes Metallica and Dr Dre accused it of promoting music piracy. 

While Napster lasted only two years, its spirit has lived on and flourished around the world in the form of all legal streaming services that followed. Spotify, the world leader, was founded in Sweden in 2006. Napster founder Sean Parker was in fact one of the early investors and helped it grow in the US. Spotify operates a `freemium’ model, where the base layer is free to use and comes with advertisements, the premium subscription provides an ad-free experience.

How does Spotify use algorithms?

40,000 new songs are added to the Spotify platform each day. Spotify has around 3,000 official playlists, curated by its employees and tastemakers. But there are millions more that are generated by its subscribers. The sheer volume of music uploaded means that a Spotify employee cannot curate every song on the platform. So, a large part of the process is automated through algorithms. The main component of the Spotify algorithm is thus provided by its users. 

Spotify begins by looking at the millions of playlists created by its users. If two of your favorite songs tend to appear on a playlist, Spotify will suggest the new song to you on individually recommended playlists such as Discover Weekly. The algorithm places more weight on the companies’ own playlists and playlists with more followers. In a nutshell, it attempts to fill in the blanks between your listening habits and those with similar tastes. 

However, the data Spotify collects on you to create your individual playlists is a lot more than just that. Spotify uses deep learning, with computers that analyze songs and learns to recognize different aspects of the music that might be desirable to you. These algorithms identify aspects of songs as concrete as distorted guitars, while also recognizing more abstract ideas like genres. The company creates a profile of each user’s taste in music, through clusters of artists and micro-genres as specific as “Synthpop”, “Dreampop”, “Vocal pop”. 

These are derived using technology from Echo Nest, a music analytics firm that Spotify acquired in 2014. Brian Whitman, who co-founded The Echo Nest, wrote in 2012 that his service scoured more than 10 million music-related web pages a day in order to understand what was happening in the world of music. “Every word anyone utters on the internet about music goes through our systems that look for descriptive terms, noun phrases, and other text,”  

The home screen of the Spotify app is another example of how algorithms govern your listening experience. Its goal is to quickly help users find something they are going to enjoy listening to. Spotify has spoken about how the home screen is governed by an A.I. system called BaRT (Bandits for Recommendations as Treatments). The system’s task is to organize each home screen in a personalized way for each user. That includes the “shelves,” or rows of playlists, that follow a theme like “best of artists” or “keep the vibe going,” and the order in which playlists appear on those shelves.

The purpose of the BaRT is to generate music that Spotify is confident you’ll like, based on your previous listening activity. But Spotify also has to get you interested in newer, fresh music so you don’t get stuck in a loop of listening to the same thing all the time. The system can be boiled down to two concepts: `Exploit’ and `Explore’. In Exploit, Spotify uses the information it knows about you. It considers your music listening history, which songs you’ve skipped, what playlists you’ve made, your activity on the platform’s social features, and even your location. Through Explore, it uses the information about the rest of the world, like playlists and artists similar to your taste in music but those that you haven’t heard yet. 

Another use of algorithms is through automatic playlist continuation. At the end of a playlist, this feature analyzes the songs and tries to predict the music that should come next — as if the person who created has added similar music that caters to their taste. Spotify wanted new ways to think about how it should be building that feature, so a few years ago it released a “Million Playlist Dataset” of user-generated Spotify playlists that could be used to understand the traits of what humans considered a good set of tracks. The company invited other A.I. researchers to try and help solve the problem. The researchers presented their solutions to Spotify at a conference in 2018.

Spotify has found that explanations of why a song has been selected by its algorithm is critical for getting the users to listen to that song. Each label, like “Jump back in” or “More of what you like”, tells the user why specific playlists have been recommended. 

On top of all the research going into the music, the company is also researching its users. Between 2016 and 2018, Spotify studied data from more than 16 million users, tracking their listening patterns including how many times someone streamed a specific artist or song per day and where in the US they lived. That data, coupled with users’ self-reported gender and age, allowed Spotify to study whether music taste changes after someone has moved to a different state (using location data), as well as how age impacts the kind of music a person listens to. 

By studying the musical tastes of people in each state, the Spotify team concluded that over a long period of time, location does factor into musical taste in some small way. By studying age, they also found that the music that is popular from ages 10 to 20 is the music that people will predominantly listen to in the future, having shaped their “musical identity.” This vast amount of data that Spotify continues to collect on its users for its service helps to maintain its competitive edge. 

How can you help the algorithm find you the perfect song?

Spotify makes decisions about what users are likely to want to listen to by filtering out some genres. This ensures that parents with young kids won’t get inundated with songs from The Wiggles, and if you listen to ocean sounds while sleeping, you won’t be swamped with “Pacific Ocean Waves Vol. 15”. However, as good as Spotify’s picks are, they aren’t perfect. Here are a few tips you can use to exploit the algorithm and find new music to listen to.

1. Add songs you like to your library or playlist.
2. Skip the songs you don’t like – if a user skips a song within 30 seconds, the algorithm sees that as a “thumbs down”.
3. Look at profiles for new artists – if you like something Spotify recommends to you and you click through the artist to discover their discography, the algorithm notes this.
4. Use private mode if you don’t want Spotify to notice – say if you have a younger sibling who is into Death Metal.