I for one Welcome Our Algorithmic Overlords

I for one Welcome Our Algorithmic Overlords


ot too long ago, before image macros and reaction videos roamed the planet, human beings created mix-tapes to express complicated feelings or share their favorite songs with family and friends. These custom compilations of songs were recorded on primitive storage media called “cassettes”, small plastic shells filled with two spools of magnetic thread constantly on the verge of getting entangled and jamming the playback device, in which case the cassette had to be carefully extracted from the mechanism, lengths of painstakingly prepared music spilling onto the floor like ugly brown tagliatelle, a scenario which could only be rectified by use of a hexagonally shaped pencil.

Nowadays our music lives in the cloud, gazillions of songs stored in ever-expanding rows within server farms across the world, managed and maintained by bots and over-caffeinated system administrators. We have access to almost every song ever written in human history and the only barrier between us and the totality of our sonic heritage is in our minds, i.e. we can only type things into the search field we’re already aware of.

This is why at the center of each streaming service like Spotify or Google Music there is a “recommendation engine”, a secret soup of algorithms constantly analyzing our listening habits and suggesting related artists, albums and playlists.

Before all of this, we used to discover music via the radio, television and most importantly, through our friends with whom we exchanged battered tapes and (more recently) compact discs hastily scrawled with black markers.

But let’s face it. Even the biggest music nerd’s record collection will pale in the face of what online streaming services have to offer. If you got a subscription and only listen to the same couple of albums over and over again, that’s fine, but you’re only accessing a minuscule slice of what you’re paying for. It’s like you’re renting a sprawling villa but never leave the tool shed. In a way, the catalog of a music streaming service is only as good as their discovery engine.

Besides, all major streaming services offer more or less the same catalog, geo-licensing shenanigans aside, so the only real difference is how good they are at enabling you to discover more great music. Pandora lets you create custom radio stations based on your taste profile, Google Play Music has a “I’m feeling lucky” button charged with the same raw computing power behind Google Search, but I often found both of these services to be a game of hit-and-miss.

One of the most interesting discovery mechanisms I’ve found so far is Spotify’s “Discover Weekly”, a custom playlist of 30 songs, generated fresh each Monday, like a personalized mix-tape. From what I could gather online, these playlists are generated by comparing your taste-profile to those of 75 million other users, searching for profiles similar to yours and then suggesting songs from these profiles which are not (yet) part of yours.

via qz.com

graphic by qz.com

The accuracy of “Discover Weekly” playlists is uncanny sometimes, up to the point where it feels like somebody who really knows your taste in music sat down and put together 30 songs you’ve never heard before.

There’s a strong sense of déja vu at times, where I feel that I’ve heard these songs before, almost as if Spotify’s algorithms knew what future-me would be listening to next week by tapping into the collective unconscious of its user base.

If these playlists weren’t so damn good, it would almost be scary, because there’s a strong sense of human curation here. Sure, there are some outliers, where sometimes a track succession feels random or there’s the odd song which just doesn’t fit, but when it get its right, boy does it get it right!

Obviously, the more you’re using Spotify, adding songs, albums, following artists, digging down deeper into the catalog, the better these playlists are supposed to get. I’ve read that when you skip a song under the 30 second mark, Spotify will register that as a thumbs-down. Also, if a family member or significant other uses your account to go on a romp of listening to stuff that’s completely different than your usual tastes, Spotify is apparently smart enough to detect that and not let it influence these weekly playlists.

Another thing I’ve noticed is that while I don’t necessary find all of the 30 songs equally great, Spotify’s algorithms sometimes seem to have a great sense of pacing where the sequence really fits, i.e. the mood of one track gets picked up by the next, evolves into something else, gets contrasted and then comes full circle.

What really goes on under the hood of “Discover Weekly”, we’ll probably never find out, but more and more often I find myself looking forward to Mondays with a strange sense of anticipation, wondering what the bots will have cooked up next.

What does it mean when algorithms are getting just as good or even better at music discovery than our networks of friends? Are we one step further towards Skynet becoming self-aware? Or are we perhaps more predictable in our preferences than we’d like to admit? Being able to automatically suggest things to people that they didn’t even know they were looking for is the Holy Grail of (online) marketing. For quite a while these mechanisms have been very primitive, but they’re slowly getting better. Eventually, Facebook might be able to predict who you’ll friend next, Amazon will automatically fill your shopping cart with highly personal Christmas gifts, and Google will show us the right search results without having to enter a single word.

It’s a brave new world in the making, questions of privacy, surveillance and morality will become more and more important. Sure, there will be pitfalls and failures along the way, but while all of this is unfolding, at least we’ll have the perfect soundtrack.

Cassette images via magnetofon.de, handwriting, automatically generated by neural network by cs.toronto.edu

P.S.: I’ve started saving some of my favorite “Discover Weekly” mixes as playlists, because the original mix gets overwritten every week. Here are two examples: