Ben Ratliff, a music critic on the New York Times, wrote in The Guardian about Spotify’s Discover Weekly recommendation engine ("Slave to the algorithm: how music fans can reclaim their playlists from Spotify" 21 Feb 2016). This service provides an automatically generated weekly playlist for subscribers to Spotify’s music streaming service, based on their profile and behaviour. His comment sums it up perfectly: “Often I hate the results, even if I like half the songs.”
This seems to reveal a common attitude toward recommendation engines – we don’t like them, even if we grudgingly admit that they are useful.
In more detail, Ratliff reveals this double attitude:
It moved me deeply a couple of times over the past few weeks with songs I hadn’t known before … but listening is creative and personal, and with Discover Weekly I am aware of being profiled by forces I do not know and cannot see… I feel intensely frustrated by what it has reduced me to. I want not to be pandered to by genre, or era, or a reduced, sellable version of a mood.
The sophistication of Amazon’s recommendation is well-known. My own experience, and my exhaustive survey of one other user, suggests that Amazon’s recommendations are not only looked at but, yes, in addition, they have provided suggestions that have been purchased. There only needs to be one or two good examples of a recommendation system working for the entire system to be a good idea. After all, there is no obligation to use a recommendation engine.
After criticising Spotify, Mr Ratliff then continues to provide his own recommendations, 10 tracks, to be precise. I was interested in following up two of them; and I wouldn’t be surprised if an automated recommendation engine produced a similar success rate, of around 20% of the recommendations being of real interest to the user.
The major downsides to the Amazon system are related to the ecommerce aspect of the site, rather than a simpler “if you looked at X, you might want to look at Y”. If you buy books for your grandmother using your own Amazon account, then the Amazon recommendation keeps suggesting more for her rather than more for you. A further problem appears to be the “I don’t want two of those” problem: if I browse for a digital camera, I’m happy for Amazon to suggest other models I might be interested in. But once I have bought a camera, I’m unlikely to buy another one. Amazon, being Amazon, have already grasped this, and now provide “inspired by your shopping trends” as well as a separate list of “recommendations for you in Books”. Even so, it is still trying to interest me in a smartphone holder for my car when I have just bought one.
The truth is that it is very easy to implement a recommendation engine. Elasticsearch even includes a “more like this” query that compares an input string to other strings in the index; but it even allows you, via “like” or “unlike” parameters, to find other related terms, thus allowing you to carry out a reasonable degree of disambiguation (for example, “bridge” unlike “river sea harbour” should exclude the structure from the card game). With tools as good as this, I’d be itching to build a recommendation engine straight away.