“We Think You Might Like This”

Our thirst for online shopping and engagement with multi-media streaming sites seems insatiable. The success of the likes of Amazon, Netflix, Spotify, and YouTube, demonstrates this major growth area, and Covid-19 has served to accelerate our consumer behaviour from offline to online. It seems ironic, that whilst our offline interactions have become ever more de-personalised through social distancing requirements, limited personal interactions and the wearing of face masks, it is our online interactions which have become increasingly more personalised. The more we consume and interact online, the more data we create. This data is the currency of the digital world. Recommender systems leverage this data to tailor the content of websites in real time to match individual predicted user preferences and needs.

But, do recommender systems influence our actual behaviour? Do they enrich our lives, or simply reinforce our own behaviour? And, how does this technology benefit organisations, customers and society?

These were just some of the questions raised in Mark Graus’s Recommender session, delivered as part of the Digital Strategy module for the MaastrichtMBA programme in November. Mark is an Assistant Professor on Data Science in Marketing at Maastricht University. His background in Human-Technology Interaction combines machine learning with fundamental psychological theory. It is this combination of expertise, which enabled a dynamic and nuanced debate on the recommender session, and yes, it got personal!

A personalised online experience is the antithesis to a one size fits all customer approach. For a successful personalised user experience, businesses need to adapt what they show people based on what they know about them. When online businesses do this well, it can generate significant income streams and create competitive advantage. All recommender systems are personalised systems and the algorithms rely on one of two main approaches: content-based filtering and collaborative-based filtering.

Content-based filtering infers a user’s preference from product attributes of items they like. So, for example, when you buy a blue shirt on Esprit’s website, you might be asked if you would also like to look at several other similar types of shirt. The interactions people have are used to predict relevance to the consumer in terms of what attributes are interesting to them. This type of filtering is very transparent, easy to understand and straightforward to implement. 

Collaborative-based filtering, on the other hand, infers a user’s preference from users like you. It matches products to what other people, similar to you, have purchased in the past and then recommends certain products. Unlike content-based filtering, it can be used without any metadata, just requiring user item interactions. It is allegedly a more reliable system, but more complicated to implement. Mark suggests that in order for businesses to achieve success with these systems of feedback, they should practise a combination of both content and collaborative-based filtering.

Apart from content-based versus collaborative filtering, a distinction can be made between the type of feedback that is used to personalise. Explicit feedback is feedback collected by users explicitly indicating their preferences. For example, the thumbs up/thumbs down on YouTube or using the heart icon to indicate you love a song on Spotify. The notion is that this type of feedback is not very reliable, mainly because what people say, isn’t necessarily what they do. A psychological human quirk, which can befuddle an algorithm based solely on explicit feedback.

The counterpart to explicit feedback is implicit feedback, or feedback derived from the natural interaction with a system. For example, the products an online shopper looks at, clicks on, adds to the shopping basket and/or eventually purchases. This type of feedback has the advantage that it does not require any interaction on the part of the shopper. At the same time,  the drawback is that it is less transparent. Since people do not provide feedback explicitly they may not be aware of what data is collected. Mark suggests that in order for businesses to achieve success with these systems of feedback, they should practise a combination of both content and collaborative-based filtering, as well as a mix of implicit and explicit feedback.

Evaluating how well the system is working is a crucial step in the personalised digital process. Taking the right action or making adjustments as a result of this, can have significant and immediate impact on the success of the system. Mark outlined three main approaches to assess how well the system is doing. These are online, offline, and user studies. Within each of these systems, there is a varying level of difficulty, reliability and depth of insight. Mark cited key pointers from Ron Kohavi’s practical guide as a useful source when doing controlled experiments on the web. The key takeaway from Mark’s insights on evaluating the recommender systems was to always define your objectives as this determines what type of evaluation to do.

The three key challenges with recommender systems are around the technical, legal and ethical issues. From a technical perspective, big data, cold start problems and predictive modelling, were cited as the main challenges. In essence, the more data we have, the better the predictions will be. Mark acknowledged that 80% of the problems with recommender systems are what is known in industry speak as a ‘cold start problem’, meaning when you have a new system, a new user or new item, you don’t have any or enough data to infer a user’s preferences to make recommendations and so it’s difficult to get started from scratch.

The legal challenges revolve around the more thorny issue of consent and transparency. Both aspects have important consequences for personalisation. The data protection regulation (GDPR), gives consumers ownership of their data and control of how that data can be used. The consequence is that there is less data available, so a smarter use of data is needed. Mark suggests a more nuanced approach could be taken by companies to demonstrate both the advantages and costs of privacy to the consumer.

Are we just living in our own bubble? Recommender systems serve to tailor our online experiences to better match our interests. They exist to give us a better service, and so increase revenue for the company, but the algorithms used to personalise our experience can also reinforce our behaviour, creating what is known as a ‘negative feedback loop’ or ‘filter bubble’. The ethical issues raised by this are hard to ignore. Mark uses his experience of watching cycling workout videos on YouTube to illustrate this. YouTube infers he likes watching videos on cycling and so shows him more cycling workout videos, which results in him watching more cycling workout videos. Sound familiar? Our observable behaviour creates a certain reaction, which then reinforces that behaviour. Much like the echo chamber idea where ideas and beliefs are repeated and shared within a closed system. The result limits our experiences, interactions and perspectives. What impact does this have on our world view and is this what we want from a personalised service?

This may seem a far cry from whether we buy that blue shirt recommended to us on Esprit’s website, but the ethical challenges of personalised systems illicit a philosophical reaction. Mark acknowledges that one of the main limitations of personalised systems is the focus on the ‘observable’ impact on behaviour, e.g. do customers purchase more, whereas the inclusion of a more user-centric model incorporating user psychology and user feedback would allow for a broader, more balanced personalised model. Perhaps this would allow recommender systems to enrich people’s lives as well as deliver on the promise of a more personalised service. Mark believes this could be the case; let’s hope so, after all, his expertise comes highly recommended.