Anyone reading this is presumably familiar with product recommendation systems at on-line stores. As you browse and buy products, e-commerce sites recommend other products that might interest you. These are typically labeled “people who bought this product also liked…” or “people who bought the things you bought also liked….”
If you ask most owners of on-line stores, they’ll tell you that product recommendation systems are a huge success. By most accounts, successful on-line stores earn between 5% and 10% of their revenue on products that customers reach through recommendation links. That translates into a lot of revenue.
But if you ask most on-line shoppers, they’ll say that the recommended products rarely match what they want to buy at the moment. Shoppers who buy running shoes are most often recommended other pairs of sneakers, while at the moment they really are more interested in fitness monitors, energy snacks, books about jogging, treadmills, or a fashionable belt to match their new fashionable sneakers.
And customers are starting to speak out. A comment in a REDDIT forum gave an example: “I've bought hundreds of books from Amazon. My last purchase was a John Sanford novel. It was okay, but I didn't feel strongly enough to write a review or even rate it. Yet when I was looking for a new book today, and clicked on "Recommended For You," it quickly gave me 20 recommendations. Nineteen of which were Sanford novels.” Similar discussions can be found on Quora and many other sites. A brief search on Twitter finds comments like “my favorite bad recommendation was when I bought an expensive new TV, and the next week they recommended me ten more,” “I got a Kindle PW for Christmas. I just bought an ebook... This leads to the recommendation: "Buy a Kindle PW!" #bigdata #notworking,” and many more.
So why are so many on-line recommendations so different from what we want to buy?
On a technical level, virtually all the recommendation systems on the Internet today use variations on a twenty-year-old approach called collaborative filtering. Collaborative filtering is a “big data” approach that compares the lists of items that many different people purchased, to find sets of people that purchased very similar products. If John and Bill have a 95% overlap in their lists of purchases, then odds are that each would like to buy the other items on the other one’s list of purchases. Obviously there are many complexities, but that’s the idea.
Amazon and others have more recently started using what they call item-to-item collaborative filtering, where instead of comparing different people’s shopping lists, they compare different items and the other items bought along with them. So instead of basing recommendations on the fact that John and Bill have similar interests, the item-to-item approach based recommendations on the fact that sneakers are most often found in shopping carts alongside other sneakers. This is a more flexible approach, allowing different recommendations on product pages (“people who bought this liked…” and shopping cart pages (“people who bought what you’re buying liked….” It also accounts for the fact that people have different interest, and may be similar to some other people in regard to one of their interests but dissimilar to them in regard to other interests.
What these approaches have in common, however, is that majority rules. If you’re buying a pair of Adidas running shoes, recommendation engines will look at what the majority of Adidas running shoe buyers buy alongside their running shoes. If you’re buying a tablet or e-book reader, the majority of people will have bought, at some time in their buying history, another tablet or e-book reader.
At the moment of purchase, however, majority does not rule. Most products are bought for a reason. When a customer buys a book on travel to Ireland, there is a strong likelihood that they’re about to go to Ireland, and might need a new carry-on bag or a lightweight e-book reader. What the customer will need at the moment of sale depends on the customer. Yet the majority-rules approach looks at the products that are most commonly bought by other buyers of travel guides to Ireland, which are travel guides to other cities. The specific things that each person needed at the same time as they bought a travel guide are ignored. The products of individual interest are not recommended, and the on-line store misses the opportunity to sell customers what they want in the moment.
Stop and consider this point, because it’s the crux of the matter. Most people are in fact repeat buyers. People that buy sneakers on-line once are likely to buy sneakers on-line a second time. People that buy travel guides before a trip to Ireland are very likely to also buy a travel guide to France before a trip to France.
But these “majority rules” similarities are very often not related to what the customer needs at the moment that he buys the travel guide or the sneakers. By looking at the majority, the immediate needs, that are more individual and less related to the majority, are missed.
In the example of the running shoes, above, it is in fact true that most people that buy running shoes on-line also buy other running shoes at some point. That’s why on-line stores recommend more running shoes when a customer buys a pair. But this majority-rules approach misses the opportunity to sell that customer a fitness monitor, or an introductory book on jogging, or energy bars, or any of the other things that the customer is thinking about at the moment of the purchase.
Cognilyze is a new start-up company, currently seeking investment, that is taking a radically different approach to product recommendation. Rather than using collaborative filtering to compare purchases of different customers, Cognilyze is analyzing each individual customer’s motivations and behaviors. Cognilyze’s technology uses several different theories from cognitive and behavioral sciences to analyze each customer’s purchases in terms of short-term motivations, personal goals, personality, social membership, and more. After determining a profile for a customer’s motivations and preferences, Cognilyze’s system uses the profile to generate recommendations, to sell the customer the other things they want in the moment.
Clearly Cognilyze’s analysis of a customer’s motivations will be somewhat speculative, especially if very little is known of the customer. In tests so far, Cognilyze has found that if their system knows more than ten products that the user is interested in, their technology can generate recommendations that are of strong interest to customers. The more products of interest the technology can analyze, the stronger the profile will match the customer.
Even more importantly, a profile of a customer’s motivations is also useful for business intelligence or personalization. Recommendation systems that work on an item-to-item basis will not profile customers in ways that can be used for BI or personalization. But Cognilyze’s approach generates a profile of overall motivations and personality types, which can be used for a wide variety of other business purposes.
How does Cognilyze’s technology analyze customer motivations? How has Cognilyze implemented cognitive and behavioral theories of motivation in a practical recommendation engine? How can Cognilyze customer profiles be used for personalization or BI? All these are the subjects of future articles. Follow Cognilyze on its blog, LinkedIn, Twitter, Google+ or Facebook to learn more.
