Improving Recommendation Systems with Deep Learning and Knowledge Graphs

Abstract

As time has moved forward, one resource has gone from a “nice-to-have” commodity to one of the world’s most valuable currencies: data. Data is further empowering and guiding our world in this modern age. Our decisions are data driven, drawing upon our historical choices and outcomes and new information to guide how we live. And at the forefront of the decisions we make are recommendation systems. Recommendation systems are a class of machine learning algorithms and techniques that suggest items that are “relevant” to their users. For example, these items could be online retail products, news videos/articles, or YouTube videos [1]. Such algorithms primarily use historical data to find items that could be relevant to a user, rank them based on relevancy, and display the most relevant items to the user, serving as their “recommendation”. Recommendation systems are at the forefront of business, employed by almost all major businesses in this day and age.

Recommendation algorithms usually fall into one of two categories: collaborative filtering (CF) based systems or content-based systems [2]. Collaborative filtering systems utilize historical interactions between users and the target items, while content-based systems will use extra information about the items and/or the users to make their recommendations. However, recent developments have led to more extensive usage of knowledge graphs in recommendation systems, utilizing their expressive power and relationships to produce even more predictive and powerful recommendation systems [3].

This article dives into recommender systems, researching how traditional recommendation systems work, how deep learning improves upon such systems, and how knowledge graphs can be used to further advance such systems.

A Data-Driven World

Our lives, now more than ever, have become data-driven. That is, we constantly are using data to guide our actions and choices, helping us make informed decisions. We are able to do this so frequently and extensively due to the sheer amount of data in the world, a number that doubles in size every two years [4]. Unsurprisingly, data is growing faster than ever before, with 90% of global data being created in the last two years. On any given day, humans generate 1,000 petabytes of data a day.

Figure 1 — Raconteur’s infographic on the scale of data generation [4]

Although there is such an overwhelming amount of data, humans have been able to harness this big data to grow businesses, create more personal user experiences, and enable more informed choices for us with the use of recommendation systems. These recommendation systems are the same systems that recommend you the YouTube videos that you enjoy everyday, and the news articles on Google that contain content you are interested in. Such systems suggest items and content that are relevant to the user based on historical user and/or item information. So, the data that we create and the online choices we make further create a profile indicating our likes and dislikes, and use the vast amount of historical data out there to make recommendations to us and help guide the decisions we make. That is the crux of recommendation systems.

Deep Learning and Knowledge Graphs — A Primer

Modern recommendation systems employ deep learning and knowledge graphs to better their design. However, before we discuss how those technologies are used in recommendation systems, I want to give a quick primer discussion on deep learning and knowledge graphs to get us ready for further discussions.

What is Deep Learning?

Deep learning is a subset of machine learning, and is a facet of representation/feature learning where the machine typically takes raw, unstructured, and unlabeled data as input. Using this input, deep learning models can learn internal representations to perform classification or regression without human intervention[5]. This is one of the biggest advantages of deep learning compared to other machine learning methods, where humans are usually needed to perform feature extraction and feed labeled data into the neural network. Deep learning is able to perform feature extraction itself to perform classification and regression later on.

Figure 2 — Machine Learning vs. Deep Learning [6]

What is a Knowledge Graph?

Knowledge graphs are a technology that have been around for a while, but are being discussed more and more lately due to more realization of their power. Knowledge graphs have several different, but equally correct definitions. However, a good and concise definition that I like is that knowledge graphs are things that “depict knowledge in terms of entities and their relationships” [7]. Simple, right? As a result, knowledge graphs serve as a very powerful tool nowadays. They enable us to do many things, like the following:

  • Make connections and establish relationships among different entities across multiple data sources
  • Act upon and pull from information based on its surrounding context
  • Infer dependencies and new relationships between entities
  • Model things and their relationships more tangibly and more aligned with how we perceive and connect such things in our heads

As you can see, knowledge graphs give great power in the context and relationship data they provide inherently in their structure. An example of a knowledge graph is shown by a simple Google search.

Figure 3 — Google Knowledge Panel of Vincent van Gogh [8]

This is the result of googling “Vincent van Gogh”. Google shows what they call a “knowledge panel” of most searches. In this case, van Gogh is the entity. This panel shows information and facts about the entity, and also displays things related to the entity (shown by the “people also search for”) and other entities produced by van Gogh (artworks and quotes). The other information other than van Gogh on these panels can be seen as other entities, related by relationships like same time period, similar art style, created by, etc. With this small example, one can see how informational and contextual knowledge graphs can be, which is why they are becoming more and more popular.

An Overview of Recommendation Systems

Now we are ready for one of the main topics of discussion: recommendation systems. Simply put, recommendation systems are systems that recommend items based on historical user data and/or item data. In these systems, an item could be a piece of clothing, a video, a song, etc. There are two main classes of recommendation systems: content based and collaborative filtering based. Most other recommendation systems are just variations of hybrids of these methods. We will now dive into both of those methods in detail.

Content Based Recommendation Systems

Content based systems are recommendation systems that use the features of items to recommend other items that are similar to what the user likes [9]. This method uses explicit feedback from a user and/or historical choices to learn what a user “likes”. The defining-point of this method is that it is independent of other users. That is, it does not consider other users and their preferences when recommending for the current user. Now, when determining what items are “similar”, different similarity metrics can be used. Some common similarity metrics include cosine similarity, dot product, and more. One of the downsides to content based recommendation systems is that it often requires people to hand-engineer features for the items, creating a bit of overhead.

To get a better idea of how content based recommendation systems work, we will walk through an example based on the image below.

Figure 4 — Content Based Example [9]

Imagine that a user–let’s call them Suzan–is searching for entertainment apps on an app store. In this image/example, each row represents an app, and each column a genre. The bottom row represents our user Suzan. As you can see, each app (row) can be modeled as a vector, where a dot is placed in a column if an app is of that genre. For example, the first row, a science app, fits the genres of education and Science R Us. Now, Suzan has indicated through explicit feedback that she likes education, healthcare, and Science R Us. With all this information, a content based recommendation system would use some specified similarity metric and choose the app candidates that are most similar to the interests specified by our user. In our example, the science app in the first row most aligns with the preferences indicated by our user, so that would likely be the top candidate recommended to our user Suzan.

Collaborative Filter Recommendation Systems

On the other hand, collaborative filtering is a recommendation technique that uses the “similarities between users and items simultaneously to provide recommendations” [9]. In this way, collaborative filtering allows for recommendations to a user based on the interests of a different, but similar user. This is how collaborative filtering mainly differs from content based filtering.

We will again walk through an example of collaborative filtering using the image below to get a better idea of how the process works.

Figure 5 — Collaborative Filtering Example [10]

With collaborative filtering, we usually want to start out by creating a user-to-item matrix, representing different users and their feedback (preferences) towards different items. This is what we see in figure 5 above. Here for example, we can see that user D has indicated they do not like photographs (column 1) and do like videos (column 3), and have not given feedback about the other columns.

After this user-to-item matrix has been constructed, we can start recommending. For this example, we want to know whether or not user E might like video (column 3), and if we should recommend that to them or not. First, we find the k-nearest neighbors to the user in question, where “nearest” is defined by some similarity function you specified beforehand, like those mentioned above (cosine similarity, dot product, etc). Then, using that similarity function, we determine what k neighbors are most similar to us and make our prediction based on the feedback of those users.

Content Based vs. Collaborative Filtering Recap

To summarize, we have content based filtering (right side of figure 6) which uses the features of items to recommend other items that are similar to what the user likes. And then we have collaborative filtering recommendation systems (left side of figure 6) that use the “similarities between users and items simultaneously to provide recommendations”.

Figure 6 — Collaborative Filtering vs. Content Based Filtering [11]

These are the traditional methods that have served as the basis for most recommendation systems.

Enhancing Recommendation Systems with Deep Learning

Now that we know how traditional recommendation systems work, we can take a look at how other methods are used to enhance them. The first method we will discuss is that of deep learning.

Now, some might ask, “Why use deep learning for recommendation systems?”. I find that there are three main reasons:

1. Modeling complex user patterns (nonlinear transformation)

Traditional recommendation systems often rely on linear models. However, user-item interaction patterns are highly complex, and cannot truly be captured by such linear models. We need some way to perform nonlinear transformations to create more complex and accurate models. Deep learning allows us to do so. Deep neural networks enable the modeling of nonlinear and complex user-item interactions through the use of nonlinear activation functions, like tanh, relu, sigmoid, and more. With such activation functions, we can attain more accurate modeling.

2. Feature/Representation Learning

As discussed earlier, humanity these days has an astronomical amount of data. Luckily, large amounts of data are what allow deep learning to thrive. The use of such large amounts of data enables deep learning techniques to learn features, reducing and sometimes eliminating the need to hand-craft features.

3. Flexibility

The landscape for deep learning frameworks is vast and varied, with frameworks like Tensorflow, PyTorch, Keras and more at your disposal. The variety of frameworks allows for great flexibility, experimentation, and optimization to allow developers to find the best frameworks and methods for them when using deep learning for recommendation systems (among other things).

As just discussed, there are a plethora of advantages and reasons to use deep learning to enhance recommendation systems. However, there are always some downsides to such methods.

The main disadvantages and possible limitations when using deep learning for recommendation systems relate to deep learning’s “black-box” behavior and its data requirement. Because weights in deep learning models are typically hidden, and activation functions are complex, deep learning models are often hard to explain and interpret (hence “black-boxes”). As such, there is an argument to be made against deep learning for recommendation systems if you need clear interpretability. Additionally, in terms of data requirement, deep learning models require large-scale data to effectively perform and learn features. This can be harder in other domains, but luckily due to the vast amounts of user data available this is not usually a big problem for recommendation systems.

An example of a deep learning recommendation system used in the real world is neural collaborative filtering (NCF). Founded by researchers from the National University of Singapore, Columbia University, Shandong University, and Texas A&M University, NCF uses a multilayer perceptron to learn user-item interaction [12].

Figure 7 — Neural Collaborative Filtering Architecture [12]

This deep learning recommendation technique takes, as input, two sparse vectors: a user vector and an item vector. The user vector signifies which user we are talking about, while the item vector signifies which items the user interacted with. Both of these vectors are one-hot encoded. The following layers are embedding layers that form the latent/dense vectors from the input layer. Then, those latent vectors are fed further into the multi-layer network, whose intent is to learn the sophisticated user-item relations from the input data. The output layer returns the predicted scores, signifying the probability that a given user u will interact with item i. This method is actually used in the real world by companies such as Spotify. Spotify uses neural collaborative filtering to help generate their Discover Weekly playlists, with music tailored to each individual [13].

Enhancing Recommendation Systems with Knowledge Graphs

The final piece to our discussion is all about how we can use knowledge graphs to empower recommendation systems. Within recommendation systems, knowledge graphs are often used as side information and context for the system. There are several ways this can be done, but the one that I will discuss is embedding-based techniques called knowledge graph embeddings (KGE).

Knowledge graph embeddings enrich the representation of users or items by using information from the knowledge graph [2]. These knowledge graph embedding algorithms encode the knowledge graph into a low-rank embedding, creating a better representation of the items/user and then integrating that external information into the recommendation system. Generally the process is as follows:

First, we form a latent vector vj for each item vj by aggregating information from the knowledge graph, user-item interaction matrix, and the item’s attributes. Next, we form a latent vector ui for each item ui from the user-item interaction matrix. Once those latent vectors are created, we use a mapping function, f, to map the embedding of the item and user into a “preference score”. This mapping function could be something like a deep neural network. The generated preference score, also denoted as ŷ (y-hat), represents the probability of user ui selecting item vj.

Figure 8 — KGE Preference Score [2]

As discussed a bit earlier, knowledge graphs are useful to recommendation systems as they introduce more external information, often solving some data sparsity issues. Collaborative filtering cannot recommend new items that it doesn’t have any historical feedback for, so adding external information could help to rectify that weak point. Another advantage of using knowledge graphs with recommendation systems is because they increase explainability. Knowledge graphs allow you to look at the knowledge base/graph and see the semantic representation of the relations and entities used to draw results, shedding context and insight into the results you acquire.

Knowledge graphs for recommendation systems are not just some toy research project either. They are used in the real world to solve real problems. An example of using knowledge graphs to recommend in the real world is seen through Google’s search engine. Google uses a knowledge graph to recommend similar things when you search for something on Google.com. As shown in the example earlier with van Gogh, it recommends similar artists, artists during van Gogh’s time period, van Gogh’s paintings, and more, using Google’s knowledge graph to gain context of your search and use the entity and its relations to recommend you another interesting search result

Closing Thoughts and Implications

In this article we’ve taken a close look at recommendation systems and how to amplify their usefulness and power using deep learning and knowledge graphs. However, we are only able to do so as a result of big data. Big data empowers deep learning and knowledge graphs to allow us to improve recommendation systems for consumers and users. Additionally, the use of such methods also increases explainability and context behind such recommendations, allowing users and developers to better understand why certain things are recommended or not.

However, as with most topics in emergent technology, there are other implications to take into account when discussing the recommendation systems. Recommendation systems often bring up ethical issues such as data privacy, addictive design, and other potential issues.

Ethical Implications of Recommendation Systems

Concerning the issue of data privacy, personalized recommendation systems make users prone to privacy violation issues due to their collection of personal data for analysis and learning purposes [14]. This data “undesirably discloses the users’ personal interests to the recommender”, which can even be sold to third parties without the consent of the user (as seen in the Facebook / Cambridge Analytica scandal). Potential solutions to this issue are keeping yourself (as a user and developer) informed about this, passing appropriate legislation, and limiting/barring the unwilling selling of personal data.

Regarding the issue of addictive design, it is apparent that recommendation systems are designed to be addictive. Platforms such as YouTube and Hulu will continue to autoplay content, keeping you watching endlessly. The core issue here is that some companies designing their recommendation engines to be addictive are taking advantage of human psychology to keep users hooked. This is acknowledged by the creators of some of the most addictive platforms, seen by how Sean Parker, a founding member of Facebook, said the thought process behind Facebook and many other current applications is “How do we consume as much of your time and conscious attention as possible” [14]. When a recommendation platform enters the gray area of persuasion to watch more content, or pseudo-coercion through features like autoplay and more, we as users must take note and evaluate if that is what we truly want out of this system. To combat such issues, we should establish new ethical standards that bar this sort of behavior and promote a much healthier and optional platform.

Final Thoughts

We have seen how powerful recommendation systems can be, especially when enhanced with deep learning and knowledge graphs. However, with great power comes great responsibility. As such, when considering employing recommendation systems, these considerations about their implications should be carefully taken into account to ensure a fair and ethical environment is formed around them, or serve as a warning if such implications are to be ignored.

Beyond the ethical considerations, recommendation systems can still aid our lives. Because of big data and technological advancements, recommendation systems are becoming more and more ingrained into modern day consumerism. They help humans with the “data flood”, sifting through large amounts of data to find a small subset of items that will likely be of interest to us. Without having to do as much research when picking items, recommendation systems allow users to spend more time on the important things in life. In these ways, recommendation systems, empowered by deep learning and knowledge graphs, serve to make the lives of users just a little bit (if not a great bit) easier.

References

  1. G. Seif, “An Easy Introduction to Machine Learning Recommender Systems,” KDnuggets. https://www.kdnuggets.com/an-easy-introduction-to-machine-learning-recommender-systems.html/ (accessed Jan. 16, 2021).
  2. Q. Guo et al., “A Survey on Knowledge Graph-Based Recommender Systems,” arXiv:2003.00911 [cs, stat], Feb. 2020, Accessed: Jan. 16, 2021. [Online]. Available: http://arxiv.org/abs/2003.00911.
  3. A. Brams, “Movie Recommendations powered by Knowledge Graphs and Neo4j,” Medium, May 03, 2020. https://towardsdatascience.com/movie-recommendations-powered-by-knowledge-graphs-and-neo4j-33603a212ad0 (accessed Jan. 16, 2021).
  4. I. Hemo, “Big Data Statistics: How Much Data Is There in the World?,” Rivery, Mar. 27, 2020. https://rivery.io/big-data-statistics-how-much-data-is-there-in-the-world/ (accessed Jan. 19, 2021).
  5. A. Said and V. Torra, Eds., Data Science in Practice, 1st ed. 2019. Cham: Springer International Publishing : Imprint: Springer, 2019.
  6. E. Sperling, “Deep Learning Spreads,” Semiconductor Engineering, Jan. 31, 2018. https://semiengineering.com/deep-learning-spreads/ (accessed Feb. 05, 2021).
  7. M. Knight, “What is a Knowledge Graph?,” DATAVERSITY, Jun. 10, 2020. https://www.dataversity.net/what-is-a-knowledge-graph/ (accessed Jan. 20, 2021).
  8. “Vincent van Gogh — Google Search.” https://www.google.com/search?kgmid=/m/07_m2&hl=en-US&q=Vincent+van+Gogh&kgs=ee10a987065e016c&shndl=0&source=sh/x/kp&entrypoint=sh/x/kp (accessed Mar. 02, 2021).
  9. “Introduction | Recommendation Systems,” Google Developers. https://developers.google.com/machine-learning/recommendation (accessed Jan. 21, 2021).
  10. “(Prediction|Recommender System) — Collaborative filtering.” https://datacadamia.com/data_mining/collaborative_filtering (accessed Feb. 18, 2021).
  11. L. N. Tondji, “Web Recommender System for Job Seeking and Recruiting,” 2018, doi: 10.13140/RG.2.2.26177.61286.
  12. A. Roy, “Introduction To Recommender Systems- 2: Deep Neural Network Based Recommendation Systems,” Medium, Aug. 02, 2020. https://towardsdatascience.com/introduction-to-recommender-systems-2-deep-neural-network-based-recommendation-systems-4e4484e64746 (accessed Jan. 24, 2021).
  13. C. Underwood, “Use Cases of Recommendation Systems in Business — Current Applications and Methods,” Emerj. https://emerj.com/ai-sector-overviews/use-cases-recommendation-systems/ (accessed Jan. 24, 2021).
  14. H. DeLeon, “The Ethical and Privacy Issues of Recommendation Engines on Media Platforms,” Medium, Apr. 24, 2019. https://towardsdatascience.com/the-ethical-and-privacy-issues-of-recommendation-engines-on-media-platforms-9bea7bcb0abc (accessed Mar. 02, 2021).

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Parker Mitchell

Parker Mitchell

Computer Science Graduate Student @ Cal Poly, San Luis Obispo. Always looking for interesting things.