Generative AI for Serendipity Recommendations

Serendipity is a concept associated with accidental and unexpected discoveries that are valuable. In the recent decade, many researchers have advocated serendipity, as part of the “beyond accuracy” metrics, to encourage a recommender system to be an exploratory discovery tool instead of a narrowly focused machine. However, due to serendipity’s elusive and subjective nature, it is challenging to model. Collecting large-scale ground truth data is also a challenge. In this dissertation, I addressed both the challenges of serendipity model construction and the ground truth collection for recommender systems.
Leveraging the recent breakthrough in generative AI and large language models, I utilized three types of generative AI models: Large Language Models (LLMs), Transformers-based cross-domain models, and Diffusion Models (DMs), to construct a serendipity recommendation model. In addition, I used Large Language Models to collect serendipity ground truth data from large-scale e-commerce reviews data. The extensive experiments demonstrated the effectiveness of generative AI in modeling serendipity and ground truth collection. This dissertation advances the understanding and implementation of serendipity in recommendation algorithms, which will empower ordinary people with opportunities of bumping into unexpected but valuable discoveries.