What Is RAG

In 2023, Canada-based Algo Communications found itself facing a challenge. The company was poised for rapid growth, but it couldn’t train customer service representatives (CSRs) quickly enough to keep up with its expansion. To tackle this challenge, the company turned to a novel solution: generative AI.

Algo adopted a large language model (LLM) to help onboard new CSRs faster. In order to train them to answer complex customer questions with accuracy and fluency, Algo knew it needed something more robust than an off-the-shelf LLM, which is typically trained on the public internet, and lacks the specific business context needed to answer questions accurately. Enter retrieval augmented generation, better known simply as RAG.

By now, many of us have already used a generative AI LLM through chat apps like OpenAI’s ChatGPT or Google’s Gemini (formerly Bard) to help write an email or craft clever social media copy. But getting the best results isn’t always easy — especially if you haven’t nailed the fine art and science of crafting a great prompt.

Here’s why: An AI model is only as good as what it’s taught. For it to thrive, it needs the proper context and reams of factual data — and not generic information. An off-the-shelf LLM is not always up to date, nor will it have trustworthy access to your data or understand your customer relationships. That’s where RAG can help.

RAG is an AI technique that allows companies to automatically embed their most current and relevant proprietary data directly into their LLM prompt. And we’re not just talking about structured data like a spreadsheet or a relational database. We mean retrieving all available data, including unstructured data: emails, PDFs, chat logs, social media posts, and other types of information that could lead to a better AI output.