Retrieval Augmented Generation (RAG) Consulting and Support
For Expert Retrieval Augmented Generation (RAG) & Consulting Support
Get in touch with us
Let's break ice
Email Us
What Is Retrieval-Augmented Generation (RAG)?
Large Language Models (LLMs) are deep learning neural networks that have been trained on
large text datasets scraped from the Internet and other sources. They are being used in an
increasing number of AI-based products and services, such as chat support agents, virtual
assistants, code generators, and much more. However, there are two major drawbacks with
LLMs:
- They only have knowledge of topics up to a certain date. For example, ChatGPT-4 does
not have knowledge of events from April 2023 onward. - The models “hallucinate” in their response, and confidently give wrong answers on
topics they don’t know about. - Publicly available models have no access to private data and are unable to generate any
content about proprietary information.
Businesses seeking to deploy their own LLM can overcome these limitations by fine-tuning
models on a custom dataset. However, the process of curating a dataset for fine-tuning takes
significant effort and time.
Retrieval-Augmented Generation (RAG) is a method for augmenting a LLM’s knowledge with
specific data beyond the general dataset it was trained on. It increases the quality and accuracy
of LLMs without needing to fine-tune a custom model. With RAG, the LLM is provided with a set
of documents to retrieve information from, and then it answers user queries with the information
from the documents. This way, the model has immediate access to the necessary information
and can use it to generate an accurate response.
RAG offers a number of benefits over traditional generation models, including:
- Improved accuracy and informativeness: RAG models can generate more accurate and informative responses by leveraging the knowledge and information contained in a large knowledge base.
- Reduced hallucinations: RAG models are less likely to generate hallucinations, which are false or misleading responses.
- Increased domain specificity: RAG models can be customized to specific domains by using domain-specific knowledge bases.
RAG can be used for a variety of NLP tasks, including:
- Question answering: RAG can be used to generate answers to questions by retrieving relevant information from a knowledge base and then summarizing it in a clear and concise way.
- Document summarization: RAG can be used to generate summaries of documents by extracting key information and then presenting it in a condensed form.
- Chatbots: RAG can be used to power chatbots that can provide informative and engaging conversations with users.
RAG models can be computationally expensive to train and deploy. Additionally, they require access to a large and high-quality knowledge base.
RAG is a relatively new technology, but it has the potential to revolutionize the field of NLP. RAG models are becoming increasingly powerful and accessible, and they are being used for a wide range of applications. As RAG technology continues to develop, we can expect to see it used in even more innovative and impactful ways.
RAG can be used in business in a number of ways, including:
- Customer service: RAG can be used to power chatbots that can provide customer support and answer customer questions.
- Marketing: RAG can be used to generate personalized marketing campaigns and product recommendations.
- Sales: RAG can be used to generate leads and qualify prospects.
- Product development: RAG can be used to generate product ideas and feedback from customers.
- Research and development: RAG can be used to generate new hypotheses and insights.
There are two main types of RAG models:
- Retrieval-based language models: These models retrieve information from a knowledge base and then use that information to generate a response.
- Generative AI: These models generate text from scratch, but they can be improved by using information from a knowledge base.
A good RAG model should have the following features:
- Accuracy: The model should be able to generate accurate and informative responses.
- Fluency: The model’s responses should be fluent and easy to read.
- Comprehensiveness: The model should be able to generate responses that are comprehensive and cover all aspects of the query.
- Relevance: The model’s responses should be relevant to the query.
- Diversity: The model should be able to generate a variety of different responses to the same query.
Some of the best-known RAG models include:
- BART: BART is a retrieval-based language model that was developed by Google AI.
- T5: T5 is a generative AI model that was developed by Google AI.
- RAG-Transformer: RAG-Transformer is a RAG model that was developed by Google AI.
Want to Unlock the Potential of Your Data with RAG?
~ Testimonials ~
Here’s what our customers have said.
Empowering Businesses with Exceptional Technology Consulting
~ Case Studies~
Retrieval Augmented Generation (RAG) Consulting Support Case Studies
RAG-Powered Delivery Support Chatbot for DoorDash
Customer Tech Support Chatbot Using RAG and Knowledge Graph
Internal Policies Knowledge Management Chatbot
AI Professor Chatbot for Entrepreneurship Education
Conversational Video Summaries with RAG
Automated Analytical Fraud Reporting with RAG
Introduction to Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) is a method for augmenting a LLM’s knowledge with specific data beyond the general dataset it was trained on. It increases the quality and accuracy of LLMs without needing to fine-tune a custom model. With RAG, the LLM is provided with a set of documents to retrieve information from, and then it answers user queries with the information from the documents. This way, the model has immediate access to the necessary information and can use it to generate an accurate response.
There are several benefits to using RAG:
- RAG increases the accuracy of LLM responses, because the LLM can directly reference the set of information provided rather than relying on its general knowledge.
- It significantly reduces the likelihood of hallucination and incorrect responses. If the provided documentation does not have the information the user is looking for, the LLM can simply say it doesn’t know the answer to the user’s query.
- The LLM’s knowledge source can be updated in real time so it can stay current with changes in information.
- RAG responses are more transparent because they can include references to the source of the information.
RAG allows companies to develop AI-enabled chatbots with specific knowledge of their product or service without needing to go through the costly and time-consuming process of fine tuning. For example, a health insurance company can provide an LLM with information about its health care plans to give it knowledge of their policies and rates. Then, when a customer asks about premiums and coverage for a certain plan, the RAG-empowered chat agent can respond with detailed and accurate information. If portions of the plan change, the company can immediately upload the changed documentation so the chatbot stays up-to-date with current knowledge.
Explanation of RAG Pipleline
A typical RAG pipeline consists of several stages and may use more than one deep learning model in the question-answering process. An example full pipeline is shown below.
The first four stages occur offline, before the model is deployed:
- The source documents are loaded using an unstructured loader (which supports .txt, .pdf, .html, etc).
- The documents are split into chunks of text, making them easier to parse. The chunks of text have to be short enough to fit in the LLM’s context window and must be small enough to be accurately representable by a single embedding.
- These text chunks are converted to vectors that numerically represent the information in the text using an embedding model.
- The vectors are stored in an indexed database which can easily be searched through. When the application is active and a user submits an input prompt:
- The user’s text input prompt is converted to a vector using the same embedding model.
- The query vectors are mathematically compared to the document vectors to determine which portions of documentation are most relevant to the query, returning the “top k” matches.
- (Optional) A cross-encoder based rerank model is used to sort the top k document matches. Rerank models use transformer networks to determine the similarity of thequery to the document chunks, so they may provide a better order of relevance to the query.
- The relevant chunks of documentation are added as context to the original user input and submitted to the LLM.
- The LLM returns an answer based on the user input and the context of the documentation it was provided.
rerank model, and a large language model.
Text Embedding Models
A fundamental part of RAG operation is text embedding. This is the process of converting a sequence of words (text) into a sequence of vectors (numbers) that represent the information contained in the words. Embeddings make it so deep learning models and other algorithms can numerically calculate the relationships between pieces of text, allowing them to perform tasks such as clustering or retrieval. In the case of RAG, embeddings are used to compare the user input to the stored documents so the pipeline can retrieve the pieces of text that are similar to the user query. It occurs at the “Embedding” stages in the pipeline.
Converting a series of words into a series of vectors that contain the semantic meaning of the words is not a straightforward task. It requires the use of embedding models, which are deep learning models trained to efficiently encode words into high-dimensional vectors. A list of popular embedding models can be found at the Massive Text Embedding Benchmark Leaderboard.
The main difference between the small and large models is the number of parameters. The large models have better scores on MTEB [reference], a benchmark that tests average performance across eight embedding tasks. The benchmark measures how well the vector conversion retains the information from the text. The large models require more memory and processing power, but the resulting vectors will contain more semantic meaning. Ultimately, the larger model helps the RAG pipeline find better documentation matches to the user query. However, the small model is likely accurate enough for most use cases. Developers can use the OpenVINO™ RAG Notebook (see Section 5.1) to test both models in a full pipeline and determine which works best for their use case. Other popular embedding models include:
● mxbai-embed-large-v1
● UAE-Large-V1 and UAE-Code-Large-V1
● gte-base-en-v1.5
Rerank Models and Two-Stage Retrieval
Rerankers compare an input query against a set of documents and rank the documents in order of relevance to the query. (This all occurs in embedded vector space.) Rerankers are part of a two-stage retrieval system. Before deployment, a full dataset of documents is encoded into vectors using a text embedding model. In the first stage, the query is converted to a vector, and then mathematically compared to the document vectors (using “cosine similarity” or a similar metric) to determine which document chunks have data relevant to the query. The first stage will return the “top_k” documents (e.g. top_k = 25).
In the second stage, a rerank model, which is more accurate than the mathematical comparison, is used to rank the top_k documents in order of relevance to the query and return the “top_n” best documents (e.g. top_n = 5). Rerankers benefit from having the context of the user query to better locate relevant information. They also require more processing than the mathematical comparison used in the first stage.
Here’s how two-stage retrieval works in the RAG pipeline:
- Chunks of a database of text is converted to vectors using a text embedding model
- A user query is converted to a vector.
- The query vector is compared to the database vectors to identify the top_k most relevant
chunks of text in the database A reranker model sorts those top_k chunks in better order of relevance to the user query.
Similar to the embedding models, the main difference between the base and large BGEreranker models is the number of parameters. The larger model requires more memory, but will do a better job of ranking documentation chunks by their relevance to the user query.
Large Language Models
Large language models (LLMs) are massive neural networks that excel in various languagerelated tasks like text completion, question-answering, and content summarization. LLMs are trained from an extensive repository of text data, which is usually collected by crawling web pages across the internet. LLMs are trained for a simple task: given a sequence of words, predict the next word in the sequence.
With RAG, LLM inference occurs at the last stage of the pipeline. The pipeline assembles a new prompt that combines the user query along with the relevant documentation. The new prompt may be arranged as:
“Answer this question based only on the following context: Question: ” This allows the LLM to reference the provided chunks of documentation when generating a response, thereby increasing the accuracy of the response and reducing the likelihood of false information.
Most LLMs available from HuggingFace Hub can be used in the RAG pipeline. The main design parameter is model size (i.e., number of parameters): a large model will generate better responses, but will also require more memory and processor power.