Multi-Agent Filtering Retrieval-Augmented Generation

Key Features :

Framework Features:

֍ Training-Free: No additional training data or fine-tuning required.
֍ Collaborative Multi-Agent Approach: Leverages multiple LLM agents to filter and score retrieved documents collaboratively.

Core Mechanisms:

֍ Adaptive Filtering: Dynamically adjusts the relevance threshold based on score distribution, reducing noise while maintaining high recall of relevant documents.
֍ Inter-Agent Consensus: Ensures robust document selection through agent agreement.

Embedding Creation

֍ Text Embedding: Using advanced NLP techniques, the textual data is transformed into embeddings, numerical representations capturing the semantic meaning of the content.
֍ Model Training: Pre-trained language models, such as GPT-3.5, are fine-tuned to understand the specific language and context of resumes and job descriptions.

Experimental Results

֍ Performance: Improved answer accuracy by 2-11% across four QA benchmarks compared to traditional RAG systems.
֍ Efficiency: Reduces irrelevant documents, decreasing computational overhead.
֍ Consistency: Provides more reliable and consistent responses.

Practicality

֍ Offers an effective, competitive, and training-free alternative to training-based RAG solutions.

Preparing Context for Vector Databases

Context Preparation

֍ Structure the context as indents paired with corresponding responses.

֍ Parse the context into small chunks based on indents and responses.

֍ Each chunk includes an indent and response, ensuring easy retrieval in a vector database.

Storing Context in Vector Databases

Vector DB (e.g., Qdrant):

֍ Efficient indexing and searching for more accurate query results.
֍ Enables faster retrieval of high-dimensional, unstructured data.
֍ Handles large datasets with billions of data points.

Retrieval Using Hybrid Search in Qdrant

֍ Combines dense and sparse retrieval methods to enhance search precision.
֍ Effective for domain-specific vocabulary, such as healthcare data.
֍ Executes hybrid search leveraging semantic understanding and keyword matching.

Ingestion Process

֍ Dense Embeddings: Use models like OpenAI's embeddings to capture text's semantic meaning.
֍ Sparse Embeddings: Employ methods like BM25 for keyword frequency and relevance.

Multi-Agent Workflow

Storing Context in Vector Databases

Agent-1 (Predictor)

֍ Analyzes retrieved document chunks and attempts to answer the query.
֍ Produces Document-Query-Answer (Doc-Q-A) triplets for further evaluation.
֍ Rephrases the query if returned chunks lack relevance.

Agent-2 (Judge)

֍ Evaluates Doc-Q-A triplets to assess relevance.
֍ Assigns "Yes" or "No" to each triplet:
      "Yes" for relevant documents supporting the query and answer.
      "No" for irrelevant or unhelpful documents.
֍ Quantifies judgments into relevance scores for filtering and ordering.

Agent-3 (Final-Predictor):

֍ Processes the refined list of relevant documents from Agent-2.
֍ Generates the final query response by extracting information from the cleaned document set.