Sachin Kottarathodi's Space

Tag: RAGs

Product Quantization

Posted:   |   Categories: tech   |   Tags: GenAI, LLMs, RAGs

AI-polished: This post originated from notes and questions while working through the topic in detail. AI was used to organize the notes, connect the ideas, and polish them into a coherent blog post. Product Quantization — From 40 GB of Vectors to 160 MB Dense retrieval introduces a scaling problem beyond search latency: memory. Suppose a vector database contains: 10,000,000 vectors Each vector contains: 1024 dimensions and each dimension is represented using: Read more...

RAG Retrieval: From Keyword Search to Vector Search

Posted:   |   Categories: tech   |   Tags: GenAI, LLMs, RAGs

Fully AI generated notes. 1. Why Does RAG Exist? An LLM primarily carries knowledge in its parameters. During training: Training Data ↓ Model Training ↓ Weights / Parameters ↓ Parametric Knowledge Once trained, asking the model a question does not normally cause it to search through its original training documents. The answer is generated from what has been learned into the model’s parameters. That creates obvious limitations: knowledge can become stale private organizational data was never part of training the model may not know niche information updating knowledge by retraining is expensive generated answers aren’t inherently grounded in a specific source RAG — Retrieval-Augmented Generation — adds external knowledge. Read more...