AI generated - These are quick reference notes meant to help reconstruct the concepts later rather than serve as a deep dive.
NLP Techniques — Quick Reference Technique Concept One-liner Bag of Words (BoW) Sparse text representation Represents text using word occurrence/counts while largely ignoring word order and context. Naive Bayes Probabilistic classifier Uses Bayes’ theorem with a strong feature-independence assumption; historically common for text classification. VADER Lexicon-based sentiment analysis Uses a predefined sentiment lexicon and rules to estimate positive, negative, neutral, and compound sentiment.Read more...
AI-polished: This post originated from notes and questions while working through the topic in detail. AI was used to organize the notes, connect the ideas, and polish them into a coherent blog post.
Product Quantization — From 40 GB of Vectors to 160 MB Dense retrieval introduces a scaling problem beyond search latency: memory.
Suppose a vector database contains:
10,000,000 vectors Each vector contains:
1024 dimensions and each dimension is represented using:Read more...
Fully AI generated notes.
1. Why Does RAG Exist? An LLM primarily carries knowledge in its parameters.
During training:
Training Data ↓ Model Training ↓ Weights / Parameters ↓ Parametric Knowledge Once trained, asking the model a question does not normally cause it to search through its original training documents.
The answer is generated from what has been learned into the model’s parameters.
That creates obvious limitations:
knowledge can become stale private organizational data was never part of training the model may not know niche information updating knowledge by retraining is expensive generated answers aren’t inherently grounded in a specific source RAG — Retrieval-Augmented Generation — adds external knowledge.Read more...
Fom Pixels to Diffusion: How Image Generation Actually Works This post began as scattered notes and questions while trying to understand each of these topics in more detail. The deeper the questions went, the more the concepts started connecting, but the notes themselves remained fragmented. AI was used to piece those fragments together into a coherent sequence. The result is a connected mental model of the topics discussed. It is intentionally simplified, as the goal at this stage is to understand the core ideas without getting lost in the deeper mathematical and implementation details.Read more...
There is a idea that the biggest returns belong to the investor who discovers a business before most others. Find the company before the market notices it. In hindsight, this looks obvious. The early investor gets the biggest part of the move, while everyone else is left paying a higher price after the story becomes established.
The earliest stage of a business is also where uncertainty is highest. For a retail investor, especially one without access to differentiated information, trying to consistently front-run these opportunities can amount to taking a large amount of business risk for a relatively small informational advantage.Read more...
Disclaimer
This post began as my scattered notes and questions while learning LLMs. I couldn’t stitch them together so used AI to do that for me. The fragments are connected into a coherent model. It is intentionally simplified as I am not there yet for this to be too complex or detailed at this point.
Why this exists I found myself collecting terms like BPE, RoPE, KV Cache, Teacher Forcing, PagedAttention, Quantization and Data Parallelism.Read more...
From File Systems to Transistors: How Operating Systems and Hardware Enforce Atomicity - Part 1 AI Generated Problem Statement: If two CPU cores attempt the same operation at almost exactly the same instant—such as creating temp.txt with O_CREAT | O_EXCL—why does only one succeed?
Every backend engineer has relied on file-system atomicity. Creating a lock file with open(..., O_CREAT | O_EXCL) or using mkdir() as a lock are common synchronization techniques.Read more...
Explanations of Long Polling start with the server “holds the connection open.”.
To understand Long Polling, it helps to go one level lower than HTTP.
Suppose a client wants to fetch messages.
GET /messages The HTTP request is converted into bytes, passed to TCP, encapsulated into IP packets and sent over the network.
Before any HTTP request is sent, TCP establishes a connection.
Client Server SYN ----------------------> <------------------- SYN-ACK ACK ----------------------> Once this handshake completes, the client and server have an established TCP connection.Read more...
AI polished. Not AI fabricated.
These notes are based primarily on the MIT 15.053 Dynamic Programming notes by Dimitris Bertsimas and John Tsitsiklis. The ideas and examples reflect my own study and understanding. AI was used to improve organization and presentation.
A Framework for Dynamic Programming Reference This note is based primarily on:
Dimitris Bertsimas and John Tsitsiklis, Introduction to Linear Optimization, Chapter 11: Dynamic Programming. MIT 15.053 Tutorial: Dynamic Programming.Read more...