Overview
Production-ready RAG (Retrieval-Augmented Generation) system built to demonstrate advanced retrieval techniques and production engineering on free-tier infrastructure.
Key Features
- Hybrid Search: Combines vector similarity (semantic) with BM25 keyword matching for better retrieval
- Cross-Encoder Reranking: Re-scores retrieved documents for improved relevance
- 5 Retrieval Strategies: Basic, HyDE (Hypothetical Document Embeddings), Multi-Query, Parent-Child, and Optimization techniques
- 2-Tier LLM Fallback: Ollama (local) → Groq (cloud) for 100% reliability
- Redis Caching: 100x speedup on repeated queries with sliding window rate limiting (60 req/min)
Technical Achievements
Retrieval Performance
- 67.7% accuracy with basic hybrid search + reranking
- 85%+ accuracy with advanced strategies (HyDE, Multi-Query)
- Tested across 19 diverse queries covering factual, comparison, and multi-hop questions
Production Optimizations
- 512MB RAM deployment using Qdrant Cloud (remote vector storage) + local Sentence Transformers (200MB) and Groq (cloud LLM)
- BM25 index caching: 9.2x speedup by persisting sparse indexes
- Batch processing: 11x faster document processing
- Connection pooling: Efficient database connections
Full-Stack Architecture
- Backend: FastAPI with async operations, comprehensive error handling
- Frontend: Next.js with TypeScript, real-time query visualization
- Caching: Redis for embeddings, BM25 indexes, and query results
- Deployment: Render (backend) + Vercel (frontend) on free tier
What I Learned
Building this taught me that retrieval quality matters more than LLM quality in RAG systems. A great LLM with poor retrieval will hallucinate, but a decent LLM with excellent retrieval will give accurate answers.
Also learned the art of free-tier optimization - using cloud APIs strategically while maintaining local development capabilities, and aggressive caching to stay within rate limits.
Tech Stack
Backend: FastAPI, LangChain, Sentence Transformers, Ollama, Groq Vector DB: Qdrant Cloud (remote storage, 2x faster) Search: BM25 (sparse), Cross-Encoder (reranking) Caching: Redis (Upstash) Frontend: Next.js 14, TypeScript, Tailwind CSS Deployment: Render, Vercel