Back to Projects

AskFiles - Multi-Tier RAG with Dual-Tier Cognitive Memory & Evidence Provenance

Resilient Multi-Tier RAG Architecture with Dual-Tier Cognitive Memory and Verifiable Evidence Provenance

AI / RAG ARCHITECTUREBuilt during: AI Systems Engineer / Architect
Tags -
AI/MLFastAPIPythonQdrant CloudNeon PostgreSQLInngestGemini 2.5 FlashQwen 2.5 72BNVIDIA NemotronLlamaIndexReact 18Docker
SourceLive Demo
AskFiles - Multi-Tier RAG with Dual-Tier Cognitive Memory & Evidence Provenance Preview

Project Overview

AskFiles is a production-grade, multi-tenant Retrieval-Augmented Generation (RAG) system with sub-second citation provenance and verifiable document grounding, orchestrated via an event-driven background pipeline (asynchronous Inngest DAGs for deterministic text chunking, document hierarchy parsing, and batched high-dimensional vector embeddings in Qdrant Cloud).

The system features a resilient distributed LLM inference cascade (Google Gemini 2.5 Flash primary → Qwen 2.5 72B Instruct fallback → NVIDIA Nemotron 120B safety net) achieving sub-2s query latency with zero-downtime fault tolerance, coupled with a Sean Chen dual-tier cognitive memory architecture (episodic conversational continuity in PostgreSQL and background semantic interest distillation).

Key Features

  • Sub-Second Citation Anchoring — Binds generated assertions to exact PDF page numbers (page_label) and section headers via regex-derived chunk metadata with interactive floating quote verification cards.
  • Distributed LLM Inference Cascade — Zero-downtime failover cascade (Google Gemini 2.5 Flash → Qwen 2.5 72B Instruct → NVIDIA Nemotron 120B) with exponential backoff and telemetry tracking.
  • Dual-Tier Cognitive Memory — Decouples multi-turn episodic history in serverless Neon PostgreSQL from background Inngest consolidation DAGs that distill user interests into persistent semantic profiles.
  • Event-Driven Background Processing — Asynchronous Inngest DAGs for deterministic text chunking, document hierarchy parsing, and batched high-dimensional vector embeddings in Qdrant Cloud.
  • Strict Multi-Tenant Isolation & TTL Reaper — Payload-level vector filtering via Qdrant MatchAny and automated background TTL reaper DAG that cleans up expired guest vectors and records after 2 hours.

Technical Highlights

  • Granular Evidence Provenance — Regex-derived chunk metadata anchors citations to page labels, powering interactive floating quote cards and enforcing zero-hallucination out-of-domain boundaries.
  • Decoupled Memory Architecture — Maintains longitudinal personalization across multi-turn sessions in Neon PostgreSQL without recurrent context window bloat.
  • Self-Healing Query Completion — Eliminates single-provider fragility and HTTP 429 rate limits via cross-model family routing across heterogeneous providers via OpenRouter.
  • Automated Vector Lifecycle Management — Enforces strict payload-level isolation and background reaper DAGs to prevent data leakage and storage bloat.

Challenges & Solutions

Challenge: Hallucination Mitigation & Grounding Verification (Granular Evidence Provenance): Solved unverifiable model claims and phantom citations in document Q&A.
Solution: Established a sub-second citation anchoring pipeline that binds generated assertions to exact PDF page numbers (page_label) and section headers via regex-derived chunk metadata.
Result: Interactive floating quote verification cards with verbatim context excerpts and enforced zero-hallucination out-of-domain boundaries.
Challenge: Context Window Saturation vs. Longitudinal Personalization (Dual-Tier Cognitive Memory): Overcame context window bloat and multi-session conversational amnesia.
Solution: Architected a decoupled memory system: persisting multi-turn episodic history in serverless Neon PostgreSQL while triggering asynchronous Inngest consolidation DAGs to distill user interests into persistent semantic profiles.
Result: Persistent personalization across sessions without recurring context inference overhead.
Challenge: High-Availability Inference & Single-Provider Fragility (Distributed Model Cascade): Eliminated single-point-of-failure vulnerabilities, provider rate limits (HTTP 429), and latency spikes.
Solution: Implemented an automated cascading fallback mechanism with exponential backoff and telemetry tracking across heterogeneous model families (Google Gemini 2.5 Flash → Qwen 2.5 72B Instruct → NVIDIA Nemotron 120B).
Result: Self-healing query completion under constrained resource limits achieving sub-2s query latency.
Challenge: Cross-Tenant Vector Contamination & Storage Bloat (Strict Multi-Tenant Isolation & TTL Reaper): Prevented data leakage across guest and authenticated user tiers.
Solution: Enforced strict payload-level vector filtering (owner_id and selective document grounding via Qdrant MatchAny), paired with an automated background TTL reaper DAG.
Result: Guaranteed multi-tenant isolation and automatic cleanup of expired guest vectors, storage artifacts, and database records upon 2-hour expiration.

Tech Stack Breakdown

frontend

React 18, Vite, Tailwind CSS, TypeScript

backend

Python, FastAPI, LlamaIndex, Inngest (Event-Driven DAGs)

database

Qdrant Cloud (Vector DB), Neon PostgreSQL

infrastructure

Docker, OpenRouter, Vercel

tools

Google Gemini 2.5 Flash, Qwen 2.5 72B, NVIDIA Nemotron 120B, JWT, Bcrypt