Back to work HYBRID RETRIEVAL AND EVALUATION

AI-search-system

A production-style search system for technical documentation that separates retrieval quality from answer generation on purpose: three retrieval modes, keyword, semantic, and hybrid, feed a two-stage reranker, and every result surfaces explainable ranking signals instead of a generated answer. Deliberately not a chatbot: the question this project answers is whether the retrieval and ranking are correct, not whether a model can write a plausible-sounding response on top of them.

View repo

Architecture & Trade-offs

What was built, and what was deliberately chosen against.

pipeline.txt

Offline (ingestion)
  documentation source
        │
        ▼
  load, normalize, chunk by headings
        │
        ▼
  OpenAI embeddings
        │
        ▼
  Typesense index (keyword + vector)

Online (query)
  user query
        │
        ▼
  keyword search   +   vector search
        │
        ▼
  Reciprocal Rank Fusion
        │
        ▼
  hosted reranker (top-N)
        │
        ▼
  ranked results + explainability signals
    keyword rank · vector rank · fusion score · rerank score

Hybrid retrieval by default, not as an add-on

Keyword and vector search run in parallel and are combined with Reciprocal Rank Fusion rather than picking one mode and bolting the other on later. Measured with Recall@5 and nDCG@5 on hand-labeled queries, hybrid consistently outperforms semantic alone and keyword alone, which is the actual argument for defaulting to it, not an assumption.

Two-stage ranking: fast retrieval, then a slower reranker

Fusion produces a candidate set quickly; a hosted reranker then reorders the top results with a higher-quality, more expensive model. Splitting retrieval and ranking into two stages keeps the fast path fast and only spends the reranker's cost on candidates that already cleared the first bar.

Every result explains its own rank

Each returned document carries its keyword rank, vector rank, fusion score, and rerank score, not just a final position. That makes a ranking decision auditable after the fact, closer to how a search engineer would debug relevance than a black-box similarity score.

No answer synthesis, on purpose

The system stops at ranked results with relevance signals; there's no RAG-style answer generation layered on top. That's a scope decision, not a missing feature: the project is built to prove retrieval and ranking are correct, and mixing in generation would make it harder to tell which part of the system is actually being evaluated.

Stack

TypeScript, Node.js, SvelteKit, Typesense, OpenAI embeddings, Cohere rerank, Docker

Status

Fully functional end to end: ingestion, search API, UI, and evaluation framework all working. Next planned phase is agentic query routing, not yet built.

Public repo. View the repo on GitHub.

Back to work