Every company my size has the same quiet problem. The answer to your question exists. It is in a slide deck from eight months ago, or a policy doc nobody re-shares, or in one person’s head, and that person is on vacation. So you ask around, you guess, and sometimes you act on a version of the truth that got replaced in March.
I am building the fix for Hybridge: an internal search where anyone signs in with their company account, types a question in plain language, and gets a real answer with the sources attached. We call it Ask Hybridge. People who are allowed to add documents drop them into a portal, and those documents flow through a careful pipeline before they ever become searchable.
The easy version of this is a weekend project now. You chunk some documents, embed them, stuff the top matches into a prompt, and call it RAG. I did not want the easy version. We work in healthcare, which means patient data, HR records, and trade secrets all live in the same building. In that setting a confident wrong answer is not a cute demo bug. It is a liability. So the whole design is organized around one idea: the system should be trustworthy first, and clever second.
The one rule everything serves
Here is the principle I keep coming back to. The system should answer from what it actually found, and when it did not find enough, it should say so. No filling the gap with a plausible guess. An honest “I don’t know, here is the closest thing” is a feature, not a failure.
That sounds obvious. It is surprisingly hard, and almost every shortcut in RAG quietly breaks it.
Why I am not building a “basic” RAG
A basic RAG fails in three ways that matter in a real company, so I planned for all three from the start.
It fails on exact terms. Pure vector search is great at meaning and bad at specifics. Search a part number, a policy code, or a person’s name and a semantic-only index will hand you something that “feels related” instead of the exact match. The fix is to run keyword search and vector search together.
It fails on conflicts. Two documents disagree. One is the current SOP, one is a draft from last year that nobody deleted. A naive system picks whichever happens to rank higher and presents it with total confidence. That is the dangerous one.
It fails on permissions. The marketing team should never retrieve a document restricted to the clinical group, even by accident, even as “context” the model quietly reads. Access has to be enforced before retrieval, not bolted on after.
How a question gets answered (the read path)
When someone asks a question, the answer travels a fixed path. I kept the common case fast and simple, and put the expensive, clever steps behind a gate so they only run when a question actually needs them.
- Rewrite the question only if it is vague or part of a back-and-forth chat. A well-formed question skips this.
- Filter by permission first, as a database condition. Documents you are not allowed to see never enter the search.
- Retrieve two ways at once: keyword search (BM25) for exact terms, vector search for meaning.
- Merge the two result lists with Reciprocal Rank Fusion.
- Rerank a small top slice with a dedicated ranking model, for precision without paying the latency on everything.
- Resolve conflicts deterministically (see below).
- Answer, citing the exact passages. If the evidence is thin, abstain.
Most questions are simple lookups, and they take the fast path and finish. Only the hard ones, the low-confidence or multi-step questions, trigger a corrective loop: re-plan, retrieve again, fall back to a web search if the answer genuinely is not in our knowledge, check the draft answer against the evidence, and only then respond.
I want to be clear about why that loop is gated rather than always-on. An agent that re-plans on every single query feels smart and behaves badly. It is slower, it costs more, and it pulls in noisier context that can make a simple answer worse. So the expensive behavior only switches on when a confidence check says the simple path did not work. The default is fast and boring. The cleverness is held in reserve.
When sources disagree, the model does not get to pick
This is the part I am most careful about. When two retrieved passages make conflicting claims, the model is never allowed to just choose one. A small piece of plain code ranks the evidence first, by rules anyone can read.
- Group the evidence by what each piece actually claims.
- Drop anything marked superseded or deprecated. A replaced document cannot win.
- Higher authority wins (an official policy outranks a personal note).
- If authority ties, the most recent effective date wins.
- If it still cannot be resolved, declare the disagreement: show both answers with their author, date, and source, and say plainly that the sources disagree.
The model’s job is only to notice that two passages genuinely contradict each other and to write the answer in a way that honors that ranking, or to surface the disagreement honestly. The decision about which source is more authoritative is made by rules, not vibes. That is the difference between a search you can run a company on and a search that sounds confident.
What happens when a document comes in (the write path)
Retrieval is only as good as what got indexed, so the ingestion side gets the same care. A document does not become searchable just because someone uploaded it.
Two details there do a lot of quiet work. The document card stays small no matter how long the document is, so the system can route and display a document without re-reading the whole thing. And every chunk is stamped with the version of the embedding model and chunking strategy that produced it, so when I improve either one later, I can re-index safely instead of guessing what is stale.
The part most people skip: measuring it
Here is the question that started the whole project for me. How do I know the answer is good before I trust it? Most RAG demos never answer that. They look impressive in a screenshot and nobody checks the hit rate.
So I am building an evaluation harness alongside the system, not after it. It measures retrieval with the standard metrics (did the right document show up, and how high), and it measures the generation for faithfulness, whether every claim in the answer is actually backed by a cited source. It even runs a managed search product in parallel as a yardstick, so I can compare what I built against a strong baseline and know whether my extra machinery is earning its keep.
The first slice of that referee now runs inside the test suite. A seeded corpus of question and answer cases is scored on every build, and the build fails if recall at 8 drops below 0.8 or mean reciprocal rank drops below 0.35. Today it scores 1.0 recall at 8 and 0.42 MRR, and dedicated cases prove the abstain path and the conflict ladder actually fire. Small corpus, honest floors, checked automatically. The broader continuous harness, with faithfulness scoring and the managed baseline comparison, is still ahead.
The harness is the referee. It is what decides when a fancier idea actually goes into production and when it stays a nice diagram. That rule is how I keep an ambitious system from turning into an over-engineered one.
The stack, in one box
I deliberately started on Cloud SQL with pgvector instead of a heavier specialized vector database. It meets every need we have at our current size of hundreds to low thousands of documents, at roughly half the cost, and there is a clean upgrade path the day the evaluation harness tells me I have outgrown it. That is the theme again: pick the simple thing, and let the measurements, not the hype, tell me when to add complexity.
Where it stands, with no rounding up
I am building this the way you would teach it, one stage at a time, because each stage is also a chunk of RAG I wanted to learn properly by hand.
| Stage | What it owns | Status |
|---|---|---|
| Foundation and auth | GCP setup, domain-locked sign-in, roles and groups, secrets, CI/CD, infrastructure as code | Shipped |
| Ingestion | upload, dedup, parse, chunk, metadata, embed, index | Shipped |
| Query serving | filter, hybrid retrieval, rerank, grounded streaming answer, abstain guardrail | Shipped, live in production |
| Evaluation | retrieval and faithfulness metrics, conflict tests, managed baseline | First version runs in CI with enforced floors; the continuous harness is in build |
| Search and Dropoff front ends | the chat experience and the upload portal | Shipped |
What is real today, as of July 2026: the whole loop is live in production. You sign in with your company account, ask a question, and watch a cited answer stream back from documents you are allowed to see, with conversation history kept. Clicking a citation opens the source document in a side pane, scrolled to the exact highlighted passage it came from. The abstain rule is not a prompt suggestion anymore, it is enforced in code: if retrieval comes back empty, the system short-circuits and says it does not have the answer, and the model is never even called. Authorized people add documents through the Dropoff portal, scoping each one at upload time with its visibility, tags, document class, and effective date, exactly the metadata the conflict ladder needs later, and then watch it move through the pipeline. Tabs and uploads are gated by role, and the API degrades gracefully instead of hallucinating when the AI backend is unavailable.
Behind it sits the discipline column: 122 automated tests across backend and frontend, 96 and 26, covering permissions in both directions, streaming, version chains, citation math, and the UI flows. Every subproject was built with tests first and then reviewed twice, once per task and once across the whole branch by a stronger model prompted to attack it. Those reviews caught a permission sentinel bug, an authority scoring bug, and a race that would have wiped the first streamed answer. The retrieval numbers above come from the seeded evaluation that runs in CI; the fuller harness with faithfulness scoring is the part still being built, and the numbers it owns stay unpublished until it says so. The referee gets built before the victory lap.
If you want the stage by stage tour with the clickable pipeline, the demo guide covers everything this post compresses, and the slides are the ten minute version.
One war story from shipping it, because it repeats a pattern I keep seeing: the bug that blocked every single login for a day was not the retrieval stack or the model. It was an auth SDK that needed initializing before its first token check. The AI is rarely the thing that breaks. The plumbing is.
