Every company my size has the same quiet problem. The answer to your question exists. It is in a slide deck from eight months ago, or a policy doc nobody re-shares, or in one person’s head, and that person is on vacation. So you ask around, you guess, and sometimes you act on a version of the truth that got replaced in March.
I am building the fix for Hybridge: an internal search where anyone signs in with their company account, types a question in plain language, and gets a real answer with the sources attached. We call it Ask Hybridge. People who are allowed to add documents drop them into a portal, and those documents flow through a careful pipeline before they ever become searchable.
The easy version of this is a weekend project now. You chunk some documents, embed them, stuff the top matches into a prompt, and call it RAG. I did not want the easy version. We work in healthcare, which means patient data, HR records, and trade secrets all live in the same building. In that setting a confident wrong answer is not a cute demo bug. It is a liability. So the whole design is organized around one idea: the system should be trustworthy first, and clever second.
The one rule everything serves
Here is the principle I keep coming back to. The system should answer from what it actually found, and when it did not find enough, it should say so. No filling the gap with a plausible guess. An honest “I don’t know, here is the closest thing” is a feature, not a failure.
That sounds obvious. It is surprisingly hard, and almost every shortcut in RAG quietly breaks it.
Why I am not building a “basic” RAG
A basic RAG fails in three ways that matter in a real company, so I planned for all three from the start.
It fails on exact terms. Pure vector search is great at meaning and bad at specifics. Search a part number, a policy code, or a person’s name and a semantic-only index will hand you something that “feels related” instead of the exact match. The fix is to run keyword search and vector search together.
It fails on conflicts. Two documents disagree. One is the current SOP, one is a draft from last year that nobody deleted. A naive system picks whichever happens to rank higher and presents it with total confidence. That is the dangerous one.
It fails on permissions. The marketing team should never retrieve a document restricted to the clinical group, even by accident, even as “context” the model quietly reads. Access has to be enforced before retrieval, not bolted on after.
How a question gets answered (the read path)
When someone asks a question, the answer travels a fixed path. I kept the common case fast and simple, and put the expensive, clever steps behind a gate so they only run when a question actually needs them.
- Rewrite the question only if it is vague or part of a back-and-forth chat. A well-formed question skips this.
- Filter by permission first, as a database condition. Documents you are not allowed to see never enter the search.
- Retrieve two ways at once: keyword search (BM25) for exact terms, vector search for meaning.
- Merge the two result lists with Reciprocal Rank Fusion.
- Rerank a small top slice with a dedicated ranking model, for precision without paying the latency on everything.
- Resolve conflicts deterministically (see below).
- Answer, citing the exact passages. If the evidence is thin, abstain.
Most questions are simple lookups, and they take the fast path and finish. Only the hard ones, the low-confidence or multi-step questions, trigger a corrective loop: re-plan, retrieve again, fall back to a web search if the answer genuinely is not in our knowledge, check the draft answer against the evidence, and only then respond.
I want to be clear about why that loop is gated rather than always-on. An agent that re-plans on every single query feels smart and behaves badly. It is slower, it costs more, and it pulls in noisier context that can make a simple answer worse. So the expensive behavior only switches on when a confidence check says the simple path did not work. The default is fast and boring. The cleverness is held in reserve.
When sources disagree, the model does not get to pick
This is the part I am most careful about. When two retrieved passages make conflicting claims, the model is never allowed to just choose one. A small piece of plain code ranks the evidence first, by rules anyone can read.
- Group the evidence by what each piece actually claims.
- Drop anything marked superseded or deprecated. A replaced document cannot win.
- Higher authority wins (an official policy outranks a personal note).
- If authority ties, the most recent effective date wins.
- If it still cannot be resolved, declare the disagreement: show both answers with their author, date, and source, and say plainly that the sources disagree.
The model’s job is only to notice that two passages genuinely contradict each other and to write the answer in a way that honors that ranking, or to surface the disagreement honestly. The decision about which source is more authoritative is made by rules, not vibes. That is the difference between a search you can run a company on and a search that sounds confident.
What happens when a document comes in (the write path)
Retrieval is only as good as what got indexed, so the ingestion side gets the same care. A document does not become searchable just because someone uploaded it.
Two details there do a lot of quiet work. The document card stays small no matter how long the document is, so the system can route and display a document without re-reading the whole thing. And every chunk is stamped with the version of the embedding model and chunking strategy that produced it, so when I improve either one later, I can re-index safely instead of guessing what is stale.
The part most people skip: measuring it
Here is the question that started the whole project for me. How do I know the answer is good before I trust it? Most RAG demos never answer that. They look impressive in a screenshot and nobody checks the hit rate.
So I am building an evaluation harness alongside the system, not after it. It measures retrieval with the standard metrics (did the right document show up, and how high), and it measures the generation for faithfulness, whether every claim in the answer is actually backed by a cited source. It even runs a managed search product in parallel as a yardstick, so I can compare what I built against a strong baseline and know whether my extra machinery is earning its keep.
The harness is the referee. It is what decides when a fancier idea actually goes into production and when it stays a nice diagram. That rule is how I keep an ambitious system from turning into an over-engineered one.
The stack, in one box
I deliberately started on Cloud SQL with pgvector instead of a heavier specialized vector database. It meets every need we have at our current size of hundreds to low thousands of documents, at roughly half the cost, and there is a clean upgrade path the day the evaluation harness tells me I have outgrown it. That is the theme again: pick the simple thing, and let the measurements, not the hype, tell me when to add complexity.
Where it stands, with no rounding up
I am building this the way you would teach it, one stage at a time, because each stage is also a chunk of RAG I wanted to learn properly by hand.
| Stage | What it owns | Status |
|---|---|---|
| Foundation and auth | GCP setup, domain-locked sign-in, roles and groups, secrets, CI/CD, infrastructure as code | Shipped |
| Ingestion | upload, dedup, parse, chunk, metadata, embed, index | Designed, in build |
| Query serving | rewrite, filter, hybrid retrieval, rerank, conflict resolution, grounded answer | Designed |
| Evaluation | retrieval and faithfulness metrics, conflict tests, managed baseline | Grows alongside the two above |
| Search and Dropoff front ends | the chat experience and the upload portal | Designed |
What is real today: the foundation and authentication layer is shipped and running. Sign-in is locked to the company domain, roles and groups are enforced, and the whole thing stands up from a clean checkout through Terraform and a tested CI/CD pipeline. The ingestion pipeline is fully specified and in build. The rest is designed in detail and queued behind it.
That is the honest status. I would rather show you a trustworthy foundation and a clear plan than a flashy demo that falls over the first time two documents disagree. The whole point of this system is to not do that.
