Private RAG: a document-search architecture that keeps data in the boundary

Short answer: Private RAG is not just a model installed locally. It is a flow where documents, permissions and answers stay inside an agreed boundary, with evidence for every data path.

The flow to design

1. OCR and intake — preserve source, language, version and document permissions; make OCR errors reviewable.
2. Chunking and embedding — record model/runtime versions, retention and the index rebuild path.
3. Access-aware retrieval — check identity and permission before returning passages, never after the answer is made.
4. Grounded answer — show citations or source passages and route material decisions to a human reviewer.
5. Update and recovery — test new index/model versions, keep the previous version and define rollback.

Boundary and audit

Separate business content from health telemetry, update packages, backups and support sessions in the Trust Center. Do not put real documents in logs just to debug. Retention and deletion must be reviewable.

What RAG does not solve

Bad OCR, missing metadata or mixed document versions make retrieval incomplete. The model can still hallucinate, so start with an approved corpus, a reproducible evaluation method and a stop path.

For local/cloud context read the decision table and request Solution Architecture when the boundary is complex.