Case File · 001 · DocRAG-Legal V2  ·  Deployed · us-central1

The contract risk
intelligence system
that argues with itself.

Not a ChatPDF. Not a legal chatbot. A system that challenges its own answers at runtime and exposes the challenge to the user — so a junior associate in Dhaka has the same evidentiary footing as a senior partner in a global firm.

>0.85
Faithfulness threshold
100%
Citation accuracy required
SHA-256
Session provenance chaining
Adversarial self-auditing Faithfulness scoring SHA-256 provenance chaining Clause risk heatmapping Statute-backed redlines Gemini-as-judge Adversarial self-auditing Faithfulness scoring SHA-256 provenance chaining Clause risk heatmapping Statute-backed redlines Gemini-as-judge
§ 01 — Information Asymmetry

Bangladesh's commercial environment runs on paper, precedent,
and who you know.

Statutes are published as degraded PDFs on government portals. Contracts are drafted from inherited templates nobody re-reads. Regulatory circulars — from Bangladesh Bank, the NBR, the RJSC — arrive weekly and are interpreted by the loudest voice in the room, not the most accurate reading of the text.

A small garment exporter in Gazipur negotiating an LC with a foreign buyer has no way to audit the indemnity clause against prevailing case law. A startup founder signing a seed term sheet has no way to flag that the liquidation preference is 3× market standard.

DocRAG-Legal ingests the statute, reads the clause, cites the source, and — critically — argues with itself in public before letting the answer reach the user.

170M
People in jurisdiction
3
Regulatory languages
$0
Cloud spend to deploy

"Not just citation-verified answers — cryptographically chained sessions. Every answer challenges itself. Every session is tamper-evident. Every risk assessment cites the exact statute it violates."

§ 02 — Design Philosophy

Not a chatbot.
An evidentiary instrument.

✗  Standard RAG System
✓  DocRAG-Legal
Retrieves chunks, synthesises, returns answer.
Retrieves, synthesises — then a second model attacks the answer.
Hallucinations are hidden from the user.
The adversarial challenge is shown to the user, always.
No way to verify synthesis faithfulness.
Faithfulness score auto-flags anything under 0.85.
Citations, if present, are decorative.
Every citation is verified against the source chunk.
A session is a list of questions with no audit trail.
Each interaction is SHA-256 chained to the previous one.
Unacceptable in legal or compliance contexts.
Designed from day one as a regulated-domain instrument.
§ 03 — The Stack

One engine.
Swappable corpus.
That is the
business model.

The processing machinery is identical across legal, clinical, cybersecurity, and financial contexts. Only the corpus changes. One engine, many data stores.

AGNOSTIC PROCESSING MACHINERY
────────────────────────────────────────
Vertex RAG → Discovery Engine Rerank
              → Gemini Pro Synthesis
                      ↓
          Gemini Flash Adversarial Judge
                      ↓
      SHA-256 Provenance → Modal Ledger
────────────────────────────────────────
            ↑ swappable corpus

MUTABLE DOMAIN CONTEXTS
────────────────────────────────────────
Statutes · IT Security · Clinical GCP
Tax Codes · Banking Circulars
Import/Export Tariffs · Sharia Law
Five endpoints
POST /query

RAG retrieval → Gemini Pro synthesis → Flash adversarial judge, inline on every single request. The judge result is visible to the user.

POST /risk

Clause heatmap — RED / YELLOW / GREEN — with statute citations for every flag. Not decorative. Legally grounded.

POST /redline

Clause diff + statute-backed rewrite + risk delta calculation. Before and after, with full citation chain.

POST /ingest

PDF upload → Cloud Storage → Vertex AI RAG async import with job tracking. Any domain corpus in minutes.

GET /eval

Audit dashboard: faithfulness verdicts, adversarial challenges, session registry, provenance hash ledger.

§ 04 — Validated Output

Real responses.
Three domains.
Same engine.

The same engine served cybersecurity SLAs, clinical trial protocols, and constitutional law without a code change. Each domain is a corpus upload and a preamble adjustment.

Case · 01 · IT Security / Procurement SLA · POST /risk
Risk Level · Yellow
"...Vendor will use commercially reasonable efforts to notify Customer within thirty (30) business days. Vendor's total aggregate liability for any security-related claims is strictly limited to $5,000."
⚠ Notification window

30-business-day notification is commercially unacceptable for a security incident.

⚠ Obligation dilution

"Commercially reasonable efforts" weakens what should be an absolute obligation.

⚠ Liability cap

$5,000 aggregate cap is nominal against real breach exposure and recovery costs.

Statutes cited: Contract Act, 1872 · ICT Act, 2006
Case · 02 · Clinical Trial Protocol · POST /redline · Risk delta: HIGH → LOW
Rewrite Complete
Before
"Any adverse events will be recorded within 14 days. Serious adverse events that do not result in death do not require immediate expedited reporting to the IRB or sponsor."
After
"Any adverse events will be recorded promptly, no later than 72 hours of discovery. All Serious Adverse Events, whether or not resulting in death, must be reported immediately — in no event later than 24 hours of awareness — to the sponsor and IRB."
Aligned with: ICH-GCP international guidelines · Confidence: 0.95
Case · 03 · Constitutional Law · POST /query
PASS · Faithfulness 1.0
Query

"How is the President of the Republic elected and what is the term of office?"

System answer: "Insufficient context to answer."

Adversarial challenge: The retrieved context contains only metadata for the Bangladesh Labour Act 2006, which does not address presidential election procedures. Speculation would constitute a faithfulness violation.

The system correctly refused to answer when the corpus didn't cover the question. A faithfulness score of 1.0 on a refusal is the safety behavior working as designed. In regulated domains, "I don't know" is the most valuable answer.

provenance_hash: dd1817e956ddf46c
§ 05 — Provenance Architecture

Ten lines of SHA-256
turn a chat log into
a tamper-evident
audit trail.

Any tampering with a single past answer breaks every subsequent hash in the chain. The session becomes an audit trail — exactly the property regulated industries need for discovery and compliance review.

Q₁ SHA256("genesis" + Q + A + context) → a7f3c2e8d1b4...
Q₂ SHA256("a7f3..." + Q + A + context) → 9c2b7a4f1e6d...
Q₃ SHA256("9c2b..." + Q + A + context) → dd1817e956df...

Genesis hash is the literal string "genesis". Every session starts clean and builds a verifiable chain from the first interaction.

§ 06 — GCP Services

Every Google Cloud service
this project touches.

Vertex AI · RAG Engine

Corpus ingestion, managed vector stores, text-embedding-004, async GCS imports. Chunk sizing: 1024 tokens / 200 overlap for statutes, 512 / 100 for contracts.

Vertex AI · Gemini

Gemini 1.5 Pro (T=0.1) for synthesis — forbids speculation, requires [N] citation format. Flash (T=0.0, structured JSON) for the adversarial judge on every request.

Vertex AI · Ranking API

semantic-ranker-512@latest re-scores top-20 candidates to top-5 before synthesis. The single biggest quality lever — faithfulness delta vs. unreranked synthesis is roughly +0.15.

Discovery Engine

Enterprise-tier Search API, unstructured PDF parsing, advanced OCR, extractive segments. Provisioned under $1,000 Gen AI App Builder credits.

Cloud Run

Source-based FastAPI deploy, autoscaling 0–5 instances, OIDC auth, enterprise security headers (HSTS, X-Frame, XSS), 1Gi memory.

Modal Persistence

Five append-only NDJSON volumes (eval logs, risk logs, redline logs, doc registry, ingest jobs) and two dicts (session state + eval cache). Zero idle cost.

Firebase Auth

JWT verification via ADC. Custom role claims: admin (ingest, batch eval), analyst (risk, redline, query), viewer (query, eval read). Zero JSON key files in container.

Cloud Scheduler

6-hourly POST /eval/batch via OIDC-authenticated HTTP job. Continuous evaluation loop — not a test suite run once and forgotten.

Secret Manager

All credentials. Zero secrets in source code, environment files, or container images.

§ 07 — What the Docs Don't Tell You

Eight production blockers.
Eight precise interventions.

01
Serverless beats managed, always

The RAG Engine's default managed-Spanner path hit capacity limits we couldn't clear. Switching to serverless via a REST PATCH we wrote ourselves unlocked the whole pipeline. When Google ships two paths, the newer one is usually less congested.

02
Reranking is not optional

Raw embeddings retrieve plausible chunks. The semantic reranker retrieves the right chunks. On regulatory text, the faithfulness delta between reranked and unreranked synthesis is roughly +0.15 — the difference between PASS and REVIEW verdicts.

03
The adversarial judge must be a runtime feature

A test-suite judge runs in CI and is forgotten. An inline judge the user sees — with severity icons and expandable challenge text — transforms the product from answer machine to evidentiary instrument. The UX is the architecture.

04
Provenance chaining is cheap and priceless

Ten lines of SHA-256 code turn a chat log into a tamper-evident audit trail. Regulated industries cannot deploy without this. It is the difference between a demo and a product.

05
ADC is a design pattern, not just auth

Every time we used Application Default Credentials instead of a JSON key file, the deployment got simpler, safer, and more portable. Firebase Admin, Discovery Engine, gcloud — all unified under one identity model.

06
Async + blocking SDKs need run_in_executor

Modal's .remote() is synchronous. FastAPI is async. Calling one from the other on the event loop destroys latency for every other request. Wrapping in asyncio.run_in_executor is non-negotiable at production traffic.

07
The refusal is a feature

The constitutional law demo answered "Insufficient context" with a perfect faithfulness score of 1.0. The system correctly refused to speculate. In regulated domains, a well-reasoned no is more valuable than a plausible-sounding wrong answer.

08
A domain-agnostic core compounds

The same engine served cybersecurity SLAs, clinical trial protocols, and constitutional law without a code change. Each new vertical is a corpus upload and a preamble tweak. This is the business model — not a product per domain.