Blueprint: Privacy-First RAG

Sanitizing Retrieved Context from Knowledge Bases

Retrieval-Augmented Generation (RAG) often pulls data from internal knowledge bases that may contain stale PII (e.g., legacy customer data, emails). If this context is injected directly into the LLM prompt, it creates a privacy leak.

This blueprint shows how the Gateway masks retrieved context in the augmentation step.

Architecture

1. Master Policy Definition

{
  "name": "rag-privacy-policy",
  "priority": 9000,
  "rules": [
    {
      "id": "allow-rag-agent",
      "effect": "allow",
      "principals": ["agent:rag-bot"],
      "actions": ["llm.generate"],
      "resources": ["*"]
    }
  ],
  "content_policy": {
    "enable_pii_detection": true,
    "pii_action": "mask"
  }
}

2. Implementation

LlamaIndex Prototype

import os
from openai import OpenAI

# Control Zero Gateway handles the masking automatically
# when the prompt is sent to the LLM.
client = OpenAI(
    api_key="ignored",
    base_url="http://cz-gateway:8001/v1",
    default_headers={"X-ControlZero-Agent-ID": "rag-bot"}
)

def answer_with_rag(query: str, retrieved_docs: list):
    # Context retrieved from Chroma/Pinecone may contain SSNs or Emails
    context_text = "\n".join(retrieved_docs)

    prompt = f"Context: {context_text}\n\nQuestion: {query}"

    try:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}]
        )
        return response.choices[0].message.content
    except Exception as e:
        return f"Policy Intervention: {e}"

# Scenario: Vector DB returns a document with an SSN
context = ["The client Jane Doe has SSN 123-45-6789 and is located in NY."]
print(answer_with_rag("What is Jane Doe's location?", context))

# Result: The LLM only sees the redacted context, protecting the SSN
# while still being able to answer the question about the location.

3. Validation Checklist

• Augmentation Check: Verify that PII within the Context: block of the prompt is masked.
• Utility preserved: Ensure the LLM can still answer non-sensitive parts of the query (e.g., location) even after redaction.
• Audit Review: Verify that ClickHouse logs the redacted version of the prompt. 埋