llm-pii-poc/docs/architecture.md

# PII Redaction POC — Solution Architecture

## Purpose

This document describes the architectural design of the **PII Redaction POC**, a .NET proof-of-concept that intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits **only sanitized text** across the LLM trust boundary. The solution is structured for enterprise adoption: clear layer separation, interface-driven composition, dependency injection, and swappable infrastructure adapters (ONNX NER, `Microsoft.Extensions.AI` chat clients).

The POC validates a compliance-oriented pattern suitable for financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.

---

## Canonical Example

The console application ships with a **sample catalog** (11 prompts). The canonical demo is sample `FullFinancialWithCustomer`. The table below shows the exact strings produced by the production pipeline when the ONNX NER model is loaded (run `scripts/download-ner-model.ps1` first).

| Stage | Value |
|-------|-------|
| **Input** | `Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.` |
| **Sanitized Output** | `Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.` |
| **Mock LLM Response** | `[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.` |

Detected entities for this prompt:

| Type | Value | Detection Source |
|------|-------|------------------|
| PERSON | Ravi Kumar | Ner |
| EMAIL | ravi.kumar@gmail.com | Regex |
| PHONE | 9876543210 | Regex |
| LOAN_NUMBER | LN-456789 | Domain |
| PAN | ABCDE1234F | Regex |

The internal placeholder map (`<PERSON_1>` → `Ravi Kumar`, etc.) is retained in-process and is **not** included in the outbound LLM request.

---

## Console Sample Catalog

Running `dotnet run --project src/PiiRedaction.ConsoleApp` executes all samples sequentially. Use `--list`, `--sample N`, or `--name SampleName` to filter.

### NER / person-name samples

These prompts exercise `OnnxNerPiiDetector` and `OnnxNerModelRunner`. Person names require the ONNX model (`models/ner-model.onnx` plus `vocab.txt` and `ner-labels.txt`). Without the model, person spans are not detected.

| Sample | Input (excerpt) | Detected person | Sanitized (excerpt) |
|--------|-----------------|-----------------|---------------------|
| **CustomerNameOnly** | Customer Anita Sharma reported unauthorized… | Anita Sharma | Customer `<PERSON_1>` reported unauthorized… |
| **MrTitlePerson** | Mr. John Smith called about a duplicate debit… | John Smith | `<PERSON_1>` called about a duplicate debit… |
| **MrsTitlePerson** | Mrs. Lakshmi Reddy requested a callback regarding LN-112233. | Lakshmi Reddy | `<PERSON_1>` requested a callback regarding `<LOAN_NUMBER_1>`. |
| **DrTitlePerson** | Dr. Jane Doe escalated a complaint… | Jane Doe | `<PERSON_1>` escalated a complaint… |
| **TwoCustomersInOnePrompt** | Customer Ravi Kumar and Customer Priya Nair… | Ravi Kumar, Priya Nair | Customer `<PERSON_1>` and Customer `<PERSON_2>`… |
| **PersonWithDomainIds** | Customer Meera Iyer holds CID-7070… | Meera Iyer | Customer `<PERSON_1>` holds `<CUSTOMER_ID_1>`… |
| **PersonWithEmailNoPhone** | Customer Arjun Mehta wrote from arjun.mehta@company.in… | Arjun Mehta | Customer `<PERSON_1>` wrote from `<EMAIL_1>`… |

### Other sample categories

| Category | Sample | Purpose |
|----------|--------|---------|
| NER + Regex + Domain | FullFinancialWithCustomer | End-to-end financial prompt (canonical) |
| Regex only | AllRegexTypes | Email, phone, PAN, Aadhaar, credit card |
| Domain only | AllDomainIds | Loan number, customer ID, account number |
| Negative | NoPiiCleanTicket | Passthrough with no detected PII |

Sample definitions live in [`SamplePromptCatalog.cs`](../src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs).

---

## High-Level Data Flow

The diagram below traces the canonical example from console input through Core sanitization to the Infrastructure LLM adapter. Data labels reflect the canonical strings at each stage.

```mermaid
flowchart TB
    subgraph consoleApp [PiiRedaction.ConsoleApp]
        program["Program.cs"]
        di["ServiceRegistration"]
    end

    subgraph core [PiiRedaction.Core]
        sanitizer["PromptSanitizer"]
        composite["CompositePiiDetector"]
        regexDet["RegexPiiDetector"]
        domainDet["DomainRulePiiDetector"]
        onnxDet["OnnxNerPiiDetector"]
        redactor["PlaceholderPiiRedactor"]
    end

    subgraph infra [PiiRedaction.Infrastructure]
        onnxRunner["OnnxNerModelRunner"]
        mockLlm["MockLlmPromptService"]
        mockChat["MockChatClient"]
    end

    rawPrompt["Raw prompt with PII"]
    sanitizedPrompt["Sanitized prompt with placeholders"]
    llmResponse["Mock LLM acknowledgment"]

    program -->|"Customer Ravi Kumar ... PAN ABCDE1234F"| sanitizer
    sanitizer --> composite
    composite --> domainDet
    composite --> regexDet
    composite --> onnxDet
    onnxDet --> onnxRunner
    sanitizer --> redactor
    redactor -->|"Customer PERSON_1 ... PAN PAN_1"| sanitizedPrompt
    program -->|"SanitizedPrompt only"| mockLlm
    mockLlm --> mockChat
    mockChat --> llmResponse

    rawPrompt -.-> program
    di -.-> sanitizer
    di -.-> mockLlm
```

---

## Detection to Redaction Detail

`PromptSanitizer` orchestrates a two-phase pipeline: **detect** then **redact**. `CompositePiiDetector` aggregates spans from all registered detectors, resolves overlaps by registration order and source priority, and returns a merged entity list. `PlaceholderPiiRedactor` replaces spans right-to-left to preserve indices, assigns stable per-type counters, and builds the in-process placeholder map.

```mermaid
flowchart LR
  inputText["Original prompt text"]

  subgraph detectPhase [Detection Phase]
    domainDet["DomainRulePiiDetector"]
    regexDet["RegexPiiDetector"]
    onnxDet["OnnxNerPiiDetector"]
    composite["CompositePiiDetector"]
    merge["Overlap merge and source priority"]
    entityList["PiiEntity list"]
  end

  subgraph redactPhase [Redaction Phase]
    redactor["PlaceholderPiiRedactor"]
    replace["Right-to-left span replacement"]
    placeholderMap["Placeholder map in-process"]
    sanitizedText["Sanitized text"]
  end

  inputText --> domainDet
  inputText --> regexDet
  inputText --> onnxDet
  domainDet --> composite
  regexDet --> composite
  onnxDet --> composite
  composite --> merge
  merge --> entityList
  entityList --> redactor
  inputText --> redactor
  redactor --> replace
  replace --> sanitizedText
  replace --> placeholderMap
```

**Overlap resolution rules** (applied by `CompositePiiDetector`):

1. Detectors run in registration order: **Domain → Regex → ONNX NER**.
2. On overlapping spans, the first registered detector wins.
3. Tie-breaking uses source priority: Domain (3) > Regex (2) > NER (1).

**Placeholder assignment** (applied by `PlaceholderPiiRedactor`):

- Format: `<{TYPE}_{n}>` (e.g. `<EMAIL_1>`, `<PERSON_1>`).
- Duplicate values of the same type reuse the same placeholder.
- Replacement proceeds from highest `StartIndex` to lowest to avoid index drift.

---

## Runtime Sequence

```mermaid
sequenceDiagram
    participant User
    participant Program as Program.cs
    participant DI as ServiceProvider
    participant Sanitizer as PromptSanitizer
    participant Detector as CompositePiiDetector
    participant Redactor as PlaceholderPiiRedactor
    participant LlmSvc as MockLlmPromptService
    participant Chat as MockChatClient

    User->>Program: Start application
    Program->>DI: Resolve IPromptSanitizer, ILlmPromptService
    DI-->>Program: Sanitizer, LlmService

    alt Interactive mode
        User->>Program: Enter prompt via console
    else Default mode
        Program->>Program: Load canonical sample prompt
    end

    Program->>Sanitizer: Sanitize(SanitizationRequest)
    Sanitizer->>Detector: Detect(originalPrompt)
    Detector-->>Sanitizer: IReadOnlyList PiiEntity
    Sanitizer->>Redactor: Redact(originalPrompt, entities)
    Redactor-->>Sanitizer: RedactionResult
    Sanitizer-->>Program: SanitizationResult

    Program->>Program: Display detected entities
    Program->>Program: Display sanitized prompt
    Program->>Program: Display placeholder map in-process

    Program->>LlmSvc: SendPromptAsync(sanitizedPrompt)
    Note over Program,LlmSvc: Placeholder map never passed
    LlmSvc->>Chat: GetResponseAsync(user message)
    Chat-->>LlmSvc: Assistant response
    LlmSvc-->>Program: Mock LLM response string
    Program->>User: Write LLM response to console
```

---

## Trust Boundary

The LLM boundary is the point at which data leaves the application process via `ILlmPromptService` / `IChatClient`. Only the sanitized prompt crosses this boundary. Original PII values, detection metadata, and the placeholder-to-value map remain in-process.

```mermaid
flowchart TB
    subgraph inProcess [In-Process Trust Zone]
        originalPrompt["Original prompt with raw PII"]
        detectedEntities["Detected PiiEntity list"]
        placeholderMap["Placeholder map"]
        sanitizationResult["SanitizationResult"]
        consoleDisplay["Console audit output"]
    end

    subgraph llmBoundary [LLM Trust Boundary]
        sanitizedOnly["Sanitized prompt text only"]
    end

    subgraph externalLlm [External LLM Provider]
        chatClient["IChatClient implementation"]
        modelInference["Model inference"]
    end

    originalPrompt --> sanitizationResult
    detectedEntities --> sanitizationResult
    placeholderMap --> sanitizationResult
    sanitizationResult --> consoleDisplay
    sanitizationResult -->|"SendPromptAsync"| sanitizedOnly
    sanitizedOnly --> chatClient
    chatClient --> modelInference

    originalPrompt -.-x|"Never transmitted"| chatClient
    placeholderMap -.-x|"Never transmitted"| chatClient
    detectedEntities -.-x|"Never transmitted"| chatClient
```

In the POC, `MockChatClient` simulates the external provider without network I/O. Replacing it with Azure OpenAI or another `IChatClient` implementation does not change the trust model: `MockLlmPromptService` (or a future production adapter) continues to accept only the sanitized string.

---

## Project Responsibilities

| Project | Layer | Responsibility |
|---------|-------|----------------|
| `PiiRedaction.ConsoleApp` | Presentation | Application entry point; reads prompt (sample or interactive); bootstraps `IHost` and DI via `AddPiiRedactionServices`; orchestrates sanitization and LLM invocation; renders audit output (detected entities, sanitized text, placeholder map). |
| `PiiRedaction.Core` | Domain / Application | Defines abstractions (`IPiiDetector`, `IPiiRedactor`, `IPromptSanitizer`, `ILlmPromptService`); implements detection strategies (`RegexPiiDetector`, `DomainRulePiiDetector`, `OnnxNerPiiDetector`, `CompositePiiDetector`); implements `PlaceholderPiiRedactor` and `PromptSanitizer`; owns domain models (`PiiEntity`, `SanitizationResult`, `RedactionResult`) and configuration (`PiiRedactionOptions`). Has no dependency on ONNX Runtime or LLM SDKs. |
| `PiiRedaction.Infrastructure` | Infrastructure | Implements technical adapters: `OnnxNerModelRunner` (ONNX Runtime inference), `MockChatClient` and `MockLlmPromptService` (`Microsoft.Extensions.AI`); depends on Core abstractions and is swappable without changing domain logic. |
| `tests/PiiRedaction.Core.Tests` | Test | Unit and integration tests for detectors, redactor, sanitizer, overlap rules, golden prompt scenarios (`PromptScenarioCatalog`), and LLM boundary assertions. |
| `tests/PiiRedaction.Infrastructure.Tests` | Test | Tests for mock LLM behavior and ONNX runner load semantics. |

**Dependency direction:** `ConsoleApp` → `Infrastructure` → `Core`. Core references no outer layers, preserving the Dependency Inversion Principle and enabling future hosts (ASP.NET Core API, worker services) to reuse the same Core and Infrastructure assemblies.

---

## Key Abstractions and Extension Points

| Abstraction | Defined In | Default Implementation | Extension |
|-------------|------------|------------------------|-----------|
| `IPiiDetector` | Core | `CompositePiiDetector` wrapping Domain, Regex, ONNX | Add new detector; register in composite order |
| `IPiiRedactor` | Core | `PlaceholderPiiRedactor` | Replace with hashing, vault-backed tokens, etc. |
| `IPromptSanitizer` | Core | `PromptSanitizer` | Unlikely to change; orchestrates detect + redact |
| `ILlmPromptService` | Core | `MockLlmPromptService` | Production adapter with telemetry, retry, policy |
| `IChatClient` | Microsoft.Extensions.AI | `MockChatClient` | Azure OpenAI, OpenAI, or other provider SDK |
| `IOnnxNerModelRunner` | Core | `OnnxNerModelRunner` | BERT WordPiece tokenization, ONNX inference, BIO label decoding |

---

## Configuration Surface

Runtime behavior is controlled via `appsettings.json` under the `PiiRedaction` section:

| Setting | Effect |
|---------|--------|
| `OnnxModelPath` | Path to ONNX NER model (`models/ner-model.onnx` by default). Companion files `vocab.txt` and `ner-labels.txt` must live in the same directory. |

Download the model assets with `scripts/download-ner-model.ps1` (exports `dslim/bert-base-NER`).

---

## Related Documentation

- [README](../README.md) — build, run, configuration, and testing instructions
- [ServiceCollectionExtensions.cs](../src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs) — DI registration and detector ordering
- [PromptScenarioCatalog.cs](../tests/PiiRedaction.Core.Tests/TestSupport/PromptScenarioCatalog.cs) — focused golden pipeline scenarios including the canonical example