llm-pii-poc/docs/architecture.md

PII Redaction POC — Solution Architecture

Purpose

This document describes the architectural design of the PII Redaction POC, a .NET proof-of-concept that intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits only sanitized text across the LLM trust boundary. The solution is structured for enterprise adoption: clear layer separation, interface-driven composition, dependency injection, and swappable infrastructure adapters (ONNX NER, Microsoft.Extensions.AI chat clients).

The POC validates a compliance-oriented pattern suitable for financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.

---

Canonical Example

The console application ships with a sample catalog (11 prompts). The canonical demo is sample FullFinancialWithCustomer. The table below shows the exact strings produced by the production pipeline when the ONNX NER model is loaded (run scripts/download-ner-model.ps1 first).

Stage	Value
Input	Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.
Sanitized Output	Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.
Mock LLM Response	[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.

Detected entities for this prompt:

Type	Value	Detection Source
PERSON	Ravi Kumar	Ner
EMAIL	ravi.kumar@gmail.com	Regex
PHONE	9876543210	Regex
LOAN_NUMBER	LN-456789	Domain
PAN	ABCDE1234F	Regex

The internal placeholder map (<PERSON_1> → Ravi Kumar, etc.) is retained in-process and is not included in the outbound LLM request.

---

Console Sample Catalog

Running dotnet run --project src/PiiRedaction.ConsoleApp executes all samples sequentially. Use --list, --sample N, or --name SampleName to filter.

NER / person-name samples

These prompts exercise OnnxNerPiiDetector and OnnxNerModelRunner. Person names require the ONNX model (models/ner-model.onnx plus vocab.txt and ner-labels.txt). Without the model, person spans are not detected.

Sample	Input (excerpt)	Detected person	Sanitized (excerpt)
CustomerNameOnly	Customer Anita Sharma reported unauthorized…	Anita Sharma	Customer <PERSON_1> reported unauthorized…
MrTitlePerson	Mr. John Smith called about a duplicate debit…	John Smith	<PERSON_1> called about a duplicate debit…
MrsTitlePerson	Mrs. Lakshmi Reddy requested a callback regarding LN-112233.	Lakshmi Reddy	<PERSON_1> requested a callback regarding <LOAN_NUMBER_1>.
DrTitlePerson	Dr. Jane Doe escalated a complaint…	Jane Doe	<PERSON_1> escalated a complaint…
TwoCustomersInOnePrompt	Customer Ravi Kumar and Customer Priya Nair…	Ravi Kumar, Priya Nair	Customer <PERSON_1> and Customer <PERSON_2>…
PersonWithDomainIds	Customer Meera Iyer holds CID-7070…	Meera Iyer	Customer <PERSON_1> holds <CUSTOMER_ID_1>…
PersonWithEmailNoPhone	Customer Arjun Mehta wrote from arjun.mehta@company.in…	Arjun Mehta	Customer <PERSON_1> wrote from <EMAIL_1>…

Other sample categories

Category	Sample	Purpose
NER + Regex + Domain	FullFinancialWithCustomer	End-to-end financial prompt (canonical)
Regex only	AllRegexTypes	Email, phone, PAN, Aadhaar, credit card
Domain only	AllDomainIds	Loan number, customer ID, account number
Negative	NoPiiCleanTicket	Passthrough with no detected PII

Sample definitions live in SamplePromptCatalog.cs.

---

High-Level Data Flow

The diagram below traces the canonical example from console input through Core sanitization to the Infrastructure LLM adapter. Data labels reflect the canonical strings at each stage.

flowchart TB
    subgraph consoleApp [PiiRedaction.ConsoleApp]
        program["Program.cs"]
        di["ServiceRegistration"]
    end

    subgraph core [PiiRedaction.Core]
        sanitizer["PromptSanitizer"]
        composite["CompositePiiDetector"]
        regexDet["RegexPiiDetector"]
        domainDet["DomainRulePiiDetector"]
        onnxDet["OnnxNerPiiDetector"]
        redactor["PlaceholderPiiRedactor"]
    end

    subgraph infra [PiiRedaction.Infrastructure]
        onnxRunner["OnnxNerModelRunner"]
        mockLlm["MockLlmPromptService"]
        mockChat["MockChatClient"]
    end

    rawPrompt["Raw prompt with PII"]
    sanitizedPrompt["Sanitized prompt with placeholders"]
    llmResponse["Mock LLM acknowledgment"]

    program -->|"Customer Ravi Kumar ... PAN ABCDE1234F"| sanitizer
    sanitizer --> composite
    composite --> domainDet
    composite --> regexDet
    composite --> onnxDet
    onnxDet --> onnxRunner
    sanitizer --> redactor
    redactor -->|"Customer PERSON_1 ... PAN PAN_1"| sanitizedPrompt
    program -->|"SanitizedPrompt only"| mockLlm
    mockLlm --> mockChat
    mockChat --> llmResponse

    rawPrompt -.-> program
    di -.-> sanitizer
    di -.-> mockLlm

---

Detection to Redaction Detail

PromptSanitizer orchestrates a two-phase pipeline: detect then redact. CompositePiiDetector aggregates spans from all registered detectors, resolves overlaps by registration order and source priority, and returns a merged entity list. PlaceholderPiiRedactor replaces spans right-to-left to preserve indices, assigns stable per-type counters, and builds the in-process placeholder map.

flowchart LR
  inputText["Original prompt text"]

  subgraph detectPhase [Detection Phase]
    domainDet["DomainRulePiiDetector"]
    regexDet["RegexPiiDetector"]
    onnxDet["OnnxNerPiiDetector"]
    composite["CompositePiiDetector"]
    merge["Overlap merge and source priority"]
    entityList["PiiEntity list"]
  end

  subgraph redactPhase [Redaction Phase]
    redactor["PlaceholderPiiRedactor"]
    replace["Right-to-left span replacement"]
    placeholderMap["Placeholder map in-process"]
    sanitizedText["Sanitized text"]
  end

  inputText --> domainDet
  inputText --> regexDet
  inputText --> onnxDet
  domainDet --> composite
  regexDet --> composite
  onnxDet --> composite
  composite --> merge
  merge --> entityList
  entityList --> redactor
  inputText --> redactor
  redactor --> replace
  replace --> sanitizedText
  replace --> placeholderMap

Overlap resolution rules (applied by CompositePiiDetector):

1. Detectors run in registration order: Domain → Regex → ONNX NER.
2. On overlapping spans, the first registered detector wins.
3. Tie-breaking uses source priority: Domain (3) > Regex (2) > NER (1).

Placeholder assignment (applied by PlaceholderPiiRedactor):

• Format: <{TYPE}_{n}> (e.g. <EMAIL_1>, <PERSON_1>).
• Duplicate values of the same type reuse the same placeholder.
• Replacement proceeds from highest StartIndex to lowest to avoid index drift.

---

Runtime Sequence

sequenceDiagram
    participant User
    participant Program as Program.cs
    participant DI as ServiceProvider
    participant Sanitizer as PromptSanitizer
    participant Detector as CompositePiiDetector
    participant Redactor as PlaceholderPiiRedactor
    participant LlmSvc as MockLlmPromptService
    participant Chat as MockChatClient

    User->>Program: Start application
    Program->>DI: Resolve IPromptSanitizer, ILlmPromptService
    DI-->>Program: Sanitizer, LlmService

    alt Interactive mode
        User->>Program: Enter prompt via console
    else Default mode
        Program->>Program: Load canonical sample prompt
    end

    Program->>Sanitizer: Sanitize(SanitizationRequest)
    Sanitizer->>Detector: Detect(originalPrompt)
    Detector-->>Sanitizer: IReadOnlyList PiiEntity
    Sanitizer->>Redactor: Redact(originalPrompt, entities)
    Redactor-->>Sanitizer: RedactionResult
    Sanitizer-->>Program: SanitizationResult

    Program->>Program: Display detected entities
    Program->>Program: Display sanitized prompt
    Program->>Program: Display placeholder map in-process

    Program->>LlmSvc: SendPromptAsync(sanitizedPrompt)
    Note over Program,LlmSvc: Placeholder map never passed
    LlmSvc->>Chat: GetResponseAsync(user message)
    Chat-->>LlmSvc: Assistant response
    LlmSvc-->>Program: Mock LLM response string
    Program->>User: Write LLM response to console

---

Trust Boundary

The LLM boundary is the point at which data leaves the application process via ILlmPromptService / IChatClient. Only the sanitized prompt crosses this boundary. Original PII values, detection metadata, and the placeholder-to-value map remain in-process.

flowchart TB
    subgraph inProcess [In-Process Trust Zone]
        originalPrompt["Original prompt with raw PII"]
        detectedEntities["Detected PiiEntity list"]
        placeholderMap["Placeholder map"]
        sanitizationResult["SanitizationResult"]
        consoleDisplay["Console audit output"]
    end

    subgraph llmBoundary [LLM Trust Boundary]
        sanitizedOnly["Sanitized prompt text only"]
    end

    subgraph externalLlm [External LLM Provider]
        chatClient["IChatClient implementation"]
        modelInference["Model inference"]
    end

    originalPrompt --> sanitizationResult
    detectedEntities --> sanitizationResult
    placeholderMap --> sanitizationResult
    sanitizationResult --> consoleDisplay
    sanitizationResult -->|"SendPromptAsync"| sanitizedOnly
    sanitizedOnly --> chatClient
    chatClient --> modelInference

    originalPrompt -.-x|"Never transmitted"| chatClient
    placeholderMap -.-x|"Never transmitted"| chatClient
    detectedEntities -.-x|"Never transmitted"| chatClient

In the POC, MockChatClient simulates the external provider without network I/O. Replacing it with Azure OpenAI or another IChatClient implementation does not change the trust model: MockLlmPromptService (or a future production adapter) continues to accept only the sanitized string.

---

Project Responsibilities

Project	Layer	Responsibility
PiiRedaction.ConsoleApp	Presentation	Application entry point; reads prompt (sample or interactive); bootstraps IHost and DI via AddPiiRedactionServices; orchestrates sanitization and LLM invocation; renders audit output (detected entities, sanitized text, placeholder map).
PiiRedaction.Core	Domain / Application	Defines abstractions (IPiiDetector, IPiiRedactor, IPromptSanitizer, ILlmPromptService); implements detection strategies (RegexPiiDetector, DomainRulePiiDetector, OnnxNerPiiDetector, CompositePiiDetector); implements PlaceholderPiiRedactor and PromptSanitizer; owns domain models (PiiEntity, SanitizationResult, RedactionResult) and configuration (PiiRedactionOptions). Has no dependency on ONNX Runtime or LLM SDKs.
PiiRedaction.Infrastructure	Infrastructure	Implements technical adapters: OnnxNerModelRunner (ONNX Runtime inference), MockChatClient and MockLlmPromptService (Microsoft.Extensions.AI); depends on Core abstractions and is swappable without changing domain logic.
tests/PiiRedaction.Core.Tests	Test	Unit and integration tests for detectors, redactor, sanitizer, overlap rules, golden prompt scenarios (PromptScenarioCatalog), and LLM boundary assertions.
tests/PiiRedaction.Infrastructure.Tests	Test	Tests for mock LLM behavior and ONNX runner load semantics.

Dependency direction: ConsoleApp → Infrastructure → Core. Core references no outer layers, preserving the Dependency Inversion Principle and enabling future hosts (ASP.NET Core API, worker services) to reuse the same Core and Infrastructure assemblies.

---

Key Abstractions and Extension Points

Abstraction	Defined In	Default Implementation	Extension
IPiiDetector	Core	CompositePiiDetector wrapping Domain, Regex, ONNX	Add new detector; register in composite order
IPiiRedactor	Core	PlaceholderPiiRedactor	Replace with hashing, vault-backed tokens, etc.
IPromptSanitizer	Core	PromptSanitizer	Unlikely to change; orchestrates detect + redact
ILlmPromptService	Core	MockLlmPromptService	Production adapter with telemetry, retry, policy
IChatClient	Microsoft.Extensions.AI	MockChatClient	Azure OpenAI, OpenAI, or other provider SDK
IOnnxNerModelRunner	Core	OnnxNerModelRunner	BERT WordPiece tokenization, ONNX inference, BIO label decoding

---

Configuration Surface

Runtime behavior is controlled via appsettings.json under the PiiRedaction section:

Setting	Effect
OnnxModelPath	Path to ONNX NER model (models/ner-model.onnx by default). Companion files vocab.txt and ner-labels.txt must live in the same directory.

Download the model assets with scripts/download-ner-model.ps1 (exports dslim/bert-base-NER).

---

Related Documentation

• README — build, run, configuration, and testing instructions
• ServiceCollectionExtensions.cs — DI registration and detector ordering
• PromptScenarioCatalog.cs — focused golden pipeline scenarios including the canonical example