llm-pii-poc/README.md

PII Redaction POC

A proof-of-concept .NET solution that redacts personally identifiable information (PII) from user prompts before sending them to a large language model (LLM). The design demonstrates enterprise-grade separation of concerns using SOLID principles, dependency injection, and the Microsoft.Extensions.AI abstractions.

Purpose

Financial and customer-service prompts often contain regulated data (names, government IDs, account numbers). This POC shows how to:

1. Accept a console prompt
2. Detect PII using Regex, ONNX NER, and domain rules
3. Replace values with stable placeholders
4. Send only the sanitized prompt to an LLM (mocked for now)

Architecture

For solution design, data-flow diagrams, trust boundaries, and project responsibilities, see docs/architecture.md.

Why Three Detection Strategies?

Strategy	Used For	Rationale
Regex	Email, phone, PAN, Aadhaar, credit card	Deterministic, format-bound identifiers with stable rules that are easy to audit and test
ONNX NER	Person names	Contextual entities without rigid formats; names vary widely in surface form
Domain rules	Loan number, customer ID, account number	Business-specific identifiers defined by internal systems, not inferable from generic models alone

Why the LLM Receives Only Sanitized Text

The placeholder map (<PERSON_1> → original value) is kept in-process for audit or downstream de-tokenization. Only the sanitized prompt crosses the LLM boundary. This reduces data-exposure risk and supports compliance requirements for regulated workloads.

Project Structure

src/
├── PiiRedaction.ConsoleApp/     # Presentation: input/output, DI bootstrap
├── PiiRedaction.Core/           # Business logic: detection, redaction, models
└── PiiRedaction.Infrastructure/ # Technical adapters: ONNX Runtime, mock LLM
models/                          # Optional ONNX model files (gitignored)

Project	Responsibility
PiiRedaction.ConsoleApp	Read prompt, call sanitizer, display results, call LLM service
PiiRedaction.Core	PII detection abstractions, redaction, sanitization orchestration
PiiRedaction.Infrastructure	ONNX model runner, IChatClient mock implementation

Prerequisites

• .NET SDK 10.x (or compatible SDK for net10.0)
• ONNX NER model for person-name detection (see ONNX Model Setup)
• Python 3.10+ (only for the model download script)

Note: This environment targets net10.0 because .NET 10 SDK is installed. The architecture is identical to the planned .NET 9 layout; change TargetFramework in .csproj files if you use .NET 9 SDK.

Build and Run

From the repository root:

dotnet restore
dotnet build
dotnet run --project src/PiiRedaction.ConsoleApp

By default the console app runs 11 curated sample prompts covering NER/person names, regex identifiers, domain IDs, combined scenarios, and a clean no-PII ticket.

List available samples:

dotnet run --project src/PiiRedaction.ConsoleApp -- --list

Run a single sample by index or name:

dotnet run --project src/PiiRedaction.ConsoleApp -- --sample 2
dotnet run --project src/PiiRedaction.ConsoleApp -- --name MrTitlePerson

Interactive mode (enter your own prompt):

dotnet run --project src/PiiRedaction.ConsoleApp -- --interactive

Console sample catalog

Samples are defined in SamplePromptCatalog.cs.

#	Name	Category	NER / Person example
0	FullFinancialWithCustomer	NER + Regex + Domain	Customer Ravi Kumar + email, phone, loan, PAN
1	CustomerNameOnly	NER	Customer Anita Sharma
2	MrTitlePerson	NER	Mr. John Smith
3	MrsTitlePerson	NER	Mrs. Lakshmi Reddy
4	DrTitlePerson	NER	Dr. Jane Doe
5	TwoCustomersInOnePrompt	NER	Customer Ravi Kumar and Customer Priya Nair
6	PersonWithDomainIds	NER + Domain	Customer Meera Iyer + CID / ACC
7	PersonWithEmailNoPhone	NER + Regex	Customer Arjun Mehta + email
8	AllRegexTypes	Regex	email, phone, PAN, Aadhaar, card
9	AllDomainIds	Domain	LN, CID, ACC
10	NoPiiCleanTicket	Negative	no redaction

Person names are detected via ONNX NER using dslim/bert-base-NER (or a compatible token-classification export). A real model is required for person-name detection; there is no regex or heuristic fallback.

NuGet Packages

# Infrastructure
dotnet add src/PiiRedaction.Infrastructure package Microsoft.ML.OnnxRuntime
dotnet add src/PiiRedaction.Infrastructure package Microsoft.ML.Tokenizers
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.AI.Abstractions
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.AI
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.Logging.Abstractions
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.Options

# Core
dotnet add src/PiiRedaction.Core package Microsoft.Extensions.Options

# ConsoleApp
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Hosting
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.DependencyInjection
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Configuration.Json
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Configuration.EnvironmentVariables

Configuration

appsettings.json:

{
  "PiiRedaction": {
    "OnnxModelPath": "models/ner-model.onnx"
  }
}

Setting	Description
OnnxModelPath	Path to ONNX NER model (relative to working directory or discovered by walking up from the current directory)

ONNX Model Setup

Person-name detection requires a token-classification ONNX model and companion tokenizer files in the models/ directory:

File	Purpose
models/ner-model.onnx	Exported NER model
models/vocab.txt	BERT WordPiece vocabulary
models/ner-labels.txt	One BIO label per line (O, B-PER, I-PER, etc.)

Download script

From the repository root:

.\scripts\download-ner-model.ps1

Or with Python directly:

python scripts/download-ner-model.py

The script exports dslim/bert-base-NER via Hugging Face Optimum when Python is available. Otherwise it downloads the pre-exported ONNX assets from Hugging Face directly.

Inference pipeline

OnnxNerModelRunner performs the full pipeline:

• BERT WordPiece tokenization (Microsoft.ML.Tokenizers)
• ONNX Runtime inference (input_ids, attention_mask, optional token_type_ids)
• BIO label decoding (B-PER / I-PER → PiiEntityType.Person)
• Character-span alignment back to the source text

When the model or tokenizer files are missing, person detection returns no results.

Swapping Mock LLM for Azure OpenAI

The application depends on ILlmPromptService (Core) and IChatClient (Microsoft.Extensions.AI). To use Azure OpenAI later, replace the mock registration in ServiceCollectionExtensions.cs:

// Remove:
// services.AddSingleton<IChatClient, MockChatClient>();

// Add (example — package and API may vary by provider SDK version):
// services.AddAzureOpenAIChatClient(
//     new Uri(configuration["AzureOpenAI:Endpoint"]!),
//     configuration["AzureOpenAI:ApiKey"]!,
//     configuration["AzureOpenAI:DeploymentName"]!);

services.AddSingleton<ILlmPromptService, MockLlmPromptService>(); // unchanged

MockLlmPromptService already uses IChatClient, so it works with any registered chat client implementation.

Sample Execution Output

=== PII Redaction POC ===

Original Prompt:
Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.

Detected PII:
  [PERSON       ] Ravi Kumar             (Ner)
  [EMAIL        ] ravi.kumar@gmail.com   (Regex)
  [PHONE        ] 9876543210             (Regex)
  [LOAN_NUMBER  ] LN-456789              (Domain)
  [PAN          ] ABCDE1234F             (Regex)

Sanitized Prompt:
Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.

Internal Placeholder Map (not sent to LLM):
  <EMAIL_1>        -> ravi.kumar@gmail.com
  <LOAN_NUMBER_1>  -> LN-456789
  <PAN_1>          -> ABCDE1234F
  <PERSON_1>       -> Ravi Kumar
  <PHONE_1>        -> 9876543210

Mock LLM Response:
[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.

SOLID Principles Applied

Principle	Application
Single Responsibility	Each detector, redactor, and runner has one job; Program.cs only orchestrates
Open/Closed	Add new IPiiDetector implementations without changing merge logic
Liskov Substitution	All detectors are interchangeable via IPiiDetector
Interface Segregation	Separate interfaces for detection, redaction, sanitization, and LLM
Dependency Inversion	Core defines abstractions; Infrastructure implements them

Testing

The solution includes an NUnit test suite across two projects:

Project	Focus
tests/PiiRedaction.Core.Tests	Detectors, redactor, sanitizer, golden pipeline scenarios (fake NER), real-model integration tests
tests/PiiRedaction.Infrastructure.Tests	Mock LLM, ONNX runner unit tests, real-model NER runner tests

Run tests

dotnet test
dotnet test --filter "FullyQualifiedName~GoldenPromptTests"
dotnet test --filter "Category=RealModel"
dotnet test --logger "console;verbosity=detailed"

Fast CI runs without the ONNX model: fake-based tests always execute; tests marked Category=RealModel are skipped when models/ner-model.onnx is absent. Download the model first:

.\scripts\download-ner-model.ps1

Test architecture

• PromptScenarioCatalog — five focused end-to-end scenarios (canonical demo, multi-regex, duplicate people, overlap stress, no-PII negative)
• ProductionPipelineFactory — builds the same Domain → Regex → OnnxNer composite stack as production DI; CreateWithRealModel(runner) wires a real OnnxNerModelRunner
• FakeOnnxNerModelRunner — unit-test double for NER; golden tests inject person spans per scenario
• GoldenPromptTests — end-to-end sanitization proof across the catalog (fake NER)
• RealNerModelFixture — shared fixture that loads models/ner-model.onnx once per class; skips when model missing
• RealNerModelRunnerTests — direct ONNX inference with span accuracy checks
• RealNerPipelineTests — full pipeline with real NER (canonical, multi-person, clean-ticket negative)
• OnnxNerModelRunnerTests — unit tests for missing/invalid model paths (no download required)
• CompositePiiDetectorTests — overlap merge and source-priority rules
• LlmBoundaryTests — verifies raw PII never appears in outbound LLM messages

Assertions use FluentAssertions for readable failures on long prompt strings.

Future Enhancements

• ASP.NET Core API host with request/response middleware
• Persistent audit log of redaction events (without storing raw PII)
• Secure vault for reversible tokenization
• Real Azure OpenAI / OpenAI provider registration