llm-pii-poc/docs/solution-guide.md

PII Redaction POC — Solution Guide

Single reference for architecture, NER models, routing, operations, Tamil/Tanglish support, Git setup, and the post-POC improvement backlog.

---

Table of contents

1. Purpose
2. Canonical example
3. Detection strategies
4. Solution architecture
5. Trust boundary
6. NER models
7. NER routing — English vs Tamil
8. Configuration and dependency injection
9. Running and testing
10. Tamil and Tanglish support
11. Git remote setup (Xenovex)
12. Improvement roadmap
13. Key source files

---

1. Purpose

This .NET proof-of-concept intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits only sanitized text across the LLM trust boundary.

The solution uses:

• Clear layer separation (Core / Infrastructure / hosts)
• Interface-driven composition and dependency injection
• Swappable ONNX NER adapters and Microsoft.Extensions.AI chat clients

It targets financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.

---

2. Canonical example

With English and Tamil ONNX models loaded (scripts/download-ner-model.ps1, scripts/download-tamil-ner-model.ps1):

Stage	Value
Input	Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.
Sanitized output	Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.
Mock LLM response	[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.

Type	Value	Source
PERSON	Ravi Kumar	NER
EMAIL	ravi.kumar@gmail.com	Regex
PHONE	9876543210	Regex
LOAN_NUMBER	LN-456789	Domain
PAN	ABCDE1234F	Regex

The placeholder map (<PERSON_1> → Ravi Kumar, etc.) stays in-process and is never sent to the LLM.

---

3. Detection strategies

Strategy	Detects	Rationale
Regex	Email, phone, PAN, Aadhaar, credit card	Deterministic, format-bound, auditable
ONNX NER	Person names	Contextual; no rigid format
Domain rules	Loan number (LN-), customer ID (CID-), account (ACC-)	Business-specific identifiers

Overlap resolution (CompositePiiDetector): detectors run in order Domain → Regex → NER. On overlapping spans, candidates are sorted by start index, length, and source priority (Domain 3 > Regex 2 > NER 1); the first non-overlapping candidate wins.

Redaction (PlaceholderPiiRedactor): format <{TYPE}_{n}>; duplicate values reuse placeholders; replacement is right-to-left to preserve indices.

---

4. Solution architecture

Project structure

src/
├── PiiRedaction.ConsoleApp/        # Console demo, DI bootstrap
├── PiiRedaction.TestHarness.Wpf/   # WPF MVVM manual test harness
├── PiiRedaction.Core/              # Detection, redaction, abstractions
└── PiiRedaction.Infrastructure/    # ONNX runners, mock LLM
models/                             # ONNX assets (gitignored)
tests/                              # NUnit unit and integration tests

Project	Layer	Responsibility
PiiRedaction.ConsoleApp	Presentation	Samples, interactive mode, audit output
PiiRedaction.TestHarness.Wpf	Presentation	Category-filtered prompts, redact UI, batch runner
PiiRedaction.Core	Domain	IPiiDetector, IPromptSanitizer, detectors, models
PiiRedaction.Infrastructure	Infrastructure	RoutingOnnxNerModelRunner, MockLlmPromptService
tests/*	Test	~112 tests; RealModel and TamilNer categories

Dependency direction: ConsoleApp / Wpf → Infrastructure → Core. Core has no ONNX or LLM SDK references.

Key abstractions

Abstraction	Default implementation	Extension
IPiiDetector	CompositePiiDetector	Add detector; register in composite order
IPiiRedactor	PlaceholderPiiRedactor	Hashing, vault tokens
IPromptSanitizer	PromptSanitizer	Orchestrates detect + redact
IOnnxNerModelRunner	RoutingOnnxNerModelRunner	Script-based EN/TA routing
ILlmPromptService	MockLlmPromptService	Production adapter
IChatClient	MockChatClient	Azure OpenAI, etc.

Data flow

flowchart TB
    subgraph hosts [Hosts]
        console[ConsoleApp / WpfHarness]
    end

    subgraph core [PiiRedaction.Core]
        sanitizer[PromptSanitizer]
        composite[CompositePiiDetector]
        domain[DomainRulePiiDetector]
        regex[RegexPiiDetector]
        onnxDet[OnnxNerPiiDetector]
        redactor[PlaceholderPiiRedactor]
    end

    subgraph infra [PiiRedaction.Infrastructure]
        router[RoutingOnnxNerModelRunner]
        en[EnglishOnnxNerRunner]
        ta[TamilOnnxNerRunner]
        llm[MockLlmPromptService]
    end

    console --> sanitizer
    sanitizer --> composite
    composite --> domain
    composite --> regex
    composite --> onnxDet
    onnxDet --> router
    router --> en
    router --> ta
    sanitizer --> redactor
    console -->|"sanitized text only"| llm

Pipeline: Sanitize → Detect (all detectors) → Redact → SanitizationResult. Optional: SendPromptAsync(sanitizedPrompt) to LLM.

---

5. Trust boundary

Only the sanitized prompt string crosses ILlmPromptService / IChatClient. Original PII, entity metadata, and the placeholder map remain in-process.

flowchart LR
    subgraph inProcess [In-Process]
        raw[Original prompt]
        entities[Detected entities]
        map[Placeholder map]
        audit[Console / WPF display]
    end

    subgraph boundary [LLM boundary]
        sanitized[Sanitized prompt only]
    end

    subgraph external [External LLM]
        chat[IChatClient]
    end

    raw --> audit
    entities --> audit
    map --> audit
    sanitized --> chat
    raw -.->|never sent| chat
    map -.->|never sent| chat

---

6. NER models

Person-name detection is the only NER responsibility. Structured PII uses regex and domain rules.

Routing summary

Script in prompt	Model(s)	Typical use
LatinOnly	English	English names, Indian names in Roman script, Tanglish
TamilOnly	Tamil	Tamil-script names
Mixed	Both; merge spans	Code-mixed prompts
NoLetters	Neither	Digits/symbols only

English model

Property	Value
Hugging Face ID	dslim/bert-base-NER
Tokenizer	WordPiece (vocab.txt, BertWordPieceEncoder)
Person labels	B-PER, I-PER, B-PERSON, I-PERSON
Primary path	models/en/ner-model.onnx
Legacy fallback	models/ner-model.onnx
Download	.\scripts\download-ner-model.ps1

Tamil model

Property	Value
Hugging Face ID	prachuryyaIITG/SampurNER_Tamil_IndicBERTv2
Tokenizer	WordPiece when vocab.txt present (export path); SentencePiece fallback
Person labels	Any BIO tag containing person (case-insensitive)
Path	models/ta/model.onnx
Download	.\scripts\download-tamil-ner-model.ps1

Why SampurNER over MuRIL: Tamil-specific NER training, smaller footprint (~0.3B vs ~0.6B), validated ONNX export in this repo. MuRIL reserved as eval-driven fallback.

Model assets (gitignored)

Directory	Key files	Approx. size
models/en/	ner-model.onnx, vocab.txt, ner-labels.txt	~431 MB ONNX
models/ta/	model.onnx, vocab.txt, ner-labels.txt	~1 GB ONNX

Only models/en/.gitkeep and models/ta/.gitkeep are committed.

Shared inference

Both runners use OnnxTokenClassifierRunner:

• Max sequence length: 128 tokens (long prompts truncate silently)
• BIO decode → PiiEntityType.Person only
• Missing model → fail-open: [] from NER (person names not redacted)

---

7. NER routing — English vs Tamil

Call chain

PromptSanitizer → CompositePiiDetector → OnnxNerPiiDetector
  → RoutingOnnxNerModelRunner.PredictEntities()
    → ScriptRouter.GetComposition(text)
    → switch (composition) { English / Tamil / both }
    → MergePersonSpans()

Step 1: ScriptRouter (Core)

File: src/PiiRedaction.Core/Detection/ScriptRouter.cs

Single pass over characters:

• Tamil letter: Unicode U+0B80 – U+0BFF
• Latin letter: char.IsAsciiLetter
• Both seen → Mixed (early exit)
• Neither → NoLetters
• Otherwise → TamilOnly or LatinOnly

// ScriptComposition enum: LatinOnly, TamilOnly, Mixed, NoLetters
public ScriptComposition GetComposition(string text) { /* scan chars */ }

Tanglish in Roman script → LatinOnly → English model only.

Step 2: RoutingOnnxNerModelRunner (Infrastructure)

File: src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs

This is the branch decision:

ScriptComposition	English	Tamil
LatinOnly	Yes if available	No
TamilOnly	No	Yes if EnableTamilNer and available
Mixed	Yes if available	Yes if EnableTamilNer and available
NoLetters	No	No

Both models run on the full prompt text for Mixed (no script segmentation).

Step 3: Merge

MergePersonSpans: overlapping PERSON spans → longer span wins; ordered by StartIndex.

Routing diagram

flowchart TD
    text[Prompt text] --> sr[ScriptRouter]
    sr --> latin[LatinOnly]
    sr --> tamil[TamilOnly]
    sr --> mixed[Mixed]
    sr --> none[NoLetters]
    latin --> en[EnglishOnnxNerRunner]
    tamil --> ta[TamilOnnxNerRunner]
    mixed --> en
    mixed --> ta
    en --> merge[MergePersonSpans]
    ta --> merge
    none --> empty[No NER]

Worked examples

Input style	Composition	Models
Customer Ravi Kumar…	LatinOnly	English
வாடிக்கையாளர் ராஜேஷ் குமார்…	TamilOnly	Tamil
Naan Suresh, phone 9003789456…	LatinOnly	English (Tanglish)
வாடிக்கையாளர் Ravi Kumar phone…	Mixed	Both
Callback on 9123456780…	NoLetters	Neither (phone via Regex)

---

8. Configuration and dependency injection

appsettings.json

{
  "PiiRedaction": {
    "OnnxModelPath": "models/ner-model.onnx",
    "EnglishOnnxModelPath": "models/en/ner-model.onnx",
    "TamilOnnxModelPath": "models/ta/model.onnx",
    "EnableTamilNer": true
  }
}

Setting	Effect
EnglishOnnxModelPath	Primary English model
OnnxModelPath	Legacy English fallback
TamilOnnxModelPath	Tamil model
EnableTamilNer	false = English-only routing

DI registration

services.AddSingleton<EnglishOnnxNerRunner>();
services.AddSingleton<TamilOnnxNerRunner>();
services.AddSingleton<IOnnxNerModelRunner, RoutingOnnxNerModelRunner>();
// OnnxNerPiiDetector receives IOnnxNerModelRunner (the router)

Same pattern in PiiRedaction.ConsoleApp and PiiRedaction.TestHarness.Wpf ServiceCollectionExtensions.cs.

Detector registration order in composite: Domain → Regex → ONNX NER.

---

9. Running and testing

Build

dotnet restore
dotnet build

Console app

# All 16 samples (English + Tamil/Tanglish/mixed)
dotnet run --project src/PiiRedaction.ConsoleApp

dotnet run --project src/PiiRedaction.ConsoleApp -- --list
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TamilCustomerNameOnly
dotnet run --project src/PiiRedaction.ConsoleApp -- --interactive

Samples: src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs

WPF test harness (Windows)

dotnet run --project src/PiiRedaction.TestHarness.Wpf

• Category dropdown: Career Guidance, Banking & Financial, Negative, Edge & Harness
• Click prompt → loads input; Redact runs pipeline
• Run All executes scenarios in the selected category
• Script badge shows predicted routing (LatinOnly, TamilOnly, Mixed)

Download models

.\scripts\download-ner-model.ps1
.\scripts\download-tamil-ner-model.ps1

If Tamil PowerShell download 404s, use Python 3.12+:

.\scripts\download-tamil-ner-model.ps1 -Python "$env:LOCALAPPDATA\Programs\Python\Python312\python.exe"

Tests

dotnet test
dotnet test --filter "Category=RealModel"
dotnet test --filter "Category=TamilNer"
dotnet test --filter "FullyQualifiedName~ScriptRouterTests|FullyQualifiedName~RoutingOnnxNerModelRunnerTests"

Tests skip gracefully when ONNX files are absent.

Console sample categories

Category	Examples
NER (English)	CustomerNameOnly, MrTitlePerson, TwoCustomersInOnePrompt
NER (Tamil)	TamilCustomerNameOnly, TamilFullFinancial
Tanglish / Mixed	TanglishCustomer, MixedTamilEnglish
Regex / Domain	AllRegexTypes, AllDomainIds
Negative	NoPiiCleanTicket

---

10. Tamil and Tanglish support

Implementation status

Phase	Status	Scope
1 Generic ONNX token classifier	Done	OnnxTokenClassifierRunner, encoders, NerLabelConfig
2 Tamil download + config	Done	Scripts, dual paths, EnableTamilNer
3 Script routing + DI	Done	ScriptRouter, RoutingOnnxNerModelRunner
4 Tests, samples, docs	Partial	Tests/samples done; eval metrics open
5 Optional enhancements	Not started	See below

Remaining gaps (Phase 4–5)

Gap	Impact
Tamil numeral normalization (௦–௯ → 0–9)	May miss phone/Aadhaar in Tamil script
Tanglish heuristics (peru, enga peru)	Better Latin-name recall in Tamil context
Label-aware regex cues	Contextual name detection
Fail-closed when NER missing	Compliance hardening
MuRIL model swap	If Tamil recall insufficient

Tanglish expectations

Input	Handler	Expected recall
Tamil script names	Tamil ONNX	High (with tuning)
Latin Indian names (Ravi Kumar)	English ONNX	High
Tanglish spellings (Senthil)	English NER + optional heuristics	Medium
Code-mixed prompts	Both models + merge	Medium–high for IDs; names variable

Success metrics (eval set target)

Metric	MVP target
Tamil script person recall	≥ 85%
Tanglish person recall	≥ 70%
False positives on clean prompts	≤ 5%
Structured PII in Tamil prompts (regex)	≥ 95%
English canonical regression	100%

---

11. Git remote setup (Xenovex)

Remote: https://xts.xenovex.com/Bilal-Nazer-Ali/llm-pii-poc.git

Create and push

cd C:\Users\bilal.n\Projects\llm-pii-poc
$git = "C:\Program Files\Git\bin\git.exe"

& $git remote add origin <YOUR_CLONE_URL>
& $git branch -M main
& $git push -u origin main

Use an empty remote repository (no README) when pushing existing history.

Committed vs gitignored

Committed	Gitignored
src/, tests/, scripts/, docs/	bin/, obj/, .vs/
README.md, PiiRedaction.slnx	models/*.onnx, vocab.txt, models/en/*, models/ta/*
models/en/.gitkeep, models/ta/.gitkeep	scratch/

After clone, run model download scripts locally.

---

12. Improvement roadmap

Post-POC items not yet implemented. Current maturity: strong architecture and tests; production needs fail-closed policy, API host, and observability.

P0 — Security and correctness

Item	Proposal
Fail-closed when NER unavailable	RequireNerOnStartup, BlockLlmWhenNerUnavailable options
Outbound LLM guard	Verify sanitized text before IChatClient
Truncation warning	Surface 128-token limit in SanitizationResult
NER confidence	Populate PiiEntity.Confidence or hide UI column
WPF leak check	Use placeholder map, not naive Contains

P1 — Platform

Item	Proposal
PiiRedaction.Composition	Shared AddPiiRedactionServices (Console + WPF duplicate today)
Unified prompt catalog	Single source for console, WPF, golden tests
Async IPromptSanitizer	Replace WPF Task.Run wrapper
PiiRedaction.Application	Shared orchestration for API/WPF
Minimal API + health checks	POST /v1/prompts/sanitize, model readiness
Tamil Phase 4	Numeral normalization, Tanglish heuristics

P2 — Operations

Item	Proposal
Placeholder audit store	TTL, encryption, correlation ID
Observability	Metrics, OpenTelemetry traces
ONNX session pool	Concurrency strategy for API load
CI pipeline	Fast tests without models; nightly real-model job

Suggested phases

1. Production hardening — fail-closed, composition root, API, truncation metadata (~1–2 weeks)
2. Detection quality — normalizer, Tamil Phase 4, regex hardening (~1 week)
3. Operations — audit store, metrics, session pool (~1–2 weeks)

---

13. Key source files

Topic	Path
Script classification	src/PiiRedaction.Core/Detection/ScriptRouter.cs
Model branch decision	src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs
English NER	src/PiiRedaction.Infrastructure/Onnx/EnglishOnnxNerRunner.cs
Tamil NER	src/PiiRedaction.Infrastructure/Onnx/TamilOnnxNerRunner.cs
Shared ONNX inference	src/PiiRedaction.Infrastructure/Onnx/OnnxTokenClassifierRunner.cs
NER detector	src/PiiRedaction.Core/Detection/OnnxNerPiiDetector.cs
Sanitizer	src/PiiRedaction.Core/Sanitization/PromptSanitizer.cs
Composite merge	src/PiiRedaction.Core/Detection/CompositePiiDetector.cs
Options	src/PiiRedaction.Core/Configuration/PiiRedactionOptions.cs
DI	src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs
Console samples	src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs
WPF catalog	src/PiiRedaction.TestHarness.Wpf/Services/TestPromptCatalog.cs
Routing tests	tests/PiiRedaction.Infrastructure.Tests/Onnx/RoutingOnnxNerModelRunnerTests.cs
Script tests	tests/PiiRedaction.Core.Tests/Detection/ScriptRouterTests.cs

---

Last consolidated: July 2026. Replaces separate architecture, NER models, routing reference, Tamil plan, improvement roadmap, and Git setup documents.