Surface NER routing provenance (ModelOrigin, NerModelsInvoked) through the pipeline and WPF harness so English/Tamil routing is observable during POC validation. Consolidate docs into solution-guide and add NER logs, topic filtering, and batch UI fixes in the test harness.
This commit is contained in:
Bilal Nazer Ali
14
README.md
14
README.md
@@ -11,13 +11,9 @@ Financial and customer-service prompts often contain regulated data (names, gove
|
||||
3. Replace values with stable placeholders
|
||||
4. Send only the **sanitized** prompt to an LLM (mocked for now)
|
||||
|
||||
## Architecture
|
||||
## Documentation
|
||||
|
||||
For solution design, data-flow diagrams, trust boundaries, and project responsibilities, see **[docs/architecture.md](docs/architecture.md)**.
|
||||
|
||||
For English and Tamil ONNX NER model IDs, assets, routing, and reproduction steps, see **[docs/ner-models.md](docs/ner-models.md)**.
|
||||
|
||||
**Planned:** Tamil / Tanglish person-name support via dual ONNX NER routing — see **[docs/tamil-tanglish-ner-plan.md](docs/tamil-tanglish-ner-plan.md)**.
|
||||
Full solution reference (architecture, NER models, routing, Tamil/Tanglish, Git setup, improvement roadmap): **[docs/solution-guide.md](docs/solution-guide.md)**
|
||||
|
||||
## Why Three Detection Strategies?
|
||||
|
||||
@@ -59,7 +55,7 @@ models/ # Optional ONNX model files (gitignored)
|
||||
|
||||
## Build and Run
|
||||
|
||||
For pushing this repository to Xenovex Git (`xts.xenovex.com`), see **[docs/git-xenovex-setup.md](docs/git-xenovex-setup.md)**.
|
||||
For pushing this repository to Xenovex Git (`xts.xenovex.com`), see **[docs/solution-guide.md § Git remote setup](docs/solution-guide.md#11-git-remote-setup-xenovex)**.
|
||||
|
||||
From the repository root:
|
||||
|
||||
@@ -96,12 +92,12 @@ dotnet run --project src/PiiRedaction.TestHarness.Wpf
|
||||
|
||||
**Workflow:**
|
||||
|
||||
1. **Select a test prompt** from the left panel (grouped by language: English, Tamil, Mixed, Tanglish) or type your own prompt in the input box.
|
||||
1. **Select a category** from the dropdown (e.g. **Career Guidance**, **Banking & Financial**) or leave **All** to see every prompt. Use the search box for finer filtering.
|
||||
2. **Click a test prompt** in the left panel to load it into the input box (previous results are cleared automatically).
|
||||
3. Click **Redact** to run the full detection pipeline. The status bar shows model availability, script composition (LatinOnly / TamilOnly / Mixed), and elapsed time.
|
||||
4. Review **Sanitized Output**, detected entities, and the placeholder map in the right panel. A leak warning appears if any detected value remains in the sanitized text.
|
||||
5. Optionally click **Send Mock LLM** to send only the sanitized prompt to the mock LLM.
|
||||
6. Click **Run All** to execute all **22 curated scenarios** (16 console samples + 6 harness-only edge cases) and view pass/fail results in the batch panel.
|
||||
6. Click **Run All** to execute scenarios in the **selected category** (or all when **All** is chosen) and view pass/fail results in the batch panel.
|
||||
|
||||
The harness uses the same DI registrations and `IPromptSanitizer` pipeline as the console app, with thin application services (`IRedactionAppService`, `ITestPromptCatalog`, `IScriptAnalysisService`, `IModelStatusService`) following SOLID principles.
|
||||
|
||||
|
||||
@@ -1,445 +0,0 @@
|
||||
# PII Redaction POC — Solution Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes the architectural design of the **PII Redaction POC**, a .NET proof-of-concept that intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits **only sanitized text** across the LLM trust boundary. The solution is structured for enterprise adoption: clear layer separation, interface-driven composition, dependency injection, and swappable infrastructure adapters (ONNX NER, `Microsoft.Extensions.AI` chat clients).
|
||||
|
||||
The POC validates a compliance-oriented pattern suitable for financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.
|
||||
|
||||
---
|
||||
|
||||
## Canonical Example
|
||||
|
||||
The console application ships with a **sample catalog** (16 prompts). The canonical demo is sample `FullFinancialWithCustomer`. Tamil script, Tanglish, and mixed-script samples run in the **default** `dotnet run` batch (no `--interactive` required). The table below shows the exact strings produced by the production pipeline when the ONNX NER models are loaded (run `scripts/download-ner-model.ps1` and `scripts/download-tamil-ner-model.ps1` first).
|
||||
|
||||
| Stage | Value |
|
||||
|-------|-------|
|
||||
| **Input** | `Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.` |
|
||||
| **Sanitized Output** | `Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.` |
|
||||
| **Mock LLM Response** | `[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.` |
|
||||
|
||||
Detected entities for this prompt:
|
||||
|
||||
| Type | Value | Detection Source |
|
||||
|------|-------|------------------|
|
||||
| PERSON | Ravi Kumar | Ner |
|
||||
| EMAIL | ravi.kumar@gmail.com | Regex |
|
||||
| PHONE | 9876543210 | Regex |
|
||||
| LOAN_NUMBER | LN-456789 | Domain |
|
||||
| PAN | ABCDE1234F | Regex |
|
||||
|
||||
The internal placeholder map (`<PERSON_1>` → `Ravi Kumar`, etc.) is retained in-process and is **not** included in the outbound LLM request.
|
||||
|
||||
---
|
||||
|
||||
## Console Sample Catalog
|
||||
|
||||
Running `dotnet run --project src/PiiRedaction.ConsoleApp` executes all samples sequentially. Use `--list`, `--sample N`, or `--name SampleName` to filter.
|
||||
|
||||
### NER / person-name samples
|
||||
|
||||
These prompts exercise `OnnxNerPiiDetector` and `RoutingOnnxNerModelRunner`. Person names require ONNX models (`models/en/` for English, `models/ta/` for Tamil script). Without models, person spans are not detected. Legacy `models/ner-model.onnx` is still supported for English.
|
||||
|
||||
| Sample | Input (excerpt) | Detected person | Sanitized (excerpt) |
|
||||
|--------|-----------------|-----------------|---------------------|
|
||||
| **CustomerNameOnly** | Customer Anita Sharma reported unauthorized… | Anita Sharma | Customer `<PERSON_1>` reported unauthorized… |
|
||||
| **MrTitlePerson** | Mr. John Smith called about a duplicate debit… | John Smith | `<PERSON_1>` called about a duplicate debit… |
|
||||
| **MrsTitlePerson** | Mrs. Lakshmi Reddy requested a callback regarding LN-112233. | Lakshmi Reddy | `<PERSON_1>` requested a callback regarding `<LOAN_NUMBER_1>`. |
|
||||
| **DrTitlePerson** | Dr. Jane Doe escalated a complaint… | Jane Doe | `<PERSON_1>` escalated a complaint… |
|
||||
| **TwoCustomersInOnePrompt** | Customer Ravi Kumar and Customer Priya Nair… | Ravi Kumar, Priya Nair | Customer `<PERSON_1>` and Customer `<PERSON_2>`… |
|
||||
| **PersonWithDomainIds** | Customer Meera Iyer holds CID-7070… | Meera Iyer | Customer `<PERSON_1>` holds `<CUSTOMER_ID_1>`… |
|
||||
| **PersonWithEmailNoPhone** | Customer Arjun Mehta wrote from arjun.mehta@company.in… | Arjun Mehta | Customer `<PERSON_1>` wrote from `<EMAIL_1>`… |
|
||||
|
||||
### Tamil / Tanglish / mixed samples
|
||||
|
||||
These prompts exercise `RoutingOnnxNerModelRunner` script routing. Tamil script uses `models/ta/`; Latin Tanglish uses `models/en/`. Mixed prompts may invoke both models.
|
||||
|
||||
| Sample | Input (excerpt) | Detected person | Sanitized (excerpt) |
|
||||
|--------|-----------------|-----------------|---------------------|
|
||||
| **TamilCustomerNameOnly** | வாடிக்கையாளர் ராஜேஷ் குமார் சேமிப்பு… | ராஜேஷ் குமார் | வாடிக்கையாளர் `<PERSON_1>` சேமிப்பு… |
|
||||
| **TamilWithPhonePan** | வாடிக்கையாளர் ராஜேஷ் குமார் தொலைபேசி 9876543210 PAN… | ராஜேஷ் குமார் | `<PERSON_1>` … `<PHONE_1>` … `<PAN_1>` |
|
||||
| **TanglishCustomer** | Customer Senthil phone 9876543210… | Senthil | Customer `<PERSON_1>` phone `<PHONE_1>`… |
|
||||
| **MixedTamilEnglish** | வாடிக்கையாளர் Ravi Kumar phone 9876543210… | Ravi Kumar | வாடிக்கையாளர் `<PERSON_1>` phone `<PHONE_1>`… |
|
||||
| **TamilFullFinancial** | வாடிக்கையாளர் ராஜேஷ் குமார் மின்னஞ்சல் ravi.kumar@gmail.com… | ராஜேஷ் குமார் | Tamil canonical — all placeholder types |
|
||||
|
||||
### Other sample categories
|
||||
|
||||
| Category | Sample | Purpose |
|
||||
|----------|--------|---------|
|
||||
| NER + Regex + Domain | FullFinancialWithCustomer | End-to-end financial prompt (canonical) |
|
||||
| Regex only | AllRegexTypes | Email, phone, PAN, Aadhaar, credit card |
|
||||
| Domain only | AllDomainIds | Loan number, customer ID, account number |
|
||||
| Negative | NoPiiCleanTicket | Passthrough with no detected PII |
|
||||
|
||||
Sample definitions live in [`SamplePromptCatalog.cs`](../src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs).
|
||||
|
||||
---
|
||||
|
||||
## High-Level Data Flow
|
||||
|
||||
The diagram below traces the canonical example from console input through Core sanitization to the Infrastructure LLM adapter. Data labels reflect the canonical strings at each stage.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph consoleApp [PiiRedaction.ConsoleApp]
|
||||
program["Program.cs"]
|
||||
di["ServiceRegistration"]
|
||||
end
|
||||
|
||||
subgraph core [PiiRedaction.Core]
|
||||
sanitizer["PromptSanitizer"]
|
||||
composite["CompositePiiDetector"]
|
||||
regexDet["RegexPiiDetector"]
|
||||
domainDet["DomainRulePiiDetector"]
|
||||
onnxDet["OnnxNerPiiDetector"]
|
||||
redactor["PlaceholderPiiRedactor"]
|
||||
end
|
||||
|
||||
subgraph infra [PiiRedaction.Infrastructure]
|
||||
onnxRunner["RoutingOnnxNerModelRunner"]
|
||||
enRunner["EnglishOnnxNerRunner"]
|
||||
taRunner["TamilOnnxNerRunner"]
|
||||
mockLlm["MockLlmPromptService"]
|
||||
mockChat["MockChatClient"]
|
||||
end
|
||||
|
||||
rawPrompt["Raw prompt with PII"]
|
||||
sanitizedPrompt["Sanitized prompt with placeholders"]
|
||||
llmResponse["Mock LLM acknowledgment"]
|
||||
|
||||
program -->|"Customer Ravi Kumar ... PAN ABCDE1234F"| sanitizer
|
||||
sanitizer --> composite
|
||||
composite --> domainDet
|
||||
composite --> regexDet
|
||||
composite --> onnxDet
|
||||
onnxDet --> onnxRunner
|
||||
sanitizer --> redactor
|
||||
redactor -->|"Customer PERSON_1 ... PAN PAN_1"| sanitizedPrompt
|
||||
program -->|"SanitizedPrompt only"| mockLlm
|
||||
mockLlm --> mockChat
|
||||
mockChat --> llmResponse
|
||||
|
||||
rawPrompt -.-> program
|
||||
di -.-> sanitizer
|
||||
di -.-> mockLlm
|
||||
```
|
||||
|
||||
`RoutingOnnxNerModelRunner` selects English and/or Tamil ONNX models based on script composition in the prompt. See [Dual-Model NER Routing (Tamil + English)](#dual-model-ner-routing-tamil--english) for the routing decision tree.
|
||||
|
||||
---
|
||||
|
||||
## Detection to Redaction Detail
|
||||
|
||||
`PromptSanitizer` orchestrates a two-phase pipeline: **detect** then **redact**. `CompositePiiDetector` aggregates spans from all registered detectors, resolves overlaps by registration order and source priority, and returns a merged entity list. `PlaceholderPiiRedactor` replaces spans right-to-left to preserve indices, assigns stable per-type counters, and builds the in-process placeholder map.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
inputText["Original prompt text"]
|
||||
|
||||
subgraph detectPhase [Detection Phase]
|
||||
domainDet["DomainRulePiiDetector"]
|
||||
regexDet["RegexPiiDetector"]
|
||||
onnxDet["OnnxNerPiiDetector<br/>(RoutingOnnxNerModelRunner)"]
|
||||
composite["CompositePiiDetector"]
|
||||
merge["Overlap merge and source priority"]
|
||||
entityList["PiiEntity list"]
|
||||
end
|
||||
|
||||
subgraph redactPhase [Redaction Phase]
|
||||
redactor["PlaceholderPiiRedactor"]
|
||||
replace["Right-to-left span replacement"]
|
||||
placeholderMap["Placeholder map in-process"]
|
||||
sanitizedText["Sanitized text"]
|
||||
end
|
||||
|
||||
inputText --> domainDet
|
||||
inputText --> regexDet
|
||||
inputText --> onnxDet
|
||||
domainDet --> composite
|
||||
regexDet --> composite
|
||||
onnxDet --> composite
|
||||
composite --> merge
|
||||
merge --> entityList
|
||||
entityList --> redactor
|
||||
inputText --> redactor
|
||||
redactor --> replace
|
||||
replace --> sanitizedText
|
||||
replace --> placeholderMap
|
||||
```
|
||||
|
||||
**Overlap resolution rules** (applied by `CompositePiiDetector`):
|
||||
|
||||
1. Detectors run in registration order: **Domain → Regex → ONNX NER**.
|
||||
2. On overlapping spans, the first registered detector wins.
|
||||
3. Tie-breaking uses source priority: Domain (3) > Regex (2) > NER (1).
|
||||
|
||||
The ONNX NER detector delegates to `RoutingOnnxNerModelRunner`, which routes inference to English and/or Tamil models by script composition. See [Dual-Model NER Routing (Tamil + English)](#dual-model-ner-routing-tamil--english).
|
||||
|
||||
**Placeholder assignment** (applied by `PlaceholderPiiRedactor`):
|
||||
|
||||
|
||||
- Format: `<{TYPE}_{n}>` (e.g. `<EMAIL_1>`, `<PERSON_1>`).
|
||||
- Duplicate values of the same type reuse the same placeholder.
|
||||
- Replacement proceeds from highest `StartIndex` to lowest to avoid index drift.
|
||||
|
||||
---
|
||||
|
||||
## Dual-Model NER Routing (Tamil + English)
|
||||
|
||||
Person-name detection uses two ONNX token-classifier models: **English** (`models/en/`, BERT WordPiece) and **Tamil** (`models/ta/`, SentencePiece or WordPiece). `OnnxNerPiiDetector` calls `RoutingOnnxNerModelRunner`, which classifies prompt script via `ScriptRouter` and dispatches to `EnglishOnnxNerRunner` and/or `TamilOnnxNerRunner`. Both runners share `OnnxTokenClassifierRunner` for BIO decoding; only **PERSON** spans are emitted.
|
||||
|
||||
The diagram below expands the detection and NER branches summarized in [High-Level Data Flow](#high-level-data-flow) and [Detection to Redaction Detail](#detection-to-redaction-detail).
|
||||
|
||||
### End-to-end pipeline (with NER branch)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Entry["Console entry"]
|
||||
A["Program.cs<br/>Host + AddPiiRedactionServices()"]
|
||||
B["PromptDemoRunner.RunAsync()"]
|
||||
A --> B
|
||||
end
|
||||
|
||||
B --> C["SanitizationRequest(OriginalPrompt)"]
|
||||
C --> D["PromptSanitizer.Sanitize()"]
|
||||
|
||||
subgraph Detect["CompositePiiDetector.Detect() — registration order"]
|
||||
direction TB
|
||||
E1["DomainRulePiiDetector<br/>LOAN_NUMBER, CUSTOMER_ID, ACCOUNT_NUMBER"]
|
||||
E2["RegexPiiDetector<br/>EMAIL, PHONE, AADHAAR, PAN, CREDIT_CARD"]
|
||||
E3["OnnxNerPiiDetector<br/>PERSON (via IOnnxNerModelRunner)"]
|
||||
E1 --> MERGE
|
||||
E2 --> MERGE
|
||||
E3 --> MERGE
|
||||
MERGE["Merge overlapping spans<br/>sort: StartIndex ↑, Length ↓, Source priority ↓<br/>(Domain=3, Regex=2, Ner=1)<br/>first candidate wins on overlap"]
|
||||
end
|
||||
|
||||
D --> Detect
|
||||
MERGE --> F["IReadOnlyList<PiiEntity>"]
|
||||
|
||||
F --> G["PlaceholderPiiRedactor.Redact()<br/>replace spans right-to-left<br/>dedupe by Type|Value → <TYPE_n>"]
|
||||
G --> H["SanitizationResult<br/>SanitizedPrompt, DetectedEntities, PlaceholderMap"]
|
||||
|
||||
H --> I["MockLlmPromptService.SendPromptAsync(SanitizedPrompt)"]
|
||||
I --> J["Mock LLM response<br/>(sanitized text only)"]
|
||||
|
||||
subgraph NerBranch["OnnxNerPiiDetector branch"]
|
||||
E3 --> N1{"RoutingOnnxNerModelRunner<br/>.IsModelAvailable?"}
|
||||
N1 -->|no| N2["return []"]
|
||||
N1 -->|yes| N3["RoutingOnnxNerModelRunner<br/>.PredictEntities()"]
|
||||
end
|
||||
```
|
||||
|
||||
### RoutingOnnxNerModelRunner decision tree
|
||||
|
||||
`ScriptRouter.GetComposition` scans each character once. Tamil letters (U+0B80–U+0BFF) and ASCII Latin letters (`char.IsAsciiLetter`) determine the route. When both scripts appear, classification is **Mixed** (early exit).
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
IN["text"] --> SR["ScriptRouter.GetComposition(text)<br/>scan each char"]
|
||||
|
||||
SR --> C1{"LatinOnly?"}
|
||||
SR --> C2{"TamilOnly?"}
|
||||
SR --> C3{"Mixed?"}
|
||||
SR --> C4{"NoLetters?"}
|
||||
|
||||
C1 -->|yes| EN1{"EnglishOnnxNerRunner<br/>.IsModelAvailable?"}
|
||||
EN1 -->|yes| EN_RUN["EnglishOnnxNerRunner.PredictEntities(text)"]
|
||||
EN1 -->|no| SKIP1["skip English"]
|
||||
EN_RUN --> ACC
|
||||
SKIP1 --> ACC
|
||||
|
||||
C2 -->|yes| TA_GATE{"EnableTamilNer<br/>&& TamilOnnxNerRunner<br/>.IsModelAvailable?"}
|
||||
TA_GATE -->|yes| TA_RUN["TamilOnnxNerRunner.PredictEntities(text)"]
|
||||
TA_GATE -->|no| SKIP2["skip Tamil"]
|
||||
TA_RUN --> ACC
|
||||
SKIP2 --> ACC
|
||||
|
||||
C3 -->|yes| EN2{"English available?"}
|
||||
EN2 -->|yes| EN_MIX["EnglishOnnxNerRunner.PredictEntities(text)"]
|
||||
EN2 -->|no| SKIP3["skip English"]
|
||||
EN_MIX --> TA_GATE2{"EnableTamilNer<br/>&& Tamil available?"}
|
||||
SKIP3 --> TA_GATE2
|
||||
TA_GATE2 -->|yes| TA_MIX["TamilOnnxNerRunner.PredictEntities(text)"]
|
||||
TA_GATE2 -->|no| SKIP4["skip Tamil"]
|
||||
TA_MIX --> ACC
|
||||
SKIP4 --> ACC
|
||||
|
||||
C4 -->|yes| EMPTY["no NER inference"]
|
||||
EMPTY --> OUT_EMPTY["return []"]
|
||||
|
||||
subgraph EN_Pipeline["EnglishOnnxNerRunner"]
|
||||
EN_RUN --> EN_ENC["BertWordPieceEncoder<br/>(model dir vocab.txt)"]
|
||||
EN_ENC --> EN_OCR["OnnxTokenClassifierRunner<br/>NerLabelConfig.English<br/>B-PER / I-PER / B-PERSON / I-PERSON"]
|
||||
end
|
||||
|
||||
subgraph TA_Pipeline["TamilOnnxNerRunner"]
|
||||
TA_RUN --> TA_ENC["TokenClassifierEncoderFactory.Create()<br/>vocab.txt → BertWordPieceEncoder<br/>else SentencePiece (*.bpe.model, spiece.model, tokenizer.model)"]
|
||||
TA_ENC --> TA_OCR["OnnxTokenClassifierRunner<br/>NerLabelConfig.Tamil<br/>label contains 'person' (case-insensitive)"]
|
||||
end
|
||||
|
||||
subgraph SharedInference["OnnxTokenClassifierRunner (shared)"]
|
||||
ENC["Encode(text, max 128 tokens)"]
|
||||
ONNX["ONNX InferenceSession.Run<br/>input_ids + attention_mask [+ token_type_ids]"]
|
||||
ARGMAX["Per-token argmax over logits"]
|
||||
BIO["BIO decode → PiiEntityType.Person<br/>PiiDetectionSource.Ner"]
|
||||
ENC --> ONNX --> ARGMAX --> BIO
|
||||
end
|
||||
|
||||
EN_OCR --> SharedInference
|
||||
TA_OCR --> SharedInference
|
||||
BIO --> ACC["accumulate entities"]
|
||||
|
||||
ACC --> MERGE["MergePersonSpans()<br/>sort: Length ↓, StartIndex ↑<br/>drop overlapping spans<br/>(longer span wins)"]
|
||||
MERGE --> OUT["return merged PERSON entities"]
|
||||
```
|
||||
|
||||
### Routing rules
|
||||
|
||||
| Rule | Source | Behavior |
|
||||
|------|--------|----------|
|
||||
| **Script classification** | `ScriptRouter.GetComposition` | Single pass over characters. Tamil letter = U+0B80–U+0BFF. Latin letter = `char.IsAsciiLetter`. Both seen → `Mixed` (early exit). Neither → `NoLetters`. Tamil only → `TamilOnly`. Latin only → `LatinOnly`. |
|
||||
| **LatinOnly** | `RoutingOnnxNerModelRunner` | Run **English only** if `englishRunner.IsModelAvailable`. |
|
||||
| **TamilOnly** | `RoutingOnnxNerModelRunner` | Run **Tamil only** if `EnableTamilNer` (default `true` in `PiiRedactionOptions`) **and** `tamilRunner.IsModelAvailable`. |
|
||||
| **Mixed** | `RoutingOnnxNerModelRunner` | Run **both** models independently on the **full text** (English if available; Tamil if `EnableTamilNer` and available). |
|
||||
| **NoLetters** | `RoutingOnnxNerModelRunner` | No NER inference; returns `[]` from routing (before merge). |
|
||||
| **Model availability gate** | `OnnxNerPiiDetector` | If `RoutingOnnxNerModelRunner.IsModelAvailable` is false, NER detector returns `[]` (English OR Tamil available when Tamil enabled). |
|
||||
| **Post-route merge** | `MergePersonSpans` | After EN/TA results are concatenated, overlapping PERSON spans are deduped; **longer span wins**, then ordered by `StartIndex`. |
|
||||
| **Composite merge** | `CompositePiiDetector` | Domain → Regex → NER all run. Overlaps resolved globally: earlier registration order + longer span + higher source priority (Domain > Regex > Ner). |
|
||||
| **Encoder choice** | `EnglishOnnxNerRunner` vs `TamilOnnxNerRunner` | English always uses `BertWordPieceEncoder`. Tamil uses factory: `vocab.txt` → WordPiece; else first SentencePiece file found; fallback WordPiece with warning. |
|
||||
| **NER output scope** | `OnnxTokenClassifierRunner` | Only **PERSON** entities decoded from BIO tags; max sequence length **128** tokens. |
|
||||
|
||||
---
|
||||
|
||||
## Runtime Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant Program as Program.cs
|
||||
participant DI as ServiceProvider
|
||||
participant Sanitizer as PromptSanitizer
|
||||
participant Detector as CompositePiiDetector
|
||||
participant Redactor as PlaceholderPiiRedactor
|
||||
participant LlmSvc as MockLlmPromptService
|
||||
participant Chat as MockChatClient
|
||||
|
||||
User->>Program: Start application
|
||||
Program->>DI: Resolve IPromptSanitizer, ILlmPromptService
|
||||
DI-->>Program: Sanitizer, LlmService
|
||||
|
||||
alt Interactive mode
|
||||
User->>Program: Enter prompt via console
|
||||
else Default mode
|
||||
Program->>Program: Load canonical sample prompt
|
||||
end
|
||||
|
||||
Program->>Sanitizer: Sanitize(SanitizationRequest)
|
||||
Sanitizer->>Detector: Detect(originalPrompt)
|
||||
Detector-->>Sanitizer: IReadOnlyList PiiEntity
|
||||
Sanitizer->>Redactor: Redact(originalPrompt, entities)
|
||||
Redactor-->>Sanitizer: RedactionResult
|
||||
Sanitizer-->>Program: SanitizationResult
|
||||
|
||||
Program->>Program: Display detected entities
|
||||
Program->>Program: Display sanitized prompt
|
||||
Program->>Program: Display placeholder map in-process
|
||||
|
||||
Program->>LlmSvc: SendPromptAsync(sanitizedPrompt)
|
||||
Note over Program,LlmSvc: Placeholder map never passed
|
||||
LlmSvc->>Chat: GetResponseAsync(user message)
|
||||
Chat-->>LlmSvc: Assistant response
|
||||
LlmSvc-->>Program: Mock LLM response string
|
||||
Program->>User: Write LLM response to console
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
The LLM boundary is the point at which data leaves the application process via `ILlmPromptService` / `IChatClient`. Only the sanitized prompt crosses this boundary. Original PII values, detection metadata, and the placeholder-to-value map remain in-process.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph inProcess [In-Process Trust Zone]
|
||||
originalPrompt["Original prompt with raw PII"]
|
||||
detectedEntities["Detected PiiEntity list"]
|
||||
placeholderMap["Placeholder map"]
|
||||
sanitizationResult["SanitizationResult"]
|
||||
consoleDisplay["Console audit output"]
|
||||
end
|
||||
|
||||
subgraph llmBoundary [LLM Trust Boundary]
|
||||
sanitizedOnly["Sanitized prompt text only"]
|
||||
end
|
||||
|
||||
subgraph externalLlm [External LLM Provider]
|
||||
chatClient["IChatClient implementation"]
|
||||
modelInference["Model inference"]
|
||||
end
|
||||
|
||||
originalPrompt --> sanitizationResult
|
||||
detectedEntities --> sanitizationResult
|
||||
placeholderMap --> sanitizationResult
|
||||
sanitizationResult --> consoleDisplay
|
||||
sanitizationResult -->|"SendPromptAsync"| sanitizedOnly
|
||||
sanitizedOnly --> chatClient
|
||||
chatClient --> modelInference
|
||||
|
||||
originalPrompt -.-x|"Never transmitted"| chatClient
|
||||
placeholderMap -.-x|"Never transmitted"| chatClient
|
||||
detectedEntities -.-x|"Never transmitted"| chatClient
|
||||
```
|
||||
|
||||
In the POC, `MockChatClient` simulates the external provider without network I/O. Replacing it with Azure OpenAI or another `IChatClient` implementation does not change the trust model: `MockLlmPromptService` (or a future production adapter) continues to accept only the sanitized string.
|
||||
|
||||
---
|
||||
|
||||
## Project Responsibilities
|
||||
|
||||
| Project | Layer | Responsibility |
|
||||
|---------|-------|----------------|
|
||||
| `PiiRedaction.ConsoleApp` | Presentation | Application entry point; reads prompt (sample or interactive); bootstraps `IHost` and DI via `AddPiiRedactionServices`; orchestrates sanitization and LLM invocation; renders audit output (detected entities, sanitized text, placeholder map). |
|
||||
| `PiiRedaction.Core` | Domain / Application | Defines abstractions (`IPiiDetector`, `IPiiRedactor`, `IPromptSanitizer`, `ILlmPromptService`); implements detection strategies (`RegexPiiDetector`, `DomainRulePiiDetector`, `OnnxNerPiiDetector`, `CompositePiiDetector`); implements `PlaceholderPiiRedactor` and `PromptSanitizer`; owns domain models (`PiiEntity`, `SanitizationResult`, `RedactionResult`) and configuration (`PiiRedactionOptions`). Has no dependency on ONNX Runtime or LLM SDKs. |
|
||||
| `PiiRedaction.Infrastructure` | Infrastructure | Implements technical adapters: `RoutingOnnxNerModelRunner`, `EnglishOnnxNerRunner`, `TamilOnnxNerRunner` (ONNX Runtime inference), `MockChatClient` and `MockLlmPromptService` (`Microsoft.Extensions.AI`); depends on Core abstractions and is swappable without changing domain logic. |
|
||||
| `tests/PiiRedaction.Core.Tests` | Test | Unit and integration tests for detectors, redactor, sanitizer, overlap rules, golden prompt scenarios (`PromptScenarioCatalog`), and LLM boundary assertions. |
|
||||
| `tests/PiiRedaction.Infrastructure.Tests` | Test | Tests for mock LLM behavior and ONNX runner load semantics. |
|
||||
|
||||
**Dependency direction:** `ConsoleApp` → `Infrastructure` → `Core`. Core references no outer layers, preserving the Dependency Inversion Principle and enabling future hosts (ASP.NET Core API, worker services) to reuse the same Core and Infrastructure assemblies.
|
||||
|
||||
---
|
||||
|
||||
## Key Abstractions and Extension Points
|
||||
|
||||
| Abstraction | Defined In | Default Implementation | Extension |
|
||||
|-------------|------------|------------------------|-----------|
|
||||
| `IPiiDetector` | Core | `CompositePiiDetector` wrapping Domain, Regex, ONNX | Add new detector; register in composite order |
|
||||
| `IPiiRedactor` | Core | `PlaceholderPiiRedactor` | Replace with hashing, vault-backed tokens, etc. |
|
||||
| `IPromptSanitizer` | Core | `PromptSanitizer` | Unlikely to change; orchestrates detect + redact |
|
||||
| `ILlmPromptService` | Core | `MockLlmPromptService` | Production adapter with telemetry, retry, policy |
|
||||
| `IChatClient` | Microsoft.Extensions.AI | `MockChatClient` | Azure OpenAI, OpenAI, or other provider SDK |
|
||||
| `IOnnxNerModelRunner` | Core | `RoutingOnnxNerModelRunner` | Script-based routing to English (BERT WordPiece) and Tamil (SentencePiece) ONNX models |
|
||||
|
||||
---
|
||||
|
||||
## Configuration Surface
|
||||
|
||||
Runtime behavior is controlled via `appsettings.json` under the `PiiRedaction` section:
|
||||
|
||||
| Setting | Effect |
|
||||
|---------|--------|
|
||||
| `OnnxModelPath` | Legacy English model path (`models/ner-model.onnx`). Used as fallback when `models/en/` is absent. |
|
||||
| `EnglishOnnxModelPath` | Primary English ONNX model (`models/en/ner-model.onnx`). |
|
||||
| `TamilOnnxModelPath` | Tamil ONNX model (`models/ta/model.onnx`). |
|
||||
| `EnableTamilNer` | When `false`, routing uses English model only. Default `true`. |
|
||||
|
||||
Download model assets with `scripts/download-ner-model.ps1` and `scripts/download-tamil-ner-model.ps1`.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [NER models](ner-models.md) — English/Tamil ONNX model IDs, assets, labels, and integration reference
|
||||
- [README](../README.md) — build, run, configuration, and testing instructions
|
||||
- [ServiceCollectionExtensions.cs](../src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs) — DI registration and detector ordering
|
||||
- [PromptScenarioCatalog.cs](../tests/PiiRedaction.Core.Tests/TestSupport/PromptScenarioCatalog.cs) — focused golden pipeline scenarios including the canonical example
|
||||
@@ -1,62 +0,0 @@
|
||||
# Git setup — Xenovex (xts.xenovex.com)
|
||||
|
||||
This repository is ready for push to your Xenovex Git server after you create a remote repository.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Git 2.x (installed at `C:\Program Files\Git\bin\git.exe`)
|
||||
- Access to https://xts.xenovex.com/explore/repos
|
||||
- .NET 10 SDK for build/test
|
||||
|
||||
## 1. Create the remote repository
|
||||
|
||||
1. Sign in to **https://xts.xenovex.com**
|
||||
2. Open **Explore repos** (or **New repository**)
|
||||
3. Create a new empty repository, e.g. `llm-pii-poc`
|
||||
4. Copy the **HTTPS** or **SSH** clone URL (example shapes):
|
||||
- `https://xts.xenovex.com/<org-or-user>/llm-pii-poc.git`
|
||||
- `git@xts.xenovex.com:<org-or-user>/llm-pii-poc.git`
|
||||
|
||||
Do **not** initialize the remote with a README if you are pushing an existing local history.
|
||||
|
||||
## 2. Add remote and push (from repository root)
|
||||
|
||||
```powershell
|
||||
cd C:\Users\bilal.n\Projects\llm-pii-poc
|
||||
|
||||
# Use full path if git is not on PATH
|
||||
$git = "C:\Program Files\Git\bin\git.exe"
|
||||
|
||||
& $git remote add origin <YOUR_CLONE_URL>
|
||||
& $git branch -M main
|
||||
& $git push -u origin main
|
||||
```
|
||||
|
||||
If the remote already has commits (e.g. auto-generated README), either use an empty remote or:
|
||||
|
||||
```powershell
|
||||
& $git pull origin main --rebase
|
||||
& $git push -u origin main
|
||||
```
|
||||
|
||||
## 3. What is committed vs excluded
|
||||
|
||||
| Included | Excluded (`.gitignore`) |
|
||||
|----------|-------------------------|
|
||||
| Source (`src/`), tests, scripts, docs | `bin/`, `obj/`, `.vs/` |
|
||||
| `README.md`, `PiiRedaction.slnx` | `models/*.onnx`, `vocab.txt`, `ner-labels.txt` (~431MB model) |
|
||||
| `models/.gitkeep` (empty models folder) | `scratch/` |
|
||||
|
||||
After clone, download the NER model locally:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
```
|
||||
|
||||
## 4. Verify after clone
|
||||
|
||||
```powershell
|
||||
dotnet build
|
||||
dotnet test
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --sample 0
|
||||
```
|
||||
@@ -1,548 +0,0 @@
|
||||
# NER Models for PII Redaction
|
||||
|
||||
This document describes the **Named Entity Recognition (NER)** ONNX models at the core of the PII Redaction POC. Person-name detection is the only NER responsibility in this solution; structured identifiers (email, phone, PAN, domain IDs) are handled by regex and domain-rule detectors.
|
||||
|
||||
For pipeline placement, trust boundaries, and routing diagrams, see [architecture.md](architecture.md). For the Tamil/Tanglish implementation plan and success metrics, see [tamil-tanglish-ner-plan.md](tamil-tanglish-ner-plan.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
The POC uses **dual-model ONNX NER routing** to redact **person-name PII** before prompts reach an LLM:
|
||||
|
||||
| Script in prompt | Model invoked | Typical use case |
|
||||
|------------------|---------------|------------------|
|
||||
| Latin only (`LatinOnly`) | English (`dslim/bert-base-NER`) | English names, Indian names in Roman script, **Tanglish** |
|
||||
| Tamil only (`TamilOnly`) | Tamil (`prachuryyaIITG/SampurNER_Tamil_IndicBERTv2`) | Tamil-script customer names |
|
||||
| Mixed (`Mixed`) | **Both** models on the full text; spans merged | Code-mixed Indian CS prompts |
|
||||
| No letters (`NoLetters`) | Neither | Digits-only or symbol-only text |
|
||||
|
||||
`RoutingOnnxNerModelRunner` classifies script via `ScriptRouter`, delegates to `EnglishOnnxNerRunner` and/or `TamilOnnxNerRunner`, and merges overlapping PERSON spans (longer span wins). Only **PERSON** entities are emitted to the redaction pipeline; all other NER labels are discarded.
|
||||
|
||||
---
|
||||
|
||||
## 2. English Model
|
||||
|
||||
### Hugging Face model ID
|
||||
|
||||
**[`dslim/bert-base-NER`](https://huggingface.co/dslim/bert-base-NER)**
|
||||
|
||||
### Architecture
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Base | BERT-base (uncased), ~110M parameters |
|
||||
| Task | Token classification (NER) |
|
||||
| Tokenizer | **WordPiece** via `vocab.txt` (`BertWordPieceEncoder`) |
|
||||
| Runtime | ONNX via Microsoft.ML.OnnxRuntime |
|
||||
| Export | Hugging Face Optimum (`ORTModelForTokenClassification`) or pre-exported ONNX from HF |
|
||||
|
||||
### Labels (BIO)
|
||||
|
||||
The English model uses standard CoNLL-style BIO tags. The POC maps only **person** labels to `PiiEntityType.Person`:
|
||||
|
||||
| Label | Mapped to PERSON |
|
||||
|-------|------------------|
|
||||
| `O` | No |
|
||||
| `B-PER`, `I-PER` | Yes |
|
||||
| `B-PERSON`, `I-PERSON` | Yes |
|
||||
| `B-ORG`, `I-ORG`, `B-LOC`, `I-LOC`, `B-MISC`, `I-MISC` | No |
|
||||
|
||||
Full label list is written to `ner-labels.txt` at download time from the model `config.json` `id2label` map (typically 9 labels for this model).
|
||||
|
||||
Label matching is implemented in `NerLabelConfig.English`:
|
||||
|
||||
```19:22:src/PiiRedaction.Infrastructure/Onnx/NerLabelConfig.cs
|
||||
private static bool IsEnglishPersonLabel(string label) =>
|
||||
label is "B-PER" or "I-PER" or "B-PERSON" or "I-PERSON"
|
||||
|| (label.EndsWith("-PER", StringComparison.Ordinal) &&
|
||||
(label.StartsWith("B-", StringComparison.Ordinal) || label.StartsWith("I-", StringComparison.Ordinal)));
|
||||
```
|
||||
|
||||
### Asset paths
|
||||
|
||||
| File | Primary path (`appsettings.json`) | Legacy fallback |
|
||||
|------|----------------------------------|-----------------|
|
||||
| ONNX model | `models/en/ner-model.onnx` | `models/ner-model.onnx` (`OnnxModelPath`) |
|
||||
| Vocabulary | `models/en/vocab.txt` | `models/vocab.txt` |
|
||||
| Labels | `models/en/ner-labels.txt` | `models/ner-labels.txt` |
|
||||
|
||||
`EnglishOnnxNerRunner` resolves the model path with primary + legacy fallback:
|
||||
|
||||
```15:22:src/PiiRedaction.Infrastructure/Onnx/EnglishOnnxNerRunner.cs
|
||||
var modelPath = OnnxAssetPathResolver.ResolveModelPath(
|
||||
options.Value.EnglishOnnxModelPath,
|
||||
options.Value.OnnxModelPath);
|
||||
|
||||
var modelDirectory = Path.GetDirectoryName(modelPath) ?? Environment.CurrentDirectory;
|
||||
var labels = OnnxAssetPathResolver.LoadLabels(modelDirectory);
|
||||
var encoder = new BertWordPieceEncoder(modelDirectory, logger);
|
||||
_runner = new OnnxTokenClassifierRunner(modelPath, encoder, NerLabelConfig.English, labels, logger);
|
||||
```
|
||||
|
||||
> **Note:** `scripts/download-ner-model.ps1` writes assets to `models/` (repository root). For the configured primary path, copy or move them into `models/en/`, or rely on the `OnnxModelPath` fallback.
|
||||
|
||||
### Download script
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
```
|
||||
|
||||
Or with Python directly:
|
||||
|
||||
```bash
|
||||
python scripts/download-ner-model.py
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
|
||||
1. If Python + Optimum are available → exports `dslim/bert-base-NER` to ONNX under `models/`.
|
||||
2. Otherwise → downloads pre-exported ONNX from `https://huggingface.co/dslim/bert-base-NER/resolve/main/onnx/` (`model.onnx`, `vocab.txt`, `config.json` → `ner-labels.txt`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Tamil Model
|
||||
|
||||
### Hugging Face model ID
|
||||
|
||||
**[`prachuryyaIITG/SampurNER_Tamil_IndicBERTv2`](https://huggingface.co/prachuryyaIITG/SampurNER_Tamil_IndicBERTv2)**
|
||||
|
||||
(SampurNER Tamil IndicBERTv2 — fine-grained NER for Tamil script.)
|
||||
|
||||
### Why SampurNER IndicBERTv2 vs MuRIL
|
||||
|
||||
| Criterion | SampurNER Tamil IndicBERTv2 | MuRIL (fallback candidate) |
|
||||
|-----------|----------------------------|----------------------------|
|
||||
| Tamil NER training | Fine-grained SampurNER dataset (Tamil-specific labels) | General multilingual; NER requires separate fine-tune |
|
||||
| Model size | ~0.3B parameters (IndicBERTv2, ~278M base) | ~0.6B parameters |
|
||||
| POC fit | Lighter memory footprint; ONNX export path validated in this repo | Reserved for Phase 5 if Tamil recall is insufficient |
|
||||
| Indian financial context | Trained on Indian-language NER corpus; person subtypes map cleanly to PERSON | Heavier; eval-driven swap only |
|
||||
|
||||
See [tamil-tanglish-ner-plan.md](tamil-tanglish-ner-plan.md) §3 for the original selection rationale.
|
||||
|
||||
### Architecture
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Base | IndicBERTv2 (AI4Bharat), ~0.3B parameters |
|
||||
| Task | Fine-grained token classification |
|
||||
| Tokenizer | **WordPiece** when `vocab.txt` is present (this repo's export path); SentencePiece fallback if `sentencepiece.bpe.model` / `spiece.model` exists |
|
||||
| Runtime | Same shared `OnnxTokenClassifierRunner` as English |
|
||||
|
||||
### Tokenizer: WordPiece, not SentencePiece (in practice)
|
||||
|
||||
The Hugging Face repo for this model does **not** ship a SentencePiece model file. The download scripts extract **WordPiece** assets from `tokenizer.json` → `vocab.txt`. `TokenClassifierEncoderFactory` prefers `vocab.txt`:
|
||||
|
||||
```10:19:src/PiiRedaction.Infrastructure/Onnx/TokenClassifierEncoderFactory.cs
|
||||
public static ITokenClassifierEncoder Create(string modelDirectory, ILogger logger)
|
||||
{
|
||||
var vocabPath = OnnxAssetPathResolver.ResolveAssetPath(Path.Combine(modelDirectory, "vocab.txt"));
|
||||
if (File.Exists(vocabPath))
|
||||
{
|
||||
logger.LogInformation(
|
||||
"Using WordPiece tokenizer (vocab.txt) from {ModelDirectory}.",
|
||||
modelDirectory);
|
||||
return new BertWordPieceEncoder(modelDirectory, logger);
|
||||
}
|
||||
```
|
||||
|
||||
The PowerShell Tamil download script emits an explicit warning when WordPiece assets are saved instead of SentencePiece.
|
||||
|
||||
### Labels (fine-grained person tags)
|
||||
|
||||
SampurNER uses fine-grained BIO tags (e.g. `B-person-politician`, `I-person-artist`, `B-location`, `O`). The POC treats **any label containing `person`** (case-insensitive) as a person span:
|
||||
|
||||
```24:25:src/PiiRedaction.Infrastructure/Onnx/NerLabelConfig.cs
|
||||
private static bool IsTamilPersonLabel(string label) =>
|
||||
label.Contains("person", StringComparison.OrdinalIgnoreCase);
|
||||
```
|
||||
|
||||
Unit tests lock this behavior:
|
||||
|
||||
```20:27:tests/PiiRedaction.Infrastructure.Tests/Onnx/NerLabelConfigTests.cs
|
||||
[TestCase("B-person-politician", true)]
|
||||
[TestCase("I-person-artist", true)]
|
||||
[TestCase("B-location", false)]
|
||||
[TestCase("O", false)]
|
||||
public void Tamil_IsPersonLabel_MatchesFineGrainedTags(string label, bool expected)
|
||||
{
|
||||
NerLabelConfig.Tamil.IsPersonLabel(label).Should().Be(expected);
|
||||
}
|
||||
```
|
||||
|
||||
### Asset paths
|
||||
|
||||
| File | Path |
|
||||
|------|------|
|
||||
| ONNX model | `models/ta/model.onnx` |
|
||||
| Tokenizer | `models/ta/vocab.txt` (WordPiece, preferred) **or** `models/ta/sentencepiece.bpe.model` |
|
||||
| Labels | `models/ta/ner-labels.txt` |
|
||||
| Optional | `models/ta/tokenizer.json` (intermediate export artifact) |
|
||||
|
||||
### Download script
|
||||
|
||||
```powershell
|
||||
.\scripts\download-tamil-ner-model.ps1
|
||||
```
|
||||
|
||||
Or with Python directly:
|
||||
|
||||
```bash
|
||||
python scripts/download-tamil-ner-model.py
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
|
||||
1. Python + Optimum → full export to `models/ta/` including ONNX, labels, and tokenizer assets.
|
||||
2. PowerShell fallback → downloads `onnx/model.onnx` from Hugging Face when published; otherwise requires Python export (pre-exported ONNX may return 404).
|
||||
|
||||
`TamilOnnxNerRunner` wiring:
|
||||
|
||||
```15:19:src/PiiRedaction.Infrastructure/Onnx/TamilOnnxNerRunner.cs
|
||||
var modelPath = OnnxAssetPathResolver.ResolveModelPath(options.Value.TamilOnnxModelPath);
|
||||
var modelDirectory = Path.GetDirectoryName(modelPath) ?? Environment.CurrentDirectory;
|
||||
var labels = OnnxAssetPathResolver.LoadLabels(modelDirectory);
|
||||
var encoder = TokenClassifierEncoderFactory.Create(modelDirectory, logger);
|
||||
_runner = new OnnxTokenClassifierRunner(modelPath, encoder, NerLabelConfig.Tamil, labels, logger);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Why These Models
|
||||
|
||||
Evidence-based rationale for this Indian financial POC:
|
||||
|
||||
| Requirement | Decision |
|
||||
|-------------|----------|
|
||||
| **English + Indian Latin names** | `dslim/bert-base-NER` is industry-standard, pre-integrated, and handles many Indian names in Roman script (e.g. `Ravi Kumar`, `Anita Sharma`) |
|
||||
| **Tamil script names** | English BERT is out-of-vocabulary for Tamil letters (U+0B80–U+0BFF); a Tamil-trained NER model is required |
|
||||
| **Tanglish (Roman-script Tamil)** | Routed to the **English** model only (`ScriptComposition.LatinOnly`); no Tamil ONNX on Latin-only text |
|
||||
| **Code-mixed prompts** | Both models run on the full prompt; `MergePersonSpans` deduplicates overlaps |
|
||||
| **Deployable size** | English ~431 MB ONNX (FP32); Tamil IndicBERTv2 ~0.3B params — smaller than MuRIL ~0.6B |
|
||||
| **ONNX export path** | Both models export via Hugging Face Optimum; English has pre-exported ONNX on HF; Tamil may require local Python export |
|
||||
| **PERSON-only scope** | POC redacts person names via NER; org/location/misc labels are intentionally ignored to limit false positives |
|
||||
|
||||
---
|
||||
|
||||
## 5. How They Integrate
|
||||
|
||||
### Configuration (`appsettings.json`)
|
||||
|
||||
```1:8:src/PiiRedaction.ConsoleApp/appsettings.json
|
||||
{
|
||||
"PiiRedaction": {
|
||||
"OnnxModelPath": "models/ner-model.onnx",
|
||||
"EnglishOnnxModelPath": "models/en/ner-model.onnx",
|
||||
"TamilOnnxModelPath": "models/ta/model.onnx",
|
||||
"EnableTamilNer": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Options type:
|
||||
|
||||
```3:14:src/PiiRedaction.Core/Configuration/PiiRedactionOptions.cs
|
||||
public sealed class PiiRedactionOptions
|
||||
{
|
||||
public const string SectionName = "PiiRedaction";
|
||||
|
||||
public string OnnxModelPath { get; set; } = "models/ner-model.onnx";
|
||||
|
||||
public string EnglishOnnxModelPath { get; set; } = "models/en/ner-model.onnx";
|
||||
|
||||
public string TamilOnnxModelPath { get; set; } = "models/ta/model.onnx";
|
||||
|
||||
public bool EnableTamilNer { get; set; } = true;
|
||||
}
|
||||
```
|
||||
|
||||
Set `EnableTamilNer` to `false` for English-only routing.
|
||||
|
||||
### Dependency injection
|
||||
|
||||
```34:36:src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs
|
||||
services.AddSingleton<EnglishOnnxNerRunner>();
|
||||
services.AddSingleton<TamilOnnxNerRunner>();
|
||||
services.AddSingleton<IOnnxNerModelRunner, RoutingOnnxNerModelRunner>();
|
||||
```
|
||||
|
||||
`OnnxNerPiiDetector` consumes `IOnnxNerModelRunner` (the router) and returns `[]` when no model is available — **fail-open** for person detection.
|
||||
|
||||
### Script routing (`ScriptRouter`)
|
||||
|
||||
```11:41:src/PiiRedaction.Core/Detection/ScriptRouter.cs
|
||||
public ScriptComposition GetComposition(string text)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(text);
|
||||
|
||||
var hasLatin = false;
|
||||
var hasTamil = false;
|
||||
|
||||
foreach (var character in text)
|
||||
{
|
||||
if (IsTamilLetter(character))
|
||||
{
|
||||
hasTamil = true;
|
||||
}
|
||||
else if (char.IsAsciiLetter(character))
|
||||
{
|
||||
hasLatin = true;
|
||||
}
|
||||
|
||||
if (hasLatin && hasTamil)
|
||||
{
|
||||
return ScriptComposition.Mixed;
|
||||
}
|
||||
}
|
||||
// ...
|
||||
return hasTamil ? ScriptComposition.TamilOnly : ScriptComposition.LatinOnly;
|
||||
}
|
||||
```
|
||||
|
||||
### Routing runner (`RoutingOnnxNerModelRunner`)
|
||||
|
||||
```39:79:src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
{
|
||||
var composition = _scriptRouter.GetComposition(text);
|
||||
var entities = new List<PiiEntity>();
|
||||
|
||||
switch (composition)
|
||||
{
|
||||
case ScriptComposition.LatinOnly:
|
||||
if (_englishRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_englishRunner.PredictEntities(text));
|
||||
}
|
||||
break;
|
||||
case ScriptComposition.TamilOnly:
|
||||
if (_enableTamilNer && _tamilRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_tamilRunner.PredictEntities(text));
|
||||
}
|
||||
break;
|
||||
case ScriptComposition.Mixed:
|
||||
if (_englishRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_englishRunner.PredictEntities(text));
|
||||
}
|
||||
if (_enableTamilNer && _tamilRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_tamilRunner.PredictEntities(text));
|
||||
}
|
||||
break;
|
||||
case ScriptComposition.NoLetters:
|
||||
break;
|
||||
}
|
||||
|
||||
return MergePersonSpans(entities);
|
||||
}
|
||||
```
|
||||
|
||||
### Shared inference (`OnnxTokenClassifierRunner`)
|
||||
|
||||
Both runners share:
|
||||
|
||||
- **Encode** → `input_ids`, `attention_mask`, optional `token_type_ids`
|
||||
- **Argmax** over per-token logits
|
||||
- **BIO decode** → `PiiEntityType.Person` with `PiiDetectionSource.Ner`
|
||||
- **Max sequence length: 128 tokens**
|
||||
|
||||
```13:13:src/PiiRedaction.Infrastructure/Onnx/OnnxTokenClassifierRunner.cs
|
||||
private const int MaxSequenceLength = 128;
|
||||
```
|
||||
|
||||
### End-to-end flow
|
||||
|
||||
```
|
||||
Prompt → CompositePiiDetector → OnnxNerPiiDetector
|
||||
→ RoutingOnnxNerModelRunner → ScriptRouter
|
||||
→ EnglishOnnxNerRunner / TamilOnnxNerRunner
|
||||
→ OnnxTokenClassifierRunner → PERSON entities
|
||||
→ PlaceholderPiiRedactor → <PERSON_n>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Evidence from Codebase
|
||||
|
||||
### Real-model tests (`Category=RealModel`)
|
||||
|
||||
English direct inference — `RealNerModelRunnerTests`:
|
||||
|
||||
```15:47:tests/PiiRedaction.Infrastructure.Tests/Onnx/RealNerModelRunnerTests.cs
|
||||
[Category("RealModel")]
|
||||
public sealed class RealNerModelRunnerTests : RealNerModelFixture
|
||||
{
|
||||
[TestCase("Customer Ravi Kumar called about billing.", "Ravi", "Ravi Kumar")]
|
||||
[TestCase("Mr. John Smith called about a duplicate debit.", "John", "John Smith")]
|
||||
public void PredictEntities_DetectsPersonWithCorrectSpan(...)
|
||||
```
|
||||
|
||||
English pipeline — `RealNerPipelineTests` (`Category=RealModel`): canonical `FullFinancialWithCustomer`, multi-person, clean-ticket negative.
|
||||
|
||||
Fixture skips when model missing:
|
||||
|
||||
```5:6:tests/TestSupport.Shared/RealNerModelPaths.cs
|
||||
public const string ModelMissingMessage =
|
||||
"ONNX model not found. Run scripts/download-ner-model.ps1 from the repository root.";
|
||||
```
|
||||
|
||||
Resolves `models/en/ner-model.onnx` then `models/ner-model.onnx`.
|
||||
|
||||
### Tamil tests (`Category=TamilNer`)
|
||||
|
||||
Direct Tamil runner — `RealTamilNerModelRunnerTests`:
|
||||
|
||||
```9:69:tests/PiiRedaction.Infrastructure.Tests/Onnx/RealTamilNerModelRunnerTests.cs
|
||||
[Category("TamilNer")]
|
||||
public sealed class RealTamilNerModelRunnerTests : RealTamilNerModelFixture
|
||||
{
|
||||
[TestCase("வாடிக்கையாளர் ராஜேஷ் குமார் அழைத்தார்.", "ராஜேஷ்", "ராஜேஷ் குமார்")]
|
||||
public void PredictEntities_TamilScript_DetectsPersonEntity(...)
|
||||
// ...
|
||||
[TestCase("Customer Senthil phone 9876543210", "Senthil")]
|
||||
public void PredictEntities_TanglishLatinScript_DoesNotInvokeTamilRunner(...)
|
||||
```
|
||||
|
||||
Routed pipeline — `RealTamilPipelineTests` covers Tamil-only, Tanglish (English path), mixed, full financial, and clean Tamil negative.
|
||||
|
||||
### Console samples (`SamplePromptCatalog`)
|
||||
|
||||
Tamil/Tanglish/mixed samples (indices 10–14):
|
||||
|
||||
| Sample | Category | Input excerpt |
|
||||
|--------|----------|---------------|
|
||||
| `TamilCustomerNameOnly` | NER (Tamil) | `வாடிக்கையாளர் ராஜேஷ் குமார் …` |
|
||||
| `TamilWithPhonePan` | NER (Tamil) + Regex | Tamil person + phone + PAN |
|
||||
| `TanglishCustomer` | NER (English/Tanglish) | `Customer Senthil phone 9876543210 …` |
|
||||
| `MixedTamilEnglish` | NER (Mixed) | `வாடிக்கையாளர் Ravi Kumar phone …` |
|
||||
| `TamilFullFinancial` | NER (Tamil) + Regex + Domain | Tamil canonical demo |
|
||||
|
||||
Run all samples (including Tamil) with no flags:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp
|
||||
```
|
||||
|
||||
Or a single Tamil sample:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TamilCustomerNameOnly
|
||||
```
|
||||
|
||||
### Expected console output (English canonical)
|
||||
|
||||
When models are loaded, person names appear as `[PERSON]` with source `Ner`:
|
||||
|
||||
```
|
||||
Detected PII:
|
||||
[PERSON ] Ravi Kumar (Ner)
|
||||
[EMAIL ] ravi.kumar@gmail.com (Regex)
|
||||
...
|
||||
|
||||
Sanitized Prompt:
|
||||
Customer <PERSON_1> with email <EMAIL_1> ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Model Assets Table
|
||||
|
||||
All model binaries are **gitignored**; only `.gitkeep` placeholders are committed.
|
||||
|
||||
| Directory | File | Approx. size | Gitignored | Purpose |
|
||||
|-----------|------|--------------|------------|---------|
|
||||
| `models/` or `models/en/` | `ner-model.onnx` | ~431 MB | Yes | English BERT NER (FP32 ONNX from HF) |
|
||||
| `models/` or `models/en/` | `vocab.txt` | ~213 KB | Yes | WordPiece vocabulary |
|
||||
| `models/` or `models/en/` | `ner-labels.txt` | < 1 KB | Yes | BIO label index (one per line) |
|
||||
| `models/ta/` | `model.onnx` | ~1.2 GB (FP32 export, varies) | Yes | Tamil IndicBERT NER |
|
||||
| `models/ta/` | `vocab.txt` | varies | Yes | WordPiece vocab (preferred tokenizer) |
|
||||
| `models/ta/` | `tokenizer.json` | varies | Yes | HF tokenizer export (optional) |
|
||||
| `models/ta/` | `sentencepiece.bpe.model` | varies | Yes | SentencePiece (if present instead of vocab) |
|
||||
| `models/ta/` | `ner-labels.txt` | few KB | Yes | Fine-grained SampurNER labels |
|
||||
|
||||
`.gitignore` entries:
|
||||
|
||||
```
|
||||
models/*.onnx
|
||||
models/vocab.txt
|
||||
models/ner-labels.txt
|
||||
models/*.json
|
||||
models/en/*
|
||||
models/ta/*
|
||||
```
|
||||
|
||||
English size reference: [docs/git-xenovex-setup.md](git-xenovex-setup.md) notes ~431 MB for the English ONNX file.
|
||||
|
||||
---
|
||||
|
||||
## 8. Limitations
|
||||
|
||||
| Limitation | Detail |
|
||||
|------------|--------|
|
||||
| **Tanglish on English model only** | Roman-script Tanglish (`Customer Senthil`) is classified `LatinOnly` and handled by English BERT. Recall is best-effort and inconsistent for non-standard spellings. Tamil ONNX is **not** invoked on Latin-only text. |
|
||||
| **Fail-open if model missing** | `OnnxNerPiiDetector` and routing runners return `[]` when models are unavailable. Person names are **not** redacted; regex/domain layers still run. No regex fallback for names. |
|
||||
| **128 token limit** | `OnnxTokenClassifierRunner` truncates encoding at 128 tokens. Very long prompts may miss person names beyond the window. |
|
||||
| **PERSON-only NER scope** | Organization, location, and misc NER labels are ignored. Only person spans become `<PERSON_n>`. |
|
||||
| **Mixed-script merge** | When both models run, overlapping spans are deduped by length; shorter overlapping spans are dropped. |
|
||||
| **Tamil ONNX availability** | Pre-exported Tamil ONNX may not exist on Hugging Face; local Python export is often required. |
|
||||
| **No fail-closed mode** | Missing NER does not block sanitization or LLM calls (optional Phase 5 enhancement). |
|
||||
|
||||
---
|
||||
|
||||
## 9. How to Reproduce
|
||||
|
||||
### Download models
|
||||
|
||||
From the repository root:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
.\scripts\download-tamil-ner-model.ps1
|
||||
```
|
||||
|
||||
If Tamil PowerShell download fails with a 404 on `onnx/model.onnx`, install Python 3.12+ and re-run:
|
||||
|
||||
```powershell
|
||||
winget install Python.Python.3.12 --accept-package-agreements --accept-source-agreements
|
||||
.\scripts\download-tamil-ner-model.ps1 -Python "$env:LOCALAPPDATA\Programs\Python\Python312\python.exe"
|
||||
```
|
||||
|
||||
Optional: copy English assets from `models/` to `models/en/` to match `EnglishOnnxModelPath`.
|
||||
|
||||
### Verify with tests
|
||||
|
||||
```bash
|
||||
dotnet build
|
||||
dotnet test
|
||||
dotnet test --filter "Category=RealModel"
|
||||
dotnet test --filter "Category=TamilNer"
|
||||
dotnet test --logger "console;verbosity=detailed"
|
||||
```
|
||||
|
||||
Tests skip gracefully when the corresponding ONNX files are absent.
|
||||
|
||||
### Verify with console
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name CustomerNameOnly
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TamilCustomerNameOnly
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TanglishCustomer
|
||||
```
|
||||
|
||||
### Disable Tamil routing (English-only)
|
||||
|
||||
Set in `appsettings.json`:
|
||||
|
||||
```json
|
||||
"EnableTamilNer": false
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [README.md](../README.md) — build, run, and test overview
|
||||
- [architecture.md](architecture.md) — dual-model routing diagrams and trust boundary
|
||||
- [tamil-tanglish-ner-plan.md](tamil-tanglish-ner-plan.md) — implementation phases and Tanglish expectations
|
||||
557
docs/solution-guide.md
Normal file
557
docs/solution-guide.md
Normal file
@@ -0,0 +1,557 @@
|
||||
# PII Redaction POC — Solution Guide
|
||||
|
||||
Single reference for architecture, NER models, routing, operations, Tamil/Tanglish support, Git setup, and the post-POC improvement backlog.
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
1. [Purpose](#1-purpose)
|
||||
2. [Canonical example](#2-canonical-example)
|
||||
3. [Detection strategies](#3-detection-strategies)
|
||||
4. [Solution architecture](#4-solution-architecture)
|
||||
5. [Trust boundary](#5-trust-boundary)
|
||||
6. [NER models](#6-ner-models)
|
||||
7. [NER routing — English vs Tamil](#7-ner-routing--english-vs-tamil)
|
||||
8. [Configuration and dependency injection](#8-configuration-and-dependency-injection)
|
||||
9. [Running and testing](#9-running-and-testing)
|
||||
10. [Tamil and Tanglish support](#10-tamil-and-tanglish-support)
|
||||
11. [Git remote setup (Xenovex)](#11-git-remote-setup-xenovex)
|
||||
12. [Improvement roadmap](#12-improvement-roadmap)
|
||||
13. [Key source files](#13-key-source-files)
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
This .NET proof-of-concept intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits **only sanitized text** across the LLM trust boundary.
|
||||
|
||||
The solution uses:
|
||||
|
||||
- Clear layer separation (Core / Infrastructure / hosts)
|
||||
- Interface-driven composition and dependency injection
|
||||
- Swappable ONNX NER adapters and `Microsoft.Extensions.AI` chat clients
|
||||
|
||||
It targets financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.
|
||||
|
||||
---
|
||||
|
||||
## 2. Canonical example
|
||||
|
||||
With English and Tamil ONNX models loaded (`scripts/download-ner-model.ps1`, `scripts/download-tamil-ner-model.ps1`):
|
||||
|
||||
| Stage | Value |
|
||||
|-------|-------|
|
||||
| **Input** | `Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.` |
|
||||
| **Sanitized output** | `Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.` |
|
||||
| **Mock LLM response** | `[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.` |
|
||||
|
||||
| Type | Value | Source |
|
||||
|------|-------|--------|
|
||||
| PERSON | Ravi Kumar | NER |
|
||||
| EMAIL | ravi.kumar@gmail.com | Regex |
|
||||
| PHONE | 9876543210 | Regex |
|
||||
| LOAN_NUMBER | LN-456789 | Domain |
|
||||
| PAN | ABCDE1234F | Regex |
|
||||
|
||||
The placeholder map (`<PERSON_1>` → `Ravi Kumar`, etc.) stays **in-process** and is never sent to the LLM.
|
||||
|
||||
---
|
||||
|
||||
## 3. Detection strategies
|
||||
|
||||
| Strategy | Detects | Rationale |
|
||||
|----------|---------|-----------|
|
||||
| **Regex** | Email, phone, PAN, Aadhaar, credit card | Deterministic, format-bound, auditable |
|
||||
| **ONNX NER** | Person names | Contextual; no rigid format |
|
||||
| **Domain rules** | Loan number (`LN-`), customer ID (`CID-`), account (`ACC-`) | Business-specific identifiers |
|
||||
|
||||
**Overlap resolution** (`CompositePiiDetector`): detectors run in order **Domain → Regex → NER**. On overlapping spans, candidates are sorted by start index, length, and source priority (**Domain 3 > Regex 2 > NER 1**); the first non-overlapping candidate wins.
|
||||
|
||||
**Redaction** (`PlaceholderPiiRedactor`): format `<{TYPE}_{n}>`; duplicate values reuse placeholders; replacement is right-to-left to preserve indices.
|
||||
|
||||
---
|
||||
|
||||
## 4. Solution architecture
|
||||
|
||||
### Project structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── PiiRedaction.ConsoleApp/ # Console demo, DI bootstrap
|
||||
├── PiiRedaction.TestHarness.Wpf/ # WPF MVVM manual test harness
|
||||
├── PiiRedaction.Core/ # Detection, redaction, abstractions
|
||||
└── PiiRedaction.Infrastructure/ # ONNX runners, mock LLM
|
||||
models/ # ONNX assets (gitignored)
|
||||
tests/ # NUnit unit and integration tests
|
||||
```
|
||||
|
||||
| Project | Layer | Responsibility |
|
||||
|---------|-------|----------------|
|
||||
| `PiiRedaction.ConsoleApp` | Presentation | Samples, interactive mode, audit output |
|
||||
| `PiiRedaction.TestHarness.Wpf` | Presentation | Category-filtered prompts, redact UI, batch runner |
|
||||
| `PiiRedaction.Core` | Domain | `IPiiDetector`, `IPromptSanitizer`, detectors, models |
|
||||
| `PiiRedaction.Infrastructure` | Infrastructure | `RoutingOnnxNerModelRunner`, `MockLlmPromptService` |
|
||||
| `tests/*` | Test | ~112 tests; `RealModel` and `TamilNer` categories |
|
||||
|
||||
**Dependency direction:** `ConsoleApp` / `Wpf` → `Infrastructure` → `Core`. Core has no ONNX or LLM SDK references.
|
||||
|
||||
### Key abstractions
|
||||
|
||||
| Abstraction | Default implementation | Extension |
|
||||
|-------------|------------------------|-----------|
|
||||
| `IPiiDetector` | `CompositePiiDetector` | Add detector; register in composite order |
|
||||
| `IPiiRedactor` | `PlaceholderPiiRedactor` | Hashing, vault tokens |
|
||||
| `IPromptSanitizer` | `PromptSanitizer` | Orchestrates detect + redact |
|
||||
| `IOnnxNerModelRunner` | `RoutingOnnxNerModelRunner` | Script-based EN/TA routing |
|
||||
| `ILlmPromptService` | `MockLlmPromptService` | Production adapter |
|
||||
| `IChatClient` | `MockChatClient` | Azure OpenAI, etc. |
|
||||
|
||||
### Data flow
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph hosts [Hosts]
|
||||
console[ConsoleApp / WpfHarness]
|
||||
end
|
||||
|
||||
subgraph core [PiiRedaction.Core]
|
||||
sanitizer[PromptSanitizer]
|
||||
composite[CompositePiiDetector]
|
||||
domain[DomainRulePiiDetector]
|
||||
regex[RegexPiiDetector]
|
||||
onnxDet[OnnxNerPiiDetector]
|
||||
redactor[PlaceholderPiiRedactor]
|
||||
end
|
||||
|
||||
subgraph infra [PiiRedaction.Infrastructure]
|
||||
router[RoutingOnnxNerModelRunner]
|
||||
en[EnglishOnnxNerRunner]
|
||||
ta[TamilOnnxNerRunner]
|
||||
llm[MockLlmPromptService]
|
||||
end
|
||||
|
||||
console --> sanitizer
|
||||
sanitizer --> composite
|
||||
composite --> domain
|
||||
composite --> regex
|
||||
composite --> onnxDet
|
||||
onnxDet --> router
|
||||
router --> en
|
||||
router --> ta
|
||||
sanitizer --> redactor
|
||||
console -->|"sanitized text only"| llm
|
||||
```
|
||||
|
||||
**Pipeline:** `Sanitize` → `Detect` (all detectors) → `Redact` → `SanitizationResult`. Optional: `SendPromptAsync(sanitizedPrompt)` to LLM.
|
||||
|
||||
---
|
||||
|
||||
## 5. Trust boundary
|
||||
|
||||
Only the **sanitized prompt string** crosses `ILlmPromptService` / `IChatClient`. Original PII, entity metadata, and the placeholder map remain in-process.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph inProcess [In-Process]
|
||||
raw[Original prompt]
|
||||
entities[Detected entities]
|
||||
map[Placeholder map]
|
||||
audit[Console / WPF display]
|
||||
end
|
||||
|
||||
subgraph boundary [LLM boundary]
|
||||
sanitized[Sanitized prompt only]
|
||||
end
|
||||
|
||||
subgraph external [External LLM]
|
||||
chat[IChatClient]
|
||||
end
|
||||
|
||||
raw --> audit
|
||||
entities --> audit
|
||||
map --> audit
|
||||
sanitized --> chat
|
||||
raw -.->|never sent| chat
|
||||
map -.->|never sent| chat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. NER models
|
||||
|
||||
Person-name detection is the **only** NER responsibility. Structured PII uses regex and domain rules.
|
||||
|
||||
### Routing summary
|
||||
|
||||
| Script in prompt | Model(s) | Typical use |
|
||||
|------------------|----------|-------------|
|
||||
| `LatinOnly` | English | English names, Indian names in Roman script, **Tanglish** |
|
||||
| `TamilOnly` | Tamil | Tamil-script names |
|
||||
| `Mixed` | Both; merge spans | Code-mixed prompts |
|
||||
| `NoLetters` | Neither | Digits/symbols only |
|
||||
|
||||
### English model
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Hugging Face ID | [`dslim/bert-base-NER`](https://huggingface.co/dslim/bert-base-NER) |
|
||||
| Tokenizer | WordPiece (`vocab.txt`, `BertWordPieceEncoder`) |
|
||||
| Person labels | `B-PER`, `I-PER`, `B-PERSON`, `I-PERSON` |
|
||||
| Primary path | `models/en/ner-model.onnx` |
|
||||
| Legacy fallback | `models/ner-model.onnx` |
|
||||
| Download | `.\scripts\download-ner-model.ps1` |
|
||||
|
||||
### Tamil model
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Hugging Face ID | [`prachuryyaIITG/SampurNER_Tamil_IndicBERTv2`](https://huggingface.co/prachuryyaIITG/SampurNER_Tamil_IndicBERTv2) |
|
||||
| Tokenizer | WordPiece when `vocab.txt` present (export path); SentencePiece fallback |
|
||||
| Person labels | Any BIO tag containing `person` (case-insensitive) |
|
||||
| Path | `models/ta/model.onnx` |
|
||||
| Download | `.\scripts\download-tamil-ner-model.ps1` |
|
||||
|
||||
**Why SampurNER over MuRIL:** Tamil-specific NER training, smaller footprint (~0.3B vs ~0.6B), validated ONNX export in this repo. MuRIL reserved as eval-driven fallback.
|
||||
|
||||
### Model assets (gitignored)
|
||||
|
||||
| Directory | Key files | Approx. size |
|
||||
|-----------|-----------|--------------|
|
||||
| `models/en/` | `ner-model.onnx`, `vocab.txt`, `ner-labels.txt` | ~431 MB ONNX |
|
||||
| `models/ta/` | `model.onnx`, `vocab.txt`, `ner-labels.txt` | ~1 GB ONNX |
|
||||
|
||||
Only `models/en/.gitkeep` and `models/ta/.gitkeep` are committed.
|
||||
|
||||
### Shared inference
|
||||
|
||||
Both runners use `OnnxTokenClassifierRunner`:
|
||||
|
||||
- Max sequence length: **128 tokens** (long prompts truncate silently)
|
||||
- BIO decode → `PiiEntityType.Person` only
|
||||
- Missing model → fail-open: `[]` from NER (person names not redacted)
|
||||
|
||||
---
|
||||
|
||||
## 7. NER routing — English vs Tamil
|
||||
|
||||
### Call chain
|
||||
|
||||
```
|
||||
PromptSanitizer → CompositePiiDetector → OnnxNerPiiDetector
|
||||
→ RoutingOnnxNerModelRunner.PredictEntities()
|
||||
→ ScriptRouter.GetComposition(text)
|
||||
→ switch (composition) { English / Tamil / both }
|
||||
→ MergePersonSpans()
|
||||
```
|
||||
|
||||
### Step 1: `ScriptRouter` (Core)
|
||||
|
||||
**File:** `src/PiiRedaction.Core/Detection/ScriptRouter.cs`
|
||||
|
||||
Single pass over characters:
|
||||
|
||||
- Tamil letter: Unicode **U+0B80 – U+0BFF**
|
||||
- Latin letter: `char.IsAsciiLetter`
|
||||
- Both seen → `Mixed` (early exit)
|
||||
- Neither → `NoLetters`
|
||||
- Otherwise → `TamilOnly` or `LatinOnly`
|
||||
|
||||
```csharp
|
||||
// ScriptComposition enum: LatinOnly, TamilOnly, Mixed, NoLetters
|
||||
public ScriptComposition GetComposition(string text) { /* scan chars */ }
|
||||
```
|
||||
|
||||
**Tanglish** in Roman script → `LatinOnly` → **English model only**.
|
||||
|
||||
### Step 2: `RoutingOnnxNerModelRunner` (Infrastructure)
|
||||
|
||||
**File:** `src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs`
|
||||
|
||||
This is the **branch decision**:
|
||||
|
||||
| `ScriptComposition` | English | Tamil |
|
||||
|---------------------|---------|-------|
|
||||
| `LatinOnly` | Yes if available | No |
|
||||
| `TamilOnly` | No | Yes if `EnableTamilNer` and available |
|
||||
| `Mixed` | Yes if available | Yes if `EnableTamilNer` and available |
|
||||
| `NoLetters` | No | No |
|
||||
|
||||
Both models run on the **full prompt text** for `Mixed` (no script segmentation).
|
||||
|
||||
### Step 3: Merge
|
||||
|
||||
`MergePersonSpans`: overlapping PERSON spans → **longer span wins**; ordered by `StartIndex`.
|
||||
|
||||
### Routing diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
text[Prompt text] --> sr[ScriptRouter]
|
||||
sr --> latin[LatinOnly]
|
||||
sr --> tamil[TamilOnly]
|
||||
sr --> mixed[Mixed]
|
||||
sr --> none[NoLetters]
|
||||
latin --> en[EnglishOnnxNerRunner]
|
||||
tamil --> ta[TamilOnnxNerRunner]
|
||||
mixed --> en
|
||||
mixed --> ta
|
||||
en --> merge[MergePersonSpans]
|
||||
ta --> merge
|
||||
none --> empty[No NER]
|
||||
```
|
||||
|
||||
### Worked examples
|
||||
|
||||
| Input style | Composition | Models |
|
||||
|-------------|---------------|--------|
|
||||
| `Customer Ravi Kumar…` | `LatinOnly` | English |
|
||||
| `வாடிக்கையாளர் ராஜேஷ் குமார்…` | `TamilOnly` | Tamil |
|
||||
| `Naan Suresh, phone 9003789456…` | `LatinOnly` | English (Tanglish) |
|
||||
| `வாடிக்கையாளர் Ravi Kumar phone…` | `Mixed` | Both |
|
||||
| `Callback on 9123456780…` | `NoLetters` | Neither (phone via Regex) |
|
||||
|
||||
---
|
||||
|
||||
## 8. Configuration and dependency injection
|
||||
|
||||
### `appsettings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"PiiRedaction": {
|
||||
"OnnxModelPath": "models/ner-model.onnx",
|
||||
"EnglishOnnxModelPath": "models/en/ner-model.onnx",
|
||||
"TamilOnnxModelPath": "models/ta/model.onnx",
|
||||
"EnableTamilNer": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Setting | Effect |
|
||||
|---------|--------|
|
||||
| `EnglishOnnxModelPath` | Primary English model |
|
||||
| `OnnxModelPath` | Legacy English fallback |
|
||||
| `TamilOnnxModelPath` | Tamil model |
|
||||
| `EnableTamilNer` | `false` = English-only routing |
|
||||
|
||||
### DI registration
|
||||
|
||||
```csharp
|
||||
services.AddSingleton<EnglishOnnxNerRunner>();
|
||||
services.AddSingleton<TamilOnnxNerRunner>();
|
||||
services.AddSingleton<IOnnxNerModelRunner, RoutingOnnxNerModelRunner>();
|
||||
// OnnxNerPiiDetector receives IOnnxNerModelRunner (the router)
|
||||
```
|
||||
|
||||
Same pattern in `PiiRedaction.ConsoleApp` and `PiiRedaction.TestHarness.Wpf` `ServiceCollectionExtensions.cs`.
|
||||
|
||||
Detector registration order in composite: **Domain → Regex → ONNX NER**.
|
||||
|
||||
---
|
||||
|
||||
## 9. Running and testing
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
dotnet restore
|
||||
dotnet build
|
||||
```
|
||||
|
||||
### Console app
|
||||
|
||||
```bash
|
||||
# All 16 samples (English + Tamil/Tanglish/mixed)
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp
|
||||
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --list
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TamilCustomerNameOnly
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --interactive
|
||||
```
|
||||
|
||||
Samples: `src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs`
|
||||
|
||||
### WPF test harness (Windows)
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.TestHarness.Wpf
|
||||
```
|
||||
|
||||
- **Category dropdown:** Career Guidance, Banking & Financial, Negative, Edge & Harness
|
||||
- Click prompt → loads input; **Redact** runs pipeline
|
||||
- **Run All** executes scenarios in the selected category
|
||||
- Script badge shows predicted routing (`LatinOnly`, `TamilOnly`, `Mixed`)
|
||||
|
||||
### Download models
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
.\scripts\download-tamil-ner-model.ps1
|
||||
```
|
||||
|
||||
If Tamil PowerShell download 404s, use Python 3.12+:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-tamil-ner-model.ps1 -Python "$env:LOCALAPPDATA\Programs\Python\Python312\python.exe"
|
||||
```
|
||||
|
||||
### Tests
|
||||
|
||||
```bash
|
||||
dotnet test
|
||||
dotnet test --filter "Category=RealModel"
|
||||
dotnet test --filter "Category=TamilNer"
|
||||
dotnet test --filter "FullyQualifiedName~ScriptRouterTests|FullyQualifiedName~RoutingOnnxNerModelRunnerTests"
|
||||
```
|
||||
|
||||
Tests skip gracefully when ONNX files are absent.
|
||||
|
||||
### Console sample categories
|
||||
|
||||
| Category | Examples |
|
||||
|----------|----------|
|
||||
| NER (English) | `CustomerNameOnly`, `MrTitlePerson`, `TwoCustomersInOnePrompt` |
|
||||
| NER (Tamil) | `TamilCustomerNameOnly`, `TamilFullFinancial` |
|
||||
| Tanglish / Mixed | `TanglishCustomer`, `MixedTamilEnglish` |
|
||||
| Regex / Domain | `AllRegexTypes`, `AllDomainIds` |
|
||||
| Negative | `NoPiiCleanTicket` |
|
||||
|
||||
---
|
||||
|
||||
## 10. Tamil and Tanglish support
|
||||
|
||||
### Implementation status
|
||||
|
||||
| Phase | Status | Scope |
|
||||
|-------|--------|-------|
|
||||
| **1** Generic ONNX token classifier | Done | `OnnxTokenClassifierRunner`, encoders, `NerLabelConfig` |
|
||||
| **2** Tamil download + config | Done | Scripts, dual paths, `EnableTamilNer` |
|
||||
| **3** Script routing + DI | Done | `ScriptRouter`, `RoutingOnnxNerModelRunner` |
|
||||
| **4** Tests, samples, docs | Partial | Tests/samples done; eval metrics open |
|
||||
| **5** Optional enhancements | Not started | See below |
|
||||
|
||||
### Remaining gaps (Phase 4–5)
|
||||
|
||||
| Gap | Impact |
|
||||
|-----|--------|
|
||||
| Tamil numeral normalization (௦–௯ → 0–9) | May miss phone/Aadhaar in Tamil script |
|
||||
| Tanglish heuristics (`peru`, `enga peru`) | Better Latin-name recall in Tamil context |
|
||||
| Label-aware regex cues | Contextual name detection |
|
||||
| Fail-closed when NER missing | Compliance hardening |
|
||||
| MuRIL model swap | If Tamil recall insufficient |
|
||||
|
||||
### Tanglish expectations
|
||||
|
||||
| Input | Handler | Expected recall |
|
||||
|-------|---------|-----------------|
|
||||
| Tamil script names | Tamil ONNX | High (with tuning) |
|
||||
| Latin Indian names (`Ravi Kumar`) | English ONNX | High |
|
||||
| Tanglish spellings (`Senthil`) | English NER + optional heuristics | Medium |
|
||||
| Code-mixed prompts | Both models + merge | Medium–high for IDs; names variable |
|
||||
|
||||
### Success metrics (eval set target)
|
||||
|
||||
| Metric | MVP target |
|
||||
|--------|------------|
|
||||
| Tamil script person recall | ≥ 85% |
|
||||
| Tanglish person recall | ≥ 70% |
|
||||
| False positives on clean prompts | ≤ 5% |
|
||||
| Structured PII in Tamil prompts (regex) | ≥ 95% |
|
||||
| English canonical regression | 100% |
|
||||
|
||||
---
|
||||
|
||||
## 11. Git remote setup (Xenovex)
|
||||
|
||||
Remote: `https://xts.xenovex.com/Bilal-Nazer-Ali/llm-pii-poc.git`
|
||||
|
||||
### Create and push
|
||||
|
||||
```powershell
|
||||
cd C:\Users\bilal.n\Projects\llm-pii-poc
|
||||
$git = "C:\Program Files\Git\bin\git.exe"
|
||||
|
||||
& $git remote add origin <YOUR_CLONE_URL>
|
||||
& $git branch -M main
|
||||
& $git push -u origin main
|
||||
```
|
||||
|
||||
Use an **empty** remote repository (no README) when pushing existing history.
|
||||
|
||||
### Committed vs gitignored
|
||||
|
||||
| Committed | Gitignored |
|
||||
|-----------|------------|
|
||||
| `src/`, `tests/`, `scripts/`, `docs/` | `bin/`, `obj/`, `.vs/` |
|
||||
| `README.md`, `PiiRedaction.slnx` | `models/*.onnx`, `vocab.txt`, `models/en/*`, `models/ta/*` |
|
||||
| `models/en/.gitkeep`, `models/ta/.gitkeep` | `scratch/` |
|
||||
|
||||
After clone, run model download scripts locally.
|
||||
|
||||
---
|
||||
|
||||
## 12. Improvement roadmap
|
||||
|
||||
Post-POC items **not yet implemented**. Current maturity: strong architecture and tests; production needs fail-closed policy, API host, and observability.
|
||||
|
||||
### P0 — Security and correctness
|
||||
|
||||
| Item | Proposal |
|
||||
|------|----------|
|
||||
| Fail-closed when NER unavailable | `RequireNerOnStartup`, `BlockLlmWhenNerUnavailable` options |
|
||||
| Outbound LLM guard | Verify sanitized text before `IChatClient` |
|
||||
| Truncation warning | Surface 128-token limit in `SanitizationResult` |
|
||||
| NER confidence | Populate `PiiEntity.Confidence` or hide UI column |
|
||||
| WPF leak check | Use placeholder map, not naive `Contains` |
|
||||
|
||||
### P1 — Platform
|
||||
|
||||
| Item | Proposal |
|
||||
|------|----------|
|
||||
| `PiiRedaction.Composition` | Shared `AddPiiRedactionServices` (Console + WPF duplicate today) |
|
||||
| Unified prompt catalog | Single source for console, WPF, golden tests |
|
||||
| Async `IPromptSanitizer` | Replace WPF `Task.Run` wrapper |
|
||||
| `PiiRedaction.Application` | Shared orchestration for API/WPF |
|
||||
| Minimal API + health checks | `POST /v1/prompts/sanitize`, model readiness |
|
||||
| Tamil Phase 4 | Numeral normalization, Tanglish heuristics |
|
||||
|
||||
### P2 — Operations
|
||||
|
||||
| Item | Proposal |
|
||||
|------|----------|
|
||||
| Placeholder audit store | TTL, encryption, correlation ID |
|
||||
| Observability | Metrics, OpenTelemetry traces |
|
||||
| ONNX session pool | Concurrency strategy for API load |
|
||||
| CI pipeline | Fast tests without models; nightly real-model job |
|
||||
|
||||
### Suggested phases
|
||||
|
||||
1. **Production hardening** — fail-closed, composition root, API, truncation metadata (~1–2 weeks)
|
||||
2. **Detection quality** — normalizer, Tamil Phase 4, regex hardening (~1 week)
|
||||
3. **Operations** — audit store, metrics, session pool (~1–2 weeks)
|
||||
|
||||
---
|
||||
|
||||
## 13. Key source files
|
||||
|
||||
| Topic | Path |
|
||||
|-------|------|
|
||||
| Script classification | `src/PiiRedaction.Core/Detection/ScriptRouter.cs` |
|
||||
| **Model branch decision** | `src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs` |
|
||||
| English NER | `src/PiiRedaction.Infrastructure/Onnx/EnglishOnnxNerRunner.cs` |
|
||||
| Tamil NER | `src/PiiRedaction.Infrastructure/Onnx/TamilOnnxNerRunner.cs` |
|
||||
| Shared ONNX inference | `src/PiiRedaction.Infrastructure/Onnx/OnnxTokenClassifierRunner.cs` |
|
||||
| NER detector | `src/PiiRedaction.Core/Detection/OnnxNerPiiDetector.cs` |
|
||||
| Sanitizer | `src/PiiRedaction.Core/Sanitization/PromptSanitizer.cs` |
|
||||
| Composite merge | `src/PiiRedaction.Core/Detection/CompositePiiDetector.cs` |
|
||||
| Options | `src/PiiRedaction.Core/Configuration/PiiRedactionOptions.cs` |
|
||||
| DI | `src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs` |
|
||||
| Console samples | `src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs` |
|
||||
| WPF catalog | `src/PiiRedaction.TestHarness.Wpf/Services/TestPromptCatalog.cs` |
|
||||
| Routing tests | `tests/PiiRedaction.Infrastructure.Tests/Onnx/RoutingOnnxNerModelRunnerTests.cs` |
|
||||
| Script tests | `tests/PiiRedaction.Core.Tests/Detection/ScriptRouterTests.cs` |
|
||||
|
||||
---
|
||||
|
||||
*Last consolidated: July 2026. Replaces separate architecture, NER models, routing reference, Tamil plan, improvement roadmap, and Git setup documents.*
|
||||
@@ -1,307 +0,0 @@
|
||||
# Tamil / Tanglish NER — Implementation Plan
|
||||
|
||||
**Goal:** Raise language coverage from ~15% to production-viable for Tamil script and Tanglish (Roman-script Tamil-English) customer prompts, without changing the secure LLM boundary pattern.
|
||||
|
||||
**Status:** Phase 1–3 implemented
|
||||
**Approach:** Dual-model ONNX NER routing (English + Tamil) + lightweight text normalization + optional Tanglish heuristics
|
||||
**Estimated effort:** 4–6 engineering days across 4 phases
|
||||
|
||||
---
|
||||
|
||||
## 1. Current State vs Gap
|
||||
|
||||
| Capability | Today | Tamil script | Tanglish (Latin) |
|
||||
|------------|-------|--------------|------------------|
|
||||
| Phone, PAN, Aadhaar, email, domain IDs | Regex + domain rules | Works (ASCII digits) | Works |
|
||||
| Person names | `dslim/bert-base-NER` (English BERT) | **Fails** — out of vocabulary | **Partial** — inconsistent |
|
||||
| Script / language routing | None | N/A | N/A |
|
||||
| Tamil numerals (௦–௯) | Not normalized | **May miss** phone/Aadhaar | N/A |
|
||||
| Label-aware cues (`பெயர்`, `peru`, `enga peru`) | None | **Misses** contextual names | **Misses** |
|
||||
|
||||
**Root cause:** Person detection is a single English-only ONNX model behind `IOnnxNerModelRunner` → `OnnxNerPiiDetector`. Regex/domain layers are already language-agnostic.
|
||||
|
||||
---
|
||||
|
||||
## 2. Target Architecture
|
||||
|
||||
No change to the trust boundary: `PromptSanitizer` → `CompositePiiDetector` → `PlaceholderPiiRedactor` → sanitized text only to LLM.
|
||||
|
||||
Only the **NER adapter** expands:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
text["Prompt text"]
|
||||
router["ScriptRouter (Core)"]
|
||||
routing["RoutingOnnxNerModelRunner"]
|
||||
en["EnglishOnnxRunner\nBERT WordPiece"]
|
||||
ta["TamilOnnxRunner\nIndicBERT SentencePiece"]
|
||||
nerDet["OnnxNerPiiDetector"]
|
||||
composite["CompositePiiDetector"]
|
||||
|
||||
text --> composite
|
||||
text --> nerDet
|
||||
nerDet --> routing
|
||||
routing --> router
|
||||
router -->|"LatinOnly / Mixed"| en
|
||||
router -->|"TamilOnly / Mixed"| ta
|
||||
en --> routing
|
||||
ta --> routing
|
||||
```
|
||||
|
||||
### Routing rules
|
||||
|
||||
| `ScriptComposition` | Models invoked | Tanglish note |
|
||||
|---------------------|----------------|---------------|
|
||||
| `LatinOnly` | English NER only | Tanglish names in Roman script |
|
||||
| `TamilOnly` | Tamil NER only | Tamil script names |
|
||||
| `Mixed` | **Both**, merge person spans | Common in Indian CS prompts |
|
||||
| `NoLetters` | Neither (or English fallback off) | Digits-only prompts |
|
||||
|
||||
**Merge inside `RoutingOnnxNerModelRunner`:** dedupe overlapping person spans (prefer longer span; tie-break Tamil vs English by start index order).
|
||||
|
||||
---
|
||||
|
||||
## 3. Model Selection
|
||||
|
||||
| Role | Model | Rationale |
|
||||
|------|-------|-----------|
|
||||
| English / Tanglish (Latin) | **Keep** `dslim/bert-base-NER` | Already integrated; works for many Indian names in Latin script |
|
||||
| Tamil script | **`prachuryyaIITG/SampurNER_Tamil_IndicBERTv2`** | Tamil NER; lighter than MuRIL (~0.6B) |
|
||||
| Fallback (optional Phase 5) | MuRIL Tamil NER | Only if IndicBERT recall is insufficient on eval set |
|
||||
|
||||
### Asset layout
|
||||
|
||||
```
|
||||
models/
|
||||
en/
|
||||
ner-model.onnx # or model.onnx (BERT export)
|
||||
vocab.txt
|
||||
ner-labels.txt
|
||||
ta/
|
||||
model.onnx
|
||||
sentencepiece.bpe.model # or tokenizer.json from HF export
|
||||
ner-labels.txt
|
||||
ner-model.onnx # legacy path — keep for backward compatibility
|
||||
```
|
||||
|
||||
### Label mapping (Tamil fine-grained NER)
|
||||
|
||||
SampurNER uses fine-grained tags (e.g. `B-person-politician`, `I-person-artist`). Map **any label containing `person`** (case-insensitive) → `PiiEntityType.Person`.
|
||||
|
||||
English labels remain: `B-PER`, `I-PER`, `B-PERSON`, `I-PERSON`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Phases
|
||||
|
||||
### Phase 1 — Generic ONNX token classifier (1–2 days)
|
||||
|
||||
**Objective:** Refactor `OnnxNerModelRunner` so BERT and SentencePiece are pluggable.
|
||||
|
||||
| Action | Location |
|
||||
|--------|----------|
|
||||
| Add `ITokenClassifierEncoder` + `EncodedSequence` | `Infrastructure/Onnx/` |
|
||||
| `BertWordPieceEncoder` — extract from current runner | Infrastructure |
|
||||
| `SentencePieceEncoder` — IndicBERT tokenizer | Infrastructure |
|
||||
| `OnnxTokenClassifierRunner` — shared inference + BIO decode | Infrastructure |
|
||||
| `NerLabelConfig` — English vs Tamil person label predicates | Infrastructure |
|
||||
| `OnnxAssetPathResolver` — resolve model dir from repo root | Infrastructure |
|
||||
| Thin wrappers: `EnglishOnnxNerRunner`, `TamilOnnxNerRunner` | Infrastructure |
|
||||
|
||||
**Backward compat:** If `models/en/` missing, fall back to `OnnxModelPath` (`models/ner-model.onnx`).
|
||||
|
||||
**No behavior change** until Phase 3 wiring — existing tests must pass.
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — Tamil model download + config (0.5–1 day)
|
||||
|
||||
| Action | Details |
|
||||
|--------|---------|
|
||||
| `scripts/download-tamil-ner-model.ps1` + `.py` | Mirror `download-ner-model.ps1`; export via `optimum-cli export onnx --task token-classification` |
|
||||
| Optional: `scripts/download-all-ner-models.ps1` | Calls English + Tamil scripts |
|
||||
| Extend `PiiRedactionOptions` | `EnglishOnnxModelPath`, `TamilOnnxModelPath`, `EnableTamilNer` (default `true`) |
|
||||
| Update `appsettings.json` | New paths under `PiiRedaction` section |
|
||||
| `.gitignore` | `models/ta/*`, `models/en/*` (same as today for onnx/vocab) |
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — Script routing + DI (0.5–1 day)
|
||||
|
||||
| Action | Location |
|
||||
|--------|----------|
|
||||
| `ScriptRouter` + `ScriptComposition` enum | `Core/Detection/` |
|
||||
| `RoutingOnnxNerModelRunner` implements `IOnnxNerModelRunner` | Infrastructure |
|
||||
| DI registration | `ServiceCollectionExtensions.cs` |
|
||||
|
||||
```csharp
|
||||
services.AddSingleton<EnglishOnnxNerRunner>();
|
||||
services.AddSingleton<TamilOnnxNerRunner>();
|
||||
services.AddSingleton<IOnnxNerModelRunner, RoutingOnnxNerModelRunner>();
|
||||
```
|
||||
|
||||
`OnnxNerPiiDetector` and `CompositePiiDetector` **unchanged**.
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — Tests, samples, docs (1 day)
|
||||
|
||||
#### Unit tests
|
||||
|
||||
| Test class | Coverage |
|
||||
|------------|----------|
|
||||
| `ScriptRouterTests` | Tamil-only, Latin-only, mixed, no-letters, boundary chars U+0B80/U+0BFF |
|
||||
| `RoutingOnnxNerModelRunnerTests` | Fake EN/TA runners; mixed script merges both |
|
||||
| `NerLabelConfigTests` | Tamil fine-grained person labels map correctly |
|
||||
|
||||
#### Real-model tests (`Category=RealModel` or `Category=TamilNer`)
|
||||
|
||||
| Scenario | Input example | Assert |
|
||||
|----------|---------------|--------|
|
||||
| Tamil name | `வாடிக்கையாளர் ராஜேஷ் தொலைபேசி 9876543210` | `<PERSON_1>`, phone redacted |
|
||||
| Tanglish name | `Customer Senthil phone 9876543210` | person + phone (best-effort) |
|
||||
| Mixed | `Rajesh மற்றும் Priya` | both persons redacted |
|
||||
| Clean Tamil | `பணத்தை திரும்பப் பெறுவது எப்படி?` | no false positives |
|
||||
| Canonical English | existing golden tests | no regression |
|
||||
|
||||
Skip gracefully when `models/ta/model.onnx` missing (mirror `RealNerModelFixture`).
|
||||
|
||||
#### Console samples
|
||||
|
||||
Add to `SamplePromptCatalog.cs`:
|
||||
|
||||
- `TamilCustomerName` — Tamil script person
|
||||
- `TanglishCustomerName` — `enga peru Rajesh` / `Customer Senthil`
|
||||
- `MixedTamilEnglish` — code-mixed prompt
|
||||
|
||||
#### Docs
|
||||
|
||||
- Update `README.md` ONNX setup (dual models)
|
||||
- Update `docs/architecture.md` NER section
|
||||
- Link this plan from README
|
||||
|
||||
---
|
||||
|
||||
### Phase 5 — Optional enhancements (post-MVP)
|
||||
|
||||
| Enhancement | Benefit | Effort |
|
||||
|-------------|---------|--------|
|
||||
| **Unicode digit normalization** pre-pass | Tamil numerals → ASCII for regex | 0.5 day |
|
||||
| **Label-based regex** (`பெயர்`, `peru`, `peyar`, `enga peru`) | Tanglish recall without ML | 0.5 day |
|
||||
| **Tamil name gazetteer** `IPiiDetector` | High precision for top names | 1 day |
|
||||
| **Fail-closed policy** when NER unavailable | Compliance option | 0.5 day |
|
||||
| MuRIL model swap | Higher Tamil recall | eval-driven |
|
||||
|
||||
---
|
||||
|
||||
## 5. Tanglish — Realistic Expectations
|
||||
|
||||
| Input type | Primary handler | Expected recall |
|
||||
|------------|-----------------|-----------------|
|
||||
| Tamil script names | Tamil ONNX NER | High (with eval tuning) |
|
||||
| Standard Latin Indian names (`Ravi Kumar`) | English ONNX NER | High (already works) |
|
||||
| Tanglish spellings (`Senthil`, `senthil`, `Centhil`) | English NER + optional gazetteer | Medium |
|
||||
| Code-mixed (`Rajesh oda account ACC-123456`) | English NER + domain regex | Medium–high for IDs; name variable |
|
||||
|
||||
**MVP target:** Tamil script person names reliably redacted; Tanglish improved but not 100% without Phase 5 heuristics.
|
||||
|
||||
---
|
||||
|
||||
## 6. Success Metrics
|
||||
|
||||
Before marking language gap closed, run an **eval set of 20–30 real prompts** (anonymized production samples):
|
||||
|
||||
| Metric | MVP target |
|
||||
|--------|------------|
|
||||
| Tamil script person-name recall | ≥ 85% |
|
||||
| Tanglish person-name recall | ≥ 70% (with English model + optional heuristics) |
|
||||
| False positive rate (clean prompts) | ≤ 5% |
|
||||
| Structured PII (phone/PAN/domain) in Tamil prompts | ≥ 95% (regex layer) |
|
||||
| Regression on English canonical demo | 100% (existing golden tests) |
|
||||
|
||||
---
|
||||
|
||||
## 7. Files to Create / Modify (checklist)
|
||||
|
||||
### New files
|
||||
|
||||
- `src/PiiRedaction.Core/Detection/ScriptRouter.cs`
|
||||
- `src/PiiRedaction.Core/Detection/ScriptComposition.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/OnnxTokenClassifierRunner.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/BertWordPieceEncoder.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/SentencePieceEncoder.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/ITokenClassifierEncoder.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/NerLabelConfig.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/OnnxAssetPathResolver.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/EnglishOnnxNerRunner.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/TamilOnnxNerRunner.cs`
|
||||
- `src/PiiRedaction.Infrastructure/Onnx/RoutingOnnxNerModelRunner.cs`
|
||||
- `scripts/download-tamil-ner-model.ps1` / `.py`
|
||||
- `tests/.../ScriptRouterTests.cs`
|
||||
- `tests/.../RoutingOnnxNerModelRunnerTests.cs`
|
||||
- `tests/.../RealTamilNerPipelineTests.cs`
|
||||
- `tests/TestSupport.Shared/RealTamilModelFixture.cs`
|
||||
|
||||
### Modified files
|
||||
|
||||
- `PiiRedactionOptions.cs` — dual model paths
|
||||
- `ServiceCollectionExtensions.cs` — routing DI
|
||||
- `appsettings.json` — config
|
||||
- `SamplePromptCatalog.cs` — Tamil/Tanglish demos
|
||||
- `ProductionPipelineFactory.cs` — `CreateWithRoutingRealModel()` for tests
|
||||
- `RealNerModelFixture.cs` / paths — support EN + TA
|
||||
- `README.md`, `docs/architecture.md`
|
||||
- `.gitignore` — `models/en/`, `models/ta/`
|
||||
|
||||
### Unchanged (by design)
|
||||
|
||||
- `PromptSanitizer`, `PlaceholderPiiRedactor`, `CompositePiiDetector`
|
||||
- `RegexPiiDetector`, `DomainRulePiiDetector`
|
||||
- `MockLlmPromptService` / LLM boundary
|
||||
|
||||
---
|
||||
|
||||
## 8. Rollout & Risk
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Tamil model export fails on Windows | PowerShell fallback downloads pre-exported ONNX from Hugging Face |
|
||||
| Larger memory (two models) | Lazy-load Tamil runner only when `EnableTamilNer` and Tamil script detected |
|
||||
| Fine-grained label mismatch | Load labels from `ner-labels.txt`; unit test label config |
|
||||
| Tanglish disappointment | Set stakeholder expectation in README; Phase 5 heuristics |
|
||||
| CI without models | Fast tests use fakes; `Category=TamilNer` skips like `RealModel` |
|
||||
|
||||
**Feature flag:** `EnableTamilNer=false` reverts to English-only behavior for gradual rollout.
|
||||
|
||||
---
|
||||
|
||||
## 9. Command Reference (after implementation)
|
||||
|
||||
```powershell
|
||||
# Download both models
|
||||
.\scripts\download-ner-model.ps1
|
||||
.\scripts\download-tamil-ner-model.ps1
|
||||
|
||||
# Run Tamil-focused console sample
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name TamilCustomerName
|
||||
|
||||
# Tests
|
||||
dotnet test
|
||||
dotnet test --filter "Category=TamilNer"
|
||||
dotnet test --filter "Category=RealModel"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Approval Checklist
|
||||
|
||||
- [ ] Stakeholder sign-off on dual-model approach (vs single multilingual model)
|
||||
- [ ] Tamil eval prompt set collected (20–30 samples)
|
||||
- [ ] Xenovex CI policy: models downloaded in pipeline or tests skip
|
||||
- [x] Phase 1–3 implementation PR
|
||||
- [ ] Phase 4 eval metrics met
|
||||
- [ ] Optional Phase 5 for Tanglish heuristics if recall < 70%
|
||||
|
||||
---
|
||||
|
||||
**Next step:** Implement Phase 1 in a feature branch (`feature/tamil-tanglish-ner`), open PR to `main` on `xts.xenovex.com/Bilal-Nazer-Ali/llm-pii-poc`.
|
||||
@@ -21,7 +21,7 @@ foreach (var t in tokens) Console.WriteLine($"{t.Id}\t{t.Value}");
|
||||
var runnerOptions = Options.Create(new PiiRedactionOptions { TamilOnnxModelPath = Path.Combine(modelDir, "model.onnx") });
|
||||
using var runner = new TamilOnnxNerRunner(runnerOptions, NullLogger<TamilOnnxNerRunner>.Instance);
|
||||
Console.WriteLine($"Available: {runner.IsModelAvailable}");
|
||||
foreach (var e in runner.PredictEntities(text))
|
||||
foreach (var e in runner.PredictEntities(text).Entities)
|
||||
{
|
||||
Console.WriteLine($"Entity: '{e.Value}' [{e.StartIndex},{e.Length}]");
|
||||
}
|
||||
|
||||
@@ -7,15 +7,20 @@ namespace PiiRedaction.Core.Detection;
|
||||
/// Aggregates multiple PII detectors and merges overlapping spans.
|
||||
/// Detectors are applied in registration order; earlier detectors win on overlap.
|
||||
/// </summary>
|
||||
public sealed class CompositePiiDetector : IPiiDetector
|
||||
public sealed class CompositePiiDetector : IPiiDetector, INerRoutingSource
|
||||
{
|
||||
private readonly IReadOnlyList<IPiiDetector> _detectors;
|
||||
private readonly INerRoutingSource? _nerRoutingSource;
|
||||
|
||||
public CompositePiiDetector(IEnumerable<IPiiDetector> detectors)
|
||||
{
|
||||
_detectors = detectors.ToList();
|
||||
_nerRoutingSource = _detectors.OfType<INerRoutingSource>().FirstOrDefault();
|
||||
}
|
||||
|
||||
public IReadOnlyList<NerModelOrigin> LastInvokedModels =>
|
||||
_nerRoutingSource?.LastInvokedModels ?? [];
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
8
src/PiiRedaction.Core/Detection/INerRoutingSource.cs
Normal file
8
src/PiiRedaction.Core/Detection/INerRoutingSource.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
public interface INerRoutingSource
|
||||
{
|
||||
IReadOnlyList<NerModelOrigin> LastInvokedModels { get; }
|
||||
}
|
||||
@@ -7,5 +7,5 @@ public interface IOnnxNerModelRunner
|
||||
{
|
||||
bool IsModelAvailable { get; }
|
||||
|
||||
IReadOnlyList<PiiEntity> PredictEntities(string text);
|
||||
NerPredictionResult PredictEntities(string text);
|
||||
}
|
||||
|
||||
11
src/PiiRedaction.Core/Detection/NerEntityTagging.cs
Normal file
11
src/PiiRedaction.Core/Detection/NerEntityTagging.cs
Normal file
@@ -0,0 +1,11 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
public static class NerEntityTagging
|
||||
{
|
||||
public static IReadOnlyList<PiiEntity> WithOrigin(
|
||||
IReadOnlyList<PiiEntity> entities,
|
||||
NerModelOrigin origin) =>
|
||||
entities.Select(entity => entity with { ModelOrigin = origin }).ToList();
|
||||
}
|
||||
10
src/PiiRedaction.Core/Detection/NerPredictionResult.cs
Normal file
10
src/PiiRedaction.Core/Detection/NerPredictionResult.cs
Normal file
@@ -0,0 +1,10 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
public sealed record NerPredictionResult(
|
||||
IReadOnlyList<PiiEntity> Entities,
|
||||
IReadOnlyList<NerModelOrigin> InvokedModels)
|
||||
{
|
||||
public static NerPredictionResult Empty { get; } = new([], []);
|
||||
}
|
||||
@@ -8,7 +8,7 @@ namespace PiiRedaction.Core.Detection;
|
||||
/// NER is used for entities that lack rigid formats and vary in surface form across prompts.
|
||||
/// Requires a loaded ONNX model; returns no person entities when the model is unavailable.
|
||||
/// </summary>
|
||||
public sealed class OnnxNerPiiDetector : IPiiDetector
|
||||
public sealed class OnnxNerPiiDetector : IPiiDetector, INerRoutingSource
|
||||
{
|
||||
private readonly IOnnxNerModelRunner _modelRunner;
|
||||
|
||||
@@ -17,15 +17,20 @@ public sealed class OnnxNerPiiDetector : IPiiDetector
|
||||
_modelRunner = modelRunner;
|
||||
}
|
||||
|
||||
public IReadOnlyList<NerModelOrigin> LastInvokedModels { get; private set; } = [];
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
if (!_modelRunner.IsModelAvailable)
|
||||
{
|
||||
LastInvokedModels = [];
|
||||
return [];
|
||||
}
|
||||
|
||||
return _modelRunner.PredictEntities(text);
|
||||
var result = _modelRunner.PredictEntities(text);
|
||||
LastInvokedModels = result.InvokedModels;
|
||||
return result.Entities;
|
||||
}
|
||||
}
|
||||
|
||||
7
src/PiiRedaction.Core/Models/NerModelOrigin.cs
Normal file
7
src/PiiRedaction.Core/Models/NerModelOrigin.cs
Normal file
@@ -0,0 +1,7 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public enum NerModelOrigin
|
||||
{
|
||||
English,
|
||||
Tamil
|
||||
}
|
||||
@@ -6,7 +6,8 @@ public sealed record PiiEntity(
|
||||
int StartIndex,
|
||||
int Length,
|
||||
PiiDetectionSource Source,
|
||||
double? Confidence = null)
|
||||
double? Confidence = null,
|
||||
NerModelOrigin? ModelOrigin = null)
|
||||
{
|
||||
public int EndIndex => StartIndex + Length;
|
||||
}
|
||||
|
||||
@@ -4,4 +4,5 @@ public sealed record SanitizationResult(
|
||||
string OriginalPrompt,
|
||||
string SanitizedPrompt,
|
||||
IReadOnlyList<PiiEntity> DetectedEntities,
|
||||
RedactionResult Redaction);
|
||||
RedactionResult Redaction,
|
||||
IReadOnlyList<NerModelOrigin> NerModelsInvoked);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Sanitization;
|
||||
@@ -21,11 +22,15 @@ public sealed class PromptSanitizer : IPromptSanitizer
|
||||
|
||||
var entities = _detector.Detect(request.OriginalPrompt);
|
||||
var redaction = _redactor.Redact(request.OriginalPrompt, entities);
|
||||
var nerModelsInvoked = _detector is INerRoutingSource routingSource
|
||||
? routingSource.LastInvokedModels
|
||||
: [];
|
||||
|
||||
return new SanitizationResult(
|
||||
request.OriginalPrompt,
|
||||
redaction.SanitizedText,
|
||||
entities,
|
||||
redaction);
|
||||
redaction,
|
||||
nerModelsInvoked);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -29,7 +29,18 @@ public class EnglishOnnxNerRunner : IOnnxNerModelRunner, IDisposable
|
||||
|
||||
public bool IsModelAvailable => _runner.IsAvailable;
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text) => _runner.PredictEntities(text);
|
||||
public NerPredictionResult PredictEntities(string text)
|
||||
{
|
||||
if (!IsModelAvailable)
|
||||
{
|
||||
return NerPredictionResult.Empty;
|
||||
}
|
||||
|
||||
var entities = _runner.PredictEntities(text);
|
||||
return new NerPredictionResult(
|
||||
NerEntityTagging.WithOrigin(entities, NerModelOrigin.English),
|
||||
[NerModelOrigin.English]);
|
||||
}
|
||||
|
||||
public void Dispose() => _runner.Dispose();
|
||||
}
|
||||
|
||||
@@ -50,7 +50,23 @@ public sealed class OnnxTokenClassifierRunner : IDisposable
|
||||
}
|
||||
|
||||
var predictedLabelIds = RunInference(encoded);
|
||||
return DecodePersonEntities(text, predictedLabelIds, encoded);
|
||||
var entities = DecodePersonEntities(text, predictedLabelIds, encoded);
|
||||
|
||||
if (entities.Count == 0)
|
||||
{
|
||||
_logger.LogDebug("Decoded 0 person span(s) from {ModelPath}.", _modelPath);
|
||||
}
|
||||
else
|
||||
{
|
||||
var spanSummary = string.Join(", ", entities.Select(entity => $"\"{entity.Value}\"@{entity.StartIndex}"));
|
||||
_logger.LogDebug(
|
||||
"Decoded {Count} person span(s) from {ModelPath}: {Spans}",
|
||||
entities.Count,
|
||||
_modelPath,
|
||||
spanSummary);
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
|
||||
private int[] RunInference(EncodedSequence encoded)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Core.Detection;
|
||||
@@ -14,39 +16,53 @@ public sealed class RoutingOnnxNerModelRunner : IOnnxNerModelRunner
|
||||
private readonly IOnnxNerModelRunner _englishRunner;
|
||||
private readonly IOnnxNerModelRunner _tamilRunner;
|
||||
private readonly bool _enableTamilNer;
|
||||
private readonly ILogger<RoutingOnnxNerModelRunner> _logger;
|
||||
|
||||
public RoutingOnnxNerModelRunner(
|
||||
EnglishOnnxNerRunner englishRunner,
|
||||
TamilOnnxNerRunner tamilRunner,
|
||||
IOptions<PiiRedactionOptions> options)
|
||||
: this(englishRunner, tamilRunner, options.Value.EnableTamilNer)
|
||||
IOptions<PiiRedactionOptions> options,
|
||||
ILogger<RoutingOnnxNerModelRunner> logger)
|
||||
: this(englishRunner, tamilRunner, options.Value.EnableTamilNer, logger)
|
||||
{
|
||||
}
|
||||
|
||||
internal RoutingOnnxNerModelRunner(
|
||||
IOnnxNerModelRunner englishRunner,
|
||||
IOnnxNerModelRunner tamilRunner,
|
||||
bool enableTamilNer)
|
||||
bool enableTamilNer,
|
||||
ILogger<RoutingOnnxNerModelRunner>? logger = null)
|
||||
{
|
||||
_englishRunner = englishRunner;
|
||||
_tamilRunner = tamilRunner;
|
||||
_enableTamilNer = enableTamilNer;
|
||||
_logger = logger ?? NullLogger<RoutingOnnxNerModelRunner>.Instance;
|
||||
}
|
||||
|
||||
public bool IsModelAvailable =>
|
||||
_englishRunner.IsModelAvailable || (_enableTamilNer && _tamilRunner.IsModelAvailable);
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
public NerPredictionResult PredictEntities(string text)
|
||||
{
|
||||
var composition = _scriptRouter.GetComposition(text);
|
||||
_logger.LogDebug("Script composition: {Composition}", composition);
|
||||
|
||||
var entities = new List<PiiEntity>();
|
||||
var invoked = new List<NerModelOrigin>();
|
||||
|
||||
switch (composition)
|
||||
{
|
||||
case ScriptComposition.LatinOnly:
|
||||
if (_englishRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_englishRunner.PredictEntities(text));
|
||||
_logger.LogDebug("Invoking English NER model.");
|
||||
var englishResult = _englishRunner.PredictEntities(text);
|
||||
entities.AddRange(englishResult.Entities);
|
||||
invoked.AddRange(englishResult.InvokedModels);
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.LogDebug("Skipping English NER model (not available).");
|
||||
}
|
||||
|
||||
break;
|
||||
@@ -54,7 +70,18 @@ public sealed class RoutingOnnxNerModelRunner : IOnnxNerModelRunner
|
||||
case ScriptComposition.TamilOnly:
|
||||
if (_enableTamilNer && _tamilRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_tamilRunner.PredictEntities(text));
|
||||
_logger.LogDebug("Invoking Tamil NER model.");
|
||||
var tamilResult = _tamilRunner.PredictEntities(text);
|
||||
entities.AddRange(tamilResult.Entities);
|
||||
invoked.AddRange(tamilResult.InvokedModels);
|
||||
}
|
||||
else if (!_enableTamilNer)
|
||||
{
|
||||
_logger.LogDebug("Skipping Tamil NER model (disabled in options).");
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.LogDebug("Skipping Tamil NER model (not available).");
|
||||
}
|
||||
|
||||
break;
|
||||
@@ -62,21 +89,46 @@ public sealed class RoutingOnnxNerModelRunner : IOnnxNerModelRunner
|
||||
case ScriptComposition.Mixed:
|
||||
if (_englishRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_englishRunner.PredictEntities(text));
|
||||
_logger.LogDebug("Invoking English NER model (mixed script).");
|
||||
var englishResult = _englishRunner.PredictEntities(text);
|
||||
entities.AddRange(englishResult.Entities);
|
||||
invoked.AddRange(englishResult.InvokedModels);
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.LogDebug("Skipping English NER model (not available, mixed script).");
|
||||
}
|
||||
|
||||
if (_enableTamilNer && _tamilRunner.IsModelAvailable)
|
||||
{
|
||||
entities.AddRange(_tamilRunner.PredictEntities(text));
|
||||
_logger.LogDebug("Invoking Tamil NER model (mixed script).");
|
||||
var tamilResult = _tamilRunner.PredictEntities(text);
|
||||
entities.AddRange(tamilResult.Entities);
|
||||
invoked.AddRange(tamilResult.InvokedModels);
|
||||
}
|
||||
else if (!_enableTamilNer)
|
||||
{
|
||||
_logger.LogDebug("Skipping Tamil NER model (disabled in options, mixed script).");
|
||||
}
|
||||
else
|
||||
{
|
||||
_logger.LogDebug("Skipping Tamil NER model (not available, mixed script).");
|
||||
}
|
||||
|
||||
break;
|
||||
|
||||
case ScriptComposition.NoLetters:
|
||||
_logger.LogDebug("No letters detected; skipping all NER models.");
|
||||
break;
|
||||
}
|
||||
|
||||
return MergePersonSpans(entities);
|
||||
var merged = MergePersonSpans(entities);
|
||||
_logger.LogInformation(
|
||||
"NER routing complete: {EntityCount} person span(s) from [{InvokedModels}].",
|
||||
merged.Count,
|
||||
invoked.Count == 0 ? "none" : string.Join(", ", invoked));
|
||||
|
||||
return new NerPredictionResult(merged, invoked);
|
||||
}
|
||||
|
||||
internal static IReadOnlyList<PiiEntity> MergePersonSpans(IReadOnlyList<PiiEntity> entities)
|
||||
|
||||
@@ -21,7 +21,18 @@ public sealed class TamilOnnxNerRunner : IOnnxNerModelRunner, IDisposable
|
||||
|
||||
public bool IsModelAvailable => _runner.IsAvailable;
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text) => _runner.PredictEntities(text);
|
||||
public NerPredictionResult PredictEntities(string text)
|
||||
{
|
||||
if (!IsModelAvailable)
|
||||
{
|
||||
return NerPredictionResult.Empty;
|
||||
}
|
||||
|
||||
var entities = _runner.PredictEntities(text);
|
||||
return new NerPredictionResult(
|
||||
NerEntityTagging.WithOrigin(entities, NerModelOrigin.Tamil),
|
||||
[NerModelOrigin.Tamil]);
|
||||
}
|
||||
|
||||
public void Dispose() => _runner.Dispose();
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
</ResourceDictionary.MergedDictionaries>
|
||||
|
||||
<converters:ScriptCompositionToBrushConverter x:Key="ScriptCompositionToBrushConverter" />
|
||||
<converters:NerInvokedToBrushConverter x:Key="NerInvokedToBrushConverter" />
|
||||
<converters:BoolToVisibilityConverter x:Key="BoolToVisibilityConverter" />
|
||||
<converters:PassFailBrushConverter x:Key="PassFailBrushConverter" />
|
||||
<converters:StringNotEmptyToVisibilityConverter x:Key="StringNotEmptyToVisibilityConverter" />
|
||||
|
||||
@@ -3,7 +3,9 @@ using System.Windows;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using PiiRedaction.TestHarness.Wpf.DependencyInjection;
|
||||
using PiiRedaction.TestHarness.Wpf.Logging;
|
||||
using PiiRedaction.TestHarness.Wpf.Services;
|
||||
using PiiRedaction.TestHarness.Wpf.ViewModels;
|
||||
|
||||
@@ -24,6 +26,11 @@ public partial class App : Application
|
||||
configuration.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
|
||||
configuration.AddEnvironmentVariables();
|
||||
})
|
||||
.ConfigureLogging((context, logging) =>
|
||||
{
|
||||
logging.AddConfiguration(context.Configuration.GetSection("Logging"));
|
||||
logging.AddFilter("PiiRedaction.Infrastructure.Onnx", LogLevel.Debug);
|
||||
})
|
||||
.ConfigureServices((context, services) =>
|
||||
{
|
||||
services.AddPiiRedactionServices(context.Configuration);
|
||||
@@ -38,6 +45,9 @@ public partial class App : Application
|
||||
|
||||
Directory.SetCurrentDirectory(AppContext.BaseDirectory);
|
||||
|
||||
var loggerFactory = _host.Services.GetRequiredService<ILoggerFactory>();
|
||||
loggerFactory.AddProvider(new UiLoggerProvider(_host.Services.GetRequiredService<INerLogService>()));
|
||||
|
||||
await _host.StartAsync().ConfigureAwait(true);
|
||||
|
||||
var mainWindow = _host.Services.GetRequiredService<MainWindow>();
|
||||
|
||||
@@ -49,6 +49,31 @@ public sealed class StringNotEmptyToVisibilityConverter : IValueConverter
|
||||
throw new NotSupportedException();
|
||||
}
|
||||
|
||||
public sealed class NerInvokedToBrushConverter : IValueConverter
|
||||
{
|
||||
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture)
|
||||
{
|
||||
if (value is not string summary)
|
||||
{
|
||||
return new SolidColorBrush(Color.FromRgb(107, 114, 128));
|
||||
}
|
||||
|
||||
return summary switch
|
||||
{
|
||||
"—" or "None" => new SolidColorBrush(Color.FromRgb(107, 114, 128)),
|
||||
"English" => new SolidColorBrush(Color.FromRgb(37, 99, 235)),
|
||||
"Tamil" => new SolidColorBrush(Color.FromRgb(124, 58, 237)),
|
||||
_ when summary.Contains("English", StringComparison.Ordinal)
|
||||
&& summary.Contains("Tamil", StringComparison.Ordinal) =>
|
||||
new SolidColorBrush(Color.FromRgb(217, 119, 6)),
|
||||
_ => new SolidColorBrush(Color.FromRgb(107, 114, 128))
|
||||
};
|
||||
}
|
||||
|
||||
public object ConvertBack(object? value, Type targetType, object? parameter, CultureInfo culture) =>
|
||||
throw new NotSupportedException();
|
||||
}
|
||||
|
||||
public sealed class PassFailBrushConverter : IValueConverter
|
||||
{
|
||||
public object Convert(object? value, Type targetType, object? parameter, CultureInfo culture)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.AI;
|
||||
using PiiRedaction.TestHarness.Wpf.Services;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Core.Detection;
|
||||
@@ -15,6 +16,7 @@ public static class ServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddPiiRedactionServices(this IServiceCollection services, IConfiguration configuration)
|
||||
{
|
||||
services.AddSingleton<INerLogService, NerLogService>();
|
||||
services.Configure<PiiRedactionOptions>(configuration.GetSection(PiiRedactionOptions.SectionName));
|
||||
|
||||
services.AddSingleton<DomainRulePiiDetector>();
|
||||
|
||||
43
src/PiiRedaction.TestHarness.Wpf/Logging/UiLogger.cs
Normal file
43
src/PiiRedaction.TestHarness.Wpf/Logging/UiLogger.cs
Normal file
@@ -0,0 +1,43 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using PiiRedaction.TestHarness.Wpf.Services;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Logging;
|
||||
|
||||
public sealed class UiLogger : ILogger
|
||||
{
|
||||
private readonly string _category;
|
||||
private readonly INerLogService _logService;
|
||||
|
||||
public UiLogger(string category, INerLogService logService)
|
||||
{
|
||||
_category = category;
|
||||
_logService = logService;
|
||||
}
|
||||
|
||||
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
|
||||
|
||||
public bool IsEnabled(LogLevel logLevel) => logLevel != LogLevel.None;
|
||||
|
||||
public void Log<TState>(
|
||||
LogLevel logLevel,
|
||||
EventId eventId,
|
||||
TState state,
|
||||
Exception? exception,
|
||||
Func<TState, Exception?, string> formatter)
|
||||
{
|
||||
if (!IsEnabled(logLevel))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var message = formatter(state, exception);
|
||||
if (exception is not null)
|
||||
{
|
||||
message = $"{message} ({exception.Message})";
|
||||
}
|
||||
|
||||
var timestamp = DateTime.Now.ToString("HH:mm:ss.fff");
|
||||
var line = $"[{timestamp}] [{logLevel}] {_category}: {message}";
|
||||
_logService.Append(line);
|
||||
}
|
||||
}
|
||||
26
src/PiiRedaction.TestHarness.Wpf/Logging/UiLoggerProvider.cs
Normal file
26
src/PiiRedaction.TestHarness.Wpf/Logging/UiLoggerProvider.cs
Normal file
@@ -0,0 +1,26 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using PiiRedaction.TestHarness.Wpf.Services;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Logging;
|
||||
|
||||
public sealed class UiLoggerProvider : ILoggerProvider
|
||||
{
|
||||
private const string NerCategoryPrefix = "PiiRedaction.Infrastructure.Onnx";
|
||||
|
||||
private readonly INerLogService _logService;
|
||||
|
||||
public UiLoggerProvider(INerLogService logService) => _logService = logService;
|
||||
|
||||
public ILogger CreateLogger(string categoryName) =>
|
||||
IsNerCategory(categoryName)
|
||||
? new UiLogger(categoryName, _logService)
|
||||
: NullLogger.Instance;
|
||||
|
||||
internal static bool IsNerCategory(string categoryName) =>
|
||||
categoryName.StartsWith(NerCategoryPrefix, StringComparison.Ordinal);
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
@@ -35,6 +35,14 @@
|
||||
Foreground="White"
|
||||
FontWeight="SemiBold" />
|
||||
</Border>
|
||||
<TextBlock Text="NER invoked:" FontWeight="SemiBold" Margin="24,0,8,0" />
|
||||
<Border Padding="6,2"
|
||||
CornerRadius="4"
|
||||
Background="{Binding NerModelsInvokedSummary, Converter={StaticResource NerInvokedToBrushConverter}}">
|
||||
<TextBlock Text="{Binding NerModelsInvokedSummary}"
|
||||
Foreground="White"
|
||||
FontWeight="SemiBold" />
|
||||
</Border>
|
||||
<TextBlock Text="Last run:" FontWeight="SemiBold" Margin="24,0,8,0" />
|
||||
<TextBlock>
|
||||
<Run Text="{Binding ElapsedMilliseconds, Mode=OneWay}" />
|
||||
@@ -67,10 +75,15 @@
|
||||
FontSize="16"
|
||||
FontWeight="SemiBold"
|
||||
Margin="0,0,0,8" />
|
||||
<ComboBox DockPanel.Dock="Top"
|
||||
Margin="0,0,0,8"
|
||||
ItemsSource="{Binding TopicCategories}"
|
||||
SelectedItem="{Binding SelectedTopicCategory}"
|
||||
ToolTip="Filter prompts by category" />
|
||||
<TextBox DockPanel.Dock="Top"
|
||||
Margin="0,0,0,8"
|
||||
Text="{Binding PromptFilter, UpdateSourceTrigger=PropertyChanged}"
|
||||
ToolTip="Filter by name, category, language, or description" />
|
||||
ToolTip="Filter by name, topic, category, language, or description" />
|
||||
<StackPanel DockPanel.Dock="Bottom" Orientation="Horizontal" Margin="0,8,0,0">
|
||||
<Button Content="Run All"
|
||||
Command="{Binding RunAllScenariosCommand}" />
|
||||
@@ -168,7 +181,8 @@
|
||||
<DataGrid.Columns>
|
||||
<DataGridTextColumn Header="Type" Binding="{Binding Type}" Width="100" />
|
||||
<DataGridTextColumn Header="Value" Binding="{Binding Value}" Width="2*" MinWidth="120" />
|
||||
<DataGridTextColumn Header="Source" Binding="{Binding Source}" Width="90" />
|
||||
<DataGridTextColumn Header="Source" Binding="{Binding Source}" Width="80" />
|
||||
<DataGridTextColumn Header="NER Model" Binding="{Binding NerModel}" Width="90" />
|
||||
<DataGridTextColumn Header="Start" Binding="{Binding StartIndex}" Width="60" />
|
||||
<DataGridTextColumn Header="Length" Binding="{Binding Length}" Width="65" />
|
||||
<DataGridTextColumn Header="Confidence" Binding="{Binding Confidence}" Width="80" />
|
||||
@@ -260,21 +274,45 @@
|
||||
</Grid>
|
||||
</Grid>
|
||||
|
||||
<!-- Batch results -->
|
||||
<Expander Grid.Row="2"
|
||||
Header="Batch Results"
|
||||
<!-- Batch results + NER logs -->
|
||||
<StackPanel Grid.Row="2" Margin="0,8,0,0">
|
||||
<Expander Header="NER Logs"
|
||||
IsExpanded="True"
|
||||
Margin="0,0,0,8"
|
||||
Background="{StaticResource PanelBrush}"
|
||||
BorderBrush="{StaticResource BorderBrushColor}"
|
||||
BorderThickness="1"
|
||||
Padding="8">
|
||||
<DockPanel MinHeight="100" MaxHeight="220">
|
||||
<StackPanel DockPanel.Dock="Top" Orientation="Horizontal" Margin="0,0,0,8">
|
||||
<Button Content="Clear Logs"
|
||||
Style="{StaticResource SecondaryButtonStyle}"
|
||||
Command="{Binding ClearNerLogsCommand}" />
|
||||
</StackPanel>
|
||||
<ListBox x:Name="NerLogListBox"
|
||||
ItemsSource="{Binding NerLogLines}"
|
||||
FontFamily="Consolas"
|
||||
FontSize="12"
|
||||
ScrollViewer.HorizontalScrollBarVisibility="Auto"
|
||||
ScrollViewer.VerticalScrollBarVisibility="Auto" />
|
||||
</DockPanel>
|
||||
</Expander>
|
||||
|
||||
<Expander Header="Batch Results"
|
||||
IsExpanded="{Binding IsBatchExpanded}"
|
||||
Margin="0,8,0,0"
|
||||
Background="{StaticResource PanelBrush}"
|
||||
BorderBrush="{StaticResource BorderBrushColor}"
|
||||
BorderThickness="1"
|
||||
Padding="8">
|
||||
<DockPanel MinHeight="120">
|
||||
<DockPanel MinHeight="120" MaxHeight="320">
|
||||
<TextBlock DockPanel.Dock="Top"
|
||||
Text="{Binding BatchSummary}"
|
||||
FontWeight="SemiBold"
|
||||
Margin="0,0,0,8" />
|
||||
<DataGrid ItemsSource="{Binding BatchResults}">
|
||||
<DataGrid ItemsSource="{Binding BatchResults}"
|
||||
VerticalScrollBarVisibility="Auto"
|
||||
HorizontalScrollBarVisibility="Auto">
|
||||
<DataGrid.Columns>
|
||||
<DataGridTextColumn Header="Scenario" Binding="{Binding Scenario.Name}" Width="180" />
|
||||
<DataGridTextColumn Header="Language" Binding="{Binding Scenario.Language}" Width="80" />
|
||||
@@ -307,5 +345,6 @@
|
||||
</DataGrid>
|
||||
</DockPanel>
|
||||
</Expander>
|
||||
</StackPanel>
|
||||
</Grid>
|
||||
</Window>
|
||||
|
||||
@@ -1,13 +1,27 @@
|
||||
using System.Collections.Specialized;
|
||||
using System.Windows;
|
||||
using PiiRedaction.TestHarness.Wpf.Services;
|
||||
using PiiRedaction.TestHarness.Wpf.ViewModels;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf;
|
||||
|
||||
public partial class MainWindow : Window
|
||||
{
|
||||
public MainWindow(MainViewModel viewModel)
|
||||
public MainWindow(MainViewModel viewModel, INerLogService nerLogService)
|
||||
{
|
||||
InitializeComponent();
|
||||
DataContext = viewModel;
|
||||
|
||||
nerLogService.Lines.CollectionChanged += OnNerLogLinesChanged;
|
||||
}
|
||||
|
||||
private void OnNerLogLinesChanged(object? sender, NotifyCollectionChangedEventArgs e)
|
||||
{
|
||||
if (NerLogListBox.Items.Count == 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
NerLogListBox.ScrollIntoView(NerLogListBox.Items[NerLogListBox.Items.Count - 1]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Models;
|
||||
|
||||
public sealed record RedactionOutcome(
|
||||
@@ -6,7 +8,9 @@ public sealed record RedactionOutcome(
|
||||
IReadOnlyList<RedactionDisplayModel> DetectedEntities,
|
||||
IReadOnlyList<PlaceholderDisplayModel> Placeholders,
|
||||
long ElapsedMilliseconds,
|
||||
bool HasLeak);
|
||||
bool HasLeak,
|
||||
IReadOnlyList<NerModelOrigin> NerModelsInvoked,
|
||||
string NerModelsInvokedSummary);
|
||||
|
||||
public sealed record BatchScenarioResult(
|
||||
TestPromptScenario Scenario,
|
||||
|
||||
20
src/PiiRedaction.TestHarness.Wpf/Models/NerRoutingDisplay.cs
Normal file
20
src/PiiRedaction.TestHarness.Wpf/Models/NerRoutingDisplay.cs
Normal file
@@ -0,0 +1,20 @@
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Models;
|
||||
|
||||
public static class NerRoutingDisplay
|
||||
{
|
||||
public static string FormatInvokedModels(IReadOnlyList<NerModelOrigin> invoked) =>
|
||||
invoked.Count switch
|
||||
{
|
||||
0 => "None",
|
||||
1 => invoked[0].ToString(),
|
||||
_ => string.Join(" + ", invoked)
|
||||
};
|
||||
|
||||
public static string FormatEntityModelOrigin(PiiEntity entity) =>
|
||||
entity.Source == PiiDetectionSource.Ner && entity.ModelOrigin.HasValue
|
||||
? entity.ModelOrigin.Value.ToString()
|
||||
: "—";
|
||||
}
|
||||
35
src/PiiRedaction.TestHarness.Wpf/Models/PromptTopics.cs
Normal file
35
src/PiiRedaction.TestHarness.Wpf/Models/PromptTopics.cs
Normal file
@@ -0,0 +1,35 @@
|
||||
namespace PiiRedaction.TestHarness.Wpf.Models;
|
||||
|
||||
/// <summary>
|
||||
/// Top-level prompt groupings for the WPF harness category filter.
|
||||
/// </summary>
|
||||
public static class PromptTopics
|
||||
{
|
||||
public const string All = "All";
|
||||
|
||||
public const string CareerGuidance = "Career Guidance";
|
||||
|
||||
public const string BankingFinancial = "Banking & Financial";
|
||||
|
||||
public const string Negative = "Negative";
|
||||
|
||||
public const string EdgeHarness = "Edge & Harness";
|
||||
|
||||
public static IReadOnlyList<string> FilterOptions { get; } =
|
||||
[
|
||||
All,
|
||||
CareerGuidance,
|
||||
BankingFinancial,
|
||||
Negative,
|
||||
EdgeHarness
|
||||
];
|
||||
|
||||
public static int SortOrder(string topic) => topic switch
|
||||
{
|
||||
CareerGuidance => 0,
|
||||
BankingFinancial => 1,
|
||||
EdgeHarness => 2,
|
||||
Negative => 3,
|
||||
_ => 99
|
||||
};
|
||||
}
|
||||
@@ -7,6 +7,7 @@ public sealed class RedactionDisplayModel
|
||||
public required string Type { get; init; }
|
||||
public required string Value { get; init; }
|
||||
public required string Source { get; init; }
|
||||
public required string NerModel { get; init; }
|
||||
public int StartIndex { get; init; }
|
||||
public int Length { get; init; }
|
||||
public string? Confidence { get; init; }
|
||||
@@ -16,6 +17,7 @@ public sealed class RedactionDisplayModel
|
||||
Type = entity.Type.ToString(),
|
||||
Value = entity.Value,
|
||||
Source = entity.Source.ToString(),
|
||||
NerModel = NerRoutingDisplay.FormatEntityModelOrigin(entity),
|
||||
StartIndex = entity.StartIndex,
|
||||
Length = entity.Length,
|
||||
Confidence = entity.Confidence?.ToString("F2")
|
||||
|
||||
@@ -11,6 +11,7 @@ public enum PromptLanguage
|
||||
public sealed record TestPromptScenario(
|
||||
string Id,
|
||||
string Name,
|
||||
string Topic,
|
||||
PromptLanguage Language,
|
||||
string Category,
|
||||
string Description,
|
||||
|
||||
12
src/PiiRedaction.TestHarness.Wpf/Services/INerLogService.cs
Normal file
12
src/PiiRedaction.TestHarness.Wpf/Services/INerLogService.cs
Normal file
@@ -0,0 +1,12 @@
|
||||
using System.Collections.ObjectModel;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Services;
|
||||
|
||||
public interface INerLogService
|
||||
{
|
||||
ObservableCollection<string> Lines { get; }
|
||||
|
||||
void Append(string line);
|
||||
|
||||
void Clear();
|
||||
}
|
||||
@@ -5,4 +5,6 @@ namespace PiiRedaction.TestHarness.Wpf.Services;
|
||||
public interface ITestPromptCatalog
|
||||
{
|
||||
IReadOnlyList<TestPromptScenario> All { get; }
|
||||
|
||||
IReadOnlyList<string> Topics { get; }
|
||||
}
|
||||
|
||||
31
src/PiiRedaction.TestHarness.Wpf/Services/NerLogService.cs
Normal file
31
src/PiiRedaction.TestHarness.Wpf/Services/NerLogService.cs
Normal file
@@ -0,0 +1,31 @@
|
||||
using System.Collections.ObjectModel;
|
||||
using System.Windows;
|
||||
|
||||
namespace PiiRedaction.TestHarness.Wpf.Services;
|
||||
|
||||
public sealed class NerLogService : INerLogService
|
||||
{
|
||||
public ObservableCollection<string> Lines { get; } = [];
|
||||
|
||||
public void Append(string line)
|
||||
{
|
||||
if (Application.Current?.Dispatcher.CheckAccess() == true)
|
||||
{
|
||||
Lines.Add(line);
|
||||
return;
|
||||
}
|
||||
|
||||
Application.Current?.Dispatcher.Invoke(() => Lines.Add(line));
|
||||
}
|
||||
|
||||
public void Clear()
|
||||
{
|
||||
if (Application.Current?.Dispatcher.CheckAccess() == true)
|
||||
{
|
||||
Lines.Clear();
|
||||
return;
|
||||
}
|
||||
|
||||
Application.Current?.Dispatcher.Invoke(() => Lines.Clear());
|
||||
}
|
||||
}
|
||||
@@ -111,6 +111,8 @@ public sealed class RedactionAppService : IRedactionAppService
|
||||
entities,
|
||||
placeholders,
|
||||
elapsedMilliseconds,
|
||||
hasLeak);
|
||||
hasLeak,
|
||||
result.NerModelsInvoked,
|
||||
NerRoutingDisplay.FormatInvokedModels(result.NerModelsInvoked));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,8 +4,130 @@ namespace PiiRedaction.TestHarness.Wpf.Services;
|
||||
|
||||
public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
{
|
||||
public IReadOnlyList<string> Topics => PromptTopics.FilterOptions;
|
||||
|
||||
public IReadOnlyList<TestPromptScenario> All { get; } =
|
||||
[
|
||||
// --- Career Guidance (Tamil students — English, Tamil, Tanglish, Mixed) ---
|
||||
|
||||
Scenario(
|
||||
"CareerEnglishItPath",
|
||||
PromptLanguage.English,
|
||||
"Career + NER + Regex",
|
||||
"English: B.Tech graduate asking IT career advice with name, email, and phone.",
|
||||
"Hello, I am Rahul Kumar from Coimbatore. My email is rahul.kumar@college.edu and mobile 9876543210. I finished B.Tech IT. Which software career path is best for me?",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerEnglishResumeHelp",
|
||||
PromptLanguage.English,
|
||||
"Career + NER + Regex",
|
||||
"English: student requesting resume guidance with name and email.",
|
||||
"I am Divya Sharma and my email is divya.sharma@gmail.com. Can you suggest how to improve my resume for data analyst internships?",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerEnglishPlacementStress",
|
||||
PromptLanguage.English,
|
||||
"Career + NER + Regex",
|
||||
"English: final-year student sharing contact details for placement counselling.",
|
||||
"I am Arjun Mehta, phone 9123456780, email arjun.mehta@campus.in. I am in my final year and campus placement offers are very low. What career options should I explore?",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTamilEngineeringChoice",
|
||||
PromptLanguage.Tamil,
|
||||
"Career + NER (Tamil) + Regex",
|
||||
"Tamil: engineering student choosing between branches with name and phone.",
|
||||
"நான் முருகன் ராஜா, தொலைபேசி 9845011223. பி.இ முதல் ஆண்டு முடித்தேன். சிவில் பொறியியலா மெக்கானிக்கலா எது நல்ல வேலை வாய்ப்பு தரும்?",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTamilMbaGuidance",
|
||||
PromptLanguage.Tamil,
|
||||
"Career + NER (Tamil) + Regex",
|
||||
"Tamil: student asking MBA finance career path with name and email.",
|
||||
"நான் வானதி குமார், மின்னஞ்சல் vanathi.k@univ.in. MBA finance career path பற்றி விளக்குங்கள்.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTamilGovtExamPrep",
|
||||
PromptLanguage.Tamil,
|
||||
"Career + NER (Tamil) + Regex",
|
||||
"Tamil: student preparing for government exams with contact details.",
|
||||
"மாணவர் கார்த்திக் செல்வம், தொலைபேசி 9003214567. TNPSC Group 2 தேர்வுக்கு எப்படி தயாராகுவது? மின்னஞ்சல் karthik.s@prep.in",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTanglishAfter12th",
|
||||
PromptLanguage.Tanglish,
|
||||
"Career + NER (Tanglish) + Regex",
|
||||
"Tanglish: student after 12th asking which course for software job.",
|
||||
"Naan Suresh, 12th complete panniten, phone 9003789456. Software job ku enna course padikanum nu sollunga.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTanglishPlacementHelp",
|
||||
PromptLanguage.Tanglish,
|
||||
"Career + NER (Tanglish) + Regex",
|
||||
"Tanglish: student worried about placements with email.",
|
||||
"Hi I am Keerthana from Madurai. Enga college la placement romba kammi. Next enna panrathu? Email keerthana.m@gmail.com",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerTanglishHigherStudies",
|
||||
PromptLanguage.Tanglish,
|
||||
"Career + NER (Tanglish) + Regex",
|
||||
"Tanglish: student asking about higher studies abroad with phone.",
|
||||
"I am Pradeep, finished BCA. MS ku apply pannanum — guide pannunga. Phone 9840098765.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerMixedStudyAbroad",
|
||||
PromptLanguage.Mixed,
|
||||
"Career + NER (Mixed) + Regex",
|
||||
"Mixed Tamil/English: study abroad guidance with name, phone, and email.",
|
||||
"நான் David Thomas, phone 9887766554. Study abroad MS computer science ku guide pannunga. Email david.t@student.in",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerMixedSkillUpgrade",
|
||||
PromptLanguage.Mixed,
|
||||
"Career + NER (Mixed) + Regex",
|
||||
"Mixed: working professional asking about upskilling with contact info.",
|
||||
"வானக்கம், I am Priya Nair working in BPO. Cloud computing ku switch panna phone 9876012345 and email priya.nair@work.com la details anupunga.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerNegativeGeneral",
|
||||
PromptLanguage.English,
|
||||
"Career + Negative",
|
||||
"General career question with no student PII.",
|
||||
"What skills are needed for a career in cloud computing after graduation?",
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
Scenario(
|
||||
"CareerNegativeTamil",
|
||||
PromptLanguage.Tamil,
|
||||
"Career + Negative",
|
||||
"Tamil career market question without personal identifiers.",
|
||||
"இன்றைய job market ல் data science career prospects எப்படி இருக்கும்? பொதுவான விளக்கம் தருங்கள்.",
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.CareerGuidance),
|
||||
|
||||
// --- Banking & Financial ---
|
||||
Scenario(
|
||||
"FullFinancialWithCustomer",
|
||||
PromptLanguage.English,
|
||||
@@ -132,7 +254,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"Negative",
|
||||
"No PII — prompt passes through unchanged.",
|
||||
"What is the status of ticket TKT-99887 and when will the API maintenance end?",
|
||||
expectDetections: false),
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.Negative),
|
||||
|
||||
Scenario(
|
||||
"NegativeWorkflowQuestion",
|
||||
@@ -140,7 +263,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"Negative",
|
||||
"General workflow question with no regulated identifiers.",
|
||||
"Summarize the retail loan approval workflow and typical SLA milestones.",
|
||||
expectDetections: false),
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.Negative),
|
||||
|
||||
Scenario(
|
||||
"EdgePhoneOnly",
|
||||
@@ -148,7 +272,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"Edge + Regex",
|
||||
"Digits-only phone without a person name.",
|
||||
"Callback requested on 9123456780 regarding branch hours.",
|
||||
expectDetections: true),
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
Scenario(
|
||||
"EdgeLongMixed",
|
||||
@@ -156,7 +281,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"Edge + NER (Mixed)",
|
||||
"Longer mixed-language prompt with person and phone.",
|
||||
"வாடிக்கையாளர் Priya Nair called from Chennai about a delayed NEFT transfer. She asked whether LoanNumber LN-909090 is linked to account ACC-808080 and wants an email confirmation sent to priya.nair@example.com on phone 9988776655.",
|
||||
expectDetections: true),
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
Scenario(
|
||||
"LeakCheckNestedEmail",
|
||||
@@ -164,7 +290,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"LeakCheck + Regex",
|
||||
"Email embedded in a sentence — placeholders must fully replace the address.",
|
||||
"Please forward the statement for customer.support@banking.example to the operations desk.",
|
||||
expectDetections: true),
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
Scenario(
|
||||
"TamilEdgePunctuation",
|
||||
@@ -172,7 +299,8 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"TamilEdge + NER (Tamil)",
|
||||
"Tamil name surrounded by punctuation and Tamil numerals.",
|
||||
"வாடிக்கையாளர் (ராஜேஷ் குமார்) — தொலைபேசி ௯௮௭௬௫௪௩௨௧௦ — உதவி தேவை.",
|
||||
expectDetections: true),
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
Scenario(
|
||||
"TanglishLatinInTamilSentence",
|
||||
@@ -180,7 +308,207 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
"Tanglish + NER",
|
||||
"Latin person name inside otherwise Tamil context.",
|
||||
"வாடிக்கையாளர் Arun Kumar அவர்களின் KYC ஆவணம் நிலுவையில் உள்ளது.",
|
||||
expectDetections: true)
|
||||
expectDetections: true),
|
||||
|
||||
// --- English banking & customer-service scenarios ---
|
||||
|
||||
Scenario(
|
||||
"MsTitlePerson",
|
||||
PromptLanguage.English,
|
||||
"NER",
|
||||
"Person detected via ONNX NER (title prefix Ms.).",
|
||||
"Ms. Kavitha Nair requested a statement for her fixed deposit renewal.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"KycAadhaarUpload",
|
||||
PromptLanguage.English,
|
||||
"NER + Regex",
|
||||
"KYC follow-up with person name and Aadhaar number.",
|
||||
"Customer Deepa Iyer uploaded Aadhaar 2345 6789 0123 for video KYC completion.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"CreditCardDispute",
|
||||
PromptLanguage.English,
|
||||
"NER + Regex",
|
||||
"Card dispute with person, masked card, and email.",
|
||||
"Customer Vikram Singh disputed charge on card 4532-1234-5678-9010 and wrote from vikram.singh@mail.com.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"SavingsAccountClosure",
|
||||
PromptLanguage.English,
|
||||
"NER + Domain",
|
||||
"Account closure request with person name and account number.",
|
||||
"Customer Sanjay Patel wants to close AccountNumber ACC-112233 and transfer the balance.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"LoanStatusByNumber",
|
||||
PromptLanguage.English,
|
||||
"Domain",
|
||||
"Loan status lookup using loan number only (no person name).",
|
||||
"Please check disbursement status for LoanNumber LN-778899 and share the expected credit date.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"CustomerIdLookup",
|
||||
PromptLanguage.English,
|
||||
"Domain",
|
||||
"CRM lookup using customer ID without a person name.",
|
||||
"Pull interaction history for CustomerId CID-5521 related to the mobile app login failure.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"EmailOnlySupport",
|
||||
PromptLanguage.English,
|
||||
"Regex",
|
||||
"Support thread with email address but no detectable person name.",
|
||||
"Reply to the customer at support.user@example.org about the delayed NEFT credit.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"PanAndPhoneNoName",
|
||||
PromptLanguage.English,
|
||||
"Regex",
|
||||
"PAN and phone provided for callback without a person name.",
|
||||
"Verify PAN FGHIJ5678K and call back on 9988123456 regarding the EMI bounce.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"RealWorldChargebackNote",
|
||||
PromptLanguage.English,
|
||||
"NER + Regex + Domain",
|
||||
"Realistic call-centre note combining person, phone, loan, and email.",
|
||||
"Customer Meera Iyer called from 9876012345 about a duplicate EMI debit on LoanNumber LN-334455. She can be reached at meera.iyer@bank.in for confirmation.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"EdgeMultiplePhones",
|
||||
PromptLanguage.English,
|
||||
"Regex",
|
||||
"Two phone numbers in one prompt (primary and alternate).",
|
||||
"Reach the customer on 9123456780 or alternate 9988776655 for OTP verification.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
// --- Tamil scenarios ---
|
||||
|
||||
Scenario(
|
||||
"TamilLoanDispute",
|
||||
PromptLanguage.Tamil,
|
||||
"NER (Tamil) + Domain",
|
||||
"Tamil person name with loan and account identifiers.",
|
||||
"வாடிக்கையாளர் பிரியா ராமன் LoanNumber LN-220011 கணக்கு ACC-445566 இல் தவறான பற்று வைக்கப்பட்டுள்ளது என புகாரளித்தார்.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"TamilAadhaarKyc",
|
||||
PromptLanguage.Tamil,
|
||||
"NER (Tamil) + Regex",
|
||||
"Tamil script KYC prompt with Aadhaar number.",
|
||||
"வாடிக்கையாளர் சுரேஷ் பாபு ஆதார் 4567 8901 2345 உடன் KYC புதுப்பிப்பை முடிக்க வேண்டும்.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"TamilEmailFollowUp",
|
||||
PromptLanguage.Tamil,
|
||||
"NER (Tamil) + Regex",
|
||||
"Tamil customer follow-up with email address.",
|
||||
"வாடிக்கையாளர் லட்சுமி மின்னஞ்சல் lakshmi.devi@example.com மூலம் சேமிப்பு வட்டி விளக்கம் கேட்டார்.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"TamilNegativeFaq",
|
||||
PromptLanguage.Tamil,
|
||||
"Negative",
|
||||
"Tamil product FAQ with no regulated identifiers.",
|
||||
"சேமிப்பு கணக்கிற்கான வட்டி விகிதம் எப்படி கணக்கிடப்படுகிறது? தயவுசெய்து விளக்கவும்.",
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.Negative),
|
||||
|
||||
// --- Tanglish scenarios ---
|
||||
|
||||
Scenario(
|
||||
"TanglishUpiRefund",
|
||||
PromptLanguage.Tanglish,
|
||||
"NER (English/Tanglish) + Regex",
|
||||
"Tanglish UPI refund complaint with person and phone.",
|
||||
"Customer Karthik said UPI payment failed, please refund. Phone 9845012345.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"TanglishMsPriyaCallback",
|
||||
PromptLanguage.Tanglish,
|
||||
"NER (English/Tanglish) + Regex",
|
||||
"Tanglish callback request with title and phone.",
|
||||
"Ms Priya called — enna panrathu? Callback 9003123456 before 6 PM.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"TanglishLoanAndCid",
|
||||
PromptLanguage.Tanglish,
|
||||
"NER + Domain",
|
||||
"Tanglish mix of person name, loan number, and customer ID.",
|
||||
"Customer Ganesh holds CID-8812 for LoanNumber LN-660033 and needs disbursement update.",
|
||||
expectDetections: true),
|
||||
|
||||
// --- Mixed-script scenarios ---
|
||||
|
||||
Scenario(
|
||||
"MixedCallCenterHandoff",
|
||||
PromptLanguage.Mixed,
|
||||
"NER (Mixed) + Regex + Domain",
|
||||
"Call-centre handoff note mixing Tamil, English, phone, and loan ID.",
|
||||
"வாடிக்கையாளர் Anitha Roy phone 9876501234 says EMI debited twice on LoanNumber LN-550077. Email anitha.roy@corp.in for receipt.",
|
||||
expectDetections: true),
|
||||
|
||||
Scenario(
|
||||
"MixedTamilEnglishAccount",
|
||||
PromptLanguage.Mixed,
|
||||
"NER (Mixed) + Domain",
|
||||
"Mixed prompt with English name embedded in Tamil sentence and account number.",
|
||||
"வாடிக்கையாளர் David Thomas அவர்கள் AccountNumber ACC-990011 ஐ மூட விரும்புகிறார்.",
|
||||
expectDetections: true),
|
||||
|
||||
// --- Additional negative / edge scenarios ---
|
||||
|
||||
Scenario(
|
||||
"NegativeProductFaq",
|
||||
PromptLanguage.English,
|
||||
"Negative",
|
||||
"Product FAQ about interest rates — no customer PII.",
|
||||
"What is the current savings account interest rate and how is it credited quarterly?",
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.Negative),
|
||||
|
||||
Scenario(
|
||||
"NegativeBranchLocator",
|
||||
PromptLanguage.English,
|
||||
"Negative",
|
||||
"Branch locator query using branch codes only.",
|
||||
"List branches open on Sunday in Chennai zone BR-CHN-04 and BR-CHN-09.",
|
||||
expectDetections: false,
|
||||
topic: PromptTopics.Negative),
|
||||
|
||||
Scenario(
|
||||
"EdgeAadhaarVariants",
|
||||
PromptLanguage.English,
|
||||
"Regex",
|
||||
"Aadhaar with spaced digits alongside PAN.",
|
||||
"Documents on file: PAN KLMPN4567Q and Aadhaar 9876 5432 1098 for verification.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness),
|
||||
|
||||
Scenario(
|
||||
"EdgeDomainWithoutLabels",
|
||||
PromptLanguage.English,
|
||||
"Domain",
|
||||
"Domain IDs embedded in natural sentence without explicit field labels.",
|
||||
"The case is tied to LN-445566 and ACC-778899 under CRM record CID-4400.",
|
||||
expectDetections: true,
|
||||
topic: PromptTopics.EdgeHarness)
|
||||
];
|
||||
|
||||
private static TestPromptScenario Scenario(
|
||||
@@ -189,6 +517,7 @@ public sealed class TestPromptCatalog : ITestPromptCatalog
|
||||
string category,
|
||||
string description,
|
||||
string prompt,
|
||||
bool expectDetections) =>
|
||||
new(name, name, language, category, description, prompt, expectDetections);
|
||||
bool expectDetections,
|
||||
string topic = PromptTopics.BankingFinancial) =>
|
||||
new(name, name, topic, language, category, description, prompt, expectDetections);
|
||||
}
|
||||
|
||||
@@ -16,26 +16,33 @@ public partial class MainViewModel : ObservableObject
|
||||
private readonly ITestPromptCatalog _promptCatalog;
|
||||
private readonly IScriptAnalysisService _scriptAnalysisService;
|
||||
private readonly IModelStatusService _modelStatusService;
|
||||
private readonly INerLogService _nerLogService;
|
||||
|
||||
public MainViewModel(
|
||||
IRedactionAppService redactionAppService,
|
||||
ITestPromptCatalog promptCatalog,
|
||||
IScriptAnalysisService scriptAnalysisService,
|
||||
IModelStatusService modelStatusService)
|
||||
IModelStatusService modelStatusService,
|
||||
INerLogService nerLogService)
|
||||
{
|
||||
_redactionAppService = redactionAppService;
|
||||
_promptCatalog = promptCatalog;
|
||||
_scriptAnalysisService = scriptAnalysisService;
|
||||
_modelStatusService = modelStatusService;
|
||||
_nerLogService = nerLogService;
|
||||
|
||||
PromptItems = new ObservableCollection<TestPromptItemViewModel>(
|
||||
_promptCatalog.All.Select(scenario => new TestPromptItemViewModel(scenario)));
|
||||
|
||||
PromptsView = CollectionViewSource.GetDefaultView(PromptItems);
|
||||
PromptsView.GroupDescriptions.Add(new PropertyGroupDescription(nameof(TestPromptItemViewModel.Language)));
|
||||
PromptsView.GroupDescriptions.Add(new PropertyGroupDescription(nameof(TestPromptItemViewModel.Topic)));
|
||||
PromptsView.SortDescriptions.Add(new SortDescription(nameof(TestPromptItemViewModel.TopicSortOrder), ListSortDirection.Ascending));
|
||||
PromptsView.SortDescriptions.Add(new SortDescription(nameof(TestPromptItemViewModel.Name), ListSortDirection.Ascending));
|
||||
PromptsView.Filter = FilterPrompt;
|
||||
|
||||
TopicCategories = new ObservableCollection<string>(_promptCatalog.Topics);
|
||||
SelectedTopicCategory = PromptTopics.All;
|
||||
|
||||
DetectedEntities = [];
|
||||
PlaceholderMap = [];
|
||||
BatchResults = [];
|
||||
@@ -69,6 +76,9 @@ public partial class MainViewModel : ObservableObject
|
||||
[ObservableProperty]
|
||||
private TestPromptItemViewModel? _selectedPrompt;
|
||||
|
||||
[ObservableProperty]
|
||||
private string _nerModelsInvokedSummary = "—";
|
||||
|
||||
[ObservableProperty]
|
||||
private ScriptComposition _scriptComposition = ScriptComposition.NoLetters;
|
||||
|
||||
@@ -93,6 +103,13 @@ public partial class MainViewModel : ObservableObject
|
||||
[ObservableProperty]
|
||||
private string _promptFilter = string.Empty;
|
||||
|
||||
[ObservableProperty]
|
||||
private string _selectedTopicCategory = PromptTopics.All;
|
||||
|
||||
public ObservableCollection<string> TopicCategories { get; }
|
||||
|
||||
public ObservableCollection<string> NerLogLines => _nerLogService.Lines;
|
||||
|
||||
[ObservableProperty]
|
||||
private string _batchSummary = string.Empty;
|
||||
|
||||
@@ -117,6 +134,13 @@ public partial class MainViewModel : ObservableObject
|
||||
StatusMessage = $"Loaded prompt: {value.Name}";
|
||||
}
|
||||
|
||||
[RelayCommand]
|
||||
private void ClearNerLogs()
|
||||
{
|
||||
_nerLogService.Clear();
|
||||
StatusMessage = "NER logs cleared.";
|
||||
}
|
||||
|
||||
[RelayCommand]
|
||||
private void Clear()
|
||||
{
|
||||
@@ -140,6 +164,7 @@ public partial class MainViewModel : ObservableObject
|
||||
LeakWarning = false;
|
||||
EntityCount = 0;
|
||||
ElapsedMilliseconds = 0;
|
||||
NerModelsInvokedSummary = "—";
|
||||
SendToMockLlmCommand.NotifyCanExecuteChanged();
|
||||
}
|
||||
|
||||
@@ -243,13 +268,15 @@ public partial class MainViewModel : ObservableObject
|
||||
BatchSummary = "Running batch validation...";
|
||||
StatusMessage = "Running all scenarios...";
|
||||
|
||||
var scenarios = GetVisibleScenarios();
|
||||
|
||||
var progress = new Progress<(int Current, int Total, string Name)>(report =>
|
||||
{
|
||||
StatusMessage = $"Batch {report.Current}/{report.Total}: {report.Name}";
|
||||
});
|
||||
|
||||
var summary = await _redactionAppService
|
||||
.RunAllScenariosAsync(_promptCatalog.All, progress, cancellationToken)
|
||||
.RunAllScenariosAsync(scenarios, progress, cancellationToken)
|
||||
.ConfigureAwait(true);
|
||||
|
||||
BatchResults.Clear();
|
||||
@@ -286,6 +313,8 @@ public partial class MainViewModel : ObservableObject
|
||||
|
||||
partial void OnPromptFilterChanged(string value) => PromptsView.Refresh();
|
||||
|
||||
partial void OnSelectedTopicCategoryChanged(string value) => PromptsView.Refresh();
|
||||
|
||||
private void ApplyOutcome(RedactionOutcome outcome)
|
||||
{
|
||||
OriginalPrompt = outcome.OriginalPrompt;
|
||||
@@ -293,6 +322,7 @@ public partial class MainViewModel : ObservableObject
|
||||
ElapsedMilliseconds = outcome.ElapsedMilliseconds;
|
||||
EntityCount = outcome.DetectedEntities.Count;
|
||||
LeakWarning = outcome.HasLeak;
|
||||
NerModelsInvokedSummary = outcome.NerModelsInvokedSummary;
|
||||
|
||||
DetectedEntities.Clear();
|
||||
foreach (var entity in outcome.DetectedEntities)
|
||||
@@ -323,6 +353,12 @@ public partial class MainViewModel : ObservableObject
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!string.Equals(SelectedTopicCategory, PromptTopics.All, StringComparison.Ordinal)
|
||||
&& !promptItem.Topic.Equals(SelectedTopicCategory, StringComparison.Ordinal))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(PromptFilter))
|
||||
{
|
||||
return true;
|
||||
@@ -330,8 +366,15 @@ public partial class MainViewModel : ObservableObject
|
||||
|
||||
var filter = PromptFilter.Trim();
|
||||
return promptItem.Name.Contains(filter, StringComparison.OrdinalIgnoreCase)
|
||||
|| promptItem.Topic.Contains(filter, StringComparison.OrdinalIgnoreCase)
|
||||
|| promptItem.Category.Contains(filter, StringComparison.OrdinalIgnoreCase)
|
||||
|| promptItem.Description.Contains(filter, StringComparison.OrdinalIgnoreCase)
|
||||
|| promptItem.Language.ToString().Contains(filter, StringComparison.OrdinalIgnoreCase);
|
||||
}
|
||||
|
||||
private IReadOnlyList<TestPromptScenario> GetVisibleScenarios() =>
|
||||
PromptItems
|
||||
.Where(item => FilterPrompt(item))
|
||||
.Select(item => item.Scenario)
|
||||
.ToList();
|
||||
}
|
||||
|
||||
@@ -13,6 +13,10 @@ public sealed class TestPromptItemViewModel
|
||||
|
||||
public string Name => Scenario.Name;
|
||||
|
||||
public string Topic => Scenario.Topic;
|
||||
|
||||
public int TopicSortOrder => PromptTopics.SortOrder(Scenario.Topic);
|
||||
|
||||
public string Category => Scenario.Category;
|
||||
|
||||
public PromptLanguage Language => Scenario.Language;
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"PiiRedaction.Infrastructure.Onnx": "Debug"
|
||||
}
|
||||
},
|
||||
"PiiRedaction": {
|
||||
"OnnxModelPath": "models/ner-model.onnx",
|
||||
"EnglishOnnxModelPath": "models/en/ner-model.onnx",
|
||||
|
||||
@@ -46,6 +46,36 @@ public sealed class OnnxNerPiiDetectorTests
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_ModelAvailable_RecordsInvokedModels()
|
||||
{
|
||||
var runner = new FakeOnnxNerModelRunner
|
||||
{
|
||||
IsModelAvailable = true,
|
||||
InvokedModelsToReturn = [NerModelOrigin.English, NerModelOrigin.Tamil]
|
||||
};
|
||||
|
||||
var detector = new OnnxNerPiiDetector(runner);
|
||||
detector.Detect("Mixed prompt");
|
||||
|
||||
detector.LastInvokedModels.Should().Equal(NerModelOrigin.English, NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_ModelUnavailable_ClearsInvokedModels()
|
||||
{
|
||||
var runner = new FakeOnnxNerModelRunner
|
||||
{
|
||||
IsModelAvailable = false,
|
||||
InvokedModelsToReturn = [NerModelOrigin.English]
|
||||
};
|
||||
|
||||
var detector = new OnnxNerPiiDetector(runner);
|
||||
detector.Detect("Any text");
|
||||
|
||||
detector.LastInvokedModels.Should().BeEmpty();
|
||||
}
|
||||
|
||||
private static OnnxNerPiiDetector CreateDetector(bool modelAvailable)
|
||||
{
|
||||
var runner = new FakeOnnxNerModelRunner { IsModelAvailable = modelAvailable };
|
||||
|
||||
@@ -32,7 +32,32 @@ public sealed class RealTamilPipelineTests : RealRoutingNerModelFixture
|
||||
result.SanitizedPrompt.Should().Contain("<PERSON_1>");
|
||||
result.SanitizedPrompt.Should().NotContain("ராஜேஷ்");
|
||||
result.DetectedEntities.Should().Contain(entity =>
|
||||
entity.Type == PiiEntityType.Person && entity.Source == PiiDetectionSource.Ner);
|
||||
entity.Type == PiiEntityType.Person
|
||||
&& entity.Source == PiiDetectionSource.Ner
|
||||
&& entity.ModelOrigin == NerModelOrigin.Tamil);
|
||||
result.NerModelsInvoked.Should().Equal(NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Sanitize_TanglishCustomer_InvokesEnglishNerOnly()
|
||||
{
|
||||
const string prompt = "Customer Senthil phone 9876543210 reported a failed UPI transfer.";
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
result.NerModelsInvoked.Should().Equal(NerModelOrigin.English);
|
||||
result.DetectedEntities.Should().Contain(entity =>
|
||||
entity.Type == PiiEntityType.Person && entity.ModelOrigin == NerModelOrigin.English);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Sanitize_MixedTamilEnglish_InvokesBothNerModels()
|
||||
{
|
||||
const string prompt = "வாடிக்கையாளர் Ravi Kumar phone 9876543210 disputed the charge.";
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
result.NerModelsInvoked.Should().Equal(NerModelOrigin.English, NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
[Test]
|
||||
|
||||
@@ -9,11 +9,13 @@ public sealed class FakeOnnxNerModelRunner : IOnnxNerModelRunner
|
||||
|
||||
public IReadOnlyList<PiiEntity> EntitiesToReturn { get; set; } = [];
|
||||
|
||||
public IReadOnlyList<NerModelOrigin> InvokedModelsToReturn { get; set; } = [NerModelOrigin.English];
|
||||
|
||||
public string? LastPredictedText { get; private set; }
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
public NerPredictionResult PredictEntities(string text)
|
||||
{
|
||||
LastPredictedText = text;
|
||||
return EntitiesToReturn;
|
||||
return new NerPredictionResult(EntitiesToReturn, InvokedModelsToReturn);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
@@ -20,7 +21,8 @@ public static class ProductionPipelineFactory
|
||||
CreateWithRealModel(new RoutingOnnxNerModelRunner(
|
||||
englishRunner,
|
||||
tamilRunner,
|
||||
options ?? Options.Create(new PiiRedactionOptions { EnableTamilNer = true })));
|
||||
options ?? Options.Create(new PiiRedactionOptions { EnableTamilNer = true }),
|
||||
NullLogger<RoutingOnnxNerModelRunner>.Instance));
|
||||
|
||||
public static IPiiDetector CreateCompositeDetector(IOnnxNerModelRunner runner) =>
|
||||
new CompositePiiDetector(
|
||||
|
||||
@@ -25,7 +25,7 @@ public sealed class OnnxNerModelRunnerTests
|
||||
var path = Path.Combine(Path.GetTempPath(), $"missing-ner-{Guid.NewGuid():N}.onnx");
|
||||
using var runner = CreateRunner(path);
|
||||
|
||||
runner.PredictEntities("Customer Ravi Kumar").Should().BeEmpty();
|
||||
runner.PredictEntities("Customer Ravi Kumar").Entities.Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
|
||||
@@ -26,16 +26,13 @@ public sealed class RealNerModelRunnerTests : RealNerModelFixture
|
||||
|
||||
{
|
||||
|
||||
var entities = Runner.PredictEntities(prompt);
|
||||
|
||||
|
||||
var result = Runner.PredictEntities(prompt);
|
||||
var entities = result.Entities;
|
||||
|
||||
entities.Should().Contain(entity =>
|
||||
|
||||
entity.Type == PiiEntityType.Person &&
|
||||
|
||||
entity.Source == PiiDetectionSource.Ner &&
|
||||
|
||||
entity.ModelOrigin == NerModelOrigin.English &&
|
||||
entity.Value.Contains(expectedNamePart, StringComparison.Ordinal) &&
|
||||
|
||||
prompt.AsSpan(entity.StartIndex, entity.Length).ToString() == entity.Value);
|
||||
@@ -43,9 +40,6 @@ public sealed class RealNerModelRunnerTests : RealNerModelFixture
|
||||
|
||||
|
||||
entities.Should().Contain(entity => entity.Value == expectedValue);
|
||||
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.English);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -21,15 +21,18 @@ public sealed class RealTamilNerModelRunnerTests : RealTamilNerModelFixture
|
||||
string expectedNamePart,
|
||||
string expectedValue)
|
||||
{
|
||||
var entities = Runner.PredictEntities(prompt);
|
||||
var result = Runner.PredictEntities(prompt);
|
||||
var entities = result.Entities;
|
||||
|
||||
entities.Should().Contain(entity =>
|
||||
entity.Type == PiiEntityType.Person &&
|
||||
entity.Source == PiiDetectionSource.Ner &&
|
||||
entity.ModelOrigin == NerModelOrigin.Tamil &&
|
||||
entity.Value.Contains(expectedNamePart, StringComparison.Ordinal) &&
|
||||
prompt.AsSpan(entity.StartIndex, entity.Length).ToString() == entity.Value);
|
||||
|
||||
entities.Should().Contain(entity => entity.Value == expectedValue);
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
[Test]
|
||||
@@ -37,11 +40,12 @@ public sealed class RealTamilNerModelRunnerTests : RealTamilNerModelFixture
|
||||
{
|
||||
const string prompt = "வாடிக்கையாளர் ராஜேஷ் குமார் தொலைபேசி 9876543210";
|
||||
|
||||
var entities = Runner.PredictEntities(prompt);
|
||||
var entities = Runner.PredictEntities(prompt).Entities;
|
||||
|
||||
entities.Should().Contain(entity =>
|
||||
entity.Type == PiiEntityType.Person &&
|
||||
entity.Source == PiiDetectionSource.Ner &&
|
||||
entity.ModelOrigin == NerModelOrigin.Tamil &&
|
||||
entity.Value.Contains("ராஜேஷ்", StringComparison.Ordinal));
|
||||
entities.Should().NotContain(entity => entity.Type == PiiEntityType.Phone);
|
||||
}
|
||||
@@ -51,7 +55,7 @@ public sealed class RealTamilNerModelRunnerTests : RealTamilNerModelFixture
|
||||
{
|
||||
const string prompt = "பணத்தை திரும்பப் பெறுவது எப்படி?";
|
||||
|
||||
Runner.PredictEntities(prompt).Should().BeEmpty();
|
||||
Runner.PredictEntities(prompt).Entities.Should().BeEmpty();
|
||||
}
|
||||
|
||||
[TestCase("Customer Senthil phone 9876543210", "Senthil")]
|
||||
@@ -61,7 +65,7 @@ public sealed class RealTamilNerModelRunnerTests : RealTamilNerModelFixture
|
||||
{
|
||||
// TamilOnnxNerRunner is script-scoped; Tanglish is handled by English routing in pipeline tests.
|
||||
// Direct Tamil runner on Latin-only text should not emit person spans.
|
||||
var entities = Runner.PredictEntities(prompt);
|
||||
var entities = Runner.PredictEntities(prompt).Entities;
|
||||
|
||||
entities.Should().NotContain(entity =>
|
||||
entity.Type == PiiEntityType.Person &&
|
||||
|
||||
@@ -11,13 +11,15 @@ public sealed class RoutingOnnxNerModelRunnerTests
|
||||
[Test]
|
||||
public void PredictEntities_LatinOnly_UsesEnglishRunnerOnly()
|
||||
{
|
||||
var english = new FakeLanguageNerRunner("Ravi Kumar");
|
||||
var tamil = new FakeLanguageNerRunner("தமிழ் பெயர்");
|
||||
var english = new FakeLanguageNerRunner("Ravi Kumar", NerModelOrigin.English);
|
||||
var tamil = new FakeLanguageNerRunner("தமிழ் பெயர்", NerModelOrigin.Tamil);
|
||||
var router = new RoutingOnnxNerModelRunner(english, tamil, enableTamilNer: true);
|
||||
|
||||
var entities = router.PredictEntities("Customer Ravi Kumar called.");
|
||||
var result = router.PredictEntities("Customer Ravi Kumar called.");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Value == "Ravi Kumar");
|
||||
result.Entities.Should().ContainSingle(entity =>
|
||||
entity.Value == "Ravi Kumar" && entity.ModelOrigin == NerModelOrigin.English);
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.English);
|
||||
english.CallCount.Should().Be(1);
|
||||
tamil.CallCount.Should().Be(0);
|
||||
}
|
||||
@@ -25,38 +27,46 @@ public sealed class RoutingOnnxNerModelRunnerTests
|
||||
[Test]
|
||||
public void PredictEntities_TamilOnly_UsesTamilRunnerOnly()
|
||||
{
|
||||
var english = new FakeLanguageNerRunner("Ravi Kumar");
|
||||
var tamil = new FakeLanguageNerRunner("ராஜேஷ்");
|
||||
var english = new FakeLanguageNerRunner("Ravi Kumar", NerModelOrigin.English);
|
||||
var tamil = new FakeLanguageNerRunner("ராஜேஷ்", NerModelOrigin.Tamil);
|
||||
var router = new RoutingOnnxNerModelRunner(english, tamil, enableTamilNer: true);
|
||||
|
||||
var entities = router.PredictEntities("வாடிக்கையாளர் ராஜேஷ்");
|
||||
var result = router.PredictEntities("வாடிக்கையாளர் ராஜேஷ்");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Value == "ராஜேஷ்");
|
||||
result.Entities.Should().ContainSingle(entity =>
|
||||
entity.Value == "ராஜேஷ்" && entity.ModelOrigin == NerModelOrigin.Tamil);
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.Tamil);
|
||||
english.CallCount.Should().Be(0);
|
||||
tamil.CallCount.Should().Be(1);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void PredictEntities_Mixed_InvokesBothRunners()
|
||||
public void PredictEntities_Mixed_InvokesBothRunnersAndTagsOrigins()
|
||||
{
|
||||
var english = new FakeLanguageNerRunner("EnglishName");
|
||||
var tamil = new FakeLanguageNerRunner("தமிழ்");
|
||||
var english = new FakeLanguageNerRunner("Priya", NerModelOrigin.English);
|
||||
var tamil = new FakeLanguageNerRunner("மற்றும்", NerModelOrigin.Tamil);
|
||||
var router = new RoutingOnnxNerModelRunner(english, tamil, enableTamilNer: true);
|
||||
|
||||
router.PredictEntities("Rajesh மற்றும் Priya");
|
||||
var result = router.PredictEntities("Rajesh மற்றும் Priya");
|
||||
|
||||
english.CallCount.Should().Be(1);
|
||||
tamil.CallCount.Should().Be(1);
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.English, NerModelOrigin.Tamil);
|
||||
result.Entities.Should().Contain(entity => entity.ModelOrigin == NerModelOrigin.English);
|
||||
result.Entities.Should().Contain(entity => entity.ModelOrigin == NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void PredictEntities_NoLetters_InvokesNeither()
|
||||
{
|
||||
var english = new FakeLanguageNerRunner("ignored");
|
||||
var tamil = new FakeLanguageNerRunner("ignored");
|
||||
var english = new FakeLanguageNerRunner("ignored", NerModelOrigin.English);
|
||||
var tamil = new FakeLanguageNerRunner("ignored", NerModelOrigin.Tamil);
|
||||
var router = new RoutingOnnxNerModelRunner(english, tamil, enableTamilNer: true);
|
||||
|
||||
router.PredictEntities("9876543210").Should().BeEmpty();
|
||||
var result = router.PredictEntities("9876543210");
|
||||
|
||||
result.Entities.Should().BeEmpty();
|
||||
result.InvokedModels.Should().BeEmpty();
|
||||
english.CallCount.Should().Be(0);
|
||||
tamil.CallCount.Should().Be(0);
|
||||
}
|
||||
@@ -64,14 +74,15 @@ public sealed class RoutingOnnxNerModelRunnerTests
|
||||
[Test]
|
||||
public void PredictEntities_TamilDisabled_SkipsTamilRunnerForMixedText()
|
||||
{
|
||||
var english = new FakeLanguageNerRunner("EnglishName");
|
||||
var tamil = new FakeLanguageNerRunner("தமிழ்");
|
||||
var english = new FakeLanguageNerRunner("EnglishName", NerModelOrigin.English);
|
||||
var tamil = new FakeLanguageNerRunner("தமிழ்", NerModelOrigin.Tamil);
|
||||
var router = new RoutingOnnxNerModelRunner(english, tamil, enableTamilNer: false);
|
||||
|
||||
router.PredictEntities("Rajesh மற்றும் Priya");
|
||||
var result = router.PredictEntities("Rajesh மற்றும் Priya");
|
||||
|
||||
english.CallCount.Should().Be(1);
|
||||
tamil.CallCount.Should().Be(0);
|
||||
result.InvokedModels.Should().Equal(NerModelOrigin.English);
|
||||
}
|
||||
|
||||
[Test]
|
||||
@@ -79,40 +90,54 @@ public sealed class RoutingOnnxNerModelRunnerTests
|
||||
{
|
||||
var entities = new[]
|
||||
{
|
||||
CreatePerson("Raj", 0, 3),
|
||||
CreatePerson("Rajesh", 0, 6)
|
||||
CreatePerson("Raj", 0, 3, NerModelOrigin.English),
|
||||
CreatePerson("Rajesh", 0, 6, NerModelOrigin.Tamil)
|
||||
};
|
||||
|
||||
var merged = RoutingOnnxNerModelRunner.MergePersonSpans(entities);
|
||||
|
||||
merged.Should().ContainSingle(entity => entity.Value == "Rajesh");
|
||||
merged.Should().ContainSingle(entity =>
|
||||
entity.Value == "Rajesh" && entity.ModelOrigin == NerModelOrigin.Tamil);
|
||||
}
|
||||
|
||||
private static PiiEntity CreatePerson(string value, int start, int length) =>
|
||||
new(PiiEntityType.Person, value, start, length, PiiDetectionSource.Ner);
|
||||
private static PiiEntity CreatePerson(string value, int start, int length, NerModelOrigin origin) =>
|
||||
new(PiiEntityType.Person, value, start, length, PiiDetectionSource.Ner, ModelOrigin: origin);
|
||||
|
||||
private sealed class FakeLanguageNerRunner : IOnnxNerModelRunner
|
||||
{
|
||||
private readonly string _personValue;
|
||||
private readonly NerModelOrigin _origin;
|
||||
|
||||
public FakeLanguageNerRunner(string personValue) => _personValue = personValue;
|
||||
public FakeLanguageNerRunner(string personValue, NerModelOrigin origin)
|
||||
{
|
||||
_personValue = personValue;
|
||||
_origin = origin;
|
||||
}
|
||||
|
||||
public int CallCount { get; private set; }
|
||||
|
||||
public bool IsModelAvailable => true;
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
public NerPredictionResult PredictEntities(string text)
|
||||
{
|
||||
CallCount++;
|
||||
return
|
||||
var start = text.IndexOf(_personValue, StringComparison.Ordinal);
|
||||
if (start < 0)
|
||||
{
|
||||
start = 0;
|
||||
}
|
||||
|
||||
return new NerPredictionResult(
|
||||
[
|
||||
new PiiEntity(
|
||||
PiiEntityType.Person,
|
||||
_personValue,
|
||||
0,
|
||||
start,
|
||||
_personValue.Length,
|
||||
PiiDetectionSource.Ner)
|
||||
];
|
||||
PiiDetectionSource.Ner,
|
||||
ModelOrigin: _origin)
|
||||
],
|
||||
[_origin]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,7 +57,7 @@ public abstract class RealRoutingNerModelFixture
|
||||
Assert.Ignore(RealTamilNerModelPaths.ModelMissingMessage);
|
||||
}
|
||||
|
||||
Runner = new RoutingOnnxNerModelRunner(EnglishRunner, TamilRunner, options);
|
||||
Runner = new RoutingOnnxNerModelRunner(EnglishRunner, TamilRunner, options, NullLogger<RoutingOnnxNerModelRunner>.Instance);
|
||||
}
|
||||
|
||||
[OneTimeTearDown]
|
||||
|
||||
Reference in New Issue
Block a user