Add PII redaction POC for secure LLM prompting.
Implements detect-redact-sanitize pipeline with regex, domain rules, and ONNX NER before the LLM boundary, plus NUnit tests and Xenovex push documentation. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
23
.gitignore
vendored
Normal file
23
.gitignore
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
## .NET
|
||||
bin/
|
||||
obj/
|
||||
*.user
|
||||
*.suo
|
||||
.vs/
|
||||
|
||||
## ONNX models (place downloaded models here locally)
|
||||
models/*.onnx
|
||||
models/vocab.txt
|
||||
models/ner-labels.txt
|
||||
models/*.json
|
||||
!models/.gitkeep
|
||||
|
||||
## IDE
|
||||
.idea/
|
||||
*.swp
|
||||
|
||||
## Local experiments (not part of the solution)
|
||||
scratch/
|
||||
|
||||
## Test artifacts
|
||||
**/nunit_random_seed.tmp
|
||||
11
PiiRedaction.slnx
Normal file
11
PiiRedaction.slnx
Normal file
@@ -0,0 +1,11 @@
|
||||
<Solution>
|
||||
<Folder Name="/src/">
|
||||
<Project Path="src/PiiRedaction.ConsoleApp/PiiRedaction.ConsoleApp.csproj" />
|
||||
<Project Path="src/PiiRedaction.Core/PiiRedaction.Core.csproj" />
|
||||
<Project Path="src/PiiRedaction.Infrastructure/PiiRedaction.Infrastructure.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/tests/">
|
||||
<Project Path="tests/PiiRedaction.Core.Tests/PiiRedaction.Core.Tests.csproj" />
|
||||
<Project Path="tests/PiiRedaction.Infrastructure.Tests/PiiRedaction.Infrastructure.Tests.csproj" />
|
||||
</Folder>
|
||||
</Solution>
|
||||
281
README.md
Normal file
281
README.md
Normal file
@@ -0,0 +1,281 @@
|
||||
# PII Redaction POC
|
||||
|
||||
A proof-of-concept .NET solution that redacts personally identifiable information (PII) from user prompts **before** sending them to a large language model (LLM). The design demonstrates enterprise-grade separation of concerns using SOLID principles, dependency injection, and the `Microsoft.Extensions.AI` abstractions.
|
||||
|
||||
## Purpose
|
||||
|
||||
Financial and customer-service prompts often contain regulated data (names, government IDs, account numbers). This POC shows how to:
|
||||
|
||||
1. Accept a console prompt
|
||||
2. Detect PII using **Regex**, **ONNX NER**, and **domain rules**
|
||||
3. Replace values with stable placeholders
|
||||
4. Send only the **sanitized** prompt to an LLM (mocked for now)
|
||||
|
||||
## Architecture
|
||||
|
||||
For solution design, data-flow diagrams, trust boundaries, and project responsibilities, see **[docs/architecture.md](docs/architecture.md)**.
|
||||
|
||||
## Why Three Detection Strategies?
|
||||
|
||||
| Strategy | Used For | Rationale |
|
||||
|----------|----------|-----------|
|
||||
| **Regex** | Email, phone, PAN, Aadhaar, credit card | Deterministic, format-bound identifiers with stable rules that are easy to audit and test |
|
||||
| **ONNX NER** | Person names | Contextual entities without rigid formats; names vary widely in surface form |
|
||||
| **Domain rules** | Loan number, customer ID, account number | Business-specific identifiers defined by internal systems, not inferable from generic models alone |
|
||||
|
||||
## Why the LLM Receives Only Sanitized Text
|
||||
|
||||
The placeholder map (`<PERSON_1>` → original value) is kept **in-process** for audit or downstream de-tokenization. Only the sanitized prompt crosses the LLM boundary. This reduces data-exposure risk and supports compliance requirements for regulated workloads.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── PiiRedaction.ConsoleApp/ # Presentation: input/output, DI bootstrap
|
||||
├── PiiRedaction.Core/ # Business logic: detection, redaction, models
|
||||
└── PiiRedaction.Infrastructure/ # Technical adapters: ONNX Runtime, mock LLM
|
||||
models/ # Optional ONNX model files (gitignored)
|
||||
```
|
||||
|
||||
| Project | Responsibility |
|
||||
|---------|----------------|
|
||||
| `PiiRedaction.ConsoleApp` | Read prompt, call sanitizer, display results, call LLM service |
|
||||
| `PiiRedaction.Core` | PII detection abstractions, redaction, sanitization orchestration |
|
||||
| `PiiRedaction.Infrastructure` | ONNX model runner, `IChatClient` mock implementation |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [.NET SDK](https://dotnet.microsoft.com/download) 10.x (or compatible SDK for `net10.0`)
|
||||
- **ONNX NER model** for person-name detection (see [ONNX Model Setup](#onnx-model-setup))
|
||||
- Python 3.10+ (only for the model download script)
|
||||
|
||||
> **Note:** This environment targets `net10.0` because .NET 10 SDK is installed. The architecture is identical to the planned .NET 9 layout; change `TargetFramework` in `.csproj` files if you use .NET 9 SDK.
|
||||
|
||||
## Build and Run
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
dotnet restore
|
||||
dotnet build
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp
|
||||
```
|
||||
|
||||
By default the console app runs **11 curated sample prompts** covering NER/person names, regex identifiers, domain IDs, combined scenarios, and a clean no-PII ticket.
|
||||
|
||||
List available samples:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --list
|
||||
```
|
||||
|
||||
Run a single sample by index or name:
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --sample 2
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --name MrTitlePerson
|
||||
```
|
||||
|
||||
Interactive mode (enter your own prompt):
|
||||
|
||||
```bash
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --interactive
|
||||
```
|
||||
|
||||
### Console sample catalog
|
||||
|
||||
Samples are defined in [`SamplePromptCatalog.cs`](src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs).
|
||||
|
||||
| # | Name | Category | NER / Person example |
|
||||
|---|------|----------|----------------------|
|
||||
| 0 | FullFinancialWithCustomer | NER + Regex + Domain | `Customer Ravi Kumar` + email, phone, loan, PAN |
|
||||
| 1 | CustomerNameOnly | NER | `Customer Anita Sharma` |
|
||||
| 2 | MrTitlePerson | NER | `Mr. John Smith` |
|
||||
| 3 | MrsTitlePerson | NER | `Mrs. Lakshmi Reddy` |
|
||||
| 4 | DrTitlePerson | NER | `Dr. Jane Doe` |
|
||||
| 5 | TwoCustomersInOnePrompt | NER | `Customer Ravi Kumar` and `Customer Priya Nair` |
|
||||
| 6 | PersonWithDomainIds | NER + Domain | `Customer Meera Iyer` + CID / ACC |
|
||||
| 7 | PersonWithEmailNoPhone | NER + Regex | `Customer Arjun Mehta` + email |
|
||||
| 8 | AllRegexTypes | Regex | email, phone, PAN, Aadhaar, card |
|
||||
| 9 | AllDomainIds | Domain | LN, CID, ACC |
|
||||
| 10 | NoPiiCleanTicket | Negative | no redaction |
|
||||
|
||||
Person names are detected via **ONNX NER** using `dslim/bert-base-NER` (or a compatible token-classification export). A real model is **required** for person-name detection; there is no regex or heuristic fallback.
|
||||
|
||||
## NuGet Packages
|
||||
|
||||
```bash
|
||||
# Infrastructure
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.ML.OnnxRuntime
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.ML.Tokenizers
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.AI.Abstractions
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.AI
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.Logging.Abstractions
|
||||
dotnet add src/PiiRedaction.Infrastructure package Microsoft.Extensions.Options
|
||||
|
||||
# Core
|
||||
dotnet add src/PiiRedaction.Core package Microsoft.Extensions.Options
|
||||
|
||||
# ConsoleApp
|
||||
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Hosting
|
||||
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.DependencyInjection
|
||||
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Configuration.Json
|
||||
dotnet add src/PiiRedaction.ConsoleApp package Microsoft.Extensions.Configuration.EnvironmentVariables
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
[`appsettings.json`](src/PiiRedaction.ConsoleApp/appsettings.json):
|
||||
|
||||
```json
|
||||
{
|
||||
"PiiRedaction": {
|
||||
"OnnxModelPath": "models/ner-model.onnx"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Setting | Description |
|
||||
|---------|-------------|
|
||||
| `OnnxModelPath` | Path to ONNX NER model (relative to working directory or discovered by walking up from the current directory) |
|
||||
|
||||
## ONNX Model Setup
|
||||
|
||||
Person-name detection requires a token-classification ONNX model and companion tokenizer files in the `models/` directory:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `models/ner-model.onnx` | Exported NER model |
|
||||
| `models/vocab.txt` | BERT WordPiece vocabulary |
|
||||
| `models/ner-labels.txt` | One BIO label per line (`O`, `B-PER`, `I-PER`, etc.) |
|
||||
|
||||
### Download script
|
||||
|
||||
From the repository root:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
```
|
||||
|
||||
Or with Python directly:
|
||||
|
||||
```bash
|
||||
python scripts/download-ner-model.py
|
||||
```
|
||||
|
||||
The script exports [`dslim/bert-base-NER`](https://huggingface.co/dslim/bert-base-NER) via Hugging Face Optimum when Python is available. Otherwise it downloads the pre-exported ONNX assets from Hugging Face directly.
|
||||
|
||||
### Inference pipeline
|
||||
|
||||
`OnnxNerModelRunner` performs the full pipeline:
|
||||
|
||||
- BERT WordPiece tokenization (`Microsoft.ML.Tokenizers`)
|
||||
- ONNX Runtime inference (`input_ids`, `attention_mask`, optional `token_type_ids`)
|
||||
- BIO label decoding (`B-PER` / `I-PER` → `PiiEntityType.Person`)
|
||||
- Character-span alignment back to the source text
|
||||
|
||||
When the model or tokenizer files are missing, person detection returns no results.
|
||||
|
||||
## Swapping Mock LLM for Azure OpenAI
|
||||
|
||||
The application depends on `ILlmPromptService` (Core) and `IChatClient` (Microsoft.Extensions.AI). To use Azure OpenAI later, replace the mock registration in [`ServiceCollectionExtensions.cs`](src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs):
|
||||
|
||||
```csharp
|
||||
// Remove:
|
||||
// services.AddSingleton<IChatClient, MockChatClient>();
|
||||
|
||||
// Add (example — package and API may vary by provider SDK version):
|
||||
// services.AddAzureOpenAIChatClient(
|
||||
// new Uri(configuration["AzureOpenAI:Endpoint"]!),
|
||||
// configuration["AzureOpenAI:ApiKey"]!,
|
||||
// configuration["AzureOpenAI:DeploymentName"]!);
|
||||
|
||||
services.AddSingleton<ILlmPromptService, MockLlmPromptService>(); // unchanged
|
||||
```
|
||||
|
||||
`MockLlmPromptService` already uses `IChatClient`, so it works with any registered chat client implementation.
|
||||
|
||||
## Sample Execution Output
|
||||
|
||||
```
|
||||
=== PII Redaction POC ===
|
||||
|
||||
Original Prompt:
|
||||
Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.
|
||||
|
||||
Detected PII:
|
||||
[PERSON ] Ravi Kumar (Ner)
|
||||
[EMAIL ] ravi.kumar@gmail.com (Regex)
|
||||
[PHONE ] 9876543210 (Regex)
|
||||
[LOAN_NUMBER ] LN-456789 (Domain)
|
||||
[PAN ] ABCDE1234F (Regex)
|
||||
|
||||
Sanitized Prompt:
|
||||
Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.
|
||||
|
||||
Internal Placeholder Map (not sent to LLM):
|
||||
<EMAIL_1> -> ravi.kumar@gmail.com
|
||||
<LOAN_NUMBER_1> -> LN-456789
|
||||
<PAN_1> -> ABCDE1234F
|
||||
<PERSON_1> -> Ravi Kumar
|
||||
<PHONE_1> -> 9876543210
|
||||
|
||||
Mock LLM Response:
|
||||
[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.
|
||||
```
|
||||
|
||||
## SOLID Principles Applied
|
||||
|
||||
| Principle | Application |
|
||||
|-----------|-------------|
|
||||
| **Single Responsibility** | Each detector, redactor, and runner has one job; `Program.cs` only orchestrates |
|
||||
| **Open/Closed** | Add new `IPiiDetector` implementations without changing merge logic |
|
||||
| **Liskov Substitution** | All detectors are interchangeable via `IPiiDetector` |
|
||||
| **Interface Segregation** | Separate interfaces for detection, redaction, sanitization, and LLM |
|
||||
| **Dependency Inversion** | Core defines abstractions; Infrastructure implements them |
|
||||
|
||||
## Testing
|
||||
|
||||
The solution includes an **NUnit** test suite across two projects:
|
||||
|
||||
| Project | Focus |
|
||||
|---------|-------|
|
||||
| `tests/PiiRedaction.Core.Tests` | Detectors, redactor, sanitizer, golden pipeline scenarios (fake NER), real-model integration tests |
|
||||
| `tests/PiiRedaction.Infrastructure.Tests` | Mock LLM, ONNX runner unit tests, real-model NER runner tests |
|
||||
|
||||
### Run tests
|
||||
|
||||
```bash
|
||||
dotnet test
|
||||
dotnet test --filter "FullyQualifiedName~GoldenPromptTests"
|
||||
dotnet test --filter "Category=RealModel"
|
||||
dotnet test --logger "console;verbosity=detailed"
|
||||
```
|
||||
|
||||
Fast CI runs without the ONNX model: fake-based tests always execute; tests marked **`Category=RealModel`** are skipped when `models/ner-model.onnx` is absent. Download the model first:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
```
|
||||
|
||||
### Test architecture
|
||||
|
||||
- **`PromptScenarioCatalog`** — five focused end-to-end scenarios (canonical demo, multi-regex, duplicate people, overlap stress, no-PII negative)
|
||||
- **`ProductionPipelineFactory`** — builds the same Domain → Regex → OnnxNer composite stack as production DI; `CreateWithRealModel(runner)` wires a real `OnnxNerModelRunner`
|
||||
- **`FakeOnnxNerModelRunner`** — unit-test double for NER; golden tests inject person spans per scenario
|
||||
- **`GoldenPromptTests`** — end-to-end sanitization proof across the catalog (fake NER)
|
||||
- **`RealNerModelFixture`** — shared fixture that loads `models/ner-model.onnx` once per class; skips when model missing
|
||||
- **`RealNerModelRunnerTests`** — direct ONNX inference with span accuracy checks
|
||||
- **`RealNerPipelineTests`** — full pipeline with real NER (canonical, multi-person, clean-ticket negative)
|
||||
- **`OnnxNerModelRunnerTests`** — unit tests for missing/invalid model paths (no download required)
|
||||
- **`CompositePiiDetectorTests`** — overlap merge and source-priority rules
|
||||
- **`LlmBoundaryTests`** — verifies raw PII never appears in outbound LLM messages
|
||||
|
||||
Assertions use **FluentAssertions** for readable failures on long prompt strings.
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- ASP.NET Core API host with request/response middleware
|
||||
- Persistent audit log of redaction events (without storing raw PII)
|
||||
- Secure vault for reversible tokenization
|
||||
- Real Azure OpenAI / OpenAI provider registration
|
||||
295
docs/architecture.md
Normal file
295
docs/architecture.md
Normal file
@@ -0,0 +1,295 @@
|
||||
# PII Redaction POC — Solution Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes the architectural design of the **PII Redaction POC**, a .NET proof-of-concept that intercepts user prompts containing regulated personally identifiable information (PII), redacts sensitive values into stable placeholders, and transmits **only sanitized text** across the LLM trust boundary. The solution is structured for enterprise adoption: clear layer separation, interface-driven composition, dependency injection, and swappable infrastructure adapters (ONNX NER, `Microsoft.Extensions.AI` chat clients).
|
||||
|
||||
The POC validates a compliance-oriented pattern suitable for financial and customer-service workloads where raw PII must not leave the application process when invoking external language models.
|
||||
|
||||
---
|
||||
|
||||
## Canonical Example
|
||||
|
||||
The console application ships with a **sample catalog** (11 prompts). The canonical demo is sample `FullFinancialWithCustomer`. The table below shows the exact strings produced by the production pipeline when the ONNX NER model is loaded (run `scripts/download-ner-model.ps1` first).
|
||||
|
||||
| Stage | Value |
|
||||
|-------|-------|
|
||||
| **Input** | `Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.` |
|
||||
| **Sanitized Output** | `Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.` |
|
||||
| **Mock LLM Response** | `[Mock LLM Response] Received sanitized prompt (146 chars). No original PII was transmitted.` |
|
||||
|
||||
Detected entities for this prompt:
|
||||
|
||||
| Type | Value | Detection Source |
|
||||
|------|-------|------------------|
|
||||
| PERSON | Ravi Kumar | Ner |
|
||||
| EMAIL | ravi.kumar@gmail.com | Regex |
|
||||
| PHONE | 9876543210 | Regex |
|
||||
| LOAN_NUMBER | LN-456789 | Domain |
|
||||
| PAN | ABCDE1234F | Regex |
|
||||
|
||||
The internal placeholder map (`<PERSON_1>` → `Ravi Kumar`, etc.) is retained in-process and is **not** included in the outbound LLM request.
|
||||
|
||||
---
|
||||
|
||||
## Console Sample Catalog
|
||||
|
||||
Running `dotnet run --project src/PiiRedaction.ConsoleApp` executes all samples sequentially. Use `--list`, `--sample N`, or `--name SampleName` to filter.
|
||||
|
||||
### NER / person-name samples
|
||||
|
||||
These prompts exercise `OnnxNerPiiDetector` and `OnnxNerModelRunner`. Person names require the ONNX model (`models/ner-model.onnx` plus `vocab.txt` and `ner-labels.txt`). Without the model, person spans are not detected.
|
||||
|
||||
| Sample | Input (excerpt) | Detected person | Sanitized (excerpt) |
|
||||
|--------|-----------------|-----------------|---------------------|
|
||||
| **CustomerNameOnly** | Customer Anita Sharma reported unauthorized… | Anita Sharma | Customer `<PERSON_1>` reported unauthorized… |
|
||||
| **MrTitlePerson** | Mr. John Smith called about a duplicate debit… | John Smith | `<PERSON_1>` called about a duplicate debit… |
|
||||
| **MrsTitlePerson** | Mrs. Lakshmi Reddy requested a callback regarding LN-112233. | Lakshmi Reddy | `<PERSON_1>` requested a callback regarding `<LOAN_NUMBER_1>`. |
|
||||
| **DrTitlePerson** | Dr. Jane Doe escalated a complaint… | Jane Doe | `<PERSON_1>` escalated a complaint… |
|
||||
| **TwoCustomersInOnePrompt** | Customer Ravi Kumar and Customer Priya Nair… | Ravi Kumar, Priya Nair | Customer `<PERSON_1>` and Customer `<PERSON_2>`… |
|
||||
| **PersonWithDomainIds** | Customer Meera Iyer holds CID-7070… | Meera Iyer | Customer `<PERSON_1>` holds `<CUSTOMER_ID_1>`… |
|
||||
| **PersonWithEmailNoPhone** | Customer Arjun Mehta wrote from arjun.mehta@company.in… | Arjun Mehta | Customer `<PERSON_1>` wrote from `<EMAIL_1>`… |
|
||||
|
||||
### Other sample categories
|
||||
|
||||
| Category | Sample | Purpose |
|
||||
|----------|--------|---------|
|
||||
| NER + Regex + Domain | FullFinancialWithCustomer | End-to-end financial prompt (canonical) |
|
||||
| Regex only | AllRegexTypes | Email, phone, PAN, Aadhaar, credit card |
|
||||
| Domain only | AllDomainIds | Loan number, customer ID, account number |
|
||||
| Negative | NoPiiCleanTicket | Passthrough with no detected PII |
|
||||
|
||||
Sample definitions live in [`SamplePromptCatalog.cs`](../src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs).
|
||||
|
||||
---
|
||||
|
||||
## High-Level Data Flow
|
||||
|
||||
The diagram below traces the canonical example from console input through Core sanitization to the Infrastructure LLM adapter. Data labels reflect the canonical strings at each stage.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph consoleApp [PiiRedaction.ConsoleApp]
|
||||
program["Program.cs"]
|
||||
di["ServiceRegistration"]
|
||||
end
|
||||
|
||||
subgraph core [PiiRedaction.Core]
|
||||
sanitizer["PromptSanitizer"]
|
||||
composite["CompositePiiDetector"]
|
||||
regexDet["RegexPiiDetector"]
|
||||
domainDet["DomainRulePiiDetector"]
|
||||
onnxDet["OnnxNerPiiDetector"]
|
||||
redactor["PlaceholderPiiRedactor"]
|
||||
end
|
||||
|
||||
subgraph infra [PiiRedaction.Infrastructure]
|
||||
onnxRunner["OnnxNerModelRunner"]
|
||||
mockLlm["MockLlmPromptService"]
|
||||
mockChat["MockChatClient"]
|
||||
end
|
||||
|
||||
rawPrompt["Raw prompt with PII"]
|
||||
sanitizedPrompt["Sanitized prompt with placeholders"]
|
||||
llmResponse["Mock LLM acknowledgment"]
|
||||
|
||||
program -->|"Customer Ravi Kumar ... PAN ABCDE1234F"| sanitizer
|
||||
sanitizer --> composite
|
||||
composite --> domainDet
|
||||
composite --> regexDet
|
||||
composite --> onnxDet
|
||||
onnxDet --> onnxRunner
|
||||
sanitizer --> redactor
|
||||
redactor -->|"Customer PERSON_1 ... PAN PAN_1"| sanitizedPrompt
|
||||
program -->|"SanitizedPrompt only"| mockLlm
|
||||
mockLlm --> mockChat
|
||||
mockChat --> llmResponse
|
||||
|
||||
rawPrompt -.-> program
|
||||
di -.-> sanitizer
|
||||
di -.-> mockLlm
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Detection to Redaction Detail
|
||||
|
||||
`PromptSanitizer` orchestrates a two-phase pipeline: **detect** then **redact**. `CompositePiiDetector` aggregates spans from all registered detectors, resolves overlaps by registration order and source priority, and returns a merged entity list. `PlaceholderPiiRedactor` replaces spans right-to-left to preserve indices, assigns stable per-type counters, and builds the in-process placeholder map.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
inputText["Original prompt text"]
|
||||
|
||||
subgraph detectPhase [Detection Phase]
|
||||
domainDet["DomainRulePiiDetector"]
|
||||
regexDet["RegexPiiDetector"]
|
||||
onnxDet["OnnxNerPiiDetector"]
|
||||
composite["CompositePiiDetector"]
|
||||
merge["Overlap merge and source priority"]
|
||||
entityList["PiiEntity list"]
|
||||
end
|
||||
|
||||
subgraph redactPhase [Redaction Phase]
|
||||
redactor["PlaceholderPiiRedactor"]
|
||||
replace["Right-to-left span replacement"]
|
||||
placeholderMap["Placeholder map in-process"]
|
||||
sanitizedText["Sanitized text"]
|
||||
end
|
||||
|
||||
inputText --> domainDet
|
||||
inputText --> regexDet
|
||||
inputText --> onnxDet
|
||||
domainDet --> composite
|
||||
regexDet --> composite
|
||||
onnxDet --> composite
|
||||
composite --> merge
|
||||
merge --> entityList
|
||||
entityList --> redactor
|
||||
inputText --> redactor
|
||||
redactor --> replace
|
||||
replace --> sanitizedText
|
||||
replace --> placeholderMap
|
||||
```
|
||||
|
||||
**Overlap resolution rules** (applied by `CompositePiiDetector`):
|
||||
|
||||
1. Detectors run in registration order: **Domain → Regex → ONNX NER**.
|
||||
2. On overlapping spans, the first registered detector wins.
|
||||
3. Tie-breaking uses source priority: Domain (3) > Regex (2) > NER (1).
|
||||
|
||||
**Placeholder assignment** (applied by `PlaceholderPiiRedactor`):
|
||||
|
||||
- Format: `<{TYPE}_{n}>` (e.g. `<EMAIL_1>`, `<PERSON_1>`).
|
||||
- Duplicate values of the same type reuse the same placeholder.
|
||||
- Replacement proceeds from highest `StartIndex` to lowest to avoid index drift.
|
||||
|
||||
---
|
||||
|
||||
## Runtime Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant Program as Program.cs
|
||||
participant DI as ServiceProvider
|
||||
participant Sanitizer as PromptSanitizer
|
||||
participant Detector as CompositePiiDetector
|
||||
participant Redactor as PlaceholderPiiRedactor
|
||||
participant LlmSvc as MockLlmPromptService
|
||||
participant Chat as MockChatClient
|
||||
|
||||
User->>Program: Start application
|
||||
Program->>DI: Resolve IPromptSanitizer, ILlmPromptService
|
||||
DI-->>Program: Sanitizer, LlmService
|
||||
|
||||
alt Interactive mode
|
||||
User->>Program: Enter prompt via console
|
||||
else Default mode
|
||||
Program->>Program: Load canonical sample prompt
|
||||
end
|
||||
|
||||
Program->>Sanitizer: Sanitize(SanitizationRequest)
|
||||
Sanitizer->>Detector: Detect(originalPrompt)
|
||||
Detector-->>Sanitizer: IReadOnlyList PiiEntity
|
||||
Sanitizer->>Redactor: Redact(originalPrompt, entities)
|
||||
Redactor-->>Sanitizer: RedactionResult
|
||||
Sanitizer-->>Program: SanitizationResult
|
||||
|
||||
Program->>Program: Display detected entities
|
||||
Program->>Program: Display sanitized prompt
|
||||
Program->>Program: Display placeholder map in-process
|
||||
|
||||
Program->>LlmSvc: SendPromptAsync(sanitizedPrompt)
|
||||
Note over Program,LlmSvc: Placeholder map never passed
|
||||
LlmSvc->>Chat: GetResponseAsync(user message)
|
||||
Chat-->>LlmSvc: Assistant response
|
||||
LlmSvc-->>Program: Mock LLM response string
|
||||
Program->>User: Write LLM response to console
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Trust Boundary
|
||||
|
||||
The LLM boundary is the point at which data leaves the application process via `ILlmPromptService` / `IChatClient`. Only the sanitized prompt crosses this boundary. Original PII values, detection metadata, and the placeholder-to-value map remain in-process.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph inProcess [In-Process Trust Zone]
|
||||
originalPrompt["Original prompt with raw PII"]
|
||||
detectedEntities["Detected PiiEntity list"]
|
||||
placeholderMap["Placeholder map"]
|
||||
sanitizationResult["SanitizationResult"]
|
||||
consoleDisplay["Console audit output"]
|
||||
end
|
||||
|
||||
subgraph llmBoundary [LLM Trust Boundary]
|
||||
sanitizedOnly["Sanitized prompt text only"]
|
||||
end
|
||||
|
||||
subgraph externalLlm [External LLM Provider]
|
||||
chatClient["IChatClient implementation"]
|
||||
modelInference["Model inference"]
|
||||
end
|
||||
|
||||
originalPrompt --> sanitizationResult
|
||||
detectedEntities --> sanitizationResult
|
||||
placeholderMap --> sanitizationResult
|
||||
sanitizationResult --> consoleDisplay
|
||||
sanitizationResult -->|"SendPromptAsync"| sanitizedOnly
|
||||
sanitizedOnly --> chatClient
|
||||
chatClient --> modelInference
|
||||
|
||||
originalPrompt -.-x|"Never transmitted"| chatClient
|
||||
placeholderMap -.-x|"Never transmitted"| chatClient
|
||||
detectedEntities -.-x|"Never transmitted"| chatClient
|
||||
```
|
||||
|
||||
In the POC, `MockChatClient` simulates the external provider without network I/O. Replacing it with Azure OpenAI or another `IChatClient` implementation does not change the trust model: `MockLlmPromptService` (or a future production adapter) continues to accept only the sanitized string.
|
||||
|
||||
---
|
||||
|
||||
## Project Responsibilities
|
||||
|
||||
| Project | Layer | Responsibility |
|
||||
|---------|-------|----------------|
|
||||
| `PiiRedaction.ConsoleApp` | Presentation | Application entry point; reads prompt (sample or interactive); bootstraps `IHost` and DI via `AddPiiRedactionServices`; orchestrates sanitization and LLM invocation; renders audit output (detected entities, sanitized text, placeholder map). |
|
||||
| `PiiRedaction.Core` | Domain / Application | Defines abstractions (`IPiiDetector`, `IPiiRedactor`, `IPromptSanitizer`, `ILlmPromptService`); implements detection strategies (`RegexPiiDetector`, `DomainRulePiiDetector`, `OnnxNerPiiDetector`, `CompositePiiDetector`); implements `PlaceholderPiiRedactor` and `PromptSanitizer`; owns domain models (`PiiEntity`, `SanitizationResult`, `RedactionResult`) and configuration (`PiiRedactionOptions`). Has no dependency on ONNX Runtime or LLM SDKs. |
|
||||
| `PiiRedaction.Infrastructure` | Infrastructure | Implements technical adapters: `OnnxNerModelRunner` (ONNX Runtime inference), `MockChatClient` and `MockLlmPromptService` (`Microsoft.Extensions.AI`); depends on Core abstractions and is swappable without changing domain logic. |
|
||||
| `tests/PiiRedaction.Core.Tests` | Test | Unit and integration tests for detectors, redactor, sanitizer, overlap rules, golden prompt scenarios (`PromptScenarioCatalog`), and LLM boundary assertions. |
|
||||
| `tests/PiiRedaction.Infrastructure.Tests` | Test | Tests for mock LLM behavior and ONNX runner load semantics. |
|
||||
|
||||
**Dependency direction:** `ConsoleApp` → `Infrastructure` → `Core`. Core references no outer layers, preserving the Dependency Inversion Principle and enabling future hosts (ASP.NET Core API, worker services) to reuse the same Core and Infrastructure assemblies.
|
||||
|
||||
---
|
||||
|
||||
## Key Abstractions and Extension Points
|
||||
|
||||
| Abstraction | Defined In | Default Implementation | Extension |
|
||||
|-------------|------------|------------------------|-----------|
|
||||
| `IPiiDetector` | Core | `CompositePiiDetector` wrapping Domain, Regex, ONNX | Add new detector; register in composite order |
|
||||
| `IPiiRedactor` | Core | `PlaceholderPiiRedactor` | Replace with hashing, vault-backed tokens, etc. |
|
||||
| `IPromptSanitizer` | Core | `PromptSanitizer` | Unlikely to change; orchestrates detect + redact |
|
||||
| `ILlmPromptService` | Core | `MockLlmPromptService` | Production adapter with telemetry, retry, policy |
|
||||
| `IChatClient` | Microsoft.Extensions.AI | `MockChatClient` | Azure OpenAI, OpenAI, or other provider SDK |
|
||||
| `IOnnxNerModelRunner` | Core | `OnnxNerModelRunner` | BERT WordPiece tokenization, ONNX inference, BIO label decoding |
|
||||
|
||||
---
|
||||
|
||||
## Configuration Surface
|
||||
|
||||
Runtime behavior is controlled via `appsettings.json` under the `PiiRedaction` section:
|
||||
|
||||
| Setting | Effect |
|
||||
|---------|--------|
|
||||
| `OnnxModelPath` | Path to ONNX NER model (`models/ner-model.onnx` by default). Companion files `vocab.txt` and `ner-labels.txt` must live in the same directory. |
|
||||
|
||||
Download the model assets with `scripts/download-ner-model.ps1` (exports `dslim/bert-base-NER`).
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [README](../README.md) — build, run, configuration, and testing instructions
|
||||
- [ServiceCollectionExtensions.cs](../src/PiiRedaction.ConsoleApp/DependencyInjection/ServiceCollectionExtensions.cs) — DI registration and detector ordering
|
||||
- [PromptScenarioCatalog.cs](../tests/PiiRedaction.Core.Tests/TestSupport/PromptScenarioCatalog.cs) — focused golden pipeline scenarios including the canonical example
|
||||
62
docs/git-xenovex-setup.md
Normal file
62
docs/git-xenovex-setup.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# Git setup — Xenovex (xts.xenovex.com)
|
||||
|
||||
This repository is ready for push to your Xenovex Git server after you create a remote repository.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Git 2.x (installed at `C:\Program Files\Git\bin\git.exe`)
|
||||
- Access to https://xts.xenovex.com/explore/repos
|
||||
- .NET 10 SDK for build/test
|
||||
|
||||
## 1. Create the remote repository
|
||||
|
||||
1. Sign in to **https://xts.xenovex.com**
|
||||
2. Open **Explore repos** (or **New repository**)
|
||||
3. Create a new empty repository, e.g. `llm-pii-poc`
|
||||
4. Copy the **HTTPS** or **SSH** clone URL (example shapes):
|
||||
- `https://xts.xenovex.com/<org-or-user>/llm-pii-poc.git`
|
||||
- `git@xts.xenovex.com:<org-or-user>/llm-pii-poc.git`
|
||||
|
||||
Do **not** initialize the remote with a README if you are pushing an existing local history.
|
||||
|
||||
## 2. Add remote and push (from repository root)
|
||||
|
||||
```powershell
|
||||
cd C:\Users\bilal.n\Projects\llm-pii-poc
|
||||
|
||||
# Use full path if git is not on PATH
|
||||
$git = "C:\Program Files\Git\bin\git.exe"
|
||||
|
||||
& $git remote add origin <YOUR_CLONE_URL>
|
||||
& $git branch -M main
|
||||
& $git push -u origin main
|
||||
```
|
||||
|
||||
If the remote already has commits (e.g. auto-generated README), either use an empty remote or:
|
||||
|
||||
```powershell
|
||||
& $git pull origin main --rebase
|
||||
& $git push -u origin main
|
||||
```
|
||||
|
||||
## 3. What is committed vs excluded
|
||||
|
||||
| Included | Excluded (`.gitignore`) |
|
||||
|----------|-------------------------|
|
||||
| Source (`src/`), tests, scripts, docs | `bin/`, `obj/`, `.vs/` |
|
||||
| `README.md`, `PiiRedaction.slnx` | `models/*.onnx`, `vocab.txt`, `ner-labels.txt` (~431MB model) |
|
||||
| `models/.gitkeep` (empty models folder) | `scratch/` |
|
||||
|
||||
After clone, download the NER model locally:
|
||||
|
||||
```powershell
|
||||
.\scripts\download-ner-model.ps1
|
||||
```
|
||||
|
||||
## 4. Verify after clone
|
||||
|
||||
```powershell
|
||||
dotnet build
|
||||
dotnet test
|
||||
dotnet run --project src/PiiRedaction.ConsoleApp -- --sample 0
|
||||
```
|
||||
0
models/.gitkeep
Normal file
0
models/.gitkeep
Normal file
101
scripts/download-ner-model.ps1
Normal file
101
scripts/download-ner-model.ps1
Normal file
@@ -0,0 +1,101 @@
|
||||
# Downloads dslim/bert-base-NER ONNX assets to models/ for the PII Redaction POC.
|
||||
param(
|
||||
[string]$Python = "python"
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$repoRoot = Split-Path -Parent $PSScriptRoot
|
||||
$modelsDir = Join-Path $repoRoot "models"
|
||||
$scriptPath = Join-Path $PSScriptRoot "download-ner-model.py"
|
||||
$baseUrl = "https://huggingface.co/dslim/bert-base-NER/resolve/main/onnx"
|
||||
|
||||
function Ensure-ModelsDirectory {
|
||||
New-Item -ItemType Directory -Force -Path $modelsDir | Out-Null
|
||||
}
|
||||
|
||||
function Download-HuggingFaceAsset {
|
||||
param(
|
||||
[string]$RelativePath,
|
||||
[string]$Destination
|
||||
)
|
||||
|
||||
$url = "$baseUrl/$RelativePath"
|
||||
Write-Host "Downloading $url"
|
||||
Invoke-WebRequest -Uri $url -OutFile $Destination -UseBasicParsing
|
||||
}
|
||||
|
||||
function Export-LabelsFromConfig {
|
||||
param([string]$ConfigPath, [string]$LabelsPath)
|
||||
|
||||
$config = Get-Content $ConfigPath -Raw | ConvertFrom-Json
|
||||
$labelMap = @{}
|
||||
foreach ($property in $config.id2label.PSObject.Properties) {
|
||||
$labelMap[[int]$property.Name] = [string]$property.Value
|
||||
}
|
||||
|
||||
$labels = for ($index = 0; $index -lt $labelMap.Count; $index++) {
|
||||
$labelMap[$index]
|
||||
}
|
||||
|
||||
$labels | Set-Content -Path $LabelsPath -Encoding utf8
|
||||
}
|
||||
|
||||
function Download-WithPowerShell {
|
||||
Ensure-ModelsDirectory
|
||||
|
||||
$modelPath = Join-Path $modelsDir "ner-model.onnx"
|
||||
$vocabPath = Join-Path $modelsDir "vocab.txt"
|
||||
$configPath = Join-Path $modelsDir "config.json"
|
||||
$labelsPath = Join-Path $modelsDir "ner-labels.txt"
|
||||
|
||||
Download-HuggingFaceAsset -RelativePath "model.onnx" -Destination $modelPath
|
||||
Download-HuggingFaceAsset -RelativePath "vocab.txt" -Destination $vocabPath
|
||||
Download-HuggingFaceAsset -RelativePath "config.json" -Destination $configPath
|
||||
Export-LabelsFromConfig -ConfigPath $configPath -LabelsPath $labelsPath
|
||||
Remove-Item $configPath -Force
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "NER model assets saved:"
|
||||
Write-Host " $modelPath"
|
||||
Write-Host " $vocabPath"
|
||||
Write-Host " $labelsPath"
|
||||
Write-Host ""
|
||||
Write-Host "Run from repository root:"
|
||||
Write-Host " dotnet run --project src/PiiRedaction.ConsoleApp -- --name CustomerNameOnly"
|
||||
}
|
||||
|
||||
function Download-WithPython {
|
||||
if (-not (Get-Command $Python -ErrorAction SilentlyContinue)) {
|
||||
return $false
|
||||
}
|
||||
|
||||
$pythonCommand = Get-Command $Python
|
||||
if ($pythonCommand.Source -like "*WindowsApps*") {
|
||||
return $false
|
||||
}
|
||||
|
||||
Write-Host "Using Python: $($pythonCommand.Source)"
|
||||
Write-Host "Repository root: $repoRoot"
|
||||
Write-Host ""
|
||||
|
||||
Push-Location $repoRoot
|
||||
try {
|
||||
& $Python $scriptPath
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "Model download script failed with exit code $LASTEXITCODE."
|
||||
}
|
||||
}
|
||||
finally {
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
return $true
|
||||
}
|
||||
|
||||
Write-Host "Repository root: $repoRoot"
|
||||
|
||||
if (-not (Download-WithPython)) {
|
||||
Write-Host "Python export unavailable; downloading pre-exported ONNX assets from Hugging Face..."
|
||||
Write-Host ""
|
||||
Download-WithPowerShell
|
||||
}
|
||||
83
scripts/download-ner-model.py
Normal file
83
scripts/download-ner-model.py
Normal file
@@ -0,0 +1,83 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Download and export dslim/bert-base-NER to ONNX for the PII Redaction POC."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
MODELS_DIR = REPO_ROOT / "models"
|
||||
MODEL_ID = "dslim/bert-base-NER"
|
||||
REQUIRED_PACKAGES = ("transformers", "optimum[onnxruntime]", "onnx", "torch")
|
||||
|
||||
|
||||
def ensure_dependencies() -> None:
|
||||
try:
|
||||
import optimum.onnxruntime # noqa: F401
|
||||
import transformers # noqa: F401
|
||||
except ImportError:
|
||||
print("Installing Python dependencies (this may take a few minutes)...")
|
||||
subprocess.check_call(
|
||||
[sys.executable, "-m", "pip", "install", *REQUIRED_PACKAGES],
|
||||
stdout=sys.stdout,
|
||||
stderr=sys.stderr,
|
||||
)
|
||||
|
||||
|
||||
def export_model() -> None:
|
||||
from optimum.onnxruntime import ORTModelForTokenClassification
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
MODELS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
temp_dir = MODELS_DIR / "_export_temp"
|
||||
if temp_dir.exists():
|
||||
shutil.rmtree(temp_dir)
|
||||
temp_dir.mkdir()
|
||||
|
||||
print(f"Exporting {MODEL_ID} to ONNX...")
|
||||
model = ORTModelForTokenClassification.from_pretrained(MODEL_ID, export=True)
|
||||
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
|
||||
|
||||
model.save_pretrained(temp_dir)
|
||||
tokenizer.save_pretrained(temp_dir)
|
||||
|
||||
onnx_files = sorted(temp_dir.glob("*.onnx"))
|
||||
if not onnx_files:
|
||||
raise FileNotFoundError("Export completed but no .onnx file was produced.")
|
||||
|
||||
target_onnx = MODELS_DIR / "ner-model.onnx"
|
||||
shutil.copy(onnx_files[0], target_onnx)
|
||||
shutil.copy(temp_dir / "vocab.txt", MODELS_DIR / "vocab.txt")
|
||||
|
||||
config_path = temp_dir / "config.json"
|
||||
with config_path.open(encoding="utf-8") as config_file:
|
||||
config = json.load(config_file)
|
||||
|
||||
id2label = config.get("id2label", {})
|
||||
labels = [id2label[str(index)] for index in range(len(id2label))]
|
||||
(MODELS_DIR / "ner-labels.txt").write_text("\n".join(labels), encoding="utf-8")
|
||||
|
||||
shutil.rmtree(temp_dir)
|
||||
|
||||
print()
|
||||
print("NER model assets saved:")
|
||||
print(f" {target_onnx}")
|
||||
print(f" {MODELS_DIR / 'vocab.txt'}")
|
||||
print(f" {MODELS_DIR / 'ner-labels.txt'}")
|
||||
print()
|
||||
print("Run from repository root:")
|
||||
print(" dotnet run --project src/PiiRedaction.ConsoleApp -- --name CustomerNameOnly")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ensure_dependencies()
|
||||
export_model()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,41 @@
|
||||
using Microsoft.Extensions.AI;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Redaction;
|
||||
using PiiRedaction.Core.Sanitization;
|
||||
using PiiRedaction.Infrastructure.Llm;
|
||||
using PiiRedaction.Infrastructure.Onnx;
|
||||
|
||||
namespace PiiRedaction.ConsoleApp.DependencyInjection;
|
||||
|
||||
public static class ServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddPiiRedactionServices(this IServiceCollection services, IConfiguration configuration)
|
||||
{
|
||||
services.Configure<PiiRedactionOptions>(configuration.GetSection(PiiRedactionOptions.SectionName));
|
||||
|
||||
services.AddSingleton<DomainRulePiiDetector>();
|
||||
services.AddSingleton<RegexPiiDetector>();
|
||||
services.AddSingleton<OnnxNerPiiDetector>();
|
||||
|
||||
services.AddSingleton<IPiiDetector>(provider => new CompositePiiDetector(
|
||||
[
|
||||
provider.GetRequiredService<DomainRulePiiDetector>(),
|
||||
provider.GetRequiredService<RegexPiiDetector>(),
|
||||
provider.GetRequiredService<OnnxNerPiiDetector>()
|
||||
]));
|
||||
|
||||
services.AddSingleton<IPiiRedactor, PlaceholderPiiRedactor>();
|
||||
services.AddSingleton<IPromptSanitizer, PromptSanitizer>();
|
||||
|
||||
services.AddSingleton<OnnxNerModelRunner>();
|
||||
services.AddSingleton<IOnnxNerModelRunner>(provider => provider.GetRequiredService<OnnxNerModelRunner>());
|
||||
services.AddSingleton<IChatClient, MockChatClient>();
|
||||
services.AddSingleton<ILlmPromptService, MockLlmPromptService>();
|
||||
|
||||
return services;
|
||||
}
|
||||
}
|
||||
28
src/PiiRedaction.ConsoleApp/PiiRedaction.ConsoleApp.csproj
Normal file
28
src/PiiRedaction.ConsoleApp/PiiRedaction.ConsoleApp.csproj
Normal file
@@ -0,0 +1,28 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\PiiRedaction.Core\PiiRedaction.Core.csproj" />
|
||||
<ProjectReference Include="..\PiiRedaction.Infrastructure\PiiRedaction.Infrastructure.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="10.0.9" />
|
||||
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="10.0.9" />
|
||||
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.9" />
|
||||
<PackageReference Include="Microsoft.Extensions.Hosting" Version="10.0.9" />
|
||||
</ItemGroup>
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<None Update="appsettings.json">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
88
src/PiiRedaction.ConsoleApp/Program.cs
Normal file
88
src/PiiRedaction.ConsoleApp/Program.cs
Normal file
@@ -0,0 +1,88 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using PiiRedaction.ConsoleApp.DependencyInjection;
|
||||
using PiiRedaction.ConsoleApp.Samples;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
var interactive = args.Contains("--interactive", StringComparer.OrdinalIgnoreCase);
|
||||
var listSamples = args.Contains("--list", StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
var host = Host.CreateDefaultBuilder(args)
|
||||
.ConfigureServices((context, services) =>
|
||||
{
|
||||
services.AddPiiRedactionServices(context.Configuration);
|
||||
})
|
||||
.Build();
|
||||
|
||||
if (listSamples)
|
||||
{
|
||||
PromptDemoRunner.ListSamples();
|
||||
return;
|
||||
}
|
||||
|
||||
var sanitizer = host.Services.GetRequiredService<IPromptSanitizer>();
|
||||
var llmService = host.Services.GetRequiredService<ILlmPromptService>();
|
||||
var runner = new PromptDemoRunner(sanitizer, llmService);
|
||||
|
||||
Console.WriteLine("=== PII Redaction POC ===");
|
||||
Console.WriteLine();
|
||||
|
||||
if (interactive)
|
||||
{
|
||||
await RunInteractiveAsync(sanitizer, llmService);
|
||||
}
|
||||
else
|
||||
{
|
||||
var samples = ResolveSamples(args);
|
||||
await RunSamplesAsync(runner, samples);
|
||||
}
|
||||
|
||||
await host.StopAsync();
|
||||
|
||||
static IReadOnlyList<SamplePromptDefinition> ResolveSamples(string[] args)
|
||||
{
|
||||
var nameIndex = Array.FindIndex(args, arg => arg.Equals("--name", StringComparison.OrdinalIgnoreCase));
|
||||
if (nameIndex >= 0 && nameIndex + 1 < args.Length)
|
||||
{
|
||||
var sample = SamplePromptCatalog.FindByName(args[nameIndex + 1])
|
||||
?? throw new ArgumentException($"Unknown sample name: {args[nameIndex + 1]}");
|
||||
|
||||
return [sample];
|
||||
}
|
||||
|
||||
var sampleIndex = Array.FindIndex(args, arg => arg.Equals("--sample", StringComparison.OrdinalIgnoreCase));
|
||||
if (sampleIndex >= 0 && sampleIndex + 1 < args.Length)
|
||||
{
|
||||
if (!int.TryParse(args[sampleIndex + 1], out var index) ||
|
||||
index < 0 ||
|
||||
index >= SamplePromptCatalog.All.Count)
|
||||
{
|
||||
throw new ArgumentException($"Sample index must be between 0 and {SamplePromptCatalog.All.Count - 1}.");
|
||||
}
|
||||
|
||||
return [SamplePromptCatalog.All[index]];
|
||||
}
|
||||
|
||||
return SamplePromptCatalog.All;
|
||||
}
|
||||
|
||||
static async Task RunSamplesAsync(PromptDemoRunner runner, IReadOnlyList<SamplePromptDefinition> samples)
|
||||
{
|
||||
for (var i = 0; i < samples.Count; i++)
|
||||
{
|
||||
await runner.RunAsync(samples[i], i + 1, samples.Count);
|
||||
}
|
||||
}
|
||||
|
||||
static async Task RunInteractiveAsync(IPromptSanitizer sanitizer, ILlmPromptService llmService)
|
||||
{
|
||||
Console.WriteLine("Enter a prompt (press Enter on an empty line to finish):");
|
||||
var prompt = Console.ReadLine() ?? string.Empty;
|
||||
|
||||
var runner = new PromptDemoRunner(sanitizer, llmService);
|
||||
await runner.RunAsync(
|
||||
new SamplePromptDefinition("Interactive", "Custom", "User-provided prompt.", prompt),
|
||||
1,
|
||||
1);
|
||||
}
|
||||
124
src/PiiRedaction.ConsoleApp/Samples/PromptDemoRunner.cs
Normal file
124
src/PiiRedaction.ConsoleApp/Samples/PromptDemoRunner.cs
Normal file
@@ -0,0 +1,124 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.ConsoleApp.Samples;
|
||||
|
||||
public sealed class PromptDemoRunner
|
||||
{
|
||||
private readonly IPromptSanitizer _sanitizer;
|
||||
private readonly ILlmPromptService _llmService;
|
||||
|
||||
public PromptDemoRunner(IPromptSanitizer sanitizer, ILlmPromptService llmService)
|
||||
{
|
||||
_sanitizer = sanitizer;
|
||||
_llmService = llmService;
|
||||
}
|
||||
|
||||
public async Task RunAsync(SamplePromptDefinition sample, int index, int total)
|
||||
{
|
||||
Console.WriteLine(new string('=', 72));
|
||||
Console.WriteLine($"Sample {index}/{total}: {sample.Name} [{sample.Category}]");
|
||||
Console.WriteLine(sample.Description);
|
||||
Console.WriteLine(new string('-', 72));
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Original Prompt:");
|
||||
Console.WriteLine(sample.Prompt);
|
||||
Console.WriteLine();
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(sample.Prompt));
|
||||
|
||||
DisplayDetectedEntities(result.DetectedEntities);
|
||||
|
||||
Console.WriteLine("Sanitized Prompt:");
|
||||
Console.WriteLine(result.SanitizedPrompt);
|
||||
Console.WriteLine();
|
||||
|
||||
DisplayPlaceholderMap(result.Redaction.PlaceholderMap);
|
||||
|
||||
var llmResponse = await _llmService.SendPromptAsync(result.SanitizedPrompt);
|
||||
|
||||
Console.WriteLine("Mock LLM Response:");
|
||||
Console.WriteLine(llmResponse);
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
public static void ListSamples()
|
||||
{
|
||||
Console.WriteLine("Available samples:");
|
||||
Console.WriteLine();
|
||||
|
||||
for (var i = 0; i < SamplePromptCatalog.All.Count; i++)
|
||||
{
|
||||
var sample = SamplePromptCatalog.All[i];
|
||||
Console.WriteLine($" [{i}] {sample.Name,-28} [{sample.Category}] {sample.Description}");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Usage:");
|
||||
Console.WriteLine(" dotnet run --project src/PiiRedaction.ConsoleApp # run all samples");
|
||||
Console.WriteLine(" dotnet run --project src/PiiRedaction.ConsoleApp -- --sample 2");
|
||||
Console.WriteLine(" dotnet run --project src/PiiRedaction.ConsoleApp -- --name MrTitlePerson");
|
||||
Console.WriteLine(" dotnet run --project src/PiiRedaction.ConsoleApp -- --list");
|
||||
Console.WriteLine(" dotnet run --project src/PiiRedaction.ConsoleApp -- --interactive");
|
||||
}
|
||||
|
||||
private static void DisplayDetectedEntities(IReadOnlyList<PiiEntity> entities)
|
||||
{
|
||||
Console.WriteLine("Detected PII:");
|
||||
|
||||
if (entities.Count == 0)
|
||||
{
|
||||
Console.WriteLine(" (none)");
|
||||
Console.WriteLine();
|
||||
return;
|
||||
}
|
||||
|
||||
foreach (var entity in entities.OrderBy(e => e.StartIndex))
|
||||
{
|
||||
var sourceLabel = entity.Source switch
|
||||
{
|
||||
PiiDetectionSource.Regex => "Regex",
|
||||
PiiDetectionSource.Domain => "Domain",
|
||||
PiiDetectionSource.Ner => "Ner",
|
||||
_ => entity.Source.ToString()
|
||||
};
|
||||
|
||||
Console.WriteLine($" [{ToDisplayType(entity.Type),-13}] {entity.Value,-28} ({sourceLabel})");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
private static void DisplayPlaceholderMap(IReadOnlyDictionary<string, string> placeholderMap)
|
||||
{
|
||||
Console.WriteLine("Internal Placeholder Map (not sent to LLM):");
|
||||
|
||||
if (placeholderMap.Count == 0)
|
||||
{
|
||||
Console.WriteLine(" (none)");
|
||||
Console.WriteLine();
|
||||
return;
|
||||
}
|
||||
|
||||
foreach (var (placeholder, originalValue) in placeholderMap.OrderBy(pair => pair.Key, StringComparer.Ordinal))
|
||||
{
|
||||
Console.WriteLine($" {placeholder,-16} -> {originalValue}");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
private static string ToDisplayType(PiiEntityType type) => type switch
|
||||
{
|
||||
PiiEntityType.Person => "PERSON",
|
||||
PiiEntityType.Email => "EMAIL",
|
||||
PiiEntityType.Phone => "PHONE",
|
||||
PiiEntityType.Pan => "PAN",
|
||||
PiiEntityType.Aadhaar => "AADHAAR",
|
||||
PiiEntityType.CreditCard => "CREDIT_CARD",
|
||||
PiiEntityType.LoanNumber => "LOAN_NUMBER",
|
||||
PiiEntityType.CustomerId => "CUSTOMER_ID",
|
||||
PiiEntityType.AccountNumber => "ACCOUNT_NUMBER",
|
||||
_ => type.ToString().ToUpperInvariant()
|
||||
};
|
||||
}
|
||||
80
src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs
Normal file
80
src/PiiRedaction.ConsoleApp/Samples/SamplePromptCatalog.cs
Normal file
@@ -0,0 +1,80 @@
|
||||
namespace PiiRedaction.ConsoleApp.Samples;
|
||||
|
||||
/// <summary>
|
||||
/// Curated demonstration prompts for the console POC.
|
||||
/// NER samples require the ONNX model (see scripts/download-ner-model.ps1).
|
||||
/// </summary>
|
||||
public static class SamplePromptCatalog
|
||||
{
|
||||
public static IReadOnlyList<SamplePromptDefinition> All { get; } =
|
||||
[
|
||||
new(
|
||||
"FullFinancialWithCustomer",
|
||||
"NER + Regex + Domain",
|
||||
"Canonical demo: person name plus email, phone, loan number, and PAN.",
|
||||
"Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue."),
|
||||
|
||||
new(
|
||||
"CustomerNameOnly",
|
||||
"NER",
|
||||
"Person name detected via ONNX NER after 'Customer' keyword.",
|
||||
"Customer Anita Sharma reported unauthorized transactions on her savings account."),
|
||||
|
||||
new(
|
||||
"MrTitlePerson",
|
||||
"NER",
|
||||
"Person detected via ONNX NER (title prefix Mr.).",
|
||||
"Mr. John Smith called about a duplicate debit on 15 March."),
|
||||
|
||||
new(
|
||||
"MrsTitlePerson",
|
||||
"NER",
|
||||
"Person detected via ONNX NER (title prefix Mrs.).",
|
||||
"Mrs. Lakshmi Reddy requested a callback regarding LN-112233."),
|
||||
|
||||
new(
|
||||
"DrTitlePerson",
|
||||
"NER",
|
||||
"Person detected via ONNX NER (title prefix Dr).",
|
||||
"Dr. Jane Doe escalated a complaint about delayed loan disbursement."),
|
||||
|
||||
new(
|
||||
"TwoCustomersInOnePrompt",
|
||||
"NER",
|
||||
"Two distinct person names in the same prompt.",
|
||||
"Customer Ravi Kumar and Customer Priya Nair disputed the same charge."),
|
||||
|
||||
new(
|
||||
"PersonWithDomainIds",
|
||||
"NER + Domain",
|
||||
"Person name combined with business identifiers.",
|
||||
"Customer Meera Iyer holds CID-7070 and account ACC-606060 for verification."),
|
||||
|
||||
new(
|
||||
"PersonWithEmailNoPhone",
|
||||
"NER + Regex",
|
||||
"Person and email without phone number.",
|
||||
"Customer Arjun Mehta wrote from arjun.mehta@company.in about KYC renewal."),
|
||||
|
||||
new(
|
||||
"AllRegexTypes",
|
||||
"Regex",
|
||||
"Email, phone, PAN, Aadhaar, and credit card in one prompt.",
|
||||
"Email a@b.co phone 9001234567 PAN ABCDE1234F aadhaar 1234 5678 9012 card 4111-1111-1111-1111."),
|
||||
|
||||
new(
|
||||
"AllDomainIds",
|
||||
"Domain",
|
||||
"Loan number, customer ID, and account number together.",
|
||||
"Please verify LN-100200 for CustomerId CID-3000 on AccountNumber ACC-400500."),
|
||||
|
||||
new(
|
||||
"NoPiiCleanTicket",
|
||||
"Negative",
|
||||
"No PII — prompt passes through unchanged.",
|
||||
"What is the status of ticket TKT-99887 and when will the API maintenance end?")
|
||||
];
|
||||
|
||||
public static SamplePromptDefinition? FindByName(string name) =>
|
||||
All.FirstOrDefault(sample => sample.Name.Equals(name, StringComparison.OrdinalIgnoreCase));
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
namespace PiiRedaction.ConsoleApp.Samples;
|
||||
|
||||
public sealed record SamplePromptDefinition(
|
||||
string Name,
|
||||
string Category,
|
||||
string Description,
|
||||
string Prompt);
|
||||
5
src/PiiRedaction.ConsoleApp/appsettings.json
Normal file
5
src/PiiRedaction.ConsoleApp/appsettings.json
Normal file
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"PiiRedaction": {
|
||||
"OnnxModelPath": "models/ner-model.onnx"
|
||||
}
|
||||
}
|
||||
6
src/PiiRedaction.Core/Abstractions/ILlmPromptService.cs
Normal file
6
src/PiiRedaction.Core/Abstractions/ILlmPromptService.cs
Normal file
@@ -0,0 +1,6 @@
|
||||
namespace PiiRedaction.Core.Abstractions;
|
||||
|
||||
public interface ILlmPromptService
|
||||
{
|
||||
Task<string> SendPromptAsync(string sanitizedPrompt, CancellationToken cancellationToken = default);
|
||||
}
|
||||
8
src/PiiRedaction.Core/Abstractions/IPiiDetector.cs
Normal file
8
src/PiiRedaction.Core/Abstractions/IPiiDetector.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Abstractions;
|
||||
|
||||
public interface IPiiDetector
|
||||
{
|
||||
IReadOnlyList<PiiEntity> Detect(string text);
|
||||
}
|
||||
8
src/PiiRedaction.Core/Abstractions/IPiiRedactor.cs
Normal file
8
src/PiiRedaction.Core/Abstractions/IPiiRedactor.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Abstractions;
|
||||
|
||||
public interface IPiiRedactor
|
||||
{
|
||||
RedactionResult Redact(string text, IReadOnlyList<PiiEntity> entities);
|
||||
}
|
||||
8
src/PiiRedaction.Core/Abstractions/IPromptSanitizer.cs
Normal file
8
src/PiiRedaction.Core/Abstractions/IPromptSanitizer.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Abstractions;
|
||||
|
||||
public interface IPromptSanitizer
|
||||
{
|
||||
SanitizationResult Sanitize(SanitizationRequest request);
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
namespace PiiRedaction.Core.Configuration;
|
||||
|
||||
public sealed class PiiRedactionOptions
|
||||
{
|
||||
public const string SectionName = "PiiRedaction";
|
||||
|
||||
public string OnnxModelPath { get; set; } = "models/ner-model.onnx";
|
||||
}
|
||||
55
src/PiiRedaction.Core/Detection/CompositePiiDetector.cs
Normal file
55
src/PiiRedaction.Core/Detection/CompositePiiDetector.cs
Normal file
@@ -0,0 +1,55 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
/// <summary>
|
||||
/// Aggregates multiple PII detectors and merges overlapping spans.
|
||||
/// Detectors are applied in registration order; earlier detectors win on overlap.
|
||||
/// </summary>
|
||||
public sealed class CompositePiiDetector : IPiiDetector
|
||||
{
|
||||
private readonly IReadOnlyList<IPiiDetector> _detectors;
|
||||
|
||||
public CompositePiiDetector(IEnumerable<IPiiDetector> detectors)
|
||||
{
|
||||
_detectors = detectors.ToList();
|
||||
}
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
var candidates = _detectors
|
||||
.SelectMany(detector => detector.Detect(text))
|
||||
.OrderBy(entity => entity.StartIndex)
|
||||
.ThenByDescending(entity => entity.Length)
|
||||
.ThenByDescending(entity => GetSourcePriority(entity.Source))
|
||||
.ToList();
|
||||
|
||||
var merged = new List<PiiEntity>();
|
||||
|
||||
foreach (var candidate in candidates)
|
||||
{
|
||||
if (merged.Any(existing => Overlaps(existing, candidate)))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
merged.Add(candidate);
|
||||
}
|
||||
|
||||
return merged.OrderBy(entity => entity.StartIndex).ToList();
|
||||
}
|
||||
|
||||
private static bool Overlaps(PiiEntity left, PiiEntity right) =>
|
||||
left.StartIndex < right.EndIndex && right.StartIndex < left.EndIndex;
|
||||
|
||||
private static int GetSourcePriority(PiiDetectionSource source) => source switch
|
||||
{
|
||||
PiiDetectionSource.Domain => 3,
|
||||
PiiDetectionSource.Regex => 2,
|
||||
PiiDetectionSource.Ner => 1,
|
||||
_ => 0
|
||||
};
|
||||
}
|
||||
56
src/PiiRedaction.Core/Detection/DomainRulePiiDetector.cs
Normal file
56
src/PiiRedaction.Core/Detection/DomainRulePiiDetector.cs
Normal file
@@ -0,0 +1,56 @@
|
||||
using System.Text.RegularExpressions;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
/// <summary>
|
||||
/// Detects organization-specific identifiers using business domain rules.
|
||||
/// Domain rules are needed because identifiers like loan numbers and customer IDs
|
||||
/// are defined by internal systems and cannot be inferred reliably by generic NER or public regex alone.
|
||||
/// </summary>
|
||||
public sealed partial class DomainRulePiiDetector : IPiiDetector
|
||||
{
|
||||
private static readonly (PiiEntityType Type, Regex Pattern)[] Patterns =
|
||||
[
|
||||
(PiiEntityType.LoanNumber, LoanNumberPattern()),
|
||||
(PiiEntityType.CustomerId, CustomerIdPattern()),
|
||||
(PiiEntityType.AccountNumber, AccountNumberPattern())
|
||||
];
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
var entities = new List<PiiEntity>();
|
||||
|
||||
foreach (var (type, pattern) in Patterns)
|
||||
{
|
||||
foreach (Match match in pattern.Matches(text))
|
||||
{
|
||||
if (!match.Success)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
entities.Add(new PiiEntity(
|
||||
type,
|
||||
match.Value,
|
||||
match.Index,
|
||||
match.Length,
|
||||
PiiDetectionSource.Domain));
|
||||
}
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
|
||||
[GeneratedRegex(@"(?i)\bLN-\d{6,}\b", RegexOptions.Compiled)]
|
||||
private static partial Regex LoanNumberPattern();
|
||||
|
||||
[GeneratedRegex(@"(?i)\bCID-\d{4,}\b", RegexOptions.Compiled)]
|
||||
private static partial Regex CustomerIdPattern();
|
||||
|
||||
[GeneratedRegex(@"(?i)\bACC-\d{6,}\b", RegexOptions.Compiled)]
|
||||
private static partial Regex AccountNumberPattern();
|
||||
}
|
||||
11
src/PiiRedaction.Core/Detection/IOnnxNerModelRunner.cs
Normal file
11
src/PiiRedaction.Core/Detection/IOnnxNerModelRunner.cs
Normal file
@@ -0,0 +1,11 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
public interface IOnnxNerModelRunner
|
||||
{
|
||||
bool IsModelAvailable { get; }
|
||||
|
||||
IReadOnlyList<PiiEntity> PredictEntities(string text);
|
||||
}
|
||||
31
src/PiiRedaction.Core/Detection/OnnxNerPiiDetector.cs
Normal file
31
src/PiiRedaction.Core/Detection/OnnxNerPiiDetector.cs
Normal file
@@ -0,0 +1,31 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
/// <summary>
|
||||
/// Detects contextual entities such as person names using ONNX-based NER.
|
||||
/// NER is used for entities that lack rigid formats and vary in surface form across prompts.
|
||||
/// Requires a loaded ONNX model; returns no person entities when the model is unavailable.
|
||||
/// </summary>
|
||||
public sealed class OnnxNerPiiDetector : IPiiDetector
|
||||
{
|
||||
private readonly IOnnxNerModelRunner _modelRunner;
|
||||
|
||||
public OnnxNerPiiDetector(IOnnxNerModelRunner modelRunner)
|
||||
{
|
||||
_modelRunner = modelRunner;
|
||||
}
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
if (!_modelRunner.IsModelAvailable)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
return _modelRunner.PredictEntities(text);
|
||||
}
|
||||
}
|
||||
64
src/PiiRedaction.Core/Detection/RegexPiiDetector.cs
Normal file
64
src/PiiRedaction.Core/Detection/RegexPiiDetector.cs
Normal file
@@ -0,0 +1,64 @@
|
||||
using System.Text.RegularExpressions;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Detection;
|
||||
|
||||
/// <summary>
|
||||
/// Detects PII using deterministic regular-expression patterns.
|
||||
/// Regex is used for format-bound identifiers (email, PAN, phone, Aadhaar, credit card)
|
||||
/// where rules are stable, auditable, and produce predictable matches without model inference.
|
||||
/// </summary>
|
||||
public sealed partial class RegexPiiDetector : IPiiDetector
|
||||
{
|
||||
private static readonly (PiiEntityType Type, Regex Pattern)[] Patterns =
|
||||
[
|
||||
(PiiEntityType.Email, EmailPattern()),
|
||||
(PiiEntityType.Phone, PhonePattern()),
|
||||
(PiiEntityType.Aadhaar, AadhaarPattern()),
|
||||
(PiiEntityType.Pan, PanPattern()),
|
||||
(PiiEntityType.CreditCard, CreditCardPattern())
|
||||
];
|
||||
|
||||
public IReadOnlyList<PiiEntity> Detect(string text)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
|
||||
var entities = new List<PiiEntity>();
|
||||
|
||||
foreach (var (type, pattern) in Patterns)
|
||||
{
|
||||
foreach (Match match in pattern.Matches(text))
|
||||
{
|
||||
if (!match.Success)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
entities.Add(new PiiEntity(
|
||||
type,
|
||||
match.Value,
|
||||
match.Index,
|
||||
match.Length,
|
||||
PiiDetectionSource.Regex));
|
||||
}
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
|
||||
[GeneratedRegex(@"[\w.+-]+@[\w.-]+\.\w+", RegexOptions.Compiled)]
|
||||
private static partial Regex EmailPattern();
|
||||
|
||||
[GeneratedRegex(@"(?<!\d)\d{10}(?!\d)", RegexOptions.Compiled)]
|
||||
private static partial Regex PhonePattern();
|
||||
|
||||
[GeneratedRegex(@"\b\d{4}\s?\d{4}\s?\d{4}\b", RegexOptions.Compiled)]
|
||||
private static partial Regex AadhaarPattern();
|
||||
|
||||
[GeneratedRegex(@"\b[A-Z]{5}\d{4}[A-Z]\b", RegexOptions.Compiled)]
|
||||
private static partial Regex PanPattern();
|
||||
|
||||
[GeneratedRegex(@"\b(?:\d{4}[-\s]?){3}\d{4}\b", RegexOptions.Compiled)]
|
||||
private static partial Regex CreditCardPattern();
|
||||
}
|
||||
8
src/PiiRedaction.Core/Models/PiiDetectionSource.cs
Normal file
8
src/PiiRedaction.Core/Models/PiiDetectionSource.cs
Normal file
@@ -0,0 +1,8 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public enum PiiDetectionSource
|
||||
{
|
||||
Regex,
|
||||
Domain,
|
||||
Ner
|
||||
}
|
||||
12
src/PiiRedaction.Core/Models/PiiEntity.cs
Normal file
12
src/PiiRedaction.Core/Models/PiiEntity.cs
Normal file
@@ -0,0 +1,12 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public sealed record PiiEntity(
|
||||
PiiEntityType Type,
|
||||
string Value,
|
||||
int StartIndex,
|
||||
int Length,
|
||||
PiiDetectionSource Source,
|
||||
double? Confidence = null)
|
||||
{
|
||||
public int EndIndex => StartIndex + Length;
|
||||
}
|
||||
14
src/PiiRedaction.Core/Models/PiiEntityType.cs
Normal file
14
src/PiiRedaction.Core/Models/PiiEntityType.cs
Normal file
@@ -0,0 +1,14 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public enum PiiEntityType
|
||||
{
|
||||
Person,
|
||||
Email,
|
||||
Phone,
|
||||
Pan,
|
||||
Aadhaar,
|
||||
CreditCard,
|
||||
LoanNumber,
|
||||
CustomerId,
|
||||
AccountNumber
|
||||
}
|
||||
5
src/PiiRedaction.Core/Models/RedactionResult.cs
Normal file
5
src/PiiRedaction.Core/Models/RedactionResult.cs
Normal file
@@ -0,0 +1,5 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public sealed record RedactionResult(
|
||||
string SanitizedText,
|
||||
IReadOnlyDictionary<string, string> PlaceholderMap);
|
||||
3
src/PiiRedaction.Core/Models/SanitizationRequest.cs
Normal file
3
src/PiiRedaction.Core/Models/SanitizationRequest.cs
Normal file
@@ -0,0 +1,3 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public sealed record SanitizationRequest(string OriginalPrompt);
|
||||
7
src/PiiRedaction.Core/Models/SanitizationResult.cs
Normal file
7
src/PiiRedaction.Core/Models/SanitizationResult.cs
Normal file
@@ -0,0 +1,7 @@
|
||||
namespace PiiRedaction.Core.Models;
|
||||
|
||||
public sealed record SanitizationResult(
|
||||
string OriginalPrompt,
|
||||
string SanitizedPrompt,
|
||||
IReadOnlyList<PiiEntity> DetectedEntities,
|
||||
RedactionResult Redaction);
|
||||
13
src/PiiRedaction.Core/PiiRedaction.Core.csproj
Normal file
13
src/PiiRedaction.Core/PiiRedaction.Core.csproj
Normal file
@@ -0,0 +1,13 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Options" Version="10.0.9" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
58
src/PiiRedaction.Core/Redaction/PlaceholderPiiRedactor.cs
Normal file
58
src/PiiRedaction.Core/Redaction/PlaceholderPiiRedactor.cs
Normal file
@@ -0,0 +1,58 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Redaction;
|
||||
|
||||
public sealed class PlaceholderPiiRedactor : IPiiRedactor
|
||||
{
|
||||
public RedactionResult Redact(string text, IReadOnlyList<PiiEntity> entities)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(text);
|
||||
ArgumentNullException.ThrowIfNull(entities);
|
||||
|
||||
var placeholderByValue = new Dictionary<string, string>(StringComparer.Ordinal);
|
||||
var counters = new Dictionary<PiiEntityType, int>();
|
||||
var orderedEntities = entities.OrderByDescending(entity => entity.StartIndex).ToList();
|
||||
var sanitized = text;
|
||||
|
||||
foreach (var entity in orderedEntities)
|
||||
{
|
||||
var mapKey = CreateValueKey(entity);
|
||||
if (!placeholderByValue.TryGetValue(mapKey, out var placeholder))
|
||||
{
|
||||
counters.TryGetValue(entity.Type, out var count);
|
||||
count++;
|
||||
counters[entity.Type] = count;
|
||||
placeholder = $"<{ToPlaceholderPrefix(entity.Type)}_{count}>";
|
||||
placeholderByValue[mapKey] = placeholder;
|
||||
}
|
||||
|
||||
sanitized = string.Concat(
|
||||
sanitized.AsSpan(0, entity.StartIndex),
|
||||
placeholder,
|
||||
sanitized.AsSpan(entity.EndIndex));
|
||||
}
|
||||
|
||||
var placeholderMap = placeholderByValue
|
||||
.ToDictionary(pair => pair.Value, pair => pair.Key.Split('|', 2)[1], StringComparer.Ordinal);
|
||||
|
||||
return new RedactionResult(sanitized, placeholderMap);
|
||||
}
|
||||
|
||||
private static string CreateValueKey(PiiEntity entity) =>
|
||||
$"{entity.Type}|{entity.Value}";
|
||||
|
||||
private static string ToPlaceholderPrefix(PiiEntityType type) => type switch
|
||||
{
|
||||
PiiEntityType.Person => "PERSON",
|
||||
PiiEntityType.Email => "EMAIL",
|
||||
PiiEntityType.Phone => "PHONE",
|
||||
PiiEntityType.Pan => "PAN",
|
||||
PiiEntityType.Aadhaar => "AADHAAR",
|
||||
PiiEntityType.CreditCard => "CREDIT_CARD",
|
||||
PiiEntityType.LoanNumber => "LOAN_NUMBER",
|
||||
PiiEntityType.CustomerId => "CUSTOMER_ID",
|
||||
PiiEntityType.AccountNumber => "ACCOUNT_NUMBER",
|
||||
_ => type.ToString().ToUpperInvariant()
|
||||
};
|
||||
}
|
||||
31
src/PiiRedaction.Core/Sanitization/PromptSanitizer.cs
Normal file
31
src/PiiRedaction.Core/Sanitization/PromptSanitizer.cs
Normal file
@@ -0,0 +1,31 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Sanitization;
|
||||
|
||||
public sealed class PromptSanitizer : IPromptSanitizer
|
||||
{
|
||||
private readonly IPiiDetector _detector;
|
||||
private readonly IPiiRedactor _redactor;
|
||||
|
||||
public PromptSanitizer(IPiiDetector detector, IPiiRedactor redactor)
|
||||
{
|
||||
_detector = detector;
|
||||
_redactor = redactor;
|
||||
}
|
||||
|
||||
public SanitizationResult Sanitize(SanitizationRequest request)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(request);
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(request.OriginalPrompt);
|
||||
|
||||
var entities = _detector.Detect(request.OriginalPrompt);
|
||||
var redaction = _redactor.Redact(request.OriginalPrompt, entities);
|
||||
|
||||
return new SanitizationResult(
|
||||
request.OriginalPrompt,
|
||||
redaction.SanitizedText,
|
||||
entities,
|
||||
redaction);
|
||||
}
|
||||
}
|
||||
41
src/PiiRedaction.Infrastructure/Llm/MockChatClient.cs
Normal file
41
src/PiiRedaction.Infrastructure/Llm/MockChatClient.cs
Normal file
@@ -0,0 +1,41 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Llm;
|
||||
|
||||
/// <summary>
|
||||
/// Mock chat client for POC demonstrations. Returns deterministic responses without external API calls.
|
||||
/// </summary>
|
||||
public sealed class MockChatClient : IChatClient
|
||||
{
|
||||
public Task<ChatResponse> GetResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
var userMessage = messages.LastOrDefault(message => message.Role == ChatRole.User);
|
||||
var userText = userMessage?.Text ?? string.Empty;
|
||||
|
||||
var response = new ChatResponse(new ChatMessage(
|
||||
ChatRole.Assistant,
|
||||
$"[Mock LLM Response] Received sanitized prompt ({userText.Length} chars). No original PII was transmitted."));
|
||||
|
||||
return Task.FromResult(response);
|
||||
}
|
||||
|
||||
public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||
{
|
||||
var response = await GetResponseAsync(messages, options, cancellationToken).ConfigureAwait(false);
|
||||
var text = response.Messages.LastOrDefault()?.Text ?? string.Empty;
|
||||
yield return new ChatResponseUpdate(ChatRole.Assistant, text);
|
||||
}
|
||||
|
||||
public object? GetService(Type serviceType, object? serviceKey = null) => null;
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
35
src/PiiRedaction.Infrastructure/Llm/MockLlmPromptService.cs
Normal file
35
src/PiiRedaction.Infrastructure/Llm/MockLlmPromptService.cs
Normal file
@@ -0,0 +1,35 @@
|
||||
using Microsoft.Extensions.AI;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Llm;
|
||||
|
||||
/// <summary>
|
||||
/// Sends only sanitized prompts to the LLM boundary.
|
||||
/// Original PII values and placeholder mappings remain in-process and are never transmitted.
|
||||
/// </summary>
|
||||
public sealed class MockLlmPromptService : ILlmPromptService
|
||||
{
|
||||
private readonly IChatClient _chatClient;
|
||||
|
||||
public MockLlmPromptService(IChatClient chatClient)
|
||||
{
|
||||
_chatClient = chatClient;
|
||||
}
|
||||
|
||||
public async Task<string> SendPromptAsync(string sanitizedPrompt, CancellationToken cancellationToken = default)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(sanitizedPrompt);
|
||||
|
||||
var messages = new[]
|
||||
{
|
||||
new ChatMessage(ChatRole.User, sanitizedPrompt)
|
||||
};
|
||||
|
||||
var response = await _chatClient
|
||||
.GetResponseAsync(messages, cancellationToken: cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
return response.Messages.LastOrDefault(message => message.Role == ChatRole.Assistant)?.Text
|
||||
?? string.Empty;
|
||||
}
|
||||
}
|
||||
325
src/PiiRedaction.Infrastructure/Onnx/OnnxNerModelRunner.cs
Normal file
325
src/PiiRedaction.Infrastructure/Onnx/OnnxNerModelRunner.cs
Normal file
@@ -0,0 +1,325 @@
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Microsoft.ML.OnnxRuntime;
|
||||
using Microsoft.ML.OnnxRuntime.Tensors;
|
||||
using Microsoft.ML.Tokenizers;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Onnx;
|
||||
|
||||
/// <summary>
|
||||
/// Wraps ONNX Runtime inference for NER models.
|
||||
/// Tokenization and tensor preparation are isolated here so detectors remain model-agnostic.
|
||||
/// </summary>
|
||||
public sealed class OnnxNerModelRunner : IOnnxNerModelRunner, IDisposable
|
||||
{
|
||||
private const int MaxSequenceLength = 128;
|
||||
|
||||
private readonly ILogger<OnnxNerModelRunner> _logger;
|
||||
private readonly string _modelPath;
|
||||
private readonly BertTokenizer? _tokenizer;
|
||||
private readonly string[] _labels;
|
||||
private InferenceSession? _session;
|
||||
|
||||
public OnnxNerModelRunner(IOptions<PiiRedactionOptions> options, ILogger<OnnxNerModelRunner> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
_modelPath = ResolveAssetPath(options.Value.OnnxModelPath);
|
||||
var modelDirectory = Path.GetDirectoryName(_modelPath) ?? Environment.CurrentDirectory;
|
||||
_labels = LoadLabels(modelDirectory);
|
||||
_tokenizer = TryLoadTokenizer(modelDirectory);
|
||||
_session = TryCreateSession();
|
||||
}
|
||||
|
||||
public bool IsModelAvailable => _session is not null && _tokenizer is not null && _labels.Length > 0;
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
{
|
||||
if (!IsModelAvailable || _session is null || _tokenizer is null)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
var encodedTokens = _tokenizer.EncodeToTokens(text, out _, considerPreTokenization: true, considerNormalization: true);
|
||||
var wordTokens = encodedTokens.Take(Math.Max(0, MaxSequenceLength - 2)).ToList();
|
||||
if (wordTokens.Count == 0)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
var sequenceLength = wordTokens.Count + 2;
|
||||
var inputIds = new long[sequenceLength];
|
||||
var attentionMask = new long[sequenceLength];
|
||||
var tokenTypeIds = new long[sequenceLength];
|
||||
var offsets = new (int Start, int End)[sequenceLength];
|
||||
var tokenIds = new int[sequenceLength];
|
||||
|
||||
inputIds[0] = _tokenizer.ClassificationTokenId;
|
||||
attentionMask[0] = 1;
|
||||
tokenIds[0] = _tokenizer.ClassificationTokenId;
|
||||
offsets[0] = (0, 0);
|
||||
|
||||
for (var i = 0; i < wordTokens.Count; i++)
|
||||
{
|
||||
var token = wordTokens[i];
|
||||
var index = i + 1;
|
||||
inputIds[index] = token.Id;
|
||||
attentionMask[index] = 1;
|
||||
tokenIds[index] = token.Id;
|
||||
offsets[index] = ToCharOffsets(token.Offset, text.Length);
|
||||
}
|
||||
|
||||
inputIds[sequenceLength - 1] = _tokenizer.SeparatorTokenId;
|
||||
attentionMask[sequenceLength - 1] = 1;
|
||||
tokenIds[sequenceLength - 1] = _tokenizer.SeparatorTokenId;
|
||||
offsets[sequenceLength - 1] = (0, 0);
|
||||
|
||||
var predictedLabelIds = RunInference(inputIds, attentionMask, tokenTypeIds, sequenceLength);
|
||||
return DecodePersonEntities(text, predictedLabelIds, offsets, tokenIds, sequenceLength);
|
||||
}
|
||||
|
||||
private int[] RunInference(long[] inputIds, long[] attentionMask, long[] tokenTypeIds, int sequenceLength)
|
||||
{
|
||||
var inputIdsTensor = CreateTensor(inputIds, sequenceLength);
|
||||
var attentionMaskTensor = CreateTensor(attentionMask, sequenceLength);
|
||||
var inputs = new List<NamedOnnxValue>
|
||||
{
|
||||
NamedOnnxValue.CreateFromTensor(_session!.InputMetadata.Keys.First(key => key.Contains("input_ids", StringComparison.OrdinalIgnoreCase)), inputIdsTensor),
|
||||
NamedOnnxValue.CreateFromTensor(_session.InputMetadata.Keys.First(key => key.Contains("attention_mask", StringComparison.OrdinalIgnoreCase)), attentionMaskTensor)
|
||||
};
|
||||
|
||||
var tokenTypeInputName = _session.InputMetadata.Keys.FirstOrDefault(key => key.Contains("token_type", StringComparison.OrdinalIgnoreCase));
|
||||
if (tokenTypeInputName is not null)
|
||||
{
|
||||
inputs.Add(NamedOnnxValue.CreateFromTensor(tokenTypeInputName, CreateTensor(tokenTypeIds, sequenceLength)));
|
||||
}
|
||||
|
||||
using var results = _session.Run(inputs);
|
||||
var outputName = _session.OutputMetadata.Keys.FirstOrDefault(key =>
|
||||
key.Contains("logits", StringComparison.OrdinalIgnoreCase))
|
||||
?? results.First().Name;
|
||||
var logits = results.First(result => result.Name == outputName).AsTensor<float>();
|
||||
var numLabels = _labels.Length;
|
||||
var predictions = new int[sequenceLength];
|
||||
|
||||
for (var tokenIndex = 0; tokenIndex < sequenceLength; tokenIndex++)
|
||||
{
|
||||
var bestLabel = 0;
|
||||
var bestScore = float.MinValue;
|
||||
|
||||
for (var labelIndex = 0; labelIndex < numLabels; labelIndex++)
|
||||
{
|
||||
var score = logits[0, tokenIndex, labelIndex];
|
||||
if (score > bestScore)
|
||||
{
|
||||
bestScore = score;
|
||||
bestLabel = labelIndex;
|
||||
}
|
||||
}
|
||||
|
||||
predictions[tokenIndex] = bestLabel;
|
||||
}
|
||||
|
||||
return predictions;
|
||||
}
|
||||
|
||||
private static DenseTensor<long> CreateTensor(long[] values, int sequenceLength)
|
||||
{
|
||||
var tensor = new DenseTensor<long>([1, sequenceLength]);
|
||||
for (var i = 0; i < sequenceLength; i++)
|
||||
{
|
||||
tensor[0, i] = values[i];
|
||||
}
|
||||
|
||||
return tensor;
|
||||
}
|
||||
|
||||
private IReadOnlyList<PiiEntity> DecodePersonEntities(
|
||||
string text,
|
||||
int[] predictedLabelIds,
|
||||
(int Start, int End)[] offsets,
|
||||
int[] tokenIds,
|
||||
int sequenceLength)
|
||||
{
|
||||
var entities = new List<PiiEntity>();
|
||||
int? entityStart = null;
|
||||
int? entityEnd = null;
|
||||
|
||||
void FlushEntity()
|
||||
{
|
||||
if (!entityStart.HasValue || !entityEnd.HasValue || entityEnd.Value <= entityStart.Value)
|
||||
{
|
||||
entityStart = null;
|
||||
entityEnd = null;
|
||||
return;
|
||||
}
|
||||
|
||||
var value = text[entityStart.Value..entityEnd.Value];
|
||||
if (!string.IsNullOrWhiteSpace(value))
|
||||
{
|
||||
entities.Add(new PiiEntity(
|
||||
PiiEntityType.Person,
|
||||
value,
|
||||
entityStart.Value,
|
||||
entityEnd.Value - entityStart.Value,
|
||||
PiiDetectionSource.Ner));
|
||||
}
|
||||
|
||||
entityStart = null;
|
||||
entityEnd = null;
|
||||
}
|
||||
|
||||
for (var i = 0; i < sequenceLength; i++)
|
||||
{
|
||||
if (IsSpecialToken(tokenIds[i]))
|
||||
{
|
||||
FlushEntity();
|
||||
continue;
|
||||
}
|
||||
|
||||
var label = _labels[predictedLabelIds[i]];
|
||||
var (start, end) = offsets[i];
|
||||
var hasOffset = end > start;
|
||||
|
||||
if (!IsPersonLabel(label))
|
||||
{
|
||||
FlushEntity();
|
||||
continue;
|
||||
}
|
||||
|
||||
if (label.StartsWith("B-", StringComparison.Ordinal))
|
||||
{
|
||||
FlushEntity();
|
||||
if (hasOffset)
|
||||
{
|
||||
entityStart = start;
|
||||
entityEnd = end;
|
||||
}
|
||||
}
|
||||
else if (label.StartsWith("I-", StringComparison.Ordinal))
|
||||
{
|
||||
if (!entityStart.HasValue && hasOffset)
|
||||
{
|
||||
entityStart = start;
|
||||
entityEnd = end;
|
||||
}
|
||||
else if (hasOffset)
|
||||
{
|
||||
entityEnd = Math.Max(entityEnd ?? end, end);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
FlushEntity();
|
||||
return entities;
|
||||
}
|
||||
|
||||
private bool IsSpecialToken(int tokenId) =>
|
||||
tokenId == _tokenizer!.ClassificationTokenId ||
|
||||
tokenId == _tokenizer.SeparatorTokenId ||
|
||||
tokenId == _tokenizer.PaddingTokenId;
|
||||
|
||||
private static (int Start, int End) ToCharOffsets(Range offset, int textLength)
|
||||
{
|
||||
var (start, length) = offset.GetOffsetAndLength(textLength);
|
||||
return (start, start + length);
|
||||
}
|
||||
|
||||
private static bool IsPersonLabel(string label) =>
|
||||
label is "B-PER" or "I-PER" or "B-PERSON" or "I-PERSON"
|
||||
|| (label.EndsWith("-PER", StringComparison.Ordinal) &&
|
||||
(label.StartsWith("B-", StringComparison.Ordinal) || label.StartsWith("I-", StringComparison.Ordinal)));
|
||||
|
||||
private InferenceSession? TryCreateSession()
|
||||
{
|
||||
if (!File.Exists(_modelPath))
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"ONNX NER model not found at {ModelPath}. Person-name detection will return no results.",
|
||||
_modelPath);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (_tokenizer is null || _labels.Length == 0)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Tokenizer vocabulary or label map missing for ONNX NER model at {ModelPath}.",
|
||||
_modelPath);
|
||||
return null;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
var session = new InferenceSession(_modelPath);
|
||||
_logger.LogInformation("ONNX NER model loaded from {ModelPath}.", _modelPath);
|
||||
return session;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to load ONNX NER model from {ModelPath}.", _modelPath);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private BertTokenizer? TryLoadTokenizer(string modelDirectory)
|
||||
{
|
||||
var vocabPath = ResolveAssetPath(Path.Combine(modelDirectory, "vocab.txt"));
|
||||
if (!File.Exists(vocabPath))
|
||||
{
|
||||
_logger.LogWarning("Tokenizer vocabulary not found at {VocabPath}.", vocabPath);
|
||||
return null;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
return BertTokenizer.Create(vocabPath, new BertOptions
|
||||
{
|
||||
LowerCaseBeforeTokenization = false
|
||||
});
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to load BERT tokenizer from {VocabPath}.", vocabPath);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static string[] LoadLabels(string modelDirectory)
|
||||
{
|
||||
var labelsPath = ResolveAssetPath(Path.Combine(modelDirectory, "ner-labels.txt"));
|
||||
if (!File.Exists(labelsPath))
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
return File.ReadAllLines(labelsPath)
|
||||
.Where(line => !string.IsNullOrWhiteSpace(line))
|
||||
.ToArray();
|
||||
}
|
||||
|
||||
internal static string ResolveAssetPath(string configuredPath)
|
||||
{
|
||||
if (Path.IsPathRooted(configuredPath) && File.Exists(configuredPath))
|
||||
{
|
||||
return configuredPath;
|
||||
}
|
||||
|
||||
var directory = new DirectoryInfo(Environment.CurrentDirectory);
|
||||
while (directory is not null)
|
||||
{
|
||||
var candidate = Path.GetFullPath(Path.Combine(directory.FullName, configuredPath));
|
||||
if (File.Exists(candidate))
|
||||
{
|
||||
return candidate;
|
||||
}
|
||||
|
||||
directory = directory.Parent;
|
||||
}
|
||||
|
||||
return Path.GetFullPath(Path.Combine(Environment.CurrentDirectory, configuredPath));
|
||||
}
|
||||
|
||||
public void Dispose() => _session?.Dispose();
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\PiiRedaction.Core\PiiRedaction.Core.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.AI" Version="10.7.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" Version="10.7.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.9" />
|
||||
<PackageReference Include="Microsoft.Extensions.Options" Version="10.0.9" />
|
||||
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.27.0" />
|
||||
<PackageReference Include="Microsoft.ML.Tokenizers" Version="2.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
</PropertyGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,127 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Detection;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class CompositePiiDetectorTests
|
||||
{
|
||||
[Test]
|
||||
public void Detect_AdjacentSpans_KeepsBothEntities()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.Email, "a@b.co", 0, 6, PiiDetectionSource.Regex),
|
||||
new PiiEntity(PiiEntityType.Phone, "9876543210", 6, 10, PiiDetectionSource.Regex))]);
|
||||
|
||||
var entities = composite.Detect("abcdef9876543210padding");
|
||||
|
||||
entities.Should().HaveCount(2);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_NestedSpan_KeepsLongerSpan()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.Aadhaar, "123456789012", 0, 12, PiiDetectionSource.Regex),
|
||||
new PiiEntity(PiiEntityType.Phone, "4567890123", 2, 10, PiiDetectionSource.Regex))]);
|
||||
|
||||
var entities = composite.Detect("123456789012");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Type == PiiEntityType.Aadhaar);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_OverlappingSameStart_LongerSpanWins()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.LoanNumber, "LN-456789", 0, 9, PiiDetectionSource.Domain),
|
||||
new PiiEntity(PiiEntityType.Phone, "456789", 3, 6, PiiDetectionSource.Regex))]);
|
||||
|
||||
var entities = composite.Detect("LN-456789");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Type == PiiEntityType.LoanNumber);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_OverlappingDifferentPriority_DomainBeatsRegex()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.LoanNumber, "LN-456789", 0, 9, PiiDetectionSource.Domain),
|
||||
new PiiEntity(PiiEntityType.Phone, "456789", 0, 6, PiiDetectionSource.Regex))]);
|
||||
|
||||
var entities = composite.Detect("LN-456789");
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.LoanNumber &&
|
||||
entity.Source == PiiDetectionSource.Domain);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_OverlappingPriority_RegexBeatsNer()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.Email, "a@b.co", 0, 6, PiiDetectionSource.Regex),
|
||||
new PiiEntity(PiiEntityType.Person, "a@b", 0, 3, PiiDetectionSource.Ner))]);
|
||||
|
||||
var entities = composite.Detect("a@b.co");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Source == PiiDetectionSource.Regex);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_OverlappingPriority_DomainBeatsNer()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.LoanNumber, "LN-456789", 0, 9, PiiDetectionSource.Domain),
|
||||
new PiiEntity(PiiEntityType.Person, "LN-456", 0, 6, PiiDetectionSource.Ner))]);
|
||||
|
||||
var entities = composite.Detect("LN-456789");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Source == PiiDetectionSource.Domain);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_DuplicateOverlappingSpan_KeepsFirstAccepted()
|
||||
{
|
||||
var composite = new CompositePiiDetector([new FixedDetector(
|
||||
new PiiEntity(PiiEntityType.Phone, "9876543210", 0, 10, PiiDetectionSource.Regex),
|
||||
new PiiEntity(PiiEntityType.Phone, "9876543210", 0, 10, PiiDetectionSource.Regex))]);
|
||||
|
||||
var entities = composite.Detect("9876543210");
|
||||
|
||||
entities.Should().HaveCount(1);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_AadhaarWithEmbeddedPhone_PrefersAadhaarSpan()
|
||||
{
|
||||
var detector = new CompositePiiDetector(
|
||||
[
|
||||
new DomainRulePiiDetector(),
|
||||
new RegexPiiDetector()
|
||||
]);
|
||||
|
||||
var entities = detector.Detect("Aadhaar 987654321012 phone 9876543210.");
|
||||
|
||||
entities.Should().Contain(entity => entity.Type == PiiEntityType.Aadhaar && entity.Value == "987654321012");
|
||||
entities.Should().Contain(entity => entity.Type == PiiEntityType.Phone && entity.Value == "9876543210");
|
||||
entities.Count(entity => entity.Type == PiiEntityType.Phone).Should().Be(1);
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Detect_InvalidInput_ThrowsArgumentException(string? text)
|
||||
{
|
||||
var composite = new CompositePiiDetector([new RegexPiiDetector()]);
|
||||
var action = () => composite.Detect(text!);
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
private sealed class FixedDetector(params PiiEntity[] entities) : IPiiDetector
|
||||
{
|
||||
public IReadOnlyList<PiiEntity> Detect(string text) => entities;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Detection;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class DomainRulePiiDetectorTests
|
||||
{
|
||||
private readonly DomainRulePiiDetector _detector = new();
|
||||
|
||||
[Test]
|
||||
public void Detect_PositiveLoanNumber_ReturnsValueOnly()
|
||||
{
|
||||
var entities = _detector.Detect("Loan LN-456789 active.");
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.LoanNumber &&
|
||||
entity.Value == "LN-456789" &&
|
||||
entity.Source == PiiDetectionSource.Domain);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_PositiveCustomerId_ReturnsEntity()
|
||||
{
|
||||
var entities = _detector.Detect("CustomerId CID-1234 found.");
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.CustomerId &&
|
||||
entity.Value == "CID-1234");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_PositiveAccountNumber_ReturnsEntity()
|
||||
{
|
||||
var entities = _detector.Detect("Account ACC-123456 open.");
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.AccountNumber &&
|
||||
entity.Value == "ACC-123456");
|
||||
}
|
||||
|
||||
[TestCase("LN-12345")]
|
||||
[TestCase("XLN-456789")]
|
||||
[TestCase("CID-123")]
|
||||
[TestCase("ACC-12345")]
|
||||
public void Detect_InvalidDomainIds_ReturnsEmpty(string text)
|
||||
{
|
||||
_detector.Detect(text).Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_LoanNumberLabel_PreservesLabelInSurroundingText()
|
||||
{
|
||||
const string text = "LoanNumber LN-456789 end.";
|
||||
var entity = _detector.Detect(text).Single();
|
||||
|
||||
entity.Value.Should().Be("LN-456789");
|
||||
entity.StartIndex.Should().Be("LoanNumber ".Length);
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Detect_InvalidInput_ThrowsArgumentException(string? text)
|
||||
{
|
||||
var action = () => _detector.Detect(text!);
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
using PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Detection;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class OnnxNerPiiDetectorTests
|
||||
{
|
||||
[Test]
|
||||
public void Detect_ModelAvailable_ReturnsRunnerEntities()
|
||||
{
|
||||
var runner = new FakeOnnxNerModelRunner
|
||||
{
|
||||
IsModelAvailable = true,
|
||||
EntitiesToReturn =
|
||||
[
|
||||
new PiiEntity(PiiEntityType.Person, "Onnx Person", 0, 11, PiiDetectionSource.Ner)
|
||||
]
|
||||
};
|
||||
|
||||
var detector = new OnnxNerPiiDetector(runner);
|
||||
var entities = detector.Detect("Any text");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Value == "Onnx Person");
|
||||
runner.LastPredictedText.Should().Be("Any text");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_ModelUnavailable_ReturnsEmpty()
|
||||
{
|
||||
var detector = CreateDetector(modelAvailable: false);
|
||||
var entities = detector.Detect("Customer Ravi Kumar with email test@x.com.");
|
||||
|
||||
entities.Should().BeEmpty();
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Detect_InvalidInput_ThrowsArgumentException(string? text)
|
||||
{
|
||||
var detector = CreateDetector(modelAvailable: false);
|
||||
var action = () => detector.Detect(text!);
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
private static OnnxNerPiiDetector CreateDetector(bool modelAvailable)
|
||||
{
|
||||
var runner = new FakeOnnxNerModelRunner { IsModelAvailable = modelAvailable };
|
||||
return new OnnxNerPiiDetector(runner);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Detection;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class RegexPiiDetectorTests
|
||||
{
|
||||
private readonly RegexPiiDetector _detector = new();
|
||||
|
||||
[Test]
|
||||
public void Detect_PositiveEmail_ReturnsEntity()
|
||||
{
|
||||
const string text = "Contact ravi.kumar@gmail.com now.";
|
||||
var entities = _detector.Detect(text);
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.Email &&
|
||||
entity.Value == "ravi.kumar@gmail.com" &&
|
||||
entity.Source == PiiDetectionSource.Regex);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_PositivePhone_ReturnsTenDigitEntity()
|
||||
{
|
||||
var entities = _detector.Detect("Call 9876543210 today.");
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.Phone &&
|
||||
entity.Value == "9876543210" &&
|
||||
entity.Source == PiiDetectionSource.Regex);
|
||||
}
|
||||
|
||||
[TestCase("Aadhaar 1234 5678 9012 linked.", "1234 5678 9012")]
|
||||
[TestCase("Aadhaar 123456789012 linked.", "123456789012")]
|
||||
public void Detect_PositiveAadhaar_ReturnsEntity(string text, string expectedValue)
|
||||
{
|
||||
var entities = _detector.Detect(text);
|
||||
|
||||
entities.Should().ContainSingle(entity =>
|
||||
entity.Type == PiiEntityType.Aadhaar &&
|
||||
entity.Value == expectedValue);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_PositivePan_ReturnsUppercaseEntity()
|
||||
{
|
||||
var entities = _detector.Detect("PAN ABCDE1234F verified.");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Type == PiiEntityType.Pan);
|
||||
entities[0].Value.Should().MatchRegex("^[A-Z]{5}\\d{4}[A-Z]$");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_PositiveCreditCard_ReturnsEntity()
|
||||
{
|
||||
var entities = _detector.Detect("Card 4111-1111-1111-1111 used.");
|
||||
|
||||
entities.Should().ContainSingle(entity => entity.Type == PiiEntityType.CreditCard);
|
||||
}
|
||||
|
||||
[TestCase("not-an-email")]
|
||||
[TestCase("@missing.com")]
|
||||
[TestCase("pan abcde1234f")]
|
||||
[TestCase("Number 987654321")]
|
||||
public void Detect_NegativePatterns_ReturnsNoMatch(string text)
|
||||
{
|
||||
_detector.Detect(text).Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Detect_EmailSpan_HasCorrectIndices()
|
||||
{
|
||||
const string text = "Email ravi@test.com end.";
|
||||
var entities = _detector.Detect(text);
|
||||
|
||||
var email = entities.Single(entity => entity.Type == PiiEntityType.Email);
|
||||
email.StartIndex.Should().Be(6);
|
||||
email.Length.Should().Be("ravi@test.com".Length);
|
||||
text[email.StartIndex..email.EndIndex].Should().Be("ravi@test.com");
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Detect_InvalidInput_ThrowsArgumentException(string? text)
|
||||
{
|
||||
var action = () => _detector.Detect(text!);
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Models;
|
||||
using PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Integration;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class GoldenPromptTests
|
||||
{
|
||||
[TestCaseSource(typeof(PromptScenarioCatalog), nameof(PromptScenarioCatalog.AllScenarios))]
|
||||
public void Sanitize_PromptScenario_ProducesExpectedOutput(PromptScenario scenario)
|
||||
{
|
||||
var sanitizer = ProductionPipelineFactory.CreateForScenario(scenario);
|
||||
var result = sanitizer.Sanitize(new SanitizationRequest(scenario.Prompt));
|
||||
|
||||
result.SanitizedPrompt.Should().Be(scenario.ExpectedSanitized, because: scenario.Name);
|
||||
result.DetectedEntities.Select(entity => entity.Type)
|
||||
.Should().BeEquivalentTo(scenario.ExpectedTypes, because: scenario.Name);
|
||||
|
||||
foreach (var forbidden in scenario.MustNotContainInSanitized)
|
||||
{
|
||||
result.SanitizedPrompt.Should().NotContain(forbidden, because: scenario.Name);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Models;
|
||||
using PiiRedaction.Core.Tests.TestSupport;
|
||||
using PiiRedaction.Infrastructure.Llm;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Integration;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class LlmBoundaryTests
|
||||
{
|
||||
[Test]
|
||||
public async Task SendPromptAsync_DoesNotTransmitOriginalPii()
|
||||
{
|
||||
var capturingClient = new CapturingChatClient();
|
||||
ILlmPromptService service = new MockLlmPromptService(capturingClient);
|
||||
|
||||
const string sanitized = "Customer <PERSON_1> with email <EMAIL_1>.";
|
||||
await service.SendPromptAsync(sanitized);
|
||||
|
||||
capturingClient.LastUserMessage.Should().Be(sanitized);
|
||||
capturingClient.LastUserMessage.Should().NotContain("ravi.kumar@gmail.com");
|
||||
capturingClient.LastUserMessage.Should().NotContain("Ravi Kumar");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task SendPromptAsync_AfterSanitization_OnlyPlaceholdersReachLlm()
|
||||
{
|
||||
var capturingClient = new CapturingChatClient();
|
||||
var sanitizer = ProductionPipelineFactory.Create().Sanitizer;
|
||||
ILlmPromptService service = new MockLlmPromptService(capturingClient);
|
||||
|
||||
const string prompt = "Email ravi.kumar@gmail.com please.";
|
||||
var result = sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
result.SanitizedPrompt.Should().Contain("<EMAIL_1>");
|
||||
await service.SendPromptAsync(result.SanitizedPrompt);
|
||||
|
||||
capturingClient.LastUserMessage.Should().Contain("<EMAIL_1>");
|
||||
capturingClient.LastUserMessage.Should().NotContain("ravi.kumar@gmail.com");
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void SendPromptAsync_InvalidSanitizedPrompt_ThrowsArgumentException(string? prompt)
|
||||
{
|
||||
ILlmPromptService service = new MockLlmPromptService(new MockChatClient());
|
||||
var action = async () => await service.SendPromptAsync(prompt!);
|
||||
action.Should().ThrowAsync<ArgumentException>();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
using FluentAssertions;
|
||||
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
using PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
using PiiRedaction.Tests.Shared;
|
||||
|
||||
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Integration;
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
|
||||
/// End-to-end pipeline proof using the real ONNX NER model (no fakes).
|
||||
|
||||
/// </summary>
|
||||
|
||||
[TestFixture]
|
||||
|
||||
[Category("RealModel")]
|
||||
|
||||
public sealed class RealNerPipelineTests : RealNerModelFixture
|
||||
|
||||
{
|
||||
|
||||
private IPromptSanitizer _sanitizer = null!;
|
||||
|
||||
|
||||
|
||||
[OneTimeSetUp]
|
||||
|
||||
public void OneTimeSetUpPipeline()
|
||||
|
||||
{
|
||||
|
||||
_sanitizer = ProductionPipelineFactory.CreateWithRealModel(Runner);
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Test]
|
||||
|
||||
public void Sanitize_FullFinancialWithCustomer_RedactsAllPiiTypes()
|
||||
|
||||
{
|
||||
|
||||
const string prompt =
|
||||
|
||||
"Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.";
|
||||
|
||||
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
|
||||
|
||||
result.SanitizedPrompt.Should().Be(
|
||||
|
||||
"Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.");
|
||||
|
||||
|
||||
|
||||
result.SanitizedPrompt.Should().NotContainAny("Ravi Kumar", "ravi.kumar@gmail.com", "9876543210", "LN-456789", "ABCDE1234F");
|
||||
|
||||
result.DetectedEntities.Should().Contain(entity =>
|
||||
|
||||
entity.Type == PiiEntityType.Person && entity.Source == PiiDetectionSource.Ner);
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Test]
|
||||
|
||||
public void Sanitize_TwoCustomersInOnePrompt_RedactsBothPeople()
|
||||
|
||||
{
|
||||
|
||||
const string prompt = "Customer Ravi Kumar and Customer Priya Nair disputed the same charge.";
|
||||
|
||||
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
|
||||
|
||||
result.SanitizedPrompt.Should().Contain("<PERSON_1>");
|
||||
|
||||
result.SanitizedPrompt.Should().Contain("<PERSON_2>");
|
||||
|
||||
result.SanitizedPrompt.Should().NotContainAny("Ravi Kumar", "Priya Nair");
|
||||
|
||||
result.DetectedEntities.Count(entity => entity.Type == PiiEntityType.Person).Should().BeGreaterThanOrEqualTo(2);
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Test]
|
||||
|
||||
public void Sanitize_NoPiiCleanTicket_PassesThroughUnchanged()
|
||||
|
||||
{
|
||||
|
||||
const string prompt = "What is the status of ticket TKT-99887 and when will the API maintenance end?";
|
||||
|
||||
|
||||
|
||||
var result = _sanitizer.Sanitize(new SanitizationRequest(prompt));
|
||||
|
||||
|
||||
|
||||
result.SanitizedPrompt.Should().Be(prompt);
|
||||
|
||||
result.DetectedEntities.Should().BeEmpty();
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
35
tests/PiiRedaction.Core.Tests/PiiRedaction.Core.Tests.csproj
Normal file
35
tests/PiiRedaction.Core.Tests/PiiRedaction.Core.Tests.csproj
Normal file
@@ -0,0 +1,35 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" Version="6.0.4" />
|
||||
<PackageReference Include="FluentAssertions" Version="8.10.0" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" Version="10.7.0" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.0" />
|
||||
<PackageReference Include="NUnit" Version="4.3.2" />
|
||||
<PackageReference Include="NUnit.Analyzers" Version="4.7.0" />
|
||||
<PackageReference Include="NUnit3TestAdapter" Version="5.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="NUnit.Framework" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Compile Include="..\TestSupport.Shared\RealNerModelPaths.cs" Link="TestSupport.Shared\RealNerModelPaths.cs" />
|
||||
<Compile Include="..\TestSupport.Shared\RealNerModelFixture.cs" Link="TestSupport.Shared\RealNerModelFixture.cs" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\src\PiiRedaction.Core\PiiRedaction.Core.csproj" />
|
||||
<ProjectReference Include="..\..\src\PiiRedaction.Infrastructure\PiiRedaction.Infrastructure.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,134 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Models;
|
||||
using PiiRedaction.Core.Redaction;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Redaction;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class PlaceholderPiiRedactorTests
|
||||
{
|
||||
private readonly PlaceholderPiiRedactor _redactor = new();
|
||||
|
||||
[Test]
|
||||
public void Redact_NoEntities_ReturnsOriginalTextAndEmptyMap()
|
||||
{
|
||||
const string text = "No PII here.";
|
||||
var result = _redactor.Redact(text, []);
|
||||
|
||||
result.SanitizedText.Should().Be(text);
|
||||
result.PlaceholderMap.Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_SingleEntity_ReplacesWithTypedPlaceholder()
|
||||
{
|
||||
const string text = "Email ravi@test.com end.";
|
||||
var entities = new[] { Entity(PiiEntityType.Email, "ravi@test.com", 6) };
|
||||
|
||||
var result = _redactor.Redact(text, entities);
|
||||
|
||||
result.SanitizedText.Should().Be("Email <EMAIL_1> end.");
|
||||
result.PlaceholderMap["<EMAIL_1>"].Should().Be("ravi@test.com");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_DuplicateSameTypeAndValue_ReusesPlaceholder()
|
||||
{
|
||||
const string text = "a@b.co and a@b.co";
|
||||
var entities = new[]
|
||||
{
|
||||
Entity(PiiEntityType.Email, "a@b.co", 0),
|
||||
Entity(PiiEntityType.Email, "a@b.co", 11)
|
||||
};
|
||||
|
||||
var result = _redactor.Redact(text, entities);
|
||||
|
||||
result.SanitizedText.Should().Be("<EMAIL_1> and <EMAIL_1>");
|
||||
result.PlaceholderMap.Should().HaveCount(1);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_MultipleTypes_AssignsIndependentCounters()
|
||||
{
|
||||
const string text = "9876543210 ravi@test.com";
|
||||
var entities = new[]
|
||||
{
|
||||
Entity(PiiEntityType.Phone, "9876543210", 0),
|
||||
Entity(PiiEntityType.Email, "ravi@test.com", 11)
|
||||
};
|
||||
|
||||
var result = _redactor.Redact(text, entities);
|
||||
|
||||
result.SanitizedText.Should().Be("<PHONE_1> <EMAIL_1>");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_MultipleSameTypeDifferentValues_IncrementsCounter()
|
||||
{
|
||||
const string text = "Customer Ravi Kumar and Customer Priya Nair";
|
||||
var entities = new[]
|
||||
{
|
||||
Entity(PiiEntityType.Person, "Ravi Kumar", 9),
|
||||
Entity(PiiEntityType.Person, "Priya Nair", 33)
|
||||
};
|
||||
|
||||
var result = _redactor.Redact(text, entities);
|
||||
|
||||
result.SanitizedText.Should().Be("Customer <PERSON_2> and Customer <PERSON_1>");
|
||||
result.PlaceholderMap.Should().HaveCount(2);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_EntityTypes_UseTypedPlaceholderPrefixes()
|
||||
{
|
||||
(PiiEntityType Type, string Prefix)[] cases =
|
||||
[
|
||||
(PiiEntityType.Person, "PERSON"),
|
||||
(PiiEntityType.Email, "EMAIL"),
|
||||
(PiiEntityType.LoanNumber, "LOAN_NUMBER")
|
||||
];
|
||||
|
||||
foreach (var (type, prefix) in cases)
|
||||
{
|
||||
const string value = "VALUE";
|
||||
var text = $"start {value} end";
|
||||
var result = _redactor.Redact(text, [Entity(type, value, 6)]);
|
||||
result.SanitizedText.Should().Contain($"<{prefix}_1>", because: type.ToString());
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_RightToLeftReplacement_PreservesCorrectOutput()
|
||||
{
|
||||
const string text = "AA BB CC";
|
||||
var entities = new[]
|
||||
{
|
||||
Entity(PiiEntityType.Phone, "AA", 0),
|
||||
Entity(PiiEntityType.Email, "BB", 3),
|
||||
Entity(PiiEntityType.Pan, "CC", 6)
|
||||
};
|
||||
|
||||
var result = _redactor.Redact(text, entities);
|
||||
|
||||
result.SanitizedText.Should().Be("<PHONE_1> <EMAIL_1> <PAN_1>");
|
||||
}
|
||||
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Redact_InvalidText_ThrowsArgumentException(string? text)
|
||||
{
|
||||
var action = () => _redactor.Redact(text!, []);
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Redact_NullEntities_ThrowsArgumentNullException()
|
||||
{
|
||||
var action = () => _redactor.Redact("text", null!);
|
||||
action.Should().Throw<ArgumentNullException>();
|
||||
}
|
||||
|
||||
private static PiiEntity Entity(PiiEntityType type, string value, int start) =>
|
||||
new(type, value, start, value.Length, PiiDetectionSource.Regex);
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
using FluentAssertions;
|
||||
using PiiRedaction.Core.Sanitization;
|
||||
using PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.Sanitization;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class PromptSanitizerTests
|
||||
{
|
||||
[Test]
|
||||
public void Sanitize_NullRequest_ThrowsArgumentNullException()
|
||||
{
|
||||
var sanitizer = ProductionPipelineFactory.Create().Sanitizer;
|
||||
var action = () => sanitizer.Sanitize(null!);
|
||||
action.Should().Throw<ArgumentNullException>();
|
||||
}
|
||||
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Sanitize_WhitespacePrompt_ThrowsArgumentException(string prompt)
|
||||
{
|
||||
var sanitizer = ProductionPipelineFactory.Create().Sanitizer;
|
||||
var action = () => sanitizer.Sanitize(new(prompt));
|
||||
action.Should().Throw<ArgumentException>();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
public sealed class CapturingChatClient : IChatClient
|
||||
{
|
||||
public string? LastUserMessage { get; private set; }
|
||||
|
||||
public Task<ChatResponse> GetResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
LastUserMessage = messages.LastOrDefault(message => message.Role == ChatRole.User)?.Text;
|
||||
|
||||
var response = new ChatResponse(new ChatMessage(
|
||||
ChatRole.Assistant,
|
||||
$"Captured {LastUserMessage?.Length ?? 0} chars."));
|
||||
|
||||
return Task.FromResult(response);
|
||||
}
|
||||
|
||||
public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||
{
|
||||
var response = await GetResponseAsync(messages, options, cancellationToken).ConfigureAwait(false);
|
||||
yield return new ChatResponseUpdate(ChatRole.Assistant, response.Messages.Last().Text);
|
||||
}
|
||||
|
||||
public object? GetService(Type serviceType, object? serviceKey = null) => null;
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
public sealed class FakeOnnxNerModelRunner : IOnnxNerModelRunner
|
||||
{
|
||||
public bool IsModelAvailable { get; set; }
|
||||
|
||||
public IReadOnlyList<PiiEntity> EntitiesToReturn { get; set; } = [];
|
||||
|
||||
public string? LastPredictedText { get; private set; }
|
||||
|
||||
public IReadOnlyList<PiiEntity> PredictEntities(string text)
|
||||
{
|
||||
LastPredictedText = text;
|
||||
return EntitiesToReturn;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
public static class NerEntityBuilder
|
||||
{
|
||||
public static IReadOnlyList<PiiEntity> BuildFromScenario(PromptScenario scenario)
|
||||
{
|
||||
var entities = new List<PiiEntity>();
|
||||
|
||||
foreach (var (type, value) in scenario.ExpectedTypes.Zip(scenario.MustNotContainInSanitized))
|
||||
{
|
||||
if (type != PiiEntityType.Person)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var searchStart = 0;
|
||||
while ((searchStart = scenario.Prompt.IndexOf(value, searchStart, StringComparison.Ordinal)) >= 0)
|
||||
{
|
||||
entities.Add(new PiiEntity(
|
||||
PiiEntityType.Person,
|
||||
value,
|
||||
searchStart,
|
||||
value.Length,
|
||||
PiiDetectionSource.Ner));
|
||||
searchStart += value.Length;
|
||||
}
|
||||
}
|
||||
|
||||
return entities;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
using PiiRedaction.Core.Abstractions;
|
||||
using PiiRedaction.Core.Detection;
|
||||
using PiiRedaction.Core.Redaction;
|
||||
using PiiRedaction.Core.Sanitization;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
public static class ProductionPipelineFactory
|
||||
{
|
||||
public static IPromptSanitizer CreateWithRealModel(IOnnxNerModelRunner runner) =>
|
||||
new PromptSanitizer(CreateCompositeDetector(runner), new PlaceholderPiiRedactor());
|
||||
|
||||
public static IPiiDetector CreateCompositeDetector(IOnnxNerModelRunner runner) =>
|
||||
new CompositePiiDetector(
|
||||
[
|
||||
new DomainRulePiiDetector(),
|
||||
new RegexPiiDetector(),
|
||||
new OnnxNerPiiDetector(runner)
|
||||
]);
|
||||
|
||||
public static (IPromptSanitizer Sanitizer, FakeOnnxNerModelRunner NerRunner) Create(
|
||||
bool modelAvailable = false,
|
||||
IReadOnlyList<Models.PiiEntity>? nerEntities = null)
|
||||
{
|
||||
var nerRunner = CreateNerRunner(modelAvailable, nerEntities);
|
||||
var sanitizer = new PromptSanitizer(CreateCompositeDetector(nerRunner), new PlaceholderPiiRedactor());
|
||||
return (sanitizer, nerRunner);
|
||||
}
|
||||
|
||||
public static IPromptSanitizer CreateForScenario(PromptScenario scenario)
|
||||
{
|
||||
var nerEntities = NerEntityBuilder.BuildFromScenario(scenario);
|
||||
return Create(modelAvailable: nerEntities.Count > 0, nerEntities: nerEntities).Sanitizer;
|
||||
}
|
||||
|
||||
public static IPiiDetector CreateCompositeDetector(
|
||||
bool modelAvailable = false,
|
||||
IReadOnlyList<Models.PiiEntity>? nerEntities = null) =>
|
||||
CreateCompositeDetector(CreateNerRunner(modelAvailable, nerEntities));
|
||||
|
||||
private static FakeOnnxNerModelRunner CreateNerRunner(
|
||||
bool modelAvailable,
|
||||
IReadOnlyList<Models.PiiEntity>? nerEntities) =>
|
||||
new()
|
||||
{
|
||||
IsModelAvailable = modelAvailable,
|
||||
EntitiesToReturn = nerEntities ?? []
|
||||
};
|
||||
}
|
||||
10
tests/PiiRedaction.Core.Tests/TestSupport/PromptScenario.cs
Normal file
10
tests/PiiRedaction.Core.Tests/TestSupport/PromptScenario.cs
Normal file
@@ -0,0 +1,10 @@
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
public sealed record PromptScenario(
|
||||
string Name,
|
||||
string Prompt,
|
||||
string ExpectedSanitized,
|
||||
PiiEntityType[] ExpectedTypes,
|
||||
string[] MustNotContainInSanitized);
|
||||
@@ -0,0 +1,68 @@
|
||||
using NUnit.Framework;
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
namespace PiiRedaction.Core.Tests.TestSupport;
|
||||
|
||||
/// <summary>
|
||||
/// Focused end-to-end scenarios that exercise the full sanitizer pipeline beyond what unit tests cover in isolation.
|
||||
/// </summary>
|
||||
public static class PromptScenarioCatalog
|
||||
{
|
||||
public static IEnumerable<TestCaseData> AllScenarios()
|
||||
{
|
||||
foreach (var scenario in BuildScenarios())
|
||||
{
|
||||
yield return new TestCaseData(scenario).SetName(scenario.Name);
|
||||
}
|
||||
}
|
||||
|
||||
private static IEnumerable<PromptScenario> BuildScenarios()
|
||||
{
|
||||
yield return CanonicalScenario();
|
||||
yield return MultiRegexScenario();
|
||||
yield return DuplicatePeopleScenario();
|
||||
yield return OverlapScenario();
|
||||
yield return NoPiiScenario();
|
||||
}
|
||||
|
||||
private static PromptScenario CanonicalScenario()
|
||||
{
|
||||
const string prompt =
|
||||
"Customer Ravi Kumar with email ravi.kumar@gmail.com and phone 9876543210 has LoanNumber LN-456789 and PAN ABCDE1234F. Please summarize this customer issue.";
|
||||
|
||||
return new PromptScenario(
|
||||
"Canonical_DemoPrompt",
|
||||
prompt,
|
||||
"Customer <PERSON_1> with email <EMAIL_1> and phone <PHONE_1> has LoanNumber <LOAN_NUMBER_1> and PAN <PAN_1>. Please summarize this customer issue.",
|
||||
[PiiEntityType.Person, PiiEntityType.Email, PiiEntityType.Phone, PiiEntityType.LoanNumber, PiiEntityType.Pan],
|
||||
["Ravi Kumar", "ravi.kumar@gmail.com", "9876543210", "LN-456789", "ABCDE1234F"]);
|
||||
}
|
||||
|
||||
private static PromptScenario MultiRegexScenario() => new(
|
||||
"Full_AllRegexTypes",
|
||||
"Email a@b.co phone 9001234567 PAN ABCDE1234F aadhaar 1234 5678 9012 card 4111-1111-1111-1111.",
|
||||
"Email <EMAIL_1> phone <PHONE_1> PAN <PAN_1> aadhaar <AADHAAR_1> card <CREDIT_CARD_1>.",
|
||||
[PiiEntityType.Email, PiiEntityType.Phone, PiiEntityType.Pan, PiiEntityType.Aadhaar, PiiEntityType.CreditCard],
|
||||
["a@b.co", "9001234567", "ABCDE1234F", "1234 5678 9012", "4111-1111-1111-1111"]);
|
||||
|
||||
private static PromptScenario DuplicatePeopleScenario() => new(
|
||||
"Duplicate_TwoPeople",
|
||||
"Customer Ravi Kumar and Customer Priya Nair.",
|
||||
"Customer <PERSON_2> and Customer <PERSON_1>.",
|
||||
[PiiEntityType.Person, PiiEntityType.Person],
|
||||
["Ravi Kumar", "Priya Nair"]);
|
||||
|
||||
private static PromptScenario OverlapScenario() => new(
|
||||
"Overlap_AadhaarAndPhone",
|
||||
"Aadhaar 987654321012 and phone 9876543210.",
|
||||
"Aadhaar <AADHAAR_1> and phone <PHONE_1>.",
|
||||
[PiiEntityType.Aadhaar, PiiEntityType.Phone],
|
||||
["9876543210"]);
|
||||
|
||||
private static PromptScenario NoPiiScenario() => new(
|
||||
"Negative_NoPii",
|
||||
"Please summarize the general policy on refunds.",
|
||||
"Please summarize the general policy on refunds.",
|
||||
[],
|
||||
[]);
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
using System.Runtime.CompilerServices;
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.AI;
|
||||
using PiiRedaction.Infrastructure.Llm;
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Tests.Llm;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class MockLlmPromptServiceTests
|
||||
{
|
||||
[Test]
|
||||
public async Task SendPromptAsync_ReturnsAssistantText()
|
||||
{
|
||||
var service = new MockLlmPromptService(new MockChatClient());
|
||||
|
||||
const string prompt = "Sanitized <EMAIL_1> content.";
|
||||
var response = await service.SendPromptAsync(prompt);
|
||||
|
||||
response.Should().Contain("Mock LLM Response");
|
||||
response.Should().Contain($"{prompt.Length} chars");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task SendPromptAsync_UsesUserRoleForOutboundMessage()
|
||||
{
|
||||
var capturingClient = new CapturingChatClient();
|
||||
var service = new MockLlmPromptService(capturingClient);
|
||||
|
||||
await service.SendPromptAsync("Only placeholders <PHONE_1>.");
|
||||
|
||||
capturingClient.LastUserMessage.Should().Be("Only placeholders <PHONE_1>.");
|
||||
}
|
||||
|
||||
private sealed class CapturingChatClient : IChatClient
|
||||
{
|
||||
public string? LastUserMessage { get; private set; }
|
||||
|
||||
public Task<ChatResponse> GetResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
LastUserMessage = messages.LastOrDefault(message => message.Role == ChatRole.User)?.Text;
|
||||
return Task.FromResult(new ChatResponse(new ChatMessage(ChatRole.Assistant, "ok")));
|
||||
}
|
||||
|
||||
public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||
{
|
||||
await GetResponseAsync(messages, options, cancellationToken);
|
||||
yield break;
|
||||
}
|
||||
|
||||
public object? GetService(Type serviceType, object? serviceKey = null) => null;
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Core.Models;
|
||||
using PiiRedaction.Infrastructure.Onnx;
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Tests.Onnx;
|
||||
|
||||
[TestFixture]
|
||||
public sealed class OnnxNerModelRunnerTests
|
||||
{
|
||||
[Test]
|
||||
public void Constructor_MissingModel_IsModelUnavailable()
|
||||
{
|
||||
var path = Path.Combine(Path.GetTempPath(), $"missing-ner-{Guid.NewGuid():N}.onnx");
|
||||
using var runner = CreateRunner(path);
|
||||
|
||||
runner.IsModelAvailable.Should().BeFalse();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void PredictEntities_WhenModelUnavailable_ReturnsEmpty()
|
||||
{
|
||||
var path = Path.Combine(Path.GetTempPath(), $"missing-ner-{Guid.NewGuid():N}.onnx");
|
||||
using var runner = CreateRunner(path);
|
||||
|
||||
runner.PredictEntities("Customer Ravi Kumar").Should().BeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Constructor_InvalidModelFile_IsModelUnavailable()
|
||||
{
|
||||
var path = Path.Combine(Path.GetTempPath(), $"invalid-ner-{Guid.NewGuid():N}.onnx");
|
||||
File.WriteAllText(path, "not-a-valid-onnx-model");
|
||||
|
||||
try
|
||||
{
|
||||
using var runner = CreateRunner(path);
|
||||
runner.IsModelAvailable.Should().BeFalse();
|
||||
}
|
||||
finally
|
||||
{
|
||||
File.Delete(path);
|
||||
}
|
||||
}
|
||||
|
||||
private static OnnxNerModelRunner CreateRunner(string modelPath)
|
||||
{
|
||||
var options = Options.Create(new PiiRedactionOptions { OnnxModelPath = modelPath });
|
||||
return new OnnxNerModelRunner(options, NullLogger<OnnxNerModelRunner>.Instance);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
using FluentAssertions;
|
||||
|
||||
using PiiRedaction.Core.Models;
|
||||
|
||||
using PiiRedaction.Tests.Shared;
|
||||
|
||||
|
||||
|
||||
namespace PiiRedaction.Infrastructure.Tests.Onnx;
|
||||
|
||||
|
||||
|
||||
[TestFixture]
|
||||
|
||||
[Category("RealModel")]
|
||||
|
||||
public sealed class RealNerModelRunnerTests : RealNerModelFixture
|
||||
|
||||
{
|
||||
|
||||
[TestCase("Customer Ravi Kumar called about billing.", "Ravi", "Ravi Kumar")]
|
||||
|
||||
[TestCase("Mr. John Smith called about a duplicate debit.", "John", "John Smith")]
|
||||
|
||||
public void PredictEntities_DetectsPersonWithCorrectSpan(string prompt, string expectedNamePart, string expectedValue)
|
||||
|
||||
{
|
||||
|
||||
var entities = Runner.PredictEntities(prompt);
|
||||
|
||||
|
||||
|
||||
entities.Should().Contain(entity =>
|
||||
|
||||
entity.Type == PiiEntityType.Person &&
|
||||
|
||||
entity.Source == PiiDetectionSource.Ner &&
|
||||
|
||||
entity.Value.Contains(expectedNamePart, StringComparison.Ordinal) &&
|
||||
|
||||
prompt.AsSpan(entity.StartIndex, entity.Length).ToString() == entity.Value);
|
||||
|
||||
|
||||
|
||||
entities.Should().Contain(entity => entity.Value == expectedValue);
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<LangVersion>latest</LangVersion>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" Version="6.0.4" />
|
||||
<PackageReference Include="FluentAssertions" Version="8.10.0" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.0" />
|
||||
<PackageReference Include="NUnit" Version="4.3.2" />
|
||||
<PackageReference Include="NUnit.Analyzers" Version="4.7.0" />
|
||||
<PackageReference Include="NUnit3TestAdapter" Version="5.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="NUnit.Framework" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Compile Include="..\TestSupport.Shared\RealNerModelPaths.cs" Link="TestSupport.Shared\RealNerModelPaths.cs" />
|
||||
<Compile Include="..\TestSupport.Shared\RealNerModelFixture.cs" Link="TestSupport.Shared\RealNerModelFixture.cs" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\src\PiiRedaction.Infrastructure\PiiRedaction.Infrastructure.csproj" />
|
||||
<ProjectReference Include="..\..\src\PiiRedaction.Core\PiiRedaction.Core.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
42
tests/TestSupport.Shared/RealNerModelFixture.cs
Normal file
42
tests/TestSupport.Shared/RealNerModelFixture.cs
Normal file
@@ -0,0 +1,42 @@
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using PiiRedaction.Core.Configuration;
|
||||
using PiiRedaction.Infrastructure.Onnx;
|
||||
|
||||
namespace PiiRedaction.Tests.Shared;
|
||||
|
||||
/// <summary>
|
||||
/// Reuses a single <see cref="OnnxNerModelRunner"/> per fixture for performance.
|
||||
/// Skips all tests in the class when the ONNX model is missing or cannot be loaded.
|
||||
/// </summary>
|
||||
public abstract class RealNerModelFixture
|
||||
{
|
||||
protected OnnxNerModelRunner Runner { get; private set; } = null!;
|
||||
|
||||
protected string ModelPath { get; private set; } = null!;
|
||||
|
||||
[OneTimeSetUp]
|
||||
public void OneTimeSetUpRealModel()
|
||||
{
|
||||
ModelPath = RealNerModelPaths.ResolveRepoModelPath();
|
||||
if (!File.Exists(ModelPath))
|
||||
{
|
||||
Assert.Ignore(RealNerModelPaths.ModelMissingMessage);
|
||||
}
|
||||
|
||||
var options = Options.Create(new PiiRedactionOptions { OnnxModelPath = ModelPath });
|
||||
Runner = new OnnxNerModelRunner(options, NullLogger<OnnxNerModelRunner>.Instance);
|
||||
|
||||
if (!Runner.IsModelAvailable)
|
||||
{
|
||||
Runner.Dispose();
|
||||
Assert.Ignore(RealNerModelPaths.ModelMissingMessage);
|
||||
}
|
||||
}
|
||||
|
||||
[OneTimeTearDown]
|
||||
public void OneTimeTearDownRealModel()
|
||||
{
|
||||
Runner?.Dispose();
|
||||
}
|
||||
}
|
||||
24
tests/TestSupport.Shared/RealNerModelPaths.cs
Normal file
24
tests/TestSupport.Shared/RealNerModelPaths.cs
Normal file
@@ -0,0 +1,24 @@
|
||||
namespace PiiRedaction.Tests.Shared;
|
||||
|
||||
public static class RealNerModelPaths
|
||||
{
|
||||
public const string ModelMissingMessage =
|
||||
"ONNX model not found. Run scripts/download-ner-model.ps1 from the repository root.";
|
||||
|
||||
public static string ResolveRepoModelPath()
|
||||
{
|
||||
var directory = new DirectoryInfo(AppContext.BaseDirectory);
|
||||
while (directory is not null)
|
||||
{
|
||||
var candidate = Path.Combine(directory.FullName, "models", "ner-model.onnx");
|
||||
if (File.Exists(candidate))
|
||||
{
|
||||
return candidate;
|
||||
}
|
||||
|
||||
directory = directory.Parent;
|
||||
}
|
||||
|
||||
return Path.Combine(Environment.CurrentDirectory, "models", "ner-model.onnx");
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user