PII Detection for AI Agents: Techniques That Work in Production
By ClawPine Team
Every customer-facing AI agent handles personally identifiable information. Even if your agent isn't designed to collect PII, users will type their names, email addresses, phone numbers, and sometimes social security numbers into the conversation. If your agent logs those conversations, forwards them to an LLM API, or stores them in memory, you're processing PII and you need to handle it properly.
Why Regex Isn't Enough
The most common approach to PII detection is regular expressions. Write a pattern for email addresses, another for phone numbers, another for SSNs, and run them against the text. This catches the obvious stuff but misses the subtle cases.
Names are the biggest gap. There's no regex that reliably detects names in free text. "Jordan called about the billing issue" contains a name. "We need to cross the Jordan River" doesn't (at least not a person's name). Context matters, and regex has no context.
Addresses are similarly tricky. "123 Main Street, Anytown, CA 94102" is easy. "Send it to my place on Oak near the coffee shop" is a location reference that regex will miss entirely. Medical record numbers, case IDs, and internal reference numbers follow organization-specific formats that generic patterns can't catch.
Machine Learning Approaches
Named Entity Recognition (NER) models solve the name problem by understanding context. Modern NER models correctly identify "Jordan" as a person name in the billing example and a geographic feature in the river example. Transformer-based NER models hit 95-98% accuracy on person names, which is dramatically better than regex.
For thorough PII detection, the best approach combines regex for structured data (emails, phones, SSNs) with NER for unstructured data (names, addresses, organization names). Run both in parallel and merge the results. Regex catches the formatted patterns. NER catches the contextual references.
Performance at Scale
PII detection must run on every agent input and output in real time. Latency matters. If your PII filter adds 500ms to every response, users will notice. The key is choosing the right model size. A full transformer NER model might take 50ms per text block. A distilled model achieves 90% of the accuracy in 5ms. For most production use cases, the distilled model is the right call.
ClawPine's PII detection pipeline runs a hybrid approach: fast regex patterns first, then a distilled NER model for contextual detection, with an optional full-model pass for high-sensitivity environments like healthcare. The default configuration adds under 10ms of latency per request.
Beyond Detection: What to Do With PII
Detecting PII is only half the problem. You also need a strategy for handling it. The three main approaches are redaction (replace PII with placeholder tokens), pseudonymization (replace with consistent fake data), and encryption (encrypt in place with reversible keys).
Redaction is simplest but lossy. Once you replace a name with [REDACTED], the context is gone. Pseudonymization preserves conversational flow ("John" becomes "Alex" consistently throughout the conversation) but requires a mapping table. Encryption preserves the original data but adds complexity.
ClawPine supports all three and lets you configure different strategies for different data types. Names might be pseudonymized for conversational quality while SSNs are always redacted. The configuration is per-compliance-profile, so your healthcare agents can have stricter rules than your general-purpose ones.