LLMFirewall Core — Defense-in-Depth Security Framework for AI & LLMs
Sub-millisecond in-process firewall intercepting prompt injection, PII leaks, and rogue tool execution.
llmfirewall-coreThis project is authored and published strictly as an in-process Python library on the Python Package Index. It does not require a hosted dashboard or cloud service — executing natively within the consumer's application runtime.
Problem Statement
Traditional Web Application Firewalls (WAFs) operate on HTTP payloads and know nothing about prompt semantics, indirect document poisoning, or autonomous agent tool execution parameters. Meanwhile, model-based guardrails add 500-1500ms latency and high token costs.
Engineering Goals
- Provide in-process security with sub-millisecond overhead (<0.1ms).
- Zero external database or cloud API dependencies.
- Deterministic Policy-as-Code for enterprise compliance (OWASP Top 10 for LLMs, NIST AI RMF).
- Publish as a clean, pip-installable package on PyPI.
LLMFirewall Core — System ArchitectureVerified Pipeline
Select any pipeline stage below to inspect verified engineering details and data flow.
Stage: Prompt / Tool Request
Role: Untrusted InputIncoming natural language prompt, document payload, or proposed agent tool call.
Execution Pipeline & Mechanics
Token Pre-Processing & De-obfuscation
Decodes hidden Unicode homoglyphs, zero-width characters, and base64 payloads to defeat bypass attempts.
Adversarial Prompt Injection Detection
Matches against known injection vectors, roleplay jailbreaks, and system override constructs.
High-Entropy Secret Detection & PII Redaction
Calculates Shannon entropy to identify high-risk strings like AWS keys and tokens, replacing them with typed placeholders.
Agent Tool-Call Argument Sandboxing
Inspects proposed tool arguments to prevent SSRF against internal IP ranges and directory traversal.
Immutable Structured Audit Logging
Emits structured JSON audit records containing violation classifications and event fingerprints.
Engineering Challenges & Design Decisions
Decision: In-process deterministic rules over secondary LLM judge
Rationale: Secondary LLM judges introduce 400-1000ms latency and their own susceptibility to adversarial persuasion. In-process deterministic checks execute in 0.02ms with zero token cost.
Alternative Rejected: Llama-Guard / OpenAI Moderation API roundtrips.
Core Architectural Highlights
LLMFirewall operates as an ASGI middleware or Python function decorator. Payloads pass through a multi-stage validation chain before dispatch to the downstream LLM or agent runtime. If an anomaly or unauthorized action is detected, it fails closed with structured error objects.
Testing & Empirical Evaluation
608 automated test cases covering adversarial prompt injection datasets, credential leakage patterns, tool argument tampering, and concurrency stress tests.
Security & Reliability Defenses
Key Engineering Learnings
- •Adversaries frequently use zero-width Unicode characters to evade naive string matching; canonicalization must precede all heuristic scans.