Washington, DC · data & AI engineering

The LLM proposes. The pipeline disposes.

I build data platforms where every model output passes a deterministic gate before it touches a number anyone acts on. Twenty years of production data systems — from 3-billion-record-a-day fraud pipelines to governed LLM workflows — and a shelf of open-source projects that show exactly how.

Book a 20-minute call See the work ↓

3B+ CDRs/day in production
10+ open-source repos
ODCS · OpenLineage · FHIR R4B
Java · Python · Scala · Spark
Available for consulting & senior roles
TS/SCI cleared

A first engagement

Fixed scope, fixed fee, written deliverables. Most relationships start with the first one.

2–3 weeks

Governance readiness assessment

For teams putting models into data pipelines and wondering what could go wrong quietly.

An inventory of every point where model output reaches a decision, and whether a gate stands in front of it
Data-contract and lineage gaps against ODCS and OpenLineage, with the provenance you can and cannot currently prove
A prioritized remediation plan your own engineers can execute

Start with this →

4–8 weeks

Contracts and lineage, implemented

Data contracts that fail the build when the schema drifts, lineage emitted on every run, and provenance keys from the trusted layer back to raw.

Start with this →

1–2 weeks

LLM-in-the-pipeline design review

Where models belong in your pipeline and where they don't: verdict caching for deterministic replay, evaluation gates, human-review queues, and the audit trail that lets you defend a number.

Start with this →

For hiring managers: also open to senior and director-level roles in data platforms and applied AI.

LinkedIn Résumé (PDF)

Work in the open

Eight projects, ordered by how much they had to get right. One habit runs through all of them: every model output or number a person acts on has a checkable boundary.

The IEF chain drawn in space and time: the telecom records disclosure walkthrough in Phase 2, where a 2,000-row file flows through the established Wrapper, Transactional and Semantic elements, the quality gate, subscriber masking and location coarsening to the exchange output.

Governed data in four dimensions, Act 1 beat 6 of 9: a stream of 2,000 call detail records splits into three releases for an internal analyst, a carrier under §222 and an agency with a warrant, with a side table showing the same record's subscriber number and cell identifier masked, coarsened or in the clear by recipient.

IEF chain in 4D · hover or tap for one source released three ways

01 · Governed Data Platform

Governed Data Platform

data in motion
profiles
lineage on every run
release sets by authority

Profile-driven governance for data as it passes through: a data plane, not a catalog. A profile declares what a dataset must contain and which controls apply; the platform enforces it, applies lineage to every run without exception, and publishes what it enforced. One upload becomes a release set, with each recipient receiving the projection their authority entitles them to. A machine may propose a profile; only a named person approves one.

ODCS-shaped profiles · OpenLineage on every run
§222 · Carpenter · E.O. 13526 · §2.3
KLV · Remote ID · SAPIENT read by their bytes

Live demo →Runbooks →

Terminal session with the aiev command: run-demo writes a 96-event ledger, verify reports OK, ask shows record AE:AIEV-001-1042:3 promoted without a review, and after a one-byte tamper verify reports BROKEN at seq 16. — aiev · the record an inspector would ask about, then a tampered ledger caught

Terminal output of model-evidence's run-evidence command: a table of two views, span and document, with accuracy, precision, recall and false-positive-rate gates, their 95% intervals and targets, stability and reproducibility results, and the line OVERALL PASS. — model-evidence · gates, intervals and the seed-stability verdict

02 · AI Evidence Record + Model Evidence

AI Evidence Record + Model Evidence

clinical trials
tamper-evident ledger
classifier acceptance

A tamper-evident ledger that sits beside a clinical trial's data systems. Every AI-generated output is quarantined until a credentialed human signs it; seven rules scan the ledger for what an inspector would find, and a per-study AI Evidence Datasheet cites the clause each row satisfies. Edit one byte and verification names the broken sequence number. Model Evidence is the companion judge: give it a classifier's predictions and a policy and it returns Pass, Fail or Insufficient evidence, with exit codes CI can gate on.

SHA-256 hash chain · 7 exception rules
21 CFR Part 11 · ICH E6(R3) · ALCOA+
Pass / Fail / Insufficient · CI exit codes

Source · ai-evidence-record Source · model-evidence

The edshield browser demo: a synthetic reflection essay on the left, and on the right the same essay with the student's name, school, personal URL, a friend's name, email, phone number, student ID and street address replaced by labelled placeholders, done in 42 ms on the device with rules only. — edshield demo · rules only, nothing sent anywhere

Diagram of the edshield agent loop: two AI agents propose changes to the filter, every try runs through edshield and the measurement bench, a fixed judge gives the verdict, guardrails keep the loop honest, and a sealed test plus human review gate any release. — edshield-evidence · the bounded experiment loop

03 · edshield + edshield-evidence

edshield + edshield-evidence

student privacy
on-device NER
protected bench

A local-first student-privacy layer for AI in education. edshield detects and removes student PII from essays, tutoring transcripts and chat messages before the text reaches any language model, on a laptop CPU or inside a Chromebook browser, so no student text leaves the device. Rules handle the identifiers where regex plus validation wins; a PIILO-trained DeBERTa model is the authority on names. edshield-evidence is the protected bench: it measures what still survives in the final output and hands the rows to a fixed judge for the verdict.

Recall 1.000 · 0 of 165 missed on 680 held-out essays
FERPA · COPPA · research policies
Apache-2.0 · on PyPI · ONNX in the browser

Source · edshield Source · edshield-evidence Live demo →

04 · corpuscle

corpuscle

corpus datasheet
signed evidence
agents propose, core disposes

An agentic corpus datasheet. Agents plan the scan, investigate the findings and brief the authorizing official, on top of a deterministic, signed measurement core. The core hashes every document, builds a Merkle root, measures duplication, gates findings against a versioned policy and signs the result as an in-toto statement. The bundle travels with the corpus and verifies offline with a directory, the bundle and a public key. An agent's investigation is recorded and signed beside the numbers, and nothing it proposes takes effect until a named approver accepts it.

Ed25519 · DSSE over an in-toto statement
97 metric IDs across ten families
Verifies offline: directory + bundle + public key

Source →

The verdict app in flexible mode: Harris Teeter wins this week at $123.17, covering all 26 items; the other stores are ranked below with coverage flags and distance chips. — flexible · store brands OK

The same list in exact-brands mode: Whole Foods becomes the only full-coverage store at $169.27. — exact brands · the modes disagree

05 · Grocery Basket Optimizer

Grocery Basket Optimizer

data pipeline
LLM gates
DuckDB medallion

A weekly grocery-price pipeline for the DC metro that picks the one store that wins a household's whole list. Bronze/silver/gold on DuckDB; five chains with five different acquisition patterns; LLMs at three gated judgment points; a phone-first verdict app. Runs on a $300 mini PC with no Spark, no orchestrator, no vector store.

ODCS v3 contract, enforced by tests
OpenLineage event per run
MIT · CI on Windows + Linux

Source →

RxGuard MFP Refund Ledger for a synthetic pharmacy: $68,227.18 floated across 78 claims awaiting payment, $19,463.50 past the 14-day standard, a 14-day median time to refund, 8 systematically short-paid claims, and a chart of working capital tied up in unpaid refunds over time. — MFP Refund Ledger · synthetic pharmacy, every amount sourced

06 · RxGuard

RxGuard

pharmacy margin
FHIR R4B
gated explanations

Evidence-first pharmacy margin and exception analysis. RxGuard joins what a payer returned, what the pharmacy paid and what changed later, then flags loss-making fills, unresolved reversals, rejected claims and incomplete audit packets. Every number carries a source reference; an explanation, whether written by the offline baseline or a model, cannot mention a dollar that is absent from the computed fact pack. The MFP Refund Ledger applies the same engine to Medicare's negotiated-price refunds: expected refund, received, variance and aging per fill, with a dispute-ready evidence pack.

FHIR R4B EOB · CARIN Blue Button Pharmacy EOB 2.2.0
6 reproducible scenarios · 47-fill MFP ledger
CycloneDX SBOM · MIT

Source →Live demo · MFP ledger →Live demo · evidence one-pager →

retro · milestone retrospectives with a curated playbook

pip install learned-behavior   · 
Successfully installed learned-behavior-0.2.0
learned-behavior init-db
Initialized shared learning DB at ~/.local/share/learned-behavior/learning.db
learned-behavior learn --workspace "$PWD" --title "Use sh, not bash, on vapor images" --rule "laravelphp/vapor has no bash; use sh -c"
Saved approved lesson for ~
learned-behavior advice --workspace "$PWD" --output claude-json   # the UserPromptSubmit hook
{"hookSpecificOutput": {"hookEventName": "UserPromptSubmit", "additionalContext": "Relevant lessons from the shared learning DB:\n- Use sh, not bash, on vapor images: laravelphp/vapor has no bash; use sh -c"}}

learned-behavior · install, teach it one lesson, watch the hook surface it

07 · retro + learned-behavior

retro + learned-behavior

Claude Code skill
agent memory
no LLM in the loop

Two tools that make an AI coding agent's lessons durable. learned-behavior observes what the agent does through Claude Code hooks, mines recurring failures and self-corrections into rules, surfaces them before the next task, and promotes or decays them on behavioral signal alone: one local SQLite file, nothing leaves the machine. retro is the curation layer above it: a structured retrospective at project milestones, human-gated, that scopes each lesson to project, stack or global and holds the cross-project playbook to a fixed budget.

25-slot playbook · a lesson must displace one to enter
5 Claude Code hooks · candidate → approved → dormant
MIT · Python 3.10+ · zero runtime dependencies

Source · retro Source · learned-behavior PyPI →

Chat Buddy's parent setup wizard on a tablet, step 'What do they love?': favorite things filled in as dinosaurs, trains and Minecraft, a note that the child loves being asked which dinosaur is the fastest, and Back and Next buttons. — parent setup wizard · placeholder family

chat view · quick-reply chips, read-aloud, parent dashboard behind a PIN

08 · Chat Buddy

Chat Buddy

assistive
free for families
installable PWA

A texting companion that helps kids with communication challenges practice conversation around the things they love. Built as a parent, given away free: a parent setup wizard, a PIN-protected dashboard with full transcripts, and an installable PWA for an iPad. The model talks through a server-side proxy so the key never reaches the browser, and the bot keeps pointing the child back toward real people.

All data stays on the family's device
No accounts · no analytics · no telemetry
MIT · Next.js PWA

Source →

Production at scale · Mobius

A fraud pipeline live since 2015, at 3 billion call records a day

SIM-box call-bypass fraud detection for multiple mobile operators in West Africa, re-architected from an hourly batch into a streaming Hadoop/Spark pipeline, and still in production.

Problem

A Java SIM-box call-bypass fraud detector running as a two-server hourly batch. Detections lagged over an hour, customers' own analysts found misses after the fact, and multiple $300K/year accounts were about to churn to a competitor.

What I built

Re-architected it as a Hadoop/Spark pipeline scoring every 30-minute CDR window against 24 hours of per-MSISDN history with an inline rules engine, plus a redesigned Node.js UI where customers author their own detection rules. Per-customer deployments sized to volume; early namenode failures moved to an HA cluster.

Outcome

Detection latency 1h+ → under 30 min
Largest deployment 28 nodes at 3B+ voice CDRs/day, later 50B+ SMS/day (1B+ per 30-minute window)
Same pipeline reused for A2P/OTP SMS bypass fraud
Live since 2015 and still in production; the at-risk accounts stayed

Hadoop
Spark
HBase + Phoenix
MongoDB
Node.js
AWS (S3, Lambda, EMR)

Standards I work in

Open Data Contract Standard (ODCS), OpenLineage, FHIR R4B, DataHub, Great Expectations — and the unglamorous discipline underneath them: raw preservation before parsing, canonical units, provenance keys on every row, and refusals that explain themselves.

Start with a 20-minute conversation

Tell me what your pipeline does and where a model touches it. I'll tell you honestly whether a first engagement makes sense — and if it does, what it would cost.

Email hi@hemangnagar.dev github.com/hemangnagar @hnagar_dev linkedin.com/in/hemangnagar