NeuroDiscovery AI · 2026 · B2B · Enterprise AI platform, real world evidence
NeuroAudit
first prototyped as NeuroValidate
An LLM platform that reads millions of clinical notes, answers a study question for every patient, quotes the sentence that proves it, and puts a human reviewer in charge of the final call.
My roleProduct owner from the first line of code. I built the prototype myself.
The challenge
Analysts read charts by hand to answer questions like "who stopped this therapy, and why". About 3 hours per 100 patients, not reproducible, not auditable.
What shipped
Four flows on one pipeline: cohort validation, score extraction, anonymisation and trial screening, with cost gates, approvals and model comparison.
What moved
2.5M clinical records processed, 23K+ reviewer decisions, 84.9% resolved with no reviewer time on the lead criterion of a flagship study
The answer to most real world evidence questions is not in a structured field. It sits in a doctor's note, where stopped, held, switched and missed a dose look alike and mean different things.
I built NeuroAudit so a model proposes and a person decides. Every answer carries the exact quote behind it, every run shows its cost before it starts, and nothing leaves the platform without a reviewer's signature.
Set up a study
From an empty board to a study with criteria, sources and validated keywords in about ten minutes.
- 01
One workspace for every study
Each card shows the client, the data sources, the patient count, the team and how far the study has got.
Projects home. Client, drug and lead names redacted. - 02
Name the study
Pick the study type: real world evidence, or a clinical trial cohort.
Create project, step 1. - 03
Define the criteria
Each criterion gets its own data source, scope, providers and keywords. The page always shows which index it will read.
Criteria and data sources. Partner site names redacted. - 04
Let an agent check the keywords
An agent flags keywords that are too broad. Broad keywords do not give wrong answers; they give huge, slow, expensive ones.
- 05
Pass the fetch gate
Pulling more than 500 patients needs approval, counted with a cheap count only query before any data moves.
The fetch gate.
Run it with cost control
Two prompts per criterion, a model per criterion, and the price on screen before anything runs.
- 01
Home base for the study
Runs by criterion, model comparisons and the criteria list on one page.
Project overview. - 02
Two stage prompts
One prompt extracts the evidence, a second checks it. Each stage picks its own model, and a model swap has to beat the baseline in a blind comparison.
Prompts and models. Drug names redacted. - 03
Scope and mode
Sample, specific patients or a full run. Fast for small checks, batch at about half the cost for big ones, with personal data removed first.
Run configuration. - 04
The cost gate
Every run shows its estimate. Up to $100 runs on its own; above that, or any full run, needs an approver.
Cost estimate and limits. - 05
Approvals
The approver sees the cost, the scope and why approval was needed, and starts the run with one click.
The approver's view.
Review: the human signs off
The deliverable is the reviewer's decision, not the model's output.
- 01
Evidence, note and verdict side by side
Green quotes support the answer, red quotes argue against it. The reviewer marks Eligible, Ineligible or Unclear.
Three pane review. Drug and protocol names redacted. - 02
A verdict needs a reason
Every decision carries a note, written to the audit trail.
Reviewer notes are required. - 03
Split the work
Assign groups of patients to reviewers and watch progress live.
Assignment and progress. - 04
Built for long review sessions
Reviewers spend hours in this screen, so every page works in light and dark.
The same review screen in light and dark. - 05
Prove a model change
An LLM judge grades two runs on the same records, blind. This is the evidence for changing a production model.
Run comparison on a demo study: wins, losses, ties and the judge's reason for each record.
Three more flows on the same pipeline
Score extraction, anonymisation and trial screening reuse the same engine.
- 01
Score extraction
Pull every clinical score (MoCA, MMSE, MIDAS, PHQ-9) from every note, each row with its own justification, then push the table to search.
Scores review. - 02
Anonymisation
Map each source column to a cleaned column, one prompt and one model per column. Clinical meaning survives; names, record numbers and birth dates do not.
- 03
Trial screening
Every patient against every criterion in one matrix, then a funnel that shows who survives each criterion and why. The funnel is what goes to the sponsor.
- 04
Agents, prompts and data
An agentic studio to test validation agents, a prompt library so teams reuse what works, and a data vault of imported cohorts.
How it was evaluated
- Grounding: a second model checks that every quote appears word for word in the source note. Anything that fails goes to a human.
- Agreement: reviewers adjudicate a sample, and their verdict is the ground truth.
- Model changes: an LLM judge compares two runs on the same records, blind, before a model goes to production.
- Product: scripted test scenarios on the live app before each release.
No platform accuracy, precision or recall. 84.9% is the share of records that needed no human check, not an accuracy score. Pairing model and reviewer decisions record by record is the next piece of work.
How it changed
- Feb 2026My prototype, as NeuroValidateProve the review loop before spending engineering time
- Mar to AprProduction: auth, projects, criteria to run, review, rolesEvery prototype flow but one shipped
- JunTrial screening; structured criteria pass automaticallyReviewer time was the expensive step
- Jun to JulLocked full runs and an anonymisation flowQuality checks found names in run outputs
- JulClear run states and reruns"Pending" looked the same for healthy and crashed jobs
- Aug to SepRun log, incremental runs, model comparison, auto prompter, approvals, access controlCost and governance asks from users
Technical choices
- Keyword retrieval over Elasticsearch with an agent checking keywords first
- Two pass extraction and grounding, model chosen per criterion from a cost ordered lineup of open and frontier models
- Go services on AWS, React, MongoDB, queue based workers, per run token and cost tracking
What I learned
- Grounding beats confidence scores. A word for word quote makes every answer checkable.
- Reliability, not intelligence, was the top complaint. A clear job status won more trust than a better model.
- Treat the prototype as the spec, and track what production actually ships.