MedTech AI Security Newsletter, May 30, 2026
There is a quiet truth about running agentic AI pipelines that does not get said often enough in vendor marketing: the house can clean you out before you finish your coffee. Not occasionally. Routinely. And when it happens, no one writes you a refund check.
We have spent the last year building and running automated product security workflows on frontier models. Threat modeling, vulnerability discovery, exploit verification, triage, regression testing of fixes. The good days are spectacular. The bad days look like a slot machine that has decided you are its retirement plan. We have started calling this the AI Casino, because the dynamics map almost perfectly: a paid B2B service where quality of service is neither guaranteed nor measurable, the operator can change the game between hands, and your chips are gone whether you won or lost.
This piece is a warning, not a pitch. If your medical device security program is about to invest in agentic pipelines, read carefully.
Frontier model APIs charge by tokens. Tokens are best thought of as the labor cost of a non-human worker. You pay per word in, per word out, and in an agentic pipeline with planners, executors, validators, and reviewers passing context around, the bill scales with the number of agents, the size of the context, and the number of loops.
The cost geometry is not linear. One analysis of agentic loops showed that a 4,000 token initial context, doubling each step as prior output gets appended, reaches 128,000 tokens by step 5, where per step cost has risen 32x. By step 15 the context overflows the model window and every call pays the maximum context rate. A documented LangChain incident from November 2025 saw two agents enter an unintended loop, an Analyzer and a Verifier requesting work from each other, with no per-agent budget ceiling. They ran for 11 days. The bill was $47,000.
That is the optimistic version. The more common, less dramatic version is the one most teams will recognize: a workflow that historically cost a few hundred dollars to run starts costing thousands, and the output quality is worse than last week.
This is where the casino metaphor becomes precise. The dealer can change the deck, and you only find out by watching your stack disappear.
Anthropic published a postmortem in April 2026 explaining six weeks of user complaints about Claude Code quality. Three changes had stacked on top of each other. On March 4, the default reasoning effort was lowered from high to medium to reduce latency. On March 26, a bug caused the model to discard its own reasoning history mid session, making it appear forgetful and burning usage faster. On April 16, a system prompt instruction was added that capped responses at 25 words between tool calls, which measurably hurt coding quality. All three were reverted by April 20.
OpenAI users have logged a similar pattern through 2025 and into 2026. Reddit, Hacker News, and developer forums fill up with reports of shorter outputs, more refusals, skeleton code where complete implementations used to land. Researchers studying behavioral drift across providers have started calling out the categories: systematic drift, stochastic drift, corrective drift, and the one that hurts most, regressive drift, where unintended degradation creeps in. One arxiv study of 2,250 model responses across providers concluded that behavioral drift is systematic and measurable across all major LLMs. Anthropic separately disclosed a September 2025 incident where a miscompiled sampling algorithm produced random anomalies that only affected specific batch sizes. A 235B parameter model running at temperature zero produced 80 unique completions across 1000 identical runs because of batch variance.
The pattern matters more than any single incident. Models get tuned. Inference stacks get re-hosted on cheaper precision tiers. Routing layers send your prompt to a smaller model. Safety filters tighten. None of these changes get a release note that says "your pipeline will perform 30 percent worse on Tuesday." You find out by measuring output quality, and by then the tokens are gone.
This is the part that has no analogue in any other paid service we can think of. When a model goes sideways, you have already paid for the tokens that produced the bad answer. There is no refund. There is no chargeback. There is no equivalent to the click fraud team at Google Ads that occasionally credits you when bots burned your budget. The closest analogue is paid advertising on a bad day, where a daily budget meant to last 12 hours evaporates in 60 minutes against junk clicks, with no explanation. Call that a lightweight casino. Frontier model APIs in an agentic loop can do the same thing exponentially, and the operator owes you nothing, because their position is that the tokens were consumed, the contract was fulfilled, and the R&D that produced the regression is internal.
We are not saying this is malicious. We are saying the asymmetry is real. You carry all of the quality risk. The provider carries none of it.
Medical device security teams are under pressure from FDA premarket cybersecurity expectations, EU NIS2 obligations, and a steady increase in CVE volume across the embedded and connected device space. The temptation to throw AI at the problem is enormous, and in many cases the math works.
Public benchmarks now support that. DARPA's AI Cyber Challenge final in August 2025 showed seven teams discovering 54 vulnerabilities across 54 million lines of code in four hours, at an average cost of roughly $152 per task. Anthropic's Mythos Preview, released April 2026, demonstrated autonomous discovery of bugs that had survived 16 to 27 years of human review, including a 27 year old OpenBSD TCP stack vulnerability. The reported cost figures for that work are striking: scanning the OpenBSD codebase under $20,000, a Linux kernel root exploit under $2,000, individual vulnerability identification under $50.
Those numbers are real. They are also the highlight reel. Behind them sit the runs that produced nothing, the runs that produced confident hallucinations, and the runs that consumed the budget without delivering. When pipelines work, they work at a productivity level no human team can match. When they do not, the burn rate exceeds human salary cost by a wide margin. A Nvidia executive recently noted that compute costs for his team now exceed employee salaries. Uber exhausted its 2026 AI budget by April. One enterprise client reportedly ran up a $500 million Claude bill in a single month after deploying it organization wide with no usage limits. Microsoft pulled most internal Claude Code licenses after the token bills exceeded the cost of the employees using them.
These are not edge cases. They are the new operating environment.
A few observations from a year of running these pipelines for security work. None of these are novel, but they are consistently underweighted in planning conversations.
Start with client side harnesses. Before any agentic pipeline goes near a paid API loop, the orchestration, prompts, context boundaries, and validators should be exercised locally with cheap or open models, and ideally with deterministic test fixtures. Most of the cost disasters we have seen would have been caught in a harness phase.
Use tight context windows on purpose. Bigger context is not better. It is more expensive and it lets agents drift. Cap context aggressively. Pass forward only what the next step actually needs. The 4,000 token loop that becomes 128,000 by step 5 is a context hygiene failure, not a model failure.
Build many small agents, not few large ones. Narrow agents with narrow tools, each with its own validator, are cheaper to run, easier to test, and easier to kill when one of them starts behaving badly. Skills, feedback loops, and checks belong inside the pipeline, not as an afterthought.
Hard budget caps at every level. Per agent, per task, per session, per day, per user. Provider alerts arrive too late when a loop is doubling context each step. The enforcement has to be in your gateway, not in the provider's billing console.
Measure quality continuously. Validator and reviewer agents should be reporting back on every run. If you cannot detect a regression within a small number of runs, you will detect it in your monthly invoice instead.
Run on Azure AI Foundry or direct API access with budget and rate controls wired in from day one, rather than letting individual developers spin up unbounded loops on personal API keys.
Do not oversell the cost savings to leadership. There will be months where token costs exceed the human labor they were supposed to replace. The honest pitch is variance, not savings. The good months are extraordinary. The bad months are expensive. The average, over a year of disciplined operation, is favorable for the work that is well suited to agents, and unfavorable for everything else.
This is why we always use AI with human validation. Humans provide feedback into the process, and more importantly, humans are the ones who can tell whether the model has changed under you or whether the input data is the actual problem. That distinction is not something an agent can reliably make about itself. With human validation in the loop, we have had great success over the last year. In the runs where validation was not there, or was there too late, it cost dearly.
Agentic AI pipelines for product security are genuinely transformative when they work. They find bugs that human teams miss. They scale into vulnerability volumes that human teams cannot handle. They produce regulatory artifacts faster than humans can write them. About 60 percent of the time, in our experience, they deliver exceptional ROI. The other 40 percent of the time, the model has shifted under you, the loop has gone sideways, or the pipeline has run up a bill that no private company would willingly pay.
That is the casino. You are not necessarily playing a losing game. You are playing a game where the house can change the rules mid hand, your bets are non refundable, and the only edge you have is discipline. Tight context. Small agents. Validators. Budgets enforced in code. Continuous measurement. A willingness to walk away from the table on bad days.
For medical device manufacturers building out AI assisted product security programs, the practical guidance is unromantic. Start small. Instrument everything. Assume the model will regress. Plan for the months when it does. The technology is real, the productivity gains are real, and so is the variance. Treat it like a serious business expense in a market with no quality guarantees, because that is exactly what it is.
---
Sources and further reading: