Explainer · AI-enabled devices

The model is the last frame.
Secure the pipeline that made it.

An AI-enabled device ships a model, and the model is the output of a data pipeline: acquisition, annotation, verification and validation, training, evaluation, packaging. Attacks land anywhere along that pipeline, and most of them land before the device exists. ELTON maps the pipeline into the digital twin, tests it as infrastructure, and manages what it finds for the life of the product.

WHERE THE MODEL IS BUILTSix of the seven FDA-named AI threats land before the device existsTRAINING ENVIRONMENT · OFF THE DEVICE · 5 OF 7 THREATSSHIPPED · 2 OF 7ACQUIREclinical sources,sensors, sitesData poisoningANNOTATElabels, referencestandard, arbitrationModel biasVERIFY + VALIDATEquality verification,splits, hold-outsIntegrity breakTRAIN + EVALUATEruns, metrics,overfitting checksOverfittingREGISTRYpackaged, versioned,signed, servedStealing + leakageDEVICEthe model at runtimeModel evasionPOSTMARKETfield inputs, updatesPerformance driftRETRAIN · every pass re-enters the pipelineattacks the dataattacks the processattacks the artifactattacks at runtimenot an attack, looks like one
FDA's January 2025 draft names seven AI-specific threats. One is tested at runtime. The rest are tested where the model is built.
Why this is a cybersecurity problem

Cybersecurity is about data.
ML never stops using it.

Every system that consumes data has an integrity problem, and every security program already knows how to handle a tampered input at runtime. Machine learning is different in one way only: it consumes data to become the software. The training set is the source code. Whatever was in it, the model learned, and a violation of that data's integrity during training is not a bug you patch later. The run is over. You start again from zero.

The data is the source

Training data is the codebase

A conventional device is defined by its code, and its data flows through it. An AI-enabled device is defined by the data it was trained on. Anyone who can write to the training set can write to the product, without touching a line of code or a single build server.

Verification and validation

The choke point everything trusts

The V&V phase is where the pipeline decides what is true: which samples are clean, which labels stand, which split is held out. Everything downstream trusts that decision. An attacker with write access there does not need to poison millions of records. They need to change what passes.

No undo

You cannot cut the data back out

Once a poisoned or corrupted sample has been trained in, it lives in the weights. There is no patch that removes it. The only remedy is to discard the run and retrain from a clean set, which means the cost of an integrity failure is the cost of the whole run, and for a shipped device, the cost of a release.

Efficacy or security? When a model starts returning wrong answers, the first question is whether the model is wrong or the world moved. A false negative from a tampered model and a false negative from drift look identical at the bedside. Telling them apart takes evidence about the pipeline, not a better look at the output. That is the evidence ELTON is built to produce.
Same risk, different clock

Cheap to train,
expensive to change.

The integrity risk is the same for every model. What differs is when the bill arrives. A frontier LLM costs millions to train once, so a poisoned run is catastrophic at training time. A model retrained every night on live data barely notices, because tomorrow overwrites today. A medical device model can be cheap to train and still carry the LLM's problem, because once it ships, changing it means a field update, a validated release, and a conversation with a regulator.

%s
The device row is the interesting one. Not costly up front, very costly in the field, so the same risk with the timing reversed.
Federated training

The pipeline you do not own

Federated learning moves the model to the data, so the training set never leaves the hospitals that hold it. It also means the pipeline now spans sites you do not control, every site is a place an update can be poisoned, and the aggregation step becomes the integrity choke point for the whole model. The surface grows, and the verification has to grow with it.

What buyers will ask

Can the model be updated at all?

Every hospital procurement will eventually ask why it should buy a device whose model cannot be updated when a better one arrives, and what happens when the current one degrades. An updatable model is a pipeline that runs after shipping, which means the pipeline is part of the product's attack surface for its whole service life, not just its development.

Performance drift

Drift is not an attack.
It looks exactly like one.

A model assumes the inputs it sees in the field are drawn from the same world as its training data. That assumption fails quietly. A site switches to new imaging instrumentation. A sensor ages and its noise floor shifts. Hardware starts to fail in ways that never appeared in the training set. Patient populations change, and biology itself changes over time, so the detector trained on one body sees another. The colon cancer model stops detecting colon cancer, and nothing in the device tells anyone.

Drift: the world moved

Inputs shifted away from the training distribution. The model is doing exactly what it learned, on data it never learned. FDA names this as threat seven, performance drift, and IEC PAS 63621 requires data quality analysis across the product lifecycle precisely because data that met requirements at verification can stop meeting them later.

Tampering: someone moved it

The data, the pipeline or the artifact was changed. The model is doing what an attacker taught it. From the output alone this is indistinguishable from drift, which is why an incident that starts as a performance complaint can be a security incident that nobody has recognized yet.

Ground truth arrives late. The only reliable signal that a fielded model is wrong today is a false negative that someone eventually notices, and the only formal channel is the complaint and MDR process, which is manual and slow. By the time a trend appears in the complaint file, the model has been wrong for months. Monitoring the pipeline's integrity is the one signal that arrives before the harm does.
What the regulators already wrote down

FDA named the threats.
IEC named the controls.

Two documents now describe the same pipeline from opposite ends. FDA's January 2025 draft guidance for AI-enabled device software functions lists seven cybersecurity threats specific to AI in Section XII, and expects sponsors of AI-enabled cyber devices under section 524B to document them premarket and manage them postmarket. IEC PAS 63621:2026 lays out the data lifecycle a manufacturer has to run for any data used to train, test or validate a device model. Read together, they are a threat list and a control list for the same infrastructure.

Lifecycle stage (IEC PAS 63621)
What the standard requires
The FDA-named threat it answers
Data requirements and planning (6.2, 6.3)
Defined intended use, data requirements, representativeness and bias mitigation planned up front, with documentation of laws, standards and clinical norms the data must meet.
Model bias, overfitting. A plan that never named the population cannot show the set was not steered.
Data acquisition (6.4)
Documented diversity of sources: population, acquisition site, equipment, parameter settings, operator qualification and acquisition time. Cloud storage named with provider, access path and permissions.
Data poisoning. Every source and every storage location is an entry point for inauthentic data.
Dataset composition and revision control (6.5.3)
Unique identifiers for every data item. Approved items cannot change; a change is a new version with a new identifier. Revision control includes integrity protection so data is authentic and unchanged.
Data poisoning, data leakage. This is the control an attacker with write access has to defeat, and the one a test has to try to defeat.
Data annotation (6.5.4)
Defined annotation process with annotator, inspector and manager roles, identity of the person who annotated, date and time of every annotation and modification, all under revision control.
Model bias. Labels are where a model learns what to believe. Unaudited annotation tooling is a write path into the model.
Data quality verification (6.5.6)
Verification that the dataset meets the defined requirements before use, using the manufacturer's documented quality characteristics: accuracy, completeness, authenticity, traceability and the rest of Annex B.
Data poisoning, integrity. The choke point. If a tampered sample can pass verification, everything downstream inherits it.
Data quality analysis (6.5.7)
Ongoing analysis across the lifecycle, with root cause, corrective action and further monitoring documented whenever verification shows a quality issue. Explicitly covers data drift and changes to hardware or software.
Performance drift. The standard's own note says data that met requirements at one point can stop meeting them later.
Provisioning and decommissioning (6.6, 6.7)
Datasets provided to the model with their description and conformance statement; data removed irretrievably when licenses or consent lapse, with security and privacy maintained if retained for reuse.
Model stealing, data leakage. Provisioned datasets and retained sets are the artifacts inversion and exfiltration go after.
What this means for a submission. The standard tells you what a well-run pipeline documents. The guidance tells you what a reviewer will ask about it. Neither one tells you whether the controls actually hold against someone trying to break them. That is a test, and it has to be run against the real pipeline.
How ELTON secures the pipeline

Map it. Test it.
Manage it for the product's life.

ELTON treats the training environment as part of the device. The pipeline goes into the digital twin next to the firmware and the interfaces, the same agentic testing that runs against the device runs against the pipeline's infrastructure, and every finding is verified, traced to a component and to the FDA-named threat it answers, then carried forward across every retrain and release.

01 · MAP

The pipeline in the twin

From the documentation the QMS already produces, including the IEC PAS 63621 dataset descriptions, ELTON models the pipeline as infrastructure: data sources and storage, annotation tooling, verification and validation jobs, training and evaluation runs, the model registry, packaging and the field update path. Trust boundaries and access paths go in with them.

  • Federated topologies modeled per site, with the aggregation step as its own component
  • Every dataset version and model artifact traceable to the release it produced
02 · TEST

The pipeline as an attack surface

Agentic vulnerability testing runs against the pipeline the way it runs against the device: access control on dataset storage and cloud buckets, integrity of revision control, authentication on annotation platforms, artifact signing and registry access, secrets in training infrastructure. Then the test that matters most: can a tampered sample pass verification and validation and reach training?

  • Model evasion tested on the deployed device, over TestLink or in our lab
  • Each test case mapped to the Section XII threat it answers
03 · MANAGE

Evidence across every retrain

Findings are verified for exploitability on the real pipeline, rated with the qualified MDDT methodology, and dispositioned with the test case attached. Pipeline components are monitored for new CVEs like any other part of the device, and every retrain or release re-enters the pipeline and is retested against what ELTON already knows.

  • Drift or tampering answered with pipeline integrity evidence, not a guess at the output
  • Living VEX and audit-ready evidence for premarket and postmarket
Get started

See your pipeline through ELTON.

Bring one AI-enabled product. We map the pipeline that built its model into the twin, test it as infrastructure, and show you which of the seven threats actually reach it.

Questions

Common questions about
securing AI inside a device.

Is the training pipeline in scope for FDA cybersecurity review?

Yes. Section XII of FDA's January 2025 draft guidance for AI-enabled device software functions names seven AI-specific threats, and six of them live in the data and the pipeline that produce the model: data poisoning, model inversion and stealing, data leakage, overfitting, model bias and performance drift. Sponsors of an AI-enabled cyber device under section 524B are expected to document these risks in the submission.

Why is an integrity violation during training worse than a runtime bug?

A runtime bug is patched in code. A poisoned training set is learned by the model, and there is no way to cut the bad data back out of the weights after the fact. The run is discarded and started again from zero. For a medical device that has already shipped, that restart is a validated release and a regulatory event, which is why the cost lands after shipping.

How do you tell model drift from tampering?

From the outside they look the same: a model that starts returning wrong answers. Drift comes from the world moving, such as new instrumentation, aging sensors or a patient population that no longer matches the training set. Tampering comes from someone changing the data, the pipeline or the artifact. The way to separate them is evidence about the pipeline itself: integrity of the dataset versions, access to the registry, and test results that show whether a tampered sample could have passed verification. ELTON produces that evidence.

What does IEC PAS 63621 require that touches security?

IEC PAS 63621:2026 sets a data lifecycle framework for data used to train, test or validate an AI model in a medical device. It asks for revision control with integrity protection so data is authentic and unchanged, unique immutable identifiers for every data item, provenance and annotator identity for labels, documented storage security and access permissions, and data quality analysis across the product lifecycle that accounts for drift. Every one of those is a control an attacker would target and a test ELTON can run.

Does ELTON test the model, or the pipeline?

Both, and the split matters. Model evasion is tested on the deployed device, over TestLink or in our lab. The other six threats are tested against the pipeline as infrastructure: dataset storage, annotation tooling, verification and validation jobs, training runs, the registry and the release path. Findings trace to a component in the digital twin and to the FDA-named threat they answer.

What about federated learning?

Federated training moves the model to the data instead of the data to the model, so the pipeline now spans sites you do not control. Every site is a place an update can be poisoned and the aggregation step becomes the integrity choke point. ELTON models the federation topology in the twin and tests the update and aggregation path the same way it tests a central pipeline.

Exploitability management for medical devices. FDA §524B methodologyExploitability proven at runtime95% faster than legacy testing Book a Demo
Platform
OverviewAvoid FDA DeficienciesAvoid Consulting FeesDigital Twin TraceabilityAI MedDevice PentestingExploitability VerificationVulnerability ChainingRemediation OptimizationRemote TestLink™Incident ResponseAutomated VEX & MetricsCVSSv4 Migration
Solutions
EnterpriseStartups / SMBs Postmarket SurveillanceIncident ResponseSecurity EngineeringRegulatory AffairsFDA §524BEU MDR/CRAEU REDNIS2IMDRF N60 / N73Japan MHLW
Why ELTON
One Solution Subscription TestingAI-NativeFDA ComplianceVerified ExploitabilityELTON vs. Legacy TestingThreat-Led AI PentestingMDDT MethodologyCredentialsDevice ModalitiesPricing
Resources
FDA Deficiency ListFDA Testing RequirementsFDA Cyber SOPs & TemplatesRemediation LibraryRegulatory GuidesWebinarsAI NewsletterThe End of Legacy TestingThe AI Vulnerability ExplosionAI Inside the ProductSecurity AdvisoriesWhitepapersIntelligence & Blog
Company
AboutLeadershipCareersPartnershipsContact Meet ELTON