In my previous post, I argued that the principles OSGi got right 25 years ago—dynamic modularity, service orientation, security isolation, dependency management, interoperability, reusability, and designing for longevity—are exactly what the AI-meets-IoT era demands. The response was overwhelming, and the most common follow-up question was always the same: "Okay, but how do we actually manage AI models on edge devices at fleet scale?"

It's the right question. And it's the one the industry is getting most dangerously wrong.

Deploying an AI model to a single edge device in a lab is a weekend project. Deploying, versioning, monitoring, updating, and rolling back AI models across tens of thousands of heterogeneous devices in production—while those devices keep running, keep making decisions, and keep generating revenue—is an engineering discipline that barely exists yet. Having chaired the IIC's Over-the-Air Updates Special Interest Group and spent 25 years managing software on IoT devices at scale, I can tell you: AI models are the hardest update payload we've ever had to deal with. And our tooling isn't ready.

AI models are not firmware — stop treating them like firmware

Here's the fundamental mistake I see organizations make: they take their existing OTA firmware update infrastructure and try to push AI models through it. It seems logical—a model is just a file, right? Ship it to the device, load it, done.

Wrong. AI models differ from firmware in ways that break every assumption traditional OTA systems are built on.

Models degrade silently. Firmware either works or it doesn't—a corrupted binary crashes at boot, and your A/B partition rollback catches it. AI models fail gradually through concept drift and data drift. The input data patterns shift over time—new product variants on a production line, changing lighting conditions in a warehouse, seasonal behavioral patterns in energy consumption. The model keeps running, keeps producing outputs, but those outputs become progressively less accurate. There is no crash to trigger a rollback. There is no error log to flag the problem. By the time anyone notices, the model has been making degraded decisions for weeks or months.

Models have non-binary compatibility. A firmware image is compiled for a specific hardware target—it runs or it doesn't. An AI model's compatibility depends on available inference runtime versions, available memory for the model's weight tensors, available compute for meeting latency requirements, available storage for input/output buffers, and the specific versions of preprocessing libraries that normalize sensor data before it reaches the model. A model that runs perfectly on a gateway with 2GB RAM and an NVIDIA Jetson may fail silently on the same model gateway with 1GB RAM—not by crashing, but by producing garbage outputs because the runtime silently truncated the model's precision to fit in memory.

Models have training provenance. Firmware has a source code repository and a build pipeline. AI models have all of that plus training data, hyperparameters, validation metrics, feature engineering pipelines, and the specific version of the training framework that produced them. When a model misbehaves in production, you don't just need to know which version it is—you need to trace back to the exact dataset, the exact preprocessing, and the exact training run that produced it. Without this provenance chain, debugging a model failure at the edge is essentially guesswork.

Models require bidirectional data flow. Firmware updates are one-directional: cloud pushes to device. AI model lifecycle management demands bidirectional flow. The device must send performance telemetry, inference confidence distributions, edge cases, and representative data samples back to the cloud or a central training pipeline—because without that feedback loop, you have no idea whether the model is still performing adequately. This is a fundamentally different data architecture than traditional OTA.

The three layers of modern IoT updates

In 2026, managing updates on IoT devices is no longer a single problem—it's three distinct problems that interact in complex ways.

Layer 1: Firmware and operating system. This is the traditional OTA domain. The RTOS, the Linux kernel, the bootloader, the hardware abstraction layer. Updates here are infrequent, high-risk, and typically require a reboot. Dual-bank (A/B) partitioning ensures rollback if the new image fails to boot. This layer is well-understood, and mature solutions exist.

Layer 2: Application software and containers. Business logic, protocol drivers, data pipelines, cloud connectors. Increasingly deployed as containers or modular application packages. Updates are more frequent than firmware, lower risk per update, but with complex dependency chains between components. This is where the OSGi principles I discussed in my previous post are most directly applicable—dynamic modularity, service binding, dependency resolution.

Layer 3: AI models. This is the new frontier. Model weights, preprocessing configurations, inference parameters, feature extraction pipelines. Updates must be frequent—potentially weekly or even daily as models are retrained on fresh data. They must be granular—updating a single model without touching the inference runtime, the preprocessing pipeline, or any other co-resident models. And they must be verifiable—not just "did the file arrive intact" but "is the model actually performing correctly on this device's specific data distribution."

The critical insight is that these three layers have different update cadences, different risk profiles, and different rollback semantics—and yet they have deep interdependencies. An AI model update may require a specific version of the inference runtime (Layer 2), which may require a specific version of a GPU driver (Layer 1). Most IoT platforms today either conflate all three layers into a single monolithic update—or handle them in completely separate systems with no awareness of cross-layer dependencies.

This is exactly the kind of problem that explicit dependency management with semantic versioning was designed to solve. The model declares: "I require inference-runtime version [3.2, 4.0) and preprocessing-pipeline version [1.5, 2.0)." The device's dependency resolver verifies these requirements can be satisfied before deploying. If they can't, the update is rejected with a clear explanation—not deployed with mysterious runtime failures.

What model versioning actually requires at the edge

Version control for source code is a solved problem. Version control for AI models is not—especially at the edge. Here's what a serious model versioning strategy must address:

Semantic model versioning with compatibility contracts. Just as OSGi bundles declared package imports with version ranges, AI models need to declare their interface contract: input tensor shapes and data types, output tensor shapes and semantics, required preprocessing steps, minimum inference runtime version, minimum hardware capabilities (memory, compute, accelerator type). A version bump in any of these dimensions is a breaking change that must be caught before deployment, not after.

Model lineage and reproducibility. Every model deployed to production must be traceable to its exact training run: the dataset version, the hyperparameters, the training framework version, the random seed, the validation metrics at training time. When a model degrades in the field, this lineage is your debugging lifeline. Without it, you're flying blind. I've seen organizations lose weeks trying to reproduce a model's training environment because nobody recorded which version of the feature extraction pipeline was used.

Fleet-aware versioning. In a heterogeneous fleet, not every device can run every model version. A gateway with an NPU can run a quantized INT8 model; the same gateway model without the NPU accelerator needs a smaller, more aggressively pruned variant. Version management must track not just "model version 3.2" but "model version 3.2, INT8 quantized, for hardware profile A" versus "model version 3.2, FP16, for hardware profile B." This is fleet-level variant management, and most MLOps tools designed for cloud deployment have no concept of it.

Configuration-model separation. A single model architecture may need different confidence thresholds, different preprocessing parameters, and different postprocessing rules depending on the deployment site. The factory in Munich has different lighting conditions than the factory in Shanghai. The model is the same; the configuration is site-specific. As I argued in my previous post: configuration must be a first-class runtime concept, decoupled from the artifact it configures. Baking inference thresholds into the model file is the edge AI equivalent of hardcoding database credentials.

Monitoring: the feedback loop most deployments lack

Here's a hard truth from 25 years of IoT device management: you cannot manage what you cannot observe. And most edge AI deployments have dangerously poor observability.

Traditional device monitoring asks: Is the device online? Is CPU usage normal? Is memory within bounds? Is the last heartbeat recent? These are necessary but nowhere near sufficient for AI workloads.

Edge AI monitoring must additionally track:

Inference performance metrics. Not just "did the model produce an output" but what is the distribution of confidence scores over time? A well-performing classification model should show a bimodal distribution—high confidence for clear cases, low confidence for ambiguous ones. When that distribution shifts toward the middle, the model is losing discriminative power. This is an early warning of drift that no traditional device metric will catch.

Data distribution monitoring. The statistical properties of the input data must be tracked continuously. If the feature distributions at inference time diverge significantly from the distributions the model was trained on, the model's predictions become unreliable—regardless of what its confidence scores say. This is data drift, and detecting it requires lightweight statistical tests running on the device itself, not just in the cloud.

Latency and resource consumption per model. When multiple models share an edge device, resource contention can cause subtle performance degradation. Model A's inference time creeps up from 15ms to 45ms because Model B's preprocessing is consuming too much memory, forcing the garbage collector into aggressive mode. Without per-model resource telemetry, this interaction is invisible.

Edge case collection. The most valuable data for model improvement comes from the cases where the model is least confident. Active learning frameworks identify these edge cases automatically and flag them for human annotation. But this requires a pipeline from device to cloud that prioritizes informativeness over volume—you don't need every inference result, you need the interesting ones.

The challenge is doing all of this on resource-constrained devices with limited bandwidth. You can't stream every inference result to the cloud—the bandwidth cost alone would be prohibitive, even before considering privacy regulations like GDPR that restrict the transmission of raw sensor data. The monitoring system itself must be edge-native: lightweight statistical aggregation on the device, with compact telemetry summaries uploaded at intervals, and selective raw data upload only for flagged edge cases.

Rollback: harder than it sounds

Everyone agrees that rollback capability is essential. A/B partitioning for firmware is well-understood: boot from partition A, install update to partition B, switch boot pointer, reboot. If B fails to boot, revert to A. Simple, robust, proven.

AI model rollback is fundamentally different—and harder—for three reasons.

First, the failure mode is different. A bad firmware image typically fails fast—boot loop, kernel panic, watchdog timeout. A bad model fails slow. It produces plausible-looking outputs that are subtly wrong. A quality inspection model that starts passing defective parts doesn't crash—it just costs you money, reputation, and potentially safety. Rollback triggers for AI models must be based on performance metrics, not on binary health checks. This means you need the monitoring infrastructure I described above before rollback can even function.

Second, the rollback scope is ambiguous. When a model update fails, what exactly do you roll back? Just the model weights? The model plus its preprocessing configuration? The model plus the inference runtime that was updated alongside it? If the new model required a new version of the preprocessing pipeline, and you roll back the model but keep the new preprocessor, you may end up in an untested combination that's worse than either version. Rollback must be atomic across the entire dependency graph—another problem that explicit dependency management was designed to solve.

Third, rollback has a data gap. When you roll back from model v3.2 to v3.1, the older model was trained on older data. If the reason v3.2 failed is that the real-world data distribution has shifted, v3.1 may perform even worse—because it was trained on an even older distribution. Rollback to a previous model version is a stopgap, not a solution. The real fix requires retraining on current data, which requires the feedback pipeline to be functioning, which requires the monitoring infrastructure to be in place. It's a system problem, not a deployment problem.

Staged rollouts: the fleet management imperative

One lesson that transfers directly from traditional OTA management to AI model deployment is: never update the entire fleet at once. Research suggests that poorly designed OTA processes can cause device failures in approximately 8.5% of a large fleet within three years. For AI models—with their subtler failure modes—the risks of fleet-wide simultaneous deployment are even higher.

A disciplined staged rollout for AI models looks like this:

Canary deployment (1-5% of fleet). Deploy the new model to a small cohort of representative devices. "Representative" is critical—canary devices must span the range of hardware variants, deployment environments, and data distributions in the fleet. A canary group that only includes devices in optimal conditions proves nothing. Run the canary for a defined observation period—days, not hours—with comprehensive performance monitoring.

Staged expansion (5% → 25% → 50% → 100%). At each stage, compare the new model's performance metrics against the previous version and against the canary cohort. Define explicit promotion criteria: inference accuracy above threshold X, latency below threshold Y, no increase in edge case rate beyond Z%. Define explicit abort criteria that trigger automatic rollback if violated.

Shadow mode deployment. For safety-critical applications, deploy the new model alongside the existing one. Both models run inference on the same inputs, but only the existing model's outputs drive actuation. The new model's outputs are logged for comparison. This eliminates risk entirely during the validation period—at the cost of doubling the inference compute load. On resource-constrained devices, this may not be feasible for every update, but for critical model changes it's the gold standard.

Time-zone and usage-aware scheduling. Schedule model updates during low-activity periods for each device's local context. A predictive maintenance model on a production line should not be updated during peak production. A building energy optimization model should not be updated during extreme weather events. This requires the update system to understand each device's operational context—yet another reason why configuration management must be a first-class capability.

The regulatory dimension: it's not optional anymore

I touched on the EU Cyber Resilience Act in my previous post. For AI model lifecycle management specifically, the regulatory implications are profound.

The CRA mandates that manufacturers provide security updates for the expected product lifetime—a minimum of five years. For AI models deployed on edge devices, this means you're committing to maintaining, retraining, and redeploying models for half a decade or more. Your model training pipeline, your OTA infrastructure, your monitoring systems—all of these must be operational and maintained for the full support period. Organizations that treat AI model deployment as a one-time event are setting themselves up for regulatory non-compliance.

Beyond the CRA, the EU AI Act introduces additional requirements for AI systems classified as high-risk. These include requirements for data governance, documentation, transparency, human oversight, accuracy, robustness, and cybersecurity—all of which must be demonstrable throughout the AI system's lifecycle. For edge AI in industrial settings, medical devices, or critical infrastructure, this means the model versioning and lineage tracking I described earlier isn't just good engineering practice—it's a legal requirement.

Bringing it together: what edge AI model lifecycle management must look like

Based on everything I've learned managing software on IoT devices for 25 years—and the new challenges AI models introduce—here's my view of what a mature edge AI model lifecycle management system requires:

A model registry with full provenance tracking. Every model artifact must be linked to its training data version, training parameters, validation metrics, and dependency requirements. This registry is the single source of truth for what's deployed where, and the starting point for any debugging or compliance audit.

Dependency-aware deployment. Model deployments must declare their requirements—inference runtime version, preprocessing pipeline version, hardware capabilities—and the deployment system must verify these requirements against each target device's actual state before attempting installation. Incompatible deployments are rejected pre-flight, not discovered post-crash.

Component-level granularity. Models must be deployable and updatable independently of the inference runtime, the preprocessing pipeline, and the application logic. Updating a model should not require redeploying the entire software stack. This is the dynamic modularity principle applied to AI—and it's the single most impactful architectural decision for reducing update risk and increasing update velocity.

Continuous on-device monitoring. Lightweight statistical monitoring of inference performance, data distribution, confidence distributions, and resource consumption—running on the device, not just in the cloud. Telemetry must be bandwidth-efficient and privacy-compliant, with selective upload of edge cases for annotation and retraining.

Automated drift detection and retraining triggers. When monitoring detects performance degradation beyond defined thresholds, the system should automatically initiate the retraining pipeline—collecting representative data from affected devices, training an updated model, and queuing it for staged deployment. The human in the loop approves the deployment, but the detection and preparation are automated.

Staged, metric-gated rollouts with atomic rollback. Canary → staged expansion → full fleet, with explicit promotion and abort criteria based on performance metrics, not just deployment success. Rollback must revert the entire dependency graph to a known-good state, not just the model file.

Regulatory-grade audit trails. Every deployment, every rollback, every configuration change, every monitoring alert must be logged immutably. For CRA and AI Act compliance, you must be able to demonstrate—years after the fact—exactly what model was running on which device at any given time, and why.

The road ahead

I'll be honest: no one has this fully figured out yet. The tools are immature. The patterns are still emerging. The gap between cloud-native MLOps and the realities of constrained edge devices remains vast.

But the principles I've advocated for 25 years—dynamic modularity, explicit dependencies, configuration as a first-class concept, designing for longevity—provide the architectural foundation on which these capabilities can be built. The organizations that start now, that architect their edge AI infrastructure around these principles from the beginning, will have a compounding advantage over those that bolt on lifecycle management as an afterthought.

AI models at the edge aren't a deploy-and-forget proposition. They're living, breathing software components that degrade, evolve, and require continuous care—for years, sometimes decades. Managing them well isn't just an engineering challenge. It's the difference between IoT products that deliver sustained value and IoT products that quietly fail.

That's the real meaning of "future-proof by design."