Container Plumbing Days 2025

In the field of AI and machine learning, model training has become an increasingly complex and resource-intensive task. Training jobs often run for days or weeks, distributed across multiple nodes with expensive GPU accelerators. Container checkpointing is a crucial technique for implementing fault tolerance, mitigating the impact of hardware and software failures by periodically saving the state of computations and resuming from the last checkpoint in the event of failures. While support for checkpointing has been recently integrated into Kubernetes, enabling checkpoint/restore coordination across multiple containers and nodes remains a challenge. In this talk, we are going to discuss how we have extended container runtimes and CRIU to synchronize checkpointing operations among multiple container instances in Kubernetes clusters. The talk will cover how we enable efficient end-to-end encryption for sensitive data in checkpoints and the integration with existing container platforms.

This session is about freeing your workloads from the shackles of traditional maintenance windows. By dynamically relocating running Virtual Machines (VMs) between Kubernetes nodes, KubeVirt keeps apps online during upgrades, scaling, or node failures.
Beneath the surface, KubeVirt orchestrates memory transfers and status checks, making VMs appear gravity-defying. “Pre-copy” sends most pages while VMs stay active, followed by a quick “stop-and-copy.” A “domain notify pipe” coordinates source and destination. For high dirty rates, “auto-converge” throttles CPU or “post-copy” starts the VM on the target, fetching pages on demand. Dedicated “migration0” and configurable parameters (e.g. completionTimeoutPerGiB) prevent stalls and manage bandwidth.
Learn best practices for migration timeouts, TLS toggling, and traffic isolation. We’ll also explore real-world scenarios like draining nodes for maintenance or using auto-converge on heavy VMs.

As ARM adoption grows for cloud-native workloads, thanks to its efficiency and cost advantages, multi-architecture container support is becoming a must-have, not a nice-to-have. But cross-arch builds come with their quirks: emulation headaches, manifest missteps, and pipeline slowdowns.

In this talk, Aditya and Anshika walk you through a CNCF-aligned, open source workflow to automate multi-arch builds using BuildKit, Kaniko, and Skopeo, producing reproducible, OCI-compliant images for both ARM and x86_64. They'll cover everything from base image strategy to manifest creation, registry publishing, and even image signing with cosign.

What you’ll learn:

1. How to configure and run multi-arch builds using open source tools like Kaniko and BuildKit.
2. Use Skopeo to inspect, sync, and validate multi-arch manifests.
3. Sign and verify images with cosign for trusted distribution.

By the end, you’ll have a solid, cloud-native blueprint to build once, run anywhere—from cloud to edge, x86 to ARM.