RuntimeClass k7

k7d plugs into Kubernetes as a containerd runtime: pods with runtimeClassName: k7 run inside k7d microVMs instead of plain containers. Four steps register the runtime on a k3s node.

The release installer does all four steps for you: sudo ./install.sh --with-k3s. The manual steps below are for understanding what it does, or for non-k3s containerd setups.

Install and start the daemon

The shim is a thin client: sandbox Create calls the daemon over /run/k7d/k7d.sock and fails loudly if it is not running.

cd dist/k7d-v0.4.0-x86_64-linux     # or the extracted release tarball
cp k7d /usr/local/bin/
mkdir -p /usr/local/share/k7d /opt/k7d
cp vmlinux initramfs.cpio.gz /usr/local/share/k7d/
cp runc.amd64 /opt/k7d/              # the shim's static runc for the guest
cat > /etc/systemd/system/k7d.service <<'EOF'
[Unit]
Description=k7d VM runtime daemon

[Service]
ExecStart=/usr/local/bin/k7d
Environment=RUST_LOG=info
Environment=K7D_KERNEL=/usr/local/share/k7d/vmlinux
Environment=K7D_INITRD=/usr/local/share/k7d/initramfs.cpio.gz
LimitNOFILE=524288

[Install]
WantedBy=multi-user.target
EOF
systemctl enable --now k7d

The guest pair is required: the daemon is the only process that decides which guest a VM boots, and without K7D_KERNEL / K7D_INITRD it refuses to create a VM rather than guessing. LimitNOFILE matters as soon as you fork clusters — the 1024 soft default runs out fast.

Install the shim binary

cp containerd-shim-k7-v1 /usr/local/bin/

containerd resolves the binary from the runtime type automatically (io.containerd.k7.v1 → containerd-shim-k7-v1).

Add to /var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.tmpl:

[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.k7]
  runtime_type = "io.containerd.k7.v1"
  pod_annotations = ["k7d.katakate.org/*"]
  privileged_without_host_devices = true
  snapshotter = "overlayfs"

Two things matter here:

pod_annotations forwards the k7d.katakate.org/* pod annotations (cluster-id, fork-vm-index, …) into the sandbox OCI spec so the shim can see them — required for cluster-mode pods deployed via kubectl.
Do not add an [options] section with BinaryName — the option value leaks into the cgroup path and breaks shim startup.

Restart k3s and create the RuntimeClass

systemctl restart k3s
kubectl apply -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: k7
handler: k7
EOF

Verify with kubectl get runtimeclass k7 — it should show HANDLER: k7.

Using it

apiVersion: v1
kind: Pod
metadata:
  name: demo
spec:
  runtimeClassName: k7
  containers:
    - name: main
      image: alpine:3.20
      command: ["sleep", "infinity"]

kubectl logs, kubectl exec (including -it with PTY, resize, Ctrl-C, detach), pod IPs, Services, DNS, ConfigMaps/Secrets, emptyDir/hostPath/PVC volumes, memory/CPU limits, init containers, natural exit / restartPolicy, and multi-vCPU guests all work — see the feature matrix for the full support table and what’s still to come (hostNetwork, NetworkPolicy, IPv6, cross-node fork).

CPU limits and vCPUs: the shim derives the guest’s vCPU count from the pod’s CPU limit (cpu: "2" boots a 2-vCPU guest), and the limit is additionally enforced on the host via a CFS quota on the VM’s vCPU threads. Fork and snapshot preserve multi-vCPU state.

Sizing writable storage

A container’s writes go to an overlay upper on the sandbox’s scratch disk — a sparse ext4 image on the host, attached as a read-write virtio-blk device — not to guest RAM. Writable capacity is therefore independent of the pod’s memory limit, and filling it gives the writing process an ordinary ENOSPC rather than guest-wide memory pressure. Size it per pod (default 8 GiB):

metadata:
  annotations:
    k7d.katakate.org/scratch-size: 20Gi

That image is shared by the pod, so one container filling it also fails its siblings. To scope the blast radius, give individual containers their own image with a name=size list over container names:

metadata:
  annotations:
    k7d.katakate.org/scratch-size: 8Gi                      # shared, everyone else
    k7d.katakate.org/container-scratch-size: builder=2Gi,logger=64Mi

Each named container gets a whole filesystem of its own, so an overrun is its own ENOSPC while siblings keep writing. Containers you don’t name go on sharing the pod’s scratch disk.

Per-container scratch is opt-in for two reasons: each image spends one of the VM’s virtio-MMIO device slots, and a pod that uses it is served by a fork rather than from the warm pool. A warm fork gets a reflink-CoW copy of every image, so a fork’s writes — and its usage numbers — are isolated from its parent and its siblings.

Kubernetes itself cannot see any of this: the CRI carries no ephemeral-storage figure in either direction, so resources.limits.ephemeral-storage never reaches the runtime and these annotations are the only sizing input. Usage is measured inside the guest and surfaced in the shim log, not in kubectl top.

Persistent data disks

Scratch dies with the pod. For state that must outlive the VM — a storage node’s replica directory, a database’s data dir — ask for named disks:

metadata:
  annotations:
    k7d.katakate.org/data-disks: longhorn:8Gi:/var/lib/longhorn,pg:20Gi:/var/lib/postgresql

Each id:size:guest-mount entry attaches an ext4 image data/<id>.img on the daemon’s reflink filesystem, created and formatted on first use and reused as-is on every later attach. Sandbox teardown leaves it in place; only the daemon’s delete_data_disk op removes it. A size that disagrees with an existing image is an error, not a resize. A warm fork gets a reflink-CoW copy, so a fork’s writes diverge from the source’s without a byte copy. Like per-container scratch, every disk spends one virtio-blk device slot, which is why it is opt-in.

Annotation reference

Annotation	Value	Meaning
`k7d.katakate.org/scratch-size`	quantity (`8Gi`)	Pod-wide writable scratch disk. Default 8 GiB
`k7d.katakate.org/container-scratch-size`	`name=size,…`	Own writable image per named container
`k7d.katakate.org/data-disks`	`id:size:mount,…`	Named persistent disks
`k7d.katakate.org/image-slots` / `image-slot-size`	count / quantity	Spare virtio-blk slots (default 4 × 2 GiB, sparse) for images assigned after boot — later containers of the pod mount their rootfs without a VM restart
`k7d.katakate.org/cluster-id`	string	Put this pod’s VM on the shared bridge `k7-br-<id>` with a stable `10.200.0.{2+index}` identity — the cluster mode adopt path
`k7d.katakate.org/fork-source-cluster` / `fork-source-vm` / `fork-vm-index`	strings	Boot this pod as a warm fork of a live sandbox VM instead of cold-booting: the fork adopts the container already running inside the forked guest. The source keeps its vsock CID

All values fail loud on a malformed entry — a pod that asked for a disk and silently did not get one would lose its data with the VM, which is the failure the annotations exist to prevent.

Because VMs outlive shims

The daemon owns the VMs, so a containerd restart or shim crash does not kill your workloads: a freshly respawned shim resolves its sandbox back to the still-running VM via the daemon’s reattach_sandbox operation, which also verifies the guest agent still answers before handing the VM back. The daemon also exposes pause_vm / resume_vm for a sandbox VM: vCPU threads are stopped and device workers parked while guest memory, devices, and the vsock CID stay put. This is what K7’s k7 pause uses against the k7d backend.

Getting started

Concepts

Guides

API

Deep dives

Using it

Sizing writable storage

Persistent data disks

Annotation reference

Because VMs outlive shims

​Using it

​Sizing writable storage

​Persistent data disks

​Annotation reference

​Because VMs outlive shims

Using it

Sizing writable storage

Persistent data disks

Annotation reference

Because VMs outlive shims