RuntimeClass k7 and k7-fc

k7d plugs into Kubernetes as a containerd runtime: pods with runtimeClassName: k7 run inside k7d microVMs instead of plain containers, and pods with runtimeClassName: k7-fc run the same guest under a stock Firecracker jail (see Backends). Four steps register the runtime on a k3s node.

The release installer does all four steps for you: sudo ./install.sh --with-k3s. The manual steps below are for understanding what it does, or for non-k3s containerd setups.

Install and start the daemon

The shim is a thin client: sandbox Create calls the daemon over /run/k7d/k7d.sock and fails loudly if it is not running.

cd dist/k7d-v0.7.0-x86_64-linux     # or the extracted release tarball
cp k7d /usr/local/bin/
mkdir -p /usr/local/share/k7d /opt/k7d
cp vmlinux initramfs.cpio.gz /usr/local/share/k7d/
cp runc.amd64 /opt/k7d/              # the shim's static runc for the guest
cat > /etc/systemd/system/k7d.service <<'EOF'
[Unit]
Description=k7d VM runtime daemon

[Service]
ExecStart=/usr/local/bin/k7d
Environment=RUST_LOG=info
Environment=K7D_KERNEL=/usr/local/share/k7d/vmlinux
Environment=K7D_INITRD=/usr/local/share/k7d/initramfs.cpio.gz
LimitNOFILE=524288

[Install]
WantedBy=multi-user.target
EOF
systemctl enable --now k7d

The guest pair is required: the daemon is the only process that decides which guest a VM boots, and without K7D_KERNEL / K7D_INITRD it refuses to create a VM rather than guessing. LimitNOFILE matters as soon as you fork clusters — the 1024 soft default runs out fast.

Install the shim binary

cp containerd-shim-k7-v1 /usr/local/bin/

containerd resolves the binary from the runtime type automatically (io.containerd.k7.v1 → containerd-shim-k7-v1).

Add to /var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.tmpl:

[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.k7]
  runtime_type = "io.containerd.k7.v1"
  pod_annotations = ["k7d.katakate.org/*"]
  privileged_without_host_devices = true
  snapshotter = "overlayfs"

Two things matter here:

pod_annotations forwards the k7d.katakate.org/* pod annotations (cluster-id, fork-vm-index, …) into the sandbox OCI spec so the shim can see them — required for cluster-mode pods deployed via kubectl.
Do not add BinaryName under [options] — the option value leaks into the cgroup path and breaks shim startup.

To also offer the Firecracker engine, add a second runtime that points the same shim at a config file selecting the backend (the k3s CRI does not stamp io.kubernetes.cri.runtime-handler, so the shim cannot tell the two handlers apart any other way):

[plugins."io.containerd.cri.v1.runtime".containerd.runtimes.k7-fc]
  runtime_type = "io.containerd.k7.v1"
  pod_annotations = ["k7d.katakate.org/*"]
  privileged_without_host_devices = true
  snapshotter = "overlayfs"
  [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.k7-fc.options]
    ConfigPath = "/etc/k7d/shim-k7-fc.toml"

mkdir -p /etc/k7d && echo 'backend = "firecracker"' > /etc/k7d/shim-k7-fc.toml
bash utils/install-firecracker.sh     # pinned firecracker + jailer, sha-verified

Restart k3s and create the RuntimeClass

systemctl restart k3s
kubectl apply -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: k7
handler: k7
EOF

For k7-fc, add a RuntimeClass with the measured PodOverhead so kubelet accounts for the out-of-process VMM:

kubectl apply -f - <<'EOF'
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: k7-fc
handler: k7-fc
overhead:
  podFixed:
    cpu: 50m
    memory: 32Mi
EOF

Verify with kubectl get runtimeclass k7 k7-fc — handlers k7 and k7-fc.

Using it

apiVersion: v1
kind: Pod
metadata:
  name: demo
spec:
  runtimeClassName: k7
  containers:
    - name: main
      image: alpine:3.20
      command: ["sleep", "infinity"]

kubectl logs, kubectl exec (including -it with PTY, resize, Ctrl-C, detach), pod IPs, Services, DNS, ConfigMaps/Secrets, emptyDir/hostPath/PVC volumes, memory/CPU limits, init containers, natural exit / restartPolicy, and multi-vCPU guests all work — see the feature matrix for the full support table and what’s still to come (hostNetwork, NetworkPolicy, IPv6, cross-node fork). Swap in runtimeClassName: k7-fc for the Firecracker engine; everything above works there too except hostPath / virtiofs volumes and time warp, which fail Create with a named error. The pod’s securityContext is honoured inside the VM: seccomp profiles, maskedPaths / readonlyPaths, device cgroups, capabilities and noNewPrivileges are passed through to the guest runc unchanged — see Security.

CPU limits and vCPUs: the shim derives the guest’s vCPU count from the pod’s CPU limit (cpu: "2" boots a 2-vCPU guest), and the limit is additionally enforced on the host via a CFS quota on the VM’s vCPU threads. Fork and snapshot preserve multi-vCPU state.

Docker inside the pod

metadata:
  annotations:
    k7d.katakate.org/docker: "true"
    k7d.katakate.org/docker-disk: 40Gi     # optional, default 20Gi

The guest agent (PID 1) supervises a pinned dockerd (27.5.1, with Compose and Buildx on PATH) whose overlay2 graph lives on a dedicated per-VM virtio-blk scratch disk; the binaries come from a read-only, host-enforced payload slot at /run/k7d/docker-tools. Nothing is added to the pod spec and no container is privileged. Because it is a guest service rather than a CRI sidecar, a warm fork keeps dockerd and its running containers (the child’s graph disk is a reflink). Any value other than "true", or docker-disk without docker, fails Create. docker.sock is guest root — read Docker as a guest service before enabling it for untrusted workloads, and prefer k7-fc for multi-tenant use. K7 exposes this as k7 create --docker.

Sizing writable storage

A container’s writes go to an overlay upper on the sandbox’s scratch disk — a sparse ext4 image on the host, attached as a read-write virtio-blk device — not to guest RAM. Writable capacity is therefore independent of the pod’s memory limit, and filling it gives the writing process an ordinary ENOSPC rather than guest-wide memory pressure. Size it per pod (default 8 GiB):

metadata:
  annotations:
    k7d.katakate.org/scratch-size: 20Gi

That image is shared by the pod, so one container filling it also fails its siblings. To scope the blast radius, give individual containers their own image with a name=size list over container names:

metadata:
  annotations:
    k7d.katakate.org/scratch-size: 8Gi                      # shared, everyone else
    k7d.katakate.org/container-scratch-size: builder=2Gi,logger=64Mi

Each named container gets a whole filesystem of its own, so an overrun is its own ENOSPC while siblings keep writing. Containers you don’t name go on sharing the pod’s scratch disk.

Per-container scratch is opt-in for two reasons: each image spends one of the VM’s virtio-MMIO device slots, and a pod that uses it is served by a fork rather than from the warm pool. A warm fork gets a reflink-CoW copy of every image, so a fork’s writes — and its usage numbers — are isolated from its parent and its siblings.

Kubernetes itself cannot see any of this: the CRI carries no ephemeral-storage figure in either direction, so resources.limits.ephemeral-storage never reaches the runtime and these annotations are the only sizing input. Usage is measured inside the guest and surfaced in the shim log, not in kubectl top.

Persistent data disks

Scratch dies with the pod. For state that must outlive the VM — a storage node’s replica directory, a database’s data dir — ask for named disks:

metadata:
  annotations:
    k7d.katakate.org/data-disks: longhorn:8Gi:/var/lib/longhorn,pg:20Gi:/var/lib/postgresql

Each id:size:guest-mount entry attaches an ext4 image data/<id>.img on the daemon’s reflink filesystem, created and formatted on first use and reused as-is on every later attach. Sandbox teardown leaves it in place; only the daemon’s delete_data_disk op removes it. A size that disagrees with an existing image is an error, not a resize. A warm fork gets a reflink-CoW copy, so a fork’s writes diverge from the source’s without a byte copy. Like per-container scratch, every disk spends one virtio-blk device slot, which is why it is opt-in.

Annotation reference

Annotation	Value	Meaning
`k7d.katakate.org/backend`	`native` \| `firecracker`	Which VMM boots the pod. `k7-fc` pods are Firecracker regardless; `native` on `k7-fc` is a Create error (`contradicts`)
`k7d.katakate.org/docker` / `docker-disk`	`"true"` / quantity	Agent-supervised in-guest dockerd with an `overlay2` graph disk (default 20Gi)
`k7d.katakate.org/scratch-size`	quantity (`8Gi`)	Pod-wide writable scratch disk. Default 8 GiB
`k7d.katakate.org/container-scratch-size`	`name=size,…`	Own writable image per named container
`k7d.katakate.org/data-disks`	`id:size:mount,…`	Named persistent disks
`k7d.katakate.org/image-slots` / `image-slot-size`	count / quantity	Spare virtio-blk slots (default 2 × 2 GiB, sparse) for images assigned after boot — later containers of the pod mount their rootfs without a VM restart. The default dropped from 4 to 2 to leave virtio-MMIO room for the Docker payload + graph pair or several hostPath disks
`k7d.katakate.org/cluster-id`	string	Put this pod’s VM on the shared bridge `k7-br-<id>` with a stable `10.200.0.{2+index}` identity — the cluster mode adopt path
`k7d.katakate.org/fork-source-cluster` / `fork-source-vm` / `fork-vm-index`	strings	Boot this pod as a warm fork of a live sandbox VM instead of cold-booting: the fork adopts the container already running inside the forked guest. The source keeps its vsock CID
`k7d.katakate.org/tree-id` / `tree-source-node-id`	strings	Record which named snapshot tree this cluster belongs to (and, for a fork, which tree node it came from)
`k7d.katakate.org/tree-adopt-cluster`	cluster-id	On sandbox Create, adopt the live VMs sharing that `cluster-id` as the root of tree `tree-id` (`TreeAdoptCluster`). Requires `tree-id`; the trigger pod must not be a member itself
`k7d.katakate.org/tree-fork-from` + `tree-fork-label`	node id + label	Fork tree `tree-id` from that node under the given branch label (`TreeFork`). Both or neither; runs after the adopt when combined
`k7d.katakate.org/tree-budget`	JSON `TreeBudget`	Optional adopt budget override (the daemon default is sized for 256 MiB busybox guests); rejected without `tree-adopt-cluster`

All values fail loud on a malformed entry — a pod that asked for a disk and silently did not get one would lose its data with the VM, which is the failure the annotations exist to prevent.

Because VMs outlive shims

The daemon owns the VMs, so a containerd restart or shim crash does not kill your workloads: a freshly respawned shim resolves its sandbox back to the still-running VM via the daemon’s reattach_sandbox operation, which also verifies the guest agent still answers before handing the VM back. The daemon also exposes pause_vm / resume_vm for a sandbox VM: vCPU threads are stopped and device workers parked while guest memory, devices, and the vsock CID stay put. This is what K7’s k7 pause uses against the k7d backend.

Getting started

Concepts

Guides

API

Deep dives

RuntimeClass k7 and k7-fc

Using it

Docker inside the pod

Sizing writable storage

Persistent data disks

Annotation reference

Because VMs outlive shims

​Using it

​Docker inside the pod

​Sizing writable storage

​Persistent data disks

​Annotation reference

​Because VMs outlive shims

Using it

Docker inside the pod

Sizing writable storage

Persistent data disks

Annotation reference

Because VMs outlive shims