Architecture

                     ┌────────────────────────────────────────┐
 your trainer /      │  Python client · MCP server · k7view   │
 agent ───────────▶  │  (examples/)                           │
                     └───────────────┬────────────────────────┘
                                     │
 user / kubectl ───▶ kube-apiserver ─┼─▶ kubelet → containerd
                                     │          │ exec()s a per-pod shim
                                     │          ▼
                                     │   ┌──────────────────────────┐
                                     │   │  containerd-shim-k7-v1   │  one process per pod;
                                     │   │  (Task ttrpc service)    │  thin client of the daemon
                                     │   └───────────┬──────────────┘
                                     │ JSON-lines over /run/k7d/k7d.sock
 ════════════════════════════════════▼═══════════════▼═════════════════ host
                     ┌────────────────────────────────────────┐
                     │  k7d daemon (k7d.service)              │  two processes when
                     │  ┌────────────┐ ┌────────┐ ┌─────────┐ │  K7D_PRIVSEP=on:
                     │  │ snapshot   │ │ native │ │ clock   │ │  root node broker +
                     │  │ tree +     │ │ KVM    │ │ owner   │ │  uid-k7d VM broker
                     │  │ budgets    │ │ VMM    │ │         │ │  owns: KVM fds, guest memory,
                     │  │ tree +     │ │ KVM    │ │ owner   │ │  vsock, TAPs/bridges, nft jail,
                     │  │ budgets    │ │ VMM    │ │         │ │  cgroups, disks, virtiofsd
                     │  └────────────┘ └────────┘ └─────────┘ │
                     └───────┬───────────────────┬────────────┘
                             │ KVM ioctls, mmap  │ jailer → firecracker  (backend: firecracker,
                             │ eventfds, vhost   │ own uid/chroot/pidns,  RuntimeClass k7-fc)
                             │ TAP               │ REST API over UDS in the jail
 ════════════════════════════════════▼═════════════════════════════════ guest
                     ┌──────────────────────────┐   ┌──────────────────────────┐
                     │ minimal CI guest         │   │ Ubuntu 24.04 node        │
                     │ k7d vmlinux + initramfs  │   │ distro kernel, ext4 root │
                     │ k7-agent = PID 1         │   │ systemd; k7-agent unit   │
                     │ runc → workload          │   │ k3s · Cilium · Tetragon  │
                     └──────────────────────────┘   └──────────────────────────┘

​

Who owns what

• The daemon owns the VMs. k7d runs as a long-lived systemd service. With K7D_PRIVSEP=on it is a root node broker (no listening socket) plus a uid-k7d VM broker that parses guest and shim input — see Privilege separation. Together they own every VM’s KVM file descriptors, guest memory, vsock, virtiofsd children, bridges, nftables jail tables, cgroups, disk images, and snapshot trees. With backend: firecracker the KVM fd and guest memory move into a per-VM jailed Firecracker child process the daemon spawns and drives over its REST API; the daemon still owns everything else (see Backends). Clients — the containerd shim, the K7 Python backend, your RL trainer, an MCP server — talk to it over a Unix socket, one JSON object per line. See Daemon API.
• The daemon owns the guest clock. Every guest reads kvm-clock; there is no RTC and no NTP client. Restore re-bases the clock with KVM_SET_CLOCK so a resumed guest knows the real time, and the same mechanism lets the daemon jump or dilate a whole cluster’s clock in lockstep. See Time warp.
• The shim is a thin client. containerd spawns one containerd-shim-k7-v1 per pod; its sandbox Create calls the daemon over /run/k7d/k7d.sock and fails loudly if the daemon is not running. Because VMs outlive shims, a respawned shim reattaches to its still-running VM (reattach_sandbox).
• The agent is not a Kubernetes component. k7-agent is a tiny static binary — PID 1 in the minimal guest, a systemd unit on an Ubuntu node. It exposes a JSON-line RPC over vsock (commands like ping, exec, read_file, write_file, container_start, container_exec, container_stop, container_output). Clients reach the agent directly over vsock using the guest_cid the daemon returns — the daemon does not proxy the agent protocol.
• runc runs inside the guest, never on the host. In the minimal guest the agent invokes it against erofs images on virtio-blk; on an Ubuntu node k3s’ own containerd does. The agent can also supervise a pinned dockerd as a guest service (k7d.katakate.org/docker), which is how k7 create --docker works without a privileged sidecar.

​

Two guests, two engines

​

Design choices worth knowing

• virtio-MMIO only, no PCI. Each virtio device is a 4 KiB MMIO window above guest RAM, discovered from the kernel cmdline. PCI bus emulation would add ~2k LoC for nothing k7d needs. Device set: virtio-net (userspace TAP pump), vhost-vsock (kernel) for the agent, virtiofs (vhost-user) for shared directories, virtio-blk for disk images, plus two legacy serial ports for console and agent fallback. The IOAPIC’s 24 pins cap a VM at 19 virtio devices — which is why per-container scratch, data disks, and image slots are opt-in.
• Guest memory is one mmap. Either memfd-backed (fresh VM) or MAP_PRIVATE over a keyframe file (restored/forked VM) — the foundation of CoW fork. Multi-vCPU guests (SMP via an MP table) snapshot, fork, and dilate like single-vCPU ones.
• Writable disks live on one reflink filesystem. The daemon loop-mounts a sparse XFS image (reflink=1) at /var/lib/k7d/disks; every scratch disk, data disk, and Ubuntu root disk is an ext4 image inside it, so a fork’s copy is a FICLONE on the same filesystem. No reflink support ⇒ error, never a silent full copy.
• No async in the VMM. Plain threads + Arc<Mutex<...>>; only the shim is tokio-based (containerd’s shim protocol is async). Every cross-thread signal is an eventfd.
• Errors are loud. Typed errors per module, no silent fallbacks, no unwrap() in library code. Resources (KVM fds, TAP devices, nft tables, cgroups, virtiofsd children) are cleaned up via Drop — drop order is deliberate so device worker threads never touch freed guest memory — and stale jail tables from a crashed daemon are reaped at startup.
• Rootfs via erofs on virtio-blk. Read-only host paths are packed into cached erofs images and attached as block devices. This is what keeps cluster forks CoW: no per-fork memory copies for filesystem shares (the virtiofs path forced a full memory copy per fork and survives only as the fallback for read-write hostPath).
• CNI is the host network contract, not a Flannel special case. When network.pod_netns is set, the daemon moves the TAP into the CNI-provided pod netns so the VM answers on the pod IP. That path follows the CNI netns + eth0 + pod IP contract; it works the same if Flannel is replaced or disabled.
• One jail per tree, on by default. nftables per bridge, cgroup per tree, seccomp on the daemon, chroot on virtiofsd. See Security model.
• The guest is hostile from its first instruction. Every host path the guest can influence is opened O_NOFOLLOW / RESOLVE_BENEATH relative to a share-root fd, anything shared by several VMs is host-enforced read-only, agent strings never reach a host path or command line, and the pod’s OCI confinement (seccomp, masked paths, device cgroups, capabilities) is passed through to guest runc rather than rewritten. See Guest-controlled paths.

​

The “start a pod” path