K7
K7dThe trade-off: one daemon, many VMs, one address space
Live copy-on-write fork requires parent and child memory to be mappings in the same process — that is what makes a ~5 ms fork possible. So the k7d daemon hosts all VMs in one address space.
If your threat model is “mutually hostile customers on shared hardware”, put tenants on separate k7d hosts (or use K7’s kata-firecracker-devmapper backend). If it is “thousands of copies of my own environment, some of which an agent will break on purpose”, this design buys you the fork — and the jailer below keeps a rogue guest from reaching anything but its own bridge.
The tree jailer
Every live cluster payload — a root, a fork, an adopted cluster — owns one bridge, and every guest on every bridge sits in the same10.200.0.0/24. The jailer is the set of host-side controls that scope a tree to itself. All of them are on by default and every escape hatch is an explicit env var that logs a warning; an unrecognised value is a hard error, never a silent fallback.
Network jail (nftables, per bridge)
An nftables table per bridge, evaluated before the iptables rules:- Inter-tree traffic is dropped. Two trees on one node can no longer route to each other’s guests, even though they share a subnet and forks reuse IPs.
- Egress policy per tree, inherited by forks:
open(WAN + CNI egress, the default and the RuntimeClass product’s posture),deny(no WAN egress; intra-node CNI still forwarded), orwhitelist(only named CIDR:port pairs, tcp+udp, plus an optional resolver on 53). SeeEgressPolicy. - Host lockdown: an
inputchain so adeny/whitelisttree cannot reach services on the host itself, except whitelist entries and established flows. - Cloud metadata (
169.254.0.0/16) is dropped in every mode unless a whitelist names it. - Connection caps: per-tree concurrent-connection limit (16384) and new-connection rate (2000/s) — an inner cluster’s image pulls sit far below, a flood far above.
K7D_NET_JAIL_CT_MAX/K7D_NET_JAIL_NEW_RATE. - Per-TAP anti-spoof: a
netdevingress chain on every TAP drops frames whose source MAC, ARP sender, or IP differs from the TAP’s assigned identity or claims another guest’s address in the shared /24, and drops every EtherType except IPv4 and ARP (which also disables guest IPv6 and VLAN-tag evasion).
nft transaction. Teardown is RAII with the bridge and the TAP; stale tables from a daemon that died without running its destructors are reaped at startup. K7D_NET_JAIL=off skips installation for debugging.
Per-tree cgroup
Each tree gets a threaded cgroup for its vCPU and device threads (pids.max) and a domain cgroup for its helper processes (memory.max + pids.max), both sized from the tree’s TreeBudget. A tree that forks itself into a fork bomb, or a helper that leaks memory, hits its own wall. K7D_TREE_CGROUP=off disables.
Daemon seccomp
k7d is one process, so one seccomp allowlist installed before any VM exists covers every tree. Theioctl surface is matched by driver type byte (KVM, vhost, TUN, FICLONE, block, loop, XFS) rather than by enumerating request numbers. The filter survives execve, so it also covers the helpers the daemon spawns (virtiofsd, nft, ip, iptables). Default enforce; K7D_SECCOMP=log records instead of killing.
virtiofsd sandbox
virtiofsd — used only for the read-write hostPath fallback — is spawned inchroot sandbox mode so it cannot name a path outside its share. K7D_VIRTIOFSD_SANDBOX=namespace|none overrides.
Guest side (Ubuntu nodes)
The Ubuntu node image ships AppArmor profiles: the k7-agent may openAF_VSOCK (it is the harness’s control plane); everything else, pods included, runs under a profile that denies the vsock family. The agent vsock channel is deliberately not jailed on the host — it is point-to-point to the host CID, and no inter-tree vsock path exists.
What survives a fork
Everything inside the forked set: in-cluster TLS sessions, established TCP between member VMs, disk state, process state. Guest clocks are re-based so time does not jump backwards. What doesn’t: TCP to the outside world (the far end never forked). Forks keep their in-cluster addresses; only the host-facing identity is new.Hardening that exists today
- ≤30k lines of Rust for the VMM + shim — a small, auditable surface, deliberately not a kitchen-sink VMM.
- Memory safety by construction: Rust, with
unsafeblocks contained, annotated with safety comments, and enforced by lint (deny(unsafe_op_in_unsafe_fn)). - Formal methods on selected paths: Kani bounded model checking over
unsafe/ address-arithmetic harnesses; Aeneas→Lean proofs of the tree budget/eviction model. Not a claim that everything is proven — see Formal verification. - Loud failures: no silent fallbacks; every error is surfaced. Fuzzing runs over the agent protocol and virtio-blk request parsing.
- Guest agent is minimal: a static binary speaking JSON-lines over vsock;
runcruns inside the guest, never on the host. - The daemon never displaces itself:
k7d --version/--helpanswer without touching a running daemon, andserverefuses to start if a live listener already owns the socket.
Current limitations
- Single host, x86_64 Linux + KVM only. Cross-node fork is on the roadmap. Does not build on macOS/Windows.
- The daemon runs as root (KVM, TAP, bridge, nftables, and cgroup setup require it). The control socket is a local Unix socket — protect host access to it.
- Young project. One primary test machine, no external security audit yet. Small surface by design, but treat it accordingly.