K7
K7dTwo ways to get a cluster tree
1. Create one from scratch — tree_create_cluster
vm_count member VMs on a tree-owned bridge and roots a cluster snapshot tree at them. The fork engine is N-node — there is no hard-coded 3. The per-member config is a regular VmConfig (its network field must be omitted: the tree creates and owns its own cluster network). Every member is booted from the same config, which is what k7d quickstart and the busybox demo use.
Practical sizing: the minimal CI guest costs ~3.2 GiB of host RAM per k3s node, an Ubuntu node ~1.1–1.4 GiB. The limit is host RAM, not the API — a 20-node cluster forks the same way the 3-node one does.
2. Adopt live VMs — tree_adopt_cluster
cluster-id annotation. The VMs must already share one cluster bridge — same cluster_id, distinct vm_indexes covering 0..N-1.
Adoption fails loud on any half-wired state (missing VM, mixed bridges, non-contiguous indices, already tree-owned). Validation failures put the VMs back untouched; a failure after extraction tears everything down — no half-adopted state survives. Adoption also writes the root’s keyframe: a full memory dump, and the only full dump the tree ever needs.
Setting up a k3s cluster inside the VMs is your scenario’s job (boot k3s server + agents, wait for Ready). k7d gives you the fork. The Ubuntu nodes guide shows the whole recipe with the command-line boot contract; the integration tests and
examples/cluster-tree-search show the busybox fixture end to end.Egress posture
Both create ops take an optionalegress policy — open (default), deny, or whitelist with CIDR:port pairs — enforced by the per-tree nftables jail. Forks inherit their source’s policy. Full shape: EgressPolicy; what the jail contains: Security model.
Forking, suspending, resuming a cluster
Once rooted, a cluster tree uses the same verbs as a single-VM tree — every operation applies to all members as one unit:guest_cids entry per member (ordered by vm_index), so your harness can dial each member’s agent, and latency_ns — the daemon’s own wall time for the fork, which is what the budgets are asserted on. Batch cluster forks run sequential warm forks — each branch gets its own bridge.
The fork_memory_full_copy flags in fork responses tell you whether each member forked CoW (false, the expected fast path) or had to fall back to a full memory copy (true — happens only for virtiofs-backed sources, e.g. pods with read-write hostPath mounts).
Forking from a child
Any live node is a fork source, so trees go as deep as the search does. The first fork from a child writes that child’s keyframe — a reflink clone of the nearest ancestor’s memory image plus only the pages the child dirtied, so a mid-tree fork point costs O(divergence), not O(guest RAM): ~0.8 s for an idle 3 × 2 GiB Ubuntu cluster versus ~7.8 s for the root’s full dump.tree_prepare_fork_source writes it explicitly if you want to show the checkpoint as its own step. See Snapshot tree — keyframes.
Forking at a different clock
tree_fork / tree_fork_batch accept dilation: the children run at that factor while the source keeps its own. One ×1 source tree can serve ×8 exploration branches and ×1 control branches at once. See Time warp.
What the numbers look like
Methodology and reproduction commands: Benchmarks. What works inside a forked cluster today (CoreDNS, Ingress, Cilium, Longhorn, …): Feature matrix.