> ## Documentation index > Fetch the complete documentation index at: https://docs.katakate.org/llms.txt > Ask it with GET https://docs.katakate.org/ask?q=. Short keywords rank best. > Use these to discover all available pages before exploring further. # Tenants, roles and OIDC > Multi-tenancy in K7 0.5.0: Tenant objects with quotas and defaults, the four roles, API keys, OIDC login and role bindings, production vs development mode K7 0.5.0 turns the API into a multi-tenant control plane. Three things changed from 0.4.0: every credential now has a **role**; namespaces are grouped into **tenants** that carry quotas, defaults and dedicated nodes; and humans can sign in with **OIDC** instead of holding a key. Keys moved from `/etc/k7/api_keys.json` on a node to the `kube-system/k7-api-keys` Secret, so every API replica sees the same store. K7 roles are **application roles inside k7-api, not Kubernetes RBAC**. The `k7-api` ServiceAccount stays the only Kubernetes actor; nobody gets a Kubernetes identity through K7, and a K7 `admin` cannot `kubectl` anything. Protect kubeconfigs and node root separately. ## Roles | Role | Scope | Can | |---|---|---| | `read-only` | one tenant | list/get sandboxes, metrics and snapshots | | `user` | one tenant | `read-only` plus create/delete, exec, logs, pause/resume, fork, snapshots | | `operator` | cluster | read everything, snapshot GC, node storage, node dedication, tenant read; no exec, no keys | | `admin` | cluster | everything, including API keys, tenants, role bindings and remote install | One table (`PERMISSIONS` in `src/k7/api/authz.py`) maps every route to its roles; a route missing from it answers 500 and fails CI. Checks compose — route role, then tenant, then the key's namespace/node narrowing — and the first failure is a `403` that names the missing role or scope. `k7 whoami` / `GET /api/v1/auth/whoami` shows what a credential resolves to. ## Tenants A `Tenant` (`tenants.k7.katakate.org/v1alpha1`, cluster-scoped) claims namespaces and owns everything a tenant key may do: ```bash k7 tenant create acme -n acme-dev -n acme-prod \ --default-size cpu=1,memory=2Gi,disk=10Gi \ --quota sandboxes=20,vcpu=16,memory=32Gi,snapshotStorage=200Gi \ --max-size cpu=8,memory=32Gi \ --network egressBps=100M,ingressBps=100M k7 nodes dedicate k7-node-01 --tenant acme k7 generate-api-key acme-ci --role user --tenant acme k7 tenant list # quotas and reconciled usage ``` - **Namespaces** are created if missing and labelled `k7.katakate.org/tenant`. A tenant key reaches every namespace of its tenant (intersected with its own `-n` list), read from the Tenant on every request — a namespace added later works without reissuing the key, and a key whose tenant is deleted reaches nothing. A namespace claimed by two Tenants (only possible by writing CRs by hand) fails every request there with 500 `TenantConflict`. - **Dedicated nodes** (`spec.nodes`, or `k7 nodes dedicate`): the tenant's sandboxes land only there (node selector + toleration), and the `NoSchedule` taint keeps everyone else off. This is what gives tenant isolation on k7d, whose daemon is shared per node — a key's `--node` pin alone restricts that key, not the others. - **`defaultSize`** (`cpu`, `memory`, `disk`) fills creates without limits. Production mode requires CPU and memory on every sandbox, so set it or make callers pass `limits`. - **Quotas** (`sandboxes`, `vcpu`, `memory`, `disk`, `snapshotStorage`) are checked at K7's two choke points — the sandbox Deployment create and the VolumeSnapshot create — so create, fork, restore, pause-with-snapshot and named snapshots are all covered. The check and the create run under a per-tenant Lease (`k7-quota-`), so concurrent creates on any number of API replicas cannot overshoot; a Lease not acquired in 5 s is 503 and creates nothing. Over-quota is `403 QuotaExceeded` with the dimension, limit, usage and request, and is audit-logged. Usage is **reserved, not used**: a paused sandbox keeps its vCPU, memory and disk until deleted. The leader replica writes `Tenant.status.usage` every 60 s. - **Per-VM limits** : `maxSandboxSize` caps a request (400 above it). `network` (`egressBps` / `ingressBps`, bits/s) is enforced by the hypervisor — Firecracker token buckets on kfd, `tc` on the tap for kql — so it works whatever the CNI; a request may lower a cap, never raise it. A backend that **cannot** enforce a configured limit refuses the create with `400 LimitNotEnforceable` instead of running unlimited: `diskIO` on every backend (no block rate limiter in Kata 3.24 or k7d yet), `network` on k7d / k7d-fc, and a `disk` quota or `maxSandboxSize.disk` on k7d / k7d-fc (K7 cannot read a k7d VM's writable-disk size). - **Enforcement is at the K7 API.** `kubectl apply` by a cluster admin bypasses it. For a single-namespace tenant a Kubernetes `ResourceQuota` (`k7-tenant-quota`) is the backstop; multi-namespace tenants have none, since a ResourceQuota cannot span namespaces. - **Migrating from 0.4.0 namespace-scoped keys:** `k7 tenant adopt --from-keys` proposes one tenant per connected group of key namespaces (dry run), `--apply` creates them and binds the keys. ## API keys ```bash k7 generate-api-key NAME --role read-only|user|operator|admin [--tenant T] [-n NS]... [--node NODE]... [--expires-days 365] k7 list-api-keys # role, tenant, scopes; warns on unscoped_legacy k7 revoke-api-key NAME ``` Tokens are `k7__`; the store holds `sha256(secret)` (the secret is 256 random bits, so a slow KDF adds nothing) in the `kube-system/k7-api-keys` Secret. The API never writes the store on authenticate: it serves a snapshot refreshed every 5 s and answers 503 rather than trust one older than 60 s, so revocation reaches every replica within one refresh. Key commands edit the Secret through kubectl (run them on a server node), or go through k7-api with `--api` using an `admin` credential (`POST /api/v1/keys`). Protect etcd (`--secrets-encryption` on K3s, KMS on managed clusters): anyone with `get secrets` in `kube-system` reads the hashes. **Upgrading from 0.4.0:** `k7 install` runs `k7 api keys migrate`, which copies `/etc/k7/api_keys.json` into the Secret and gives every pre-role key a role: `admin: true` → `admin`, namespace-scoped → `user` (still narrowed), unscoped → `admin` flagged `unscoped_legacy`. Such keys keep working in development mode; re-issue them with a real role before switching to production. ## OIDC login Install with all five settings (or none; `https://` issuers only): ```bash k7 install ... \ --oidc-issuer https://idp.example.com/realms/k7 \ --oidc-audience k7-api \ --oidc-client-id k7-cli \ --oidc-groups-claim groups \ --oidc-username-claim email ``` Then `k7 login` (device flow; `--browser` for Authorization Code + PKCE on a loopback redirect) caches the tokens in `~/.config/k7/credentials.json` (0600) and refreshes them itself; `k7 logout` deletes the cache and revokes at the IdP. Clients send `Authorization: Bearer `; k7-api validates statelessly — signature against the issuer's JWKS (cached `--oidc-jwks-ttl` seconds, default 3600; a failed refresh is 503, never a stale allow), `alg` RS256/ES256 only, `iss`, `aud`, `exp` (60 s leeway) and `sub` required, `email_verified: true` when the username claim is `email`. k7-api stores no sessions and never sees refresh tokens. Register `k7-cli` as a **public** client with the device grant enabled. Per-IdP claim settings (Keycloak, Entra ID, Okta, Auth0, Dex) are in [docs/HARDENING.md](https://github.com/Katakate/k7/blob/main/docs/HARDENING.md#idp-claim-settings). ### Role bindings Groups, usernames or subjects map to roles through ConfigMap `k7-role-bindings`, read on every request: ```yaml # k7 role-bindings apply -f bindings.yaml bindings: - {group: k7-admins, role: admin} - {group: sre, role: operator} - {group: acme-devs, role: user, tenant: acme} - {group: acme-viewers, role: read-only, tenant: acme} deny_subjects: [] # OIDC `sub` or username; blocks on the next request ``` A subject in several groups gets the highest role per tenant plus the highest cluster role. An unbound subject gets a 403 that echoes its username and groups, to debug claim shapes. A write that leaves no `admin` binding while no unexpired `admin` key exists is refused. A JWT is valid until `exp` — configure 1 h or shorter tokens at the IdP; `deny_subjects` is the immediate stop. Keep one **break-glass `admin` key** offline: keys keep working during an IdP outage, and `k7 generate-api-key` with the node kubeconfig always works. ## Production and development modes `--api-mode` is chosen at install and never defaulted (chart: `api.mode`). Every response carries `X-K7-Mode`; `/health` and `k7_api_info{mode}` report it. | | `production` | `development` | |---|---|---| | Keys without a tenant or cluster role (`unscoped_legacy`, pre-role, namespace-only) | refused at startup (k7-api names each one) and `403` on every route | work as cluster `admin` | | `read-only` / `user` keys | need `--tenant` | may be unscoped | | Sandbox CPU and memory limits | required (from `limits` or the tenant's `defaultSize`) | optional | | `--api-insecure-http` | refused | allowed | | Metering | `--metering-sample-interval` required | 15 s default | | `--api-remote-install` | refused before deploying | allowed | Switch a development cluster to production by re-running `k7 install --api-mode production …` once every key has been re-scoped (`k7 list-api-keys` shows none flagged `unscoped_legacy`). ## Checklist 1. Migrate keys (`k7 api keys migrate` runs on install) and re-issue every `unscoped_legacy` key with the narrowest real role. 2. Give machines tenant keys with `--expires-days`; give humans OIDC. 3. Create tenants with `defaultSize` and quotas; dedicate nodes for tenants that must not share a k7d daemon. 4. Bind groups, not people; keep a break-glass `admin` key offline. 5. Ship the audit log (k7-api stdout, logger `k7.audit`). 6. Install with `--api-mode production`. Reference: [SECURITY.md](https://github.com/Katakate/k7/blob/main/SECURITY.md), [docs/HARDENING.md](https://github.com/Katakate/k7/blob/main/docs/HARDENING.md), API: [Identity](/k7/api/endpoints/identity), [Tenants](/k7/api/endpoints/tenants).