Skip to content
Hakiri

ADR-0015 — Inference zones with Incognito mode, enforced by policy + capability token

Context

Section titled “Context”

Hakiri’s data plane is local-first; the inference plane that consumes Hakiri’s MCP responses (the LLM the agent client talks to) is not, and v0 has no opinion about where it runs. Users today must mentally track which model endpoint is hot before they ask a sensitive question. Two failure modes follow:

  1. Accidental exfiltration. A user with a Stripe / Slack / EHR-backed Hakiri context loaded asks a question; their MCP client is configured against the Anthropic public API; private data is sent to a multi-tenant cloud inference endpoint they didn’t intend.
  2. No portable answer to “where can this data be processed.” Compliance buyers (PRD strategic audience) need a written, enforceable answer; the only thing they can quote today is the data-plane residency story from 11-compliance.md, which is silent on inference.

The product ask is two-part: a seamless flip between local/on-prem and cloud inference, plus an Incognito mode guarantee that private-tagged context never leaves a local/on-prem boundary regardless of which model the user has selected in their client.

Three architectural options were considered:

A. Policy-only. Hakiri tags data with a privacy class, capability tokens carry an attested inference zone, and MCP responses refuse to return cloud-disallowed data unless the calling client’s zone is in the allowed set. Hakiri does not host inference or embed an LLM gateway.

B. Policy + optional thin LLM gateway. Same policy model, plus an optional hakiri-llm-gateway component that routes inference calls to local or cloud endpoints and attests the zone to the runtime.

C. Policy + hard host attestation. Same policy model, but require SPIFFE-flavored host attestation before serving private-tagged data. Pushes v0’s self-asserted model harder; would delay landing.

Decision

Section titled “Decision”

Option A: policy-only enforcement. Inference placement is governed by a typed zone label on the subject tuple and a typed inference_zone_allowed policy on tables and columns. Incognito mode is a client-side switch that pins the subject’s zone to local:device / on-prem:* and surfaces honestly into the audit trail. Hakiri does not ship an LLM gateway, does not import any model-vendor SDK, and does not verify zone claims cryptographically in v0 — the zone is self-asserted on the same honesty model as agent:// and host:// per 09-access-control.md § Attestation. Mechanics live in 15-inference-placement.md.

Consequences

Section titled “Consequences”

Positive

  • Anti-pillar preserved. “Not an LLM SDK or prompt framework” stays intact; no model-vendor SDK enters any Hakiri crate. Pillar 3’s MCP-only commitment is the boundary; this ADR strengthens it instead of bending it.
  • Composable with existing access control. The subject tuple gains one field; the policy validator gains one check. Both compose with RLS / CLS / write-time redaction without rewiring the enforcement layers.
  • Provider-agnostic by construction. The policy is “which zone,” not “which vendor.” Anthropic, OpenAI, Bedrock, Vertex, Ollama, vLLM, llama.cpp all collapse to the same four zone categories. Switching providers is a client concern; the policy keeps holding.
  • Audit-shaped. Every MCP read records the asserted zone. Compliance teams answer “did anyone ever read this row under public-cloud:*” in one OTel query, today.
  • Cheap to ship. Manifest validator + one subject field + one filter pass over result rows. Lands in the M2 access-control window alongside capability tokens.

Negative

  • Self-asserted in v0. A compromised client claims whatever it likes — Incognito is “defense against accident,” not “defense against a compromised laptop.” The audit trail labels every zone with .asserted = true to make this honest; cryptographic attestation is M3+.
  • No “seamless flip” without a client that respects the zone. The desktop app exposes the toggle; an arbitrary third-party MCP client must implement the zone-assertion surface or it cannot enter Incognito. The renderer’s localhost-endpoint health check is the only UX guardrail in v0.
  • One more knob in hakiri.toml. Operators have one more axis to think about. Mitigated by defaulting unlabeled tables to ["*"] so the cost is opt-in.

Neutral

  • The optional hakiri-llm-gateway (Option B’s extension) is not foreclosed. If user demand is overwhelming, it ships as a separate crate / repo — never in the data plane — and reuses the same zone-label vocabulary. Tracked as an open question in 15-inference-placement.md.

Alternatives considered

Section titled “Alternatives considered”

Option B: policy + optional thin LLM gateway. Same policy model plus a sidecar hakiri-llm-gateway exposing an OpenAI-compatible API that routes by declared zone + task tag. Rejected for v0 because: (a) it leaks model-vendor concerns into the Hakiri repo, blurring Anti-pillar “Not an LLM SDK”; (b) the seamless-flip UX is achievable client-side (the Electron app already feature-detects local endpoints); (c) the gateway can ship later as a separate component without revisiting the data-plane policy. Kept on the table as an open question.

Option C: policy + hard host attestation. Same policy model, but require SPIFFE-flavored host attestation before serving private-tagged data. Rejected for v0 because: (a) the host attestation work is already deferred to M3+ per 09-access-control.md § Attestation; (b) gating Incognito on the harder attestation story would either delay v0 or force a half-built attestation surface; (c) the OTel labeling (.asserted = true) is honest enough for v0’s audit posture. Upgrade path is built in — when SPIFFE lands, inference_zone.asserted flips to .verified without policy rewrites.

Do nothing. Lean on replicate = false and proxy-mode replicas to keep sensitive data off laptops, and rely on documentation to warn users about cloud LLM exposure. Rejected because: (a) it doesn’t address the in-region case (a Worker reading proxy-served PHI and forwarding it to Anthropic’s public API would pass every existing check); (b) the audit trail has nothing to say about which zone consumed each read; (c) compliance buyers need a written policy axis, not documentation.

References

Section titled “References”