System Map
Heimdall sits between API clients and local runtimes. The client sees one API-like service; Heimdall handles auth, routing, backend translation, and response normalization.
/v1/chat/completions enters the chat adapter. /v1/embeddings is forwarded to the embeddings backend.claude -p. Embeddings bypass Claude Code and are generated by the local FastEmbed service.What It Is
Heimdall is a compatibility layer, not a model host. A shim is a small adapter that makes one interface look like another. A router chooses which backend should handle a request based on the URL path.
In practice, Heimdall makes selected local capabilities look like one authenticated API surface. The backend selected for the route still does the actual work.
Chat
The chat endpoint accepts a familiar /v1/chat/completions request, then spawns claude -p. To the caller it looks like an HTTP model endpoint; on the server it is a local CLI job.
Embeddings
Embeddings are numeric vectors used for search, matching, and similarity. Heimdall exposes them through a familiar /v1/embeddings request shape, while FastEmbed generates the vectors locally.
Chat Flow
Chat is the current local chat adapter. Heimdall translates a chat-completion request into a local CLI invocation and maps the result back into the response shape clients expect.
messages, stream, response_format, and the optional heimdall controls.--system-prompt; user messages are joined and written to stdin.claude -p --model ... --max-turns ..., watches for timeout or disconnect, and collects stdout.Fields That Matter Today
| Field | Runtime effect |
|---|---|
messages | Used. System content becomes --system-prompt; user messages are joined into stdin. |
stream | Used. Heimdall sends server-sent-event heartbeats and a final chunk, not live token-by-token streaming. |
response_format.type = json_object | Used. Heimdall adds JSON-only instructions and extracts a JSON object from the response. |
response_format.type = json_schema | Used. Heimdall passes a JSON schema to Claude Code and returns the structured output as message content. |
heimdall.max_turns | Used, but clamped by service policy. |
heimdall.timeout_ms | Used, but cannot exceed the service maximum. |
model, max_tokens, temperature | Accepted for client compatibility, but not fully wired through to Claude Code behavior yet. |
POST /v1/chat/completions
Authorization: Bearer <token>
Content-Type: application/json
{
"model": "sonnet",
"stream": false,
"messages": [
{ "role": "system", "content": "Return concise JSON." },
{ "role": "user", "content": "Extract the key facts." }
],
"response_format": { "type": "json_object" },
"heimdall": { "max_turns": 3, "timeout_ms": 180000 }
}Embeddings Flow
The embeddings route uses the same Heimdall front door but a different backend. Heimdall authenticates the request, then forwards the embedding body to a private FastAPI service that is not part of the public client surface.
- Backend: FastAPI via Uvicorn.
- Library: FastEmbed.
- Model:
BAAI/bge-small-en-v1.5. - Dimension:
384. - Default input limits: up to
32inputs and16000characters per input.
POST /v1/embeddings
Authorization: Bearer <token>
Content-Type: application/json
{
"model": "BAAI/bge-small-en-v1.5",
"input": ["text to embed"]
}Security Model
Heimdall's own security boundary starts at the Node front door. Every documented client route requires a bearer token before any backend is reached. Network exposure is handled outside the application; Heimdall's private backend topology is intentionally not published here.
Secrets
The Heimdall bearer token is loaded from a host-local env file that is not tracked in source. This page intentionally does not publish token values, credential paths, or private identifiers.
Systemd containment
Both services run under systemd with restricted write paths, no ambient capabilities, restart-on-failure behavior, and journald logging. Chat and embeddings run as separate service users.
Operations
The private Heimdall checkout tracks the Node proxy, the embeddings backend, and the systemd unit templates that keep both processes running. The public explanation stops at component roles and client contract; exact paths, ports, users, and env files stay in private operator notes.
| Component | Role | Public detail |
|---|---|---|
| Node front door | Auth, routing, chat adapter, metrics, embeddings proxy. | Call through the documented API routes only. |
| Embeddings worker | FastAPI embeddings backend with local model cache. | Private backend behind the authenticated front door. |
Operational status, logs, metrics, and restart commands are deliberately left out of the public client contract. They should live in private runbooks where the real service names and host-specific values can be used safely.
Client Contract
- Send
Authorization: Bearer <token>on every request. - Use
POST /v1/chat/completionsfor the current local chat adapter. - Use
POST /v1/embeddingsfor local embedding vectors. - Use
response_formatfor JSON-only or JSON-schema output. - Use
heimdall.max_turnsandheimdall.timeout_msonly when a call needs tighter runtime controls.
Callers should treat Heimdall as a narrow compatibility contract around selected local backends, not as a full vendor API implementation.
Limits And Caveats
- The current chat path is backed by a local CLI process, not a production vendor Messages API deployment.
- Latency can be high and variable because each chat request starts a CLI process and may run for minutes.
- Concurrency is intentionally low. The service queues requests and returns overload errors when the queue is full or times out.
- Multi-turn chat is flattened: assistant messages are detected as a warning, but only system and user content are forwarded to
claude -p. - Streaming is heartbeat plus final response, not true token-by-token streaming.