ComputerAgent

One framework to define, harness, run, and remember. Build computer-using agents that ship — without lock-in.

Open source · maintained by Shreyas Kapale @ Lyzr
scroll

// architecture

One framework.
Every layer, swappable.

Compose definition, harness, runtime, and memory. Each slot accepts the providers you already trust — or your own.

ComputerAgent framework diagram

// architecture

Harness Protocol. One system.

Every orthogonal axis of ComputerAgent, drawn out. Swipe through the full architecture.

High-Level ArchitectureHarness Protocol — HTTP + SSEGit URL Is The Agent IdentitySubstrate Runtime MatrixPermission / Governance FlowAudit & Telemetry PipelineLibrary vs Server ModeSessionStore ArchitectureEnd-to-End TopologyMultitenancy Architecture
01 / 10High-Level Architecture

// pillars

Eight slots. Infinite stacks.

01 / Full Observability

Every step, traced and replayable

OpenTelemetry-native traces, structured logs, token and cost metrics on every run. Inspect prompts, tool calls, and decisions side-by-side — debug agents like you debug code.

// Ten diagrams

From import to running agent.

No glue code. No vendor adapters. Just composition.

> import { ComputerAgent } from "computeragent";
> import { E2BSubstrate } from "@computeragent/runtime-e2b";
>
> const agent = new ComputerAgent({
    source:       { type: "git", url: "github.com/open-gitagent/trading-agent" },
    harness:      "claude-agent-sdk",
    runtime:      new E2BSubstrate({ apiKey: process.env.E2B_API_KEY! }),
    sessionStore: { kind: "mongo", options: { url: MONGO_URL, database: "agentos" } },
    envs:         { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY! },
  });
>
> await agent.chat("book my flight");
              

// Docs

Reference.

The complete API and protocol reference for computeragent and the @open-gitagent/* / @computeragent/* workspace packages.

Install

The umbrella package gives you ComputerAgent, runTask, LocalSubstrate, and all SDK types — enough for a complete agent in one import.

Umbrella SDK
$ npm install computeragent
E2B microVM runtime
$ npm install @computeragent/runtime-e2b
Linux bwrap runtime
$ npm install @computeragent/runtime-bwrap
Apple VZ runtime
$ npm install @computeragent/runtime-vzvm
Mongo session store
$ npm install @open-gitagent/session-store-mongo
SQLite session store
$ npm install @computeragent/session-store-sqlite
Mongo telemetry + registry
$ npm install @open-gitagent/agent-registry-mongo
Observability (OTel gen_ai.*)
$ npm install @computeragent/observability

Requires Node 22+ or Bun 1.1+ (the SDK targets modern async iterators + `using` semantics).

Quickstart

import { ComputerAgent, LocalSubstrate } from "computeragent";

const agent = new ComputerAgent({
  source:  { type: "git", url: "github.com/open-gitagent/general-agent" },
  harness: "claude-agent-sdk",
  runtime: new LocalSubstrate(),
  envs:    { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY! },
});

const result = await agent.chat("Summarize this repo in two sentences.");
console.log(result.messages.at(-1));
console.log(`Cost: $${result.usage.costUsd}`);

Same shape works against any substrate (E2BSubstrate, BwrapSubstrate, …), any engine, and any session store.

Core API — ComputerAgentOptions

Field
Req
Type
What it does
source
IdentitySource | string
Where to load the agent from. String form is shorthand for { type: 'git', url }.
harness
"claude-agent-sdk" | "gitagent" | "deepagents" | string
Engine to drive the loop. Custom engines registered on the harness server are accepted.
runtime
"local" | Substrate | string
Where the harness runs. Pass a Substrate to have the SDK call bootHarness() on first chat.
harnessUrl
string
Override the harness URL. Defaults to http://127.0.0.1:7700.
envs
Record<string, string>
Env vars forwarded to the engine subprocess (ANTHROPIC_API_KEY, GITHUB_TOKEN, …).
model
string
Override the agent's model.preferred. Engine-specific value.
temperature
number
Override model.constraints.temperature. Honored by engines that expose it.
baseUrl
string
Custom Anthropic-compatible endpoint. Useful for Helicone, OpenRouter, LiteLLM.
sessionId
string
Resume a specific session. With sessionStore, replays prior entries.
sessionStore
SessionStoreConfig
Conversation memory backend: memory, file, mongo, sqlite.
attachments
Attachment[]
Files materialized into the agent's workdir before the engine starts. Path-jailed.
options
Record<string, unknown>
Engine-specific options forwarded as body.options (permissionMode, maxTurns, …).
onToolCall
(ctx) => PermissionDecision
HITL callback for tool gating. Auto-allows if omitted.
policy
{ kind: 'srs', endpoint, apiKey, policyId, principalId }
Per-session policy decider config.
telemetry
AgentTelemetry
Telemetry hook (e.g. MongoTelemetry).
identityLoader
string
Identity loader. Default: 'gitagentprotocol'.
fetch
typeof fetch
Custom fetch impl (tests, proxies).
debug
boolean
Forces COMPUTERAGENT_LOG=debug and emits one client log per consumed event.

Methods

agent.chat(input)ChatHandle
The main turn. Dual interface — iterate events or await result.
agent.dispose()Promise<void>
Tear down the substrate, delete the server session. Implicit with `await using`.
agent.harnessUrl()Promise<string>
The resolved harness URL.
agent.fetchArtifact(path)Promise<Uint8Array | null>
Pull a file the agent wrote out of its workdir.
agent.listWorkdir(opts?)Promise<FsTreeEntry[]>
List the agent's workdir contents.

Disposal

// Explicit:
const agent = new ComputerAgent({...});
try { /* … */ } finally { await agent.dispose(); }

// Modern (auto-dispose at scope exit):
await using agent = new ComputerAgent({...});
// no try/finally needed

agent.chat() and ChatHandle

ChatHandle has a dual interface, inspired by client.messages.stream(...) in the Anthropic SDK.

  • for await (const ev of handle)Raw HarnessEvents as they arrive
  • await handle / handle.result()ChatResult — drained to completion
  • handle.getUsage()Snapshot of running UsageRollup at any time
  • handle.sessionId()Resolves to the real session id once POST /v1/sessions returns
  • handle.cancel()POST /v1/sessions/:id/cancel to abort the turn
  • handle.respondToPermission(callId, d)Manually answer a permission request

ChatResult

interface ChatResult {
  readonly sessionId: string;
  readonly messages: ReadonlyArray<unknown>;
  readonly ended: { kind: "ca_session_ended"; reason: string; errorMessage?: string };
  readonly usage: UsageRollup;
}

interface UsageRollup {
  readonly inputTokens: number;
  readonly outputTokens: number;
  readonly cacheCreationInputTokens: number;
  readonly cacheReadInputTokens: number;
  readonly costUsd: number | undefined;
}

Cost semantics: tokens always SUM across ca_usage_snapshot events. Cost depends on the engine's costSemantic: "cumulative" (claude-agent-sdk) takes the MAX; "delta" (gitclaw) SUMs.

ChatInput

type ChatInput =
  | string                              // user text
  | UserMessage                         // {role:"user", content:[...]}
  | UserMessage[]                       // multi-message turn
  | AsyncIterable<UserMessage>;         // streaming input

runTask — one-shot

import { runTask, LocalSubstrate } from "computeragent";

const result = await runTask({
  source:  { type: "git", url: "github.com/<org>/<repo>" },
  harness: "claude-agent-sdk",
  runtime: new LocalSubstrate(),
  envs:    { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY! },
  message: "Summarize the code in 3 bullets.",
});

IdentitySource — where the agent comes from

{ type: "git" } — clone from a remote repo

source: {
  type:   "git",
  url:    "github.com/<org>/<repo>",
  ref:    "v1.2.3",            // optional — branch/tag/SHA
  subdir: "agents/triage",     // optional — sub-path inside the repo
}

The git URL is the canonical agent identity. With MongoTelemetry, the same URL across machines deduplicates to the same agent_registry row. Authentication: set GITHUB_TOKEN in envs — the harness bakes it into the clone URL; your token never reaches the engine subprocess.

{ type: "local" } — use a directory already on disk

source: {
  type: "local",
  path: "/Users/me/my-agent",
}

{ type: "inline" } — pass the manifest in-memory

source: {
  type: "inline",
  manifest: { name: "hello", version: "0.1.0" },
  files: {
    "agent.yaml": [
      'spec_version: "0.1.0"',
      "name: hello",
      "version: 0.1.0",
      "model:",
      "  preferred: claude-haiku-4-5-20251001",
    ].join("\n"),
    "SOUL.md": "Respond in one short sentence.",
  },
}

String shorthand

source: "github.com/<org>/<repo>"   // → { type: "git", url: "github.com/<org>/<repo>" }

Substrates — where the agent runs

A Substrate hosts the harness server. The SDK calls substrate.bootHarness() on first chat(), gets back a URL, and proxies everything else through it.

LocalSubstrate<100ms (subprocess)
computeragent (bundled)

Dev, library-mode in your existing worker

BwrapSubstrate~50ms (user-namespaces)
@computeragent/runtime-bwrap

Isolation without containers on Linux

E2BSubstrate~2s (Firecracker microVM)
@computeragent/runtime-e2b

Strong isolation, untrusted code

VZSubstrate~3s (Tart-managed VZ)
@computeragent/runtime-vzvm

macOS-native VM, full OS, persistent disk

// E2B cloud microVM
import { E2BSubstrate } from "@computeragent/runtime-e2b";
runtime: new E2BSubstrate({
  apiKey:     process.env.E2B_API_KEY!,
  templateId: "computeragent-base",
}),

// Linux bubblewrap
import { BwrapSubstrate } from "@computeragent/runtime-bwrap";
runtime: new BwrapSubstrate({
  bind: ["/etc/ssl/certs:/etc/ssl/certs:ro"],
}),

Engines (harnesses) — the agent loop

claude-agent-sdk
@anthropic-ai/claude-agent-sdk v0.2.x
Default. Streaming + tool use + permission callback + sessions + budget.
gitagent
gitclaw CLI
Any OpenAI-compatible model (openai:<model>@<baseUrl>); GAP-native agents.
deepagents
LangChain deepagents
LangGraph-style agents needing LangChain ecosystem tool integrations.

harness is a string — open to extension. TypeScript narrows the built-ins for autocomplete; any string the running harness server has registered is accepted at runtime. Custom engines: implement EngineDriver and register via createHarnessServer({ engines }).

Session stores — conversation memory

Kind
Package
Backend
"memory"built-in
In-process map (default — non-persistent)
"file"built-in
JSONL on local disk
"mongo"@open-gitagent/session-store-mongo
MongoDB collection
"sqlite"@computeragent/session-store-sqlite
Local SQLite DB
// MongoDB — shared across pods, resume from any worker
sessionStore: {
  kind: "mongo",
  options: { url: process.env.MONGO_URL!, database: "agentos" },
},

// Resume a session
const agent = new ComputerAgent({
  ...opts,
  sessionId:    "sess_abc123",
  sessionStore: { kind: "mongo", options: { url, database } },
});
await agent.chat("Continue where we left off.");

Telemetry — AgentTelemetry + AuditSink

AgentTelemetry (SDK-side, library-mode)

import { ComputerAgent } from "computeragent";
import { MongoTelemetry } from "@open-gitagent/agent-registry-mongo";

const agent = new ComputerAgent({
  ...opts,
  telemetry: new MongoTelemetry({
    url:      process.env.MONGO_URL!,
    database: "agentos",
    agent:    { name: "triage", source: opts.source, harness: opts.harness },
  }),
});

Hooks fired (all optional, fire-and-forget — exceptions never propagate):

interface AgentTelemetry {
  onAgentConstructed?(info: AgentConstructedInfo): void | Promise<void>;
  onChatStart?(info: ChatStartInfo): void | Promise<unknown>;
  onChatEnd?(info: ChatEndInfo, context?: unknown): void | Promise<void>;
  onClose?(): void | Promise<void>;
}

AuditSink (harness-side, server-mode)

import { configureOtel, OtelAuditSink } from "@computeragent/observability";

configureOtel({
  serviceName: "computeragent",
  exporter:    "otlp-http",
  endpoint:    process.env.OTEL_EXPORTER_OTLP_ENDPOINT!,
});

const auditSink = new OtelAuditSink();
const server    = new ComputerAgentServer({ /* ..., */ auditSink });

Spans use the OpenTelemetry gen_ai.* semantic conventions — any OTel-compatible APM (Honeycomb, Datadog, Grafana, ClickHouse) renders them natively.

Permissions / Human-in-the-loop

onToolCall callback (per-agent)

const agent = new ComputerAgent({
  ...opts,
  onToolCall: async ({ callId, toolName, input, risk }) => {
    if (toolName === "Bash" && (input as { command?: string }).command?.startsWith("rm")) {
      return { decision: "deny", reason: "rm denied by policy" };
    }
    return { decision: "allow" };
  },
});

PermissionDecision shapes

type PermissionDecision =
  | { decision: "allow" }
  | { decision: "deny";   reason?: string }
  | { decision: "modify"; input: Record<string, unknown> };

TTY approval (CLI scripts)

import { ttyApproval } from "computeragent";

const agent = new ComputerAgent({
  ...opts,
  onToolCall: ttyApproval(),   // prompts at the terminal for each tool call
});

Manual iteration

const handle = agent.chat("...");
for await (const ev of handle) {
  if (ev.kind === "ca_permission_request") {
    await handle.respondToPermission(ev.callId, { decision: "allow" });
  }
}

Policy — Cedar + OPA guardrails

const agent = new ComputerAgent({
  ...opts,
  policy: {
    kind:        "srs",
    endpoint:    "https://your-policy-service.example.com",
    apiKey:      process.env.SRS_API_KEY!,
    policyId:    "policy_abc123",
    principalId: "user:alice",
  },
});

The harness fetches the policy once (cached) and evaluates every tool call against the cedar_guardrail + opa_guardrail subsections. Decisions are emitted as ca_permission_decision events for audit. Fail-closed: on 5xx / timeout, the harness defaults to deny.

Configuration reference

Environment variables (read by the engine subprocess)

  • ANTHROPIC_API_KEYAnthropic direct path
  • ANTHROPIC_BASE_URLOverride Anthropic endpoint (Helicone, OpenRouter, proxies)
  • CLAUDE_CODE_USE_BEDROCKRoute through AWS Bedrock instead of Anthropic direct
  • AWS_REGION / AWS_DEFAULT_REGIONBedrock region
  • AWS_BEDROCK_MODEL_IDe.g. us.anthropic.claude-haiku-4-5-20251001-v1:0
  • AWS_ROLE_ARN / AWS_WEB_IDENTITY_TOKEN_FILEIRSA-injected on EKS
  • AWS_PROFILE / AWS_SHARED_CREDENTIALS_FILE / AWS_CONFIG_FILEAlternative AWS auth paths
  • GITHUB_TOKENFor cloning private agent source repos
  • OTEL_EXPORTER_OTLP_ENDPOINTOTLP collector URL (turns on OtelAuditSink)
  • OTEL_SERVICE_NAMEService name on emitted spans (default: computeragent)
  • COMPUTERAGENT_LOGdebug / info / warn / error / silent

Model overrides — resolution order (highest wins)

  1. options.model constructor arg
  2. model: constructor arg
  3. agent.yaml's model.preferred
  4. Engine default (claude-agent-sdk → claude-haiku-4-5-20251001)

HTTP wire protocol

The harness server exposes a small REST + SSE surface — curl can drive every endpoint. Schemas are Zod-validated.

REST

GET/health
Liveness
POST/v1/sessions
Start a session — returns SSE event stream
GET/v1/sessions/:id
Session status
POST/v1/sessions/:id/messages
Push a user turn (multi-turn flow)
POST/v1/sessions/:id/permission/:callId
Respond to a permission request
POST/v1/sessions/:id/cancel
Abort the in-flight turn
DELETE/v1/sessions/:id
Tear down + free workdir
GET/v1/sessions/:id/fs/tree?depth=N
List the agent's workdir
GET/v1/sessions/:id/fs/file?path=...
Fetch a file the agent wrote

SSE event stream

type HarnessEvent =
  | { kind: "ca_session_started";    sessionId; engine; identity; capabilities }
  | { kind: "sdk_message";           sessionId; payload }
  | { kind: "ca_permission_request"; sessionId; callId; toolName; input; risk? }
  | { kind: "ca_permission_decision";sessionId; callId; decision; reason? }
  | { kind: "ca_turn_started";       sessionId; userTextLen? }
  | { kind: "ca_usage_snapshot";     sessionId;
                                     inputTokens?; outputTokens?;
                                     cacheCreationInputTokens?; cacheReadInputTokens?;
                                     costUsd?; costSemantic?: "cumulative" | "delta" }
  | { kind: "ca_session_ended";      sessionId; reason; errorMessage? };

Every event has a monotonic id. Reconnect with Last-Event-ID: <last-id> and the harness replays from the per-session ring buffer (default: last 1,000 events / 5 minutes).

One-line curl

curl -N -X POST http://127.0.0.1:7700/v1/sessions \
  -H 'content-type: application/json' \
  -d '{
    "source": "github.com/open-gitagent/general-agent",
    "harness": "claude-agent-sdk",
    "envs": { "ANTHROPIC_API_KEY": "sk-ant-…" },
    "message": "Reply: PING"
  }'

CLI

npx computeragent run \
  --source github.com/open-gitagent/general-agent \
  --harness claude-agent-sdk \
  --message "Summarize the README in 3 bullets"

Common flags

  • --sourcesource (git URL or local path)
  • --harnessharness
  • --modelmodel
  • --temperaturetemperature
  • --runtimeruntime (local / e2b / bwrap / vz)
  • --session-idsessionId
  • --session-storesessionStore.kind (memory / file / mongo / sqlite)
  • --debugdebug: true
  • --messageone-shot agent.chat(input) and exit

Errors

HarnessProtocolError
Server returned a non-2xx, malformed SSE, or unexpected message order
UnknownEngineError
harness: value isn't registered on the running server
UnknownLoaderError
identityLoader: value isn't registered
UnknownStoreError
sessionStore.kind isn't registered
import { HarnessProtocolError, UnknownEngineError } from "computeragent";

try {
  await agent.chat("…");
} catch (e) {
  if (e instanceof HarnessProtocolError) { /* retry, swap harness url, … */ }
  if (e instanceof UnknownEngineError)   { /* "did you mean…" */ }
  throw e;
}

AuditSink / AgentTelemetry errors never propagate. They're caught, logged at debug level, and swallowed. Telemetry must never break an agent run.

Companion packages

@open-gitagent/protocol
Wire-protocol schemas (Zod) — HarnessEvent, IdentitySource, REST shapes
@open-gitagent/sdk
The user-facing SDK (ComputerAgent, ChatHandle, runTask)
@open-gitagent/runtime-local
Default LocalSubstrate
@computeragent/runtime-bwrap
Linux user-namespace sandbox
@computeragent/runtime-e2b
E2B microVM substrate
@computeragent/runtime-vzvm
Apple VZ.framework substrate (Tart)
@computeragent/harness-server
The Hono-based server that hosts engines + substrates
@computeragent/engine-claude-agent-sdk
Engine wrapping @anthropic-ai/claude-agent-sdk
@computeragent/engine-gitagent
Engine wrapping gitclaw (any OpenAI-compatible upstream)
@computeragent/engine-deepagents
Engine wrapping LangChain deepagents
@open-gitagent/agent-registry-mongo
MongoTelemetry + Mongo-backed AgentRegistry
@open-gitagent/session-store-mongo
Mongo SessionStore
@computeragent/session-store-sqlite
SQLite SessionStore
@computeragent/observability
configureOtel() + OtelAuditSink (gen_ai.* spans)
@computeragent/observability-api
Express read API over ClickHouse
@computeragent/llm-proxy-openai
Anthropic Messages ↔ OpenAI Chat Completions translator
@computeragent/identity-gitagentprotocol
Default IdentityLoader — clones GAP repos
@computeragent/testing
Table-driven conformance suite
@computeragent/cli
CLI
computeragent
Umbrella entry point — re-exports SDK + LocalSubstrate
create-computeragent
npx create-computeragent my-agent scaffolder

Versioning

Public packages follow semver and are independently published. The umbrella computeragent is currently at 0.2.x — minor versions may include breaking changes until 1.0. Pin a major if stability matters.

Wire-protocol changes are versioned separately: events carry a protocolVersion field; the harness server rejects requests from incompatible client versions with HarnessProtocolError rather than malforming.