Computer use with Claude
The adapter binds the computer_toolset_20260801 tool shape to the desktop API. Not installable yet; available when the beta opens. Model credentials stay in the local process, separate from the Dory API key and outside the computer. The sample's fixed model name is only an example, not a promise that your account can access it.
Complete loop
read -rs CE_API_KEY && export CE_API_KEY
read -rs ANTHROPIC_API_KEY && export ANTHROPIC_API_KEY
import sys
import anthropic
from computeengines import Client
from computeengines.adapters.anthropic import ComputerToolset
ce = Client() # CE_API_KEY
claude = anthropic.Anthropic() # your own Anthropic key; CE never sees it
computer = ce.computers.create(size="ac-4", name="computer-use-demo")
try:
computer.desktop.set_display(1280, 800) # POST /desktop/display
toolset = ComputerToolset(computer) # definition: {"type":"computer_toolset_20260801"}
messages = [{"role": "user", "content":
"Open Chromium, go to https://example.com and tell me the page's main heading."}]
for _ in range(50): # hard stop for a runaway loop
response = claude.messages.create(
model="claude-opus-5-5",
max_tokens=16000,
tools=[toolset.definition],
messages=messages,
)
messages.append({"role": "assistant", "content": response.content})
if response.stop_reason == "refusal":
break
calls = [block for block in response.content if block.type == "tool_use"]
if not calls:
break
results = []
halted = False
for call in calls:
if halted:
result = {
"type": "tool_result", "tool_use_id": call.id,
"toolset_name": call.toolset_name, "is_error": True,
"content": "Not executed: an earlier computer action in this turn failed.",
}
else:
try:
if call.toolset_name != "computer":
raise ValueError("Unsupported toolset")
result = toolset.run(call)
except Exception:
result = {
"type": "tool_result", "tool_use_id": call.id,
"toolset_name": call.toolset_name, "is_error": True,
"content": "Computer action failed; inspect the local diagnostic log before retrying.",
}
halted = bool(result.get("is_error", False))
results.append(result)
messages.append({"role": "user", "content": results})
print(next((b.text for b in response.content if b.type == "text"), ""))
finally:
try:
computer.pause()
except Exception:
print(f"Cleanup failed for {computer.id}; inspect it in the CE console because it may still be running and billed.", file=sys.stderr)
raise
Results and stopping
The adapter echoes toolset_name for each result and handles multiple calls in a turn. Check stop_reason before selecting content, retain earlier screenshots in message history, and stop further actions in a turn after an error. Do not strip earlier image blocks from a history containing later thinking blocks. Provider-side history-clearing parameters require separate current provider verification; none are invented here.
The loop has a 50-turn bound. Creation happens before the cleanup block: an OperationFailed or WaitTimeout carries a computer_id for recovery even if no Computer was returned. Cleanup is successful only when the bounded pause waiter reaches paused. A cleanup error must remain visible because the computer may still be running and billed.
Coordinates and control
The adapter rescales screenshot coordinates to the display. Zoom images use display resolution. Finish a held left-button drag within one turn; the service releases held buttons after ten seconds. hold_key renews held keys and may repeat a key press. An exclusive human holder produces an error result rather than an action. Unknown keys fail rather than being silently substituted. See Desktop for the coordinate and input rules.
Complete TypeScript loop
This program imports the Anthropic SDK and the Dory adapter. It uses a running computer reserved for the task through CE_COMPUTER_ID and pauses it on every exit after lookup. It replies to every tool_use in order; after a failed action, later calls receive the exact skipped-action result. Provider and API keys stay in the local environment, never in the computer.
import Anthropic from "@anthropic-ai/sdk";
import { Client } from "@computeengines/agent-computers";
import { ComputerToolset } from "@computeengines/agent-computers/adapters/anthropic";
/** Convert only the adapter's actual text/image blocks to provider types.
* @param {unknown} block
* @returns {Anthropic.TextBlockParam | Anthropic.ImageBlockParam}
*/
function providerContent(block) {
if (block !== null && typeof block === "object" && "type" in block) {
if (block.type === "text" && "text" in block && typeof block.text === "string") {
return { type: "text", text: block.text };
}
if (block.type === "image" && "source" in block && block.source !== null && typeof block.source === "object") {
const source = block.source;
if ("type" in source && source.type === "base64" && "data" in source && typeof source.data === "string"
&& "media_type" in source && (source.media_type === "image/png" || source.media_type === "image/jpeg")) {
return { type: "image", source: { type: "base64", media_type: source.media_type, data: source.data } };
}
}
}
throw new Error("Unsupported adapter result representation");
}
const id = process.env.CE_COMPUTER_ID;
if (!id) throw new Error("Set CE_COMPUTER_ID to a running computer reserved for this task");
const ce = new Client();
try {
const claude = new Anthropic(); // reads ANTHROPIC_API_KEY.
const computer = await ce.computers.get(id);
try {
await computer.desktop.setDisplay({ width: 1280, height: 800 });
const toolset = new ComputerToolset(computer);
/** @type {Anthropic.MessageParam[]} */
const messages = [{ role: "user", content: "Take a screenshot and describe the current desktop." }];
for (let turn = 0; turn < 50; turn++) {
const response = await claude.messages.create({
model: "claude-opus-5-5", max_tokens: 16000,
tools: [toolset.definition], messages,
});
messages.push({ role: "assistant", content: response.content });
if (response.stop_reason === "refusal") break;
const calls = response.content.filter(block => block.type === "tool_use");
if (calls.length === 0) break;
/** @type {Anthropic.ToolResultBlockParam[]} */
const results = [];
let halted = false;
for (const call of calls) {
const toolset_name = "toolset_name" in call ? call.toolset_name : null;
/** @type {Anthropic.ToolResultBlockParam} */
let result;
if (halted) {
result = { type: "tool_result", tool_use_id: call.id, toolset_name, is_error: true,
content: "Not executed: an earlier computer action in this turn failed." };
} else {
try {
if (toolset_name !== "computer") throw new Error("Unsupported toolset");
const actual = await toolset.run({ id: call.id, name: call.name, input: call.input, toolset_name });
result = { type: "tool_result", tool_use_id: actual.tool_use_id, toolset_name: actual.toolset_name,
content: actual.content.map(providerContent), ...(actual.is_error ? { is_error: true } : {}) };
} catch {
result = { type: "tool_result", tool_use_id: call.id, toolset_name, is_error: true,
content: "Computer action failed; inspect the local diagnostic log before retrying." };
}
halted = result.is_error === true;
}
results.push(result); // Exactly one reply for every tool_use, including skipped calls.
}
messages.push({ role: "user", content: results });
}
console.log("Bounded model loop ended");
} finally {
// A failed pause propagates: do not claim that billing stopped without readback.
await computer.pause();
}
} finally { ce.close(); }
Dory is in development; these pages describe the planned service and may change. Questions: support@computeengines.com.