dotsfeed
← Updates

Model

Building a computer-use agent on the API? The new SDK toolsets run the loop, so you only write the driver

Verified· Oct 16, 2026Published Oct 16, 2026

OpenAI's Python and Node SDKs now handle the computer-use tool loop for you, with a screenshot-based ComputerToolset and a page-text BrowserToolset. Here is how to pick one and keep the agent fenced in.

Benchmarks

Benchmarks not published by the source yet.

What changed

The latest OpenAI Python (3.31.0) and Node (7.33.0) SDKs now include helpers that run the Responses tool loop for computer-use agents, so you no longer have to hand-write the request, act, screenshot, repeat cycle. Kevin Whinnery from the OpenAI SDK team walked through it on X on October 16, and @OpenAIDevs reposted it. There are two ways in. ComputerToolset works from screenshots plus mouse and keyboard actions. BrowserToolset hands the model page text, controls, and tabs as ordinary function tools. Either way, you write (or import) a small driver that connects those calls to your own browser or desktop.

Who is affected

Developers building agents that fill forms, test user flows, or operate web apps through the OpenAI API, especially anyone who has been maintaining their own computer-use loop. You need Python 3.11 or later or Node 22.6 or later, and a model your project can use with the computer tool. This is the API and SDK side. It doesn't change computer use inside ChatGPT, Codex, or dots.

What to do now

  • Pick the path that matches what the model needs to see. Use BrowserToolset when page text and controls are enough. It is easier to fence in. Use ComputerToolset when the model has to look at the actual screen.
  • Expose as little as possible. With BrowserToolset, list only the operations the task needs in members. With ComputerToolset, refuse every action type you don't support.
  • Approve at the point of action. Use the confirm or before-action callback to allow only the specific targets and values you expect, and re-check the page right before the click, because the screen can change after approval.
  • Cap the run with a step limit, and check the real result on the page before you report success instead of trusting the model's final answer.
  • Before writing your own driver, run the Fieldwork restaurant booking demo in the SDK repos. It has a rehearsal mode with no API calls, and it asks before submitting. If you'd rather not host the browser yourself, OpenAI's docs list ready guides for Browserbase, Browser Use, Daytona, and E2B.

What is not confirmed

We read the computer use guide and the SDK example READMEs, but we haven't built an agent with the new helpers or measured how reliable they are. Each step sends screenshots or page text to OpenAI and costs normal API tokens, and screenshots count as image input. Setting store to false doesn't remove every kind of data retention, so check OpenAI's data controls first. Hosted browser providers may charge separately and have their own data policies.

Sources

Sources

Get updates like this every morning

  1. ① Email
  2. ② Card on Stripe
  3. ③ 7 days free

Then $2/month · cancel anytime in one click

Building a computer-use agent on the API? The new SDK toolsets run the loop, so you only write the driver · Dotsfeed