Fire Your Fetch Tool — Let your Agent Use a Browser

“Fetching a URL” sounds trivial, but inside an agent it can get messy. Part of the confusion comes from documentation that blurs what each web tool actually does. Remember when our dear agents didn’t even bring anything with them?

With Codex, the situation is still less clear than with Claude Code: Codex brings a web-search tool, that has been introduced several weeks ago. The documentation indicates it can retrieve full pages (when using a reasoning model, which Codex CLI clearly does)—yet the public docs do not establish that it executes client-side JavaScript to produce a post-JS DOM. Maybe it does, maybe not. I’m not sure, to be honest. But: that distinction matters on SPAs where content appears only after hydration.

By contrast, Claude Code cleanly separates WebSearch (find sources) and WebFetch (retrieve a specific page); not running arbitrary site JavaScript—so JS-heavy SPAs still come back thin unless you use a real browser.

The web fetch tool currently does not support web sites dynamically rendered via Javascript.
— Anthropic documentation about the ”web fetch tool”, October 24th, 2025

Because of that gap, a small local fallback helps: render once with a headless browser, then hand the model normalized text. A tiny wrapper script can keep the path noise (and token overhead) out of your context:

# ~/bin/chrome
#!/usr/bin/env bash
exec "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" "$@"

From there, this will cover most needs:

# 1) Fully rendered DOM (post-JS) as HTML
chrome --headless --disable-gpu --dump-dom "https://example.com"

# 2) Rendered DOM → Markdown (model-friendly)
chrome --headless --dump-dom "https://example.com" | uvx markitdown

# 3) Rendered DOM with stripped tags (token-budgeted)
chrome --headless --dump-dom "https://example.com" | uvx strip-tags

This pattern delegates networking, JS execution, and DOM construction to the browser engine once, and lets the agent get a stable artifact for its context. Keep the built-in search tool for discovery purposes, but use real browser for fetching page content.

To make this actionable for agents, here’s an excerpt of my current AGENTS.md (using it with Codex). Should work for CLAUDE.md as well, but haven’t given it a try. Should require more shouting, though.

## Agent’s toolbox (complementary to MCP server tools)
- When the user wants you to read websites, fetch their content using 
  `chrome --headless --dump-dom | uvx markitdown` 
  (if `chrome` is available). 
  The call will return the website’s content converted to markdown. 
  If markdown is not practicable, leave out the piping. 
  As a generic fallback use curl or wget. 
  Don’t use the search tool for fetching!

The best tools an agent could use are often already on your system. In many cases you don’t need an MCP server at all: you save configuration time and token budget—since every extra MCP tool pulls its tool description into context. Models generally know how to use well-known CLI tools; if not, they can invoke them and read the usage output.