# Agents deserve the whole web. (https://staging.expand.ai/docs) Set up Expand inside your agent, or run one Fetch from your terminal and inspect the output yourself.
## What your agent gets [#what-your-agent-gets] Every Fetch returns one working document plus the evidence behind it. Main Markdown is the readable default. State JSON ships alongside it when it fits, and stays addressable by reference when a page carries too much. Appendix holds secondary regions, and every claim traces back to a snapshot. ## Start here [#start-here] Run one Fetch and inspect the output. Choose agent, CLI, SDK, API, or Playground. Understand what Expand preserves beyond plain Markdown. ## Developer surfaces [#developer-surfaces] Fetch is the product. The CLI, SDKs, HTTP API, MCP, and the Playground are surfaces that bring Fetch into the place where your work happens. See [How Fetch Works](/docs/fetch/how-fetch-works) for the model behind all of them. * [Agents and Expand Skill](/docs/agent-quickstarts) * [CLI](/docs/reference/cli-commands) * [TypeScript SDK](/docs/reference/typescript-sdk) * [Python SDK](/docs/reference/python-sdk) * [Direct API](/api-reference) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [Playground & Replay](/docs/fetch/playground-and-replay) ## Machine-readable docs [#machine-readable-docs] Pointing an agent or IDE tool at these docs? Start it with [start.md](/start.md) or the [llms.txt](/llms.txt) index, pull the full bundle from [llms-full.txt](/docs/llms-full.txt), or read any page [as Markdown](/docs/machine-readable-docs/docs-as-markdown). `llms.txt` is an index for agents you point at it directly, not a file that major web crawlers fetch on their own. --- # FAQ (https://staging.expand.ai/docs/account-and-billing/faq) Short answers to common questions about Fetch, agents, Highlights, billing, tiers, and reference docs. Each answer is two to five sentences and links to the canonical page when you need detail. Jump to a group: Fetch, usage surfaces, and which path to pick. Markdown, State JSON, progressive disclosure, Include Options. Searching captures and building evidence links. Hooks, the Expand Skill, MCP, and auth. Credits, usage, browser concurrency, provisioned slots. API Reference, schemas, and machine-readable routes. ## Using Expand [#using-expand] ### What is Fetch? [#what-is-fetch] Fetch is Expand's core product: it renders a live page and returns agent-ready Markdown, State JSON, evidence, and replayable artifacts. Start with [Fetch Overview](/docs/fetch). ### Are CLI, MCP, SDKs, and the API separate products? [#are-cli-mcp-sdks-and-the-api-separate-products] No. They are ways to use Fetch. Use the CLI for terminal workflows, SDKs or the API for app code, MCP for agent tools, and hooks or skills for agent harness integrations. See [Ways to Use Expand](/docs/get-started/ways-to-use-expand). ### Which path should I use? [#which-path-should-i-use] | Need | Path | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | Try one URL from your terminal | [Quickstart](/docs/get-started/quickstart) / CLI | | Add Fetch to an app | [TypeScript SDK](/docs/reference/typescript-sdk), [Python SDK](/docs/reference/python-sdk), or [API Reference](/api-reference) | | Give a coding agent web context | [Agent Quickstarts](/docs/agent-quickstarts) | | Inspect evidence visually | [Playground & Replay](/docs/fetch/playground-and-replay) | ### When should I use the CLI instead of an SDK? [#when-should-i-use-the-cli-instead-of-an-sdk] Use the CLI for local inspection, scripts, and agent shell workflows. Use an SDK or the API when Fetch is part of your application code. See [CLI Commands](/docs/reference/cli-commands) and the [TypeScript](/docs/reference/typescript-sdk) and [Python](/docs/reference/python-sdk) SDKs. ## Fetch output [#fetch-output] ### Why does `/v1/fetch` return Markdown directly? [#why-does-v1fetch-return-markdown-directly] `/v1/fetch` is optimized for the common path: get the main readable document quickly. Use `/v1/fetch/json` when you need metadata, State JSON, snapshot IDs, or structured response fields. See [Output Model](/docs/fetch/output-model). ### When should I use JSON Mode? [#when-should-i-use-json-mode] Use JSON Mode when application code needs Main Markdown plus State JSON, metadata, links, Highlights, or snapshot references in one object response. See [Output Model](/docs/fetch/output-model) and the [TypeScript](/docs/reference/typescript-sdk) and [Python](/docs/reference/python-sdk) SDKs. ### What are Main Markdown and State JSON? [#what-are-main-markdown-and-state-json] Main Markdown is the primary readable page content. State JSON is structured page and network state that can answer questions Markdown alone may miss. See [How Fetch Works](/docs/fetch/how-fetch-works). ### Is State JSON always included? [#is-state-json-always-included] State JSON is part of the core output model for JSON Mode, but large or secondary state can be referenced through progressive disclosure instead of being forced into the main response. See [Output Model](/docs/fetch/output-model). ### What is progressive disclosure? [#what-is-progressive-disclosure] Progressive disclosure means Fetch gives the agent the pragmatic default first, while preserving references to deeper context when the task needs it. See [How Fetch Works](/docs/fetch/how-fetch-works). ### Are Appendix links, `snapshotId`, or `moreTokens` missing data? [#are-appendix-links-snapshotid-or-moretokens-missing-data] Usually no. They are handles to more context, replay, evidence, or continued output. Fetch avoids flooding the default response while keeping deeper material recoverable. See [Output Model](/docs/fetch/output-model) and [Playground & Replay](/docs/fetch/playground-and-replay). ### Should I turn on every Include Option? [#should-i-turn-on-every-include-option] No. Start with defaults. Add Include Options only when the task needs a specific artifact such as HTML, screenshot, links, response metadata, Appendix, or State JSON detail. See [Include Options](/docs/fetch/include-options). ### Does Batched Fetch support every single-Fetch output? [#does-batched-fetch-support-every-single-fetch-output] No. Batched Fetch is optimized for many URLs and has limitations. It does not support every single-Fetch artifact or Highlights path. See [Batched Fetch](/docs/fetch/batched-fetch). ## Highlights and citations [#highlights-and-citations] ### What are Highlights? [#what-are-highlights] Highlights finds the most relevant snippets inside a Fetch capture so an agent can ask focused questions without loading the whole document again. See [Highlights](/docs/fetch/highlights). ### Why do technical surfaces still say `fetch_search`, `/v1/fetch/search`, or `expandai search`? [#why-do-technical-surfaces-still-say-fetch_search-v1fetchsearch-or-expandai-search] Highlights is the product name. Some technical surfaces keep their existing endpoint, tool, or command names for compatibility. See [Highlights](/docs/fetch/highlights) and the [API Reference](/api-reference). ### Does Highlights re-fetch the page? [#does-highlights-re-fetch-the-page] Snapshot Highlights searches an existing capture by snapshot ID. Inline Highlights can run as part of a Fetch request. See [Highlights](/docs/fetch/highlights). ### Can Highlights search Markdown, State JSON, and Appendix? [#can-highlights-search-markdown-state-json-and-appendix] Yes, when those artifacts are present in the searchable corpus. Defaults and include choices determine what is available. See [Highlights](/docs/fetch/highlights) and [Include Options](/docs/fetch/include-options). ### Do Highlights results include citations? [#do-highlights-results-include-citations] Highlights results include enough location information to build evidence links. MCP can add `citationUrl` fields; raw API and SDK responses may require using citation helper functions. See [Playground & Replay](/docs/fetch/playground-and-replay) and the [TypeScript](/docs/reference/typescript-sdk) and [Python](/docs/reference/python-sdk) SDKs. ### Can I show Playground citation links to my own users? [#can-i-show-playground-citation-links-to-my-own-users] Yes, if the captured content is safe to share publicly. Playground links are designed for evidence inspection and can be embedded into deep research products, but you should not expose private pages, secrets, or customer data. See [Playground & Replay](/docs/fetch/playground-and-replay). ## Agents [#agents] ### What is the difference between hooks, the Expand Skill, and MCP? [#what-is-the-difference-between-hooks-the-expand-skill-and-mcp] Hooks can route a host's native web fetch behavior through Expand. The Skill teaches an agent how to use Expand. MCP gives an agent native Fetch tools. See [Agent Quickstarts](/docs/agent-quickstarts). ### Which agents support hooks? [#which-agents-support-hooks] First-class hook support exists for Claude Code and OpenCode. Cursor and Codex use Skill plus MCP instead. See [Claude Code](/docs/agent-quickstarts/claude-code) and [OpenCode](/docs/agent-quickstarts/opencode). ### Is installing the Expand Skill enough? [#is-installing-the-expand-skill-enough] It can be enough when the agent can read the skill and run terminal commands. Add MCP when you want native tool calls or when the agent should not rely only on shell commands. See [Expand Skill](/docs/agent-quickstarts/expand-skill). ### Should agent setup use `expandai login` or `EXPAND_API_KEY`? [#should-agent-setup-use-expandai-login-or-expand_api_key] For normal local agent setup, use `expandai login`; the local MCP server and CLI read local auth. Use `EXPAND_API_KEY` for SDK, API, server, or CI-style contexts. See [Agent Quickstarts](/docs/agent-quickstarts) and the [TypeScript](/docs/reference/typescript-sdk) and [Python](/docs/reference/python-sdk) SDKs. ### What is `/start.md`? [#what-is-startmd] `/start.md` is the canonical public setup prompt for agents. Give it to an agent when you want it to choose the right Expand setup path. See [`start.md`](/docs/machine-readable-docs/start-md). ### Why can't my agent see the installed skill or MCP server? [#why-cant-my-agent-see-the-installed-skill-or-mcp-server] Most often, the install was run from the wrong directory, the agent needs to restart, or the MCP config was written to a different scope than the agent is using. See [Agent Quickstarts](/docs/agent-quickstarts) and your agent's setup page. ## Billing and tiers [#billing-and-tiers] ### Is Expand billed by tokens? [#is-expand-billed-by-tokens] No. Expand helps agents answer more questions with fewer tokens, but billing is credit and usage based, not token based. See [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). ### What creates billable usage? [#what-creates-billable-usage] Fetch usage is metered through billable outputs. Highlights and summaries add usage when requested. See [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). ### What are credits? [#what-are-credits] Credits are a dollar balance on your organization. Usage draws down that balance as billable outputs are recorded. See [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). ### What happens when my balance reaches zero? [#what-happens-when-my-balance-reaches-zero] Usage can be blocked until funds are available again. Auto-Recharge helps avoid hitting a zero-balance block. See [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). ### Are tiers pricing plans? [#are-tiers-pricing-plans] No. Tiers are automatic browser-concurrency unlocks based on successful spend and time since your first transaction. See [Tiers](/docs/account-and-billing/tiers). ### Why am I not in the next tier after spending enough? [#why-am-i-not-in-the-next-tier-after-spending-enough] Most tiers require both successful spend and elapsed time since the first transaction. If spend is high enough but the time requirement has not elapsed, the next tier is not active yet. See [Tiers](/docs/account-and-billing/tiers). ### What is browser concurrency? [#what-is-browser-concurrency] Browser concurrency is the number of browser-backed Fetch jobs that can run at the same time before additional work queues. See [Tiers](/docs/account-and-billing/tiers) and [Batched Fetch](/docs/fetch/batched-fetch). ### Do provisioned browser slots upgrade my tier? [#do-provisioned-browser-slots-upgrade-my-tier] No. Provisioned slots add browser capacity on top of your tier. They do not change your tier status. See [Tiers](/docs/account-and-billing/tiers). ### When do provisioned slot upgrades and downgrades take effect? [#when-do-provisioned-slot-upgrades-and-downgrades-take-effect] Upgrades apply immediately. Downgrades take effect at the next billing cycle. See [Tiers](/docs/account-and-billing/tiers). ## Reference and Markdown docs [#reference-and-markdown-docs] ### Why is API Reference separate from authored docs? [#why-is-api-reference-separate-from-authored-docs] Authored docs explain concepts and workflows. API Reference owns exact endpoint contracts, schemas, errors, and generated examples. Open the [API Reference](/api-reference). ### Where do exact request and response schemas live? [#where-do-exact-request-and-response-schemas-live] Use API Reference for exact endpoint schemas. Authored docs intentionally link to reference pages instead of duplicating full schemas. Open the [API Reference](/api-reference). ### Can agents read the docs as Markdown? [#can-agents-read-the-docs-as-markdown] Yes. Docs and API Reference pages are available through Markdown-oriented routes, and machine-readable docs provide compact agent entry points. See [Docs as Markdown](/docs/machine-readable-docs/docs-as-markdown). ### What is the difference between `/start.md`, `llms.txt`, and `llms-full.txt`? [#what-is-the-difference-between-startmd-llmstxt-and-llms-fulltxt] `/start.md` is an agent setup prompt. `llms.txt` is a compact docs index. `llms-full.txt` is a larger docs bundle for agents that need more context. See [`start.md`](/docs/machine-readable-docs/start-md), [`llms.txt`](/docs/machine-readable-docs/llms-txt), and [`llms-full.txt`](/docs/machine-readable-docs/llms-full-txt). ### Can API Reference pages be fetched as Markdown? [#can-api-reference-pages-be-fetched-as-markdown] Yes. The API Reference area stays available through Markdown routes and `Accept: text/markdown` requests. Open the [API Reference](/api-reference). --- # Pricing & Usage (https://staging.expand.ai/docs/account-and-billing/pricing-and-usage) Expand is credit-based. Fetch and Highlights create usage; credits pay for that usage; tiers and provisioned browser slots control how much work can run at once. Start with welcome credits, add funds when needed, and keep usage running with Auto-Recharge. Fetch is metered by billable outputs. Highlights add usage when requested. Your tier gives you a base browser limit. Provisioned slots add more browser sessions immediately. Manage balance, payment methods, invoices, Auto-Recharge, and browser concurrency from Billing. Act from these three places: * **Open Billing** — the Billing view in your dashboard, for `Account Balance`, `Add funds`, `Auto-Recharge`, payment methods, invoices, and `Browser Concurrency`. * **View Usage** — the Usage view in your dashboard, for request activity over time. * **Read Tiers** — [Tiers](/docs/account-and-billing/tiers), for tier qualification rules and worked examples. Need more browser capacity before your tier unlocks? Buy provisioned browser slots and they apply immediately. See [Browser concurrency](#browser-concurrency). ## Pricing summary [#pricing-summary] | Item | How it is billed | Notes | | ------------------- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | Fetch | Per billable Fetch output | Main content retrieval. `$2 / 1,000` (`$0.002` each). | | Highlights | Per billable Highlights output | Called Fetch Search on some technical surfaces; docs say Highlights. `$1 / 1,000` (`$0.001` each). | | Browser concurrency | Included base tier plus optional provisioned slots | Higher tiers unlock more base concurrency; provisioned slots add capacity sooner. | | Enterprise / volume | Custom | Bulk discount, higher support, and enterprise terms. | Current public pricing is `$2 / 1,000` for Fetch outputs and `$1 / 1,000` for Highlights outputs. New organizations start with `$10` in welcome credits. Enterprise and high-volume usage is billed at custom rates — [contact sales](/docs/account-and-billing/faq). This is usage and credit based, not token based. Expand's value is that agents answer more questions with fewer tokens, but you pay per billable output, not per token. ## Credits and balance [#credits-and-balance] Credits are dollar balance on your organization. Fetch usage draws down that balance as billable outputs are recorded. Your first organization starts with `$10` in welcome credits, which expire after their configured period (currently 365 days). Anonymous landing and demo usage runs on separate demo credits and does not draw from your account balance. Billing shows your balance broken into: * **Available balance** — credits you can spend now. * **Pending balance** — credits recorded but not yet settled. * **Expiring soon** — credits set to expire within 30 days. * **Next funds date** — when your next scheduled credits become available. Add more with `Add funds`. Added funds use Stripe through Orb, require a payment method, and create credits that expire after their configured period. To keep balance from reaching zero, turn on `Auto-Recharge`. ## Usage metering [#usage-metering] A normal Fetch creates a billable Fetch output. If you ask for Highlights, that adds Highlights usage. Each requested output is metered independently: | What you fetch | Usage recorded | | ------------------------ | --------------------- | | A page for Main Markdown | Fetch | | A page with Highlights | Fetch plus Highlights | View request activity over time in the Usage view of your dashboard. Billing units in this page — Fetch and Highlights outputs — are the source of truth for what you are charged; the Usage view is for inspecting request activity. ## Browser concurrency [#browser-concurrency] Browser concurrency is how many browser sessions can run at once. It comes from two parts: ```txt Total browser concurrency = base tier concurrency + provisioned browser slots ``` ### Tier overview [#tier-overview] Your tier sets your included base browser concurrency. Tiers increase as cumulative spend and elapsed time thresholds are met. | Tier | Qualification | Base browser concurrency | | ---- | ---------------------------------------------------------- | ------------------------ | | 0 | Default | 2 | | 1 | `$10` cumulative spend | 5 | | 2 | `$50` cumulative spend + 7 days since first transaction | 10 | | 3 | `$200` cumulative spend + 14 days since first transaction | 20 | | 4 | `$500` cumulative spend + 30 days since first transaction | 50 | | 5 | `$1000` cumulative spend + 60 days since first transaction | 100 | Spend thresholds must be met for every tier above 0; for tiers 2 and up, the elapsed-time threshold must also be met. See [Tiers](/docs/account-and-billing/tiers) for qualification rules, tier-up timing, and worked examples. ### Provisioned browser concurrency [#provisioned-browser-concurrency] Tiers increase your included base browser concurrency over time. If you need more capacity now, add provisioned browser slots. Provisioned slots are available immediately on upgrade and are added on top of your tier limit. Provisioned browser slots are a fixed-fee subscription item. Use them when you need predictable parallel throughput before tier progression catches up — Batched Fetch and live Fetch both queue under your browser session limit. Removing provisioned browser slots takes effect at the next billing cycle. ## Auto-Recharge [#auto-recharge] Auto-Recharge adds funds when your balance falls below a threshold. | Setting | Default | | --------------- | ------- | | Threshold | `$5` | | Recharge amount | `$10` | When your available balance drops below the threshold, Expand charges your payment method for the recharge amount and adds the credits. Auto-Recharge needs a payment method on file. Keeping it on is the main way to avoid a zero-balance block that pauses usage. Adjust both values, or turn Auto-Recharge off, in Billing. ## Invoices and payment methods [#invoices-and-payment-methods] Payment methods are card records handled through Stripe. You can add a payment method, remove one, and set a default. Invoices are generated through Orb and can include a hosted invoice link and a PDF link. Find them in Billing. You cannot remove your last payment method while Auto-Recharge or a browser concurrency subscription is active — those need a card on file to keep running. ## Access when credits run out [#access-when-credits-run-out] If your organization has positive credits, Fetch can run. If credits run out, usage can be blocked until funds are available again. * A positive balance allows usage. * A zero balance can block usage. * A next funds date can explain when future credits become available, and a temporary block may clear then. * Auto-Recharge reduces the chance of hitting a zero-balance block. * Repeated payment failure (dunning exhaustion) can block access. A single failed payment does not immediately disable all access. Blocks follow your credit balance, not one declined charge. ## What this page does not cover [#what-this-page-does-not-cover] | Need | Where to go | | ------------------------------------- | ---------------------------------------------- | | Detailed tier examples and edge cases | [Tiers](/docs/account-and-billing/tiers) | | Exact endpoint schemas | [API Reference](/api-reference) | | Rate limiting | [Rate limiting](/api-reference/rate-limiting) | | Enterprise contract terms | [Contact sales](/docs/account-and-billing/faq) | | Billing questions | [FAQ](/docs/account-and-billing/faq) | ## Related pages [#related-pages] * [Fetch Overview](/docs/fetch) * [Highlights](/docs/fetch/highlights) * [Batched Fetch](/docs/fetch/batched-fetch) * [Tiers](/docs/account-and-billing/tiers) * [FAQ](/docs/account-and-billing/faq) --- # Tiers (https://staging.expand.ai/docs/account-and-billing/tiers) Tiers unlock included browser concurrency as your organization builds successful spend history. They are calculated automatically from cumulative successful spend and the days since your first transaction. There is no plan to pick and no upgrade button to press. Open your billing dashboard to check your current tier, [read Pricing & Usage](/docs/account-and-billing/pricing-and-usage) for the credit and usage model, or [add browser concurrency](#provisioned-browser-slots) when you need capacity before the next tier unlocks. Tiers update from billing state. There is no manual tier upgrade flow. The next tier unlocks only after both the spend threshold and the time threshold are met. Each tier increases the number of browser sessions included with your account. Need more capacity now? Add browser slots on top of your tier. ## Tier table [#tier-table] | Tier | Qualification | Included browser concurrency | | ---- | ---------------------------------------------------------- | ---------------------------: | | 0 | Default | 2 | | 1 | `$10` successful spend | 5 | | 2 | `$50` successful spend + 7 days since first transaction | 10 | | 3 | `$200` successful spend + 14 days since first transaction | 20 | | 4 | `$500` successful spend + 30 days since first transaction | 50 | | 5 | `$1000` successful spend + 60 days since first transaction | 100 | Tier 0 is the default for every new account. For tiers 2 and up, both requirements must be met. See [what counts toward spend](#what-counts-toward-spend), [when the timer starts](#how-tiers-are-calculated), and [provisioned browser slots](#provisioned-browser-slots) for the details behind each column. ## How tiers are calculated [#how-tiers-are-calculated] Expand calculates your tier from two values: cumulative successful spend and days since your first successful transaction. Both requirements must be met. If multiple tiers match, Expand uses the highest matching tier. The rules: * Tier 0 is the default until you qualify higher. * Spend alone is not always enough. Tiers 2 through 5 also require elapsed time. * Time alone is never enough. Every tier above 0 has a spend threshold. * The timer starts at your first successful transaction, not at signup. * Your current tier is the highest tier whose spend and time requirements are both satisfied. ## What counts toward spend [#what-counts-toward-spend] Successful spend counts toward tier spend. Pending or failed payments do not unlock tiers. For accounts billed through Expand's billing provider, successful invoices count. The backend counts invoices in a successful state, derives cumulative spend from invoice totals, and uses the earliest successful invoice date as your first transaction date. For local billing, adding funds updates your cumulative spend, first transaction date, and tier directly. Use the term `successful spend` when reasoning about tiers. Pending charges, failed charges, and not-yet-synced invoices do not move you up a tier. ## When your tier updates [#when-your-tier-updates] Your tier can update after a successful payment, during billing synchronization, or when your account billing state is refreshed. If you have just paid, expect a short delay while billing state syncs before your tier reflects the new spend. The tier itself is recomputed from your current billing state, so it catches up once the payment is recorded. > If your spend is high enough but your tier has not changed, check the time requirement first. Most tier surprises come from the days-since-first-transaction rule. ## Browser concurrency [#browser-concurrency] The concurrency number in the tier table is included browser concurrency. It controls how many browser-backed Fetch jobs can run at once before additional work queues. ```txt Total browser concurrency = included tier concurrency + provisioned browser slots ``` | Term | Meaning | | ------------------------- | -------------------------------------------------------------------- | | Included tier concurrency | Browser sessions included by the current tier. | | Provisioned browser slots | Additional paid browser sessions added on top. | | Total browser concurrency | The effective browser session limit used by Fetch and Batched Fetch. | This is browser concurrency: the number of live browser sessions running at once. It is not a generic request-rate limit. When you exceed it, additional Fetch and [Batched Fetch](/docs/fetch/batched-fetch) work queues rather than fails. ## Provisioned browser slots [#provisioned-browser-slots] Tiers increase included concurrency over time. If you need more capacity before the next tier unlocks, add provisioned browser slots. They are added on top of your tier limit. * Upgrades apply immediately. * Downgrades take effect at the next billing cycle. * Provisioned slots do not change your tier. * Provisioned slots increase total browser concurrency. Tier 2 includes 10 browser sessions. If you add 20 provisioned browser slots, your total browser concurrency is 30. Provisioned slots buy capacity, not tier status. They are the right tool when you need parallel throughput now and do not want to wait for spend and time thresholds to catch up. ## Examples [#examples] These follow the [tier table](#tier-table) exactly. Tier mechanics are easy to misread, so each case is worked through. ### Spend before time [#spend-before-time] You spend `$50` on day 1. You are not Tier 2 yet because Tier 2 also requires 7 full days since the first transaction. Until 7 full days have elapsed, the highest matching tier is Tier 1. ### Time without spend [#time-without-spend] Your account is 30 days old but has only `$10` in successful spend. Time alone does not unlock Tier 4; the spend threshold must also be met. You stay at Tier 1. ### Higher spend, waiting on time [#higher-spend-waiting-on-time] You spend `$1000` after 30 full days have elapsed. Tier 5 still requires 60 full days since the first transaction, so the highest matching tier is Tier 4 until 60 full days have elapsed. ### Provisioned capacity [#provisioned-capacity] You are Tier 2 with 10 included browser sessions and add 25 provisioned browser slots. Your total browser concurrency is 35. ## Why am I not in the next tier? [#why-am-i-not-in-the-next-tier] Work down this checklist: * Has the successful spend threshold been reached? * Has enough time passed since the first successful transaction? * Has the latest payment finished? * Has billing state synced? * Are you looking at included tier concurrency, or total concurrency after provisioned slots? The most common reason is that the spend threshold is met but the time threshold is not. ## What tiers do not control [#what-tiers-do-not-control] * Tiers are not pricing plans. * Tiers do not change the per-output billing model for Fetch or Highlights outputs. See [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). * Tiers do not include or remove provisioned browser slots. * Tiers do not replace rate limit docs. See the [API Reference](/api-reference) for endpoint-level limits. Storage and retention may appear in pricing copy, but this page does not claim tier-specific retention. Treat retention as confirmed only when the product source of truth says it is tier-backed. ## Related pages [#related-pages] * [Pricing & Usage](/docs/account-and-billing/pricing-and-usage) * [Batched Fetch](/docs/fetch/batched-fetch) * [API Reference](/api-reference) * [FAQ](/docs/account-and-billing/faq) Need concurrency beyond Tier 5 or a committed plan? Contact sales through your account team. --- # Claude Code (https://staging.expand.ai/docs/agent-quickstarts/claude-code) Route Claude Code's `WebFetch` through Expand so pages return as rendered Markdown, and register the Expand MCP server in the same step. One install command does both. ```bash expandai login expandai hook install claude-code project ``` Run this from your project root. Project setup writes `.claude/settings.json` and `.mcp.json`. The hook install also registers MCP, so you do not run a separate `mcp install`. Verify it worked: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` Optional auth check: ```bash expandai whoami ``` ## What gets installed [#what-gets-installed] | Piece | What it does | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | WebFetch hook | Adds a Claude Code `PreToolUse` hook for `WebFetch` that runs `expandai hook claude-code`, falling back to `npx -y expandai` when the CLI is not installed globally. | | `WebFetch` deny rule | Adds `WebFetch` to `permissions.deny` in the same `settings.json`, which removes Claude Code's built-in `WebFetch` from the toolset for that scope. Global scope means every Claude Code session on the machine. Uninstall restores it. | | MCP server | Registers the Expand MCP server as `expandai` with `expandai mcp run` (written as `npx -y expandai mcp run` when the CLI is not on PATH at install time). | | Shared auth | Uses local Expand CLI auth from `expandai login`; no API key is written into Claude Code config. | With the deny rule in place, Claude Code does not offer `WebFetch` at all, so the model reaches for Expand's `fetch` tool and reads a clean, rendered version of the page instead of raw HTML. Denying the tool rather than intercepting each call is what keeps the transcript free of a refusal error on every fetch. The hook remains installed as a fallback: if the deny rule is removed or overridden, the hook catches `WebFetch` and routes it through Expand instead. The MCP server adds native `fetch` and `fetch_search` tools for follow-up fetches and Highlights search across your snapshots. For Claude Code, the hook path is the recommended full setup. It handles `WebFetch` and installs MCP in one command. ## Project and global scope [#project-and-global-scope] Project scope is the default. Use global scope when you want Expand in every Claude Code session. | Scope | Command | Hook config | MCP config | Use when | | ------- | ------------------------------------------- | -------------------------------------- | -------------------------- | ------------------------------------- | | Project | `expandai hook install claude-code project` | `/.claude/settings.json` | `/.mcp.json` | You want Expand enabled for one repo. | | Global | `expandai hook install claude-code global` | `~/.claude/settings.json` | `~/.claude.json` | You want Expand across projects. | Commit project-scoped config only if the team wants Expand enabled for collaborators. Collaborators do not need a global CLI install: the hook falls back to `npx -y expandai` at runtime. They still need to authenticate once with `npx expandai login`. ## Hook, MCP, and Skill [#hook-mcp-and-skill] | Integration | Claude Code support | Use it when | | ----------- | ----------------------- | ------------------------------------------------------------ | | Hook | Recommended | You want Claude Code `WebFetch` to use Expand automatically. | | MCP | Installed by hook setup | You want native tools such as `fetch` and `fetch_search`. | | Skill | Optional | You want extra instructions for shell-based CLI use. | The hook is the recommended full setup, and it installs MCP for you. The Skill is an optional add-on for shell-based CLI use, not a replacement for the hook. ```bash expandai skill install claude-code project ``` Read [Expand Skill](/docs/agent-quickstarts/expand-skill) for Skill behavior and lifecycle. ## Uninstall [#uninstall] Remove the Expand `WebFetch` hook, the `WebFetch` deny rule, and the Expand MCP entry for a scope. A `WebFetch` deny rule you added yourself is left in place. The hook is what marks the rule as ours — install writes both together — so uninstall removes the rule only when it also removes the hook. If you delete the hook by hand first, clear the deny rule yourself: ```bash expandai hook uninstall claude-code project ``` For global setup: ```bash expandai hook uninstall claude-code global ``` ## Troubleshooting [#troubleshooting] ### Claude Code still uses normal WebFetch [#claude-code-still-uses-normal-webfetch] * Confirm you installed the hook in the same project where Claude Code is running. * Restart Claude Code if it did not reload settings. * Confirm `/.claude/settings.json` contains `expandai hook claude-code`. * Make sure the prompt actually triggers `WebFetch`. ### Fetch fails or auth is missing [#fetch-fails-or-auth-is-missing] Run `expandai login` from your terminal, then try the verification prompt again. Confirm your session with `expandai whoami`. Do not paste API keys into Claude Code config for normal local setup. ### MCP tools are missing [#mcp-tools-are-missing] * Confirm the hook install command completed without errors. * For project scope, confirm `/.mcp.json` contains an `expandai` server. * For global scope, confirm `~/.claude.json` contains an `expandai` server. * Use Claude Code's MCP UI or command to inspect connected servers if needed. ### Wrong command [#wrong-command] Use `expandai hook install claude-code project`. The stale command `expandai hook claude` is not correct and will not install the hook. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Expand Skill](/docs/agent-quickstarts/expand-skill) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [CLI Commands](/docs/reference/cli-commands) * [Claude Code docs](https://code.claude.com/docs) --- # Codex (https://staging.expand.ai/docs/agent-quickstarts/codex) Install the Expand Skill so Codex knows when to use Expand, then install MCP so Codex can call Expand Fetch tools directly. Codex uses both: the skill teaches intent, MCP provides the tools. Codex has no Expand hook path. ```bash expandai login expandai skill install codex project expandai mcp install codex project ``` Run this from your project root. Project setup writes `.agents/skills/expandai/` and `.codex/config.toml`. If Codex does not pick up the new skill or MCP server, restart or reopen Codex. MCP verification prompt: ```txt Use the Expand MCP fetch tool to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` If MCP is not wired up yet, the skill can still drive the CLI. Cross-agent check: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` Optional local auth check: ```bash expandai whoami ``` ## What gets installed [#what-gets-installed] | Piece | What it does | | ------------ | --------------------------------------------------------------------------------------------------------- | | Expand Skill | Installs `SKILL.md` so Codex knows when to use Expand and how to fall back to the CLI. | | MCP server | Registers the Expand MCP server as `expandai`, run with `expandai mcp run`. | | Shared auth | Codex config does not receive API keys. The MCP server reads local Expand CLI auth from `expandai login`. | Codex does not have an Expand hook path. For Codex, install both Skill and MCP. ## Project and global scope [#project-and-global-scope] Project scope is the default. It keeps Expand scoped to one repo and lets you commit the config for collaborators. | Scope | Install commands | Writes to | When to use | | ------- | -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | --------------------------- | | Project | `expandai skill install codex project`
`expandai mcp install codex project` | `/.agents/skills/expandai/`
`/.codex/config.toml` | Recommended for one repo. | | Global | `expandai skill install codex global`
`expandai mcp install codex global` | `~/.agents/skills/expandai/`
`~/.codex/config.toml` | Use Expand across projects. | Codex uses `.agents/skills/expandai/` for skills, not `.codex/skills/expandai/`. Commit project-scoped `.agents/skills/expandai/` and `.codex/config.toml` only if the team wants Expand enabled for collaborators. If Codex ignores project config, make sure the project is trusted in Codex. ## Skill and MCP [#skill-and-mcp] The skill gives Codex instructions for when to use Expand. MCP gives Codex native tools such as `fetch` and `fetch_search`. Skill alone can work when Codex can run shell commands. Add MCP when you want Codex to call Expand as native tools. For Codex, install both. Codex does not have an Expand hook path, so there is no automatic interception of its built-in fetching. ## What the installer writes [#what-the-installer-writes] This is a reference for debugging. The installer writes these for you, and you usually should not edit them by hand. Skill path: ```txt /.agents/skills/expandai/SKILL.md ``` MCP config shape in `.codex/config.toml`: ```toml [mcp_servers.expandai] command = "expandai" args = ["mcp", "run"] enabled = true ``` The installed MCP server runs without `--docs`, so the optional `expand://` docs resources are not exposed by default. To expose them, see [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources). ## Uninstall [#uninstall] Uninstall both pieces if you want Codex to forget the Expand instructions and remove the native MCP tools. ```bash expandai skill uninstall codex project expandai mcp uninstall codex project ``` Global uninstall: ```bash expandai skill uninstall codex global expandai mcp uninstall codex global ``` ## Troubleshooting [#troubleshooting] ### Codex does not see the skill [#codex-does-not-see-the-skill] * Confirm the skill was installed for the same scope Codex is using. * Confirm `/.agents/skills/expandai/SKILL.md` exists for project setup. * Do not look for the skill under `.codex/skills/`. * Restart or reopen Codex if it does not reload skills. ### MCP tools are missing [#mcp-tools-are-missing] * Confirm `/.codex/config.toml` contains `[mcp_servers.expandai]` for project setup. * Confirm `~/.codex/config.toml` contains `[mcp_servers.expandai]` for global setup. * Restart or reopen Codex. * Use Codex MCP inspection, such as `/mcp`, if available. ### Project config is ignored [#project-config-is-ignored] Codex may ignore project config until the project is trusted. Trust the project, then restart or reopen Codex. ### Config landed in the wrong place [#config-landed-in-the-wrong-place] Expand writes project-scoped Codex config to your current directory, not a detected repo root. If Codex cannot find the skill or MCP server, confirm you ran the install commands from the project root and check where `.codex/config.toml` and `.agents/skills/expandai/` actually landed. ### Fetch fails or auth is missing [#fetch-fails-or-auth-is-missing] Run `expandai login` and `expandai whoami` in your terminal, then try the verification prompt again. Do not paste API keys into Codex config. ### Codex does not use the tool [#codex-does-not-use-the-tool] Ask Codex explicitly to use the Expand MCP `fetch` tool, then retry. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Expand Skill](/docs/agent-quickstarts/expand-skill) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [CLI Commands](/docs/reference/cli-commands) --- # Cursor (https://staging.expand.ai/docs/agent-quickstarts/cursor) Install the Expand Skill so Cursor knows when to use Expand, then install MCP so Cursor can call Expand Fetch tools directly. Cursor has no hook path, so Skill + MCP is the full recommended setup. ```bash expandai login expandai skill install cursor project expandai mcp install cursor project ``` Run this from your project root. Project setup writes `.cursor/skills/expandai/` and `.cursor/mcp.json`. Restart or reload Cursor so it picks up the new skill and MCP server. Verification prompt: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` Optional auth check: ```bash expandai whoami ``` ## What gets installed [#what-gets-installed] | Piece | What it does | | ------------ | ---------------------------------------------------------------------------------------------------------- | | Expand Skill | Installs `SKILL.md` so Cursor knows when to use Expand and how to fall back to the CLI. | | MCP server | Registers the Expand MCP server as `expandai` with `expandai mcp run`. | | Shared auth | Cursor config does not receive API keys. The MCP server reads local Expand CLI auth from `expandai login`. | ## Project and global scope [#project-and-global-scope] Project scope is the default. Use global scope to make Expand available across every project. | Scope | Install commands | Writes to | When to use | | ------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | ------------------------- | | Project | `expandai skill install cursor project`
`expandai mcp install cursor project` | `/.cursor/skills/expandai/`
`/.cursor/mcp.json` | Recommended for one repo. | | Global | `expandai skill install cursor global`
`expandai mcp install cursor global` | `~/.cursor/skills/expandai/`
`~/.cursor/mcp.json` | Use across projects. | Commit project-scoped config only if the team wants Expand enabled for collaborators. ## Skill and MCP [#skill-and-mcp] The skill gives Cursor instructions for when to use Expand. MCP gives Cursor native tools such as `fetch` and `fetch_search`. Skill alone can work when Cursor can run shell commands. Add MCP when you want Cursor to call Expand as native tools. For Cursor, install both. Cursor does not have an Expand hook path. Unlike Claude Code, there is no way for Cursor to route its built-in fetch through Expand automatically, so the skill is what tells Cursor when to reach for Expand. ## What the installer writes [#what-the-installer-writes] This is reference for debugging. The installer writes both files for you, and you usually should not edit them by hand. Skill path: ```txt /.cursor/skills/expandai/SKILL.md ``` MCP config shape: ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run"] } } } ``` The config holds no API key. Auth comes from your local `expandai login` session, which the MCP server reads at runtime. ## Uninstall [#uninstall] Uninstall both pieces if you want Cursor to forget the Expand instructions and remove the native MCP tools. ```bash expandai skill uninstall cursor project expandai mcp uninstall cursor project ``` Global uninstall: ```bash expandai skill uninstall cursor global expandai mcp uninstall cursor global ``` ## Troubleshooting [#troubleshooting] ### Cursor does not see the skill [#cursor-does-not-see-the-skill] * Confirm you installed the skill for the same scope Cursor is using. * Confirm `/.cursor/skills/expandai/SKILL.md` exists for project setup. * Restart or reload Cursor. * Try global setup only if you want Expand available across every project. ### MCP server is missing [#mcp-server-is-missing] * Confirm `/.cursor/mcp.json` contains an `expandai` server for project setup. * Confirm `~/.cursor/mcp.json` contains an `expandai` server for global setup. * Restart or reload Cursor. * Inspect Cursor's MCP logs if it reports a connection error. ### Fetch fails or auth is missing [#fetch-fails-or-auth-is-missing] Run `expandai login` and `expandai whoami` in your terminal, then try the verification prompt again. Do not paste API keys into Cursor config. ### Cursor does not use the tool [#cursor-does-not-use-the-tool] Ask Cursor explicitly to use the Expand MCP `fetch` tool, then retry. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Expand Skill](/docs/agent-quickstarts/expand-skill) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [CLI Commands](/docs/reference/cli-commands) * [Cursor MCP docs](https://cursor.com/docs/mcp) * [Cursor Skills docs](https://cursor.com/docs/skills) --- # Expand Skill (https://staging.expand.ai/docs/agent-quickstarts/expand-skill) The Expand Skill teaches your agent when to fetch live pages with Expand, reuse captured snapshots, search them for more context, and return citation links instead of falling back to raw HTML. Authenticate once, then install the skill for your agent. Project scope is the default because it keeps the setup in the repo where the agent works. ```bash expandai login expandai skill install codex project ``` Pick your agent. Both the host and the scope are required arguments. | Agent | Project install | Recommended full setup | | ----------- | -------------------------------------------- | ------------------------------------------------- | | Claude Code | `expandai skill install claude-code project` | [Hook setup](/docs/agent-quickstarts/claude-code) | | Cursor | `expandai skill install cursor project` | [Skill + MCP](/docs/agent-quickstarts/cursor) | | Codex | `expandai skill install codex project` | [Skill + MCP](/docs/agent-quickstarts/codex) | | OpenCode | `expandai skill install opencode project` | [Hook setup](/docs/agent-quickstarts/opencode) | Verify it works by asking the agent: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` The skill works on its own whenever your agent can run terminal commands. Add MCP when you want the agent to call Expand as native tools instead of shelling out. For Claude Code and OpenCode, the recommended full setup is a hook. ## Install commands [#install-commands] Run the command from your project root when you use `project` scope, so the files land next to the code the agent works on. Global variants make the skill available across every project: ```bash expandai skill install claude-code global expandai skill install cursor global expandai skill install codex global expandai skill install opencode global ``` Use `project` scope when the setup should travel with one repository. Use `global` scope when you want the same skill in every project on your machine. ## Skill, MCP, and hooks [#skill-mcp-and-hooks] These three surfaces stack. The skill gives your agent instructions for using the Expand CLI from the shell. MCP gives it native Fetch tools. Hooks wire Expand into the host's own fetch behavior where the host supports it. | Agent | Skill support | Recommended full setup | Why | | ----------- | ------------- | ---------------------- | ----------------------------------------------- | | Claude Code | yes | Hook | Routes the native WebFetch tool through Expand. | | Cursor | yes | Skill + MCP | Skill teaches usage; MCP exposes direct tools. | | Codex | yes | Skill + MCP | Skill teaches usage; MCP exposes direct tools. | | OpenCode | yes | Hook | Routes native fetch through Expand. | Skill alone is enough when your agent can run terminal commands. Add MCP when you want the agent to call Expand as native tools instead of shelling out. For Cursor and Codex, the full setup is Skill + MCP. For Claude Code and OpenCode, a hook is usually the best full setup because it upgrades the host's own fetch behavior. Cursor and Codex do not have an Expand hook path. ## What the skill teaches [#what-the-skill-teaches] ### Fetch live pages [#fetch-live-pages] Use `expandai fetch ` when the agent needs rendered page content instead of raw HTML. Expand renders the live page, so it handles SPAs and bot-protected sites that plain `curl` cannot. ```bash expandai fetch https://docs.example.com/api expandai fetch github.com/owner/repo ``` ### Inspect object-mode output [#inspect-object-mode-output] Use JSON output when the agent needs the page metadata, Main Markdown, and State JSON together. ```bash expandai fetch https://docs.example.com/api --format json ``` In object-mode output, the reusable snapshot handle is `meta.snapshotId`. ### Reuse snapshots [#reuse-snapshots] Use `expandai search "your query"` to search a page that was already captured instead of fetching the same URL again. Expand calls this search experience Highlights. ```bash expandai search "pricing limits" expandai search "pricing limits" --format json --max-results 10 --min-score 0.5 ``` Use `expandai fetch --search "query"` only for a one-shot fetch and search of a URL you have not captured yet. Once the agent has a `snapshotId`, prefer `expandai search`, which reuses the existing capture. ### Return citations [#return-citations] Search-backed claims should carry citation links. In text output, `expandai search` and `expandai fetch --search` print a public playground citation link above each snippet. MCP tool results expose `citationUrl` fields directly. The skill instructs the agent to return those links alongside any claim grounded in fetched content. ### Ask for more context [#ask-for-more-context] Start with Markdown. Reach for JSON, snapshot search, and referenced artifacts only when the task needs more than the main document. This keeps the agent's first read cheap and lets it pull detail on demand. ### Stay in the user's auth boundary [#stay-in-the-users-auth-boundary] If Expand is not authenticated, the agent should ask the user to run `expandai login` rather than authenticating on the user's behalf. ```bash expandai login ``` ## Installed locations [#installed-locations] Most users never need these paths. Reach for them when the agent does not see the skill or when project versus global scope looks wrong. | Agent | Project path | Global path | | ----------- | ------------------------------------------- | ------------------------------------- | | Claude Code | `/.claude/skills/expandai/` | `~/.claude/skills/expandai/` | | Cursor | `/.cursor/skills/expandai/` | `~/.cursor/skills/expandai/` | | Codex | `/.agents/skills/expandai/` | `~/.agents/skills/expandai/` | | OpenCode | `/.opencode/skills/expandai/` | `~/.config/opencode/skills/expandai/` | Codex uses `.agents/skills/expandai/`, not `.codex/skills/expandai/`. Project-scoped skill files are written into the repository. Commit them only when the team wants Expand enabled for collaborators. ## Update and uninstall [#update-and-uninstall] There is no separate `expandai skill update` command. Reinstall the skill to update it. The installer overwrites the existing Expand Skill directory for the selected host and scope. ```bash expandai skill install codex project ``` Uninstall removes the skill directory for that host and scope: ```bash expandai skill uninstall codex project expandai skill uninstall cursor global ``` ## Troubleshooting [#troubleshooting] ### My agent does not see the skill [#my-agent-does-not-see-the-skill] * Confirm you installed for the right host. * Confirm project versus global scope. * Restart the agent if the host does not hot-reload skills. * Confirm `SKILL.md` exists at the expected path from the table above. * For project scope, confirm you ran the install command from the project root. ### The skill runs but Fetch fails [#the-skill-runs-but-fetch-fails] Run `expandai login` in your terminal, then ask the agent to try again. A non-zero exit with `Not authenticated` means the CLI has no session. ### I need tools, not just instructions [#i-need-tools-not-just-instructions] Install MCP for Cursor, Codex, or another MCP-compatible client when you want the agent to call Expand tools directly instead of relying on shell commands. ```bash expandai mcp install cursor project expandai mcp install codex project ``` Read [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) for the full tool list. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Claude Code](/docs/agent-quickstarts/claude-code) * [Cursor](/docs/agent-quickstarts/cursor) * [Codex](/docs/agent-quickstarts/codex) * [OpenCode](/docs/agent-quickstarts/opencode) * [Skill-Based Agents](/docs/agent-quickstarts/skill-based-agents) * [CLI Commands](/docs/reference/cli-commands) --- # Overview (https://staging.expand.ai/docs/agent-quickstarts) Choose the fastest setup for your agent. Expand gives agents Fetch through hooks, the Expand Skill, and MCP. Most local setups authenticate once with `expandai login`, then your agent can fetch web context when it needs it. If you want your agent to wire itself up, paste this prompt: ```txt Read https://expand.ai/start.md and set up Expand. ``` The prompt is a shortcut, not a requirement. Pick your agent in the matrix below and follow its setup page directly. Need the CLI? Start with [Quickstart](/docs/get-started/quickstart) or [CLI Commands](/docs/reference/cli-commands), then come back to configure your agent. `expandai login` and `expandai mcp run` both come from the Expand CLI. ## Supported agents [#supported-agents] Each row routes to a dedicated setup page. Supported modes are shown as `Hook`, `Skill`, and `MCP`. | Agent | Start with | Supported modes | Auth | Set up | | ----------------- | ----------- | -------------------- | -------------------------- | ----------------------------------------------------------------- | | Claude Code | Hook | `Hook` `Skill` `MCP` | `expandai login` | [Set up Claude Code](/docs/agent-quickstarts/claude-code) | | Cursor | Skill + MCP | `Skill` `MCP` | `expandai login` | [Set up Cursor](/docs/agent-quickstarts/cursor) | | Codex | Skill + MCP | `Skill` `MCP` | `expandai login` | [Set up Codex](/docs/agent-quickstarts/codex) | | OpenCode | Hook | `Hook` `Skill` `MCP` | `expandai login` | [Set up OpenCode](/docs/agent-quickstarts/opencode) | | Other MCP clients | Manual MCP | `MCP` | local login or client auth | [Configure MCP client](/docs/agent-quickstarts/other-mcp-clients) | For Claude Code and OpenCode, the hook is the recommended full setup: one command installs the hook and registers the MCP server, so you do not configure MCP separately. Cursor and Codex have no hook path; they use the Expand Skill alongside MCP tools. ## Setup modes [#setup-modes] You will see four setup modes across the agent pages. Here is when each one applies. ### Hook [#hook] A hook lets Expand intercept or enhance an agent's native web-fetch behavior, so existing fetches return rendered Markdown instead of raw HTML. The hook install also registers the MCP server in the same command. First-class hook agents: * Claude Code * OpenCode ### Expand Skill [#expand-skill] The Expand Skill installs agent-readable instructions for when and how to use Expand: when to fetch, when to run Highlights over a snapshot, how to read State JSON, and how to return citations. Use it for agents that load skills or custom instructions. See [Expand Skill](/docs/agent-quickstarts/expand-skill) for installation, contents, and lifecycle. ### MCP [#mcp] MCP gives the agent native Fetch tools over stdio. Use it when your agent or assistant supports Model Context Protocol. ```txt MCP is an agent integration mechanism, not a separate Expand product. ``` Tool and resource names live in [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources). ### Manual MCP [#manual-mcp] For MCP-compatible clients without a first-class installer, put the server command in the client's MCP config: ```bash expandai mcp run ``` The client launches that command over stdio. Full configuration shapes for JSON and TOML clients are in [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients). ## Authentication [#authentication] For local agent setup, run `expandai login` once. The agent reuses the same local Expand credentials as the CLI, so no API key is written into agent config. ```bash expandai login ``` Use `EXPAND_API_KEY` for CI, servers, direct API calls, or non-interactive environments. Keep keys out of local agent config unless you are deliberately running an advanced, non-interactive setup. ## Verify the setup [#verify-the-setup] After setup, ask your agent: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` Each setup page also lists an MCP-specific check so you can confirm the native tool is visible to the agent. ## Docs for agents [#docs-for-agents] Point an agent at these when you want it to understand Expand setup or reference docs without browsing the human docs UI. Agent-readable setup guide. The same source the setup prompt reads. Index of the docs for agents and IDE tools you point at it explicitly. The full docs bundle in one file for deep agent context. Every docs page is available as plain Markdown for ingestion. Major AI crawlers do not fetch `llms.txt` on their own. It is for agents and tools you explicitly point at it. ## Other agents [#other-agents] Agents that work through the Expand Skill rather than a first-class installer live on [Skill-Based Agents](/docs/agent-quickstarts/skill-based-agents). Those pages route back to [Expand Skill](/docs/agent-quickstarts/expand-skill) for installation. ## Next steps [#next-steps] * [Claude Code](/docs/agent-quickstarts/claude-code) * [Cursor](/docs/agent-quickstarts/cursor) * [Codex](/docs/agent-quickstarts/codex) * [OpenCode](/docs/agent-quickstarts/opencode) * [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) * [Expand Skill](/docs/agent-quickstarts/expand-skill) --- # OpenCode (https://staging.expand.ai/docs/agent-quickstarts/opencode) Route OpenCode's native webfetch through Expand. Install the hook to replace webfetch output with rendered Markdown and a snapshot reference, and register the Expand MCP server in the same command so OpenCode can also fetch pages directly or call Expand tools when it needs more control. ```bash expandai login expandai hook install opencode project ``` Run the project command from the project root. Expand writes OpenCode project config into the current working directory, not a detected git root. Verify the hook intercepts native webfetch: ```txt Use OpenCode webfetch to read https://news.ycombinator.com through Expand. Confirm that the output includes an Expand snapshot reference, then show me the first 10 lines of page Markdown. ``` The hook output can start with a snapshot note before the page content. That is expected, and it is the fastest signal that OpenCode is seeing the Expand hook. Verify the MCP tools are wired up: ```txt Use the Expand MCP fetch tool to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the tool call succeeded. ``` See also: [Expand Skill](/docs/agent-quickstarts/expand-skill), [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources), and [CLI Commands](/docs/reference/cli-commands). ## Recommended setup [#recommended-setup] The hook path is the OpenCode default because it improves the thing users already ask OpenCode to do: webfetch. OpenCode still runs its native `webfetch` tool, then the Expand plugin takes over the result. The plugin hooks `tool.execute.after` for the native `webfetch` tool, shells out to `expandai hook opencode`, and replaces the tool output with Expand Markdown. The same installer registers the Expand MCP server, so OpenCode can call Expand tools directly when a task needs snapshot search, JSON output, or more precise control. MCP is not a separate Expand product. It is one integration surface onto Fetch, the same product the hook uses. ## What gets installed [#what-gets-installed] `expandai hook install opencode project` writes two things: the webfetch plugin and the MCP server entry. | Piece | Project path | Global path | | --------------- | ---------------------------------------- | ------------------------------------------------- | | Webfetch plugin | `.opencode/plugins/expandai-webfetch.js` | `~/.config/opencode/plugins/expandai-webfetch.js` | | MCP config | `opencode.json` | `~/.config/opencode/opencode.json` | OpenCode MCP config uses a top-level `mcp` key, not `mcpServers`. The installed entry is a local server, enabled, that runs `expandai mcp run`: ```json { "mcp": { "expandai": { "type": "local", "command": ["expandai", "mcp", "run"], "enabled": true } } } ``` The installer writes this for you. Copy the install command rather than hand-editing JSON. ## Project and global scope [#project-and-global-scope] Project scope is the safer default. | Scope | Command | Writes to | Use when | | ------- | ---------------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------ | | Project | `expandai hook install opencode project` | `.opencode/plugins/`, `opencode.json` | You want Expand enabled for this project. | | Global | `expandai hook install opencode global` | `~/.config/opencode/plugins/`, `~/.config/opencode/opencode.json` | You want Expand available in every OpenCode workspace. | Commit `.opencode/plugins/expandai-webfetch.js` and `opencode.json` only if the team wants Expand enabled for collaborators. ## Hook, MCP, and Skill [#hook-mcp-and-skill] OpenCode has three Expand surfaces. Start with the hook; the others are additive. | Surface | Command | What it does | Recommended | | ------- | ----------------------------------------- | ------------------------------------------------------------------------ | ------------------------ | | Hook | `expandai hook install opencode project` | Replaces native `webfetch` output with Expand Markdown and installs MCP. | Yes, default | | MCP | `expandai mcp install opencode project` | Registers Expand tools without changing native `webfetch`. | When you only want tools | | Skill | `expandai skill install opencode project` | Adds agent-readable instructions for when and how to use Expand. | Optional | Use the hook first. Add the Skill only if you want OpenCode to carry explicit Expand usage guidance in its skill system. The Skill installs to `.opencode/skills/expandai/SKILL.md` for project scope or `~/.config/opencode/skills/expandai/SKILL.md` for global scope. Read [Expand Skill](/docs/agent-quickstarts/expand-skill) for its behavior and lifecycle. ## Uninstall [#uninstall] ```bash expandai hook uninstall opencode project expandai hook uninstall opencode global ``` Hook uninstall removes the OpenCode webfetch plugin and the Expand MCP entry for the selected scope. It does not touch unrelated OpenCode config or other MCP servers. If you installed only MCP or only the Skill, uninstall it with the matching command (`expandai mcp uninstall opencode ` or `expandai skill uninstall opencode `). ## Troubleshooting [#troubleshooting] ### OpenCode still uses normal webfetch [#opencode-still-uses-normal-webfetch] * Confirm the plugin exists at `.opencode/plugins/expandai-webfetch.js` or `~/.config/opencode/plugins/expandai-webfetch.js`. * Restart OpenCode if it does not pick up plugin changes. * Confirm the prompt actually uses OpenCode `webfetch`. * Run `expandai login` and retry. ### MCP tools are missing [#mcp-tools-are-missing] * Confirm `opencode.json` or `~/.config/opencode/opencode.json` contains the `mcp.expandai` entry. * Confirm the command is `expandai mcp run`. * Confirm the CLI is on the same PATH OpenCode uses. * Restart OpenCode if needed. ### Config landed in the wrong place [#config-landed-in-the-wrong-place] Expand writes project-scoped OpenCode files into the current directory, not a detected git root. If OpenCode cannot see the plugin or MCP server, check whether you ran the installer from a subfolder. ### Authentication fails [#authentication-fails] Run `expandai login` in your terminal, then ask OpenCode to try again. Local OpenCode setup should not require putting an API key into OpenCode config. ### I want instructions too [#i-want-instructions-too] Install the [Expand Skill](/docs/agent-quickstarts/expand-skill) if you want OpenCode to keep explicit guidance for when to fetch, reuse snapshots, and return citations. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Expand Skill](/docs/agent-quickstarts/expand-skill) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [CLI Commands](/docs/reference/cli-commands) --- # Other MCP Clients (https://staging.expand.ai/docs/agent-quickstarts/other-mcp-clients) Point any MCP-compatible agent at the local Expand MCP server and it gets Fetch as native tools. Use this page when your agent supports MCP but does not have a first-class Expand installer. The server runs over stdio through `expandai mcp run`. Need the CLI first? Start with [Quickstart](/docs/get-started/quickstart) or [CLI Commands](/docs/reference/cli-commands), then come back to configure your MCP client. Authenticate once in your terminal: ```bash expandai login ``` Then give your client this stdio config. The client launches the server; you do not run it yourself. ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run"] } } } ``` Most clients should not run `expandai mcp run` in your terminal directly. Put it in the client's MCP config so the client starts the server over stdio and owns its lifecycle. Next: read [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) for the full tool schemas, or jump to your first-class page if you use [Claude Code](/docs/agent-quickstarts/claude-code), [Cursor](/docs/agent-quickstarts/cursor), [Codex](/docs/agent-quickstarts/codex), or [OpenCode](/docs/agent-quickstarts/opencode). ## Client config shapes [#client-config-shapes] The command is always the same: `command` is `expandai`, `args` are `["mcp", "run"]`. Only the surrounding schema changes between clients. Claude-style JSON: ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run"] } } } ``` OpenCode-style JSON: ```json { "mcp": { "expandai": { "type": "local", "command": ["expandai", "mcp", "run"], "enabled": true } } } ``` Codex-style TOML: ```toml [mcp_servers.expandai] command = "expandai" args = ["mcp", "run"] enabled = true ``` These are schema patterns, not a substitute for first-class setup pages. If you actually use Claude Code, Cursor, Codex, or OpenCode, use the dedicated page — the installer writes and maintains this config for you. ## Optional docs resources [#optional-docs-resources] Add `--docs` when you want the MCP server to expose local documentation resources alongside the Fetch tools. This adds the `expand://about` and `expand://start` resources. ```bash expandai mcp run --docs ``` ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run", "--docs"] } } } ``` First-class installers do not enable `--docs` by default. Do not assume `expand://start` exists unless the configured MCP command includes `--docs`. ## MCP tools overview [#mcp-tools-overview] The raw MCP server exposes two tools. Read [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) for parameters, schemas, and examples. | Tool | Use | | -------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `fetch` | Fetch a URL and return Markdown, optional metadata, optional State JSON, optional appendix, and optional Highlights snippets. | | `fetch_search` | Technical tool name for Highlights on a previous snapshot by `snapshotId`, without fetching the URL again. | Search snippets include `citationUrl`; agents should return those links when grounding claims in fetched content. ## Should I use this page? [#should-i-use-this-page] Use this page if: * your MCP client is not Claude Code, Cursor, Codex, or OpenCode; * your client can launch a local stdio command; * you want native tool calls instead of shelling out through a skill. Use another page if: | You use | Go to | | -------------- | -------------------------------------------------------------------------------------------------------------- | | Claude Code | [Claude Code](/docs/agent-quickstarts/claude-code) | | OpenCode | [OpenCode](/docs/agent-quickstarts/opencode) | | Cursor | [Cursor](/docs/agent-quickstarts/cursor) | | Codex | [Codex](/docs/agent-quickstarts/codex) | | No MCP support | [Skill-Based Agents](/docs/agent-quickstarts/skill-based-agents) or [Quickstart](/docs/get-started/quickstart) | ## Authentication [#authentication] For local MCP clients, run `expandai login` once in your terminal. The MCP server uses the same local Expand credentials as the CLI, so the config carries no secrets. Do not put `EXPAND_API_KEY` in the primary local config. API keys belong in CI, server, direct API, advanced, or non-interactive setups. If advanced users need environment variables, see [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) or [CLI Commands](/docs/reference/cli-commands). ## Troubleshooting [#troubleshooting] ### `expandai` is not found [#expandai-is-not-found] Your MCP client may have a different PATH than your terminal. Use an absolute path to the `expandai` binary or configure the client's environment. ### Authentication fails [#authentication-fails] Run `expandai login`, then verify with `expandai whoami`. Restart the MCP client after authentication if it keeps a long-running server process. ### My client uses a different config shape [#my-client-uses-a-different-config-shape] Keep the command the same and adapt the surrounding schema to your client's MCP format: command `expandai`, args `["mcp", "run"]`. ### Docs resources are missing [#docs-resources-are-missing] Add `--docs` to the MCP server args if you want `expand://about` and `expand://start`. ### Fetch works but search does not [#fetch-works-but-search-does-not] Call `fetch` first, read the returned snapshot ID, then use Highlights through `fetch_search` with that `snapshotId`. --- # Skill-Based Agents (https://staging.expand.ai/docs/agent-quickstarts/skill-based-agents) Set up Expand for an agent that has no first-class installer yet. If your agent can read a `SKILL.md`, install reusable instructions, or follow a setup prompt, it can use Expand through the bundled Skill or MCP. Pi, Hermes, OpenClaw, and other skill-based agents do not currently have dedicated `expandai skill install` commands. Use the Expand Skill model below, or configure MCP if your agent supports tools. Give the agent the `SKILL.md` body so it knows when to fetch and how to cite. Wire Expand as native tools if your agent can call MCP. Hand the agent the machine-readable prompt and let it pick the path. Each named agent below carries a `No native installer yet` badge. These are manual setup paths, not first-class CLI support. No native installer yet Use the Expand Skill if Pi can install or read a `SKILL.md`. Otherwise hand it `/start.md`. **Use the Skill.** No native installer yet Use reusable instructions first, then add MCP if Hermes supports native tools. **Use the Skill.** No native installer yet The Skill gives behavior guidance; MCP gives native tool calls. **Compare Skill and MCP.** No native installer yet Put the `SKILL.md` where the agent loads reusable instructions. **Open Skill docs.** Verify the setup with one prompt: ```txt Use Expand to fetch https://news.ycombinator.com. Confirm you used Expand, show the first 10 lines of Markdown, and include any snapshot or citation link you received. ``` ## Recommended setup model [#recommended-setup-model] If your agent supports `SKILL.md` files, copy the Expand Skill into the location your agent expects. If your agent supports MCP tools, configure Expand MCP instead or in addition. The first-class hosts get this skill from `expandai skill install`. For an agent without an installer, you place the same skill body yourself: the host decides where the file must live, and Expand provides the content. ### Get the skill body [#get-the-skill-body] The canonical source is `packages/cli/skills/expandai/SKILL.md`. Copy the block below into the skill location your agent expects. ````md --- name: expandai description: Fetch any URL as clean markdown using the `expandai` CLI. Use when reading documentation, articles, GitHub READMEs, blog posts, or any web page — the output is rendered from the live page so it handles SPAs and bot-protected sites that plain `curl` does not. --- # expandai Fetch any URL as clean markdown. ```bash expandai fetch ``` ## When to use Use `expandai fetch` instead of `curl` or `WebFetch` when you need to read web content. It renders the live page (handles JavaScript, SPAs, bot protection) and returns markdown rather than raw HTML. ## Usage ```bash expandai fetch https://docs.example.com/api expandai fetch github.com/owner/repo # https:// is added automatically expandai fetch https://example.com --format json # object-mode { meta, markdown, json } ``` ## Output - Default (`--format text`): full markdown document — pipe to grep, less, or write to a file. - `--format json`: object-mode response with `meta`, `markdown`, and `json`. The handle for later searches is `meta.snapshotId`. ## Search a page you already fetched Every fetch produces a snapshot handle (`meta.snapshotId`). When the WebFetch hook serves a page, it injects that handle as a note at the top of the markdown. Once a page is captured, extract specific passages by searching the snapshot by handle — don't fetch the URL again: ```bash expandai search "your query" # snippets from the stored snapshot expandai search "your query" --format json --max-results 10 --min-score 0.5 ``` Agents with the expandai MCP server can call the `fetch_search` tool with the same `snapshotId` instead. Use `expandai fetch --search "query"` only for a one-shot fetch+search of a URL you have not captured yet — it starts a new browser capture. If you already have a `snapshotId`, prefer `expandai search` (or `fetch_search`), which reuses the existing capture. ## Citations Search results include enough location data to audit the cited passage. In text output, `expandai search` and `expandai fetch --search` print a public playground citation link above each snippet. MCP tool results expose `citationUrl` fields directly. ## Common patterns ```bash # Read a doc page expandai fetch https://effect.website/docs/getting-started # Save to a file expandai fetch https://example.com > page.md # Inspect object-mode output, including State JSON when present expandai fetch https://docs.example.com/api --format json ``` ## Failure modes - Exit code non-zero with `Not authenticated` — the user has not run `expandai login`. Tell them to run it; do not attempt to authenticate yourself. - Empty markdown output — try `--format json` to inspect the response shape. ```` Authenticate the CLI once so the skill's commands work: ```bash expandai login ``` API keys are for CI, servers, direct API, and other non-interactive use. For local agent setup, prefer `expandai login`. ## Skill vs MCP [#skill-vs-mcp] | Need | Use | | ----------------------------------------------------------------------------- | ------------------------------- | | Agent can read instructions and run shell commands, but cannot call MCP tools | Expand Skill | | Agent supports MCP tools | MCP | | Agent supports both | Skill + MCP | | Agent cannot install skills or MCP | `/start.md` setup prompt or CLI | Use the Skill when the agent needs instructions for when to fetch, how to reuse Highlights on captured pages, and how to return citations. Use MCP when the agent supports native tools and you want it to call Fetch directly. Use both when the agent supports both. * Skill details: [Expand Skill](/docs/agent-quickstarts/expand-skill) * MCP config: [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) * CLI fallback: [Quickstart](/docs/get-started/quickstart) or [CLI Commands](/docs/reference/cli-commands) ## First-class alternatives [#first-class-alternatives] If you are using Claude Code, Cursor, Codex, or OpenCode, use the first-class setup page instead. Those pages use the installer paths supported by the Expand CLI. * [Claude Code](/docs/agent-quickstarts/claude-code) * [Cursor](/docs/agent-quickstarts/cursor) * [Codex](/docs/agent-quickstarts/codex) * [OpenCode](/docs/agent-quickstarts/opencode) ## Troubleshooting [#troubleshooting] ### My agent does not load the skill [#my-agent-does-not-load-the-skill] Check the skill location your agent expects. Expand only provides the `SKILL.md` content for non-first-class hosts; the host decides where that file must live. ### My agent can run tools [#my-agent-can-run-tools] If your agent supports MCP, use [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) instead of manually wiring shell commands into the skill. ### My agent cannot run shell commands [#my-agent-cannot-run-shell-commands] The Expand Skill teaches CLI usage. If your agent cannot run shell commands, configure MCP or use the direct API/SDK from your application code. ### I need an official installer [#i-need-an-official-installer] Official skill installers exist today for Claude Code, Cursor, Codex, and OpenCode. Other agents should use manual skill placement, MCP, or `/start.md` until a first-class installer exists. ## Next steps [#next-steps] * [Agent Quickstarts overview](/docs/agent-quickstarts) * [Expand Skill](/docs/agent-quickstarts/expand-skill) * [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) * [start.md](/docs/machine-readable-docs/start-md) * [CLI Commands](/docs/reference/cli-commands) --- # Batched Fetch (https://staging.expand.ai/docs/fetch/batched-fetch) Start many browser captures, poll one run ID, and consume paginated Fetch results as they finish. Batched Fetch is the async fan-out path for processing a list of URLs that share the same settings, without holding open one request per page. ```bash # Start a run curl -X POST https://api.expand.ai/v1/fetch/batched \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{"urls":["https://example.com","https://example.com/about"]}' # Poll the run ID it returns curl "https://api.expand.ai/v1/fetch/batched/019...?limit=10&offset=0" \ -H "x-expand-api-key: $EXPAND_API_KEY" ``` ```txt URLs -> POST /v1/fetch/batched -> { id } -> GET /v1/fetch/batched/{id} -> status + paginated results ``` | Use Batched Fetch when | Use single Fetch when | | ------------------------------------------ | --------------------------------------------------------------------- | | You have many URLs with the same settings. | You need one URL now. | | Your pipeline can poll asynchronously. | You need screenshots, summaries, State JSON, Appendix, or Highlights. | | You want results paged from one run ID. | You need the full single-request output model. | Jump to: [Start a Run](#start-a-run) · [Poll Results](#poll-results) · [API Reference](/api-reference/fetch/fetch.batched) Batched Fetch is API/SDK-only today. There is no CLI command or MCP tool for starting a batch. ## Start a Run [#start-a-run] Send all the URLs in one POST. The call returns once Hatchet durably accepts the run command; billing reservation, lifecycle projection, and browser captures continue asynchronously in the background. ```bash curl -X POST https://api.expand.ai/v1/fetch/batched \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "x-idempotency-key: fastenersolutions:2026-07" \ -H "Content-Type: application/json" \ -d '{ "urls": [ "https://example.com", "https://example.com/about" ] }' ``` ```json { "id": "019..." } ``` `urls` is required and must be non-empty. `include` is optional, and the only public `browserConfig` field at start is `scrollFullPage`. Unknown top-level keys are rejected. * Every submitted URL uses the same shared settings. The start body does not accept per-URL settings. * Duplicate URLs are deduped by resolved URL, so the same page is captured once. * Production callers should authenticate with an API key. Anonymous sessions cannot start batched runs. Keep the run ID. Every status check and every page of results is read back through it. An immediate poll can return `QUEUED` with no results while the accepted command is being projected. This is expected. Billing is checked authoritatively during that initialization, so a run can later become `FAILED` even though the start request was accepted. ### Safe retries [#safe-retries] When `x-idempotency-key` is omitted, the API generates a unique key. Send a stable key when the caller needs to retry a separate request or recover after a process restart. Retrying the same payload with the same explicit key is safe: the API returns a successful response with the same run ID and does not create or charge for another run. Only reusing the same key with a different request body returns `409 BatchedIdempotencyConflict`: ```json { "_tag": "BatchedIdempotencyConflict", "existingRunId": "019...", "reason": "payload_mismatch" } ``` Treat `payload_mismatch` as a client bug rather than polling a run created for different input. * Keys are scoped to your organization and may be up to 255 characters. * The API generates a key when omitted. Persist or deterministically reconstruct an explicit key when deduplication must survive separate calls or process restarts. * Include a logical period in recurring-job keys, such as `fastenersolutions:2026-07:`. ## Poll Results [#poll-results] Read status and results back with the run ID. Page through results with `limit` and `offset`. ```bash curl "https://api.expand.ai/v1/fetch/batched/019...?limit=10&offset=0" \ -H "x-expand-api-key: $EXPAND_API_KEY" ``` Keep polling while `batchedStatus` is `QUEUED` or `RUNNING`. Stop once it reaches a terminal status. ```ts const terminal = new Set(["COMPLETED", "FAILED", "CANCELLED"]) let status = await client.getBatched(run.id) while (!terminal.has(status.batchedStatus)) { await new Promise((resolve) => setTimeout(resolve, 1000)) status = await client.getBatched(run.id) } ``` * Use a short delay or backoff between polls instead of a tight loop. * Handle API errors from the poll call the way you handle any request. * `FAILED` or `CANCELLED` means stop polling the run. * After the run is terminal, inspect each result's `status` before using it. ## Status Model [#status-model] `batchedStatus` describes the whole run. The top-level `status` describes the current page of results, and each result has its own persisted `status`. | Field | Meaning | How to use it | | ------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------- | | `batchedStatus` | Overall run status. | Decide whether to keep polling the run. | | `status` | Current page status, including extraction quality. | Decide whether this page is stable and its requested content was extracted. | | `results[].status` | Persisted state for one URL. | Handle successful, failed, queued, running, and cancelled URLs independently. | ```txt batchedStatus = whole run status = current page ``` * `QUEUED` means the run was accepted but may not be executing yet. * `RUNNING` means work is still in progress. * `COMPLETED` on `batchedStatus` means the run is done, not that every URL produced Markdown. * `FAILED` and `CANCELLED` are terminal. * A page reads `COMPLETED` only when its rows are stable and requested content extraction succeeded. Requested content can legitimately be empty. * A page can read `FAILED` after a completed capture if snapshot content could not be read or rendered. Successful siblings on that page remain usable. * Failed extraction outcomes are cached for one minute. Re-poll the same failed page with backoff to retry extraction without starting or charging for another browser capture. * A page can read `COMPLETED` while the whole run is still `RUNNING`. Keep the two fields separate in your code; do not collapse them into one `status`. ## Response Shape [#response-shape] A poll returns the run status, pagination metadata, and a page of per-URL results. ```json { "id": "019...", "status": "COMPLETED", "batchedStatus": "RUNNING", "totalUrls": 20, "pagination": { "total": 12, "limit": 10, "offset": 0, "hasMore": true }, "results": [ { "status": "SUCCEEDED", "data": { "response": { "url": "https://example.com", "originStatusCode": 200 }, "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples." } }, { "status": "SUCCEEDED", "data": { "response": { "url": "https://blocked.example", "originStatusCode": 403 }, "blocked": { "blockedType": "botProtection" } } }, { "status": "FAILED", "data": { "response": { "url": "https://unavailable.example" } } } ] } ``` Handle each result independently. A batch can finish even when one URL was blocked, failed, or redirected. Use `results[].status` rather than missing Markdown to identify failed items. A blocked result remains `SUCCEEDED` and carries `data.blocked`; fully populated siblings remain usable. If no poll `include.markdown` is provided, Batched Fetch normalizes results to Markdown by default. For the exact per-result schema, see the [API Reference](/api-reference/fetch/fetch.getBatched). ## Pagination [#pagination] Batched results are paginated. Fetch page 1 with `offset=0`, then keep increasing `offset` by `limit` while `pagination.hasMore` is true. ```txt offset 0 -> 10 results offset 10 -> 10 results offset 20 -> ... ``` * `limit` defaults to `10` and accepts `1` through `100`. * `offset` defaults to `0` and must be non-negative. * `pagination.total` is the number of available result rows for the run. * `pagination.hasMore` tells you whether to request the next page. ```bash curl "https://api.expand.ai/v1/fetch/batched/019...?limit=100&offset=100" \ -H "x-expand-api-key: $EXPAND_API_KEY" ``` * Polling only the first page can miss later results. * Page-level `status` can differ across pages while the run is still active. ## Partial Failures [#partial-failures] Batched Fetch is item-oriented. Treat the batch as a container for many Fetch attempts, then inspect each result before using it. * Blocked target pages can appear as per-item `data.blocked`, for example `blockedType: "botProtection"`. * `data.response.originStatusCode` helps classify each outcome, for example `403` on a blocked page. * Pending rows can initially return only minimal `data.response`. * A URL-only, unblocked result on a `FAILED` page can indicate an extraction failure; missing Markdown on a `COMPLETED` page can be legitimate empty content. * Failed extraction outcomes are cached for one minute. Re-poll the same `limit`/`offset` page with backoff to retry reading existing capture artifacts without another capture or charge. * Terminal `COMPLETED` on the run does not override a page-level extraction failure. * Store per-URL outcome state so a retry collects only the URLs that need it. Retry a start request with the same key and payload after an ambiguous timeout. The API returns the same run ID if the first request already created the run. ## Supported Options [#supported-options] | Capability | Batched Fetch support | Notes | | ------------------------------ | -------------------------------- | -------------------------------------------------------------------------------- | | Markdown | Supported | Default result content. | | HTML | Supported on completed results | Request through `include`. | | Meta | Supported | Included by default when available. | | Links | Supported on completed results | Request through `include.links`. | | Response info | Basic metadata supported | Batched polling returns URL/status metadata, but not response headers today. | | `browserConfig.scrollFullPage` | Supported at start | Applies to every child fetch. | | Screenshots | Not supported in batched results | Use single Fetch. | | State JSON / `json` | Not supported in batched results | Use single Fetch or JSON Mode. | | Appendix | Not supported in batched results | Use single Fetch. | | Highlights/search | Not supported | Use [Highlights](/docs/fetch/highlights) on a single Fetch or existing snapshot. | Polling rejects `screenshot`, `json`, and `appendix` include fields with `BatchedIncludeUnsupported`. Batched Fetch is narrower than single Fetch by design. Use it when many URLs and asynchronous collection matter more than every single Fetch artifact. For the full set of fields and nested defaults, see [Include Options](/docs/fetch/include-options). ## SDK Examples [#sdk-examples] The SDKs wrap the same two calls: `batched` to start a run, `getBatched` to poll it. ### TypeScript [#typescript] ```ts import { ExpandClient } from "@expandai/sdk" const client = new ExpandClient() const run = await client.batched({ urls: ["https://example.com", "https://example.com/about"], }) let page = await client.getBatched(run.id, { limit: "10", offset: "0" }) while (page.batchedStatus === "QUEUED" || page.batchedStatus === "RUNNING") { await new Promise((resolve) => setTimeout(resolve, 1000)) page = await client.getBatched(run.id, { limit: "10", offset: "0" }) } console.log(page.results) ``` * When omitted, `batched()` generates one `idempotencyKey` and reuses it for the client's configured retry policy. * Supply a stable `idempotencyKey` to deduplicate separate calls or calls made after a process restart. * The same key and payload return the same run ID. Catch `ExpandClientApiError` with status `409` only for a different payload; its body contains `existingRunId` and `reason: "payload_mismatch"`. * The generated `getBatched` query params are strings today, so the examples use `"10"` and `"0"`. ### Python [#python] ```python from expandai import BatchedParams, Expand client = Expand() run = client.batched(BatchedParams( urls=["https://example.com", "https://example.com/about"], )) status = client.get_batched(run.id) print(status.results) ``` * When omitted, `batched()` generates one `idempotency_key` and reuses it for the client's configured retry policy. * Supply a stable `idempotency_key` to deduplicate separate calls or calls made after a process restart. * The same key and payload return the same run ID. Catch `ExpandAPIError` with status code `409` only for a different payload; its body contains `existingRunId` and `reason: "payload_mismatch"`. ## Operational Notes [#operational-notes] * Batched Fetch queues its child browser captures under the organization's browser concurrency rather than cancelling each other. * Cancellation is cooperative. A successful cancel request returns `CANCELLING` while running items drain, or `CANCELLED` when no work remains; queued items are cancelled immediately. * Higher tiers can have higher browser concurrency. Exact tier details live in [Tiers](/docs/account-and-billing/tiers) and [Pricing & Usage](/docs/account-and-billing/pricing-and-usage). * Duplicate submitted URLs are deduped by resolved URL. * Anonymous sessions cannot start batched runs; API-key auth is the normal production setup. * There is no Batched-specific public rate-limit number to plan against today. ## Common Mistakes [#common-mistakes] * Polling once, seeing no results, and assuming the run failed. * Stopping because a page `status` is `COMPLETED` while `batchedStatus` is still `RUNNING`. * Ignoring pagination and reading only the first page. * Assuming `COMPLETED` means every item has Markdown. * Requesting unsupported include fields (`screenshot`, `json`, `appendix`) on polling. * Looking for a CLI or MCP batch command. There is none today. * Expecting an automatically generated key to deduplicate separate calls or calls made after a process restart. * Treating `payload_mismatch` as a recoverable duplicate instead of fixing key reuse. ## Next steps [#next-steps] * [Start Batched Fetch API Reference](/api-reference/fetch/fetch.batched): exact create schema. * [Get Batched Fetch API Reference](/api-reference/fetch/fetch.getBatched): exact poll and result schema. * [Include Options](/docs/fetch/include-options): the full include surface and limits. * [Browser Behavior](/docs/fetch/browser-behavior): capture and `scrollFullPage` behavior. * [Highlights](/docs/fetch/highlights): search a single Fetch or existing snapshot. --- # Browser Behavior (https://staging.expand.ai/docs/fetch/browser-behavior) Fetch renders live pages before extraction. It runs JavaScript, observes the page, captures browser state, and returns agent-ready Markdown, State JSON, screenshots when requested, and replayable evidence. Public Fetch keeps browser controls intentionally small: today, `browserConfig.scrollFullPage` is the only public browser behavior option. ```bash curl https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://stripe.com/pricing", "browserConfig": { "scrollFullPage": true } }' ``` Choose Markdown, screenshot, JSON, and response output. Replay what Fetch saw at capture time. See exact request and response schemas. ## What Fetch does in the browser [#what-fetch-does-in-the-browser] Fetch is not just downloading HTML. It opens the page in a browser, lets client-side JavaScript run, observes the rendered page, and then extracts the content and state that matter for agents. That difference matters on pages where raw HTML misses the useful content: * JavaScript-rendered docs that hydrate after load; * dashboards and app shells backed by JSON state; * pricing pages, tables, and grids where layout carries meaning; * icon-heavy pages where the icons are the signal. Fetch reads and preserves page context. It is not a public browser-automation engine: this page does not promise that Fetch can fill forms, click through multi-step flows, or drive arbitrary page interactions. ## Readiness and timing [#readiness-and-timing] Fetch decides when the page is ready internally. Public Fetch does not currently expose timeout, wait selector, network-idle, or custom readiness controls. What that means in practice: * pages are rendered before extraction, not captured as raw source; * a page that never settles may be captured partially; * captures that fail or are refused surface as errors or blocked outcomes; * when output looks off, replay shows what Fetch actually saw. Internal timing constants are not a stable public contract and can change. ## Scrolling [#scrolling] `scrollFullPage` is the one public browser behavior control. It defaults to `false`. When `true`, Fetch attempts to scroll the page before capture to trigger lazy-loaded content, then returns to the top before capturing. ```json { "url": "https://example.com", "browserConfig": { "scrollFullPage": true } } ``` The scroll pass is best-effort. It does not guarantee that every infinite-scroll page is fully exhausted, and a scroll that fails is logged and does not fail the request. Use `scrollFullPage` when important content appears only after scrolling. Do not use it as a promise that every infinite-scroll page will be fully exhausted. ```txt scrollFullPage: false -> capture the initial rendered page scrollFullPage: true -> scroll to trigger lazy content -> return to top -> capture ``` ## Screenshots [#screenshots] Screenshots are requested through `include.screenshot`, not `browserConfig`. ```json { "url": "https://example.com", "include": { "screenshot": true } } ``` Screenshots are opt-in. A requested screenshot defaults to full-page. Set `include.screenshot.fullPage: false` for a viewport screenshot instead. Fetch returns one screenshot per request, not both a full-page and a viewport image. ```txt fullPage: true -> one full-page screenshot fullPage: false -> one viewport screenshot ``` The response carries a screenshot URL and a `fullPage` boolean. The exact schema belongs to [Output Model](/docs/fetch/output-model) and the [API Reference](/api-reference); see [Include Options](/docs/fetch/include-options) for the full request surface. ## Redirects and response metadata [#redirects-and-response-metadata] If your application needs to know what happened at the HTTP layer, request response metadata. ```json { "url": "https://example.com", "include": { "response": { "includeHeaders": true } } } ``` Response metadata can report: * the final URL Fetch landed on; * the originally requested URL when it differs; * origin status; * redirect status; * headers, only when `include.response.includeHeaders` is `true`. The full response schema lives in [Output Model](/docs/fetch/output-model) and the [API Reference](/api-reference). ## Blocked pages [#blocked-pages] Fetch can handle many pages that raw HTTP cannot, but it does not promise to bypass every block, CAPTCHA, auth wall, or bot-protection system. Fetch renders through a real browser and can handle many sites where raw HTTP fails. Some sites still block automated access; those may return `FetchBlocked`. * A blocked page may return `FetchBlocked`. * Synchronous `POST /v1/fetch` can surface this as HTTP 403. * Some retry behavior is internal; Fetch may retry retryable block classes on its own. * Proxy behavior is internal and is not a public request option. * When a capture result is surprising, inspect replay or the error details before assuming the page is unreachable. ## Browser controls that are not public [#browser-controls-that-are-not-public] This is the full set of public browser behavior today. The rest of the table exists so you do not hunt for options that Fetch does not expose. | Control | Public today? | Notes | | ---------------------------- | ------------- | ------------------------------------------------------ | | `scrollFullPage` | Yes | Best-effort scroll before capture. | | Timeout | No | Readiness and timeout behavior are managed internally. | | Proxy / location | No | Proxy retry behavior is internal. | | Viewport / device | No | Not a public Fetch option. | | User agent | No | Not a public Fetch option. | | Cookies / session auth | No | Authenticated browsing is not owned by this page. | | Geolocation | No | Not a public Fetch option. | | Wait selector / action steps | No | Fetch is not a public browser-actions API today. | | Bot detector bypass | No | Internal-only; not a public option. | If any of these become public, this page and the [API Reference](/api-reference) change together. ## Debug with replay [#debug-with-replay] When output looks incomplete or surprising, replay shows what Fetch saw at capture time. * Replay helps debug missing or lazy-loaded content. * Replay lets you inspect citations and evidence. * Replay separates page behavior (what the site rendered) from extraction behavior (what Fetch kept). The common failure modes to look for: timeout, blocked, auth wall, CAPTCHA, missing lazy content, and unexpected redirects. [Playground & Replay](/docs/fetch/playground-and-replay) covers the inspection workflow in full. ## Next steps [#next-steps] * [Include Options](/docs/fetch/include-options): control which artifacts Fetch returns. * [Output Model](/docs/fetch/output-model): see the response fields and schemas. * [Playground & Replay](/docs/fetch/playground-and-replay): inspect evidence visually. * [How Fetch Works](/docs/fetch/how-fetch-works): the capture mental model. --- # Highlights (https://staging.expand.ai/docs/fetch/highlights) Highlights finds the most relevant passages inside a Fetch capture, so your agent can answer focused questions without re-fetching the page or loading the whole document into context. It searches the evidence Fetch already captured and returns the best snippets from Main Markdown, State JSON, and optionally Appendix, each with a relevance score and a source location you can cite. The mental model: ```txt snapshotId + query -> ranked snippets -> source: markdown | appendix | statejson -> citation / Playground link ``` `Highlights` is the product name. The API, CLI, and MCP surfaces still use technical names such as `/v1/fetch/search`, `expandai search`, and `fetch_search`. Jump to: [Choose Corpus](/docs/fetch/include-options) · [Inspect Citations](/docs/fetch/playground-and-replay) · [Open API Reference](/api-reference) ## Two ways to call Highlights [#two-ways-to-call-highlights] Search while fetching, or search a snapshot you already captured. ```bash # Search a snapshot you already have (no re-capture) expandai search "AI agent browser" ``` ```bash # Search while fetching, in one request curl https://api.expand.ai/v1/fetch/json \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://news.ycombinator.com", "search": { "query": "AI agent browser", "maxResults": 3 }, "include": { "markdown": true, "json": true } }' ``` A response carries ranked snippets, not the whole page: ```json { "search": { "query": "AI agent browser", "snippets": [ { "source": "markdown", "text": "Show HN: peerd - AI agent harness that runs entirely in your browser...", "score": 1, "location": { "evidenceId": 858 } } ] } } ``` ## What Highlights does [#what-highlights-does] Highlights searches the evidence Fetch already captured. It returns the best snippets from Main Markdown, State JSON, and optionally Appendix, instead of returning the whole page again. This is how an agent can answer more questions with fewer tokens: read the main document first, then use Highlights when the task needs a focused passage or structured evidence. Reach for it to: * ask a targeted question about a long captured page; * retrieve exact evidence for an answer; * search State JSON without dumping all of it into context; * reuse a snapshot instead of fetching the same URL again; * return citations to a user. ```txt Fetch once -> snapshotId -> Highlights queries without re-capture ``` ## Search while fetching [#search-while-fetching] Inline Highlights starts a new Fetch capture and returns matching snippets from the included corpus. Add a `search` object to a `/v1/fetch` or `/v1/fetch/json` request. JSON Mode shows the response shape most clearly. ```bash curl https://api.expand.ai/v1/fetch/json \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://news.ycombinator.com", "search": { "query": "AI agent browser", "maxResults": 3, "minScore": 0 }, "include": { "markdown": true, "json": true } }' ``` In search mode, JSON Mode returns snippets under `data.search` and suppresses the ordinary payloads (full Markdown, State JSON, HTML, screenshots, links, Appendix): ```json { "meta": { "version": 1, "url": "https://news.ycombinator.com/", "capturedAt": "2026-06-24T17:44:50.852Z", "snapshotId": "019...", "lang": "en", "pageType": "article", "playground": "https://expand.land/s/019..." }, "markdown": "", "json": [], "data": { "search": { "query": "AI agent browser", "snippets": [ { "type": "text", "source": "markdown", "text": "Show HN: peerd - AI agent harness that runs entirely in your browser...", "score": 1, "location": { "evidenceId": 858 } } ] } } } ``` In search mode, `markdown` and `json` are not the full page output. They may be empty while `data.search.snippets` carries the focused results. ## Search an existing Fetch [#search-an-existing-fetch] Snapshot Highlights searches a previous Fetch by `snapshotId`. It does not start a new browser capture. The CLI is the shortest path for humans and agents: ```bash expandai search "AI agent browser" ``` The same call over the raw API hits `POST /v1/fetch/search`: ```bash curl https://api.expand.ai/v1/fetch/search \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "snapshotId": "019...", "search": { "query": "AI agent browser", "maxResults": 2, "minScore": 0 }, "include": { "markdown": true, "json": true, "appendix": false } }' ``` The snapshot search response wraps the snippets with the source response and timing: ```json { "snapshotId": "019...", "response": { "url": "https://news.ycombinator.com/", "originStatusCode": 200 }, "search": { "query": "AI agent browser", "snippets": [ { "source": "markdown", "text": "Show HN: peerd - AI agent harness that runs entirely in your browser...", "score": 1, "location": { "evidenceId": 858 } } ] }, "durationMs": 107 } ``` Batched Fetch does not support Highlights. Search a single snapshot, or run inline Highlights on a single `/v1/fetch` request. ## Markdown and State JSON together [#markdown-and-state-json-together] One Highlights response can mix readable Markdown evidence and structured State JSON evidence. Here, a query for `plan limit` matches a table row in Markdown and the underlying frontend value in State JSON: ```json { "search": { "query": "plan limit", "snippets": [ { "source": "markdown", "text": "| Pro | 10 projects | 50 seats |", "score": 0.94, "location": { "evidenceId": 1204 } }, { "source": "statejson", "text": "planLimits.pro.projects: 10", "json": { "plan": "pro", "projects": 10, "seats": 50 }, "score": 0.91, "location": { "evidenceId": 3, "jsonPath": "$.planLimits.pro" } } ] } } ``` Markdown snippets are readable evidence. State JSON snippets can carry the structured value that produced the match. Agents should use both: quote the readable text when it answers the question, and use State JSON when the page's frontend data contains the precise value. State JSON snippets are not guaranteed. They appear when State JSON is included and relevant structured state exists. ```txt Markdown snippet State JSON snippet readable sentence structured value evidenceId evidenceId + jsonPath ``` ## Corpus selection [#corpus-selection] For Highlights, include options choose the search corpus. They are not just output toggles. * `include.markdown` searches Main Markdown. * `include.json` searches State JSON. * `include.appendix` searches Appendix when Appendix text exists. * Inline Highlights with no include defaults to Markdown plus State JSON. * Snapshot Highlights with no include defaults to Markdown plus State JSON. * Appendix is opt-in. * A request with no searchable corpus is rejected. ```txt Main Markdown State JSON Appendix | v Highlights | v ranked snippets + source locations ``` Turning every corpus off leaves nothing to search: ```json { "search": { "query": "pricing" }, "include": { "markdown": false, "json": false, "appendix": false } } ``` This request has no corpus to search, so it should fail before ranking. See [Include Options](/docs/fetch/include-options) for the full field matrix. ## Snippet fields [#snippet-fields] | Field | Meaning | | --------------------- | ------------------------------------------------------------------------------------- | | `query` | The query that was searched. | | `snippets[]` | Ranked snippet results. | | `source` | `markdown`, `appendix`, or `statejson`. | | `text` | Readable snippet content. | | `json` | Structured State JSON value when the source is `statejson` and the value fits inline. | | `score` | Normalized relevance score from 0 to 1. | | `location.evidenceId` | Canonical public Evidence ID for the snippet when available. | | `location.jsonPath` | JSON path for State JSON snippets when available. | ## Citations and Playground links [#citations-and-playground-links] Raw API snippets expose `location`. CLI text output prints an auditable citation link in each snippet header. MCP adds `citationUrl` fields so agents can return Playground links directly. CLI text output: ```txt [markdown · score 1.00] https://expand.land/s/019...?id=858 Show HN: peerd - AI agent harness that runs entirely in your browser... ``` MCP-enriched snippet: ```json { "source": "statejson", "text": "planLimits.pro.projects: 10", "json": { "projects": 10 }, "score": 0.91, "citationUrl": "https://expand.land/s/019...?id=3" } ``` `citationUrl` is not a raw API field. MCP adds it from the snippet's evidence id, and the CLI adds it to both text and JSON output. You do not need that field to cite. Expand stamps an evidence id into the captured Markdown next to the text it belongs to, written as `{1199}`, and every result carries a `playground` URL for the snapshot. Any citation is those two joined: ```txt ?id= ``` That rule works for anything you read, not just search snippets — which is why the markers are inline in the Markdown. To inspect a citation visually, open the snapshot in [Playground & Replay](/docs/fetch/playground-and-replay). ## Defaults and tuning [#defaults-and-tuning] | Surface | Knob | Default | | ------------------ | ----------------- | ------- | | API | `maxResults` | 5 | | API / CLI | `minScore` | 0.6 | | CLI | `--max-results` | 5 | | CLI | `--min-score` | 0.6 | | MCP `fetch_search` | nested `minScore` | 0 | The CLI supports `--format text` and `--format json`, plus `--min-score` and `--max-results`. MCP `fetch_search` exposes `includeMarkdown`, `includeAppendix`, and `includeJson`. ```bash expandai search "pricing limits" --max-results 10 --min-score 0.5 ``` The same knobs over the raw API: ```json { "snapshotId": "019...", "search": { "query": "pricing limits", "maxResults": 10, "minScore": 0.5 } } ``` ## Empty results and errors [#empty-results-and-errors] Tell these apart: * **Zero snippets** means the corpus was valid but nothing met the score threshold. Lower `minScore` or widen the corpus. * **Empty corpus** means the request is invalid. The current error wording is `Fetch Search requires at least one included corpus: markdown, json, or appendix.` * **Missing or inaccessible `snapshotId`** is a snapshot lookup or ownership error, not a ranking outcome. * **Blocked fetches** are Fetch errors, not Highlights ranking errors. Ranking is not deterministic for every query. It can change as the capture, corpus, and ranker improve, so do not depend on a fixed snippet order or on debug-only ranking metadata. ## Technical name mapping [#technical-name-mapping] Use `Highlights` in prose. Use the technical names only when showing API routes, CLI commands, MCP tools, or field names. | Product term | Technical surface | | ------------------- | ----------------------------------------------------------------------- | | Highlights | Product capability | | Inline Highlights | `search` on `/v1/fetch` or `/v1/fetch/json` | | Snapshot Highlights | `POST /v1/fetch/search` | | JSON Mode field | `data.search` | | MCP tool | `fetch_search` | | CLI command | `expandai search` | | Citation link | Printed by CLI text output; `citationUrl` field in MCP-enriched results | ## Next steps [#next-steps] * [Include Options](/docs/fetch/include-options): choose the search corpus. * [Playground & Replay](/docs/fetch/playground-and-replay): inspect citations visually. * [Output Model](/docs/fetch/output-model): understand `snapshotId`, State JSON, and evidence fields. * [API Reference](/api-reference): see exact endpoint schemas and response examples. --- # How Fetch Works (https://staging.expand.ai/docs/fetch/how-fetch-works) Fetch renders the page, reads the visible content and structured state, then returns the useful working context first. When the task needs more, the output keeps stable references back to State JSON, Appendix, Highlights, citations, and replay. ```txt Useful first, complete when needed. ``` Most extraction tools force a bad choice: return everything and waste context, or prune aggressively and lose evidence. Fetch takes a different path. It gives the agent a useful working document first, then keeps the rest addressable through stable references.

Keep everything

Complete, but expensive. Raw payloads, navigation, footers, duplicated UI, and low-signal text crowd out the answer.

Keep only main text

Concise, but lossy. Tables, icons, app state, links, and source evidence disappear when the agent needs the next step.

Fetch

Useful first, complete when needed. Main Markdown and compact State JSON carry the common case; references preserve depth.

Map these product layers to the `markdown`, `json`, `appendix`, and `meta` fields. Run one request and read the output for yourself. Open a snapshot, follow citations, and replay what Fetch saw. ## A web page is more than text [#a-web-page-is-more-than-text] Useful page context includes the visible article or product copy, but it also includes layout, tables, icons, app state, secondary regions, links, and source evidence. Plain HTML-to-Markdown converters usually flatten that into one stream. Fetch turns the page into layers an agent can reason about. Main Markdown is the first thing the agent should read, not the only thing Fetch knows. ## Useful first [#useful-first] Main Markdown is the readable working document. It is designed to answer the common case with fewer low-signal tokens, while preserving page structure that matters for reasoning. State JSON is structured page and network state. It can answer questions Markdown alone cannot, especially when the page is backed by frontend data or API responses. It is part of the useful working context when it fits; when it is too large or better retrieved later, Fetch keeps references so the agent can ask for more. Spatial Markdown keeps layout-sensitive information readable. Tables, pricing grids, comparison rows, icons, and visual affordances should not collapse into loose paragraphs when they carry meaning. Frontmatter and metadata give the agent request and page context such as the source URL, title, capture metadata, `snapshotId`, and links to inspect deeper evidence. Exact fields belong in [Output Model](/docs/fetch/output-model); this page is the mental model. ## Complete when needed [#complete-when-needed] Progressive disclosure is how Fetch stays compact without becoming lossy. Less context does not mean less evidence. The default output gives the agent the likely answer path first, then leaves handles for the rest: * Appendix keeps secondary regions such as navigation, sidebars, footers, and link collections reachable without flooding the main document. * State JSON can be included directly when useful and can be referenced when it is too large for the main response. * Highlights searches captured Markdown, State JSON, and optional Appendix from a `snapshotId`. * Citations and evidence IDs point back to exact source material. * Playground and Replay make the same handles inspectable for humans. This is why Fetch can help agents answer more questions with fewer tokens: it improves signal density instead of merely making pages shorter. ## What Iris does [#what-iris-does] Iris is the part of Fetch that decides which page signals should become working context, structured state, spatial Markdown, or referenced evidence. Iris is not an API field and it is not a separate product you configure. It is the product mechanism behind three jobs: * selecting useful State JSON; * preserving layout and icons as Spatial Markdown; * keeping evidence handles for source inspection. ## The output layers [#the-output-layers] These are product layers, not exact response field names. For fields and schemas, use [Output Model](/docs/fetch/output-model). Read top to bottom, the output moves from the working document the agent reads first to the evidence it can reach for later. None of these layers is discarded; the lower ones stay addressable by reference. Frontmatter, Main Markdown, and Spatial Markdown form the working document. State JSON and Appendix add structured state and secondary regions when they carry signal. Evidence handles, `snapshotId`, and Playground / Replay keep every claim traceable back to what Fetch saw. Product terms map to current technical names in a few places: * Main Markdown maps to the public `markdown` field. * State JSON maps to `json` in current API fields and include options. * Highlights maps to technical surfaces such as `/v1/fetch/search`, `fetch_search`, and `expandai search` where exact names are required. * `snapshotId` is the public handle used for replay, evidence, and follow-up retrieval. ## Why this helps agents [#why-this-helps-agents] Fetch does not win by making pages shorter at any cost. It wins by improving signal density: the agent starts with the context most likely to answer the question, while deeper evidence stays recoverable. That matters when an agent needs to: * answer from a clean document without paying for every repeated nav item; * keep tables, grids, icons, and product comparisons intact; * recover links and secondary regions without another page capture; * search a previous capture with Highlights instead of fetching again; * attach citations that users can inspect in Playground and Replay. ## Next steps [#next-steps] * [Output Model](/docs/fetch/output-model): see the response fields and examples. * [Quickstart](/docs/get-started/quickstart): run one Fetch. * [Playground & Replay](/docs/fetch/playground-and-replay): inspect evidence visually. * [Fetch overview](/docs/fetch): return to the Fetch section. --- # Include Options (https://staging.expand.ai/docs/fetch/include-options) Include options control which artifacts a Fetch returns: Markdown, State JSON, Appendix, page metadata, links, raw HTML, screenshots, summaries, and response metadata. Start with the default. Add include options only when the task needs more context, a different artifact, or a narrower search corpus. ```bash curl https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "include": { "markdown": true, "json": true } }' ``` Do not turn everything on by default. Fetch is designed to return the useful working context first, then keep deeper evidence reachable when the task needs it. Jump to: [Output Model](/docs/fetch/output-model) · [Highlights](/docs/fetch/highlights) · [Browser Behavior](/docs/fetch/browser-behavior) · [API Reference](/api-reference) ## Which surface am I using? [#which-surface-am-i-using] Include works differently depending on where you call Fetch. Pick the surface, then read its rules. | Surface | What it controls | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | Body `include` | API and SDK output artifacts: Markdown, State JSON, Appendix, page metadata, links, HTML, screenshots, summaries, and response metadata. | | Markdown `?include=` | Markdown document parcels only: `statejson` and `appendix`. | | MCP flags | Agent-facing booleans such as `includeJson`, `includeAppendix`, and `includeMeta`. | | CLI | Common-path Fetch output. No public include flag today. | | Batched Fetch | Shared include options with narrower result support. | ## Start with the default [#start-with-the-default] Most requests should start with the default Fetch output. It gives the agent the main working document and preserves enough structure to answer more questions without flooding context. Add options deliberately: * Use the default for first-pass reading, summarization, extraction, and agent browsing. * Add **State JSON** when frontend state, embedded JSON, or network data may answer questions Markdown cannot. * Add **Appendix** when navigation, sidebars, related links, footers, or secondary regions matter. * Add **`links`** when the application needs link inventories rather than prose. * Add **`html`** only when the integration needs raw source. * Add **`screenshot`** when visual inspection or debugging matters. * Add **`response`** when HTTP status, final URL, redirects, or headers matter. More output is not automatically better output. A larger response costs context and rarely improves the answer. ## Include options matrix [#include-options-matrix] `State JSON` is the product name; the request field is `json`. `meta` is page metadata; `response` is response metadata. | Option | Product meaning | Default / common behavior | Use when | Link deeper | | ------------ | ------------------------------ | --------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------ | | `markdown` | Main Markdown working document | Added when omitted for normal Fetch | The agent needs readable page content | [Output Model](/docs/fetch/output-model) | | `json` | State JSON | Public include default false; MCP defaults true; search default includes it | Frontend state, network data, hidden JSON, or structured evidence matters | [Output Model](/docs/fetch/output-model) | | `appendix` | Secondary page regions | Off unless requested | Related links, sidebars, navigation, references, or secondary context matter | [How Fetch Works](/docs/fetch/how-fetch-works) | | `meta` | Page metadata | Public include default true | Title, URL, snapshot, page metadata, and request context matter | [Output Model](/docs/fetch/output-model) | | `response` | HTTP response metadata | Headers are off unless `response.includeHeaders` is true | Status, redirects, final URL, response metadata, or headers matter | [API Reference](/api-reference) | | `links` | Extracted links | Off unless requested | The application needs link inventories or filtered link lists | [API Reference](/api-reference) | | `html` | Raw HTML | Off unless requested | The integration needs source HTML for its own parser or audit path | [API Reference](/api-reference) | | `screenshot` | Visual capture | Off unless requested | Visual debugging, inspection, or screenshot artifacts matter | [Browser Behavior](/docs/fetch/browser-behavior) | State JSON is core signal, not just an overflow reference, and Appendix is secondary context that is available when needed, not evidence-only. ## Body include [#body-include] Body `include` asks Fetch to return specific artifacts from the capture. Use it when your application needs structured control over the response. It is the main control surface for the API and SDKs. ```json { "url": "https://example.com", "include": { "markdown": true, "json": true, "appendix": true, "meta": true, "response": { "includeHeaders": true } } } ``` A few options take nested configuration: * `links` can be configured for filtered link extraction, such as same-domain filtering and include/exclude patterns. * `screenshot.fullPage` controls whether screenshot capture is full-page. See [Browser Behavior](/docs/fetch/browser-behavior) and the [API Reference](/api-reference) for exact capture defaults. * `response.includeHeaders` includes response headers. See the [API Reference](/api-reference) for exact nested schemas and field-level defaults. ## Markdown include parameter [#markdown-include-parameter] Markdown `?include=` controls document parcels, not every Fetch artifact. It only decides whether `statejson` and `appendix` are appended to the Markdown document. ```txt /v1/fetch -> Markdown + frontmatter + default State JSON /v1/fetch?include= -> Markdown + frontmatter only /v1/fetch?include=appendix -> Markdown + frontmatter + Appendix /v1/fetch?include=appendix,statejson -> Markdown + frontmatter + Appendix + State JSON ``` Rules to remember: * Use the canonical token `statejson` in URLs. `stateJson`, `state-json`, and `state_json` may work as aliases, but examples should use `statejson`. * Unknown tokens are ignored. * `include=appendix` does not also include State JSON unless `statejson` is listed. * Empty `include=` is a deliberate opt-out of extra document parcels. The Markdown document is assembled as parcels: ```txt Markdown document frontmatter Main Markdown + statejson? + appendix? ``` ## JSON Mode and mixed includes [#json-mode-and-mixed-includes] JSON Mode is the same capture split into fields. Body `include` controls requested artifacts, while the URL `?include=` parameter still controls Markdown document parcels such as Appendix and State JSON. Avoid mixing body `include` and URL `?include=` casually. If you need precise behavior, pick one documented surface for the thing you are controlling and verify the output shape in JSON Mode. ## Highlights corpus [#highlights-corpus] When [Highlights](/docs/fetch/highlights) searches a Fetch, include options choose the corpus. They are not just display toggles. * Markdown is a searchable corpus source. * State JSON is a searchable corpus source. * Appendix is a searchable corpus source when included. * If `search` is present and no include is provided, Fetch defaults to Markdown plus State JSON. * A request with no searchable corpus is rejected. ```json { "url": "https://example.com", "search": { "query": "pricing table limits" }, "include": { "markdown": true, "json": true, "appendix": false } } ``` The selected corpus flows into ranked results: ```txt Markdown + State JSON + Appendix | v Highlights corpus | v ranked snippets + citations ``` See [Highlights](/docs/fetch/highlights) for queries, ranking, snippets, scores, citations, and searching an existing `snapshotId`. To inspect a citation back to its exact source block, see [Playground & Replay](/docs/fetch/playground-and-replay). ## MCP include flags [#mcp-include-flags] MCP uses agent-friendly booleans instead of the full body `include` object. | MCP tool | Include flags | | -------------- | --------------------------------------------------- | | `fetch` | `includeMeta`, `includeAppendix`, `includeJson` | | `fetch_search` | `includeMarkdown`, `includeAppendix`, `includeJson` | What the flags mean: * `includeJson` means State JSON. * `includeAppendix` means Appendix. * `includeMeta` means page metadata on `fetch`. * `includeMarkdown` controls whether `fetch_search` searches stored Markdown. MCP defaults are chosen for agent usefulness, not to mirror every raw API default. For setup, see [Agent Quickstarts](/docs/agent-quickstarts); for exact tool schemas, see [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources). ## CLI behavior [#cli-behavior] The CLI is optimized for the common Fetch path. `expandai fetch` does not currently expose a public `--include` flag. * Use `expandai fetch ` for the common path. * Use `--format json` when you need the JSON output shape. * Use `--search`, `--max-results`, and `--min-score` for CLI Highlights behavior. * Use the API, SDKs, or MCP when you need fine-grained include control. When you need to choose individual artifacts, reach for body `include` rather than the CLI. ## Batched Fetch limitations [#batched-fetch-limitations] Batched Fetch accepts shared include options at the start of a run, but the polling path cannot retroactively serve every artifact. * Batched Fetch covers common Markdown and metadata use cases. * Polling cannot serve `screenshot`, `json`, or `appendix` as retroactive include fields. * Highlights is not the Batched Fetch workflow. See [Batched Fetch](/docs/fetch/batched-fetch) for status, pagination, and lifecycle behavior. ## Next steps [#next-steps] * [Output Model](/docs/fetch/output-model): understand the fields and layers each include option returns. * [Highlights](/docs/fetch/highlights): search a capture once the corpus is selected. * [Browser Behavior](/docs/fetch/browser-behavior): screenshot and capture behavior. * [Batched Fetch](/docs/fetch/batched-fetch): include limits when fetching many URLs. * [API Reference](/api-reference): exact schemas and nested include defaults. --- # Overview (https://staging.expand.ai/docs/fetch) Fetch renders live web pages and returns agent-ready Markdown, structured state, citations, and replayable snapshots. Start with the main content, then ask for more when the task needs it. **More answers, fewer tokens.** Fetch separates signal from noise before your agent sees the page. It returns the main working document first, preserves structured state and spatial layout when they matter, and keeps deeper evidence reachable by reference instead of flooding the context window. Less context does not mean throwing data away. Fetch preserves the page, then gives the agent the right entry point. HTML-to-Markdown is not enough anymore. Fetch keeps the tables, icons, layout, hidden JSON, citations, and replay links that ordinary Markdown drops. ## One request [#one-request] Send a URL, get back agent-ready context. The canonical call is `POST /v1/fetch`: ```bash curl https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url":"https://news.ycombinator.com"}' ``` Need a key and a first run? Start with the [Quickstart](/docs/get-started/quickstart). ## What comes back [#what-comes-back] Fetch gives agents the page as a working document, not a screenshot of the DOM and not a lossy scrape. One request returns layers an agent can read in order: ```txt Frontmatter Main Markdown Spatial Markdown for tables, grids, icons, and layout State JSON when useful Appendix references snapshotId citations / Playground link ``` The default answer is pragmatic: Markdown first, structured state when useful, references to more when the agent needs depth. Run one Fetch and inspect the output. Learn the mental model behind the output layers. Map product layers to exact response fields. ## Read the Fetch section [#read-the-fetch-section] Understand Main Markdown, State JSON, Appendix, snapshots, citations, and progressive disclosure. See response fields, frontmatter, metadata, JSON Mode, snapshot IDs, and references. Choose Markdown, State JSON, appendix, metadata, screenshots, HTML, response data, and more. Understand JavaScript rendering, scrolling, screenshots, timeouts, and page limitations. Ask focused questions over captured evidence without fetching the same page again. Inspect what Fetch saw, debug citations, and share replayable evidence links. Fetch many URLs asynchronously and poll for results. ## What Fetch preserves [#what-fetch-preserves] Fetch turns one page into layers instead of one flattened stream. Each layer links to where it is defined in full. | Layer | What it gives the agent | Go deeper | | ----------------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------- | | Frontmatter | Page metadata, URL, title, snapshot ID, and request context before the document body. | [Output Model](/docs/fetch/output-model) | | Main Markdown | The high-signal working document your agent should read first. | [How Fetch Works](/docs/fetch/how-fetch-works) | | Spatial Markdown | Layout-aware Markdown for tables, grids, icons, pricing pages, dashboards, and UI-heavy pages. | [How Fetch Works](/docs/fetch/how-fetch-works) | | State JSON | Structured app and network state, included when it helps answer questions Markdown cannot. | [How Fetch Works](/docs/fetch/how-fetch-works) | | Appendix | Secondary regions, navigation, sidebars, and links without flooding the main document. | [Include Options](/docs/fetch/include-options) | | Evidence & Replay | Citations, source IDs, `snapshotId`, Playground links, and replayable captures. | [Playground & Replay](/docs/fetch/playground-and-replay) | ## Ways to call Fetch [#ways-to-call-fetch] The API, SDKs, CLI, MCP, and agents are all surfaces onto the same Fetch. Pick the one that fits your stack; the output model is the same. | Path | Where to start | | -------------- | ---------------------------------------------------------------- | | API | [API Reference](/api-reference) | | TypeScript SDK | [TypeScript SDK](/docs/reference/typescript-sdk) | | Python SDK | [Python SDK](/docs/reference/python-sdk) | | CLI | [CLI Commands](/docs/reference/cli-commands) | | MCP | [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) | | Agents | [Agent Quickstarts](/docs/agent-quickstarts) | ## When to use Fetch [#when-to-use-fetch] Use Fetch for agent research, docs ingestion, product and pricing pages, dashboard-like pages, JavaScript-rendered content, and any workflow where source evidence matters. A simpler path may be enough for already-clean Markdown or a known API that exposes exactly the data you need. Browser automation is the better tool when the primary task is operating a site, filling forms, or completing a multi-step workflow. ## Next steps [#next-steps] * [Quickstart](/docs/get-started/quickstart): run one Fetch. * [Why Expand](/docs/get-started/why-expand): understand what Fetch preserves beyond plain Markdown. * [How Fetch Works](/docs/fetch/how-fetch-works): learn the mental model. * [API Reference](/api-reference): use exact endpoint schemas. --- # Output Model (https://staging.expand.ai/docs/fetch/output-model) Fetch has two primary output views: a Markdown document for agents and a structured JSON Mode response for applications. Both carry the same capture, split differently: frontmatter/meta, Main Markdown, State JSON, Appendix references, snapshot handles, and evidence. ```txt Markdown mode POST /v1/fetch -> text/markdown JSON Mode POST /v1/fetch/json -> { meta, markdown, json, data } ``` ```bash curl https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url":"https://news.ycombinator.com"}' ``` Next: [See Include Options](/docs/fetch/include-options) to control which fields come back, [Open API Reference](/api-reference) for exact schemas, or read [How Fetch Works](/docs/fetch/how-fetch-works) for the capture model behind these fields. ## Markdown mode [#markdown-mode] `POST /v1/fetch` returns a Markdown document, not JSON. The document opens with YAML frontmatter and is followed by the Main Markdown body. ```md --- version: 1 url: https://example.com capturedAt: 2026-06-24T... snapshotId: ... playground: https://expand.land/s/... title: ... stateJson: moreTokens: 1200 appendix: url: ... links: sameDomain: 42 external: 7 --- # Main page title Main Markdown body... ``` * Frontmatter mirrors JSON Mode `meta`. Same values, expressed as YAML. * `snapshotId` is the stable handle for follow-up retrieval, Highlights, citations, and replay. * `playground` is the whole-capture inspection link, openable at `https://expand.land/s/`. * `stateJson`, `appendix`, and `markdown` in frontmatter can be references or hints to more content rather than the content itself. * `?include=` controls whether Markdown mode embeds State JSON and Appendix directly. The full request controls live in [Include Options](/docs/fetch/include-options). Frontmatter has more fields than shown here. See the [API Reference](/api-reference) for the exact schema instead of relying on this sketch. ## JSON Mode [#json-mode] `POST /v1/fetch/json` returns the same capture as named fields an application can read directly. ```ts { meta: { version: 1, url: string, capturedAt: string, snapshotId: string, playground: string, title?: string, finalUrl?: string, appendix?: { url: string, links: { sameDomain: number, external: number } }, stateJson?: { moreTokens: number }, markdown?: { url: string, moreTokens: number }, blocked?: string, loadingStatus?: string }, markdown: string, json: [ { id: number, kind: "inline" | "network", role: "ld+json" | "next-data" | "network-json" | "...", score: number, url?: string, method?: string, body?: unknown, moreTokens?: number, keys?: string[], urlRedacted?: true, urlOmitted?: "invalid" | "unsafe" | "too-long" } ], data?: { search?: { query: string, snippets: [...] }, assets?: [...] } } ``` * `meta` is the frontmatter expressed as JSON. * `markdown` is the Main Markdown body without frontmatter or evidence sections. * `json` is State JSON evidence, one entry per preserved source. * Network State JSON includes a sanitized originating `url` and captured HTTP `method` when available. * `urlRedacted: true` means credentials were removed from the URL; `urlOmitted` explains why no URL was disclosed. * `data.search` holds Highlights results when you request them. * `meta.appendix`, `meta.stateJson`, and `meta.markdown` are references to more, not the content itself. JSON Mode is not a different capture. It is the same Fetch output split into fields for applications. State JSON is called `json` on the wire because it is data. In docs, State JSON means the selected page and network state Fetch preserved for reasoning. ## Product terms vs field names [#product-terms-vs-field-names] Product names map to specific wire fields and surfaces. This table resolves the naming, it does not add a new layer. | Product term | Field / surface | | ------------- | ----------------------------------------------------------------------------- | | Main Markdown | `markdown` | | Frontmatter | Markdown YAML frontmatter / JSON Mode `meta` | | State JSON | `json`, `meta.stateJson`, `include=json`, `statejson` in Markdown `?include=` | | Appendix | `appendix`, `meta.appendix`, `include=appendix` | | Highlights | `data.search`, `/v1/fetch/search`, `fetch_search`, `expandai search` | | Evidence ID | `json[].id`, `location.evidenceId`, citation URLs | | Snapshot | `snapshotId`, `meta.playground` | ## References to more [#references-to-more] When output would be too large or secondary to the main task, Fetch returns a reference instead of flooding the response. You will see references in fields like: * `meta.stateJson.moreTokens` * `meta.appendix.url` * `meta.markdown.url` * `json[].moreTokens` * `json[].keys` * `snapshotId` * `playground` References are not missing data. They are stable paths back to the capture when the task needs more depth. Follow them with [Highlights](/docs/fetch/highlights) or by opening the snapshot in [Playground & Replay](/docs/fetch/playground-and-replay). ## Citations and evidence fields [#citations-and-evidence-fields] Evidence lives at the field level in Fetch output. Raw API Highlights snippets may include: * `source` * `text` * `score` * `json` * `location` MCP and SDK helpers may add: * `citationUrl` `citationUrl` is added by MCP tooling and SDK helpers. Raw API responses expose the location data needed to build citation links. For snippet retrieval, see [Highlights](/docs/fetch/highlights). For citation inspection and auditing, see [Playground & Replay](/docs/fetch/playground-and-replay). For exact snippet schemas, see the [API Reference](/api-reference). ## Blocked and loading fields [#blocked-and-loading-fields] A synchronous `POST /v1/fetch` can return a `FetchBlocked` error when a page cannot be captured. In JSON Mode, `meta.blocked` or `meta.loadingStatus` may appear on short-circuited or partial responses, and batched results can carry per-item blocked state. These fields signal capture state only. For why a capture was blocked or partial, see [Browser Behavior](/docs/fetch/browser-behavior) and the [API Reference](/api-reference). ## Next steps [#next-steps] * [Include Options](/docs/fetch/include-options): control which fields are returned. * [Highlights](/docs/fetch/highlights): search captured evidence. * [Playground & Replay](/docs/fetch/playground-and-replay): inspect snapshots and citations visually. * [API Reference](/api-reference): see exact schemas. --- # Playground & Replay (https://staging.expand.ai/docs/fetch/playground-and-replay) Playground and Replay let you inspect what Fetch saw, compare the captured page with the Markdown and State JSON it produced, and share citation links that open the exact evidence behind an answer. Every Fetch response carries a `snapshotId`. Open it to replay the captured page, pair the output with the source side by side, and hand auditable evidence links to your users. ## The Playground layout [#the-playground-layout] The Playground is two synchronized panes: generated Markdown on the left, the replayed page on the right. Selecting a source block in the Markdown highlights the region it came from in the replay. State JSON citations open the matched structured evidence instead. Replay the captured page, then pair it with the Markdown side by side. Map a raw snippet `location` to a link that opens the exact evidence. Return citation links so your users can inspect the source behind a claim. ## URL model [#url-model] | Use case | URL | | ---------------------------- | ---------------------------------------------------- | | Whole replay | `https://expand.land/s/` | | Side-by-side Playground | `https://expand.land/s//playground` | | Markdown / Appendix evidence | `https://expand.land/s/?id=` | | State JSON evidence | `https://expand.land/s/?id=` | | WACZ archive | `https://expand.land/s//archive.wacz` | The `snapshotId` is the stable handle for replay, Highlights, citation links, and follow-up inspection. All precise evidence links use the same snapshot-scoped `?id=` contract. Playground may use internal hash state while synchronizing its panes, but that state is not the public citation URL. ## Replay vs Playground [#replay-vs-playground] ### Replay [#replay] Replay opens the captured page from the snapshot archive. Use it to see the page state Fetch captured. ```txt https://expand.land/s/ ``` Browser replay is backed by the snapshot's WACZ archive. Adding `?id=` opens an evidence-specific view: depending on the evidence kind, that may render as a source-hunk in replay or open State JSON evidence as JSON instead of browser replay. ### Playground [#playground] Playground pairs the generated Markdown with the replayed page so you can inspect how output maps back to the source. ```txt https://expand.land/s//playground ``` The left pane shows Markdown (raw or rendered); the right pane shows the replay iframe. Hovering or clicking a source block links it to the matching region in the replay. Public citation links use `/s/?id=`. ## Citation links [#citation-links] Citations are not just footnotes. A citation link should let a human inspect the exact captured evidence behind an agent's claim. Each Highlights snippet can expose a canonical public Evidence ID in `location.evidenceId`: | Snippet source | Raw location | Citation target | | -------------- | --------------------------------------------------- | --------------------------------- | | `markdown` | owning Source Block in `location.evidenceId` | `/s/?id=` | | `appendix` | owning Source Block in `location.evidenceId` | `/s/?id=` | | `statejson` | State JSON evidence object in `location.evidenceId` | `/s/?id=` | How each surface exposes the link: * raw API snippets expose `location`, so you build the URL yourself; * CLI text output prints citation links above each snippet; * MCP results add a `citationUrl` field; * SDK helpers can build citation URLs from `location`; * if no precise location exists, link to the whole snapshot at `/s/`. A Markdown or Appendix citation: ```txt https://expand.land/s/019...?id=858 ``` A State JSON citation: ```txt https://expand.land/s/019...?id=3 ``` State JSON citation links use the same `location.evidenceId` contract as Markdown and Appendix. Internal source-object indexes are not public citation targets. ## Inspect State JSON [#inspect-state-json] State JSON citations open the structured evidence object that matched the answer, not just the rendered page. ```txt https://expand.land/s/?id= ``` `statejson` remains the snippet source name. `evidenceId` is the only public target identity and is accepted by `?id=`. Internal State JSON source indexes are not exposed. ## Fetch snapshot content [#fetch-snapshot-content] Snapshot links are useful for humans and machines. Browsers get replay; non-HTML clients can request Markdown or JSON evidence. * `GET /s/` can return full Markdown for non-HTML clients via content negotiation. * `GET /s//content?id=` can return source-hunk Markdown or State JSON evidence, depending on the evidence and the `Accept` header. * invalid or missing evidence IDs return not-found behavior. For the exact content-negotiation contract, see the [API Reference](/api-reference). ## WACZ archive [#wacz-archive] Replay is backed by a WACZ archive: the captured page, assets, metadata, Markdown, evidence, and State JSON needed to replay and inspect the snapshot. ```txt https://expand.land/s//archive.wacz ``` The WACZ is public for the snapshot and powers replay loading. It is not the primary integration surface for most users, and its internal format is not owned by this page. Snapshot-land does not publicly serve screenshot or bundle artifacts; screenshots are returned by the Fetch API when you request them through [Include Options](/docs/fetch/include-options). ## Embed evidence in your product [#embed-evidence-in-your-product] If you build a deep research agent, return citation links with the answer. Your users should be able to click a claim and inspect the captured source behind it. A useful citation payload pairs the claim with everything needed to render and open it: ```txt Agent answer claim citationUrl source label snippet text optional structured JSON value ``` Rendered as a citation card in your product: ```txt Enterprise includes 50 seats. [View evidence] Source: pricing page Evidence: State JSON Link: https://expand.land/s/019...?id=3 ``` Implementation guidance: * store `snapshotId` with the research result; * store snippet `source`, `text`, `location`, and `citationUrl` when available; * for API-only integrations, build links from `location` or use SDK helpers; * for MCP integrations, pass through `citationUrl` verbatim; * for CLI text workflows, preserve the printed citation link; * render citation links in your product UI, not only in logs. This is an example product UI for teams building on Expand, not a promise that Playground renders citation cards or copy/share controls. Recommend linking to Playground first; product teams can then open or embed those links according to their own UX constraints. ## Shareability, privacy, and indexing [#shareability-privacy-and-indexing] Snapshot links are designed to be shareable evidence links. Treat them like public links to captured source material. * snapshot URLs are public in the snapshot service; * snapshot pages use non-indexable and no-referrer page metadata; * this is not an auth boundary, and that metadata may not apply as HTTP headers to non-HTML responses such as `/content` or `archive.wacz`; * the links are still shareable by anyone who has them; * avoid fetching sensitive private pages if you do not want public evidence links; * do not include secrets, tokens, or private customer data in pages intended for citation sharing. If auth, TTL, retention, or access controls are added later, this page must be updated. ## What to read next [#what-to-read-next] Retrieve snippets and scores from a snapshot. Understand `snapshotId`, `location`, `meta.playground`, and field shapes. Request screenshots, response metadata, State JSON, and Appendix. Understand capture timing, scrolling, screenshots, and blocked pages. --- # Quickstart (https://staging.expand.ai/docs/get-started/quickstart) ## Run Fetch in seconds. [#run-fetch-in-seconds] Fetch a live page yourself, or give Expand to your coding agent. The human path starts with one command. The agent path starts with your harness integration.

Human quickstart

Run one command in your terminal. Prints agent-ready Markdown for a live page, usually in seconds.

Run once · Install globally · Use the SDK · Call the API

Agent quickstart

Give Fetch to Claude Code, Codex, Cursor, or OpenCode, then verify with one prompt.

Agent Quickstarts · Expand Skill · Verify setup

The primary human command: ```bash npx expandai fetch https://news.ycombinator.com ``` This prints agent-ready Markdown for a live page. No project, no config. > The short `expandai` CLI package is pending publish. Until it is live, run `npx @expandai/cli fetch https://news.ycombinator.com`. The `npx expandai ...` form above is the target command. The primary agent prompt: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` If your agent does not have Expand yet, point it at setup first: ```txt Read /start.md or the Expand Agent Quickstart for your harness, install the recommended integration, then run the verification Fetch. ``` ## Human path [#human-path] ### Run once with npx [#run-once-with-npx] ```bash npx expandai fetch https://news.ycombinator.com ``` * No global install required. * Prints Markdown by default. * Best for trying Fetch once. ### Install globally [#install-globally] Once `expandai` is published under the short name: ```bash npm install -g expandai expandai fetch https://news.ycombinator.com ``` If the package is still scoped at install time, use the scoped name with the same binary: ```bash npm install -g @expandai/cli expandai fetch https://news.ycombinator.com ``` ### Authenticate when needed [#authenticate-when-needed] Log in once and the CLI reuses your local session: ```bash expandai login expandai whoami ``` For CI, servers, and non-interactive runs, set an API key instead. The CLI, SDKs, and direct API all read `EXPAND_API_KEY`: ```bash export EXPAND_API_KEY="xpnd_..." ``` > Keep API keys server-side. Raw keys are shown only once after creation. ## Expected output [#expected-output] Markdown mode prints the readable page body. A Hacker News fetch starts like this: ```md # Hacker News 1. Example story title 2. Another story title ... ``` When you want metadata and structured signal alongside the Markdown, request JSON mode: ```bash expandai fetch https://news.ycombinator.com --format json ``` Object-mode JSON carries: * `markdown` — Main Markdown for the page; * State JSON — included in the response when it is useful and fits, addressable by reference when it is too large; * `meta.snapshotId` — the reusable snapshot handle for replay and follow-up Highlights; * metadata and evidence/source handles where returned. Markdown mode does not print a `snapshotId`. Use `--format json` or an SDK `fetchJson` call when you need the snapshot handle. ## API and SDK paths [#api-and-sdk-paths] These are secondary to the CLI for a first run, but they are the path into application code. ### Direct API [#direct-api] ```bash curl -X POST https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url":"https://news.ycombinator.com"}' ``` * `POST /v1/fetch` returns Markdown. * `POST /v1/fetch/json` returns object-mode JSON with Main Markdown, State JSON, and metadata. * Authored cURL examples use the `x-expand-api-key` header. See [API Reference](/api-reference) and [Output Model](/docs/fetch/output-model) for exact schemas. ### TypeScript SDK [#typescript-sdk] Install `@expandai/sdk`. The client reads `EXPAND_API_KEY` from the environment. ```ts import { ExpandClient } from "@expandai/sdk" const client = new ExpandClient() const markdown = await client.fetch({ url: "https://news.ycombinator.com" }) console.log(markdown) ``` Use `client.fetchJson({ url })` when you need Main Markdown, State JSON, and `meta.snapshotId` together. See the [TypeScript SDK](/docs/reference/typescript-sdk) reference. ### Python SDK [#python-sdk] Install `expandai` and set `EXPAND_API_KEY`. The client opens as a context manager. ```python from expandai import Expand, FetchParams, FetchJsonParams with Expand() as expand: markdown = expand.fetch(FetchParams(url="https://news.ycombinator.com")) print(markdown) ``` Use `expand.fetch_json(FetchJsonParams(url=...))` for Main Markdown plus State JSON and `meta.snapshot_id`. See the [Python SDK](/docs/reference/python-sdk) reference. ## Agent path [#agent-path] The fastest way to give an agent web context is its native setup, not a paste. Pick your harness, install once, then verify. | Agent | First path | | -------------------------------------------------- | ------------------------ | | [Claude Code](/docs/agent-quickstarts/claude-code) | Hook — also installs MCP | | [Codex](/docs/agent-quickstarts/codex) | Skill + MCP | | [Cursor](/docs/agent-quickstarts/cursor) | Skill + MCP | | [OpenCode](/docs/agent-quickstarts/opencode) | Hook — also installs MCP | Hooks are available for Claude Code and OpenCode. Skill and MCP setup accept `claude-code`, `cursor`, `opencode`, and `codex`. Pi, Hermes, and OpenClaw are skill-based — set them up from [Skill-Based Agents](/docs/agent-quickstarts/skill-based-agents). After setup, confirm the integration with one prompt: ```txt Use Expand to fetch https://news.ycombinator.com. Show me the first 10 lines of Markdown and confirm the Fetch succeeded. ``` ## Troubleshooting [#troubleshooting] * `npx expandai` not found: the short package is pending publish — use `npx @expandai/cli fetch ...` or install globally. * CLI not authenticated: run `expandai login`, then `expandai whoami` to confirm. * API key missing: set `EXPAND_API_KEY`, or send the `x-expand-api-key` header on direct API calls. * Empty or blocked output: try another URL, then see [Browser Behavior](/docs/fetch/browser-behavior). * Need a structured response: use `--format json` on the CLI or `POST /v1/fetch/json`. * Need links, State JSON, or evidence: see [Output Model](/docs/fetch/output-model), [Include Options](/docs/fetch/include-options), and [How Fetch Works](/docs/fetch/how-fetch-works). ## Next steps [#next-steps] * [Ways to Use Expand](/docs/get-started/ways-to-use-expand): choose agent, CLI, SDK, API, or Playground. * [Agent Quickstarts](/docs/agent-quickstarts): configure your harness. * [CLI Commands](/docs/reference/cli-commands): learn every CLI command. * [TypeScript SDK](/docs/reference/typescript-sdk) and [Python SDK](/docs/reference/python-sdk): integrate in application code. * [How Fetch Works](/docs/fetch/how-fetch-works): learn Main Markdown, State JSON, Appendix, Highlights, and evidence. * [API Reference](/api-reference): the exact endpoint contract. --- # Ways to Use Expand (https://staging.expand.ai/docs/get-started/ways-to-use-expand) ## Pick the path that matches how you want to use Fetch. [#pick-the-path-that-matches-how-you-want-to-use-fetch] Use Expand from an agent, terminal, app, API workflow, or visual Playground. Each path gives you one useful next action and a canonical docs link. The choice is never MCP versus CLI versus SDK. The product is Fetch. These are the surfaces that bring Fetch into the place where your work already happens, and they all return the same output: Main Markdown, State JSON when useful, citations, and snapshot handles. Match the path to where you are right now. Inside an agent, take the agent path so it fetches the web automatically. Testing by hand, run one CLI command. Building a product, call the SDK or the API and hand the same result to your own users. Want to see the evidence behind a Fetch, open the Playground. ## Compare paths [#compare-paths] The cards above route you out. This table is the side-by-side check before you commit. ## MCP belongs under agents [#mcp-belongs-under-agents] MCP is one agent integration path, not a separate product. Use [Agent Quickstarts](/docs/agent-quickstarts) for setup and [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) for exact tool and resource names, such as `fetch_search` for Highlights. Hooks, skills, and MCP combine. Hooks improve a client's native web-fetch behavior, skills teach the agent when to reach for Expand, and MCP exposes Fetch as callable tools. The agent pages give the recommended default for each client, so you do not have to assemble it yourself. ## Common workflows [#common-workflows] * Give web context to an agent → [Agents](/docs/agent-quickstarts) * Fetch from a terminal → [CLI Commands](/docs/reference/cli-commands) * Build Fetch into a product → [TypeScript SDK](/docs/reference/typescript-sdk), [Python SDK](/docs/reference/python-sdk), or [API Reference](/api-reference) * Inspect evidence visually → [Playground & Replay](/docs/fetch/playground-and-replay) ## Machine-readable entrypoints [#machine-readable-entrypoints] Point an agent or IDE tool at these routes when you want it to bootstrap from the docs instead of scraping the site: * [start.md](/start.md): setup instructions an agent can read and follow. * [llms.txt](/llms.txt): an index of the docs. * [llms-full.txt](/docs/llms-full.txt): the full docs bundled into one file. Major AI crawlers do not fetch `llms.txt` on their own. These routes pay off when you explicitly hand the link to an agent, so it loads accurate, current instructions in one step. --- # Why Expand (https://staging.expand.ai/docs/get-started/why-expand) ## Answer more questions with fewer tokens. [#answer-more-questions-with-fewer-tokens] Expand Fetch is a new kind of website-to-Markdown API that saves you money and preserves more information at the same time. Other extraction APIs are HTML-to-Markdown: they convert the raw HTML and drop everything that doesn't survive that conversion — JavaScript-rendered content, app state, layout, and page assets. Expand renders the full website the way a browser does and extracts from there, so the data those tools lose is still there. We'll explain how this is possible with progressive disclosure and the small language models we trained. The result is a clean document plus hidden JSON from the page that other tools don't consider. Oversized secondary regions go in a separate section we call the Appendix. The goal is not shorter Markdown at any cost, but higher information density. * [Run a Quickstart](/docs/get-started/quickstart): get a working Fetch result in one command. * [See how Fetch works](/docs/fetch/how-fetch-works): the full Main Markdown, State JSON, and evidence model. ## Completeness vs. cost [#completeness-vs-cost]

Keep everything

More complete, but noisy and token-heavy. Navigation, footers, repeated UI, and raw payloads crowd out the useful answer and inflate the bill.

Keep only main content

Cheaper but lossy. Hidden state, links, and source evidence vanish exactly when the agent needs them for the next question.

Expand

Useful first, recoverable by reference. Main Markdown and curated State JSON carry the common case; deeper evidence stays one handle away.

A web page is not just the main content. It carries app state, links, sidebars, and unwanted noise like ads and cookie banners. This often leads to choosing between two failure modes. Keeping everything floods the model with low-signal tokens, costing you money. Keeping only main content swings the other way. Pruning to the article body is cheap, but it throws away the links to follow next and the app state that was never in the prose. Expand takes a third path. It intelligently selects what to show first by running small language models over all HTML and internal JSON. ## Progressive disclosure [#progressive-disclosure] Progressive disclosure is how Expand stays pragmatic without becoming lossy. Main Markdown is the readable default document. State JSON rides along when it is useful and fits. The rest is retrievable as needed. The fetch output comes in two layers: ```txt Useful content first: Main Markdown + curated State JSON when useful and when it fits. References to more: Appendix, complete original HTML and JSON, and page assets (images, SVGs, etc). ``` Most questions are answered by the first layer and when a question needs content from the Appendix, the agent fetches it in a follow-up call. ## What Expand preserves [#what-expand-preserves] Layout-aware Markdown keeps tables, grids, pricing pages, feature comparisons, and icons understandable instead of flattening them into loose paragraphs. Structured page and app data that answers questions Markdown alone cannot. Part of the working context when it fits; recoverable by reference when it does not. Navigation, links, sidebars, footers, and secondary regions stay reachable without flooding the main output. Focused retrieval over captured evidence when the agent needs a targeted follow-up instead of another full page read. Citations, source blocks, snapshots, screenshots, Playground, and Replay map extracted output back to the page regions it came from. Expand Fetch uses new methods to preserve information that may have been missed otherwise. It preserves icons and layouts that make things like pricing tables legible and it returns State JSON which answers questions Markdown alone cannot, especially for data-driven pages. There is a reason most agents currently don't use the internal JSON. It would overwhelm their context windows. So we use a small language model to select just the useful fields and elements. Expand's snapshotting system records the complete HTML and all page assets. You can see this for yourself in how the Playground is able to replay the page as it originally appeared. Since page assets such as images are saved, the agent can retrieve them even if the website has been shut down. The snapshotting enables the citations feature, which is a sourcemap for the extracted web page, mapping Markdown+JSON back to the original so you can verify a claim. The agent can include a citation in its response to precisely point to the evidence for a claim. Then humans can click on those citations to highlight the source regions in the Playground. ## Why this improves agent answers [#why-this-improves-agent-answers] The benefits are: * Get information that's usually overlooked: hidden JSON state, icons, and table layout. * Token reduction helps models whose performance degrades when they approach the limits of their context window. * The agent can always retrieve any piece of data from the snapshot, not just the commonly needed content. * Verifiable claims with citations. ## When Expand is worth it [#when-expand-is-worth-it] | Use Expand for | A simpler path may be enough | Use browser automation when | | ------------------------------------ | --------------------------------------- | ------------------------------------------------- | | agent research | simple static articles | the task is a long workflow | | docs ingestion | already-clean Markdown | the goal is form filling | | product and pricing pages | a known API exposes the exact data | extraction is secondary to interaction | | dashboards and app-like pages | source verification does not matter | complex interaction determines the result | | marketplaces and search/detail pages | the page is already structured for LLMs | the agent must operate the page, not just read it | JS-rendered pages and anything with important tables, icons, layout, hidden JSON, or evidence needs are where Expand is most useful. When the source is a clean static article or a known API that already returns the exact data, a plain fetch or a simpler converter is fine. ## Where to go next [#where-to-go-next] * [Quickstart](/docs/get-started/quickstart): run one Fetch. * [Ways to Use Expand](/docs/get-started/ways-to-use-expand): choose agent, CLI, SDK, API, or Playground. * [How Fetch Works](/docs/fetch/how-fetch-works): the full output model including Iris, the rendering engine behind Fetch. Secondary reading: [Output Model](/docs/fetch/output-model), [Include Options](/docs/fetch/include-options), [Browser Behavior](/docs/fetch/browser-behavior), and [Playground & Replay](/docs/fetch/playground-and-replay). --- # Docs as Markdown (https://staging.expand.ai/docs/machine-readable-docs/docs-as-markdown) Every Expand docs page is available as Markdown. Append `.md` to any docs URL when you want to give an agent one specific page without the surrounding docs UI. ```txt HTML https://expand.ai/docs/get-started/quickstart Markdown https://expand.ai/docs/get-started/quickstart.md ``` Fetch the page from a terminal or pipe it straight to your clipboard: ```bash curl https://expand.ai/docs/get-started/quickstart.md ``` ```bash curl https://expand.ai/docs/get-started/quickstart.md | pbcopy ``` * **Open Markdown** — [`/docs/get-started/quickstart.md`](/docs/get-started/quickstart.md) * **Copy URL** — `https://expand.ai/docs/get-started/quickstart.md` * **Open llms.txt** — [`/llms.txt`](/llms.txt) ## .md endpoints [#md-endpoints] Append `.md` to an Expand docs URL to get the Markdown version of that page. | Human page | Markdown page | | ---------------------------------- | ------------------------------------- | | `/docs/get-started/quickstart` | `/docs/get-started/quickstart.md` | | `/docs/fetch/how-fetch-works` | `/docs/fetch/how-fetch-works.md` | | `/api-reference` | `/api-reference.md` | | `/api-reference/fetch/fetch.fetch` | `/api-reference/fetch/fetch.fetch.md` | `.mdx` routes are also supported, but `.md` is the recommended form to share with agents. ## What the Markdown contains [#what-the-markdown-contains] The Markdown opens with the page title and its canonical human URL, then the processed page content: ```md # Quickstart (https://expand.ai/docs/get-started/quickstart) ... ``` Each Markdown response includes: * page title; * canonical human URL; * processed Markdown content; * code blocks and tables as Markdown; * links preserved as Markdown links where the docs renderer emits them; * no docs navigation, sidebar, or page chrome. Docs-as-Markdown returns processed Markdown, not raw MDX source. It is meant for reading and retrieval, not for reconstructing the docs page implementation. ## Copy as Markdown [#copy-as-markdown] On authored docs pages, use the Copy Content menu to copy the current page as Markdown or open the Markdown URL directly. The copied content is the same processed Markdown the `.md` URL serves, so a teammate or agent reading the link sees exactly what you copied. ## Using Markdown with agents [#using-markdown-with-agents] ### Single page context [#single-page-context] Use one `.md` page when the agent needs help with one feature or workflow. ```txt Read https://expand.ai/docs/fetch/output-model.md and show me how to access main Markdown and State JSON from the SDK. ``` ### Multiple page context [#multiple-page-context] Use two or three `.md` pages when the task spans a workflow. ```txt Read these Expand docs before answering: - https://expand.ai/docs/fetch/output-model.md - https://expand.ai/docs/fetch/highlights.md - https://expand.ai/docs/fetch/playground-and-replay.md Then explain how to build a deep research agent with user-visible citations. ``` ### Full docs context [#full-docs-context] For one topic, prefer the specific `.md` page. Use `/docs/llms-full.txt` only when the agent needs broad authored-docs context and has enough context budget. ## Content negotiation [#content-negotiation] This is advanced behavior, not the primary path. Tools that support request headers can ask for Markdown from the human URL with `Accept: text/markdown`. ```bash curl -H "Accept: text/markdown" https://expand.ai/docs/get-started/quickstart ``` This is useful for agents that prefer to keep canonical human URLs. For explicit links, `.md` URLs are easier to inspect and share. Negotiation applies to `/docs` and `/api-reference` pages, not to every route on the site. ## Docs versus API Reference [#docs-versus-api-reference] Use authored docs Markdown for workflows and explanations. Use API Reference Markdown for exact endpoint schemas, parameters, errors, and generated examples. ```txt Authored docs https://expand.ai/docs/fetch/output-model.md API Reference https://expand.ai/api-reference/fetch/fetch.fetch.md ``` ## Which machine-readable route should I use? [#which-machine-readable-route-should-i-use] | Need | Use | | ------------------------- | ---------------------- | | Agent setup | `/start.md` | | First orientation | `/llms.txt` | | Docs table of contents | `/docs/llms.txt` | | One authored docs page | `/docs/...md` | | One API Reference page | `/api-reference/...md` | | Full authored-docs bundle | `/docs/llms-full.txt` | ## Agent guidance [#agent-guidance] * Prefer current Markdown docs over memorized behavior. * Use API Reference Markdown for exact request and response shapes. * Use specific `.md` pages before pulling the full docs bundle. * Do not assume undocumented endpoints, CLI flags, SDK methods, or schemas exist. * Follow canonical URLs in Markdown headers when citing or revisiting docs. ## Related pages [#related-pages] * [start.md](/docs/machine-readable-docs/start-md) * [llms.txt](/docs/machine-readable-docs/llms-txt) * [llms-full.txt](/docs/machine-readable-docs/llms-full-txt) --- # llms-full.txt (https://staging.expand.ai/docs/machine-readable-docs/llms-full-txt) `llms-full.txt` is the full authored-docs bundle for agents that need the docs content, not just the docs index. ```txt https://expand.ai/docs/llms-full.txt ``` Open `/docs/llms-full.txt` to hand an agent the whole authored corpus in one pull, or open [`/docs/llms.txt`](/docs/machine-readable-docs/llms-txt) when you only need the index. The two are not interchangeable: `/docs/llms.txt` lists the pages, `/docs/llms-full.txt` ships their full processed Markdown. | File | Use when | | ---------------------- | ----------------------------------------------------- | | `/llms.txt` | First orientation and setup routing. | | `/docs/llms.txt` | Compact authored-docs index. | | `/docs/llms-full.txt` | Full authored-docs bundle. | | API Reference Markdown | Exact endpoint schemas and generated reference pages. | ## What it contains [#what-it-contains] `/docs/llms-full.txt` is generated from authored docs pages. It is not the same as the API Reference Markdown export. The bundle is assembled from the authored docs source and contains: * processed Markdown for authored docs pages; * one chunk per authored docs page; * the page title before each page body; * the canonical source URL before each page body; * separators between page chunks; * no generated API Reference pages unless the implementation intentionally changes. Each chunk uses the processed Markdown, not the raw `.mdx` source, so the agent reads the same rendered content a human sees. ## Output shape [#output-shape] Pages are concatenated in docs order. Each chunk opens with a heading that carries the page title and its canonical URL, then the page body, then a `---` separator before the next page. ```md # Quickstart (https://expand.ai/docs/get-started/quickstart) ... --- # Fetch Overview (https://expand.ai/docs/fetch/overview) ... ``` Each page chunk must preserve a title and canonical source URL so agents can cite or revisit the source page. ## What it does not contain [#what-it-does-not-contain] * It is not the root orientation file. * It is not the generated API Reference bundle. * It is not a replacement for exact endpoint schemas. * It is not a setup guide by itself. * It may be too large for small context windows. Use API Reference Markdown routes when the agent needs exact endpoint schemas, parameters, errors, or generated examples. ## When to use it [#when-to-use-it] Use `/docs/llms-full.txt` when: * the agent needs broad authored-docs context; * the agent has enough context budget for a large bundle; * docs search is not available; * a single full-docs pull is simpler than fetching pages one by one. Avoid `/docs/llms-full.txt` when: * the agent only needs setup — use [`/start.md`](/docs/machine-readable-docs/start-md); * the agent only needs orientation — use [`/llms.txt`](/docs/machine-readable-docs/llms-txt); * the agent only needs the docs table of contents — use [`/docs/llms.txt`](/docs/machine-readable-docs/llms-txt); * the agent needs one page — use that page's `.md` route; * the agent needs exact endpoint schemas — use API Reference Markdown routes. ## Relationship to API Reference [#relationship-to-api-reference] API Reference is separate from authored docs. `/docs/llms-full.txt` should not be assumed to contain generated endpoint schemas. Authored docs explain how to use Expand. The generated API Reference owns exact endpoint schemas, request bodies, response models, and status codes. When the agent needs that level of precision, reach for an API Reference Markdown route instead of the bundle: * `/api-reference.md`; * `/api-reference.mdx`; * nested `/api-reference/...md`; * nested `/api-reference/...mdx`; * a request to any API Reference page with `Accept: text/markdown`. These schemas live only in the API Reference routes; they are not duplicated in `/docs/llms-full.txt`. ## Generation and maintenance [#generation-and-maintenance] The bundle is regenerated from the authored docs source, so it stays in sync as pages change. When updating it, keep these invariants: * generate from the authored docs source, not a hand-maintained list; * emit processed Markdown, not raw MDX source; * preserve stable page ordering; * preserve the title and canonical URL for every page chunk; * use a consistent separator between pages; * confirm whether API Reference stays excluded — if it is ever included, measure the output size and update this page's copy. Tests for the route should assert the content type, that `Accept: text/markdown` requests resolve correctly, that each chunk carries a title and canonical URL, that the separator is present, that at least one known authored page appears, that processed Markdown is non-empty for included pages, and that ordering stays stable after IA changes. ## Related pages [#related-pages] * [llms.txt](/docs/machine-readable-docs/llms-txt) * [Docs as Markdown](/docs/machine-readable-docs/docs-as-markdown) * [start.md](/docs/machine-readable-docs/start-md) * [API Reference](/api-reference) --- # llms.txt (https://staging.expand.ai/docs/machine-readable-docs/llms-txt) Root `llms.txt` gives agents a compact entrypoint into Expand docs: setup first, then docs, API Reference, and the full docs bundle. ```txt https://expand.ai/llms.txt ``` `llms.txt` is for agents and tools you explicitly point at Expand's docs. Do not treat it as a guarantee that major AI search engines crawl or rank the site differently. Its practical value is as a handoff URL for coding agents, IDE assistants, and MCP tools. Curated root entrypoint. Generated authored-docs table of contents. Full authored docs as Markdown. ## Which file should I use? [#which-file-should-i-use] | URL | Use when | Source | | -------------------------------------- | --------------------------------------------------- | ------------------------------------------------ | | `https://expand.ai/llms.txt` | An agent needs first orientation. | Curated root route. | | `https://expand.ai/docs/llms.txt` | An agent needs the authored docs table of contents. | Generated from authored docs pages. | | `https://expand.ai/docs/llms-full.txt` | An agent needs the full authored docs bundle. | Generated from processed authored docs Markdown. | Root `llms.txt` is curated. `/docs/llms.txt` is generated. `/docs/llms-full.txt` is the full authored-docs bundle. ## Recommended root shape [#recommended-root-shape] Root `llms.txt` follows the llms.txt convention: an H1, a one-line blockquote summary, short guidance, then H2 sections of Markdown link lists with concise descriptions. Setup links come before general docs, and `## Optional` holds links an agent can skip when context is tight. ```md # Expand > Expand Fetch turns live web pages into agent-ready Markdown, State JSON, evidence, and replayable snapshots. Use `start.md` when setting up an agent. Use `llms-full.txt` when you need the full authored docs bundle. Use API Reference Markdown routes for exact endpoint schemas. ## Start Here - [Agent setup guide](https://expand.ai/start.md): Set up Expand across every detected coding agent. - [Docs](https://expand.ai/docs): Human-authored docs. - [API Reference](https://expand.ai/api-reference): Exact endpoint schemas and errors. - [Full docs bundle](https://expand.ai/docs/llms-full.txt): Full authored docs as Markdown. ## Recommended Paths - [Quickstart](https://expand.ai/docs/get-started/quickstart.mdx): Run the first Fetch. - [Ways to Use Expand](https://expand.ai/docs/get-started/ways-to-use-expand.mdx): Choose agents, CLI, SDKs, API, or Playground. - [Fetch Overview](https://expand.ai/docs/fetch/overview.mdx): Understand the Fetch product. ## Agent Setup Prompt Read https://expand.ai/start.md and set up Expand. ## Optional - [Docs index](https://expand.ai/docs/llms.txt): Compact authored-docs index. ``` The root file stays curated and small. It should include: * the project name and short summary; * the start guide link; * the human docs link; * the API Reference link; * the full authored docs bundle link; * the canonical agent setup prompt; * optional links for secondary machine-readable resources. It should not become the full docs dump. Where a Markdown-readable target exists, link the `.md` or `.mdx` route instead of the HTML page so agents read Markdown directly. ## What /docs/llms.txt includes [#what-docsllmstxt-includes] `/docs/llms.txt` is a generated index of authored docs pages. It should list page titles, links, and short descriptions. * It is generated from the authored docs source, not hand-written. * Use it when an agent needs a docs table of contents. * Page links should prefer Markdown-readable targets such as `.md` or `.mdx` routes. * It does not replace root `/llms.txt` for setup and routing. * It does not include generated API Reference pages. For endpoint schemas, use `/api-reference` and its Markdown export routes. ## What /docs/llms-full.txt includes [#what-docsllms-fulltxt-includes] `/docs/llms-full.txt` is a larger authored-docs bundle. Use it when the agent has enough context budget and needs the docs content, not just the index. * It is generated from processed Markdown for authored docs pages. * Each page section includes a title and canonical source URL before the page body. * Pages are separated within the bundle. * It is not the same as the API Reference Markdown export. * It can be too large for some prompts. For a single topic, append `.md` to the docs URL instead. ## Related machine-readable docs [#related-machine-readable-docs] | Resource | Purpose | | ----------------------------- | -------------------------------------------- | | `start.md` | Agent setup guide and setup prompt. | | `llms.txt` | Compact curated agent entrypoint. | | `llms-full.txt` | Full authored-docs bundle. | | Docs `.md` / `.mdx` routes | One authored docs page as Markdown. | | API Reference Markdown routes | Exact endpoint docs and schemas as Markdown. | ## Related pages [#related-pages] * [llms-full.txt](/docs/machine-readable-docs/llms-full-txt) * [start.md](/docs/machine-readable-docs/start-md) * [Docs as Markdown](/docs/machine-readable-docs/docs-as-markdown) * [API Reference](/api-reference) --- # start.md (https://staging.expand.ai/docs/machine-readable-docs/start-md) `start.md` is the canonical machine-readable setup guide for agents. It carries the whole setup procedure in one document — an agent fetches it once and can start immediately. ```txt https://expand.ai/start.md ``` The route returns the guide as `text/markdown`, so any agent that can read a URL can ingest it directly. Read the hosted guide that the setup prompt points an agent at. The curated docs index. It links `start.md` before general docs. ## Setup prompt [#setup-prompt] Paste this into any coding agent. ```txt Read https://expand.ai/start.md and set up Expand. ``` This mirrors the setup prompt the hosted `/llms.txt` and the CLI emit. The **Copy setup prompt** button in the app produces the same instruction, and adds a single-use authorization when you are signed in. ## What the guide contains [#what-the-guide-contains] | Section | What the agent does | | ------------------------ | ----------------------------------------------------------------------------------------------------------- | | Setup | Identifies its own harness, asks for project or global scope, and runs one non-interactive `expandai init`. | | Authorization | Relays the browser approval URL and resumes `init` with the one-time setup code. | | Proof | Fetches a live page through the installed integration and answers with an exact cited sentence. | | Optional SDK setup | Asks before adding the SDK to a confirmed project path. | | Other ways to use Expand | HTTP API and CLI usage, for callers that are not coding agents. | The exact commands live in the guide itself. This page does not mirror them. `init` detects every supported harness on the machine, and the guide asks which of them to configure before writing anything. Only the selected ones are touched; the rest are left alone. ## When to use it [#when-to-use-it] * Use it whenever you want Expand set up in a coding agent — it is the default path. * Use it for generic or skill-based agents that do not have a first-class quickstart. * Use it when a human wants a copyable setup prompt for an agent. * Use the [Agent Quickstarts](/docs/agent-quickstarts) when you would rather run the setup steps by hand. ## Hosted route vs MCP resource [#hosted-route-vs-mcp-resource] `https://expand.ai/start.md` is the hosted public Markdown route. `expand://start` is a local MCP docs resource. `expand://start` is only available when the MCP server is run with docs resources enabled, for example through `expandai mcp run --docs`. Do not assume it exists after a normal first-class MCP install. See [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) for how to wire and run the MCP server. ## Related machine-readable docs [#related-machine-readable-docs] | Resource | Use when | | ------------------------------------------------------------------- | ----------------------------------------------- | | [`start.md`](/start.md) | An agent needs setup instructions. | | [`llms.txt`](/docs/machine-readable-docs/llms-txt) | An agent needs the compact docs index. | | [`llms-full.txt`](/docs/machine-readable-docs/llms-full-txt) | An agent needs a larger docs bundle. | | [Markdown docs pages](/docs/machine-readable-docs/docs-as-markdown) | An agent needs one specific authored docs page. | --- # API Reference (https://staging.expand.ai/docs/reference/api-reference) Exact endpoint contracts for Expand live in the generated API Reference. Open it for exact endpoints, request schemas, response schemas, errors, and rate limits. [Open API Reference](/api-reference) The authored docs on this site explain concepts and tasks. The generated reference at `/api-reference` is the source of truth for request bodies, response fields, parameter tables, status codes, and generated code samples. When you open it, the left nav switches to the API Reference sidebar instead of mixing endpoint pages into the authored tree. ## Generated reference pages [#generated-reference-pages] * [Fetch (`POST /v1/fetch`)](/api-reference/fetch/fetch.fetch) * [Fetch JSON](/api-reference/fetch/fetch.fetchJson) * [Highlights search](/api-reference/fetch/fetch.fetchSearch) * [Batched Fetch](/api-reference/fetch/fetch.batched) * [Get batched result](/api-reference/fetch/fetch.getBatched) * [Errors](/api-reference/errors) * [Rate limiting](/api-reference/rate-limiting) ## Find the endpoint for an authored page [#find-the-endpoint-for-an-authored-page] Each product page maps to a generated endpoint. Use this table to jump from a task to its exact contract. | Authored page | Generated reference | | ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | [Fetch Overview](/docs/fetch) | [`fetch.fetch`](/api-reference/fetch/fetch.fetch) | | [Output Model](/docs/fetch/output-model) | [`fetch.fetchJson`](/api-reference/fetch/fetch.fetchJson) | | [Include Options](/docs/fetch/include-options) | [`fetch.fetch`](/api-reference/fetch/fetch.fetch) and [`fetch.fetchJson`](/api-reference/fetch/fetch.fetchJson) | | [Highlights](/docs/fetch/highlights) | [`fetch.fetchSearch`](/api-reference/fetch/fetch.fetchSearch) | | [Batched Fetch](/docs/fetch/batched-fetch) | [`fetch.batched`](/api-reference/fetch/fetch.batched) and [`fetch.getBatched`](/api-reference/fetch/fetch.getBatched) | Highlights is exposed by the `/v1/fetch/search` endpoint. The product label is Highlights; the technical path keeps its `search` name. ## Auth header [#auth-header] Authored cURL examples authenticate with the `x-expand-api-key` header: ```bash curl https://api.expand.ai/v1/fetch \ -H "x-expand-api-key: $EXPAND_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url":"https://news.ycombinator.com"}' ``` SDKs read `EXPAND_API_KEY` from the environment by default. The generated reference may also document bearer and cookie auth, which are runtime-supported schemes; for task-focused authored examples, keep the API-key header. ## Agent and markdown access [#agent-and-markdown-access] The API Reference area is available to agents and IDE tools as Markdown: * `/api-reference.md` * `/api-reference.mdx` * nested `.md` and `.mdx` routes for each endpoint page * any request sent with `Accept: text/markdown` ## Related authored pages [#related-authored-pages] * [CLI Commands](/docs/reference/cli-commands) * [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) * [TypeScript SDK](/docs/reference/typescript-sdk) * [Python SDK](/docs/reference/python-sdk) --- # CLI Commands (https://staging.expand.ai/docs/reference/cli-commands) Reference for the `expandai` CLI: fetch pages, search existing snapshots, manage local auth, and install agent integrations. Every command, argument, and flag below is documented exactly as the CLI exposes it. Need the CLI? Start with [Quickstart](/docs/get-started/quickstart), then come back here for the full command reference. The `expandai` package is not published yet, so treat install instructions as provisional and get the current install method from Quickstart. Once `expandai` is on your `PATH`, the commands here are the complete surface. ## Most common [#most-common] ```bash expandai fetch https://news.ycombinator.com expandai fetch https://example.com --format json expandai fetch https://example.com --search "authentication" expandai login expandai whoami ``` Jump to: [Fetch](#fetch-content) · [Highlights](#highlights) · [Auth](#auth-and-configuration) · [Agent setup](#agent-setup) · [Cleanup](#cleanup) ## Command index [#command-index] Commands are grouped by job. Each one is copyable as written. | Job | Command | What it does | | ---------- | -------------------------------------------- | --------------------------------------------- | | Fetch | `expandai fetch ` | Fetch a page, print agent-ready Markdown. | | Fetch | `expandai fetch --format json` | Fetch a page, print structured JSON. | | Highlights | `expandai fetch --search ` | Fetch a page, return Highlights snippets. | | Highlights | `expandai search ` | Search an existing snapshot, no recapture. | | Auth | `expandai login` | Store local credentials (browser or API key). | | Auth | `expandai whoami` | Report the current auth state. | | Auth | `expandai logout` | Clear local stored credentials. | | MCP | `expandai mcp run [--docs]` | Run the local stdio MCP server. | | MCP | `expandai mcp install ` | Install Expand MCP config for a host. | | MCP | `expandai mcp uninstall ` | Remove Expand MCP config for a host. | | Skills | `expandai skill install ` | Install the Expand Skill for a host. | | Skills | `expandai skill uninstall ` | Remove the Expand Skill for a host. | | Hooks | `expandai hook install ` | Install an Expand fetch hook for a host. | | Hooks | `expandai hook uninstall ` | Remove an Expand fetch hook for a host. | | Cleanup | `expandai uninstall [target] [host] [scope]` | Remove installed Expand integration files. | Config is set through environment variables (`EXPAND_API_KEY`, `EXPAND_API_URL`) and the local config file under `XDG_CONFIG_HOME`. See [Auth and configuration](#auth-and-configuration). Batched Fetch has no CLI command today. Start and poll batched runs through the API or SDKs. See [Batched Fetch](/docs/fetch/batched-fetch). ## Fetch content [#fetch-content] ### `expandai fetch ` [#expandai-fetch-url] Fetch a rendered web page and print agent-ready Markdown. ```bash expandai fetch ``` ```bash expandai fetch https://news.ycombinator.com ``` | Argument | Required | Description | | -------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `url` | Yes | URL to fetch. Common bare hosts may be normalized to `https://...`; localhost URLs are supported. Flag-like or otherwise invalid URLs are rejected. | | Option | Values | Default | Description | | --------------- | -------------- | ------- | ---------------------------------------------------------------------------- | | `--format` | `text`, `json` | `text` | Controls CLI output format. | | `--search` | string | none | Returns Highlights snippets for the query instead of the full page Markdown. | | `--max-results` | `1` to `50` | `5` | Maximum Highlights snippets when `--search` is used. | | `--min-score` | `0` to `1` | `0.6` | Minimum Highlights relevance score when `--search` is used. | **Output.** Default text output prints Markdown to stdout. `--format json` prints structured Fetch output. With `--search`, text mode prints Highlights snippets instead of the full page. The CLI does not expose Include Options through an `--include` flag today, and `--format` accepts only `text` or `json`, not `markdown`. Related: [Fetch Overview](/docs/fetch) · [Output Model](/docs/fetch/output-model) · [Include Options](/docs/fetch/include-options) · [API Reference](/docs/reference/api-reference) ### `expandai fetch --format json` [#expandai-fetch-url---format-json] Fetch a page and print structured JSON instead of text Markdown. ```bash expandai fetch https://example.com --format json ``` JSON output is what you want when a script needs fields — metadata, Markdown, and State JSON when present — rather than terminal Markdown. State JSON is usually included in the main content; large state may be returned by reference instead. This is not equivalent to every API Include Options combination. The CLI does not expose the nested `include` controls that the API offers; for that level of control, call the API directly. See [Include Options](/docs/fetch/include-options). ## Highlights [#highlights] `--search` and `expandai search` both return Highlights — the most relevant snippets of a page for a query. text/json Both accept `--max-results` (`1`–`50`, default `5`) and `--min-score` (`0`–`1`, default `0.6`). ### `expandai fetch --search ` [#expandai-fetch-url---search-query] Fetch a page and return the most relevant Highlights snippets for a query. ```bash expandai fetch https://example.com --search "pricing" ``` This performs a new Fetch, then runs the query against the captured page. It uses inline Fetch JSON Mode with `search` in the request payload — not the snapshot-search endpoint — so it always recaptures the URL. Text output includes source and score headers plus an auditable citation link for each snippet. JSON output follows the API result shape. Default corpus behavior is covered on the Highlights page. Related: [Highlights](/docs/fetch/highlights) · [Output Model](/docs/fetch/output-model) · [Playground & Replay](/docs/fetch/playground-and-replay) · [`fetch.fetchJson`](/api-reference/fetch/fetch.fetchJson) ### `expandai search ` [#expandai-search-snapshotid-query] Search an existing snapshot without fetching the page again. ```bash expandai search ``` ```bash expandai search 019... "return policy" ``` | Argument | Required | Description | | ------------ | -------- | ------------------------------- | | `snapshotId` | Yes | Existing snapshot ID to search. | | `query` | Yes | Search query for Highlights. | | Option | Values | Default | Description | | --------------- | -------------- | ------- | --------------------------- | | `--format` | `text`, `json` | `text` | Controls CLI output format. | | `--max-results` | `1` to `50` | `5` | Maximum snippets. | | `--min-score` | `0` to `1` | `0.6` | Minimum relevance score. | Use `search` for snapshot reuse: it queries a page you already captured and skips the network round trip. If you do not have a `snapshotId` yet, run `expandai fetch --search ` instead. Related: [Highlights](/docs/fetch/highlights) · [Playground & Replay](/docs/fetch/playground-and-replay) · [`fetch.fetchSearch` (`/v1/fetch/search`)](/api-reference/fetch/fetch.fetchSearch) ## Auth and configuration [#auth-and-configuration] ### `expandai login` [#expandai-login] Store local credentials for CLI and local integration use. interactive ```bash expandai login ``` The command is interactive and supports two paths: browser login and API-key login. API-key auth is the normal production and server path. Credentials are written to the local Expand CLI config file (see [Environment and config](#environment-and-config)). Do not paste API keys into shell history; let `login` prompt for them. ### `expandai whoami` [#expandai-whoami] Check which Expand account or credential the CLI is using. ```bash expandai whoami ``` This is the first command to run when Fetch, MCP, skills, or hooks fail because of auth. If `whoami` shows the wrong account or no credential, run `expandai login`. ### `expandai logout` [#expandai-logout] Clear local stored Expand credentials. ```bash expandai logout ``` ### Environment and config [#environment-and-config] | Name | Type | Description | | -------------------------------- | ------- | ---------------------------------------------------------------------- | | `EXPAND_API_KEY` | env var | API key override for CLI and API clients. | | `EXPAND_API_URL` | env var | API base URL override, mainly for staging or self-hosted internal use. | | `XDG_CONFIG_HOME` | env var | Changes where the CLI stores `expandai/config.json`. | | `~/.config/expandai/config.json` | file | Default local config path when `XDG_CONFIG_HOME` is not set. | Authored cURL examples authenticate with the API-key header, not bearer: ```bash x-expand-api-key: $EXPAND_API_KEY ``` ## Agent setup [#agent-setup] These commands wire Expand into local agent hosts: MCP config, the Expand Skill, and native-fetch hooks. The per-host quickstarts own the recommended setup and verification flow — [Claude Code](/docs/agent-quickstarts/claude-code), [Cursor](/docs/agent-quickstarts/cursor), [Codex](/docs/agent-quickstarts/codex), and [OpenCode](/docs/agent-quickstarts/opencode). agent setup Install and uninstall commands take an explicit `` of `global` or `project`. Project scope writes files into the current working directory, so run these from the project root you want them in. ### `expandai mcp run [--docs]` [#expandai-mcp-run---docs] Run the local Expand MCP server over stdio. ```bash expandai mcp run ``` ```bash expandai mcp run --docs ``` MCP clients and generated MCP config launch this command; you rarely run it by hand. Plain `mcp run` exposes the Fetch tools. `--docs` additionally exposes Expand docs resources. First-class installers do not enable `--docs` by default. Full tool schemas live in [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources); there is no hosted or remote Expand MCP server today. ### `expandai mcp install ` [#expandai-mcp-install-host-scope] Install Expand MCP config for a supported local agent host. ```bash expandai mcp install ``` Hosts: `claude-code`, `cursor`, `opencode`, `codex`. Scopes: `global`, `project`. ```bash expandai mcp install codex project expandai mcp install cursor project ``` ### `expandai mcp uninstall ` [#expandai-mcp-uninstall-host-scope] Remove Expand MCP config for a supported host and scope. ```bash expandai mcp uninstall codex project ``` ### `expandai skill install ` [#expandai-skill-install-host-scope] Install the Expand Skill for a supported agent host. ```bash expandai skill install ``` Hosts: `claude-code`, `cursor`, `opencode`, `codex`. Scopes: `global`, `project`. ```bash expandai skill install codex project expandai skill install claude-code project ``` Skills teach agents how to use Expand. Project scope writes skill files into the current project — commit them only if you want the skill enabled for collaborators. The command requires an explicit scope; it does not default to `project`. Detailed behavior lives on the [Expand Skill](/docs/agent-quickstarts/expand-skill) page. ### `expandai skill uninstall ` [#expandai-skill-uninstall-host-scope] Remove the Expand Skill for a host and scope. ```bash expandai skill uninstall codex project ``` ### `expandai hook install ` [#expandai-hook-install-host-scope] Install an Expand hook for hosts that can route native web-fetch behavior through Expand. ```bash expandai hook install ``` Hosts: `claude-code`, `opencode`. Scopes: `global`, `project`. ```bash expandai hook install claude-code project expandai hook install opencode project ``` Hook behavior is host-specific. The host pages own the full setup and verification flow. ### `expandai hook uninstall ` [#expandai-hook-uninstall-host-scope] Remove an Expand hook for a host and scope. ```bash expandai hook uninstall opencode project ``` ### `expandai hook claude-code` and `expandai hook opencode` [#expandai-hook-claude-code-and-expandai-hook-opencode] Runtime commands invoked by the Claude Code and OpenCode hook integrations. integration runtime ```bash expandai hook claude-code expandai hook opencode ``` These are integration-facing, not primary user commands. Most users should `hook install` or `hook uninstall` and never call them directly — the host invokes them when a hook fires. ## Cleanup [#cleanup] ### `expandai uninstall [target] [host] [scope]` [#expandai-uninstall-target-host-scope] Remove Expand integration files across hooks, skills, and MCP config. ```bash expandai uninstall [target] [host] [scope] ``` * `target` selects a surface such as all, hook, skill, or MCP. * `host` narrows cleanup to a supported host. * `scope` narrows cleanup to `global` or `project` files. * With no arguments, the command uses broad cleanup defaults. ```bash expandai uninstall expandai uninstall mcp codex project expandai uninstall skill claude-code project ``` Project-scoped cleanup depends on the current working directory. Run cleanup from the same project root you used during installation, or it will not find the files to remove. ## Related pages [#related-pages] | Topic | Page | | ----------------------- | ---------------------------------------------------------------- | | Install and first fetch | [Quickstart](/docs/get-started/quickstart) | | Human vs. agent routing | [Ways to Use Expand](/docs/get-started/ways-to-use-expand) | | Fetch product behavior | [Fetch Overview](/docs/fetch) | | Output fields | [Output Model](/docs/fetch/output-model) | | Include controls | [Include Options](/docs/fetch/include-options) | | Highlights | [Highlights](/docs/fetch/highlights) | | Citations and replay | [Playground & Replay](/docs/fetch/playground-and-replay) | | Agent setup | [Agent Quickstarts](/docs/agent-quickstarts) | | Skill behavior | [Expand Skill](/docs/agent-quickstarts/expand-skill) | | MCP schemas | [MCP Tools & Resources](/docs/reference/mcp-tools-and-resources) | | Endpoint contracts | [API Reference](/docs/reference/api-reference) | --- # MCP Tools & Resources (https://staging.expand.ai/docs/reference/mcp-tools-and-resources) The Expand MCP server gives an agent two native Fetch tools — `fetch` and `fetch_search` — and, with a flag, two documentation resources. It runs locally over stdio: ```bash expandai mcp run ``` Most clients should launch this command from their MCP config, not from a terminal. Authenticate once with `expandai login`; the server reuses your local CLI credentials. ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run"] } } } ``` Different clients name the surrounding fields differently. See [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) for JSON and TOML variants, or use a first-class installer for [Claude Code](/docs/agent-quickstarts/claude-code), [Cursor](/docs/agent-quickstarts/cursor), [Codex](/docs/agent-quickstarts/codex), or [OpenCode](/docs/agent-quickstarts/opencode). ## Tools [#tools] The server registers two tools. Both are read-only and non-destructive. | Tool | What it does | | -------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `fetch` | Captures a URL in a browser and returns Markdown, optional page metadata, State JSON, appendix, and optional Highlights snippets. | | `fetch_search` | Runs Highlights against an existing snapshot by `snapshotId`, without capturing the page again. | ### `fetch` [#fetch] Fetches and extracts content from any URL. The response is object-mode output: rendered Markdown, front-matter metadata, and the extracted State JSON evidence. When you pass `search`, the response also carries ranked snippets under `data.search.snippets`. | Parameter | Type | Default | Description | | ----------------- | ------- | ------- | ------------------------------------------------------------------------------------- | | `url` | string | — | The URL to fetch content from. | | `search` | object | omitted | Highlights configuration (see below). When set, the response returns ranked snippets. | | `includeMeta` | boolean | `false` | Include page meta tags (title, description, Open Graph). | | `includeAppendix` | boolean | `false` | Include extracted links and sidebar content. | | `includeJson` | boolean | `true` | Include State JSON extracted from network responses and the DOM. | The `search` object takes: | Field | Type | Default | Description | | ------------ | -------------- | ------- | ------------------------------------------------------------------------- | | `query` | string | — | Highlights query. When provided, `fetch` returns snippets for this query. | | `maxResults` | integer (1–50) | `5` | Maximum snippets to return. | | `minScore` | number (0–1) | `0` | Minimum snippet relevance score. | Example call: ```json { "name": "fetch", "arguments": { "url": "https://news.ycombinator.com", "includeMeta": true, "search": { "query": "top front-page stories", "maxResults": 5 } } } ``` The result includes `meta` (with `snapshotId` and a `playground` link), `markdown`, the State JSON `json` evidence array, and — because `search` was set — `data.search.snippets`. Keep the returned `snapshotId` if you want to run more Highlights queries later without re-capturing the page. ### `fetch_search` [#fetch_search] Searches a previous fetch by `snapshotId` without starting a new capture. Use it after a `fetch` call when you need additional evidence from the same page — it reuses the stored artifacts instead of paying for another browser capture. | Parameter | Type | Default | Description | | ----------------- | ------- | ------- | --------------------------------------------------------------------------------------------------- | | `snapshotId` | string | — | Snapshot ID returned by a previous `fetch` response. | | `search` | object | — | Highlights configuration: `query`, `maxResults` (1–50, default `5`), `minScore` (0–1, default `0`). | | `includeMarkdown` | boolean | `true` | Search the stored Markdown corpus. | | `includeAppendix` | boolean | `false` | Search extracted appendix content. | | `includeJson` | boolean | `true` | Search stored State JSON. | The result returns the `snapshotId`, the ranked `search` snippets, a `durationMs` timing, and a whole-snapshot `playground` link. ### Snippets and citations [#snippets-and-citations] Every Highlights snippet — from `fetch` or `fetch_search` — carries a `citationUrl`: a public Playground link that opens the exact passage so a human can audit it. Snippets also expose: | Field | Description | | ------------- | -------------------------------------------------------------------- | | `source` | Where the snippet came from: `markdown`, `appendix`, or `statejson`. | | `text` | The matched passage. | | `json` | The extracted value, present when `source` is `statejson`. | | `score` | Relevance score from 0 to 1. | | `citationUrl` | Public Playground link auditing this exact snippet. | Return the `citationUrl` to the user verbatim whenever you ground a claim in fetched content, so the source stays verifiable. ## Optional documentation resources [#optional-documentation-resources] Add `--docs` to also expose two local documentation resources alongside the tools: ```bash expandai mcp run --docs ``` | Resource | Name | Contents | | ---------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------- | | `expand://about` | About expand.ai | Key links: website, docs, and dashboard. | | `expand://start` | Expand Start Guide | A local copy of the agent and editor setup guide, including the `expandai login` step and per-host install commands. | First-class installers do not enable `--docs` by default. Do not assume `expand://start` exists unless the configured MCP command includes `--docs`. ```json { "mcpServers": { "expandai": { "type": "stdio", "command": "expandai", "args": ["mcp", "run", "--docs"] } } } ``` ## Authentication [#authentication] For local MCP clients, run `expandai login` once in your terminal. The server uses the same local Expand credentials as the CLI, so there is no API key in your config: ```bash expandai login expandai whoami ``` Keep `EXPAND_API_KEY` out of the primary local config. API keys belong in CI, servers, direct API calls, and other non-interactive setups. For environment-variable configuration, see [CLI Commands](/docs/reference/cli-commands). ## How the tools map to the API [#how-the-tools-map-to-the-api] The MCP tools are a thin surface over the Fetch HTTP API. `fetch` calls JSON-mode Fetch; `fetch_search` calls the Highlights endpoint. For exact request and response schemas, error responses, and generated examples, open the [API Reference](/api-reference) — specifically [Fetch (JSON mode)](/api-reference/fetch/fetch.fetchJson) and [Highlights](/api-reference/fetch/fetch.fetchSearch). ## Related pages [#related-pages] * [Other MCP Clients](/docs/agent-quickstarts/other-mcp-clients) * [CLI Commands](/docs/reference/cli-commands) * [Highlights](/docs/fetch/highlights) * [Output Model](/docs/fetch/output-model) --- # Python SDK (https://staging.expand.ai/docs/reference/python-sdk) Use `expandai` to call Fetch from Python: Main Markdown, State JSON, Highlights, Batched Fetch, async clients, and citation helpers. The package requires Python 3.10 or newer. ```bash pip install expandai ``` ```bash uv add expandai ``` ```bash poetry add expandai ``` `fetch_json()` is the method you reach for first. It reads Main Markdown and State JSON in one call: ```python from expandai import Expand, FetchJsonParams with Expand() as expand: page = expand.fetch_json(FetchJsonParams(url="https://news.ycombinator.com")) main_markdown = page.markdown state_json = page.json snapshot_id = page.meta.snapshot_id playground = page.meta.playground ``` Use `fetch_json()` for application code that needs Main Markdown plus State JSON. Use `fetch()` when you only want the Markdown string. Jump to a section: * [JSON Mode](#json-mode-with-fetch_json) * [Markdown-only Fetch](#markdown-only-fetch-with-fetch) * [Highlights](#highlights) * [Batched Fetch](#batched-fetch) * [Async](#async-client) * [Raw Responses](#raw-responses) * [Citation Helpers](#citation-helpers) * [Errors](#errors-and-retries) ## Install and auth [#install-and-auth] The SDK reads your key from `EXPAND_API_KEY`: ```bash export EXPAND_API_KEY="xpnd_..." ``` With the variable set, construct a client and let it pick up the key. The `Expand` client owns an `httpx.Client`, so close it with a context manager or `client.close()`: ```python from expandai import Expand with Expand() as expand: ... ``` Pass the key explicitly when you manage configuration yourself: ```python import os from expandai import Expand with Expand(api_key=os.environ["EXPAND_API_KEY"]) as expand: ... ``` ### Constructor options [#constructor-options] | Option | Default | Description | | ------------- | ------------------------------------------------ | ---------------------------------------- | | `api_key` | `os.environ["EXPAND_API_KEY"]` | Expand API key. | | `base_url` | `https://api.expand.ai` | API base URL. | | `timeout_ms` | `60000` | Total wall-clock budget across attempts. | | `max_retries` | `2` | Retry attempts for retryable failures. | | `http_client` | SDK-created `httpx.Client` / `httpx.AsyncClient` | Optional custom client. | The SDK sends your key on every request using the `x-expand-api-key` header. You never set that header by hand. ## Method index [#method-index] | Method | Use when | Returns | | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ----------------------------- | | `fetch_json(body, include=None, request_options=None)` | App code needs Main Markdown, State JSON, snapshot metadata, or inline Highlights. | Object-mode Fetch model. | | `fetch(body, include=None, request_options=None)` | You only need the Markdown string. | `str` | | `fetch_search(body, request_options=None)` | You already have `snapshot_id` and want Highlights from stored artifacts. | Snapshot Highlights model. | | `batched(body, *, idempotency_key=None, request_options=None)` | You want to start many async Fetch jobs. The SDK generates `idempotency_key` when omitted. | Batched run model with `id`. | | `get_batched(id, options=None)` | You want to poll a Batched Fetch run. | Batched status/results model. | `AsyncExpand` exposes the same methods as `async` methods. The names and model classes match the sync client one to one. ## Models and serialization [#models-and-serialization] The public SDK exports the request and response models you need from `expandai`. You should not import from `expandai._generated` in normal application code. ```python from expandai import ( FetchParams, FetchJsonParams, FetchSearchParams, FetchJsonSearch, FetchSearchQuery, BatchedParams, to_dict, to_json, ) ``` * Models use Python snake\_case attributes, such as `snapshot_id`, `captured_at`, and `max_results`. * `to_dict(model)` serializes a model to a wire-format dictionary. * `to_json(model, indent=2)` is handy when you want to inspect a request body while debugging. * Exact generated field lists belong to the [API Reference](/api-reference), not this page. ## JSON mode with `fetch_json` [#json-mode-with-fetch_json] `fetch_json()` is the recommended SDK method for applications. It returns the Main Markdown, State JSON, snapshot metadata, and optional search results in one model. ```python from expandai import Expand, FetchJsonParams with Expand() as expand: page = expand.fetch_json(FetchJsonParams(url="https://example.com")) print(page.markdown) print(page.json) print(page.meta.snapshot_id) print(page.meta.playground) ``` The fields you read most often (abbreviated — see the [Output Model](/docs/fetch/output-model) for the full shape): ```python page.meta.snapshot_id # snapshot handle for follow-up search page.meta.playground # human inspection URL page.markdown # Main Markdown page.json # State JSON / extracted evidence page.data # optional extra data such as search results ``` * `markdown` is the Main Markdown. * `json` is the State JSON and extracted evidence. * `meta.snapshot_id` is the handle you pass to later Highlights. * `meta.playground` is the human inspection link. * `data.search` appears only when you request inline search. This list is not exhaustive. The [Output Model](/docs/fetch/output-model) and [API Reference](/api-reference) own the exact fields. ## Markdown-only Fetch with `fetch` [#markdown-only-fetch-with-fetch] `fetch()` is the convenience method for scripts and agents that only need the Markdown string. ```python from expandai import Expand, FetchParams with Expand() as expand: markdown = expand.fetch(FetchParams(url="https://example.com")) print(markdown) ``` * The return type is `str`. * This calls `/v1/fetch`. * Reach for `fetch_json()` when the application needs State JSON, snapshot metadata, or structured search results. ## Include options [#include-options] Pass `include` as a keyword argument to widen what a Fetch returns: ```python markdown = expand.fetch( FetchParams(url="https://example.com"), include="appendix,statejson", ) ``` ```python page = expand.fetch_json( FetchJsonParams(url="https://example.com"), include="appendix", ) ``` Body include models such as `FetchJsonInclude` are available when you prefer structured request-body controls. The exact include semantics belong to [Include Options](/docs/fetch/include-options) and the [API Reference](/api-reference) — this page does not repeat the full matrix. ## Highlights [#highlights] Highlights are search over a captured page. Run them inline with a fresh Fetch, or against a snapshot you already captured. ### Inline Highlights with `fetch_json` [#inline-highlights-with-fetch_json] ```python from expandai import Expand, FetchJsonParams, FetchJsonSearch with Expand() as expand: page = expand.fetch_json( FetchJsonParams( url="https://docs.example.com", search=FetchJsonSearch( query="authentication limits", max_results=5, min_score=0.6, ), ) ) snippets = page.data.search.snippets if page.data and page.data.search else [] ``` * This starts a new Fetch and searches it in the same call. * Snippets live under `page.data.search.snippets`. * Python attributes use `max_results`, not `maxResults`. * State JSON snippets may include `json`. * Raw SDK snippets expose `location`, not the MCP-only `citationUrl` field. ### Snapshot Highlights with `fetch_search` [#snapshot-highlights-with-fetch_search] ```python from expandai import Expand, FetchJsonParams, FetchSearchParams, FetchSearchQuery with Expand() as expand: page = expand.fetch_json(FetchJsonParams(url="https://docs.example.com")) result = expand.fetch_search( FetchSearchParams( snapshot_id=page.meta.snapshot_id, search=FetchSearchQuery( query="authentication limits", max_results=5, min_score=0.6, ), ) ) print(result.search.snippets) ``` * `fetch_search` calls `/v1/fetch/search`. * It searches stored artifacts without recapturing the URL. * Use it to refine a query after a previous Fetch. * The exact request schema belongs to the [API Reference](/api-reference). See [Highlights](/docs/fetch/highlights) for behavior. ## Citation helpers [#citation-helpers] Raw SDK results expose snippet `location`. Use Playground helpers when you want to turn a snippet into a user-visible citation link. ```python from expandai import DEFAULT_PLAYGROUND_HOST, resolve_playground_host, snippet_citation_url ``` ```python host = resolve_playground_host(page.meta.playground, DEFAULT_PLAYGROUND_HOST) for snippet in snippets: url = snippet_citation_url(page.meta.snapshot_id, snippet, host) print(snippet.text, url) ``` | Helper | Purpose | | ------------------------------------------------------ | ----------------------------------------------------------------- | | `playground_base(snapshot_id, host=...)` | Build the whole-snapshot Playground URL. | | `playground_origin(meta_playground)` | Extract the origin from `meta.playground`. | | `resolve_playground_host(meta_playground, fallback)` | Prefer the server-provided Playground host, fall back if missing. | | `citation_url(snapshot_id, location, host=...)` | Build a citation link from `location.evidence_id`. | | `snippet_citation_url(snapshot_id, snippet, host=...)` | Build a citation link from a snippet model. | SDK API responses do not already contain `citationUrl`. MCP adds that field; Python SDK users build it with these helpers. See [Playground & Replay](/docs/fetch/playground-and-replay) for URL semantics. ## Batched Fetch [#batched-fetch] `batched()` starts many Fetch jobs at once; `get_batched()` polls the run until it finishes: ```python import time from expandai import Expand, BatchedParams with Expand(timeout_ms=120_000) as expand: run = expand.batched( BatchedParams( urls=[ "https://example.com", "https://example.com/about", ] ), ) status = expand.get_batched(run.id, options={"limit": 10, "offset": 0}) while str(status.batched_status) in {"QUEUED", "RUNNING"}: time.sleep(1) status = expand.get_batched(run.id, options={"limit": 10, "offset": 0}) print(status.results) ``` * `batched()` returns a run model with `id`. * `get_batched()` polls the run. * Each result includes its persisted `status` alongside `data`, so handle failed or cancelled URLs independently. * Python `get_batched` options accept numeric `limit` and `offset`; the SDK converts them to strings for the generated request internally. * When omitted, the SDK generates one `idempotency_key` and reuses it for the configured retry policy. * Supply a stable `idempotency_key` to deduplicate separate calls or calls made after a process restart. * Reusing a key with a different payload returns `409 BatchedIdempotencyConflict` with `reason: "payload_mismatch"`. * The full lifecycle belongs to [Batched Fetch](/docs/fetch/batched-fetch). ## Async client [#async-client] Use `AsyncExpand` in asyncio applications. It exposes the same operations as `Expand`, but each request method is awaited. ```python import asyncio from expandai import AsyncExpand, FetchJsonParams async def main() -> None: async with AsyncExpand() as expand: page = await expand.fetch_json(FetchJsonParams(url="https://example.com")) print(page.markdown) asyncio.run(main()) ``` * Use `async with` to close the underlying `httpx.AsyncClient`. * The method names and model classes match the sync client. * `with_raw_response` also exists on `AsyncExpand`. ## Raw responses [#raw-responses] Use `with_raw_response` when you need headers, status codes, request IDs, or the underlying `httpx.Response`. ```python from expandai import Expand, FetchJsonParams with Expand() as expand: response = expand.with_raw_response.fetch_json( FetchJsonParams(url="https://example.com") ) print(response.status_code) print(response.headers) page = response.parse() ``` The raw response is an `APIResponse`: | Field | Meaning | | --------------- | ---------------------------- | | `parsed` | Parsed SDK model. | | `http_response` | Underlying `httpx.Response`. | | `status_code` | HTTP status code. | | `headers` | Response headers. | | `request_id` | Request ID when present. | | `parse()` | Returns `parsed`. | ## Errors and retries [#errors-and-retries] Every SDK error subclasses `ExpandError`: | Error | Meaning | | ----------------------- | ------------------------------------------------------------------------------------------------- | | `ExpandError` | Base SDK error. | | `ExpandAPIError` | Non-2xx API response or failed response parsing. Inspect `status_code`, `body`, and `request_id`. | | `ExpandSdkError` | Invalid SDK options or setup error. | | `ExpandTimeoutError` | Request exceeded `timeout_ms`. | | `ExpandConnectionError` | Network or HTTP transport failure. | Retry and timeout defaults: * Default `timeout_ms` is `60000`. * Default `max_retries` is `2`. * Retryable failures include timeouts, connection errors, and HTTP statuses `408`, `409`, `429`, `500`, `502`, `503`, and `504`. ```python from expandai import Expand, ExpandAPIError, FetchJsonParams with Expand() as expand: try: page = expand.fetch_json(FetchJsonParams(url="https://example.com")) except ExpandAPIError as error: print(error.status_code) print(error.body) raise ``` ## Related pages [#related-pages] | Topic | Page | | ------------------------------------ | ---------------------------------------------------------- | | First SDK or API setup | [Quickstart](/docs/get-started/quickstart) | | Choosing the SDK vs CLI, MCP, or API | [Ways to Use Expand](/docs/get-started/ways-to-use-expand) | | Product behavior | [Fetch Overview](/docs/fetch) | | Main Markdown and State JSON | [Output Model](/docs/fetch/output-model) | | Include semantics | [Include Options](/docs/fetch/include-options) | | Highlights behavior | [Highlights](/docs/fetch/highlights) | | Citation URLs and replay | [Playground & Replay](/docs/fetch/playground-and-replay) | | Batched lifecycle | [Batched Fetch](/docs/fetch/batched-fetch) | | Exact endpoint schemas | [API Reference](/api-reference) | | TypeScript equivalent | [TypeScript SDK](/docs/reference/typescript-sdk) | --- # TypeScript SDK (https://staging.expand.ai/docs/reference/typescript-sdk) Use `@expandai/sdk` to call Fetch from TypeScript: Main Markdown, State JSON, Highlights, Batched Fetch, and citation helpers. ```bash npm install @expandai/sdk ``` ```bash pnpm add @expandai/sdk ``` ```bash bun add @expandai/sdk ``` ```ts import { ExpandClient } from '@expandai/sdk' const expand = new ExpandClient({ apiKey: process.env.EXPAND_API_KEY }) const page = await expand.fetchJson({ url: 'https://news.ycombinator.com', }) const mainMarkdown = page.markdown const stateJson = page.json const snapshotId = page.meta.snapshotId const playground = page.meta.playground ``` Use `fetchJson()` for application code that needs Main Markdown plus State JSON. Use `fetch()` when you only want the Markdown string. Jump to: [JSON Mode](#json-mode-with-fetchjson) · [Markdown-only Fetch](#markdown-only-fetch-with-fetch) · [Highlights](#highlights) · [Batched Fetch](#batched-fetch) · [Citation Helpers](#citation-helpers) · [Errors](#errors-and-retries) · [Effect](#effect-service) ## Install and auth [#install-and-auth] Install the package, then set your API key. The SDK reads `EXPAND_API_KEY` from the environment by default. ```bash export EXPAND_API_KEY="xpnd_..." ``` ```ts import { ExpandClient } from '@expandai/sdk' const expand = new ExpandClient({ apiKey: process.env.EXPAND_API_KEY, }) ``` The SDK sends your key on every request as the `x-expand-api-key` header. | Option | Default | Description | | ------------ | ---------------------------- | ---------------------------------------- | | `apiKey` | `process.env.EXPAND_API_KEY` | Expand API key. | | `baseUrl` | `https://api.expand.ai` | API base URL. | | `timeoutMs` | `60000` | Total wall-clock budget across attempts. | | `maxRetries` | `2` | Retry attempts for retryable failures. | ## Method index [#method-index] | Method | Use when | Returns | | ------------------------------- | ------------------------------------------------------------------------------------------------- | --------------------------- | | `fetchJson(params, options?)` | App code needs Main Markdown, State JSON, snapshot metadata, or inline Highlights. | Object-mode Fetch result. | | `fetch(params, options?)` | You only need the Markdown string. | `string` | | `fetchSearch(params, options?)` | You already have `snapshotId` and want Highlights from stored artifacts. | Snapshot Highlights result. | | `batched(params, options?)` | You want to start many async Fetch jobs. The SDK generates `options.idempotencyKey` when omitted. | `{ id }` | | `getBatched(id, options?)` | You want to poll a Batched Fetch run. | Batched status/results. | ## JSON Mode with `fetchJson` [#json-mode-with-fetchjson] `fetchJson()` is the recommended SDK method for applications. It returns the Main Markdown, State JSON, snapshot metadata, and optional search results in one object. ```ts const page = await expand.fetchJson({ url: 'https://example.com', }) console.log(page.markdown) console.log(page.json) console.log(page.meta.snapshotId) console.log(page.meta.playground) ``` The result shape, abbreviated: ```ts type FetchJsonResult = { meta: { snapshotId: string playground: string url: string capturedAt: string // ... } markdown: string json: Array data?: { search?: { query: string snippets: Array<{ source: 'markdown' | 'appendix' | 'statejson' text: string json?: unknown score: number location?: unknown }> } } } ``` * `markdown` is the Main Markdown. * `json` is State JSON and extracted evidence. * `meta.snapshotId` is the handle for later Highlights with `fetchSearch`. * `meta.playground` is the human inspection link. * `data.search` appears when inline search is requested. This shape is not exhaustive. See [Output Model](/docs/fetch/output-model) and the [API Reference](/api-reference) for exact fields. ## Markdown-only Fetch with `fetch` [#markdown-only-fetch-with-fetch] `fetch()` is the convenience method for scripts and agents that only need the Markdown string. ```ts const markdown = await expand.fetch({ url: 'https://example.com', }) console.log(markdown) ``` * The return type is `string`. * This calls `/v1/fetch`. * Use `fetchJson()` if the application needs State JSON, snapshot metadata, or structured search results. ## Include options [#include-options] Both `fetch` and `fetchJson` accept query include options through the second argument. ```ts const markdownWithAppendix = await expand.fetch( { url: 'https://example.com' }, { include: 'appendix,statejson' }, ) ``` ```ts const objectMode = await expand.fetchJson( { url: 'https://example.com' }, { include: 'appendix' }, ) ``` * `include` is a query option passed as the second argument. * `fetchJson` body include options are available through the request body when you need exact structured control. * Passing `{ include: null }` omits the include query param. Exact include semantics belong to [Include Options](/docs/fetch/include-options) and the [API Reference](/api-reference). ## Highlights [#highlights] Highlights ("search") run two ways: inline with a fresh Fetch, or against a stored snapshot. ### Inline Highlights with `fetchJson` [#inline-highlights-with-fetchjson] ```ts const page = await expand.fetchJson({ url: 'https://docs.example.com', search: { query: 'authentication limits', maxResults: 5, minScore: 0.6, }, }) const snippets = page.data?.search?.snippets ?? [] ``` * This starts a new Fetch. * Snippets live under `data.search.snippets`. * State JSON snippets may include `json`. * Raw SDK snippets expose `location`, not the MCP-only `citationUrl` field. ### Snapshot Highlights with `fetchSearch` [#snapshot-highlights-with-fetchsearch] ```ts const page = await expand.fetchJson({ url: 'https://docs.example.com', }) const result = await expand.fetchSearch({ snapshotId: page.meta.snapshotId, search: { query: 'authentication limits', maxResults: 5, minScore: 0.6, }, include: { markdown: true, json: true, appendix: false, }, }) console.log(result.search.snippets) ``` * `fetchSearch` calls `/v1/fetch/search`. * It searches stored artifacts without recapturing the URL. * Use it to refine results after a previous Fetch. * The exact request schema belongs to the [API Reference](/api-reference). See [Highlights](/docs/fetch/highlights) for behavior. ## Citation Helpers [#citation-helpers] Raw SDK results expose snippet `location`. Use Playground helpers when you want to turn a snippet into a user-visible citation link. ```ts import { citationUrl, resolvePlaygroundHost } from '@expandai/sdk/Playground' ``` ```ts const host = resolvePlaygroundHost(page.meta.playground, 'https://expand.land') for (const snippet of page.data?.search?.snippets ?? []) { const url = citationUrl(page.meta.snapshotId, snippet.location, host) console.log(snippet.text, url) } ``` | Helper | Purpose | | ------------------------------------------------- | ----------------------------------------------------------------- | | `playgroundBase(snapshotId, host?)` | Build the whole-snapshot Playground URL. | | `playgroundOrigin(metaPlayground)` | Extract the origin from `meta.playground`. | | `resolvePlaygroundHost(metaPlayground, fallback)` | Prefer the server-provided Playground host, fall back if missing. | | `citationUrl(snapshotId, location, host?)` | Build a citation link from `location.evidenceId`. | SDK API responses do not already contain `citationUrl`. MCP adds that field; SDK users build it with these helpers. See [Playground & Replay](/docs/fetch/playground-and-replay) for URL semantics. ## Batched Fetch [#batched-fetch] ```ts const run = await expand.batched({ urls: [ 'https://example.com', 'https://example.com/about', ], }) let status = await expand.getBatched(run.id, { limit: '10', offset: '0' }) while (status.batchedStatus === 'QUEUED' || status.batchedStatus === 'RUNNING') { await new Promise((resolve) => setTimeout(resolve, 1000)) status = await expand.getBatched(run.id, { limit: '10', offset: '0' }) } console.log(status.results) ``` * `batched()` returns a run ID. * `getBatched()` polls the run. * Each result includes its persisted `status` alongside `data`, so handle failed or cancelled URLs independently. * When omitted, the SDK generates one `idempotencyKey` and reuses it for the configured retry policy. * Supply a stable `idempotencyKey` to deduplicate separate calls or calls made after a process restart. * Reusing a key with a different payload returns `409 BatchedIdempotencyConflict` with `reason: 'payload_mismatch'`. * The current generated TypeScript `getBatched` query params use strings for `limit`, `offset`, and `include`. The full lifecycle belongs to [Batched Fetch](/docs/fetch/batched-fetch). ## Effect Service [#effect-service] Use `ExpandService` when your application is already Effect-native. It exposes the same operations as `ExpandClient`, but failures are typed in the Effect error channel. ```ts import { NodeRuntime } from '@effect/platform-node' import { Effect } from 'effect' import { ExpandService } from '@expandai/sdk' const program = Effect.gen(function* () { const expand = yield* ExpandService const page = yield* expand.fetchJson({ url: 'https://example.com' }) yield* Effect.log(page.markdown) }) NodeRuntime.runMain( program.pipe( Effect.provide( ExpandService.layer({ apiKey: process.env.EXPAND_API_KEY }), ), ), ) ``` You do not need this section to use the SDK from plain Promise code. ## Errors and retries [#errors-and-retries] Promise-side errors, thrown by `ExpandClient`: | Error | Meaning | | ----------------------------- | ------------------------------------------------------- | | `ExpandClientError` | Base Promise-side SDK error and invalid client options. | | `ExpandClientApiError` | Non-2xx API response. Inspect `status` and `body`. | | `ExpandClientConnectionError` | Network failure. | | `ExpandClientTimeoutError` | Request exceeded `timeoutMs`. | Effect-side errors, surfaced in the `ExpandService` error channel: | Error | Meaning | | ----------------------- | ------------------------------------------- | | `ExpandSdkError` | Invalid service options or SDK setup error. | | `ExpandApiError` | Non-2xx API response. | | `ExpandConnectionError` | Network failure. | | `ExpandTimeoutError` | Request exceeded `timeoutMs`. | Retry and timeout behavior: * Default `timeoutMs` is `60000`. * Default `maxRetries` is `2`. * Retryable failures include timeouts, connection errors, `408`, `409`, `429`, and `5xx`. ```ts import { ExpandClientApiError } from '@expandai/sdk' try { await expand.fetchJson({ url: 'https://example.com' }) } catch (error) { if (error instanceof ExpandClientApiError) { console.error(error.status, error.body) } throw error } ``` ## Related pages [#related-pages] | Topic | Where to go | | ---------------------------- | ---------------------------------------------------------- | | First SDK or API setup | [Quickstart](/docs/get-started/quickstart) | | Choosing SDK vs CLI/MCP/API | [Ways to Use Expand](/docs/get-started/ways-to-use-expand) | | Product behavior | [Fetch Overview](/docs/fetch) | | Main Markdown and State JSON | [Output Model](/docs/fetch/output-model) | | Include semantics | [Include Options](/docs/fetch/include-options) | | Highlights behavior | [Highlights](/docs/fetch/highlights) | | Citation URLs and replay | [Playground & Replay](/docs/fetch/playground-and-replay) | | Batched lifecycle | [Batched Fetch](/docs/fetch/batched-fetch) | | Exact endpoint schemas | [API Reference](/api-reference) | | Python equivalent | [Python SDK](/docs/reference/python-sdk) |