Expand AI logo
DocsDocs
Glow Active
API Reference
Login

Documentation

Get Started

OverviewWhy ExpandQuickstartWays to Use Expand

Agent Quickstarts

OverviewExpand SkillClaude CodeCursorCodexOpenCodeSkill-Based AgentsOther MCP Clients

Fetch

OverviewHow Fetch WorksOutput ModelInclude OptionsBrowser BehaviorHighlightsPlayground & ReplayBatched Fetch

Reference

API ReferenceCLI CommandsMCP Tools & ResourcesTypeScript SDKPython SDK

Account & Billing

Pricing & UsageTiersFAQ

Machine-Readable Docs

start.mdllms.txtllms-full.txtDocs as Markdown
Browse docs

Get Started

OverviewWhy ExpandQuickstartWays to Use Expand

Agent Quickstarts

OverviewExpand SkillClaude CodeCursorCodexOpenCodeSkill-Based AgentsOther MCP Clients

Fetch

OverviewHow Fetch WorksOutput ModelInclude OptionsBrowser BehaviorHighlightsPlayground & ReplayBatched Fetch

Reference

API ReferenceCLI CommandsMCP Tools & ResourcesTypeScript SDKPython SDK

Account & Billing

Pricing & UsageTiersFAQ

Machine-Readable Docs

start.mdllms.txtllms-full.txtDocs as Markdown

Python SDK

Call Fetch from Python with the expandai package — Main Markdown, State JSON, Highlights, Batched Fetch, async clients, and citation helpers.

Use expandai to call Fetch from Python: Main Markdown, State JSON, Highlights, Batched Fetch, async clients, and citation helpers. The package requires Python 3.10 or newer.

pip install expandai
uv add expandai
poetry add expandai

fetch_json() is the method you reach for first. It reads Main Markdown and State JSON in one call:

from expandai
import
Expand, FetchJsonParams
with Expand() as expand:
page = expand.fetch_json(FetchJsonParams(url="https://news.ycombinator.com"))
main_markdown = page.markdown
state_json = page.json
snapshot_id = page.meta.snapshot_id
playground = page.meta.playground

Use fetch_json() for application code that needs Main Markdown plus State JSON. Use fetch() when you only want the Markdown string.

Jump to a section:

  • JSON Mode
  • Markdown-only Fetch
  • Highlights
  • Batched Fetch
  • Async
  • Raw Responses
  • Citation Helpers
  • Errors

Install and auth

The SDK reads your key from EXPAND_API_KEY:

export EXPAND_API_KEY="xpnd_..."

With the variable set, construct a client and let it pick up the key. The Expand client owns an httpx.Client, so close it with a context manager or client.close():

from expandai import Expand

with Expand() as expand:
    ...

Pass the key explicitly when you manage configuration yourself:

import os
from expandai import Expand

with Expand(api_key=os.environ["EXPAND_API_KEY"]) as expand:
    ...

Constructor options

OptionDefaultDescription
api_keyos.environ["EXPAND_API_KEY"]Expand API key.
base_urlhttps://api.expand.aiAPI base URL.

The SDK sends your key on every request using the x-expand-api-key header. You never set that header by hand.

Method index

MethodUse whenReturns
fetch_json(body, include=None, request_options=None)App code needs Main Markdown, State JSON, snapshot metadata, or inline Highlights.Object-mode Fetch model.
fetch(body, include=None, request_options=None)You only need the Markdown string.str

AsyncExpand exposes the same methods as async methods. The names and model classes match the sync client one to one.

Models and serialization

The public SDK exports the request and response models you need from expandai. You should not import from expandai._generated in normal application code.

from expandai import (
    FetchParams,
    FetchJsonParams,
    FetchSearchParams,
    FetchJsonSearch,
    FetchSearchQuery,
    BatchedParams,
    to_dict,
    to_json,
)
  • Models use Python snake_case attributes, such as snapshot_id, captured_at, and max_results.
  • to_dict(model) serializes a model to a wire-format dictionary.
  • to_json(model, indent=2) is handy when you want to inspect a request body while debugging.
  • Exact generated field lists belong to the API Reference, not this page.

JSON mode with fetch_json

fetch_json() is the recommended SDK method for applications. It returns the Main Markdown, State JSON, snapshot metadata, and optional search results in one model.

from expandai import Expand, FetchJsonParams

with Expand() as expand:
    page = expand.fetch_json(FetchJsonParams(url="https://example.com"))

print(page.markdown)
print(page.json)
print(page.meta.snapshot_id)
print(page.meta.playground)

The fields you read most often (abbreviated — see the Output Model for the full shape):

page.meta.snapshot_id    # snapshot handle for follow-up search
page.meta.playground     # human inspection URL
page.markdown            # Main Markdown
page.json                # State JSON / extracted evidence
page.data                # optional extra data such as search results
  • markdown is the Main Markdown.
  • json is the State JSON and extracted evidence.
  • meta.snapshot_id is the handle you pass to later Highlights.
  • meta.playground is the human inspection link.
  • data.search appears only when you request inline search.

This list is not exhaustive. The Output Model and API Reference own the exact fields.

Markdown-only Fetch with fetch

fetch() is the convenience method for scripts and agents that only need the Markdown string.

from expandai import Expand, FetchParams

with Expand() as expand:
    markdown = expand.fetch(FetchParams(url="https://example.com"))

print(markdown)
  • The return type is str.
  • This calls /v1/fetch.
  • Reach for fetch_json() when the application needs State JSON, snapshot metadata, or structured search results.

Include options

Pass include as a keyword argument to widen what a Fetch returns:

markdown = expand.fetch(
    FetchParams(url="https://example.com"),
    include="appendix,statejson",
)
page = expand.fetch_json(
    FetchJsonParams(url="https://example.com"),
    include="appendix",
)

Body include models such as FetchJsonInclude are available when you prefer structured request-body controls. The exact include semantics belong to Include Options and the API Reference — this page does not repeat the full matrix.

Highlights

Highlights are search over a captured page. Run them inline with a fresh Fetch, or against a snapshot you already captured.

Inline Highlights with fetch_json

from expandai import Expand, FetchJsonParams, FetchJsonSearch

with Expand() as expand:
    page = expand.fetch_json(
        FetchJsonParams(
            url="https://docs.example.com",
            search=FetchJsonSearch(
                query="authentication limits",
                max_results=5,
                min_score=0.6,
            ),
        )
    )

snippets = page.data.search.snippets if page.data and page.data.search
  • This starts a new Fetch and searches it in the same call.
  • Snippets live under page.data.search.snippets.
  • Python attributes use max_results, not maxResults.
  • State JSON snippets may include json.
  • Raw SDK snippets expose location, not the MCP-only citationUrl field.

Snapshot Highlights with fetch_search

from expandai import Expand, FetchJsonParams, FetchSearchParams, FetchSearchQuery

with Expand() as expand:
    page = expand.fetch_json(FetchJsonParams(url="https://docs.example.com"))

    result = expand.fetch_search(
        FetchSearchParams(
            snapshot_id=page.meta.snapshot_id,
            search=FetchSearchQuery(
                query="authentication limits",
                max_results=5,
                min_score=0.6,
            ),
        )
    )

print(result.search.snippets)
  • fetch_search calls /v1/fetch/search.
  • It searches stored artifacts without recapturing the URL.
  • Use it to refine a query after a previous Fetch.
  • The exact request schema belongs to the API Reference. See Highlights for behavior.

Citation helpers

Raw SDK results expose snippet location. Use Playground helpers when you want to turn a snippet into a user-visible citation link.

from expandai import DEFAULT_PLAYGROUND_HOST, resolve_playground_host, snippet_citation_url
host = resolve_playground_host(page.meta.playground, DEFAULT_PLAYGROUND_HOST)

for snippet in snippets:
    url = snippet_citation_url(page.meta.snapshot_id, snippet, host)
    print(snippet.text, url)
HelperPurpose
playground_base(snapshot_id, host=...)Build the whole-snapshot Playground URL.
playground_origin(meta_playground)Extract the origin from meta.playground.
resolve_playground_host(meta_playground, fallback)Prefer the server-provided Playground host, fall back if missing.

SDK API responses do not already contain citationUrl. MCP adds that field; Python SDK users build it with these helpers. See Playground & Replay for URL semantics.

Batched Fetch

batched() starts many Fetch jobs at once; get_batched() polls the run until it finishes:

import time
from expandai import Expand, BatchedParams

with Expand(timeout_ms=120_000) as expand:
    run = expand.batched(
        BatchedParams(
            urls=[
                "https://example.com",
                "https://example.com/about",
            ]
        ),
    )

    status = expand.get_batched(run.id, options={"limit": 10, "offset": 0})

    while
  • batched() returns a run model with id.
  • get_batched() polls the run.
  • Each result includes its persisted status alongside data, so handle failed or cancelled URLs independently.
  • Python get_batched options accept numeric limit and offset; the SDK converts them to strings for the generated request internally.
  • When omitted, the SDK generates one idempotency_key and reuses it for the configured retry policy.
  • Supply a stable idempotency_key to deduplicate separate calls or calls made after a process restart.
  • Reusing a key with a different payload returns 409 BatchedIdempotencyConflict with reason: "payload_mismatch".
  • The full lifecycle belongs to Batched Fetch.

Async client

Use AsyncExpand in asyncio applications. It exposes the same operations as Expand, but each request method is awaited.

import asyncio
from expandai import AsyncExpand, FetchJsonParams

async def main() -> None:
    async with AsyncExpand() as expand:
        page = await expand.fetch_json(FetchJsonParams(url="https://example.com"))
        print(page.markdown)

asyncio.run(main())
  • Use async with to close the underlying httpx.AsyncClient.
  • The method names and model classes match the sync client.
  • with_raw_response also exists on AsyncExpand.

Raw responses

Use with_raw_response when you need headers, status codes, request IDs, or the underlying httpx.Response.

from expandai import Expand, FetchJsonParams

with Expand() as expand:
    response = expand.with_raw_response.fetch_json(
        FetchJsonParams(url="https://example.com")
    )

print(response.status_code)
print(response.headers)

page = response.parse()

The raw response is an APIResponse:

FieldMeaning
parsedParsed SDK model.
http_responseUnderlying httpx.Response.
status_codeHTTP status code.
headers

Errors and retries

Every SDK error subclasses ExpandError:

ErrorMeaning
ExpandErrorBase SDK error.
ExpandAPIErrorNon-2xx API response or failed response parsing. Inspect status_code, body, and request_id.
ExpandSdkErrorInvalid SDK options or setup error.

Retry and timeout defaults:

  • Default timeout_ms is 60000.
  • Default max_retries is 2.
  • Retryable failures include timeouts, connection errors, and HTTP statuses 408, 409, 429, 500, 502, 503, and 504.
from expandai import Expand, ExpandAPIError, FetchJsonParams

with Expand() as expand:
    try:
        page = expand.fetch_json(FetchJsonParams(url="https://example.com"))
    except ExpandAPIError as error:
        print(error.status_code)
        print(error.body)
        raise

Related pages

TopicPage
First SDK or API setupQuickstart
Choosing the SDK vs CLI, MCP, or APIWays to Use Expand
Product behaviorFetch Overview
Main Markdown and State JSON
PreviousTypeScript SDK
NextPricing & Usage

On This Page

Install and authConstructor optionsMethod indexModels and serializationJSON mode with fetch_jsonMarkdown-only Fetch with fetchInclude optionsHighlightsInline Highlights with fetch_jsonSnapshot Highlights with fetch_searchCitation helpersBatched FetchAsync clientRaw responsesErrors and retriesRelated pages
timeout_ms
60000
Total wall-clock budget across attempts.
max_retries2Retry attempts for retryable failures.
http_clientSDK-created httpx.Client / httpx.AsyncClientOptional custom client.
fetch_search(body, request_options=None)
You already have snapshot_id and want Highlights from stored artifacts.
Snapshot Highlights model.
batched(body, *, idempotency_key=None, request_options=None)You want to start many async Fetch jobs. The SDK generates idempotency_key when omitted.Batched run model with id.
get_batched(id, options=None)You want to poll a Batched Fetch run.Batched status/results model.
else
[]
citation_url(snapshot_id, location, host=...)
Build a citation link from location.evidence_id.
snippet_citation_url(snapshot_id, snippet, host=...)Build a citation link from a snippet model.
str(status.batched_status) in {
"QUEUED"
,
"RUNNING"
}:
time.sleep(1)
status = expand.get_batched(run.id, options={"limit": 10, "offset": 0})
print(status.results)
Response headers.
request_idRequest ID when present.
parse()Returns parsed.
ExpandTimeoutError
Request exceeded timeout_ms.
ExpandConnectionErrorNetwork or HTTP transport failure.
Output Model
Include semanticsInclude Options
Highlights behaviorHighlights
Citation URLs and replayPlayground & Replay
Batched lifecycleBatched Fetch
Exact endpoint schemasAPI Reference
TypeScript equivalentTypeScript SDK