Expand AI logo
DocsDocs
Login

Documentation

Introduction

WelcomeQuickstartConcepts

Product

FetchMCPCLI

SDKs

SDKsTypeScript SDKPython SDK

Product

Fetch

Extract web content as markdown, HTML, screenshots, and more

fetch is the primary interface for extracting data from the web. It turns any URL into a structured response containing exactly the formats you need.

It works out of the box without any configuration—JavaScript rendering, residential proxies, and bot protection are handled automatically for every request.

Usage

Send a POST request to https://api.expand.ai/v1/fetch with the target URL and your desired data selection.

import Expand from 'expandai'

const client = new Expand({
  apiKey: '{{API_KEY}}'
})

const result =
await
client.fetch({
url: 'https://example.com',
select: {
markdown: true,
meta: true
}
})
from expandai import Expand

client = Expand(api_key="{{API_KEY}}")

result = client.fetch(
    url="https://example.com",
    select={
        "markdown": True,
        "meta": True
    }
)
curl -X POST https://api.expand.ai/v1/fetch \
  -H "x-expand-api-key: {{API_KEY}}" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "select": {
      "markdown": true,
      "meta": true
    }
  }'

The API returns a structured object with your requested data:

{
  "data": {
    "response": {
      "originStatusCode": 200,
      "url": "https://example.com"
    },
    "meta": {
      "title":

Select Options

The select object controls which data formats are returned. You can mix and match any combination of options. For full type definitions, see the API Reference.

PropertyDescription
markdownLLM-optimized markdown representation of the page.
metaPage metadata including title, description, and social tags.
htmlRaw HTML content of the page.
screenshotVisual capture of the page (viewport or full-page).

AI Summaries

When summary is enabled, the API generates a high-level overview of the page content. You can pass a custom prompt to focus the summary on specific information.

const result = await client.fetch({
  url: 'https://example.com',
  select: {
    summary: {
      prompt: 'Extract the key features and pricing information'
    }
  }
})

console.log(result.data.summary)
result = client.fetch(
    url="https://example.com",
    select={
        "summary"






{
  "data": {
    "summary": "This page provides an overview of the Expand API, covering its main features such as web scraping, AI-powered extraction, and residential proxy support."
  }
}

AI Snippets

Snippets allow you to perform RAG (Retrieval-Augmented Generation) directly on a single page. Instead of processing the whole document, you can extract only the parts relevant to your query.

const result = await client.fetch({
  url: 'https://example.com',
  select: {
    snippets: {
      query: 'How do I authenticate with the API?'
    }
  }
})

console.log(result.data.snippets)
result = client.fetch(
    url="https://example.com",
    select={
        "snippets": {






{
  "data": {
    "snippets": [
      {
        "text": "To authenticate with the API, include your API key in the 'x-expand-api-key' header of your requests.",
        "score": 0.92,
        "index": 12
      }
    ]

Browser Configuration

Configure how the browser behaves before content is extracted.

PropertyDescription
scrollFullPageWhether to scroll the entire page to trigger lazy-loaded content.
PreviousConcepts
NextMCP

On This Page

UsageSelect OptionsAI SummariesAI SnippetsBrowser Configuration
"Example Domain"
,
"description": "This domain is for use in illustrative examples..."
},
"markdown": "# Example Domain\n\nThis domain is for use in illustrative examples in documents..."
}
}
summaryAI-powered summary of the page content.
snippetsRelevant snippets extracted using AI.
responseHTTP status and optionally response headers.
: {
"prompt": "Extract the key features and pricing information"
}
}
)
print(result.data.summary)
curl -X POST https://api.expand.ai/v1/fetch \
  -H "x-expand-api-key: {{API_KEY}}" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "select": {
      "summary": {
        "prompt": "Extract the key features and pricing information"
      }
    }
  }'
"query": "How do I authenticate with the API?"
}
}
)
print(result.data.snippets)
curl -X POST https://api.expand.ai/v1/fetch \
  -H "x-expand-api-key: {{API_KEY}}" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "select": {
      "snippets": {
        "query": "How do I authenticate with the API?"
      }
    }
  }'
}
}