Error Handling
When Expand fails, when the origin fails, and how to tell the two apart.
Expand separates its own failures from the origin's. That split is the single rule behind every status code and every error body in this API.
When Expand fails, when the origin fails, and how to tell the two apart.
Expand separates its own failures from the origin's. That split is the single rule behind every status code and every error body in this API.
Fetching a URL involves two parties. Expand can fail — infrastructure trouble, a bot wall it could not get past, a page beyond its processing limits. The origin can also fail — DNS that does not resolve, a broken certificate, a refused connection, a 404. A status code alone cannot express whose failure it was, so Expand does not try: origin outcomes are typed data inside a successful response.
| Whose failure | How it is reported |
|---|---|
| Expand's | Non-2xx status with a _tag-discriminated error body |
| The origin's | 200 plus a FetchOriginError in the body (or in response headers, on /v1/fetch) |
The practical consequence: nothing retries a dead URL by accident. Generic retry middleware retries 502 and 503. It does not retry 200. A permanently dead origin no longer looks like a temporary Expand outage.
Every error carries a _tag naming its class, and the classes whose status alone cannot settle the
question also carry a retryable verdict. Status and tag together are the contract;
the tag is what you should branch on.
| Status | _tag | What happened | Retry |
|---|---|---|---|
200 | FetchOriginError in the body or response headers | Expand completed the request; the origin failed | Follow the origin outcome's retryable field |
400 | , |
A successful request returns 200. Statuses that carry more than one tag return them as an anyOf
in the OpenAPI spec: the status tells you the category, the _tag tells you which member of it.
The fetch failure path is a three-way partition. Proven origin failures are 200 outcomes. Proven
deterministic page or capture failures are 502 with retryable: false. Failures that fit neither
proof stay in the smaller 503 residue with retryable: true; a missing or unreviewed network code
is not enough to blame the origin or the page.
Every response — success or error — carries the x-expand-trace-id header. Quote it when contacting support.
Every error body carries a _tag field identifying the error type:
{
"_tag": "ErrorType"
// additional fields depend on the error type
}When the origin — not Expand — is the thing that failed, the response is a 200 carrying a FetchOriginError.
{
"_tag": "FetchOriginError",
"url": "https://checkout.deadstartup.example/pricing",
"kind": "dns",
"upstreamError": "net::ERR_NAME_NOT_RESOLVED",
"retryable": false
}| Field | Description |
|---|---|
url | The requested URL this outcome describes. |
kind | What went wrong at the origin. See the table below. |
upstreamError | The underlying browser network error, for the network kinds. Optional. |
originStatusCode |
The document-only fields contentAvailable, redirectLimitReached, and finalUrl describe the upcoming document-origin outcome. Document refusals return FetchOriginHttpError with HTTP 404 or 410 when the document is missing or gone, and 502 for other origin statuses.
kind | Meaning | retryable |
|---|---|---|
dns | DNS resolution failed. The exact code distinguishes a stable missing name from resolver timeouts/failures. | Depends on upstreamError. |
tls | The TLS handshake or certificate validation failed. | Depends on upstreamError. |
The network kinds mean no HTTP response ever existed. http means the origin answered and told you it had a problem.
For current captures, retryable is derived from the exact reviewed Chromium net::ERR_* code, not merely from kind. For example, net::ERR_NAME_NOT_RESOLVED is a stable DNS failure and reports false, while net::ERR_DNS_TIMED_OUT is transient and reports true. Unknown codes from current native capture stay Expand-side failures rather than receiving a guessed origin kind or retry verdict. Older stored failures may not carry a reviewed code; when those already contain a legacy failureKind, the API preserves that kind and uses its conservative default retry verdict.
For kind: "http", retryable is true when originStatusCode is 408, 429, or 500 and above — the origin either asked for a later attempt or hit a server-side condition that may clear. Every other 4xx describes the request itself, so repeating it unchanged reports false.
POST /v1/fetch/jsonThe JSON endpoint returns one of two shapes, and they are disjoint.
Origin never reached — no document exists, so the origin outcome is the response. Recognize this variant by its top-level _tag:
HTTP/1.1 200 OK
{
"_tag": "FetchOriginError",
"url": "https://checkout.deadstartup.example/pricing",
"kind": "dns",
"upstreamError": "net::ERR_NAME_NOT_RESOLVED",
"retryable": false
Origin responded with an error status — the page was captured, so you get the content and the outcome. Error pages are frequently useful: soft-404s, maintenance notices, status dashboards returning 500. Recognize the content variant by its meta field; it has no top-level _tag.
HTTP/1.1 200 OK
{
"meta": {
"version": 1,
"url": "https://site.example/removed-page",
"title": "Page not found"
},
"markdown": "# Page not found\n\nThe page you are looking for has been removed…",
"
originError is absent whenever the origin answered normally, so its presence alone tells you the origin reported a problem.
POST /v1/fetch/v1/fetch answers in text/markdown, which has nowhere to put a typed field, so the origin outcome rides response headers.
| Header | Value |
|---|---|
x-expand-origin-error | The kind — dns, tls, connectionRefused, connectionReset, connectionClosed, addressUnreachable, emptyResponse, connectionTimeout, or http. |
x-expand-origin-error-detail |
When the origin was never reached there is no document to return, so the body is a short Markdown error document written to be read by an LLM as readily as by a human:
HTTP/1.1 200 OK
content-type: text/markdown; charset=utf-8
x-expand-origin-error: dns
x-expand-origin-error-detail: net::ERR_NAME_NOT_RESOLVED
x-expand-origin-error-retryable: false
# Origin unreachable
DNS resolution failed for `https://checkout.deadstartup.example/pricing`
(`net::ERR_NAME_NOT_RESOLVED`). No HTTP response exists at this URL. Retrying
will not succeed until the domain's DNS is fixed.When an error page was captured, its content is returned as usual and only the headers change:
HTTP/1.1 200 OK
content-type: text/markdown; charset=utf-8
x-expand-origin-error: http
x-expand-origin-error-detail: 500
x-expand-origin-error-retryable: true
---
url: https://api-status.example/dashboard
title: Internal Server Error
---
# Internal Server Error
…captured page content…The headers are absent entirely when the origin answered normally.
The high-level SDK fetch convenience returns the Markdown string. Use the SDK's raw-response surface where available, or a direct HTTP request, when you also need to inspect these response headers.
Batched Fetch reports the same type per URL, in-band. A run never fails because one URL was unreachable.
{
"status": "SUCCEEDED",
"error": {
"_tag": "FetchOriginError",
"url": "https://checkout.deadstartup.example/pricing",
"kind": "dns",
"upstreamError
An item carries exactly one of data or error: error means no snapshot was produced. An origin that responded with an error status did produce one, so that item keeps its data and reports the outcome in a separate originError field alongside it. See Batched Fetch.
retryableA status code cannot always tell you whether a retry could succeed: a 502 here means the page
itself could not be captured and will fail the same way next time, while a 503 means our side had
a transient problem. Every error whose status leaves that open — 429, 500, 502, 503 —
therefore states the answer outright:
{
"_tag": "ServiceUnavailable",
"retryable": true
}Prefer this field over the status code. Errors that are unambiguous from their status alone (400,
401, 403, 404, 409, 410, 413) omit it; treat a missing field on those as false.
429 and 503 responses also carry a Retry-After header, in seconds. Wait at least that long
before retrying — retrying sooner spends your own timeout budget on a refusal we already predicted.
Returned when the request body does not match the expected schema. Framework-level decoding produces HttpApiDecodeError:
{
"_tag": "HttpApiDecodeError",
"message": "Invalid request parameters",
"issues": [
{
"_tag": "Missing",
"path": ["url"],
"
Endpoint-level validation — unknown properties, or a search request with no searchable corpus — produces BadRequest:
{
"_tag": "BadRequest",
"message": "Invalid fetch request body"
}Common validation issues:
url fieldhttp:// or https://)include.links.includePatternssearch.query{
"_tag": "AuthFailed",
"reason": "InvalidApiKey",
"description": "The provided API key is not valid"
}| Reason | Description |
|---|---|
InvalidApiKey | The API key is missing or invalid |
InvalidToken | The bearer token is invalid |
InvalidSession | The session has expired |
InvalidTenant |
AccessBlocked means your organization cannot currently run fetches — most often too few credits to cover the request. See Pricing & Usage.
{
"_tag": "AccessBlocked",
"organizationId": "org_123"
}FetchBlocked means the target site blocked the capture and Expand could not get past it. This is Expand failing to deliver the page, which is why it is a 403 rather than an origin outcome.
{
"_tag": "FetchBlocked",
"url": "https://example.com",
"blockedType": "botProtection"
}blockedType is optional and describes the classification, for example captcha, botProtection, ipBlock, or authWall. Retrying immediately rarely helps; Expand already retried through a different proxy exit where that could plausibly clear the block.
PayloadTooLarge is about your request. FetchPageTooLarge is about the target page: the request was well-formed, the page is simply beyond what Expand will process.
{
"_tag": "FetchPageTooLarge",
"url": "https://example.com/huge",
"limitKind": "domsnapshot_bytes",
"limitBytes": 52428800,
"measuredBytes": 91234567
}limitKind is one of transfer_bytes, domsnapshot_bytes, or document_frame_bytes. Older stored results can also carry ast_bytes, a retired limit that is no longer produced. Retrying the same URL will not help.
FetchNavigationFailed means navigation committed no HTTP(S) document and no recognized Chromium network code proved the origin was responsible. Browser documents such as about:blank, blob:, data:, extension/PDF viewers, and download outcomes are not evidence of a refused origin connection, so they remain Expand-side failures.
{
"_tag": "FetchNavigationFailed",
"url": "https://example.com/report.pdf",
"failureType": "browserInternalDocument"
}This also preserves the honest response for legacy batched results that stored only navigationFailed without a network code.
Returned when you exceed your rate limit. See Rate Limiting.
{
"_tag": "TooManyRequests",
"retryable": true
}Returned when your organization already has as many captures in flight as your plan allows. Nothing was captured, so the same request succeeds once a slot frees:
{
"_tag": "FetchConcurrencyLimitExceeded",
"maxConcurrency": 5,
"retryAfterSeconds": 20,
"requestId": "0f4c...",
"retryable": true
}maxConcurrency is your own limit, not a global one. Either lower the number of requests you run at
once, or send the URLs to /v1/fetch/batched, which queues against the same limit instead of
refusing.
Returned when an unexpected server-side failure interrupts the request:
{
"_tag": "InternalError",
"retryable": true
}An unexpected failure may not recur, so a retry is worthwhile.
Returned when the browser could not produce a usable page for the URL. Two tags share this status.
FetchNavigationFailed — navigation itself did not reach the page:
{
"_tag": "FetchNavigationFailed",
"url": "https://example.com/",
"failureType": "browserInternalDocument",
"retryable": false
}FetchCaptureFailed — navigation succeeded, but the capture did not:
{
"_tag": "FetchCaptureFailed",
"url": "https://example.com/",
"failureType": "pageNeverSettled",
"retryable": false
}failureType | Meaning |
|---|---|
pageNeverSettled | The page never produced any navigation activity to capture |
capabilityFailed | The page rendered, but no capture satisfied a format you requested |
Both verdicts are deterministic for this URL: the captured evidence proves another attempt will reach the same outcome. Fix the URL or the target page instead.
The SDKs do not retry the page verdicts or FetchCaptureTimeout. 403 and 413 are not retryable statuses, and the 502 and 504 tags are explicitly exempted. Keeping ambiguous 504s out of automatic retry loops avoids amplifying an Expand-side incident; a deliberate manual retry may still be appropriate. The SDKs do retry 529 FetchCapacityTimeout with their normal backoff; the response also advertises Retry-After for direct HTTP clients.
Returned when the URL served a document rather than a page and the conversion could not be delivered. Documents lists what Fetch converts.
415 FetchUnsupportedContentType means nothing here can be converted. That covers an unknown format, a file too damaged to read, and a request that bounced between the page and document paths without either one handling it. contentType is present when Fetch could detect it.
{
"_tag": "FetchUnsupportedContentType",
"url": "https://example.com/archive.7z",
"contentType": "application/x-7z-compressed"
}An unavailable document is not charged. Document refusals return FetchOriginHttpError with the origin status in statusCode. Missing or removed documents return HTTP 404 or 410. Check the URL and access permissions before trying again. Other origin statuses return HTTP 502.
The upcoming response below uses 200 FetchOriginError with contentAvailable: false. Clients should support both shapes during the transition. originStatusCode is what the origin sent. Redirect exhaustion also sets redirectLimitReached: true and includes the last responding finalUrl. It does not necessarily mean a loop; the redirect chain may simply be too long.
{
"_tag": "FetchOriginError",
"url": "https://example.com/report.pdf",
"kind": "http",
"originStatusCode": 404,
"contentAvailable": false,
"retryable
This is deliberately unlike a page fetch, which renders an origin's error page and reports its status in data.response.originStatusCode. Rendering the error page is right for a caller who asked for a page. There is no rendering step for a document, so the only alternative would be converting a block page into Markdown and calling it the document.
503 FetchDocumentProcessingUnavailable means a vision or transcription provider could not finish the work, or, for a YouTube URL, that the platform refused to hand over the video. operation says which one: "ocr", "transcription", or "video".
{
"_tag": "FetchDocumentProcessingUnavailable",
"url": "https://example.com/scan.pdf",
"operation": "ocr",
"message": "OCR provider unavailable"
}422 FetchVideoUnavailable means the video platform will not serve this video to anyone in our position, so there is no transcript to return. availability is one of private, removed, geoRestricted, ageRestricted, membersOnly, live, upcoming, or unknown. Only live and upcoming change on their own, once the stream ends. YouTube answers 200 for all of them, which is why this is not a 502 FetchOriginHttpError. See YouTube.
{
"_tag": "FetchVideoUnavailable",
"url": "https://www.youtube.com/watch?v=aBcDeFgHiJk",
"availability": "private",
"message": "Private video. Sign in if you've been granted access to this video"
}413 FetchPageTooLarge means the document exceeded a limit. This can be an explicit documentConfig.maxPages limit, maxAudioSeconds, or a limit on image or audio processing. PDFs have no default page limit or public 50 MB transfer cap. Worker resource failures are reported separately. Readable pages use their text layer; pages that need OCR are processed separately. limitKind tells you which one. For document_pages and audio_seconds the payload reports limit and measured in that unit, while the byte-shaped limits use limitBytes and measuredBytes. Nothing converts, and you are charged no credits.
{
"_tag": "FetchPageTooLarge",
"url": "https://example.com/book.pdf",
"limitKind": "document_pages",
"limit": 50,
"measured": 312
}The SDKs do not retry 415 or a 200 origin outcome automatically. For a contentless origin outcome, retryable is true for origin 408, 429, and 5xx statuses, and false for redirect exhaustion and other 4xx statuses. A 503 FetchDocumentProcessingUnavailable still uses normal backoff. By default, missing or removed documents return 404 or 410 with the existing FetchOriginHttpError body, so SDKs do not retry them. Other document refusals remain 502 FetchOriginHttpError, which SDKs retry with normal backoff. Updated SDKs decode these error bodies. Older Node SDK versions expose an unrecognized 404 or 410 body as raw JSON text; upgrade the SDK if you read structured fields from the error body.
Returned when processing started but no result was ready inside its budget: 150 seconds for browser captures or 55 minutes for documents. It is also the conservative fallback when Expand cannot tell whether the capture started:
{
"_tag": "FetchCaptureTimeout",
"url": "https://example.com/",
"timeoutMs": 150000,
"retryable": false
}timeoutMs reports the configured request ceiling: 150000 for browser captures or 3300000 for documents, not the time remaining.retryable: false is why the SDKs do not retry this automatically: it keeps ambiguous 504s out of
automatic retry loops, so an Expand-side incident is not amplified. It is not proof the page was
at fault, and a deliberate manual retry may still be appropriate — it starts a separate capture
that can consume another full 150-second budget.timeoutMs default of 60000 aborts long captures client-side. See
TypeScript SDK.FetchCapacityTimeout; one that started stays on the
504 side.Returned when the request was still waiting for capacity when its deadline passed:
{
"_tag": "FetchCapacityTimeout",
"url": "https://example.com/",
"timeoutMs": 150000,
"retryable": true
}retryable: true says, and it is the one
timeout the SDKs retry for you.Retry-After: 5 — a short overload backoff that avoids an immediate retry
stampede while spending little of the next request's 150-second budget. Direct HTTP clients should
wait at least that long.504 FetchCaptureTimeout.ServiceUnavailable now means only one thing: an Expand-side infrastructure problem. It is no longer returned for unreachable origins.
{
"_tag": "ServiceUnavailable",
"message": "Expand could not complete the request. Retry shortly, or contact support with the request ID if the problem persists.",
"requestId": "0f4c...",
"retryable": true
}This is reserved for transient failures on our side. Retry after the delay named by Retry-After,
and quote requestId if the problem persists. requestId is the same trace ID sent in the
x-expand-trace-id header.
| Outcome | Retry? |
|---|---|
200 with FetchOriginError, retryable: true | Yes, after a delay. The condition may clear. |
200 with FetchOriginError, retryable: false | No. The URL is dead; fix the URL or drop it. |
200 with no origin error | Nothing to retry. |
Do not treat a 200 carrying an origin error as a transport failure. It is a completed answer about a URL that could not serve content.
You pay for delivered content, not for attempts. Charges follow the outcome, not the status code.
| Outcome | Billed |
|---|---|
| Page captured successfully | Yes — one credit |
Origin responded with an error status (400+) and content was captured | Yes — the content was delivered alongside the originError |
Origin unreachable — any network FetchOriginError kind | No |
| (403) |
The one case that surprises people is the second row: an origin's 404 or 500 page still contains content — soft-404s, maintenance pages, status dashboards — and Expand captured and returned it, so it costs a credit like any other page. If you do not want to pay for error pages, check originError before treating a response as useful.
Everything else is straightforward: if no content reached you, no charge was made. See Pricing & Usage for what each operation costs in credits.
ExpandClient throws only when Expand failed. Inspect status and body on ExpandClientApiError to branch on the _tag. See the TypeScript SDK for the full error hierarchy.
import { ExpandClient, ExpandClientApiError, ExpandClientError } from '@expandai/sdk'
const expand = new ExpandClient({ apiKey: '{{API_KEY}}' })
try {
const result = await expand.fetchJson({ url: 'https://example.com' })
// Reached here means Expand succeeded.
} catch (error) {
if (error instanceof ExpandClientApiError) {
// A non-2xx response. `body` is the parsed error above — branch on `_tag`.
console.log('Status:', error.status)
fetchJson resolves to either shape, so narrow before reading content. Only the origin-outcome variant carries a top-level _tag:
import { isFetchOriginError } from '@expandai/sdk'
const result = await expand.fetchJson({ url: 'https://example.com' })
if (isFetchOriginError(result)) {
// The origin was never reached — there is no content.
console.log(result.kind, result.retryable)
} else if (result.originError) {
// The origin answered with an error status, but the page was captured.
console.log(result.originError.originStatusCode, result.markdown)
} else {
console.log(result.markdown)
}Check the status code first, then the body. On /v1/fetch, the origin outcome is in the response headers.
response=$(curl -s -D /tmp/expand-headers -w "\n%{http_code}" \
-X POST https://api.expand.ai/v1/fetch \
-H "x-expand-api-key: {{API_KEY}}" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}')
status_code=$(echo "$response" | tail -n 1)
body=$(echo "$response
200 may still carry an origin outcome._tag, not on message text.retryable. It is derived from the exact Chromium network code; false means stop, while true means the condition may clear.retryable: true, with backoff and never sooner than Retry-After.403, 413, 502) or the conservative timeout fallback (504). A deliberate 504 retry may still be appropriate, but each attempt re-runs a full capture.180000 ms client timeout for browser captures or 3600000 ms for documents. Documents can process for up to 55 minutes.x-expand-trace-id on every response. It is what support needs, and it is present even when the body is not.HttpApiDecodeErrorBadRequest| The request body did not match the schema |
| Never |
401 | AuthFailed | The API key, token, or session is not valid | Never |
403 | AccessBlocked | Your organization's access is suspended | Never |
403 | FetchBlocked | The target site refused the capture (bot protection, captcha) | Never |
404 | FetchSnapshotSearchNotFound | No snapshot exists to search | Never |
404, 410 | FetchOriginHttpError | The origin reports the requested document is missing or gone | Never |
409 | BatchedIdempotencyConflict, BatchedCancellationConflict | The batched run is in a conflicting state | Never |
413 | PayloadTooLarge | The request body exceeds the limit | Never |
413 | FetchPageTooLarge | The captured page or document conversion exceeds a limit | Never |
415 | FetchUnsupportedContentType | The origin served content no converter handles | Never |
422 | FetchVideoUnavailable | The video platform will not serve this video; inspect availability | Only for live and upcoming, after the stream ends |
429 | TooManyRequests | You exceeded your rate limit | retryable: true |
429 | FetchConcurrencyLimitExceeded | You already have as many captures running as your plan allows | retryable: true |
500 | InternalError | An unexpected server-side failure interrupted the request | retryable: true |
502 | FetchNavigationFailed | The browser could not navigate to the page | retryable: false |
502 | FetchCaptureFailed | The page could not be captured | retryable: false |
502 | FetchOriginHttpError | The origin refused a document request; inspect statusCode | Depends on the origin status |
503 | FetchDocumentProcessingUnavailable | OCR, transcription, or video acquisition failed after provider retries | Retry later |
503 | ServiceUnavailable | A transient failure on our side | retryable: true |
504 | FetchCaptureTimeout | The capture started but produced no result before its deadline | retryable: false |
529 | FetchCapacityTimeout | The request expired in the queue; the capture never started | retryable: true |
The origin's HTTP status. Present only for kind: "http". |
retryable | Whether retrying the same URL later could plausibly succeed. |
contentAvailable | false for a standalone HTTP-origin refusal. No document was delivered and the request is refunded. |
redirectLimitReached | true when the origin exceeded the document redirect limit. |
finalUrl | The last URL that responded, when recorded for a document refusal. |
connectionRefused | The connection attempt was explicitly refused; nothing accepted it. | false |
connectionReset | A peer or intermediary reset an established connection (TCP RST). | true |
connectionClosed | A peer closed an established connection unexpectedly (TCP FIN). | true |
addressUnreachable | The network had no working route to the origin address. | true |
emptyResponse | The server accepted the connection, then closed it without sending data. | true |
connectionTimeout | The connection never answered in time. | true |
http | The origin responded, with a status of 400 or above. | Depends on the status — see below. |
The net::ERR_* code for the network kinds, or the origin's status for http. Omitted when neither is known. |
x-expand-origin-error-retryable | true or false. |
x-expand-content-available | false for the upcoming document-refusal outcome. The body explains the refusal and contains no captured document. |
| The organization was not found |
400, 401, 403, 413 | No. Fix the request, the credentials, the balance, or the URL. |
429 | Yes, following retryable and Retry-After. See Rate Limiting. |
500 | Yes when the body says retryable: true; report it with the trace ID if it persists. |
502 with FetchNavigationFailed or FetchCaptureFailed | No. The page or capture failure was classified as deterministic. |
503 | Yes, after Retry-After; this is the unproven Expand-side residue. |
504 with FetchCaptureTimeout | No automatic retry. A deliberate retry may still be appropriate. |
529 with FetchCapacityTimeout | Yes, after Retry-After; the capture never started. |
FetchBlocked| No |
FetchPageTooLarge (413) | No |
ServiceUnavailable (503) or InternalError (500) | No |
The SDK already retries for you, using the same retryable field: it never retries a
502 FetchCaptureFailed, and it waits out Retry-After before retrying a 429. Anything that
reaches your catch block has run out of attempts.
400 errors.