Fetch - Crustdata Docs

Authorizations

Authorization

string

header

required

Bearer token authentication. Pass your API key as Authorization: Bearer <your_api_key>.

Headers

x-api-version

enum<string>

default:2025-11-01

required

API version to use. This endpoint currently requires 2025-11-01.

Available options:

2025-11-01

Example:

"2025-11-01"

Body

application/json

List of URLs to fetch and optional settings.

Request body for fetching webpage content.

urls

string[]

required

List of URLs to fetch web content for. Each URL must include the protocol prefix (http:// or https://). Maximum of 10 URLs per request.

Required array length: 1 - 10 elements

A fully-qualified URL to fetch.

Maximum string length: 2000

Example:

["https://example.com"]

human_mode

boolean

default:false

Whether to use a browser-like retrieval path when standard access is blocked.

Example:

false

Response

Array of fetch results, one per URL. Each result includes a success flag. Failed URLs have null fields.

success

boolean | null

Whether the content was fetched successfully for this URL.

Example:

true

url

string | null

The URL that was fetched. Null if the fetch failed.

Example:

"https://example.com"

timestamp

integer | null

Unix timestamp (seconds) of when the content was fetched. Null if the fetch failed.

Example:

1775193366

title

string | null

The title of the fetched webpage from the HTML title tag. Null if the fetch failed.

Example:

"Example Domain"

content

string | null

The full HTML content of the fetched webpage. Null if the fetch failed.