Skip to main content
POST
https://api.crustdata.com
/
web
/
enrich
/
live
curl --request POST \ --url https://api.crustdata.com/web/enrich/live \ --header 'Authorization: Bearer <token>' \ --header 'Content-Type: application/json' \ --header 'x-api-version: <x-api-version>' \ --data ' { "urls": [ "https://example.com" ] } '
[ { "success": true, "url": "https://example.com", "timestamp": 1775193366, "title": "Example Domain", "content": "<html lang=\"en\"><head><title>Example Domain</title><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\"><style>body{background:#eee;width:60vw;margin:15vh auto;font-family:system-ui,sans-serif}h1{font-size:1.5em}div{opacity:0.8}a:link,a:visited{color:#348}</style></head><body><div><h1>Example Domain</h1><p>This domain is for use in documentation examples without needing permission. Avoid use in operations.</p><p><a href=\"https://iana.org/domains/example\">Learn more</a></p></div>\n</body></html>" } ]

Documentation Index

Fetch the complete documentation index at: https://docs.crustdata.com/llms.txt

Use this file to discover all available pages before exploring further.

Authorizations

​
Authorization
string
header
required

Bearer token authentication. Pass your API key as Authorization: Bearer <your_api_key>.

Headers

​
x-api-version
enum<string>
default:2025-11-01
required

API version to use. This endpoint currently requires 2025-11-01.

Available options:
2025-11-01
Example:

"2025-11-01"

Body

application/json

List of URLs to fetch and optional settings.

Request body for fetching webpage content.

​
urls
string[]
required

List of URLs to fetch web content for. Each URL must include the protocol prefix (http:// or https://). Maximum of 10 URLs per request.

Required array length: 1 - 10 elements

A fully-qualified URL to fetch.

Maximum string length: 2000
Example:
["https://example.com"]
​
human_mode
boolean
default:false

Whether to attempt bypassing Cloudflare protection when fetching pages behind Cloudflare.

Example:

false

Response

Array of fetch results, one per URL. Each result includes a success flag. Failed URLs have null fields.

​
success
boolean | null

Whether the content was fetched successfully for this URL.

Example:

true

​
url
string | null

The URL that was fetched. Null if the fetch failed.

Example:

"https://example.com"

​
timestamp
integer | null

Unix timestamp (seconds) of when the content was fetched. Null if the fetch failed.

Example:

1775193366

​
title
string | null

The title of the fetched webpage from the HTML title tag. Null if the fetch failed.

Example:

"Example Domain"

​
content
string | null

The full HTML content of the fetched webpage. Null if the fetch failed.