Skip to main content
Use this when you want to find web pages, news articles, academic papers, author profiles, AI-generated overviews, or social media posts matching a search query. The Web Search API accepts a query and returns results from one or more source types. The result shape varies by source — always specify sources explicitly when you need predictable parsing. Every request goes to the same endpoint: 
POST https://api.crustdata.com/web/search/live
Pricing: 1 credit per query.
Default rate-limit is 15 requests per minute. Send an email to gtm@crustdata.co to discuss higher limits if needed for your use case.

Source types

Result shapes and field presence for each source: web, news, scholar-articles, scholar-author, AI, social.

Recipes

Site/date filtering, multi-page, academic, deep research mode, discovery patterns, and search → fetch workflows.

Reference

Request parameters, response body, error handling, and common gotchas.
The simplest search uses a query with an explicit sources array. Always specify sources for predictable result parsing. 
curl --request POST \
  --url https://api.crustdata.com/web/search/live \
  --header 'authorization: Bearer YOUR_API_KEY' \
  --header 'content-type: application/json' \
  --header 'x-api-version: 2025-11-01' \
  --data '{
    "query": "crustdata",
    "sources": ["web"],
    "location": "US"
  }'
Response trimmed for clarity.
Extract: Each result in results[] contains source, title, url, snippet, and position. Use position for ranking and url for follow-up fetching.

​
Use human mode when standard retrieval is blocked

Set human_mode: true when you want the API to attempt a browser-like retrieval path for the search request. 
curl --request POST \
    --url https://api.crustdata.com/web/search/live \
    --header 'authorization: Bearer YOUR_API_KEY' \
    --header 'content-type: application/json' \
    --header 'x-api-version: 2025-11-01' \
    --data '{
        "query": "crustdata",
        "sources": ["web"],
        "human_mode": true
    }'
Current platform behavior: human_mode: true returns the normal search response shape with success, query, timestamp, results, and metadata.

​
What to do next

  • Understand result shapes — see Source types for field presence by source.
  • Try more patterns — see Recipes for site/date filtering, discovery, and workflows.
  • Look up request/response details — see Reference.
  • Fetch page content — see Web Fetch to fetch HTML content of URLs returned by search.