Web Search source types - Crustdata Docs

Web Search supports seven source types. Each returns a different result shape — always specify sources explicitly when you need predictable parsing.

Source capabilities

Current platform behavior — not guaranteed by the OpenAPI contract. Parameter applicability varies by source. This table reflects observed behavior.

Source	Best use case	Fetchable `url`?	`site` effective?	Date filters effective?
`web`	General web search	Yes	Yes	Yes
`news`	News articles	Yes	Yes	Yes
`scholar-articles`	Academic papers	Yes	No	Yes
`scholar-articles-enriched`	Papers + author profiles	Yes	No	Yes
`scholar-author`	Researcher profiles	No	No	No
`ai`	AI-generated summaries	No	No	No
`social`	Social media mentions	Yes	No	No

Result shapes by source

The results[] array shape depends on the source field of each result. Use this reference when parsing multi-source responses.

web / news
scholar-articles
scholar-author
ai
social

Standard web and news results share the same shape.

Field	Type	Description
`source`	string	`"web"` or `"news"`.
`title`	string	Page title.
`url`	string	Page URL.
`snippet`	string	Text excerpt.
`position`	integer	Result position (1-based).

{
    "source": "web",
    "title": "Crustdata: Real-Time B2B Data Broker via API or Data Feed",
    "url": "https://crustdata.com/",
    "snippet": "Crustdata is a B2B data provider offering real-time company & people datasets.",
    "position": 1
}

Academic article results include citation data, author information, and optional PDF links.

Field	Type	Description
`source`	string	`"scholar-articles"` or `"scholar-articles-enriched"`.
`title`	string	Article title.
`url`	string	Link to the article.
`snippet`	string	Abstract excerpt.
`metadata`	string	Citation string: `"Author - Year - Publisher"`.
`pdf_url`	string?	Direct PDF link, if available.
`position`	integer	Result position (1-based).
`authors`	array	`[{ name, profile_url, profile_id }]`.
`citations`	integer	Total citation count.

{
    "source": "scholar-articles",
    "title": "Understanding deep learning",
    "url": "https://books.google.com/books?hl=en&lr=lang_en&id=rvyxEAAAQBAJ",
    "snippet": "...to this field understand the principles behind deep learning.",
    "metadata": "SJD Prince - 2023 - books.google.com",
    "pdf_url": null,
    "position": 1,
    "authors": [
        {
            "name": "SJD Prince",
            "profile_url": "https://scholar.google.com/citations?user=fjm67xYAAAAJ&hl=en&oi=sra",
            "profile_id": "fjm67xYAAAAJ"
        }
    ],
    "citations": 618
}

Use scholar-articles-enriched instead of scholar-articles to get richer author profile data. The result shape is the same, with more author fields populated.

Author profile results have a completely different shape — no snippet, position, or title. Instead, you get a full researcher profile.

Field	Type	Description
`source`	string	`"scholar-author"`.
`url`	string	Academic profile URL.
`name`	string	Author full name.
`affiliation`	string	Institutional affiliation.
`website`	string?	Personal or institutional website.
`interests`	array	`[{ title, link }]` — research interests.
`thumbnail`	string?	Profile photo URL.
`citations`	object	`{ all, since_2020 }` — total and recent counts.
`h_index`	object	`{ all, since_2020 }`.
`i10_index`	object	`{ all, since_2020 }`.
`articles`	array	Top publications: `[{ title, url, year, citations, authors, publication }]`.

{
    "source": "scholar-author",
    "url": "https://scholar.google.com/citations?user=NMS69lQAAAAJ&hl=en&oi=ao",
    "name": "Jeff Dean",
    "affiliation": "Google Chief Scientist, Google Research and Google DeepMind",
    "website": "http://research.google.com/people/jeff",
    "interests": [
        { "title": "Distributed systems", "link": "https://scholar.google.com/..." }
    ],
    "citations": { "all": 401624, "since_2020": 231008 },
    "h_index": { "all": 114, "since_2020": 78 },
    "i10_index": { "all": 319, "since_2020": 203 },
    "articles": [
        {
            "title": "MapReduce: simplified data processing on large clusters",
            "url": "https://scholar.google.com/...",
            "year": "2008",
            "citations": "37255",
            "authors": "J Dean, S Ghemawat",
            "publication": "Communications of the ACM 51 (1), 107-113, 2008"
        }
    ]
}

Deep research mode returns a single AI-generated overview with source references. No snippet, position, or standard search fields.

Field	Type	Description
`source`	string	`"ai"`.
`title`	string	Always `"AI Overview"`.
`content`	string	AI-generated overview text.
`references`	array	Source articles: `[{ title, url, snippet }]`.
`images`	array	Embedded images: `[{ url, alt, width, height }]`.

{
    "source": "ai",
    "title": "AI Overview",
    "content": "The primary difference between uv and pip is speed and scope...",
    "references": [
        {
            "title": "uv vs pip: Managing Python Packages and Dependencies",
            "url": "https://realpython.com/uv-vs-pip/",
            "snippet": "When it comes to Python package managers..."
        }
    ],
    "images": []
}

Social media results use the same shape as web/news results.

Field	Type	Description
`source`	string	`"social"`.
`title`	string	Post or page title.
`url`	string	Post URL.
`snippet`	string	Post excerpt.
`position`	integer	Result position (1-based).

Current platform behavior: Social search results may return empty for some queries depending on availability. Always check results.length before processing.

Result ordering and ranking

Current platform behavior: When querying a single source, position reflects the source’s natural ranking order. When querying multiple sources, results from different sources are interleaved and position may reflect a per-source rank rather than a global rank. metadata.totalResults is the total count across all requested sources and pages.

Parsing multi-source responses

When you query multiple sources at once (or omit sources), the results[] array can contain items with different shapes. Always check the source field of each result to determine which fields are available:

for (const result of response.results) {
    switch (result.source) {
        case "web":
        case "news":
        case "social":
            // Standard: title, url, snippet, position
            console.log(result.title, result.url);
            break;
        case "scholar-articles":
        case "scholar-articles-enriched":
            // Academic: standard fields + authors, citations, pdf_url, metadata
            console.log(result.title, result.citations, result.authors);
            break;
        case "scholar-author":
            // Author profile: name, affiliation, h_index, articles[]
            console.log(result.name, result.affiliation, result.h_index);
            break;
        case "ai":
            // AI overview: content, references[]
            console.log(result.content, result.references);
            break;
    }
}

Field presence by source

Use this reference to determine which fields are present for each source type.

Naming note: The API uses metadata in two different contexts. The response-level metadata is an object with totalResults, failedPages, and emptyPages. The per-result metadata field (scholar-articles only) is a citation string like "Author - Year - Publisher". Always use the full path (response.metadata vs result.metadata) to avoid confusion.

Standard fields — present in web, news, social, and scholar-articles / scholar-articles-enriched:

Field	Sources with this field	Notes
`source`	All sources	Always present
`title`	`web`, `news`, `social`, `scholar-articles*`, `ai`	AI: always `"AI Overview"`
`url`	`web`, `news`, `social`, `scholar-articles*`, `scholar-author`	Academic author: profile link
`snippet`	`web`, `news`, `social`, `scholar-articles*`	Absent in `ai`, `scholar-author`
`position`	`web`, `news`, `social`, `scholar-articles*`	Absent in `ai`, `scholar-author`

Academic article fields — scholar-articles and scholar-articles-enriched only:

Field	Type	Notes
`metadata`	string	Citation string: `"Author - Year - Publisher"`
`pdf_url`	string?	Direct PDF download link — handle outside Web Fetch
`authors`	array	`[{ name, profile_url, profile_id }]`
`citations`	integer	Total citation count

Academic author fields — scholar-author only:

Field	Type	Notes
`name`	string	Author full name
`affiliation`	string	Institutional affiliation
`website`	string?	Personal or institutional website
`interests`	array	`[{ title, link }]`
`thumbnail`	string?	Profile photo URL
`citations`	object	`{ all, since_2020 }` — different type than scholar-articles
`h_index`	object	`{ all, since_2020 }`
`i10_index`	object	`{ all, since_2020 }`
`articles`	array	`[{ title, url, year, citations, authors, publication }]`

Deep research mode fields — ai only:

Field	Type	Notes
`content`	string	AI-generated overview text
`references`	array	`[{ title, url, snippet }]` — fetch these URLs
`images`	array	`[{ url, alt, width, height }]`

Next steps

Recipes — site/date filtering, multi-page, academic, deep research, social, and workflow examples.
Web Search — main Search page with quickstart, request/response, and errors.
Web Fetch — fetch the HTML content of URLs returned by search results.

​Source capabilities

​Result shapes by source

​Result ordering and ranking

​Parsing multi-source responses

​Field presence by source

​Next steps

Source capabilities

Result shapes by source

Result ordering and ranking

Parsing multi-source responses

Field presence by source

Next steps