llms.txt is a machine-readable index of this documentation, written for large
language models and AI agents. It maps every page to a one-line summary so a
model can pull the right reference directly instead of crawling the whole site.
It’s hosted at the root of the docs and stays in sync with the site.
The index is served at
/llms.txt and always reflects the
latest documentation. Every page is also available as Markdown — append
.md to any docs URL (for example, /general/pricing.md).What’s in it
The index groups every page by product area, mirroring this site:| Section | Contents |
|---|---|
| Getting started | Introduction, Pricing, Rate limits, Credits, and the MCP server |
| Company, Person, Job, Web, Social Post | Each API area’s guides and reference |
| Migration guides | Legacy endpoint → current replacement mappings |
| API reference | Every endpoint in the 2025-11-01 version |
Use it with an AI tool
Give your AI tool or agent the URLhttps://docs.crustdata.com/llms.txt. Tools
that support the llms.txt convention discover it automatically; others let you
paste the URL. The model reads the index, then fetches only the pages it needs
as Markdown — keeping answers accurate and context small.
What to do next
- Read the index — open
/llms.txtto see every page and its summary. - Connect an assistant — the MCP server lets Claude, Cursor, and other clients query Crustdata in natural language.
- Start from the top — the Introduction maps the five API areas.