How do I list a site's URLs? Does Map find every page?
Use POST /v1/web/map from a public starting page to list URLs inside a site. Map does not guarantee every page on a domain: each request returns at most 100 deduplicated URLs and follows links only to max_depth (1–3, default 1).
Scope a crawl with depth and path filters
Start from a structured entry page (such as a help-center index) and narrow the output before spending credits on unrelated sections:
limit:1–100(default50). Billing is$0.001per returned URL; a call that returns zero URLs costs nothing and adds anotewhen a page builds links with JavaScript or links only externally.max_depth:1–3hops from the startingurl(default1).select_pathsandexclude_paths: Up to 10 path regular expressions each (up to 200 characters per pattern; invalid regex returns400 bad_request).include_externaldefaults tofalse.
Verify completeness and read target pages
Record the starting url, max_depth, and path filters alongside the URL inventory, group discovered URLs by topic, and pass selected pages to POST /v1/web/read to inspect page content. When a section looks incomplete, compare the Map output against the site’s sitemap.xml or official directory.
Contracts and related questions
Before running a request, inspect the contract for free: data:web.search, data:web.read, data:web.map.

