Skip to navigation

Map a website

Synchronously lists the URLs of one website by following its links from a starting page, optionally only the paths that match select_paths. Use it to find the right pages inside a site, then read them with POST /v1/web/read instead of guessing URLs with search. Billed per URL returned, so a call that finds none costs nothing. Prices are in the billing section.

Unknown request fields and path expressions that do not compile are rejected with 400. The same operation is the data:web.map capability of POST /v1/capabilities/run and the web_map tool of the BeatAPI MCP endpoint.

Authentication

AuthorizationBearer

Send your API key as Authorization: Bearer <BEATAPI_API_KEY>.

Headers

Idempotency-KeystringOptional<=255 characters

Request

This endpoint expects an object.
urlstringRequiredformat: "^https?://"<=2048 characters

The page to start from. It must be a public http or https URL without a user name or password, and not localhost or a private-network IP address.

limitintegerOptional1-100Defaults to 50
At most this many URLs are returned. Each URL returned is billed.
max_depthintegerOptional1-3Defaults to 1
How many links away from the starting page to follow.
include_externalbooleanOptionalDefaults to false
Also list links that leave the site.
select_pathslist of stringsOptional

Regular expressions over the URL path, such as /docs/.*. Only matching URLs are listed. An expression that does not compile is rejected with 400.

exclude_pathslist of stringsOptional
Regular expressions over the URL path. Matching URLs are left out.

Response

The URLs found from the starting page.
objectenum
Allowed values:
request_idstring
Quote it in support requests.
urlstring
The starting page.
urlslist of strings

URLs found from the starting page, without duplicates and at most limit.

notestringOptional

Present only when urls is empty, saying what to try next (an empty map is not charged).

usageobjectOptional

What this call was charged, in US dollars. The same object a social-data call answers with.

Errors

400
Bad Request Error
401
Unauthorized Error
402
Payment Required Error
403
Forbidden Error
409
Conflict Error
429
Too Many Requests Error
500
Internal Server Error
503
Service Unavailable Error