How do I get the next search page?
To fetch the next X search page, call twitter.web.fetch_search_timeline again with the same keyword and search_type, passing data.next_cursor from the previous response unchanged as params.cursor. Omit cursor on page one, and start a fresh page-one search whenever you change keyword or switch between Top and Latest.
For example, when page one queries a brand name with search_type:"Latest", page two sends the exact same brand name, search_type:"Latest", and page one’s data.next_cursor. See Search and CSV export.
When do I stop, and what can I report?
Read the stop state recorded in coverage.json and act accordingly:
HTTP errors, business failures, wrong actions, missing arrays, and malformed JSON are failures rather than successful empty pages. Keep the attempt logs and raw response bytes, then follow the empty-result diagnosis. This workflow never retries automatically. If you receive approval to investigate after an empty_with_cursor stop, record that investigation as a new batch instead of appending the request to the stopped cursor chain.
Does a two-page budget mean forty posts?
No. A two-page budget limits the number of requests rather than guaranteeing a fixed post count per page, unique post count, or date range. Page two can overlap with page one, so fewer rows after deduplication does not mean posts were lost.
At the current Inspect rate of $0.003 per request, two successful pages cost an estimated $0.006, excluding retries or extra pages. Stop early on an empty page or an absent cursor instead of spending the remaining budget just to complete two pages.
How do I continue saved JSON without paying for it again?
Export saved responses locally using export_saved_x.py from the search guide; fetching responses and exporting saved responses are separate workflows.
- Record each page’s actual parameters, query time, and original response filename in
pages.json. - Run
export_saved_x.py.export_saved_x.pyverifies that every page shares the samekeywordandsearch_type, that each cursor continues the previous page, that request IDs are distinct, and that collection timestamps are ordered, then writesposts.csv,coverage.json,sources.json, and byte-identical copies of the source files without network access. - Inspect
stop_reasonbefore fetching another page. Onlysaved_batch_endedwith validnext_paramsprepares a continuation, and sending that continuation still requires budget approval. - Save the new response, append the page’s actual metadata to
pages.json, and export into a new output directory.
Never rerun page-one collection solely to add CSV columns or adjust local parsing.
How should overlaps, timing and evidence appear in the report?
posts.csv retains the first observation of each original tweet_id. Later observations of the same post remain in the raw JSON files, and coverage.json records the duplicate count. Do not overwrite the first observation’s like count with a later value.
State the actual earliest and latest created_at timestamps and every page’s collection time in the deliverable. A two-page sample count is not total mention volume. File hashes identify the saved bytes without proving server authenticity, timestamp accuracy, or completeness.
Contract reference: data:twitter.web.fetch_search_timeline.

