Tutorial

How to Read Public X Posts with Python and the X API

Use the official X API to fetch one page of public posts, then handle permissions, pagination, request budgets, and duplicate records.

To measure public discussions of your own product on X, start with the official post search endpoint. Keep the first query narrow, such as a public product name or an account you manage. Confirm the app's access and usage allowance before choosing what to store.

Steps

  1. 1

    Check access and choose the time window

    Review the app's search access and usage in the developer console. Recent search is suitable for the last seven days. For an older period, confirm what historical access your plan includes and how its usage is billed. Put credentials in environment variables or a secret manager. Log status and request identifiers without logging tokens.

  2. 2

    Request the first page with Python

    This example uses Requests to read one page. Replace the example query with a subject you are authorized to process and set X_BEARER_TOKEN in your environment. Printing the post ID and time is enough to confirm whether the first sample meets the reporting need.

    python
    import os
    import requests
    
    response = requests.get(
        "https://api.x.com/2/tweets/search/recent",
        headers={"Authorization": "Bearer " + os.environ["X_BEARER_TOKEN"]},
        params={
            "query": '"example product name" lang:en',
            "max_results": 10,
            "tweet.fields": "created_at",
        },
        timeout=20,
    )
    response.raise_for_status()
    for post in response.json().get("data", []):
        print(post["id"], post.get("created_at"))
  3. 3

    Paginate and deduplicate during incremental updates

    After checking the first response, use the API's pagination token for subsequent pages. Record the time range processed by each run and deduplicate by post ID. Put a ceiling on pages and requests per run so the report has a predictable cost.

    For 401 or 403, review credentials, endpoint access, and app status. For 429, use the wait indicated by the response or API guidance. Retain the fields needed for analysis and apply the deletion and retention rules that cover your use.

Frequently asked questions

Why does the same query return different counts later?

The recent-search window moves and the underlying posts can change. Keep the query time, requested time window, page position, and deduplicated count with each run.

Do I need to rotate proxy IPs for this integration?

X controls access and usage through the API. Use a stable connection and authorized credentials, and manage cost through query limits and incremental reads.

References

X API post search

X API authentication

Original topic from Miya IP

Related resource

REDROCK / CONTACT

Get in touch

Choose a channel to discuss products and setup.

Telegram@RedrockIP
Email supportsupport@redrock-ip.com