Pagination & Cursors

The Streamforge External API uses cursor-based pagination for all list endpoints. This approach provides stable, efficient pagination that avoids duplicates when data changes between requests.

How Cursor Pagination Works

Unlike offset-based pagination (e.g., page=1, page=2), cursor pagination uses opaque tokens that point to a specific position in the dataset. This means:

Stable results: New items added to the beginning won’t cause duplicates
Efficient: No need to skip through large offsets
Consistent: Results remain consistent even as data updates

Using Cursors

All list endpoints return a next_cursor in the response metadata:

{
  "payload": [...],
  "meta": {
    "paging": {
      "limit": 10,
      "next_cursor": "eyJpZCI6Ijk4NzY1NDMyMSIsImVuZGVkX2F0IjoiMjAyNC0wNy0wNFQxMTozMDowMC4wMDBaIn0="
    }
  }
}

Basic Pagination Pattern

async function fetchAllContent(platform, profileId) {
  let cursor = null;
  const allContent = [];

  do {
    const url = `/platforms/${platform}/profiles/${profileId}/content?limit=50${cursor ? `&cursor=${cursor}` : ''}`;
    const response = await fetch(url, {
      headers: { 'x-api-key': 'YOUR_API_KEY' }
    });

    const data = await response.json();
    allContent.push(...data.payload);

    cursor = data.meta.paging.next_cursor;
  } while (cursor !== null);

  return allContent;
}

Choosing a Limit

The limit parameter controls how many items are returned per page:

Minimum: 1 item
Maximum: 50 items (for most endpoints)
Default: 10 items
Quota cost: Each item returned costs 1 quota unit

Limit Selection Guidelines

Small datasets: Use limit=50 to minimize requests
Large datasets: Use limit=10-25 for better progress tracking
Real-time updates: Use smaller limits to detect new items faster

Cursor Stability

Cursors are stable tokens that remain valid even as new data arrives. However:

Cursors don’t expire, but they may become less efficient over time
If you pause pagination for days/weeks, consider restarting from the beginning
Cursors encode the sort position, so changing sort order requires a new cursor

Endpoints with Pagination

The following endpoints support cursor-based pagination:

GET /platforms/{platform}/content - List all content
GET /platforms/{platform}/profiles/{profile_id}/content - List content by creator
GET /platforms/{platform}/games/{game_id}/content - List content for a game

Best Practices

Always check if next_cursor is null before making another request. A null cursor means you’ve reached the end of the results.

Respect rate limits: Don’t paginate too aggressively; monitor your quota usage
Handle errors: Implement retry logic for failed pagination requests
Store cursors: If resuming pagination, store the last cursor to avoid re-fetching data
Batch processing: For large datasets, process items in batches rather than loading everything into memory

Example: Paginating Content

# First request
curl -H "x-api-key: YOUR_KEY" \
  "https://external-api.streamforge.com/platforms/twitch/content?limit=25"

# Response includes next_cursor
# Use it in the next request
curl -H "x-api-key: YOUR_KEY" \
  "https://external-api.streamforge.com/platforms/twitch/content?limit=25&cursor=eyJpZCI6Ijk4NzY1NDMyMSIsImVuZGVkX2F0IjoiMjAyNC0wNy0wNFQxMTozMDowMC4wMDBaIn0="

Common Pitfalls

Don’t use offset-based pagination patterns. The API doesn’t support page or offset parameters.

Assuming page numbers: There are no page numbers; only cursors
Reusing old cursors: While stable, very old cursors may not reflect current data
Ignoring null cursors: Always check for null to avoid infinite loops
Not handling empty results: An empty payload array with a null cursor means no results

Getting started

Core concepts

Workflows

Advanced topics

Using Cursors

Choosing a Limit

Limit Selection Guidelines

Cursor Stability

Best Practices

Example: Paginating Content

Common Pitfalls

​How Cursor Pagination Works

​Using Cursors

​Basic Pagination Pattern

​Choosing a Limit

​Limit Selection Guidelines

​Cursor Stability

​Endpoints with Pagination

​Best Practices

​Example: Paginating Content

​Common Pitfalls

How Cursor Pagination Works

Using Cursors

Basic Pagination Pattern

Choosing a Limit

Limit Selection Guidelines

Cursor Stability

Endpoints with Pagination

Best Practices

Example: Paginating Content

Common Pitfalls