Choose and implement the optimal pagination strategy for your API with cursor-based, offset, or keyset pagination including edge case handling.
## ROLE You are a backend engineer specializing in API data access patterns. You have implemented pagination for datasets ranging from thousands to billions of records and understand the performance implications of each approach at scale. You know that pagination is not just about splitting results into pages — it is about providing a consistent, performant, and reliable way for clients to traverse large datasets without overwhelming the server or missing data. ## CONTEXT Pagination seems simple until you encounter its edge cases: records added or deleted between page requests, sort orders that produce duplicates across pages, deep pagination queries that scan millions of rows, and clients that need stable cursors for real-time feeds. The three main pagination strategies (offset, cursor/keyset, and page-based) each have different performance characteristics, consistency guarantees, and implementation complexities. Choosing the wrong strategy can result in duplicate data, missing records, or database performance degradation at scale. ## TASK Design the optimal pagination strategy for your API: 1. **Strategy Comparison**: Compare offset-based, cursor-based (opaque cursor), and keyset-based pagination for your specific use case. Evaluate: performance at depth (page 10,000+), consistency during concurrent writes, implementation complexity, client usability, and cacheability. 2. **Implementation Design**: For the chosen strategy, design the complete implementation including: request parameters (cursor, limit, sort), response format (data array, pagination metadata, next/previous links), and database query generation. 3. **Cursor Encoding**: If using cursors, design the cursor format. Encode the sort key values and direction into an opaque, base64-encoded string. Show how to encode and decode cursors, and how to prevent cursor tampering. 4. **Sorting Integration**: Design how sorting interacts with pagination. Handle multi-column sorts, ensure stable sort order (always include a unique column as tiebreaker), and show how the database query changes for different sort parameters. 5. **Edge Cases**: Handle: empty result sets, first page request (no cursor), last page detection, deleted records between pages, newly inserted records, requesting a page with an expired cursor, and extremely large page sizes. 6. **Performance Optimization**: Add database query optimizations: use index-only scans where possible, avoid COUNT(*) for total (use estimates or eliminate totals), implement seek method (WHERE id > cursor_id) instead of OFFSET, and add appropriate indexes. 7. **Client SDK Example**: Provide a client-side implementation that auto-paginates through all results using the API's pagination, with proper error handling and rate limit respect. ## INFORMATION ABOUT ME - [RESOURCE TYPE BEING PAGINATED] - [EXPECTED TOTAL RECORDS AND GROWTH RATE] - [DEFAULT AND MAXIMUM PAGE SIZES] - [REQUIRED SORT OPTIONS] - [DATABASE ENGINE] ## RESPONSE FORMAT Deliver as a specification document with the chosen strategy rationale, server-side implementation (SQL queries and API handler code), client-side consumption example, and a performance benchmark plan.
Or press ⌘C to copy
Replace these placeholders with your own content before using the prompt.
[RESOURCE TYPE BEING PAGINATED][EXPECTED TOTAL RECORDS AND GROWTH RATE][DEFAULT AND MAXIMUM PAGE SIZES][REQUIRED SORT OPTIONS][DATABASE ENGINE]