Pagination: Handling Large Request Resources Like a Pro

Okay, imagine someone asked you to send them a list of every person in your city. You'd probably say "That's impossible! There are too many!"

That's exactly the problem pagination solves.

The Phone Book Story

Remember old phone books? Phone books had thousands of listings. Did they print one giant list? No! They:

1. Organized alphabetically
2. Added page numbers
3. Let you jump to sections (A-D, E-H, etc.)

That's pagination!

Why Do We Need Pagination?

Let's see this instance. You're building Instagram's API. Someone requests:

Your database has 500 MILLION posts.

Without pagination, you'd try to send:

Problems:

❌ Takes 10 minutes to load

❌ Uses 50 GB of bandwidth

❌ Crashes the user's app

❌ Kills your server

❌ Costs you thousands in bandwidth fees

Solution: Only send a small "page" at a time!

Method 1: Offset Pagination (The Page Number System)

This is the most intuitive method. It's like reading a book:

Page 1: Items 1-10

Page 2: Items 11-20

Page 3: Items 21-30

...and so on

Let’s see how this looks in an API:

Request for Page 1:

━━━━━━━━━━━━━━━━━━

Server thinks:"Page 1, limit 10... let me calculate:

Offset = (page - 1) × limit = (1 - 1) × 10 = 0

So, fetch items from position 0 to 9"

Response:

Now, the user wants page 2:

Request for Page 2:

━━━━━━━━━━━━━━━━━━

Server calculates:

Offset = (2 - 1) × 10 = 10

So, fetch items from position 10 to 19

Response:

Let’s see a Complete Example

Imagine you're building a blog API with 243 posts. A client wants 20 posts per page:

First Request:

Your API returns:

Notice how we are giving the client helpful info:

• How many total items exist

• How many pages there are

• Whether there's a next/previous page

• Direct links to navigate

The Problem with Offset Pagination

Let’s see why this method has issues. Imagine you're on page 5, looking at posts:

Page 5 shows posts 41-50

While you're reading, someone deletes post #35

Now:

Post #41 becomes post #40

Post #42 becomes post #41

When you click "Next" to page 6:

(Expecting to see posts 51-60)

But now you see posts 50-59!

You MISSED post #50 because everything shifted!

This is called the "shifting data problem."

Method 2: Cursor Pagination (The Bookmark System)

Cursor pagination is like using a bookmark. Instead of remembering page numbers, you remember "where you left off."

Let see how:

First Request:

━━━━━━━━━━━━━━

Response:

That next_cursor is encoded data saying "you left off at ID 110"

Second Request: ━━━━━━━━━━━━━━━

Server decodes cursor: "They left off at ID 110"

Why Cursor Pagination Solves the Shifting Problem

The same deletion scenario:

You're looking at items with cursor pointing to ID 110

Someone deletes item ID 35

Next request:

Server: "Give me items AFTER ID 110"

Result: You still get items 111-120 Nothing shifted because you're using absolute identifiers!

Real-World Example: Social Media Feed

Think about your Instagram/Twitter feed. Let me walk you through what happens:

You open the app:

━━━━━━━━━━━━━━━━━

App:

Server returns:

You scroll down:

━━━━━━━━━━━━━━━━━

App:

Server: "Show 20 posts AFTER post_980"

Even if:

• New posts are added
• Posts are deleted
• Posts are edited

You'll never see duplicates or miss posts!

Which Pagination Method Should You Use?

Let see a pagination technique:

Use Offset (page numbers) when:

• Users need to jump to specific pages (e.g., search results)
• Total count is important
• Data doesn't change frequently
• Example: Product catalogs, blog archives

Use Cursor (bookmarks) when:

• Data changes frequently
• Users mostly scroll forward
• Exact position isn't important
• Example: Social feeds, notifications, real-time data

Key Takeaways

1. Pagination prevents returning millions of records in one response — essential for performance and usability
2. Offset-based pagination is simple but breaks with changing data — page 2 may show duplicates if new items are inserted on page 1
3. Cursor-based pagination is more reliable for real-time data — uses a pointer to the last seen item, unaffected by insertions
4. Always include pagination metadata in responses — total count, next/previous links, and current page help clients navigate