We live in a brave new (to some) world of databases other than a relational database with a SQL interface. Normally end users never notice a difference, but the astute viewer may notice the slow demise of an old friend: pagination.
Traditionally with SQL databases pagination has looked something like this:
There are previous and next links as well as links for jumping right to the beginning and end. Pretty boring stuff.
What’s interesting is that this standard interface is disappearing in favor of something like this:
Twitter
Facebook
And soon beta testers of Urban Airship’s push service for Android will see a More link on the page that lists devices associated with their app:
The simplest possible explanation for this dumbing down of pagination is that count (for total pages) and skip/offset are expensive operations.
Not only are those operations expensive, but in eventually consistent databases, which many modern non-relational databases are, they’re extremely expensive, if not impossible, to perform.
Cassandra
At Urban Airship we, like Facebook, use Cassandra: a distributed column-based database. This deals two deadly blows to traditional pagination:
- No way to count columns in a row (without reading every column).
- No way to skip by numeric offset (so you can’t say, skip to page 5).
In Cassandra columns are ordered, so you start reading from the beginning and read N+1 columns where N is the number of items you’d like to display. The last column’s key is then used to determine whether the More link is enabled, and if so, what key to start the next “page” at.
Both of those are solvable problems if you really need them, but I would suspect you would end up creating a column count cache as well as some sort of table of contents for the various page offsets. Not what I want to spend my time implementing.
The fact of the matter is that for many use cases, a simple More button works just as well (if not better) than traditional pagination. It’s also far cheaper to implement, which means more developer time free to work on features and more hardware resources available to push your 140 character insights around the web.
MongoDB
I should note that MongoDB is fairly unique in the non-relational database world as its dynamic querying features include count and skip operations. However, as with any database, you’ll want to make sure these queries hit indexes.
Sadly MongoDB currently doesn’t have the distributed features necessary to automatically handle data too big for a single server.
