Joinutility seperatorLogin utility separator Infobright.com

Infobright Blog

11
Aug

Billions and Billions of Rows - That’s A Lot Of Data!

Bob Zurek's photo
by Bob Zurek     Wed, Aug 11, 2010

As you recall, Carl Sagan popularized the statement "Billions and Billions" as he referred to the number of planets, stars and galaxies in the universe. In speaking with customers who use Infobright as their analytic database of choice, I get very excited when I hear customers talk about how much data they are loading into and storing in Infobright at a very low cost and with a very high level of compression. In fact, it wouldn't be unusual for me to respond with something akin to the Staples commercial (that's a low price)  with "That's A Lot Of Rows".  Here are a few recent real world stats from several of our customers where we asked the question, How many rows in your largest table? By the way, that's just one table.

How many rows in your largest table:
Mobile Analytics: 2 billion rows
Financial Services: 19 billions rows
Entertainment Company: 810 million rows
Social Networking: 800 million rows

With lots of rows comes a requirement for more storage. The trouble is that the cost of managing that storage is said to range from 2-10x the cost of acquiring the storage hardware itself. To help reduce the cost of storage and to enable the ability to store more historical data, data compression is now a key required feature for in analytic database solutions. On average one should be able to achieve 10-40:1 compression ratios (raw data to size on disk), which is what Infobright customers  routinely get. As I wrote in a previous blog, here are some real world stats on compression being achieved by a sampling of customers:

A large media company: 16:1
One of the top search engine/portals: 150:1 on one table, another 50:1
A large retailer: 30:1
A performance based digital marketing company: 40:1

At Infobright we are taking steps in our roadmap to further expand our compression capabilities without sacrificing query performance and administrative simplicity.

If you are interested in learning more about our compression capabilities in Infobright, I'd like to encourage you to take a look at our YouTube Channel http://www.youtube.com/infobrightdb. Here you will find a quick 3 minute tutorial on this subject. In the mean time, we will continue to capture billions and billions of rows on low cost commodity hardware. 

Infobright     Tags:
Please login or register to post a comment.