Infobright Blog

Polyglot Persistence

We have all witnessed the significant growth in the use of a wide variety of programming languages over the past ten years. Much debate has also occurred about which language is best suited for a particular use case. Clearly developers have discovered that one language doesn't fit all. This has resulted in all kinds of conversations around the topic of polyglot programming over the years,  which frequently involves using a combination of languages to achieve a particular use case objective. 

Just like programming languages, we are now experiencing an amazing amount of innovation along with significant growth in data persistence solutions as volume of data created and manipulated grows at a significant pace. With this growth in data comes much debate, as we saw with programming languages, as to whether NoSQL, Traditional SQL Relational Database, Columnar Database, Graph Databases, Key Value Stores, Hadoop  and the like are the perfect solution for handling a data persistency requirement for a particular use case. The industry is now describing this as Polyglot Persistence. It reminds me of my days in the integration space where we saw much debate about ETL, Data Integration, Application Integration, Content Integration, Message Based Integration, etc. Ah, Polyglot Integration. Just like languages and data, integration is in a similar boat. Not all integration solutions are the same or even usable for a particular use case. But back to this topic of Polyglot Persistence. 

At the upcoming Boston Big Data Summit, we have a unique opportunity to hear from Infobright, VoltDB, Cloudera and 10Gen (MongoDB) as we all discuss real world use cases that have achieved success with our data solutions. My sense is that the panel conversation might revolve around the topic of Polyglot persistence as many of these solutions can absolutely work together in the context of an overall applications requirement for persistent data. 

Much of the discussion around Polyglot Persistence involves some discussion of how traditional SQL databases are not always the best for the persistent problem at hand. The discussion then turns to the topic of NoSQL, Key Value stores and transient stores like Memcached as illustrated by this conversation by Adam Wiggins at http://blog.heroku.com/archives/2010/7/20/nosql/ I agree with many of the points that Adam makes in this blog post. We will clearly see companies taking advantage of multiple data persistent solutions as time goes on and the debates about SQL versus NoSQL versus Hadoop versus (name the database solution here) will become something of the past as we have seen in languages and even integration.