Polyglot Persistence: the Future of Database Management

I recently went to the NoSQL Now! Conference in San Jose, and I found it to be pretty interesting. The conference was not heavily commercialized, but there were definitely a lot of companies there, and the people were great. Particularly interesting to me was this relatively new notion that I learned about called polyglot persistence.

Polyglot persistence refers to the use of both an RDBMS and one or more NoSQL databases as the backing store for modern applications. In the past, when you created an application you needed some way to store information about the objects that the software was modeling. You needed a backing store, and relational databases won out during a brief battle with object databases in the 1990s.

Object databases used technology that enabled more complicated relationships between objects, also allowing those objects to be quickly stored and retrieved. But it never quite caught on. Relational databases won out, either due to better marketing or a better focus on implementation.

From Relational Databases to NoSQL Databases

I worked on several applications in the 90s, and for business applications, storing and retrieving data from a relational database was just fine. Occasionally there were slow queries, and things sometimes had to be changed in order to improve performance. Otherwise, relational databases got the job done.

With the growth of the Internet and the processing of massive quantities of data in the early 2000s, relational database technology was pushed to its limits. One way to exceed those limits through clustering: having multiple copies of the database in different locations and online, then having a way to keep and synchronize those databases. Transactions had to be stored across all of those copies, and that turned out to be very expensive.

With relational databases, there’s this object relational mapping – or ORM – that needs to occur, because relational databases don’t store complex lists or other data structures within the tables, which are simply a row-and-column structure. The mapping has to take place between this complex list of structures and other data structures, with relationships in between them. This can be dispensed within these NoSQL databases.

Why Use a NoSQL Database?

The first of two main reasons for using a NoSQL database is the need for performance scale and for quantities of data that can only be handled by very expensive clusters of relational database technology. The second reason is that you want to have a better match between the model of data in your application and how the data is actually stored in the database.

Companies such as Google and Amazon developed their own NoSQL databases. Google developed Bigtable; Amazon created DynamoDB. The concept of polyglot persistence means that you might still have a relational database to store some information, but not necessarily alone. You would also use one of these NoSQL databases to store massive amounts of data. So although you might have a relational database to manage user accounts within a system, the data that’s actually being consumed – whether it’s video or just huge quantities of transactional data – would be stored in one of these NoSQL databases instead.

Utilizing polyglot persistence, you can maximize the potential of relational databases yet still benefit from the flexibility and scalability that NoSQL provides.