Whither Data Warehousing in the Teens?

James Kobielus
Information Management Blogs, January 4, 2010

People often use the end of a decade to say goodbye to trends that have played themselves out, or good riddance to things that have long since passed their cultural expiration dates. I like to use the beginnings of decades for that same purpose. What, we should ask ourselves, is not likely to last beyond the close of this new ten-year cycle?

In data warehousing, the most likely casualty of the Teens will be the very notion of a data warehouse. You can tell that a concept is on its last legs when its proponents spend more time on the defense, fighting definitional trench wars, than evolving it in useful new directions. Here’s a perfect case in point: a recent article by Bill Inmon, self-described “father of the data warehouse,” where he takes pains to specify what is not a data warehouse. Apparently, many of the approaches that we normally implement in our data warehousing architectures—such as subject-specific data marts, dimensional data structures, federated architectures, and real-time  data integration—don’t pass muster in Inmon’s way of looking at things. Though he didn’t mention hybrid row-columnar and in-memory databases by name, one suspects that Inmon has a similarly jaundiced view of these leading-edge data warehousing technologies.

What’s likely to happen in this decade is that the hidebound Inmon approach to data warehousing will become the exception, not the rule, in most enterprise analytic architectures. Actually, that’s already happened, for the most part. Few Forrester customers have built unified enterprise data warehouses on rigid Inmon design principles, such as third-normal-form relational data structures, hub-and-spoke topologies, and conformed data marts. Fewer still have signalled any intention to evolve toward that architecture. That’s because most companies’ requirements are too diverse, complex, and dynamic to be constrained in a design paradigm from the decade before last.

The very term “data warehouse” feels outdated. It implies two design principles that fit the Teens as poorly as ten-year-old sneakers on a real-world teenager. First, the proportion of “data”—in other words, rigidly structured information sets—will shrink as the amount analytic information originating in content management, social networking, and other non-traditional sources grows. Second, the role of the “warehouse”—in other words, a single master table of all reference data—will diminish as businesses seek architectures for managing analytic information and functions across stubbornly distributed environments.

Call this new paradigm “analytic clouds” or whatever. You can even call it “DW 2.0” if the old term feels less scary. But no amount of rearguard sniping at these emerging analytics architectures can stop this new order from becoming mainstream in the Teens.

Happy New Decade!