Among the myriad of year-end analyses and forecasts I was very much amused by reading Jill Dyché’s forecasts. I like her style because unlike other analysts who generally forecast what will happen she chose to forecast what will not happen in 2011. I always sympathize with her when I read her blog posts as she seems to come across similar data quality issues as I do. In her forecasts she comes to the conclusion that in 2011 management won’t want to invest in data governance and business users won’t help to define business rules. All this despite the slowly dawning realization that data is valuable.
My experiences are similar. I believe that developers will continue to spend countless hours to wrangle data: to cleanse, transform and load into the data warehouse, to resolve ambiguities, to join incompatible data sources and so on. In order to avoid all this work, the company should implement data governance on a corporate level. This should include master data management, capturing business rules, implementing best practices and industry standards as well as consolidation of disparate data sources. Will all this happen in the near future? In this regard I’m a pessimist just like Jill and I totally agree with her dark cloud prediction.