Agile business intelligence

Agile business intelligence

As data volumes increase to the point where we call them "big data", users' appetites increase alongside. They want to be able to quickly analyze these huge volumes of data and achieve immediate business intelligence results. Agile approaches are becoming common in software development because they enable users to receive tangible results in short time intervals. Can we use such agile approaches in business intelligence as well? -- MonitorPro 01/14, p. 28-29...
Read More
Data quality vs. data gatekeeping

Data quality vs. data gatekeeping

Too often I meet data warehouse development teams who are implementing what they call »data quality« in order to help the business have cleaner data to work with. But what they are really implementing is »data gatekeeping« which hurts the business even more than if they didn't have any data quality in the first place. Let's look at an example. A data quality rule may say that the value for education level in the customer table must exist in the education level reference table. Fair enough, right? We must ensure that all reference data values in our tables match their corresponding values in the reference tables. What happens when we receive a customer record with a value for education level that doesn't exist in the education level reference table? Our data gatekeepers say: »No, this record is not allowed to enter the data warehouse because it violates a data quality rule.« Fair enough, right? But think about it. We are not loading a customer...
Read More
Visualization: a different view of data

Visualization: a different view of data

Humans often understand data better if we see it in a graphical format rather than just looking at numbers in a spreadsheet. Data visualization technologies enable us to transform data into visual building blocks that enable us to easily spot trends, discover connections among data elements, present interesting discoveries to others or to detect trends. -- MonitorPro winter 2013, p. 24-26...
Read More
A quick introduction to MapReduce

A quick introduction to MapReduce

Whenever we talk about large volumes of data or »Big Data«, we almost always mention Hadoop, which may have even become a synonym for storing and accessing huge amounts of unstructured data. Although there is lots of chatter about using Hadoop in business environments, many business users don't know what it is and how to make sue of it. -- MonitorPro autumn 2013, p. 30-33...
Read More
Diving into data

Diving into data

All of us who work with data could donate some of our expertise to charitable causes through non-profit organizations. The organization DataKind helps to bring together volunteers with organizations who need data related services. I decided to attend an event organized by DataKind's European operation DataKindUK based in London. The event, which was basically a hackathon, took place the last weekend of July and was named the "DataDive". The event was well attended, both by volunteers and by organizations who wanted our help. Four organizations were chosen to present their requirements, goals and expectations. They each brought their own data and made it available to the volunteers. I decided to help the organization HelpAge International who was looking for meaningful data visualizations to be used in their brochures and marketing materials. Their aim is to bring attention to the issues that older people face when trying to make a decent living in all the countries in the world. The event in London had a very hacker feel. Each...
Read More
Using social media in business intelligence

Using social media in business intelligence

Social media is increasing in popularity, both among individuals and businesses. Companies have their social media presence where they listen to what customers say about their products, services and brands. Such information can be useful in business intelligence solutions because they represent another useful source of data. -- MonitorPro magazine, summer 2013, p. 16-19...
Read More
Relational databases are still important despite more modern technologies

Relational databases are still important despite more modern technologies

A surprising revelation related to data warehousing was that Facebook uses a traditional data warehouse based on relational technology. The news is interesting because Facebook is one of the representatives of the new generation of companies who deal with very large amounts of data and therefore don't want relational technology because it is too cumbersome for their needs. They have been inventing new database technologies that can handle such large amounts of data, have fast response times and they typically don't support the widely used query language SQL. At the recent The Data Warehousing Institute conference, a representative from Facebook explained that they decided to use relational technology for their data warehouse because the right technology should be used for the right purpose. Newer database technologies, especially those that utilize the Hadoop platform like Facebook does, are better at handling large amounts of unstructured data. Relational databases, on the other hand, require a more strict database structure that typically includes constraints on data elements...
Read More