IDC Big Data & Business Analytics Forum 2014

IDC Big Data & Business Analytics Forum 2014

Dealing with today’s huge amounts of data requires analytics. IDC’s recent Big Data and Business Analytics forum was all about connecting data and analytics. The organizers of the event addressed questions such as: what are the forces behind the explosion within the digital universe? Technology has certainly helped, by driving down the cost of creating, capturing, managing, and storing information to a sixth of what it was in 2005. The real impetus, nonetheless, is financial. Since 2005, investments in the digital universe among enterprises have expanded some 50% to $4 trillion – money spent on hardware, software, services, and staff, ultimately, to generate revenue. I was one of the speakers at the event. My presentation Quick results in the conversion of large amounts of data into business information was about agile approaches in the implementation of solutions for data analysis, business solutions in real time and predicting the likelihood of business events. The magazine MonitorPro wrote about me in the February 2014...
Read More
Agile business intelligence

Agile business intelligence

As data volumes increase to the point where we call them "big data", users' appetites increase alongside. They want to be able to quickly analyze these huge volumes of data and achieve immediate business intelligence results. Agile approaches are becoming common in software development because they enable users to receive tangible results in short time intervals. Can we use such agile approaches in business intelligence as well? -- MonitorPro 01/14, p. 28-29...
Read More
Data quality vs. data gatekeeping

Data quality vs. data gatekeeping

Too often I meet data warehouse development teams who are implementing what they call »data quality« in order to help the business have cleaner data to work with. But what they are really implementing is »data gatekeeping« which hurts the business even more than if they didn't have any data quality in the first place. Let's look at an example. A data quality rule may say that the value for education level in the customer table must exist in the education level reference table. Fair enough, right? We must ensure that all reference data values in our tables match their corresponding values in the reference tables. What happens when we receive a customer record with a value for education level that doesn't exist in the education level reference table? Our data gatekeepers say: »No, this record is not allowed to enter the data warehouse because it violates a data quality rule.« Fair enough, right? But think about it. We are not loading a customer...
Read More
Visualization: a different view of data

Visualization: a different view of data

Humans often understand data better if we see it in a graphical format rather than just looking at numbers in a spreadsheet. Data visualization technologies enable us to transform data into visual building blocks that enable us to easily spot trends, discover connections among data elements, present interesting discoveries to others or to detect trends. -- MonitorPro winter 2013, p. 24-26...
Read More
A quick introduction to MapReduce

A quick introduction to MapReduce

Whenever we talk about large volumes of data or »Big Data«, we almost always mention Hadoop, which may have even become a synonym for storing and accessing huge amounts of unstructured data. Although there is lots of chatter about using Hadoop in business environments, many business users don't know what it is and how to make sue of it. -- MonitorPro autumn 2013, p. 30-33...
Read More
Diving into data

Diving into data

All of us who work with data could donate some of our expertise to charitable causes through non-profit organizations. The organization DataKind helps to bring together volunteers with organizations who need data related services. I decided to attend an event organized by DataKind's European operation DataKindUK based in London. The event, which was basically a hackathon, took place the last weekend of July and was named the "DataDive". The event was well attended, both by volunteers and by organizations who wanted our help. Four organizations were chosen to present their requirements, goals and expectations. They each brought their own data and made it available to the volunteers. I decided to help the organization HelpAge International who was looking for meaningful data visualizations to be used in their brochures and marketing materials. Their aim is to bring attention to the issues that older people face when trying to make a decent living in all the countries in the world. The event in London had a very hacker feel. Each...
Read More
Being more productive on projects

Being more productive on projects

Once again I was invited to participate in PMI's publication PM Network, this time in their June 2013 issue. I contributed tips about how to be more productive on projects. The article lists ten miscellaneous tips for project managers. One of them was mine, about resolving minor issues by clearing the air via telephone call rather than entering a lengthy email correspondence: “Whenever I detect an undertone such as sarcasm or anger in an email, I never respond by email. I usually telephone the sender and ask him or her to clarify, which allows me to get a clearer understanding of his or her motives. We can resolve minor issues in passing, not having to explain them in lengthy emails.” In my project management experience, I find it much more efficient to pick up the telephone and call someone when there is an issue to be resolved, a matter to be settled or something ambiguous to be clarified rather than firing off an email. By...
Read More
Using social media in business intelligence

Using social media in business intelligence

Social media is increasing in popularity, both among individuals and businesses. Companies have their social media presence where they listen to what customers say about their products, services and brands. Such information can be useful in business intelligence solutions because they represent another useful source of data. -- MonitorPro magazine, summer 2013, p. 16-19...
Read More
Relational databases are still important despite more modern technologies

Relational databases are still important despite more modern technologies

A surprising revelation related to data warehousing was that Facebook uses a traditional data warehouse based on relational technology. The news is interesting because Facebook is one of the representatives of the new generation of companies who deal with very large amounts of data and therefore don't want relational technology because it is too cumbersome for their needs. They have been inventing new database technologies that can handle such large amounts of data, have fast response times and they typically don't support the widely used query language SQL. At the recent The Data Warehousing Institute conference, a representative from Facebook explained that they decided to use relational technology for their data warehouse because the right technology should be used for the right purpose. Newer database technologies, especially those that utilize the Hadoop platform like Facebook does, are better at handling large amounts of unstructured data. Relational databases, on the other hand, require a more strict database structure that typically includes constraints on data elements...
Read More