Big Data – a problem or a promise?

Posted on November 12, 2012 by JP Fendrich

The term “Big Data” is not well defined. It’s a buzzword used in many different ways for different reasons. I often notice a confusion between what is data and what is information. A huge lot of data doesn’t mean that we have a huge lot of information. The data is just ones and zeros or other ways to represent information but it is not information until we interpret the data into something understandable into our brains.

According to the analysis firm Gartner the amount of data in the world doubles every 18 months. That means that the last 18 months more data was generated than all other previous years combined. The question is how useful all this data is and how it can be utilized.

Let’s make a practical way of looking at this.
Within the automotive business the volumes of sensor data, streaming data and other types as well is increasing exponentially. We are starting to have:

• Huge volumes of data that can’t be handled in normal database managment systems.
• Types of data, such as streming data, office documents etc that are not well handled by standard databases.
• Real time data that we also want to access in (near) real time. Ordinary database systems are not built for this.

The classical way of analyzing the content in our databases is to extract “cubes” or some other type of predefined structure and get reports or to make predefined searches. Nowadays most “normal” databases can be hosted as in-memory databases. That means that you can search them in any way you want very fast compared to on-disk databases. The demand on a predifined structure is not at all that This is one practical way of looking at “Big Data”

Here is an example of what can be done in real time or very close to real time:

When it comes to volumes of data that are so huge that it can’t be handled by normal database management systems there some new Non-SQL dabase systems coming up. The field is still very unmature.

I know that withinin the automotive business, we have so much data collected and not fully used in any way. There are so many opportunities by combining, structuring, analyzing current data to get valuable information and by that making better decisions.

Let’s get analyzing!

 

Related articles

• Big Data: The Next Frontier in Competition, Innovation and Productivity
• Big Data To Become Powerful Driver Of IT Spending – Gartner
• Big Data Is No BuzzWord – It’s Real | Forrester Blogs