Companies have been analyzing data for years. That’s not new. Before we get too far into this topic, let’s set the stage for this topic. Let’s explain what has changed. What does ‘big data’ mean?
It’s actually a difficult area to define. It’s actually a little bit like cloud computing. Any discussion of cloud usually spends the first 45 minutes arguing what everyone means by it. I am going to try to avoid that when we’re talking about big data trends.
Companies have had big databases for years, and some of them are very, very large. One of the things that can be implied by big data is that the database has become so large that it is no longer easily dealt with, managed, reported on, or analyzed with traditional database tools.
That’s only part of the problem though. One of the other parts is sometimes traditional database tools can look at a really large data set, but they can’t do it particularly fast. So maybe if you want to look at a smaller data set, but get the data in real time, that would be another instance where you might want to use the new big data tools.
| #1 Ranking: Read how InetSoft was rated #1 for user adoption in G2's user survey-based index |
|
Read More |
Third when you’re just looking at a large data set or maybe a data set where you want numbers fast, you may also want to extract new information that is not produced by traditional database tools. This is something that a lot of people are really excited about.
People are talking about it, yes, but it’s still relatively new on the business horizon. It was really just in 2009 when it was roughly estimated as a $100M business. Now the prediction this year is that it will be $1 to 1.5B. Where is all that growth going to come from?
First of all, before we get to that, let’s take a look at how that number was arrived at. When you go out there and try to get any estimates on the size of the industry, there’s very little out there. To a large part that’s due to people fighting about the definition.
If you take a look at total IT spending forecast, and then look at the part that is responsible for traditional databases and some associated areas like ERP software or business intelligence software. If you assume a certain percentage of databases are too big or need data too fast, or are looking for specific insights, what kind of number does that come up with? That’s how you get a 1 to 1 and a half billon number.
 |
“Flexible product with great training and support. The product has been very useful for quickly creating dashboards and data views. Support and training has always been available to us and quick to respond.
- George R, Information Technology Specialist at Sonepar USA
|
Simultaneously, people have done bottom-up analyses interviewing some of the largest companies in the world, and asked them what their big data spending plans were for next year. When you do that, you’re once again on that order of 1 to 1 and a half billion. Now just to be clear, total IT spending around the world is trillions of dollars, so big data is still an incredibly small part of a really big pie.
Now where is that coming from? Not surprisingly technology is the first place. Big data came out of very large technology companies grappling with enormous data sets, especially in the search area, not surprisingly. That was the first area where it came out of.
Social media generates a lot of data that sometimes best handled using big data tools. Other ones that make a lot of sense, and you can probably guess these. These are companies that have lots of data about tens of millions of customers and touch them on an hourly or more frequent basis.
So media companies are really looking at this. The gambling industry, gaming, is a big one. Financial services, almost every bank around the world is looking at or thinking about big data projects. Media is another one where you have got companies with a lot of customers, tens of millions in the audience, and they want to get greater insights. So those are the industries that appear to be first. One I want that I almost forgot is retailing.