InetSoft Webinar: Data Lake as a Necessary But Not Sufficient Foundation

This is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "10 Biggest Big Data Trends."

Holly: I was just going to say, I think of the data lake as a necessary but not sufficient foundation for analytics and BI, and so it's great that we can get so much data into the lake so quickly. Our next challenge is getting the data out of the lake in an agile fashion to perform the analytics, and that's where the focus is shifting now. A lot of customers have expanded and hydrated the lake. It's already built out. It's there.

Now we need to take advantage of that lake and keep it agile. Part of the whole benefit of the Big Data platforms is that they are kind of moving away from less agile solutions like traditional data warehouses which could take a long time to settle. While they're building out the lake we need to keep that agility of what do we do with the data and have it available more on demand, at the time of need, for the analysis, and the data lake is a foundation for that.

Abhishek: Great, alright our next trend --- architectures mature to reject one-size-fits-all frameworks. Larry, can you tell us some more what are you thinking with this one.

View a 2-minute demonstration of InetSoft's easy, agile, and robust BI software.

Larry: Sure, this trend stems from an earlier trend we talked about which was that Hadoop is no longer just a batch processing platform for data science use cases. It has become the engine for ad hoc analysis in many cases. It's being used for operational reporting on day-to-day workloads, the kind that have traditionally in the past have been handled by data warehouses.

So going forward we will increasingly see customers rejecting one-size-fits-all frameworks, and by that wheat I mean, it is not simply deploying Hadoop and putting InetSoft on top of it. How you deploy Hadoop? How do you access data from Hadoop? It depends on a lot of factors, which include the personas who are asking these questions, the volumes of data you are looking at, how frequently that data needs to be answered, needs to be accessed, the speed of data and the level of aggregation.

All of these questions need to be considered before committing to a data strategy, and to give you an example, you know we are actually going to profile an InetSoft customer story in a week's time in a webinar, and when I look at their architecture, they use, Kafka for events processing, Hadoop for data processing, Hadoop Spark and Presto.

All of this gets fed into Amazon S3, and a fast layer of Redshift is used to brew it, and InetSoft extracts it for visualization. They are using InetSoft, and this by no means is a template. Organizations need to evaluate their needs and the sites of the users and the questions that are going to be asked before committing to these frameworks.

Abhishek: Great, that is, and I highly encourage people to check out that customer story they presented at the InetSoft user conference. It's super interesting, and for people that listen to this recording later that webinar is on February 25th, and then it will be available on demand after that. Holly any other, any other thoughts on this one, or how you actually see customers implementing this in real life scenarios?

Read the top 10 reasons for selecting InetSoft as your BI partner.

Holly: Well there are a couple of questions that have come in. We will speak to this a little bit, and one of the questions is, do we see relational databases or traditional databases, all of the data sources become one with the one thing in the future? That's the sort of question, and why that might not become one thing is definitely the case that is part of the admin. Some of the end-users should see it as one thing, and what they might see is different types of databases in the whole solution, in the whole platform, different types of storage mechanisms, as well. So that's when you say one size fits all doesn't work, there is no one particular storage technology or computer technology or processing interface that's going to solve every BI business problem or analytics problem. So we'll see them merging and the technology edges blurring and a lot with no overlap in combination eventually absolutely, but mostly there is a pragmatic approach to that.