Analyzing and Exploring Data: Time to Wait for a Result

This is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "The Evolution from Information Publishing to Insight Discovery and Analytics." The speaker is Mark Flaherty, CMO at InetSoft.

Now, what they’ve discovered is when you get beyond a certain point of response time, of interaction time, where you say or do something, and then there’s an expected response that you have to wait for, it breaks your brain. Basically the model of your brain building up a memory buffer of what you’re thinking about, your next thought, your next input, your next task, gets derailed if you have to wait too long, and you have to start over.

And so, what you would expect to see is response time and how you thought are directly correlated. You do something, and something happens. It is a straight line graph. And now in general this wait time needs to be less than three seconds, and it’s a cross-cultural thing. And what happens when you don’t have that ability to interact in real time, you have a disruption of your work. And when you have that disruption of work, you have this interrupted flow of work which means an interrupted flow of thought, which makes it very difficult do analysis.

When you’re trying to analyze things then you can’t get into what’s called flow. And these different places you can end up with are charted out on this graph which based on the skill level and the challenge level. At least initially, you can end up in these different points where the skill and the challenge are matched. That’s the first precursor to be able to get into a state of flow which you can think of as that mental state where you’re in the zone, where you’re in an optimal state of what’s called intrinsic motivation. You’re immersed, and you can work. And the tools or the things that you’re using or working with come naturally exactly when you need them. You lose track of time. Things work.

You can think about people playing videogames being in that zone, sports, music, a lot of different people fall into that. But the only way that you get there is if the challenge is there, and if skill level is up to it, and you’re interested enough, and the tools support it. And we often disable that because the interaction model of a reporting paradigm doesn’t support it. To some extent the dashboard and interactive reporting tools get us there, and the new BI tools get us there. But they quickly break off in somewhat of a different direction because often they’re too rigid. They don’t allow further discovery of information.

View live interactive examples in InetSoft's dashboard and visualization gallery.

So for example, on a dashboard you have six KPI, and you can drill down very much like a master detailed report falling back into that old metaphor of thinking about a problem. And you don’t have the ability to go the direction you need to go which means the tools increase the challenge, and you fall out of flow in one way or another. And that is what happens when you have these scripted interactions.

And when you think about analyzing and exploring data which is the next step after you get your report or after you get a piece of information out of a dashboard, you ask what’s going on here? What’s wrong? And when you look at the interaction time scales in BI, I talk to people all the time who wait days, especially for the more advanced analytics and complex reporting where you’re trying to understand things like market baskets, or consumer behaviors, or other types of correlated problems. It’s this sort of scale of “come back tomorrow”, or “go to lunch”, or “take a break”, or “get some coffee”.

When you’re down below that three second barrier, closer to instantaneous, then you can really begin to immerse yourself in work. And that’s the point where things change, and that’s the point where a lot of newer analytic tools, and newer BI products are coming from. But until you resolve this task performance gap, you can’t really do things like look at data and say, “Well, I wonder about ‘x’”. “I wonder what the impact of this is” which is why a lot of business analysts take the data and dump it into Excel, Access, or SAS or some other tool.

Read why choosing InetSoft's cloud-flexible BI provides advantages over other BI options.

The reason for dumping it into such a tool is that you can stage your information. You can say, “Give me this, this, that, and that” which comes from four different reports to support the information that was pulled out of the dashboard signaling a problem. And then you’re buried in Excel. You can interact directly on top of the data, not through a design/execute/view model and not through this publishing model. And so, you can get to analyze what works at the same speed and performance as you’re thinking because you’re operating directly on that data.