Monitoring ETL Processes Via a Dashboard

This is of the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "How to Use Your Business Intelligence As a Competitive Edge" The speaker is Mark Flaherty, CMO at InetSoft.

Another part of these ETL processes is monitoring them via a dashboard. Once you’ve loaded that data, you’re going to be doing data loads, you’re going to be loading new data into that on a recurring basis. Think of your ERP loading into a data warehouse.

You have to monitor that data as well. So what that does is that it allows us to not only do the loading and correcting of data but also monitor over time as new data comes in, our new data quality problems and what’s the impact of those down the line? This lets us do is monitor those data quality issues more regularly and more proactively.

Let’s take a look at monitoring some of these jobs over time. What I have here are four different data warehouses, and we’ve got some uploads going on. We want to see how those are behaving over time. So in my sales warehouse here, I’ve got a chart, and I’ll click on that. Open it up, and what it shows me is a high level overview. We’re uploading a customer master and a product master here, and there’s several rules we’ve defined based on our own business logic.

Three Major Categories of ETL Monitoring

We’ve grouped those into three major categories, whether that will affect the accuracy of the data, the conformity of that data, whether it conforms to our business rules or the integrity of that data. Now if I want to then drill down and understand how that trend is going over time, I can click on more when it loads. On the dashboard here, I can see that I have all my different custom rules on the right hand side.

When I click on any one of these top level rules, it shows me the rules that apply specifically to that, and when I click on an individual rule it shows me how that rule has behaved over time. So if I saw a data problem introduced, I’d see that dropping down. I can see my one warehouse field on the custom rule table, that we’re only conforming to that rule about 55 percent of the time, so that’s a problem that I want to drill into. Now, I won’t go too deep into this, but what we could do is look at this failed data load, or even view the impact. And what that would do is take us into a view where we could see that this field is taken into the data services job and then it’s loaded into some reports. and then this report is actually visible to the business.

We’re coming to the end of our time, and we’ve covered a lot of ground. We started off talking about pixel perfect reporting, canned reporting solutions and specifically around how InetSoft's Style Report Enterprise can help you with your enterprise reporting. Next, we showed how you can quickly and easy empower your end users to get the information that they need to make their own decisions. We showed you on both a traditional desktop or laptop view as well as showing it using an iPad.

View a 2-minute demonstration of InetSoft's easy, agile, and robust BI software.

Using an Exploration Tool

Then we saw the exploration tool in Style Scope, which lets your executives or other users see their key metrics, really pivot around the data quickly and easily to share out results. They could be marketing campaigns, sales performance, or sales trends over time. And you get a solution that’s both available on desktops as well as mobile devices.

And then finally we showed how to use ETL tools to transform our data, cleanse it, as well as monitor over time. This was a great demo integrated in with Google maps using an iPad, and we saw how that works and ties together. Now although we’ve kind of presented these as separate concepts, except for the ETL tools, they’re all part of the same InetSoft BI platform. So, not only do you get enterprise reporting, you can also have ad hoc analysis of exploration views, in dashboards as well as the trusted backend data sources, all tied in and integrated within a single platform.

Now we can handle some Q and A’s and handle any of the questions that people have been typing in. If you’d like to submit your question, all you have to do is hit the ask a question button on your console and just type in if you have an observation, if you’d just like to make a comment feel free, but also questions are great too.

Can you use both a traditional PC as well as tablets and some of the newer devices to utilize these dashboards and reports?

Yes, iPads, other tablet devices and smart phones.

Learn how InetSoft's data intelligence technology is central to delivering efficient business intelligence.

Can some of the ad hoc reports be pushed to other users?

The ad hoc reporting solution that you saw earlier is something that can be useful to cerate your own reports, modify existing reports, save those reports, which is a very common workflow, either into your private folder, something that you just want to see, or if you have rights, share it into a public folder that other people on your team, or other people in your organization can also see. Depending on their level, they could also go in and do some drilling or filtering. So really the ad hoc portion allows you to both modify it and share it out.