InetSoft Webinar: Successful Data Mining Projects

This is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "Best Practices in Data Mining." The speaker is Mark Flaherty, CMO at InetSoft.

Moderator: Yes, that’s a really good point in terms of preparing and massaging the data for a successful data mining project. That time factor could be, for example, if we don’t see any purchases from Joe Smith over a period of 18 months, we probably lost that customer, right?

Flaherty: Exactly, in that time window, and I think that’s where the come difference between the data scientist and the decision maker is that time window is a marketing and sales business question. It’s not a science question, right? For example, the example you gave, you might be able to find out if somebody is going to leave over 18 month window.

There are a lot of drivers that come directly from the business and have to be applied to the predictive analytics problem and the data transformation problem which allow you to answer these questions. But they are not directly from, they are not necessarily things that fall out of the data.

They’re business drivers. They had to be taken into account when you perform predictive analytics. And there is kind of a balance where you have to make sure that the business drivers don’t interfere with the ability to do any predictive analytics.

Yes, there are lots of things that come in there to play or out of control which you have to realize as well. There are a lot of industries where there are certain legislative restrictions on what information can be used or what information can be used in conjunction with each other. And the predictive algorithms obviously don’t know about whatever acts of congress are controlling how to use data.

But if they find out rules that conjoin two pieces of information, and these pieces of information can't be conjoined, but might have particular power when to use it independently, then you actually have to be able to look at that, interpret it and figure out what to do with those results after the fact, which you might not be able to do it in an automated kind of way.

Moderator: Right. So one of the things that you are implicitly communicating here is the importance of the best practice of taking an iterative approach with things, right? You may be set. You bring your data forward. Let's say you are analyzing it to notice some high level patterns, but then you take an iterative approach to building models to better understand certain aspects like customer attrition or your next best offer or fraud detection or whatever the use case might be.

Flaherty: Exactly, and I think the best practices in making decisions is to include the people that would know all of these special cases, such as how things really apply in the business world or legislative world, or the special circumstances around the analysis that you are doing and how those apply them. And I think it’s really important to bring those analytic tools and those concepts into the hands of people who can actually make the decisions and actually really understand the context around the information that’s coming out.

Read what InetSoft customers and partners have said about their selection of Style Scope for their solution for dashboard reporting.

The information comes into the modeling process as kind of a very abstract form, and the information comes out in the very abstract form, and the people who can actually quickly react to it, aren’t going to be able to interpret those results. Then the exercise becomes a scientific exercise rather than a business driving it.

Moderator: So a best practices for data mining is if we are going to do data mining of any kind, you really need to have your ducks in a row, you need to think about things. It’s not the easiest thing in the world to map it all out. But from your perspective what are two or three best practices that you share with clients when you begin the process of helping them do data mining.

Flaherty: I think one of the areas I would like to focus on is how there has been kind of a revolution in the way that statistical analysts, people with deep knowledge in doing data mining or predictive analytics, tend to work with business people. They work with them to guide their results. I think that that’s been an established connection that people have talked about for 20 odd years now.

But I think in recent years, there has really been a vast change in how that’s occurred, and I think the way that’s happened is as predictive analytics has become much more popular, a lot more business analyst types are interested in trying to apply the technology. And what they have done is they have actually gone out and tried it. And this is different from the past where they offered information about the business to professional analyst, but they didn’t really understand how the process worked.