InetSoft Webinar: Tools to Solve Data Integration Problems

This is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "The Alphabet Soup of Data Integration Technologies." The speaker is Abhishek Gupta, Product Manager at InetSoft.

Let us talk about some other tools and methods that people use to solve data integration problems. First I would like to go to the list of terms that we hear a lot and give short definitions to help put some of them into context, but before we a three letter acronym game, how familiar are people with these terms already?

I often ask my clients and people I talk to at conferences and seminars and users groups whether they have heard of these terms. The usual response is no, or maybe one or two of the terms. It is real test to keep track of all the terms since they seem to be created regularly.

So let us demystify them. I am going to just go down the list and let us start with an oldie but a goodie: ETL or Extract Transform and Load. Well, actually, before we look at ETL, remember that a quick definition again of data integration is transforming data into information and delivering to the business. Note in my definition doesn’t talk about how it happens, just what it is.

View a 2-minute demonstration of InetSoft's easy, agile, and robust BI software.

People often assume data integration is installation of the data in the data warehouse, but that is not necessarily true. Hopefully when you go through this alphabet soup, that fact will come out. Now to back to extract, transform, and load or ETL, now that is a classic data warehousing tool where you physically move data from many sources to a data warehouse.

It is often used for data migration. It is a tool that many people use. It does compete with manual coding in many enterprises, but using it is a best practice. It is a pragmatic practice to use an ETL tool. In the long run it increases productivity, consistency and reuse and in the short run, it provides code management and documentation for data integration process.

The next one is the definition of EAI, or enterprise application integration. EAI is a traditional application used for application integration. It is really oriented towards data transfer or exchange between applications, and its origins are in messaging, which again implies just like an e-mail, you transfer or exchange in the information, and it provides real time capability.

How about the definition of EII, or Enterprise Information Integration? This is a really a query on steroids, where it’s basically pulling data from many sources across an enterprise. Sometimes, the term federated query is used. It enables data integration without physically moving the data, sort of like a virtual data warehouse, and it also provides real time capability.

ETL, EII, and EAI all provide data integration. They all transform data into information by pulling the data from various sources and delivering it to the business. It is not what they do, but how they do it at different ends, and that is the decision you need to make when you are selecting the tools.

What is data mashup? This is a relatively new technology that is most similar to EII. The data still resides in the source systems, but data from these difference sources can be mashed up in real-time specifically for information presentation such as in dashboards or reports.

View live interactive examples in InetSoft's dashboard and visualization gallery.

How about Service Oriented Architecture or SOA technology, how does that play into the integration and space? Service Oriented Architecture used to be that have all or nothing opportunity with regards to using applications. You couldn’t just pick the parts you wanted. You are forced to use the whole thing, and when you wanted to use multiple applications, that involved a lot of system integration work.

With SOA you access services rather in a monolithic application. An example is connecting to your bank’s Web site. Behind the scenes, SOA is being used to gather information from multiple systems, but you see it as if it is from one system, and hence that is why that supports the data integration.

The term EIM or Enterprise Information Management, what does that mean? EIM is an overall approach to managing information in enterprise. Rather than just a tool, it includes the architecture, processes, and technology to manage and deliver information, and the emphasis here is on reference data and another data management.