How to Manage Data Distribution Across the Enterprise

Below is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "Managing Data Complexity." The presenter is Mark Flaherty, CMO at InetSoft

Mark Flaherty (MF):Now we have all sorts of different large scale inter-organizational requirements whether it's for customer relationship management, enterprise resource planning, whether it's for supply chain management, whether it's data warehousing, business intelligence, integrated analytics, complex event processing, a significant amount of repurposing and data reuse is now required and the data is being distributed across the organization.

We don’t really have a good handle on our data investments, and that gives us a little bit of pause. So the question I have heard a lot of people talking about is that the context of creating data, or the managing data is a strategic corporate asset. And what does that really mean to manage data as an asset?

We have to look at it from what we do and what we use assets for. We essentially rely on information to add value to the organization. And if that’s true, then what techniques and what processes are in place to manage and control and communicate the increased value that we can get out of information?

Different Data for Different Groups

Given the fact that we have got many different business units in organization, many different departments, lots of different data platforms, lots of different databases, lots of different kinds of systems, many different applications, different kinds of tools, different channels of collecting and managing information, different levels of management, different types of people with different levels of skills to understand information in different types of ways, with different levels of experience and expertise in using and exploiting information, we have a high degree of complexity in managing these assets.

OK, we have got a significant amount of complexity, and it's worth looking at these different levels of complexity. In fact, we can take a walk from the top of the organization on down. If we start out with the organization itself and say, okay well, how our organization is organized? Well, it's interesting because we don’t begin creating a big mess or as my friends refer to this as a “pile of hair.”

We don’t start out by engineering an organization that’s got 30 different types of systems with 500 different types of applications, each of which is relying on one out of seven types of database management systems with different types of reporting in it and analysis platforms. And in fact if you look at this picture, this is intended to kind of give an overview, where we have got different kinds of computers, we have got different kinds of storage systems, they are all interconnected.

And then even outside of that network, we have got, what you might call uncontrolled use of information and that typically falls into desktop applications like Excel or Access, in which people download data from our portal or take it out of the warehouse or some data mart and then put it into their own spreadsheet, and start manipulating the data on their own. So we have got all sorts of replication of data, reuse, pulling data from different places, copying it, and various levels of control.

Read what InetSoft customers and partners have said about their selection of Style Scope for their solution for dashboard reporting.

Creating data marts, while essential for facilitating analytical insights, often presents a myriad of challenges for organizations and data professionals alike. One prevalent issue is the complexity involved in designing and building data marts that accurately reflect the business's needs and objectives. Without a clear understanding of user requirements and data sources, organizations risk developing data marts that fail to deliver actionable insights or provide value to stakeholders.

The process of defining data models, extracting, transforming, and loading data into data marts can be time-consuming and resource-intensive, leading to delays in project delivery and increased costs. Another common problem faced by those creating data marts is ensuring data quality and consistency across disparate sources. Organizations often grapple with data silos, where information resides in separate systems or departments, each with its own structure and format. Integrating data from these disparate sources into a cohesive data mart can be challenging, especially when dealing with inconsistencies, duplications, and inaccuracies. Poor data quality not only undermines the reliability of insights derived from data marts but also erodes trust in analytical outputs, hindering effective decision-making and business performance.

Maintaining and updating data marts over time poses significant challenges for organizations, particularly as business requirements evolve, and new data sources emerge. Without robust governance processes and scalable infrastructure in place, data professionals may struggle to keep data marts aligned with changing business needs and technological advancements. Additionally, ensuring data security and compliance with regulatory requirements adds another layer of complexity to the management of data marts. Failure to address these challenges effectively can result in outdated or obsolete data marts, limiting their usefulness and diminishing their impact on driving strategic initiatives and business growth.