Relational Databases Are Not Going Away

This is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of "10 Biggest Big Data Trends."

Abhishek: Yeah, that was a good question. We do not see things converging, and we think that the relational database is not going away. It feels like that was a little bit of hype years ago that that may happen, but in reality we are probably seeing the opposite. To the end-user or the data analyst or the knowledge worker who is working with data and seeing data, they want to see one interface to it.

Not to sound overly self-serving of something InetSoft tries very hard to be, but we can be that one interface. All the very different formats of data doesn't matter in reality in the back end of it. We are not seeing customers retiring the relational databases and moving to favorite non-relational store.

What they're doing, they are rightsizing potentially, and so there's less storing in proprietary MPP data warehouse platforms, as an example and storing more in the data lake and so they have a multi-tiered strategy at that point or they are using technologies for data virtualization, data federation to unify querying across a number different data sources.

#1 Ranking: Read how InetSoft was rated #1 for user adoption in G2's user survey-based index Read More

There's a lot of options out here which are interesting. It's really cool to see how customers are putting this together and implementing, but I don't think we are seeing one type of database or store taking over everything else. That's a really, really interesting, kind of multi-tiered architecture.

Larry: Here is a related question from Sally, and she's asking what is the key to focus on when translating data lakes into business intelligence. Should the end-user be an enabled data specialist? Well the answer to that is well the end-user could be a non-technical business user, it could also be the PHD data scientist trying to query billions of records.

Again to the point we've been making, the three-tiered architecture actually allows you to satisfy both end-user communities. If you want to query billions of records to try to understand clusters of customer behavior you should absolutely do that, but if you want to use it more from a KPI reporting standpoint, well we should enable that as well. That's where it makes sense to have data both in a relational data warehouse. It's an "and" not an "or."

Abhishek: That's right, great okay, our next trend, variety not volume or velocity drives big data investments, and this kind of gets down to those three Vs by which we define Big Data.

Holly, do you want to kick us off on some thoughts on this? Holly: Yes, so as data government defines Big Data or differentiates Big Data from other data originally is by the three Vs or the scale of the issue. It's the volume, the variety, and the velocity and we have tended to focus on the volume to call it Big Data. We have tended to focus on the volume, but the council here is to not just focus on the volume, and what we are seeing is it's the variety.

And the variety of the data is becoming just as important. One of the current big drivers is to make a big investment in these data platforms. For the new data platform investments, though I want to minimize velocity, certainly variety is definitely a key driver. We touched on it when mentioning the internet of things.

Read what InetSoft customers and partners have said about their selection of Style Scope for their solution for dashboard reporting.

All the different types of data that is coming in from new data sources all the time, and even and we are going to touch on this a little bit later on it speaks to some of the questions and some of the other trends, but a lot of people think of the data just coming in to the platform. But you should know the primary sources are your file system, web logs, internet of things, and increasingly the work platform itself is creating data.

Dashboards for File System and Web Logs

All the different types of data that is coming in from new data sources all the time, and even and we are going to touch on this a little bit later on it speaks to some of the questions and some of the other trends, but a lot of people think of the data just coming in to the platform. But you should know the primary sources are your file system, web logs, internet of things, and increasingly the work platform itself is creating data.

Dashboards for file system logs and web logs are powerful tools that enable organizations to visualize and analyze large volumes of log data efficiently. These dashboards provide a centralized view of system activities, user interactions, performance metrics, and security events, offering valuable insights for IT teams, developers, and business stakeholders. By translating raw log data into actionable information, these dashboards enhance monitoring, troubleshooting, and decision-making.

File system log dashboards focus on the activities and events occurring within a file system. They offer visual representations of data such as file access patterns, modifications, deletions, and permission changes. Key metrics displayed include file creation and deletion rates, user activity logs, and alerts for unauthorized access attempts.

For example, administrators can quickly identify anomalies like a spike in file deletions or failed access attempts, which might indicate a potential ransomware attack. These dashboards often feature real-time tracking and historical analysis, making them essential for maintaining data integrity and security.

Key Features of File System Log Dashboards

  1. User Activity Monitoring: Dashboards show who accessed specific files and when, highlighting patterns that may require further investigation.
  2. Audit Trails: Detailed records of file changes help in compliance reporting and forensic analysis.
  3. Anomaly Detection: Integration with AI or machine learning allows for the identification of unusual patterns, such as bulk file deletions.
  4. Storage Usage Analysis: Visualizations show storage consumption trends, helping organizations plan for capacity upgrades or cleanups.

These features help IT teams ensure that file systems operate smoothly while minimizing risks related to data breaches or unauthorized activities.

Dashboards for Web Logs: An Overview

Web log dashboards focus on the activities and events occurring on websites or web applications. They aggregate data from server logs, application logs, and user interaction logs, presenting insights into website performance, user behavior, and potential security threats. Key metrics include page views, session durations, error rates, and traffic sources.

For example, an e-commerce business can use a web log dashboard to track user navigation paths, identify pages with high bounce rates, and analyze conversion funnels. This information helps optimize the user experience and improve website performance.

Key Features of Web Log Dashboards

  1. Traffic Analysis: Visualizations display metrics such as total visits, unique visitors, and traffic sources (e.g., organic search, social media).
  2. Error Tracking: Dashboards flag HTTP error codes (e.g., 404, 500) and correlate them with affected pages, enabling quick resolution.
  3. User Behavior Insights: Heatmaps, session recordings, and clickstream analysis provide deep insights into user interactions.
  4. Security Alerts: Dashboards monitor for suspicious activities such as repeated failed login attempts or unusual traffic spikes, which may indicate DDoS attacks.

These capabilities empower developers and marketing teams to optimize both the technical and user experience aspects of their websites.

Commonalities Between File System and Web Log Dashboards

Both types of dashboards share some core functionalities, such as real-time monitoring, historical trend analysis, and alerting systems. They also integrate with broader analytics or security platforms to provide a comprehensive view of organizational activities. Additionally, both dashboards are highly customizable, allowing users to tailor the visualizations and alerts to their specific needs.

For example, a unified dashboard could combine insights from file system logs and web logs to detect patterns that span across systems, such as coordinated data exfiltration attempts involving file downloads and anomalous web activity.

Use Cases for File System Log Dashboards

File system log dashboards are critical in industries where data security and compliance are paramount. For instance:

  • Healthcare: Dashboards monitor file access to ensure compliance with HIPAA regulations.
  • Finance: They track document changes to safeguard sensitive financial data.
  • Technology: Companies use these dashboards to audit development activities and manage intellectual property.

By providing clear, actionable insights, these dashboards help organizations maintain compliance and reduce operational risks.

Use Cases for Web Log Dashboards

Web log dashboards are indispensable for optimizing web performance and user engagement. For example:

  • E-commerce: Dashboards analyze user paths to improve product recommendations and reduce cart abandonment.
  • Media and Publishing: They track content popularity to tailor editorial strategies.
  • SaaS Platforms: Dashboards monitor server uptime and user engagement to ensure service reliability and customer satisfaction.

These insights enable businesses to deliver better user experiences while maintaining robust web application performance.

Previous: Data Lake as a Necessary But Not Sufficient Foundation Next: Machine Learning and Predictive Analytics Create Data to Analyze