All modern organizations of any size live and die by their data and the processes surrounding that data. Processes can include how data is:
– Acquired
– Organized
– Stored
– Used to gain insights/make tactical/strategic decisions
…and more.
Many companies now struggle with their data because traditional approaches to managing it are increasingly inadequate. These processes often rely on humans performing manual discovery and documentation tasks for:
– Metadata management
– ETL specifications
– Report definitions
– Reporting errors
– Changes to calculations
…and many other aspects of the data environment.
The Data Just Keeps on Coming…and Changing
At the same time, more data sources are added, changes are being made in the data landscape, and end users (who now have access to powerful, self-service analytics and reporting tools) are demanding more from the data.
It would take an army of data specialists to get a handle on all this, but then you would have problems with consistency. Not everyone in your “army” is going to characterize the data quite the same way. You may end up back where you started.
In this situation, control of an organization’s data becomes nearly impossible. Those organizations with the maturity to recognize this, understand something needs to be done. That something has to do with metadata management.
First, the Bad News
Here’s the bad news: “metadata management” means different things to different organizations. (In fact, it can mean different things within an organization.)
For example:
– Some regard it as a tool for enabling data lineage, tracking reporting errors, and other day-to-day BI (business intelligence) operations.
– Some see it as an input in implementing a migration project from legacy to modern BI reporting tools.
– Some see it as the first step in compliance with new data privacy regulations.
– Some see metadata management as a means to the end of data governance.
– Some think of it as a tool for developing data catalogs.
Now, the Good News
The good news is, everyone’s right. Metadata management is the key to understanding the data landscape and environment. Many companies approach metadata management implementations without a clear idea of what it means or what exactly why they need it. But the simple truth is, there are many ways to leverage good metadata.
How do you execute a metadata management initiative without taking months, spending millions, and causing more problems than you solve?
Metadata Management Automation.
Automated tools are the only practical solution to the problem of metadata discovery and management. You can’t realistically ask a business to stop generating data, stop acquiring new data sources, and stop creating and modifying reports and dashboards while the BI team takes months to go through every data asset and document the metadata.
With automated tools, the metadata discovery task can be reduced to days or even hours. It can continuously monitor the data environment for changes. The BI team and other stakeholders can focus on extracting value from the data instead of manual grunt work.
Regardless of your motivation and needs, the first step in taking control of your data is metadata management. And the first step in metadata management is automation.
