Most businesses have one profit center—sales—and many cost centers.
Of all the cost centers, the BI team is in a unique position. Their activities can have a direct impact on the ability of the profit center to increase revenue by providing the right information at the right time to management to make the right decisions. The trouble is, as cost centers go, BI is among the costliest in the business.
Because of the way BI professionals traditionally do their work, they may actually cost the business more than the revenues they increase.
There are two primary reasons for this: Manual data discovery and manual data lineage.
Money Loser #1: Manual Data Discovery
Metadata is at the heart of every report, dashboard, data warehouse, visualization, and anything else the BI team produces. Without an understanding of the organization’s metadata, the BI team can’t match the data from multiple sources to produce a single view of the business.
This means that much of what the BI team does involves searching for and cataloging metadata. Traditionally, this is a manual process, and it’s more difficult accessing some sources of data than others.
Consider these challenges:
-The same type of data—names, birthdates, phone numbers, etc.—is stored and labeled differently in different systems. On one system, a table column with birthdates may have an obvious name, “Birthdate.” In another system, it might be “DOB.” BI professionals often have to do some laborious detective work to understand exactly what is being stored in each column and field.
-The same type of data may be formatted differently in different systems. “Birthdate” in one system might be stored as a UNIX date (the number of seconds since January 1, 1970), “MMDDYYYY” in another, and “YYYYMMDD” in another. These formats have to be parsed and standardized in order to bring them together in a single view.
Because it’s a manual process, it becomes tedious, error-prone, and time-consuming for the BI team to perform useful functions, such as:
–Root cause analysis—what caused these reporting errors?
–ETL impact analysis—If we change the data type or format of a column in this database table, how does it affect downstream data consumers?
-Understanding ETL operations column-to-column—What calculations and transformations were performed on a given column to produce the corresponding column(s) in a new table?
Money Loser #2: Manual Data Lineage
When the inevitable metadata errors manifest themselves in reports that don’t add up and dashboards that provide misleading performance metrics, the BI team has to go back and manually find the source of the problem. This is a painstaking process and a huge time suck for everyone.
The Solution: Automation
The typical BI team spends more than half its time manually tracking down metadata and errors. This is why it’s such a money-loser for the business. That time and expertise could be put to much better, productive use.
The key is automating manual processes. Automated metadata discovery tools can track and catalog metadata, then add to the catalog when new data sources are added. Data lineage tools can present visual report representations. These tools enable the BI team to perform tasks 80% faster and with much higher accuracy. It frees them up to really help the business gain insights from the data.
BI will never be a profit center by the textbook definition, but with automation, at least BI teams can make a better contribution to the bottom line.
