Sit back and imagine how you would feel when…
Your CFO says to you, “You were right. This data catalog is helping boost our bottom line.”
Your data team spends more time doing high-level analysis than it does searching for relevant datasets.
Your most technologically-challenged business user searches the data catalog at least once a week – without coming to you for help!
YEE-HAW! Data catalog SUCCESS!
Okay, come down to earth. Visualization may work wonders for athlete performance, but if you want to make data catalog adoption into a rousing success, you’re going to sweat a little bit.
But don’t worry. You don’t have to go it alone; we’re here to share our recipe for data catalog management that will have you basking in the glory.
Pick a data catalog that matches your enterprise’s needs
You did notice how we bolded the word “your” in the above sub-header, right?
Because that’s important.
Different organizations will have different data management needs and priorities. The capabilities of a data catalog service under consideration should be prioritized in accordance.
Potential priorities and corresponding catalog functionalities include:
- Data intelligence, making support for every system and tool in your BI stack essential
- Self-service BI, making user-friendly search, discovery, and UI critical
- Regulatory compliance, making data lineage capabilities important
- Preserving tribal knowledge, making collaboration and communication features a need
- Data governance, making data status, classification, and access functionalities a priority
Work out your organization’s list of needs, sort them in order of priority, and use that as the basis for evaluating data catalog candidates.
Get an automated, self-updating data catalog
You know when you look up contact information in some online directory, and when you try the resulting number or email you get a “This number is not in service” or “A message that you sent could not be delivered. This is a permanent error”?
How many of those will it take before you dismiss that directory as a reliable source of information?
Your data catalog is no different.
If a user sees that the catalog lists the data steward of a particular asset as his colleague Ashley who left the company three months ago, that’s a big strike against his trust in the catalog. Reduced trust means reduced use means… why did you invest in a data catalog, again?
So your data catalog needs to be kept up-to-date. How are you going to accomplish that? Manually updating the catalog every time a data asset changes is a Sisyphean task, and you’d require an army of Sisyphuses to even attempt it.
An automated, self-updating data catalog, on the other hand, will take care of keeping itself current. It will automatically review your data landscape for the relevant metadata on your thousands upon thousands of data assets. When Ashley leaves the company and Kim takes over this area of her responsibilities, your automated data catalog will quickly figure that out and update the right catalog entries accordingly.
For data catalog adoption success, make sure that the data catalog you choose is automated, with self-creating and self-updating capabilities. Double-check that it can connect and retrieve metadata from all the systems across your data landscape. Your data catalog and metadata management tools need to integrate smoothly. If your enterprise uses Azure Data Factory but the data catalog you’re considering won’t talk to anything Azure, this is not a match made in heaven.
Show users how this will make life easier (and not harder)
“So, Joe, I see you created this great data asset. I’m just going to put you down in the data catalog as the data asset owner so that anyone who has a question can come to you…”
Joe, if he is like most of us, does not have a happy expression on his face right now.
What?! You’re putting me in charge of more things? You’re giving me more responsibilities? I have more than enough on my plate! I wish I’d never created the dataset… or that you’d never had the bright idea of getting us a data catalog.
The key to successful data catalog adoption is to emphasize how this catalog will make life and work easier.
“So, Joe, I see you created this great data asset that so many people in our organization could gain from. But very few people know about it, and of those who do, almost all of them have the same questions that you have to answer over and over again. I’m going to put you down in the data catalog as the data asset owner so that you can answer any questions once and only once in the notes on the catalog entry. Many more people will be able to leverage the asset you created, but fewer will disturb you unnecessarily.”
See Joe’s smile?
Create department data catalog ambassadors
If Joe is convinced of the data catalog’s value for himself and his fellow co-workers, he’s a perfect candidate for a department ambassador.
Pick one or two staff members from each business department that could benefit from using the data catalog. Give them training. Show them real-world scenarios where their department specifically can gain concrete benefit from data catalog use. Then let ‘em loose – with encouragement (and possibly incentives) to share their new knowledge with their co-workers.
Integrate the data catalog into your existing routines and tasks
Even an automated, self-creating, self-updating data catalog has its limits. For the time being (and probably for a while into the future of AI), humans need to play a role in reviewing, correcting and enhancing the automated catalog entries.
How can you get your already-too-busy staff to take part in this new data catalog project?
What we’ve found effective for Octopai’s clients is to integrate human checks and updates to data catalog entries into pre-existing data management responsibilities, for example:
- Make it a standard part of a new project to document its data assets in the catalog
- Make it a step in closing a change request to update the data catalog for the specific asset that the change request addressed
Just adding these checkboxes to your existing checklists and routines can be a powerful data catalog management tool. Leverage it to pave the way for a well-oiled data catalog management workflow.
The recipe for success
We’re not going to lie: if you want to make your data catalog a rousing business success, you’re going to have to put in work.
Once you put in the work, however, give yourself a little time to visualize success: visualize the statistics on data catalog use increasing every month; visualize your data team raving about how you’ve made them exponentially more productive; visualize C-suite executives singing your praises…
After all, everyone raves about how amazing visualizations are for data.
