Like a Needle in a Haystack – Discover Metadata in a Few Clicks [Part 1]

It was February 2014. I met Lisa Parker, a finance manager at Airwave, a commercial telecom outfit specializing in discount services for large consumer markets. Lisa approached me as I was coming off the speaker stage at a metadata management conference in NYC, where I discussed the importance of integrated metadata management for enterprise business intelligence (BI) teams.

I knew Lisa for quite some time. Always talkative, energetic and direct.  She didn’t waste a moment before snapping: “That was a nice talk, but I don’t believe it – you consultants always inflate reality to scare people, so they hire you”. I knew Lisa long enough to know that arguing with her wouldn’t do us any good, so I kept quiet, and quickly changed the subject.

A few months later Lisa was promoted to Chief Financial Officer (CFO) at Airwave. At that time, due to limited competition in the discounted mobile services, the regulator, The Ministry of Communication (MOC) asked Airwave to provide monthly reports which demonstrated the pricing in various product categories to ensure that customers paid a fair market price for Airwave’s services.

In her new role, Lisa was also responsible for designing reports according to specifications received from MOC. The reports were to be uploaded during the first week of every month starting Q2, reporting on the previous month’s activities. The MOC required three reports:

1. Average pricing for each product category

2. Total revenue for all product categories

3. Change in average pricing in the reported month compared to the average pricing in the previous quarter

Lisa worked with the BI team to communicate the requirements and to ensure that reports would be produced according to specs.  After several months of a tight development process, Lisa received a sample of the acceptance testing and approved the reports for production delivery. Lisa was also responsible for reports QA and tasked one of her deputies with reviewing the reports before sending them via the secured VPN to MOC.

When the CEO calls you in to give you that dreaded task

Everyone was happy. But not for long. Three months into the process, the CEO, Larry Ford, received a memo from MOC demanding that all reports be amended to produce a different calculating formula, which would reflect the gross margin gained for each of the product categories. Naturally, Larry gave the job to his CFO, Lisa Parker. Lisa was furious. She didn’t have time for useless tasks, but she didn’t have much of a choice.

Lisa reluctantly called Susan Desoto, the CIO, and asked her to include the MOC new requirement in the workload. Susan responded with a flat NO.  After pleading with Larry, Susan agreed to review the request and come back with a resource estimate. Two days later, Lisa received a phone call. It was Susan. “It’s going to take us four months and three technicians, which I don’t have. And if you want to know why, it’s because we have to manually search for the data in every database table that represents a client transaction. Do you know how many transactions there are every day? I think you do, after all, you’re the CFO. So either you find the resources or this thing is going to the backlog list”.

Never enough time, never enough manpower

Lisa was devastated. She never missed a deadline and on this she didn’t even know where to begin. To make things worse, the BI team refused to let outsiders dig into their systems, and they practically stopped the project, waiting for a decision by the IT executive committee. The committee did not approve the hiring of an outside consulting firm and asked Lisa to request an extension from MOC. The regulator agreed to extend the date by two months, but that was not enough for Lisa and the BI team to complete the task.

Lisa decided to search for solutions outside the company, and she came across Amara, a BI consulting company specializing in metadata management. Lisa quickly scheduled a phone conference with one of Amara’s partners. Hoping for a miracle, Lisa attended the conference with Airwave’s BI manager to make sure she understood the ins and outs of tracking enterprise metadata.  The conference was quite productive but did not yield a practical solution for the problem at hand.

Lisa left her office as helpless as before, and although she was not able to resolve this particular reporting issue, she managed to raise the visibility of data administration for the BI team.

On one cold morning, when I was on my way to visit a client in Europe, I got an urgent phone call. It was Lisa.

“Hey Avi, you’ve got to help me, I don’t know where else to go.”

“What do you mean?” I said. “We consultants always inflate reality to scare people. Right?”

“Oh don’t do that, I am really desperate… remember I told you about the MOC new requirements?”

“How could I forget? — OK, Lisa. I won’t hold it against you this time” I said.

After a noticeable pause, I asked Lisa: “Have you heard of Octopai?”

To be continued

***

Is your organization Octopied?

With effortless onboarding and no implementation costs, Octopai’s data intelligence platform gives you unprecedented visibility and trust into the most complex data environments.

Announcement ! We are happy to share that Octopai has been acquired by Cloudera