While an important part of information processing in many industries, implementing data lineage is a time consuming and expensive process that will change the way your company operates. Because it can have such a sweeping effect on a business, to implement it successfully you need to start at the top and remain committed to seeing it through.
Identify the most important business reasons that your company must begin utilizing data lineage. These could include changes stemming from recent legislation, a data quality concerns, business expansion or regulatory audit preparation.
Use these reasons to get management on board and define the scope for data lineage. Correspond the reasons the business needs these processes to data sets that will be affected by implementation.
Decide how to process these data sets. Use either descriptive or automated data lineage. Descriptive relies on manual entrance of data, and even with the assistance of data governance applications is still a very labor intensive and time-consuming process. Automated processing of the metadata can save a lot of time but is more expensive and may be incompatible with some legacy systems.
Once you have decided on what method will be best for your business you can choose a tool to base the process around. Start with a small pilot program with a tight scope to test the lineage solution. Work from there until the business has a stable process of data lineage adherence in place.
Is your organization Octopied?
With effortless onboarding and no implementation costs, Octopai’s data intelligence platform gives you unprecedented visibility and trust into the most complex data environments.
