The worldwide economy was shaken in 2007 when the United States stock market had its largest drop since the Great Depression. While there are many factors that led to this event, one critical dynamic was the inadequacy of the data architectures supporting banks and their risk management systems. Let’s examine how these processes failed, what regulations were put in place as a result, and what this means for business intelligence teams today.
Inaccurate Data Management Leads to Financial Collapse
One reason the financial collapse took the world by surprise was the lack of data transparency. In 2007 many high-risk sectors of the financial industry such as hedge funds, depended on complex data. This data was unregulated by the Security Exchange Committee. Large sums of money were moved around based on inaccessible or inscrutable data.
The governor of the Federal Reserve, Randall Kirszner, observed collateralized debt obligations were so complex it was impossible to determine their real value. Investors then paid whatever was asked without any information to justify the cost. This allowed the price to be determined by the reputation of the seller alone.
New Regulations Lead to New Challenges
The opaque and complex nature of these systems led to a 2007 market panic and that led to the Great Recession. Afterwards, the recognition of banks’ inability to manage these risks led to the passage of the Basel Committee on Banking Supervision’s landmark piece of legislation, BCBS 239. These regulations required quarterly risk-evaluation reports.
One of the most important components of that legislation was BCBS 239 Principle 2. It required banks to maintain data architecture supporting risk aggregation at all times. In order to comply, banks needed to implement tools making risk reports more accessible using:
- Automated data discovery functions
- Documented flow of data through all systems or data lineage capability
- Visual representation of data movement
In order to achieve the above requirements, organizations often turn to automated metadata management platforms such as Octopai, to provide a single source of truth that stakeholders and government regulators can trust for accurate risk-reporting.
Automated Data Lineage Tools Provide Regulatory Solutions
BCBS-239 compliance looks great on paper. In reality, banks have hundreds of metrics to regulate, most coming from distinct systems. To be in complete compliance, the entire spectrum of data coming from these metrics needed to be accounted for. According to a McKinsey report, banks quickly found their primary challenge was an inefficient data architecture.
To solve this problem, banks implemented data lineage tools in conjunction with the other metadata analysis capabilities, such as data discovery and a business glossary. These tools extract, sort, and integrate thousands of metrics.
This task would have once required an investment of hundreds of hours from an entire team of BI professionals, and could now be completed in a fraction of the time. The problems banks faced before the 2007 crisis, including lack of transparency and complex data, were eliminated by data lineage implementation.
Data Lineage Provides Further Benefits for Enterprise Organizations
Once regulatory standards are met, automated data lineage delivers many benefits for your company. Along with automated data discovery and automated business glossary, automated data lineage helps organizations meet the European Union’s GDPR compliance automation regulations, and is also critical for ensuring CCPA compliance as well. Data lineage also spurs value creation within the organization, creating value in otherwise inert data. This allows for:
- Business glossary implementation
- Easier data mapping
- Faster discrepancy identification
- Higher accuracy
- Lower project costs
- More efficient analysis and reporting
Automated data lineage is a key tool for compliance in the financial world. Get ahead of the curve. Connect this function to the many other features of a fully equipped data management platform such as Octopai to position your company for success.
