Data and Analytics Transformation-Getting the Fundamentals Right
There have been significant advancements in data and analytics during the past decade and the evolution continues at a dramatic pace. The progress experienced to date in artificial intelligence and machine learning has been astonishing, underpinned by an equally exponential growth in data volumes and other technology advancements. In this constant and often competing arms race towards the ‘next frontier,’ one fundamental factor for success is often overlooked: data governance and quality management. This is echoed in the parallel and seismic shifts in customer, government, and regulator expectations and requirements of security and privacy world-wide. It is critical therefore, that organisations balance their focus to ensure that the fundamentals are addressed right relatively quickly.
Why It Matters and Why Now?
1) Data Governance: One of the key drivers of the urgency around data governance is the rapidly shifting regulatory and compliance landscape across the globe; this is further spurred on by high-profile personal data access, security and privacy issues such as Cambridge Analytica, and others to name a few.
Regulators world-wide are responding with stringent legislation–the European Union’s general data protection regulation (GDPR) legislation which came into effect 25th May 2018 is a prime example of change which had global ramifications. The European Union was not alone–California’s Consumer Privacy Act (CCPA), for example, became effective 1st January 2020 and places significant obligations on organisations to ensure ownership and control sits with consumers. Whilst this is limited to California now, it is widely expected that other states will follow.
Locally, we can see the same focus by industry-specific regulators such as the Australian Prudential Regulatory Authority (APRA) in Australia also signalling its interest in this area, with further guidelines being issued in relation to financial services.
Other notable developments include the rapid progress in open data and open banking initiatives across the world. The Consumer Data Right legislation in Australia (Banks and Utilities), Open Banking in Unkind PSD2 in EU are dramatically shifting the landscape and the fundamentals of data ownership.
Whilst this change is being driven by regulatory regimes, it is important to note that they are reflective of the shift in customer expectations with respect to how organisations treat their data.
To effectively respond to this rapidly shifting landscape, having a well-co-ordinated strategy across the businesses within and organisation is crucial. Data governance frameworks and protocols play a key role in ensuring long-term sustainability.
2) Data Quality: Data Quality has been a topic of discussion since the inception of data warehouses in the 1990s, but until recently, had never gained the right visibility or focus to shift the dial. As volumes increase and technology is further advanced, data quality has been recognised for its critical role in ensuring accurate and timely inferences are made from data for decisions and for data-driven customer engagement. Regulators are increasingly putting a high importance on data quality. The other key important factor is the role of quality in supporting initiatives in AI–if the quality is not right outcomes can be diluted.
Where to from here?
Against this backdrop, we are seeing lot of examples where organisations are being forced to move quickly in governance and quality management–this is usually driven by regulatory and compliance pressures. This is not ideal nor are these investments perceived as ‘growth related’. It is important for data and analytics leaders to balance and communicate the context whilst approaching actions in a more strategic and pre-emptive manner.
It would be prudent to rapidly assess your organisation against industry, local and global standards such as data management capability assessment model (DCAM) to understand your current position. Then the next step is making a start: this is important since governance and quality programs can take a prolonged period to initiate / implement depending on the size and complexity of the organisation and the level of business awareness and sponsorship. Active business participation in every stage is critical for the success of governance and quality initiatives.
Making technology choices and implementations can often be tricky. In many cases where issues such as data lineage must be addressed, rework is often required and migrations to new tools can be expensive, time consuming and difficult to obtain stakeholder buy-in. Pragmatically selecting technology must often be balanced between the future data strategy (e.g. Cloud, Data fabric) and existing legacy systems. Some of the usual challenges are it is often difficult to retrofit capabilities such as quality. The key to success is striking the balance between the focus on getting Governance and Quality right whilst keeping up with the progress required to take the Data Strategy forward. Whichever way it is addressed, the key is to ensure that both Governance and Quality are embedded in the Growth strategies in Data.