There are no quick fixes when it comes to dirty data

For a lot of people managing dirty data is an everyday occurrence, and they spend each day mapping, adjusting and compensating for dirty underlying data. In this article, I highlight what data management is, and what it is not, and why data management is a better alternative than fixing your data issues with everyday data cleansing and patchwork solutions. 

When a business invites us to help them with data management, they often start the first meeting by asking us a hopeful question such as “Can’t we get a gadget or a hub or something that can help us manage our data?”, or “We want a system that can help us manage our data – what system should we choose?”.

Often, the first thing I have to say is that unfortunately there are no quick fixes as such to solve their problems. But there is definitely a way out of the mess, and some technology may be an option. However, this is not necessarily where we begin when we want data to remain good over time.  

So, where do we start?

Often, it is the IT department who contacts us; maybe because they are made the scapegoat by the business when the “system” does not work, and IT repeatedly ascertains that there is nothing wrong with the system but that the data is bad. And this is our point of departure, because most data is “owned” by the business – not IT. And the good reason for this is that it is the business that knows what good data looks like and why data has to look a certain way in order for their processes to work.

We know from experience that data management is a hard slog. So, to succeed we have to create results - fast. Consequently, I quickly turn the conversation and ask where the business is hurting. If we are lucky, IT actually has a pretty good idea. If not, then this is clearly where we have to start. If you are not planning to have a new ERP system or are not facing a merger of some kind, then do not make the scope too big. Start with a small scope but with something that will make a noticeable difference for the business. Once the business experiences that their working day becomes less difficult, it is easier to attract others and increase scope. Also, it allows for getting actual experience in what works for the business. 

Another central prerequisite for success is that you do not start your data management project as a grass roots movement. Data cannot become good if only a small part of the organisation is trying to patch something together that has to apply globally. The data management initiative has to be embedded high enough in the organisation that a person with a global mandate takes the lead and says: “Friends, if we are to succeed, we have to agree on what our data should look like across the business”. Because a central challenge when it comes to dirty data is that you risk ending up with a mismatch of data across your systems and departments, and no one knows what is “the truth”. Silos are fatal for good data and are a significant reason for the most prevalent data issues.

To start a data management project is the beginning of a new practice. A new and lasting practice that has to be practised and improved as long as the business exists. It must be seen as just as significant as the tasks that are solved in some of the more well-established functions such as HR and Finance. If you have bought into the idea that data is important, that you want to be data-driven and want to go for automation or want to make use of some of the hottest technology trends, then you should also prioritise ensuring a sufficiently good data quality. As you know, the amount of data increases constantly, existing data has to be changed and there are new requirements, etc. Consequently, it is obvious that the data management practice is also a permanent set-up – not just a project. 

When you are about to start your data management practice, you can read a lot of information on what it takes to succeed. There are plenty of books on data management. However, my experience is that some businesses actually have made an attempt at data management and have tried to implement some kind of data governance or have bought a ”gadget”. But something went wrong; the initiative has not provided the required results, and over time “metal fatigue” has set in, until you have almost given up and the dirty data wins. And that is a shame! Because if you approach this correctly, by for example getting help from or employing someone who has done it before, then you gain access to some short cuts to getting off to a good start.

Gartner also recommends getting help from a third party in order to “fast-track time to value”.¹ However, remember that data management cannot be a ”consultancy project”. The key to success and sustainable data management lies in a large transfer of knowledge and active involvement of employees. Once you have seen how you can work systematically with data management, the business can take over the practice from there. I promise that this is not rocket science, but if you have never done it before, then maybe you end up on a winding road, which in the worst-case scenario can lead to the end of something that it is really important to succeed with.

In my next blog post I will write about master data, what it is, and why it is important to manage your master data first, when you no longer want to be a victim to dirty data. If you have other topics that you would like me to write about or have questions, then please just drop me a line.   

¹https://s3.amazonaws.com/bizzabo.file.upload/VVo19sNqRyKqLd2EqWNL_Oct5TCraig.pdf