Data migration is a critical step in an ERP implementation, but it’s one that often doesn’t get the required time and attention. On its own, data migration is the process of moving your data from one system to another. Sounds simple enough right? Where companies tend to underestimate the effort is on the data cleaning and validation. One of the most common reasons for implementing an ERP system is the desire for better visibility in performance and business health. Having clean, quality data is the key to achieving that. And it takes time to do properly. Your goal is to achieve confidence that your data is accurate, complete and consistent from day one in your new system.
Risks of poor data cleaning
One of the worst things you can do is invest in a new ERP system – with the goal of improving your operations – then dump a bunch of bad data into it. That’s a recipe for a failed implementation because you’ll still be working with data that’s questionable at best. Ultimately, this hinders your ability to use your new system to its full potential. The new and improved processes only work well if you’re feeding in accurate information. So what’s the impact of poor data cleaning on the business?
Cleaning the data after you’ve gone live is much more difficult, taking longer and costing you more.
You may not fully realize the benefits of your investment.
Data issues on the back end have a real impact to customer satisfaction.
You run the risk of making bad business decisions based on incorrect data.
How to properly clean your ERP data
1. Develop a strategy
Treat this phase as a mini project. Create a logical and realistic schedule for cleaning and validating your data. Assign roles, responsibilities and deadlines to each member of your team so everyone’s clear on the plan. You’ll want to involve your executive sponsors to determine what data you’re migrating and what you’re not (keep reading for more on that topic). Be sure to allocate adequate time to this portion of the project. It often takes longer than you expect, so be realistic with the timelines.
2. Assign the right resources
When it comes to reviewing, analyzing and cleaning the data, assign your functional resources – the people working with the data day in and day out. They know the data best and will be best equipped to identify and properly correct errors. You may need some technical experts to help you export data from your existing systems. If you don’t have this expertise in-house, your implementation partner may be able to assist. If your team isn’t particularly skilled in data handling and manipulation, be open to accepting guidance from your partner. They can provide some tips on the most effective way to do this, rather than going through a spreadsheet line by line.
3. Prioritize data migration
Within the context of your ERP implementation project plan, you need to prioritize data migration. Start early and allocate plenty of time. You’ll need to free up your resources from their day-to-day to make time for this, so build that into your plan. If you’re not sure how much time you need, talk with your implementation partner. Based on your size and complexity, they can provide an estimate.
4. Define data standards
The way your data is classified in your existing systems may or may not match up exactly the same way in your new system. It’s important to work with your partner to map the data from the old source to the new and define any translation rules to transform the data. This requires a field-by-field analysis to make sure all required information is captured. To our point about wanting clean data in your new system, you should standardize the data format as much as possible. This will require collaboration across departments to make sure everyone’s in agreement on the standard. If you’re moving from a collection of disparate systems, this step is even more important. You’ll likely have the same information in multiple systems, classified differently in each one. Now is the time to set a standard, document it and validate against that standard moving forward.
5. Think carefully about what data you migrate
Some companies think it’s easiest to just move all their data. But it’s not necessary or advisable. Be critical about what data you need to move into your new system. How often is the data recorded? How often is it used or referenced? How much value does it add having it in your core system? For data that’s irrelevant or obsolete, don’t move it. It won’t add value and you can keep an offline back-up for historical purposes.
6. Assume your data needs heavy cleaning
Most companies we work with are honest about the fact that their data quality needs improving. When you go into the cleaning process with that mindset, it will be easier (and less frustrating) to spot the issues. Take time to analyze the data, identify and correct issues and document new standard formats. Here’s what to look for:
Completeness – Is all the information being recorded? How often are fields left empty? You might identify some lesser-used fields that aren’t necessary moving forward. You could also uncover opportunities to add validation rules into your new system, prompting users to fill in required information.
Consistency – Is the data format consistent? Look at common examples like province – do you have a combination of British Columbia, BC and B.C.? Small details like this can impact reporting.
Accuracy – Is the data correct?
Duplication – Do you have multiple records for the same data? Common culprits are companies, contacts, items, etc. Identify and resolve duplicates before migrating. If you leave it until later, there’s a greater chance it won’t get done and it becomes harder to do over time.
7. Test thoroughly throughout data migration
Rather than importing your entire data set at once, import a small batch first. Then test thoroughly before you import more data. This approach allows you to troubleshoot any issues that come up before go-live. Reconcile data between your old and new systems by comparing the number of records, checking that the data populated as expected, and that the data is correct and in the right format.
Throughout the data cleaning and migration process, keep regular communication with your implementation partner. This is a critical step in your ERP implementation, so you want to do it right. Remember that your team is best equipped to do the heavy lifting on data cleaning, but your partner can support you and provide guidance along the way. Don’t be afraid to reach out for help if you need it.