Data Migration Part III : Data load strategy

Data migration deployment often happens along with D365 application Go-Live and it can take significant time to complete. Go-live deployment means downtime for application and adding data migration could really increase that downtime by tenfolds. So how do we do them together while keeping the application outages to a minimum? let’s dive in look at some of the strategies.

This is my third blog on data migration, you can check out other blogs in the series here.

Strategy I – Big Bang Approach: As the name suggest its an approach where data migration is done along with other deployment activities likely over a weekend. This does work and could be ideal in some cases but is prone to failures and risk. Let’s look into some of the Pros and cons:

Pros: 

    • Quicker deployment
    • All deployment activities are done together, leading to some efficiency.
    • only suited for a very small data set migration

Cons:

    • longer Application Outage
    • Likely not enough time for Data Validation
    • Failure in data migration could impact Go live deployment and vice versa

Strategy II – Delta Migration Approach: This is my preferred Data migration approach, In this approach data, is migrated using 2 passes:

Primary data load: loading all the data in the system up to a pre-selected date.

Delta load: Delta data load to load the final set of data accumulated since primary data load. 

This approach allows for the majority of data to be pre-loaded during primary load and reduce the time required for data migration during Go live. Early data load also allows for data validation, correction, and optimization.

Strategy III – Multi-stage migration: Enterprise uses Dynamics 365 for mission-critical business purposes as such it can not have downtime. in enterprise application data generation happens at a lot quicker rate as such even Delta data migration requires a lot of time. In such cases data migration is done in 3 stages:

Primary data load: This is same as in Delta data Migration where we load all the data in the system up to a pre-selected date. This will be ideally done a couple of weeks before Go-live.

Delta data load: Delta data load will be done a few days to a week before the go-live and will load the data accumulated since the last data migration date.

Go-live Migration: Go live migration will be done on the day/hour of Go live and will migrate the final set of data accumulated since delta data migration. This data set should be kept as small as possible.

Strategy IV – Iterative Data Migration: This is best suited for with complex enterprise data migration where a lot of historical data needs to be migrated. A lot of historical data means application should be able to work with all different kind and formats of data which would have gone numerous changes over the years. The challenge is not only to move the data but also ensure that the application can work with legacy data.

In such cases, you would do a Primary Data migration long before the go-live and will do Corrections as well as additive data Migration in sprints until you reach Go Live. At go-live, the final set of data will be migrated

once the deployment strategy that works for your project/implementation is identified, the data migration scripts should be built to support running it for the initial run, delta run and go live run. 

Let me know thoughts on deployment approach for data migration and what migration approach worked for you. In the next blog, we will look at Data cleansing approaches for data migration.

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *