If you've ever been tasked with migrating data, then you know there are a wide variety of challenges that you must overcome. I've migrated data to Microsoft Dynamics CRM from many disparate systems in my career, which typically involves pulling data from a live system. The key word is "live", so you need to tread lightly when querying millions of rows of data. It is very easy to negatively impact performance, which will instantly translate into end user complaints.
My best practice is to simply run the data migration during off hours, when the end users are not in the system. I will typically schedule the data migration to launch on a Friday afternoon after everyone has left for the day. While most migration tools allow you to control when to start your migration, they do not account for a hard stop if all of your data is not migrated when the end users come back on Monday morning. Prepare for a whole new onslaught of performance complaints!
I find Scribe Online is a great cloud based platform to migrate and transform data between systems, especially when working with Microsoft Dynamics CRM. This product has a very intuitive interface, and can be configured to handle this scenario. Are you ready to see how it's done?
First, let's start with a simple map that queries SQL data. It will loop thru the data source and create or update an account in CRM. Take a closer look, and you will see that I have a conditional block that will check to see if it is 7am on a Monday morning. If so, it will execute a Map Exit block, which will stop the data migration.
Dive into the conditional block and you will see the logic used to determine if it is 7am on Monday. Since I don't know the local time zone of the server, I'm going to focus on Coordinated Universal Time (UTC).
You can even customize the error message in your Map Exit block, which is displayed in the execution history.
A well designed query block should always focus on a date field. When you resume your data migration, you will see it pick up right where it left off.
Beringer is always here to provide expert knowledge in topics like these and has deep experience integrating data. Can we help migrate you to Microsoft Dynamics CRM?