Working with customer data is never an easy task, especially when syncing data from one system to another. At Beringer, we implement plenty of custom integrations that handle data transfer between an ERP database and CRM. It is important to understand the data structure behind both systems thoroughly as well as have an accurate mapping of what specific information goes where when syncing these systems. Today I will outline for you what I consider the top 3 most important things when designing a custom integration with Microsoft Dynamics CRM.
Data Map Documentation
One crucial document to the construction of a custom integration is the Data Map. A Data Map will contain the fields and data types from the source database and show in clear detail where they will go in the target database. The client will initially design this mapping as they would have the best understanding of the ERP system and what data they will want to bring to the CRM. The client will then send that document to a project manager who will determine if the mapping is reasonable for CRM at a baseline functionality level. The document can be sent back to the client if there needs to be a change and this process will repeat. Once this document is agreed upon between the client and project manager, this will be sent to the developer who will verify the fields and design the target system as well as the integration tool to fit the data listed in the Data Map. If anything on the Data Map requires a revision, the document is sent through the process again so the client can approve or alter any data that would be sent to the integration tool to fit the target database’s specifications.
Timing of Data Processing
Timing the integration process itself is also something to take into consideration. CRM for example is a highly-organized system that is modeled after a data hierarchy. An example of this is any parent-child entity relationship (Parent Customer ->Child Contacts; Parent Order -> Child Order Products). This means that to complete this hierarchy, you will need to place the Parent entities into the system before putting the child entities in so the system is properly created. For a recent integration, I chose Scribe Online as my tool of choice as it gives me the option to add data synchronously and asynchronously. Using their solution/mapping model, I was able to align the hierarchical relationships in a synchronous order (Order Mapping completes before Order Product Mapping starts), and have processes that do not rely on the hierarchy to run asynchronously.
Working With Live Data
Another concern to address when working with these integrations is the need for caution when working with live data. If the client is syncing data between two databases, most likely both data sets are being worked on and changing during business hours. User traffic and any system load brought on by your integration implementation are things to consider for integration design. You do not want your integration to lock up the source system while other users are actively using the system. One solution for this issue you may consider is the use of the WITH (NOLOCK) SQL statement when gathering information for your integration. The idea is to make sure that your reading of the source’s data does not interrupt standard use of the system by other users. Please refer to my colleague’s article on Minimizing SQL Dead Locks for more information.
Obviously there are many things to consider while working with client data, but when it comes to integrating two systems, these are some of the more critical points. I hope this article was helpful and got you thinking about your next data integration design. Thanks for reading and happy coding!