A government client needed to restructure 1.3 million contact records in Microsoft Dynamics 365 CRM. At that scale, traditional UI tools and flows can throttle, fail silently, or leave a migration half-finished.
Instead of relying on standard imports, we used a custom console application built on the Dataverse SDK to execute the migration safely and deterministically, even when the job was interrupted mid-run.
Key Takeaways
- Large-scale updates in Dynamics 365 CRM / Dataverse require a different approach than standard UI tools or imports.
- A custom console app using the Dataverse SDK provides control over batching, retries, and execution logic.
- Idempotent update logic ensures migrations can safely resume if interrupted.
- Testing with dry runs and small batches first significantly reduces migration risk.
The Challenge: Migrating 1.3 Million Dynamics 365 CRM Records
Recently, one of our firm’s clients in the government sector needed to overhaul how they track contact origins in Microsoft Dynamics 365. The goal was straightforward on paper: migrate data from a legacy “Source” field into a newly redesigned, structured field and map the old option set values into new grouped categories.
The catch? We were dealing with 1.3 million Contact records.
Why Updating 1.3 Million Dynamics 365 Records Requires a Different Approach
Updating a few hundred records is a quick afternoon task. Updating 1.3 million records becomes an operational risk if it isn’t handled carefully. At that scale, standard out-of-the-box tools and event-driven automations simply cannot handle the load reliably.
Several factors made this migration particularly complex:
- Throttling and Timeouts
Standard UI tools and flows will inevitably throttle, time out, or fail silently when pushed to seven-digit volumes. - Inactive Records
A large portion of the database consisted of inactive contacts. The standard Dataverse UI locks these records down, preventing edits without temporarily reactivating them — a messy and risky workaround. - Data Integrity
We had to preserve existing, populated values. Accidentally overwriting valid data across millions of rows was not an option. - The “Half-Finished” Nightmare
If a standard import fails halfway through, figuring out exactly where it stopped and how to resume without duplicating work or corrupting data can become a massive headache.
Our Solution: Using a Dataverse Console App for a Large-Scale CRM Data Migration
To eliminate these risks, we bypassed the UI entirely and built a custom console app leveraging the Dataverse SDK. This wasn’t about building a flashy interface; it was about having absolute control and deterministic execution.
We implemented a strict “test small first” methodology:
- Idempotent Logic
The update rule was designed so that running it twice would not change anything the second time. It only targeted records where the old field had data and the new field was empty. - Dry Runs
We ran the logic in a simulated state first to validate mappings and confirm exact record counts without touching the live database. - Direct SDK Access
Using the Dataverse SDK allowed us to update inactive records without triggering UI-level restrictions. - Controlled Batching
We managed the exact size and speed of the data payloads being sent to the server to prevent throttling or system instability.
Ensuring Reliable Dynamics 365 Data Migration at Scale
This approach protected the client from data loss, downtime, and operational chaos.
The true test of our methodology happened mid-deployment: the power went out while the job was running. Because we designed the console app with idempotent logic, it wasn’t a crisis. We simply rebooted, restarted the application, and it picked up exactly where it left off, ignoring the records that had already been processed.
When your data volume reaches seven digits, you need a solution that guarantees success on the first try. By stepping outside the standard UI and using a controlled SDK-based approach, we delivered a flawless, verifiable migration.
Organizations planning a Dynamics 365 CRM implementation or large-scale data restructuring often encounter similar challenges, where careful planning and the right technical approach make all the difference.
Deeper Technical Breakdown of the Dataverse Migration
For a deeper technical breakdown of the migration, including the code logic, execution pattern, and the pre-flight checklist used for this project, read the full article on updating millions of Dataverse records using a custom console app.
