Tag: Data Migration

Improved duplicate detection and data merge in Microsoft Dataverse

In case you haven’t followed the news, Microsoft Common Data Service has a new name, Its “Microsoft Dataverse”.

The change is just the name change at this time, behind the scene it’s the same CDS which has powered Dynamics 365 and Power Platform for years.

Dataverse Duplicate Detection and merge capabilities were due for a feature update in a long time, The alert and merge prompts were not optimized for UCI and looked out of Place on UCI apps.

The latest update( Release Wave 2 – 2020) to the platform provides new and improved Duplicate detection and merge capabilities as well as adds UCI compliant forms for merge.

This feature could be enabled by In Power Platform Admin Center under Duplicate Detection settings.

Continue reading “Improved duplicate detection and data merge in Microsoft Dataverse”

Data Migration Part IV : Data Cleansing

Data quality is determined by 3 key factors: Accuracy, Completeness and Relevancy/Validity. Data Quality is the most important aspect for any enterprise as it enables a business to make more informed decisions, better audience target, effective usage, and better user experience.

As an enterprise, any opportunity to improve on Data quality should be leveraged and Data Migration is one the best time to do a reset and clean the data. So let’s look into what are a few common data quality issues and how we deal with them when using Dynamics 365. Continue reading “Data Migration Part IV : Data Cleansing”

Data Migration Part III : Data load strategy

Data migration deployment often happens along with D365 application Go-Live and it can take significant time to complete. Go-live deployment means downtime for application and adding data migration could really increase that downtime by tenfolds. So how do we do them together while keeping the application outages to a minimum? let’s dive in look at some of the strategies.

Continue reading “Data Migration Part III : Data load strategy”

Data Migration Part II : Evaluation & Planning

As Antoine de Saint-Exupéry said, “A goal without a plan is just a wish.” In terms of Data migration that is so true,  according to Gartner, 83 % of data migrations fail outright or exceed their allotted budgets. A lot of this failure is attributed to a lack of planning.

Analysis of data migration projects over the years has shown that they meet with mixed results. While mission-critical to the success of the business initiatives they are meant to facilitate and support lack of planning structure and attention to risks causes many data migration efforts fail.
— Gartner, “Risks and Challenges in Data Migrations and Conversions,” February 2009, ID Number: G00165710

In the last blog, I mentioned about Data Migration stages. In this blog will do a deeper dive and look in the first 2 stages,  So let’s talk about what planning and Evaluation look like.

Evaluate: Data

  1. Data Sources: Identify all different data sources for Data migration, often time Dynamics 365 implementation replaces more than one applications as such you might be migrating data from multiple applications. 
  2. Data Quality: it’s important to analyze the quality of data at the source to avoid Garbage In, Garbage Out. Some of the common quality issues are
    1. Inconsistent and incomplete Data: inconsistency is a big indicator that there’s a data quality problem. When data has not been entered in the system correctly like missing zip code, province or other information remaining data could be of less value.
    2. Duplicates: In many circumstances, the same records might exist multiple times in a database, Common culprits are Account, Contacts.
    3. Inconsistent Formats: if source application did not have much validation the data will have more typos of errors, Address is the most common place to find with City and provinces being named differently.
    4. Obsolete Data:  Every day, people move, marry, and change their names. Since Peoples’ information is rarely static, obsolete data is a common challenge which is hard to rectify as well.
  3. Data Volume: Volume of Data could be a key factor in determining Data migration Tool/Process and the Deployment Plan.
  4. Operational Data vs Archival Data: Legacy applications when replaced will often have a lot of historical data, you might not want to bring all of this data into Dynamics 365 as this data might be not be used for day to day operation but useful for Data Analytics and Reporting Purposes. Identify what data needs to be in Dynamics 365.

Continue reading “Data Migration Part II : Evaluation & Planning”

Dynamics 365 : Data Migration demystified-Part I

Data migration is often overlooked in Project implementation primarily because the project team is focussed on application development and customization which takes away the priority of data integration and data migration. As data migration is planned to be the last activity on the project it also gets most impacted from any budget cuts and application scope increase. 

Talking about the budget, many implementation’s approaches to data migration is for data migration to be done using the Import and Export feature of Dynamics 365 CE over the weekend before Go Live, this works sometimes but is a recipe of disaster in most cases. 

When implementing Dynamics 365 Project, configuration and customization to meet the users need is only half the battle. Data migration is key to getting the right head start for users to start using the application. In this blog series, I will focus on data migration, the challenges, strategy, Process, Tools and my own learnings over the years. Let’s start with what is Data migration?

Continue reading “Dynamics 365 : Data Migration demystified-Part I”

Microsoft Dynamics 365 Connector for Logic App gets an update

Last week Microsoft released an update for Dynamics 365 Connector for Logic App, there are few long-awaited features in this update which I am excited about.

With this update Microsoft has added support for option sets being exposed as strings in the response, this used to be a challenge earlier as Optionset value and mapping had to be hardcoded in the Logic app or Azure Functions.

the difference in the options set value in the output:


Continue reading “Microsoft Dynamics 365 Connector for Logic App gets an update”

Basics of Azure Logic Apps Schema and Codeview editing

Integrations using Logic App could be a fun exercise. Logic App provides a visual designer to model and automate your workflow process as a series of steps called as workflow designer UI,  for complex requirement Code view comes handy. In this blog, I will talk about the structure of logic app definition.

Let’s start with some basic components of Logic App.

    • Logic App Visual designer actually creates a JSON file with workflow definitions. JSON structure for Logic app

{
"$schema": "",
"contentVersion": "",
"parameters": { },
"triggers": [ { } ],
"actions": [ { } ],
"outputs": { }
}

$Schema, content version and static values and are set to schema and version automatically.

Parameters are important as they help parameterize the logic app and assist with Deployments and change management. I will go more in-depth in my subsequent blog on Logic App and Change management/ALM.

Triggers: define the trigger criteria for the workflow. to access the output of trigger @triggerBody() could be used.

Actions: define the actions within the workflow, actions are nested and using runAfter you may define the dependent action or parallel action flow.

Outputs: refer to the response of the workflow post execution of all the actions. Most of our Integration scenario doesn’t use the output as we will have another action to consume the output.

Continue reading “Basics of Azure Logic Apps Schema and Codeview editing”