Flow: Connectors for Common Data Service Platform

Last week, I was prepping for my session at Dynamics 365 Summit on Logic Apps and noticed my Flow connector got few more features added. Often time Logic Apps connector is more advanced than the same connector in Flow, this is so not true when for CDS/Dynamics. 

Flow currently has 3 Connectors for connecting to CDS/Dynamics 365:

Dynamics 365 Connector: was the first D365 connector published and has been around since Flow & Logic Apps were released. This connector went through a couple of revision and has been deemed deprecated since April 22nd in Favour of CDS Connector

Common Data Service: This was released when CDS was released this is the recommended connector to be used with Flow and Logic Apps when creating independent flow. This connector has parity with Dynamics 365 connector which has been deprecated.

Common Data Service (current environment): is the newest Flow connector, released earlier this year. As the name suggests this connector is Environment specific and solution aware. it also has a lot of new features(actions/Triggers).

Continue reading “Flow: Connectors for Common Data Service Platform”

Data Migration Part IV : Data Cleansing

Data quality is determined by 3 key factors: Accuracy, Completeness and Relevancy/Validity. Data Quality is the most important aspect for any enterprise as it enables a business to make more informed decisions, better audience target, effective usage, and better user experience.

As an enterprise, any opportunity to improve on Data quality should be leveraged and Data Migration is one the best time to do a reset and clean the data. So let’s look into what are a few common data quality issues and how we deal with them when using Dynamics 365. Continue reading “Data Migration Part IV : Data Cleansing”

Data Migration Part II : Evaluation & Planning

As Antoine de Saint-Exupéry said, “A goal without a plan is just a wish.” In terms of Data migration that is so true,  according to Gartner, 83 % of data migrations fail outright or exceed their allotted budgets. A lot of this failure is attributed to a lack of planning.

Analysis of data migration projects over the years has shown that they meet with mixed results. While mission-critical to the success of the business initiatives they are meant to facilitate and support lack of planning structure and attention to risks causes many data migration efforts fail.
— Gartner, “Risks and Challenges in Data Migrations and Conversions,” February 2009, ID Number: G00165710

In the last blog, I mentioned about Data Migration stages. In this blog will do a deeper dive and look in the first 2 stages,  So let’s talk about what planning and Evaluation look like.

Evaluate: Data

  1. Data Sources: Identify all different data sources for Data migration, often time Dynamics 365 implementation replaces more than one applications as such you might be migrating data from multiple applications. 
  2. Data Quality: it’s important to analyze the quality of data at the source to avoid Garbage In, Garbage Out. Some of the common quality issues are
    1. Inconsistent and incomplete Data: inconsistency is a big indicator that there’s a data quality problem. When data has not been entered in the system correctly like missing zip code, province or other information remaining data could be of less value.
    2. Duplicates: In many circumstances, the same records might exist multiple times in a database, Common culprits are Account, Contacts.
    3. Inconsistent Formats: if source application did not have much validation the data will have more typos of errors, Address is the most common place to find with City and provinces being named differently.
    4. Obsolete Data:  Every day, people move, marry, and change their names. Since Peoples’ information is rarely static, obsolete data is a common challenge which is hard to rectify as well.
  3. Data Volume: Volume of Data could be a key factor in determining Data migration Tool/Process and the Deployment Plan.
  4. Operational Data vs Archival Data: Legacy applications when replaced will often have a lot of historical data, you might not want to bring all of this data into Dynamics 365 as this data might be not be used for day to day operation but useful for Data Analytics and Reporting Purposes. Identify what data needs to be in Dynamics 365.

Continue reading “Data Migration Part II : Evaluation & Planning”

Dynamics 365 : Data Migration demystified-Part I

Data migration is often overlooked in Project implementation primarily because the project team is focussed on application development and customization which takes away the priority of data integration and data migration. As data migration is planned to be the last activity on the project it also gets most impacted from any budget cuts and application scope increase. 

Talking about the budget, many implementation’s approaches to data migration is for data migration to be done using the Import and Export feature of Dynamics 365 CE over the weekend before Go Live, this works sometimes but is a recipe of disaster in most cases. 

When implementing Dynamics 365 Project, configuration and customization to meet the users need is only half the battle. Data migration is key to getting the right head start for users to start using the application. In this blog series, I will focus on data migration, the challenges, strategy, Process, Tools and my own learnings over the years. Let’s start with what is Data migration?

Continue reading “Dynamics 365 : Data Migration demystified-Part I”

Logic App ALM Process for Dynamics 365 Integration – ALM Part 2

In the previous blog  – we looked into working with Logic Apps from Visual Studio, which allowed us to manage logic app code in TFS or Git or any other source control system. But to truly do DevOps for Logic Apps while integrating with Dynamics 365 you need to be able to do ALM process for Logic Apps better yet do continuous delivery/deployment.

when we are editing and deploying a Logic app in Visual Studio we are actually editing an ARM Template, So what is an ARM template?

ARM templates are JSON format template, it enables you to deploy Azure resources in a declarative manner and could be used to define all the resources in a resource group.

An Empty ARM Template Schema

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": { },
"variables" : { },
"resources": [],
"outputs": { }
}

To add a resource in ARM template, resource details are declared under resources properties, there could be multiple resources. Below is a sample of Empty Logic app added to empty ARM Template.

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {}, //ARM parameters
  "variables": {}, //ARM template variables
  "resources": [
    {
      "name": "Logic APP Name",
      "type": "Microsoft.Logic/workflows",
      "location": "West US",
      "tags": {"displayName": "LogicApp" },
      "apiVersion": "2016-06-01",
      "properties": {
        "definition": {
          "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
          "contentVersion": "1.0.0.0",
          "actions": {},
          "outputs": {},
          "parameters": {},//Logic App defination parameters defination
          "triggers": {}
        },
        "parameters": {}   //parameter values
      }
    }
  ],
  "outputs": {}
}

Continue reading “Logic App ALM Process for Dynamics 365 Integration – ALM Part 2”

Migrate Dynamics 365 Notes Attachments to SharePoint

Quite often we run across a Dynamics 365 implementation where not much thought was giving to Document Management while implementation, resulting in user using Notes(Annotation) entity for document management.

Notes do offer a lightweight solution for managing and uploading documents to related record but it lacks the capabilities of a full-fledged document management capabilities. Notes are also stored in CRM database, storing documents as notes could easily become a costly endeavor considering Dynamics 365 storage cost. Jukka Niiranen has written a great blog on file storage and disadvantages of using Notes/Annotation.

The first step to transition from using CRM Notes to SharePoint for document management is migrating existing documents from Notes to SharePoint. This may seem like a daunting task but could easily be accomplished using SSIS package for migration.

I have an SSIS starter project on GitHub which illustrates the migration for notes related to Case entity, It could be easily configured to migrate notes related to any other entity.

SSIS by itself doesn’t provide capabilities to connect with Dynamics 365, for this, we use a third-party connector like KingswaySoft or CozyRoc, in below example we are using KingswaySoft connector, license and pricing details could be found here. Kingsway does provide the developer license free of cost.

Continue reading “Migrate Dynamics 365 Notes Attachments to SharePoint”

eXtreme365 Dubrovnik 2018

Extreme365, just completed in Dubrovnik, Croatia. What an amazing, beautiful and fun location for a great Dynamics conference. A week surrounded by amazing people sharing knowledge while having fun.

This was special for me as this was my first Extreme 365 conference speaking engagement. I presented my take on “Modern Approach to integration with Dynamics 365“, the session was well attended and very well received. Thank you all who attended my session.

The conference coincided with spring release announcement of Next version/ generation of Dynamics 365. This was one of the biggest Dynamics release announcement. for the first time, Microsoft had to release document rather than release notes primarily because of the number of changes being shipped on this release. With this release Dynamics, 365 becomes the Application platform for business application.

We also watched live streaming of  Business Forward’s in Amsterdam with Jean-Philippe Courtois, Alysa Taylor, and James Phillips at the conference.

Key Highlights from Dubrovnik: Continue reading “eXtreme365 Dubrovnik 2018”

Dynamics 365 Portals: Create new entity record from within Lookups

If you have used Dynamics 365 you must be familiar with the + New button on the lookup. This enables the user to create new item instead of selecting exiting one.

This feature has been a frequent ask for Dynamics 365 Portals. Good news is that this functionality was added by the product team few releases back the bad is that it is only partially done.

In this post, I’ll walk through the steps for configuring it and its limitation.
Continue reading “Dynamics 365 Portals: Create new entity record from within Lookups”

Microsoft Dynamics 365 Connector for Logic App gets an update

Last week Microsoft released an update for Dynamics 365 Connector for Logic App, there are few long-awaited features in this update which I am excited about.

With this update Microsoft has added support for option sets being exposed as strings in the response, this used to be a challenge earlier as Optionset value and mapping had to be hardcoded in the Logic app or Azure Functions.

the difference in the options set value in the output:


Continue reading “Microsoft Dynamics 365 Connector for Logic App gets an update”

Integration with Microsoft Dynamics 365 using Azure Logic App Service

In this post I’ll discuss Data Integration using Azure Logic App Service, we will create a Logic app that gets triggered when a case is created in Dynamics 365 and inturns creates a work item in Visual Studio Team services for the team to work on.

Logic App workflow is built using series of actions and Connectors, and has its own JSON based workflow definition schema, for details refer to my blog on The Basics of Azure Logic Apps Schema and Codeview editing

Pre-requisites
  • Microsoft Dynamics 365 Online instance (On-premise instance could be used as well using Azure Data Gateway)
  • Microsoft Azure subscription for creating a Logic APP (If you don’t have an Azure subscription you can either use Try a Logic App feature which creates a logic app for an hour or you may sign up for a Trial account.)
  • Visual Studio Team services account (SignUp link).

Continue reading “Integration with Microsoft Dynamics 365 using Azure Logic App Service”

Basics of Azure Logic Apps Schema and Codeview editing

Integrations using Logic App could be a fun exercise. Logic App provides a visual designer to model and automate your workflow process as a series of steps called as workflow designer UI,  for complex requirement Code view comes handy. In this blog, I will talk about the structure of logic app definition.

Let’s start with some basic components of Logic App.

    • Logic App Visual designer actually creates a JSON file with workflow definitions. JSON structure for Logic app

{
"$schema": "",
"contentVersion": "",
"parameters": { },
"triggers": [ { } ],
"actions": [ { } ],
"outputs": { }
}

$Schema, content version and static values and are set to schema and version automatically.

Parameters are important as they help parameterize the logic app and assist with Deployments and change management. I will go more in-depth in my subsequent blog on Logic App and Change management/ALM.

Triggers: define the trigger criteria for the workflow. to access the output of trigger @triggerBody() could be used.

Actions: define the actions within the workflow, actions are nested and using runAfter you may define the dependent action or parallel action flow.

Outputs: refer to the response of the workflow post execution of all the actions. Most of our Integration scenario doesn’t use the output as we will have another action to consume the output.

Continue reading “Basics of Azure Logic Apps Schema and Codeview editing”