Microsoft Intelligent Data Platform Integration

Delphix for Microsoft Intelligent Data Platform

Together, Delphix and Microsoft IDP combine industry-leading compliance and automation services to make the delivery of compliant data easy for everyone

There has been an explosion of data in recent years. In order to unlock the strategic value of data, it needs to be dynamic and portable. Data present in silos limits its strategic value and is difficult to use for analytical purposes.
Breaking down data silos is difficult:

  • Data must be manipulated to fit to a common format. ETL pipelines must be adapted to each system of record and must scale to support the massive data sets of modern enterprises.
     
  • Compliance with regulations regarding sensitive information must be maintained when data is moved from systems of record. PII and other sensitive elements must be obscured without impacting the business value of the data set.

Realize Your Goals

Delphix enables compliance and security for the Microsoft Intelligent Data Platform by automatically discovering and masking PII and other sensitive information. With Delphix, enterprises stay compliant with privacy regulations and mitigate risk of breach, while accelerating data delivery for development, analytics, and AI/ML.
 

Data Compliance & Security

  • Discovery for PII, other sensitive information
  • Masking for privacy, industry compliance
  • Automation for zero trust

Frictionless, Democratized Data

  • Universal connectivity
  • Automated masking, API integration for IDP data flows
  • No-code / low-code
  • Accelerators for key enterprise apps

Quality Data for Development, Analytics

  • Use real, masked data for analytics
  • Robust OOTB or customizable algorithms
  • Preserve referential integrity

Enabling Data Compliance and Security for Microsoft IDP

Automate discovery and masking of sensitive information for development and analytics

The following architecture outlines the use of Delphix Continuous Compliance in an ADF ETL pipeline to identify and mask sensitive data.

Benefits

  • Universal connectivity
     
  • Realistic, deterministic masking that maintains referential integrity
     
  • Preemptive identification of sensitive data for key enterprise applications
     
  • Native cloud execution
     
  • Template-based deployment
     
  • Scalable

Dataflow

The data flows through the scenario below as follows:

  1. ADF extracts data from source datastore(s) to a container in Azure File Storage using the Copy Data activity. This container is referred to as the Source Data Container and the data is in CSV format.
     
  2. ADF initiates an iterator (ForEach activity) that loops through a list of masking jobs configured within Delphix. These masking jobs will be pre configured and will collectively mask sensitive data present in the Source Data Container.
     
  3. For each job in the list, the Initiate Masking activity authenticates and initiates the masking job by calling the REST API endpoints on the Delphix Continuous Compliance Engine.
     
  4. The Delphix Continuous Compliance Engine reads data from the Source Data Container and runs through the masking process.
     
  5. In this masking process, Delphix masks data in-memory and writes the resultant masked data back to a target Azure File Storage container (referred to as Target Data Container).
     
  6. ADF now initiates a second iterator (ForEach activity) that monitors the executions.
     
  7. For each execution (Masking Job) that was started, the Check Status activity checks the result of masking.
     
  8. Once all masking jobs have successfully completed, ADF loads the masked data from Target Data Container to the specified destination.
     
AzureDelphixArchitecture

Frequently Asked Questions

Where can I learn more about Delphix and Azure?
Read more about accelerating your Azure journey
Where does Delphix fit in the Microsoft Intellegent Data Platform?
Delphix for Microsoft Intelligent Data Platform and
Introducing the Microsoft Intelligent Data Platform Partner Ecosystem
Where can I learn more about the solution?
Read our whitepaper
How can I get started using the solution?
Navigate to adf.azure.com, log in and open the Template Gallery, then search for Delphix.