Delphix for Microsoft Intelligent Data Platform
Together, Delphix and Microsoft IDP combine industry-leading compliance and automation services to make the delivery of compliant data easy for everyone
There has been an explosion of data in recent years. In order to unlock the strategic value of data, it needs to be dynamic and portable. Data present in silos limits its strategic value and is difficult to use for analytical purposes.
Breaking down data silos is difficult:
- Data must be manipulated to fit to a common format. ETL pipelines must be adapted to each system of record and must scale to support the massive data sets of modern enterprises.
- Compliance with regulations regarding sensitive information must be maintained when data is moved from systems of record. PII and other sensitive elements must be obscured without impacting the business value of the data set.
Realize Your Goals
Delphix enables compliance and security for the Microsoft Intelligent Data Platform by automatically discovering and masking PII and other sensitive information. With Delphix, enterprises stay compliant with privacy regulations and mitigate risk of breach, while accelerating data delivery for development, analytics, and AI/ML.
Data Compliance & Security
- Discovery for PII, other sensitive information
- Masking for privacy, industry compliance
- Automation for zero trust
Frictionless, Democratized Data
- Universal connectivity
- Automated masking, API integration for IDP data flows
- No-code / low-code
- Accelerators for key enterprise apps
Quality Data for Development, Analytics
- Use real, masked data for analytics
- Robust OOTB or customizable algorithms
- Preserve referential integrity
Enabling Data Compliance and Security for Microsoft IDP
Automate discovery and masking of sensitive information for development and analytics
The following architecture outlines the use of Delphix Continuous Compliance in an ADF ETL pipeline to identify and mask sensitive data.
Benefits
- Universal connectivity
- Realistic, deterministic masking that maintains referential integrity
- Preemptive identification of sensitive data for key enterprise applications
- Native cloud execution
- Template-based deployment
- Scalable
Dataflow
The data flows through the scenario below as follows:
- ADF extracts data from source datastore(s) to a container in Azure File Storage using the Copy Data activity. This container is referred to as the Source Data Container and the data is in CSV format.
- ADF initiates an iterator (ForEach activity) that loops through a list of masking jobs configured within Delphix. These masking jobs will be pre configured and will collectively mask sensitive data present in the Source Data Container.
- For each job in the list, the Initiate Masking activity authenticates and initiates the masking job by calling the REST API endpoints on the Delphix Continuous Compliance Engine.
- The Delphix Continuous Compliance Engine reads data from the Source Data Container and runs through the masking process.
- In this masking process, Delphix masks data in-memory and writes the resultant masked data back to a target Azure File Storage container (referred to as Target Data Container).
- ADF now initiates a second iterator (ForEach activity) that monitors the executions.
- For each execution (Masking Job) that was started, the Check Status activity checks the result of masking.
- Once all masking jobs have successfully completed, ADF loads the masked data from Target Data Container to the specified destination.
Frequently Asked Questions