DQ Workflows

Benefits
The DQ Workflows package listed on Collibra Marketplace allows you to 1) create and manage Data Quality Issues, 2) receive Notifications on Rule Metrics, and 3) request Rule Creation and Modification within Collibra Data Intelligence Cloud. Data stewards will be able to organize and prioritize all requests within DIC before they take any action within Collibra Data Quality.
Once deployed, the workflows will facilitate quicker data issue remediation by involving business analysts and other personas who can now participate in your data quality workstreams.
Please note: DQ Workflows are listed on Collibra Marketplace and are templates to get customers started. Collibra-provided Marketplace listings are not subject to the same SLA obligations (https://marketplace.collibra.com/marketplace-terms/) In addition, they can only be leveraged within Collibra Data Intelligence Cloud. In the future, we will work towards releasing bi-directional workflows.
Step 0: Prerequisites
Text
Text
Resource
Notes
Collibra Edge Site
DQ Connector is a capability of Edge
Collibra Data Intelligence Cloud
2021.07 Release (or newer)
Collibra Data Quality
2.15 (or newer)
Collibra DQ Connector
Synchronized Rules from Data Quality to Catalog
Let's proceed after gathering all prerequisites!
Step 1: Download, Deploy and Start DQ Workflows
1A. Download Package from Collibra Marketplace and Unzip Files
**1B. Deploy Workflows **
1C. Adjust Workflow Settings (One-Time Setup)
Text
Text
Workflows
Adjustments To Default Load Settings
DQ Rule Request
DQ Sync Request
Applies To: Edit: Global -> Asset
Asset Type: Add Rules: Column, Table
Other: Any signed in user can start workflow (Check)
Other: This workflow can only run once at the same time on specific resource (Uncheck)
DQ Rule Modification
Applies To: Edit: Global -> Asset
Asset Type: Add Rules: Column, Table, Data Quality Rule
Other: Any signed in user can start workflow (Check)
Other: This workflow can only run once at the same time on specific resource (Uncheck)
DQ Data Remediation
Applies To: Edit: Global -> Asset
Asset Type: Add Rules: Column, Table
Other: Any signed in user can start workflow (Check)
DQ Issue Resolution
Applies To: Edit: Global -> Asset
Asset Type: Add Rules: Issue
Manage DQ Subscriptions
Other: Any signed in user can start workflow (Check)
Other: Show in global create (Check)
Notify of DQ Metrics
None
Example: DQ Rule Request
Example: Manage DQ Subscriptions
Step 2: Create Data Quality Requests / Issues
2A. Create Data Quality Issues
Workflow
Main Requestor Persona
Description
Steward Taking Action
DQ Data Remediation
Data Steward
Business Analyst
Tracking / management for confirmed data issues which may require underlying data remediation
Data Lake Admin, ETL Engineer
DQ Rule Request
Business Analyst
Proposing data quality rules in plain language e.g. "flag any German phone numbers in this dataset" or "identify customers with churn risk based on engagement time with our platform"
Data Steward
DQ Rule Modification
Business Analyst
Proposing adjustments to existing rules e.g. values, dimensions, passing thresholds
Data Steward
DQ Sync Request
Business Analyst
Request for synchronization of the DQ Connector to synchronize and/or onboarding a new dataset with pre-populated rules
Data Steward
Example: Data Quality Rule Request
Completed Rule Request Example
Example: Data Quality Rule Modification Request
Completed Data Quality Rule Modification Example
Example: DQ Synchronization Request with Rules Request optional checkbox
Step 3: Manage Data Quality Issues
3A. Setup Data Helpdesk Filter
Data Helpdesk
Select Issues
Navigate to 'Filters'
Properties > Attributes > Relations > Issue **categorized by **Issue Category > Input 'Data Quality Issue' > Apply
Save button > Save View as > 'Data Quality Issues'
Optional settings for View: Can pin, promote, make public, make default
3B. Manage Issues From Data Helpdesk View
3C. Alternate: Manage Issues From Tasks
Step 4: Receive Notifications Of DQ Issues And Metrics
4A. Set Up DQ Metric Subscription
Who? Anyone can set up a DQ subscription, for yourself or for your teammates
Alerts will be sent based on reviewing rules and metrics associated with Tables or Columns that violate the specified Threshold
Assuming an e-mail is associated with the Subscriber within Collibra, the Subscriber will receive e-mail notifications by default at 12pm local server time. This, along with other settings within the provided workflow, can be adjusted in Eclipse, Collibra's recommended workflow editor.
4B. Review DQ Metric Alerts
Ensure that the DQ alerts set for you are providing helpful details
4C. Update Subscription Settings
For every subscription set up for a Subscriber, the Manage DQ Subscriptions modal will cycle through for your review. You can update Threshold, add or delete Notification Days, add or delete Tables or Columns, rename the Subscription title, Save the new settings, or simply Unsubscribe.