DQ Workflows
We've moved! To improve customer experience, the Collibra Data Quality User Guide has moved to the Collibra Documentation Center as part of the Collibra Data Quality 2022.11 release. To ensure a seamless transition, dq-docs.collibra.com will remain accessible, but the DQ User Guide is now maintained exclusively in the Documentation Center.
The DQ Workflows package listed on Collibra Marketplace allows you to 1) create and manage Data Quality Issues, 2) receive Notifications on Rule Metrics, and 3) request Rule Creation and Modification within Collibra Data Intelligence Cloud. Data stewards will be able to organize and prioritize all requests within DIC before they take any action within Collibra Data Quality.
Once deployed, the workflows will facilitate quicker data issue remediation by involving business analysts and other personas who can now participate in your data quality workstreams.
Please note: DQ Workflows are listed on Collibra Marketplace and are templates to get customers started. Collibra-provided Marketplace listings are not subject to the same SLA obligations (https://marketplace.collibra.com/marketplace-terms/) In addition, they can only be leveraged within Collibra Data Intelligence Cloud. In the future, we will work towards releasing bi-directional workflows.
Resource | Notes |
Collibra Edge Site | DQ Connector is a capability of Edge |
Collibra Data Intelligence Cloud | 2021.07 Release (or newer) |
Collibra Data Quality | 2.15 (or newer) |
Collibra DQ Connector | Synchronized Rules from Data Quality to Catalog |
Let's proceed after gathering all prerequisites!
1A. Download Package from Collibra Marketplace and Unzip Files
**1B. Deploy Workflows **
1C. Adjust Workflow Settings (One-Time Setup)
Workflow Configuration Setting | DQ Rule / DQ Sync Request | DQ Rule Modification | DQ Data Remediation | DQ Issue Resolution | Manage DQ Subscriptions | Notify of DQ Metrics |
|---|---|---|---|---|---|---|
Applies To | Asset | Asset | Asset | Asset | Global | Global |
Applies To Asset Type | Column, Table | Column, Table, Data Quality Rule | Column, Table | Issue | | |
Other: Any Signed In User Can Start Workflow | Y | Y | | | Y | Y |
Other: Perform Candidate User Check on Workflow Start | Y | Y | Y | | Y | Y |
Other: This Workflow Can Only Run Once At Same Time on Specific Resource | | | | | Y | Y |
Other: Show In Global Create | | | | | Y | Y |
Roles: Start Workflow | | | | | Sysadmin | Sysadmin |
Roles: Stop Workflow | | | | | Sysadmin | Sysadmin |
Roles: Reassign Tasks | | | | | Sysadmin | Sysadmin |
Example: DQ Rule Request
Example: Manage DQ Subscriptions
Workflow | Main Requestor Persona | Description | Steward Taking Action |
|---|---|---|---|
DQ Data Remediation | Data Steward Business Analyst | Tracking / management for confirmed data issues which may require underlying data remediation | Data Lake Admin, ETL Engineer |
DQ Rule Request | Business Analyst | Proposing data quality rules in plain language e.g. "flag any German phone numbers in this dataset" or "identify customers with churn risk based on engagement time with our platform" | Data Steward |
DQ Rule Modification | Business Analyst | Proposing adjustments to existing rules e.g. values, dimensions, passing thresholds | Data Steward |
DQ Sync Request | Business Analyst | Request for synchronization of the DQ Connector to synchronize and/or onboarding a new dataset with pre-populated rules | Data Steward |
Example: Data Quality Rule Request
Completed Rule Request Example
Example: Data Quality Rule Modification Request
Completed Data Quality Rule Modification Example
Example: DQ Synchronization Request with Rules Request optional checkbox
Data Helpdesk
- Select Issues
- Navigate to 'Filters'
- Properties > Attributes > Relations > Issue **categorized by **Issue Category > Input 'Data Quality Issue' > Apply
- Save button > Save View as > 'Data Quality Issues'
- Optional settings for View: Can pin, promote, make public, make default
Who? Anyone can set up a DQ subscription, for yourself or for your teammates
Alerts will be sent based on reviewing rules and metrics associated with Tables or Columns that violate the specified Threshold
Assuming an e-mail is associated with the Subscriber within Collibra, the Subscriber will receive e-mail notifications by default at 12pm local server time. This, along with other settings within the provided workflow, can be adjusted in Eclipse, Collibra's recommended workflow editor.
Ensure that the DQ alerts set for you are providing helpful details
For every subscription set up for a Subscriber, the Manage DQ Subscriptions modal will cycle through for your review. You can update Threshold, add or delete Notification Days, add or delete Tables or Columns, rename the Subscription title, Save the new settings, or simply Unsubscribe.
Last modified 2mo ago