DQ Job MongoDB

Browse MongoDB like any other relational database

Using Owl's file tree explorer browse Mongo "collections" like "tables". Then use the wizard to create standard DQ scans.

CMD Line

Copy paste-able cmdline example for simple spark submit job.
1
-lib "/opt/owl/drivers/mongodb/"
2
-h localhost:5432/postgres
3
-master local[*]
4
-ds tpch.lineitem_7
5
-br 10 -deploymode client
6
-q "select * from tpch.lineitem where l_shipdate between '${rd} 00:00:00.000+0000'
7
and '${rdEnd} 00:00:00.000+0000' "
8
-bhlb 10 -rd "1998-12-01"
9
-driver "mongodb.jdbc.MongoDriver"
10
-loglevel INFO -cxn MongoDB -rdEnd "1998-12-02"
Copied!

Drivers and Config

In order to make this possible OwlDQ requires 2 drivers, MongoDB driver and UnityJDBC Driver. Out of the box OwlDQ comes preconfigured with these drivers. You simply open the MongoDB connection template and paste in your JDBC URL.
1
driverClass: mongodb.jdbc.MongoDriver
2
3
path: /opt/owl/drivers/mongodb/
4
+-- mongoJdbc.jar
5
+-- unityJDBC.jar
Copied!

Simply paste in JDBC Info

Discover Correlations, Relationships, DQ issues and Much More...

The following table presents the various SQL statements related to table-level actions and the corresponding MongoDB statements.https://docs.mongodb.com/manual/reference/sql-comparison/

Limiting Collections in the JDBC URL

1
jdbc:mongodb://<dbuser>:<password>@datalake0-dza1q.a.query.mongodb.net/<mydatabase>?ssl=true&authSource=admin&rebuildschema=true&tables=orders
Copied!
There are 3 collections in this mongodb atlas lake. By adding &tables=orders in the URL params you can see only order collections show up in the explorer.

3 Collections in MongoDB Atlas

The total number of collections in mongodb atlas lake.
Last modified 20d ago