2021.10
Collibra DIC Integration
Powered By GitBook
Cloudera CLASSPATH

What is a CLASSPATH?

A CLASSPATH is essentially a list of jars that get injected into a JVM on the start of a job execution. Like many applications, Spark can have jars injected when a job is run. Cloudera has defined a list of predefined jars (rightfully called classpath.txt):
1
/etc/spark2/conf/classpath.txt
Copied!
that will get injected whenever Spark is called. Here is an example list of jars as defined within a cluster we have stood up @ Owl:
1
[[email protected] ~]$ cat /etc/spark2/conf/classpath.txt
2
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/activation-1.1.jar
3
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/aopalliance-1.0.jar
4
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/apacheds-i18n-2.0.0-M15.jar
5
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/apacheds-kerberos-codec-2.0.0-M15.jar
6
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/api-asn1-api-1.0.0-M20.jar
7
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/api-util-1.0.0-M20.jar
8
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/asm-3.2.jar
9
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/avro-1.7.6-cdh5.16.1.jar
10
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/aws-java-sdk-bundle-1.11.134.jar
11
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/azure-data-lake-store-sdk-2.2.9.jar
12
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-beanutils-1.9.2.jar
13
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-beanutils-core-1.8.0.jar
14
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-codec-1.4.jar
15
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-codec-1.9.jar
16
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-configuration-1.6.jar
17
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-daemon-1.0.13.jar
18
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-digester-1.8.jar
19
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-el-1.0.jar
20
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-logging-1.2.jar
21
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-math-2.1.jar
22
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-math3-3.1.1.jar
23
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/commons-net-3.1.jar
24
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/core-3.1.1.jar
25
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/curator-client-2.7.1.jar
26
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/curator-framework-2.7.1.jar
27
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/curator-recipes-2.7.1.jar
28
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/disruptor-3.3.0.jar
29
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/findbugs-annotations-1.3.9-1.jar
30
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/guava-11.0.2.jar
31
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/guava-12.0.1.jar
32
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/guice-3.0.jar
33
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-annotations-2.6.0-cdh5.16.1.jar
34
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-ant-2.6.0-cdh5.16.1.jar
35
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-archive-logs-2.6.0-cdh5.16.1.jar
36
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-archives-2.6.0-cdh5.16.1.jar
37
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-auth-2.6.0-cdh5.16.1.jar
38
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-aws-2.6.0-cdh5.16.1.jar
39
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-azure-2.6.0-cdh5.16.1.jar
40
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-azure-datalake-2.6.0-cdh5.16.1.jar
41
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-common-2.6.0-cdh5.16.1.jar
42
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-datajoin-2.6.0-cdh5.16.1.jar
43
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-distcp-2.6.0-cdh5.16.1.jar
44
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-extras-2.6.0-cdh5.16.1.jar
45
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-gridmix-2.6.0-cdh5.16.1.jar
46
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-hdfs-2.6.0-cdh5.16.1.jar
47
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-hdfs-nfs-2.6.0-cdh5.16.1.jar
48
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-app-2.6.0-cdh5.16.1.jar
49
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-common-2.6.0-cdh5.16.1.jar
50
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-core-2.6.0-cdh5.16.1.jar
51
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-hs-2.6.0-cdh5.16.1.jar
52
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.16.1.jar
53
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.16.1.jar
54
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-nativetask-2.6.0-cdh5.16.1.jar
55
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.16.1.jar
56
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-mapreduce-examples-2.6.0-cdh5.16.1.jar
57
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-nfs-2.6.0-cdh5.16.1.jar
58
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-openstack-2.6.0-cdh5.16.1.jar
59
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-rumen-2.6.0-cdh5.16.1.jar
60
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-sls-2.6.0-cdh5.16.1.jar
61
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-streaming-2.6.0-cdh5.16.1.jar
62
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-api-2.6.0-cdh5.16.1.jar
63
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.16.1.jar
64
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.16.1.jar
65
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-client-2.6.0-cdh5.16.1.jar
66
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-common-2.6.0-cdh5.16.1.jar
67
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-registry-2.6.0-cdh5.16.1.jar
68
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.16.1.jar
69
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-server-common-2.6.0-cdh5.16.1.jar
70
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-server-nodemanager-2.6.0-cdh5.16.1.jar
71
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.16.1.jar
72
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hadoop-yarn-server-web-proxy-2.6.0-cdh5.16.1.jar
73
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hamcrest-core-1.3.jar
74
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-annotations-1.2.0-cdh5.16.1.jar
75
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-client-1.2.0-cdh5.16.1.jar
76
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-common-1.2.0-cdh5.16.1.jar
77
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-examples-1.2.0-cdh5.16.1.jar
78
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-external-blockcache-1.2.0-cdh5.16.1.jar
79
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-hadoop-compat-1.2.0-cdh5.16.1.jar
80
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-hadoop2-compat-1.2.0-cdh5.16.1.jar
81
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-it-1.2.0-cdh5.16.1.jar
82
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-prefix-tree-1.2.0-cdh5.16.1.jar
83
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-procedure-1.2.0-cdh5.16.1.jar
84
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-protocol-1.2.0-cdh5.16.1.jar
85
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-resource-bundle-1.2.0-cdh5.16.1.jar
86
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-rest-1.2.0-cdh5.16.1.jar
87
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-rsgroup-1.2.0-cdh5.16.1.jar
88
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-server-1.2.0-cdh5.16.1.jar
89
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-shell-1.2.0-cdh5.16.1.jar
90
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hbase-thrift-1.2.0-cdh5.16.1.jar
91
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/high-scale-lib-1.1.1.jar
92
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hsqldb-1.8.0.10.jar
93
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/htrace-core-3.2.0-incubating.jar
94
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/htrace-core4-4.0.1-incubating.jar
95
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/httpclient-4.2.5.jar
96
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/httpcore-4.2.5.jar
97
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/hue-plugins-3.9.0-cdh5.16.1.jar
98
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-annotations-2.2.3.jar
99
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-core-2.2.3.jar
100
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-core-asl-1.8.10.jar
101
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-databind-2.2.3-cloudera.1.jar
102
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-jaxrs-1.8.10.jar
103
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-mapper-asl-1.8.10-cloudera.1.jar
104
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jackson-xc-1.8.10.jar
105
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jamon-runtime-2.4.1.jar
106
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jasper-compiler-5.5.23.jar
107
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jasper-runtime-5.5.23.jar
108
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/java-xmlbuilder-0.4.jar
109
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/javax.inject-1.jar
110
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jaxb-api-2.1.jar
111
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jaxb-api-2.2.2.jar
112
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jaxb-impl-2.2.3-1.jar
113
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jcodings-1.0.8.jar
114
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jets3t-0.9.0.jar
115
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jettison-1.1.jar
116
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jettison-1.3.3.jar
117
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jline-2.11.jar
118
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/joni-2.1.2.jar
119
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jruby-cloudera-1.0.0.jar
120
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jsch-0.1.42.jar
121
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jsp-2.1-6.1.14.jar
122
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jsp-api-2.1-6.1.14.jar
123
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jsp-api-2.1.jar
124
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/jsr305-3.0.0.jar
125
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/leveldbjni-all-1.8.jar
126
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/log4j-1.2.16.jar
127
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/log4j-1.2.17.jar
128
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/metrics-core-2.2.0.jar
129
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/metrics-core-3.0.2.jar
130
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/microsoft-windowsazure-storage-sdk-0.6.0.jar
131
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/mockito-all-1.8.5.jar
132
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/netty-3.10.5.Final.jar
133
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/netty-all-4.0.50.Final.jar
134
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/okhttp-2.4.0.jar
135
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/okio-1.4.0.jar
136
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/paranamer-2.3.jar
137
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/protobuf-java-2.5.0.jar
138
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/slf4j-api-1.7.5.jar
139
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/slf4j-log4j12-1.7.5.jar
140
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/snappy-java-1.0.4.1.jar
141
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/spark-1.6.0-cdh5.16.1-yarn-shuffle.jar
142
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/spymemcached-2.11.6.jar
143
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/stax-api-1.0-2.jar
144
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/xercesImpl-2.9.1.jar
145
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/xml-apis-1.3.04.jar
146
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/xmlenc-0.52.jar
147
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/jars/zookeeper-3.4.5-cdh5.16.1.jar
148
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/LICENSE.txt
149
/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/NOTICE.txt
150
/usr/java/jdk1.8.0_131/lib/tools.jar
Copied!
At each execution of any Spark job (including the use of spark-submit) this list of jars above will automatically get loaded.

What is a JAR?

A jar file is essential a compressed list of classes and methods. It is important to note that when jar files are built they will typically have an associated version number.
Someone can look at the contents of a jar file by executing
1
jar -tvf phoenix-4.13.1-HBase-1.3-client.jar
Copied!
Or you can wrap the above in a for loop that will look at the contents of every jar that might contain a method you are looking for.
1
for i in `ls -1 *.jar`;do jar -tvf $i | grep -i htrace/trace;echo $i;done;
Copied!

Common issues that can occur with CLASSPATH's

Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:218) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:850) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:635)
When a situation like this occurs it means that a method cannot be found in the classpath for the job that is trying to execute. This can indicate a couple things:
    1.
    The job cannot find a jar file that contains the method flagged (in the example above the org/apache/htrace/Trace method)
    2.
    Sometimes different versions of the same jar file gets loaded and the first jar loaded will always win. Older jars that get loaded first may not have a method defined in new jars.
At owl we have solved CLASSPATH / CLASSLOAD issues by automatically injecting jars defined in our owl/libs directory, and allowing users the ability to simply toggle loading them or not.
Last modified 2yr ago