Benjamin BONNET created ATLAS-1910: -------------------------------------- Summary: Classpath issue in import-hive.sh script Key: ATLAS-1910 URL: https://issues.apache.org/jira/browse/ATLAS-1910 Project: Atlas Issue Type: Bug Affects Versions: 0.8-incubating, trunk, 0.9-incubating Environment: HDP2.6 sandbox Reporter: Benjamin BONNET
Hi, trying to run Hive bridge, I get the following error : {{[root@sandbox hook-bin]# ./import-hive.sh /usr/hdp/2.6.0.3-8/hadoop/conf:/usr/hdp/2.6.0.3-8/hadoop/lib/*:/usr/hdp/2.6.0.3-8/hadoop/.//*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/./:/usr/hdp/2.6.0.3-8/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.3-8/hadoop-hdfs/.//*:/usr/hdp/2.6.0.3-8/hadoop-yarn/lib/*:/usr/hdp/2.6.0.3-8/hadoop-yarn/.//*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.3-8/hadoop-mapreduce/.//*::jdbc-mysql.jar:mysql-connector-java-5.1.17.jar:mysql-connector-java-5.1.37.jar:mysql-connector-java.jar:/usr/hdp/2.6.0.3-8/tez/*:/usr/hdp/2.6.0.3-8/tez/lib/*:/usr/hdp/2.6.0.3-8/tez/conf Using Hive configuration directory [/etc/hive/conf] Log file for import is /usr/hdp/current/atlas-client/logs/import-hive.log CP=:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-client-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-common-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-notification-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/atlas-typesystem-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/gson-2.5.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/guice-multibindings-4.1.0.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hdfs-model-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/hive-bridge-0.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-ast_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-core_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/json4s-native_2.11-3.2.11.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka_2.11-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/kafka-clients-0.10.1.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-compiler-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-library-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scalap-2.11.8.jar:/usr/hdp/current/atlas-client/hook/hive/atlas-hive-plugin-impl/scala-reflect-2.11.8.jar:/etc/hive/conf:/usr/hdp/current/atlas-client/../hive/lib/accumulo-core-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-fate-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-start-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/accumulo-trace-1.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/activation-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/ant-launcher-1.9.1.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-2.7.7.jar:/usr/hdp/current/atlas-client/../hive/lib/antlr-runtime-3.4.jar:/usr/hdp/current/atlas-client/../hive/lib/apache-log4j-extras-1.2.17.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-commons-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/asm-tree-3.1.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avatica-metrics-1.8.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/avro-1.7.5.jar:/usr/hdp/current/atlas-client/../hive/lib/bonecp-0.8.0.RELEASE.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-core-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/calcite-linq4j-1.2.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-cli-1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-codec-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-collections-3.2.2.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compiler-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-compress-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-dbcp-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-httpclient-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-io-2.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-lang-2.6.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-logging-1.1.3.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-math-2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-pool-1.5.4.jar:/usr/hdp/current/atlas-client/../hive/lib/commons-vfs2-2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-client-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-framework-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/curator-recipes-2.6.0.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-api-jdo-4.2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-core-4.1.6.jar:/usr/hdp/current/atlas-client/../hive/lib/datanucleus-rdbms-4.1.7.jar:/usr/hdp/current/atlas-client/../hive/lib/derby-10.10.2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/usr/hdp/current/atlas-client/../hive/lib/eigenbase-properties-1.1.5.jar:/usr/hdp/current/atlas-client/../hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:/usr/hdp/current/atlas-client/../hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/groovy-all-2.4.4.jar:/usr/hdp/current/atlas-client/../hive/lib/guava-14.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/HikariCP-2.5.1.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-accumulo-handler-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-accumulo-handler.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-ant-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-ant.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-beeline-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-beeline.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-cli-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-cli.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-common-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-common.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-contrib-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-contrib.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-exec-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-exec.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-hbase-handler-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-hbase-handler.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-hwi-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-hwi.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-jdbc-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-jdbc.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-metastore-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-metastore.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-serde-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-serde.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-service-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-service.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-0.20S-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-0.23-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-common-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-common.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-scheduler-1.2.1000.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/hive-shims-scheduler.jar:/usr/hdp/current/atlas-client/../hive/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/atlas-client/../hive/lib/httpclient-4.4.jar:/usr/hdp/current/atlas-client/../hive/lib/httpcore-4.4.jar:/usr/hdp/current/atlas-client/../hive/lib/ivy-2.4.0.jar:/usr/hdp/current/atlas-client/../hive/lib/jackson-annotations-2.4.0.jar:/usr/hdp/current/atlas-client/../hive/lib/jackson-core-2.4.2.jar:/usr/hdp/current/atlas-client/../hive/lib/jackson-databind-2.4.2.jar:/usr/hdp/current/atlas-client/../hive/lib/janino-2.7.6.jar:/usr/hdp/current/atlas-client/../hive/lib/javax.jdo-3.2.0-m3.jar:/usr/hdp/current/atlas-client/../hive/lib/jcommander-1.32.jar:/usr/hdp/current/atlas-client/../hive/lib/jdo-api-3.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/jetty-all-7.6.0.v20120127.jar:/usr/hdp/current/atlas-client/../hive/lib/jetty-all-server-7.6.0.v20120127.jar:/usr/hdp/current/atlas-client/../hive/lib/jline-2.12.jar:/usr/hdp/current/atlas-client/../hive/lib/joda-time-2.8.1.jar:/usr/hdp/current/atlas-client/../hive/lib/jpam-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/json-20090211.jar:/usr/hdp/current/atlas-client/../hive/lib/jsr305-3.0.0.jar:/usr/hdp/current/atlas-client/../hive/lib/jta-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/libfb303-0.9.3.jar:/usr/hdp/current/atlas-client/../hive/lib/libthrift-0.9.3.jar:/usr/hdp/current/atlas-client/../hive/lib/log4j-1.2.16.jar:/usr/hdp/current/atlas-client/../hive/lib/mail-1.4.1.jar:/usr/hdp/current/atlas-client/../hive/lib/maven-scm-api-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/maven-scm-provider-svn-commons-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/maven-scm-provider-svnexe-1.4.jar:/usr/hdp/current/atlas-client/../hive/lib/metrics-core-3.1.0.jar:/usr/hdp/current/atlas-client/../hive/lib/metrics-json-3.1.0.jar:/usr/hdp/current/atlas-client/../hive/lib/metrics-jvm-3.1.0.jar:/usr/hdp/current/atlas-client/../hive/lib/mysql-connector-java.jar:/usr/hdp/current/atlas-client/../hive/lib/netty-3.7.0.Final.jar:/usr/hdp/current/atlas-client/../hive/lib/ojdbc6.jar:/usr/hdp/current/atlas-client/../hive/lib/opencsv-2.3.jar:/usr/hdp/current/atlas-client/../hive/lib/oro-2.0.8.jar:/usr/hdp/current/atlas-client/../hive/lib/paranamer-2.3.jar:/usr/hdp/current/atlas-client/../hive/lib/parquet-hadoop-bundle-1.8.1.jar:/usr/hdp/current/atlas-client/../hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:/usr/hdp/current/atlas-client/../hive/lib/plexus-utils-1.5.6.jar:/usr/hdp/current/atlas-client/../hive/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/atlas-client/../hive/lib/ranger-hive-plugin-shim-0.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/ranger-plugin-classloader-0.7.0.2.6.0.3-8.jar:/usr/hdp/current/atlas-client/../hive/lib/regexp-1.3.jar:/usr/hdp/current/atlas-client/../hive/lib/servlet-api-2.5.jar:/usr/hdp/current/atlas-client/../hive/lib/snappy-java-1.0.5.jar:/usr/hdp/current/atlas-client/../hive/lib/ST4-4.0.4.jar:/usr/hdp/current/atlas-client/../hive/lib/stax-api-1.0.1.jar:/usr/hdp/current/atlas-client/../hive/lib/stringtemplate-3.2.1.jar:/usr/hdp/current/atlas-client/../hive/lib/super-csv-2.2.0.jar:/usr/hdp/current/atlas-client/../hive/lib/transaction-api-1.1.jar:/usr/hdp/current/atlas-client/../hive/lib/velocity-1.5.jar:/usr/hdp/current/atlas-client/../hive/lib/xz-1.0.jar:/usr/hdp/current/atlas-client/../hive/lib/zookeeper-3.4.6.2.6.0.3-8.jar::/hook/hive: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/PropertiesConfiguration at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:622) Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.PropertiesConfiguration at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 13 more Failed to import Hive Data Model!!!}} I am using a HDP 2.6 sandbox, but import-hive.sh is quite the same on Master. The problem comes from a side effect of HIVE_CLASSPATH computation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)