[ https://issues.apache.org/jira/browse/HIVE-13696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283769#comment-15283769 ]
Hive QA commented on HIVE-13696: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12804034/HIVE-13696.11.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/292/testReport Console output: http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/292/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-292/ Messages: {noformat} **** This message was trimmed, see log for full details **** [INFO] Excluding org.apache.spark:spark-core_2.10:jar:1.6.0 from the shaded jar. [INFO] Excluding com.twitter:chill_2.10:jar:0.5.0 from the shaded jar. [INFO] Excluding com.twitter:chill-java:jar:0.5.0 from the shaded jar. [INFO] Excluding org.apache.xbean:xbean-asm5-shaded:jar:4.4 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-client:jar:2.6.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.6.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.6.0 from the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.6.0 from the shaded jar. [INFO] Excluding org.apache.spark:spark-launcher_2.10:jar:1.6.0 from the shaded jar. [INFO] Excluding org.apache.spark:spark-network-common_2.10:jar:1.6.0 from the shaded jar. [INFO] Excluding org.apache.spark:spark-network-shuffle_2.10:jar:1.6.0 from the shaded jar. [INFO] Excluding org.apache.spark:spark-unsafe_2.10:jar:1.6.0 from the shaded jar. [INFO] Excluding org.slf4j:jul-to-slf4j:jar:1.7.10 from the shaded jar. [INFO] Excluding org.slf4j:jcl-over-slf4j:jar:1.7.10 from the shaded jar. [INFO] Excluding com.ning:compress-lzf:jar:1.0.3 from the shaded jar. [INFO] Excluding net.jpountz.lz4:lz4:jar:1.3.0 from the shaded jar. [INFO] Excluding com.typesafe.akka:akka-remote_2.10:jar:2.3.11 from the shaded jar. [INFO] Excluding com.typesafe.akka:akka-actor_2.10:jar:2.3.11 from the shaded jar. [INFO] Excluding com.typesafe:config:jar:1.2.1 from the shaded jar. [INFO] Excluding org.uncommons.maths:uncommons-maths:jar:1.2.2a from the shaded jar. [INFO] Excluding com.typesafe.akka:akka-slf4j_2.10:jar:2.3.11 from the shaded jar. [INFO] Excluding org.scala-lang:scala-library:jar:2.10.4 from the shaded jar. [INFO] Excluding org.json4s:json4s-jackson_2.10:jar:3.2.10 from the shaded jar. [INFO] Excluding org.json4s:json4s-core_2.10:jar:3.2.10 from the shaded jar. [INFO] Excluding org.json4s:json4s-ast_2.10:jar:3.2.10 from the shaded jar. [INFO] Excluding org.scala-lang:scalap:jar:2.10.0 from the shaded jar. [INFO] Excluding org.scala-lang:scala-compiler:jar:2.10.0 from the shaded jar. [INFO] Excluding org.apache.mesos:mesos:jar:shaded-protobuf:0.21.1 from the shaded jar. [INFO] Excluding com.clearspring.analytics:stream:jar:2.7.0 from the shaded jar. [INFO] Excluding io.dropwizard.metrics:metrics-graphite:jar:3.1.2 from the shaded jar. [INFO] Excluding com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4 from the shaded jar. [INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded jar. [INFO] Excluding oro:oro:jar:2.0.8 from the shaded jar. [INFO] Excluding org.tachyonproject:tachyon-client:jar:0.8.2 from the shaded jar. [INFO] Excluding org.tachyonproject:tachyon-underfs-hdfs:jar:0.8.2 from the shaded jar. [INFO] Excluding org.tachyonproject:tachyon-underfs-s3:jar:0.8.2 from the shaded jar. [INFO] Excluding org.tachyonproject:tachyon-underfs-local:jar:0.8.2 from the shaded jar. [INFO] Excluding net.razorvine:pyrolite:jar:4.9 from the shaded jar. [INFO] Excluding net.sf.py4j:py4j:jar:0.9 from the shaded jar. [INFO] Excluding org.spark-project.spark:unused:jar:1.0.0 from the shaded jar. [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.10 from the shaded jar. [INFO] Replacing original artifact with shaded artifact. [INFO] Replacing /data/hive-ptest/working/apache-github-source-source/ql/target/hive-exec-2.1.0-SNAPSHOT.jar with /data/hive-ptest/working/apache-github-source-source/ql/target/hive-exec-2.1.0-SNAPSHOT-shaded.jar [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] Dependency-reduced POM written at: /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-exec --- [INFO] Installing /data/hive-ptest/working/apache-github-source-source/ql/target/hive-exec-2.1.0-SNAPSHOT.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-exec/2.1.0-SNAPSHOT/hive-exec-2.1.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/ql/dependency-reduced-pom.xml to /home/hiveptest/.m2/repository/org/apache/hive/hive-exec/2.1.0-SNAPSHOT/hive-exec-2.1.0-SNAPSHOT.pom [INFO] Installing /data/hive-ptest/working/apache-github-source-source/ql/target/hive-exec-2.1.0-SNAPSHOT-tests.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-exec/2.1.0-SNAPSHOT/hive-exec-2.1.0-SNAPSHOT-tests.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/ql/target/hive-exec-2.1.0-SNAPSHOT-core.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-exec/2.1.0-SNAPSHOT/hive-exec-2.1.0-SNAPSHOT-core.jar [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Llap Server 2.1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-llap-server --- [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/llap-server/target [INFO] Deleting /data/hive-ptest/working/apache-github-source-source/llap-server (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-llap-server --- [INFO] [INFO] --- build-helper-maven-plugin:1.8:add-source (add-source) @ hive-llap-server --- [INFO] Source directory: /data/hive-ptest/working/apache-github-source-source/llap-server/src/gen/protobuf/gen-java added. [INFO] Source directory: /data/hive-ptest/working/apache-github-source-source/llap-server/src/gen/thrift/gen-javabean added. [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-llap-server --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-llap-server --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 19 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-llap-server --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-llap-server --- [INFO] Compiling 87 source files to /data/hive-ptest/working/apache-github-source-source/llap-server/target/classes [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/cache/SimpleAllocator.java:[29,16] sun.misc.Cleaner is internal proprietary API and may be removed in a future release [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/cache/SimpleAllocator.java:[29,16] sun.misc.Cleaner is internal proprietary API and may be removed in a future release [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/cache/SimpleAllocator.java:[29,16] sun.misc.Cleaner is internal proprietary API and may be removed in a future release [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/cache/SimpleAllocator.java:[74,9] sun.misc.Cleaner is internal proprietary API and may be removed in a future release [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java: Some input files use or override a deprecated API. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/io/metadata/OrcFileMetadata.java: Recompile with -Xlint:deprecation for details. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/DirWatcher.java: Some input files use unchecked or unsafe operations. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/DirWatcher.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-llap-server --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 4 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-llap-server --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/llap-server/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/llap-server/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-github-source-source/llap-server/target/tmp/conf [copy] Copying 15 files to /data/hive-ptest/working/apache-github-source-source/llap-server/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-llap-server --- [INFO] Compiling 13 source files to /data/hive-ptest/working/apache-github-source-source/llap-server/target/test-classes [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java: Some input files use unchecked or unsafe operations. [WARNING] /data/hive-ptest/working/apache-github-source-source/llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestFirstInFirstOutComparator.java: Recompile with -Xlint:unchecked for details. [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-llap-server --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-llap-server --- [INFO] Building jar: /data/hive-ptest/working/apache-github-source-source/llap-server/target/hive-llap-server-2.1.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-llap-server --- [INFO] [INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-llap-server --- [INFO] Building jar: /data/hive-ptest/working/apache-github-source-source/llap-server/target/hive-llap-server-2.1.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-llap-server --- [INFO] Installing /data/hive-ptest/working/apache-github-source-source/llap-server/target/hive-llap-server-2.1.0-SNAPSHOT.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-llap-server/2.1.0-SNAPSHOT/hive-llap-server-2.1.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-github-source-source/llap-server/pom.xml to /home/hiveptest/.m2/repository/org/apache/hive/hive-llap-server/2.1.0-SNAPSHOT/hive-llap-server-2.1.0-SNAPSHOT.pom [INFO] Installing /data/hive-ptest/working/apache-github-source-source/llap-server/target/hive-llap-server-2.1.0-SNAPSHOT-tests.jar to /home/hiveptest/.m2/repository/org/apache/hive/hive-llap-server/2.1.0-SNAPSHOT/hive-llap-server-2.1.0-SNAPSHOT-tests.jar [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Service 2.1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: http://repository.apache.org/snapshots/org/apache/directory/client/ldap/ldap-client-api/0.1-SNAPSHOT/maven-metadata.xml [WARNING] Could not transfer metadata org.apache.directory.client.ldap:ldap-client-api:0.1-SNAPSHOT/maven-metadata.xml from/to apache.snapshots (http://repository.apache.org/snapshots): Failed to transfer file: http://repository.apache.org/snapshots/org/apache/directory/client/ldap/ldap-client-api/0.1-SNAPSHOT/maven-metadata.xml. Return code is: 503 , ReasonPhrase:Service Unavailable. [WARNING] Failure to transfer org.apache.directory.client.ldap:ldap-client-api:0.1-SNAPSHOT/maven-metadata.xml from http://repository.apache.org/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots has elapsed or updates are forced. Original error: Could not transfer metadata org.apache.directory.client.ldap:ldap-client-api:0.1-SNAPSHOT/maven-metadata.xml from/to apache.snapshots (http://repository.apache.org/snapshots): Failed to transfer file: http://repository.apache.org/snapshots/org/apache/directory/client/ldap/ldap-client-api/0.1-SNAPSHOT/maven-metadata.xml. Return code is: 503 , ReasonPhrase:Service Unavailable. Downloading: http://repository.apache.org/snapshots/org/apache/directory/client/ldap/ldap-client-api/0.1-SNAPSHOT/ldap-client-api-0.1-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hive .............................................. SUCCESS [2.825s] [INFO] Hive Shims Common ................................. SUCCESS [4.572s] [INFO] Hive Shims 0.23 ................................... SUCCESS [3.230s] [INFO] Hive Shims Scheduler .............................. SUCCESS [0.885s] [INFO] Hive Shims ........................................ SUCCESS [0.528s] [INFO] Hive Storage API .................................. SUCCESS [1.261s] [INFO] Hive ORC .......................................... SUCCESS [4.352s] [INFO] Hive Common ....................................... SUCCESS [4.846s] [INFO] Hive Service RPC .................................. SUCCESS [2.770s] [INFO] Hive Serde ........................................ SUCCESS [3.847s] [INFO] Hive Metastore .................................... SUCCESS [15.602s] [INFO] Hive Ant Utilities ................................ SUCCESS [0.552s] [INFO] Hive Llap Common .................................. SUCCESS [4.670s] [INFO] Hive Llap Client .................................. SUCCESS [1.518s] [INFO] Hive Llap Tez ..................................... SUCCESS [1.300s] [INFO] Spark Remote Client ............................... SUCCESS [2.410s] [INFO] Hive Query Language ............................... SUCCESS [51.941s] [INFO] Hive Llap Server .................................. SUCCESS [2.774s] [INFO] Hive Service ...................................... FAILURE [9.452s] [INFO] Hive Accumulo Handler ............................. SKIPPED [INFO] Hive JDBC ......................................... SKIPPED [INFO] Hive Beeline ...................................... SKIPPED [INFO] Hive CLI .......................................... SKIPPED [INFO] Hive Contrib ...................................... SKIPPED [INFO] Hive HBase Handler ................................ SKIPPED [INFO] Hive HCatalog ..................................... SKIPPED [INFO] Hive HCatalog Core ................................ SKIPPED [INFO] Hive HCatalog Pig Adapter ......................... SKIPPED [INFO] Hive HCatalog Server Extensions ................... SKIPPED [INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED [INFO] Hive HCatalog Webhcat ............................. SKIPPED [INFO] Hive HCatalog Streaming ........................... SKIPPED [INFO] Hive HPL/SQL ...................................... SKIPPED [INFO] Hive HWI .......................................... SKIPPED [INFO] Hive Llap External Client ......................... SKIPPED [INFO] Hive Shims Aggregator ............................. SKIPPED [INFO] Hive TestUtils .................................... SKIPPED [INFO] Hive Packaging .................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2:00.409s [INFO] Finished at: Sun May 15 02:15:56 GMT 2016 [INFO] Final Memory: 159M/1349M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "hadoop-1" could not be activated because it does not exist. [ERROR] Failed to execute goal on project hive-service: Could not resolve dependencies for project org.apache.hive:hive-service:jar:2.1.0-SNAPSHOT: Failed to collect dependencies for [org.apache.hive:hive-exec:jar:2.1.0-SNAPSHOT (compile), org.apache.hive:hive-metastore:jar:2.1.0-SNAPSHOT (compile), org.apache.hive:hive-service-rpc:jar:2.1.0-SNAPSHOT (compile), org.apache.hive:hive-llap-server:jar:2.1.0-SNAPSHOT (compile), commons-codec:commons-codec:jar:1.4 (compile), commons-cli:commons-cli:jar:1.2 (compile), net.sf.jpam:jpam:jar:1.1 (compile), commons-lang:commons-lang:jar:2.6 (compile), org.eclipse.jetty.aggregate:jetty-all:jar:7.6.0.v20120127 (compile), tomcat:jasper-compiler:jar:5.5.23 (compile), tomcat:jasper-runtime:jar:5.5.23 (compile), org.apache.thrift:libfb303:jar:0.9.3 (compile), org.apache.thrift:libthrift:jar:0.9.3 (compile), org.apache.curator:curator-framework:jar:2.6.0 (compile), org.apache.curator:curator-recipes:jar:2.6.0 (compile), org.apache.hadoop:hadoop-common:jar:2.6.0 (compile?), org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.6.0 (compile?), org.jamon:jamon-runtime:jar:2.3.1 (compile), org.apache.hive:hive-exec:jar:tests:2.1.0-SNAPSHOT (test), org.apache.hive:hive-common:jar:tests:2.1.0-SNAPSHOT (test), junit:junit:jar:4.11 (test), org.apache.directory.client.ldap:ldap-client-api:jar:0.1 (test), org.apache.directory.server:apacheds-server-integ:jar:1.5.6 (test), org.apache.directory.server:apacheds-test-framework:jar:1.5.6 (test), org.slf4j:slf4j-api:jar:1.7.10 (compile)]: Failed to read artifact descriptor for org.apache.directory.client.ldap:ldap-client-api:jar:0.1-SNAPSHOT: Could not transfer artifact org.apache.directory.client.ldap:ldap-client-api:pom:0.1-SNAPSHOT from/to apache.snapshots (http://repository.apache.org/snapshots): Failed to transfer file: http://repository.apache.org/snapshots/org/apache/directory/client/ldap/ldap-client-api/0.1-SNAPSHOT/ldap-client-api-0.1-SNAPSHOT.pom. Return code is: 503 , ReasonPhrase:Service Unavailable. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-service + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12804034 - PreCommit-HIVE-MASTER-Build > Monitor fair-scheduler.xml and automatically update/validate jobs submitted > to fair-scheduler > --------------------------------------------------------------------------------------------- > > Key: HIVE-13696 > URL: https://issues.apache.org/jira/browse/HIVE-13696 > Project: Hive > Issue Type: Improvement > Reporter: Reuben Kuhnert > Assignee: Reuben Kuhnert > Attachments: HIVE-13696.01.patch, HIVE-13696.02.patch, > HIVE-13696.06.patch, HIVE-13696.08.patch, HIVE-13696.11.patch > > > Ensure that jobs are placed into the correct queue according to > {{fair-scheduler.xml}}. Jobs should be placed into the correct queue, and > users should not be able to submit jobs to queues they do not have access to. > This patch builds on the existing functionality in {{FairSchedulerShim}} to > route jobs to user-specific queue based on {{fair-scheduler.xml}} > configuration (leveraging the Yarn {{QueuePlacementPolicy}} class). In > addition to configuring job routing at session connect (current behavior), > the routing is validated per submission to yarn (when impersonation is off). > A {{FileSystemWatcher}} class is included to monitor changes in the > {{fair-scheduler.xml}} file (so updates are automatically reloaded when the > file pointed to by {{yarn.scheduler.fair.allocation.file}} is changed). -- This message was sent by Atlassian JIRA (v6.3.4#6332)