See <https://builds.apache.org/job/Tez-Build/1697/display/redirect?page=changes>
Changes: [jeagles] TEZ-3355. Tez Custom Shuffle Handler POC (jeagles) [jeagles] TEZ-3380. Shuffle Handler: Replace primitive wrapper's valueOf method [jeagles] TEZ-3377. Remove ShuffleHandler dependency on mapred.FadvisedChunkedFile [jeagles] TEZ-3334-CHANGES.txt fixup [jeagles] TEZ-3378. Move Shuffle Handler configuration into the Tez namespace [jeagles] TEZ-3393. Remove extra jetty dependency from Shuffle Handler (jeagles) [jeagles] TEZ-3390. Package Shuffle Handler as a shaded uber-jar (jeagles) [jeagles] TEZ-3238. TEZ-3238. Shuffle service name should be configureable and [jeagles] TEZ-3408. Allow Task Output Files to reside in DAG specific directories [jeagles] TEZ-3410. ShuffleHandler should use Path.SEPARATOR instead of '/' (Kuhu [jeagles] TEZ-3412. Modify ShuffleHandler to use Constants.DAG_PREFIX and fix [jeagles] TEZ-3411. TestShuffleHandler#testSendMapCount should not used hard coded [jeagles] TEZ-3360. Tez Custom Shuffle Handler Documentation (jeagles) [jeagles] TEZ-3362. Delete intermediate data at DAG level for Shuffle Handler [jeagles] TEZ-3480. Port MAPREDUCE-6763 to Tez ShuffleHandler (Kuhu Shukla via [jeagles] TEZ-3509. Make DAG Deletion path based (Kuhu Shukla via jeagles) [jeagles] TEZ-3509. Make DAG Deletion path based (Kuhu Shukla via jeagles) [jeagles] TEZ-3361. Fetch Multiple Partitions from the Shuffle Handler (jeagles) [jeagles] TEZ-3557. TEZ-3362 causes TestContainerLauncherWrapper#testDelegation to [jeagles] TEZ-3564. TezConfiguration#TEZ_DELETION_TRACKER_CLASS has the wrong [jeagles] TEZ-3563. Tez Shuffle Handler logging fails to initialize (jeagles) [jeagles] TEZ-3532. Backport MAPREDUCE-6808. Log map attempts as part of shuffle [jeagles] TEZ-3586. Remove fusesource.leveldbjni from the tez-auxservices shaded [jeagles] TEZ-3587. Fetcher fetchInputs() can NPE on srcAttempt due to missing [jeagles] TEZ-3590. Remove google.protobuf from the tez-auxservices shaded jar [jeagles] TEZ-3595. Composite Fetch account error for disk direct (jeagles) [jeagles] TEZ-3597. Composite Fetch hangs on certain DME empty events. (jeagles) [jeagles] TEZ-3596. Number of Empty DME logged for Composite fetch is too high [jeagles] TEZ-3599. Unordered Fetcher can hang if empty partitions are present [jeagles] TEZ-3604. Remove the compositeInputAttemptIdentifier from remaining list [jeagles] TEZ-3606. Fix debug log for empty partitions to the expanded partitionId [jeagles] TEZ-3608. Fetcher can hang if copyMapOutput/fetchInputs returns early [jeagles] TEZ-3612. Tez Shuffle Handler Content length does not match actual [jeagles] TEZ-3618. Shuffle Handler Loading cache equality tests always results is [jeagles] TEZ-3620. UnorderedPartitionedKVOutput is missing the shuffle service [jeagles] TEZ-3621. Optimize the Shuffle Handler content length calculation for [jeagles] TEZ-3628. Give Tez shuffle handler threads custom names (jeagles) [jeagles] Preparing TEZ-3334 for mergo to trunk [jeagles] TEZ-3682. Pass parameters instead of configuration for changes to [jeagles] TEZ-3683. LocalContainerLauncher#shouldDelete member variable is not [jeagles] TEZ-3684. Incorporate first pass non-essential TEZ-3334 pre-merge [jeagles] TEZ-3685. ShuffleHandler completedInputSet off-by-one error (jeagles) [jlowe] TEZ-3702. Tez shuffle jar includes service loader entry for [jeagles] TEZ-3705. Modify DeletionTracker and deletion threads to be initialized [kshukla] TEZ-3725. Cleanup http connections and other unnecessary fields in DAG [kshukla] TEZ-3726. Clean up DeletionTracker's reflection instantiation and [jeagles] TEZ-3334 merge conflict [jeagles] TEZ-3735. Test failures in TestTaskAttempt and TestAMContainerMap (Kuhu [kshukla] TEZ-3740. Clean up TEZ-3334-CHANGES.txt (Jonathan Eagles via kshukla) ------------------------------------------ [...truncated 393.94 KB...] [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665765898 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ hadoop-shim-impls --- [INFO] [INFO] --- maven-jar-plugin:2.4:test-jar (default) @ hadoop-shim-impls --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building hadoop-shim-2.7 0.9.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ hadoop-shim-2.7 --- [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.7 --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665765954 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ hadoop-shim-2.7 --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-shim-2.7 --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 2 source files to <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/classes> [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/src/test/resources> [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-shim-2.7 --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 1 source file to <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/test-classes> [INFO] [INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ hadoop-shim-2.7 --- [INFO] Surefire report directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/surefire-reports> ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.tez.hadoop.shim.TestHadoop25_26_27ShimProvider Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.228 sec Results : Tests run: 4, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.7 --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665766577 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ hadoop-shim-2.7 --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-shim-2.7 --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/src/test/resources> [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-shim-2.7 --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ hadoop-shim-2.7 --- [INFO] Skipping execution of surefire because it has already been run for this configuration [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ hadoop-shim-2.7 --- [INFO] Building jar: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/hadoop-shim-2.7-0.9.0-SNAPSHOT.jar> [INFO] [INFO] --- maven-jar-plugin:2.4:test-jar (default) @ hadoop-shim-2.7 --- [INFO] Building jar: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/hadoop-shim-2.7-0.9.0-SNAPSHOT-tests.jar> [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hadoop-shim-2.7 --- [INFO] Installing <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/hadoop-shim-2.7-0.9.0-SNAPSHOT.jar> to /home/jenkins/.m2/repository/org/apache/tez/hadoop-shim-2.7/0.9.0-SNAPSHOT/hadoop-shim-2.7-0.9.0-SNAPSHOT.jar [INFO] Installing <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/pom.xml> to /home/jenkins/.m2/repository/org/apache/tez/hadoop-shim-2.7/0.9.0-SNAPSHOT/hadoop-shim-2.7-0.9.0-SNAPSHOT.pom [INFO] Installing <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/target/hadoop-shim-2.7-0.9.0-SNAPSHOT-tests.jar> to /home/jenkins/.m2/repository/org/apache/tez/hadoop-shim-2.7/0.9.0-SNAPSHOT/hadoop-shim-2.7-0.9.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-deploy-plugin:2.8.1:deploy (default-deploy) @ hadoop-shim-2.7 --- [INFO] Deploying org.apache.tez:hadoop-shim-2.7:0.9.0-SNAPSHOT at end [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.7 --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665766640 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ hadoop-shim-2.7 --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-shim-2.7 --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-shim-2.7 --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory <https://builds.apache.org/job/Tez-Build/ws/hadoop-shim-impls/hadoop-shim-2.7/src/test/resources> [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-shim-2.7 --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ hadoop-shim-2.7 --- [INFO] Skipping execution of surefire because it has already been run for this configuration [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ hadoop-shim-2.7 --- [INFO] [INFO] --- maven-jar-plugin:2.4:test-jar (default) @ hadoop-shim-2.7 --- [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Skipping tez-dist [INFO] This project has been banned from the build due to previous failures. [INFO] ------------------------------------------------------------------------ [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Tez 0.9.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ tez-docs --- [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-docs --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665766682 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ tez-docs --- [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-docs --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665766692 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ tez-docs --- [INFO] [INFO] --- maven-jar-plugin:2.4:test-jar (default) @ tez-docs --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] Building jar: <https://builds.apache.org/job/Tez-Build/ws/docs/target/tez-docs-0.9.0-SNAPSHOT-tests.jar> [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ tez-docs --- [INFO] Installing <https://builds.apache.org/job/Tez-Build/ws/docs/pom.xml> to /home/jenkins/.m2/repository/org/apache/tez/tez-docs/0.9.0-SNAPSHOT/tez-docs-0.9.0-SNAPSHOT.pom [INFO] Installing <https://builds.apache.org/job/Tez-Build/ws/docs/target/tez-docs-0.9.0-SNAPSHOT-tests.jar> to /home/jenkins/.m2/repository/org/apache/tez/tez-docs/0.9.0-SNAPSHOT/tez-docs-0.9.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-deploy-plugin:2.8.1:deploy (default-deploy) @ tez-docs --- [INFO] Skipping artifact deployment [INFO] [INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-docs --- [INFO] Checking for local modifications: skipped. [INFO] Updating project files from SCM: skipped. [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildNumber: a991f17e423b1dcfa098f50942ffa4e5a9c83ee4 at timestamp: 1495665766724 [INFO] Executing: /bin/sh -c cd <https://builds.apache.org/job/Tez-Build/ws/docs> && git rev-parse --verify HEAD [INFO] Working directory: <https://builds.apache.org/job/Tez-Build/ws/docs> [INFO] Storing buildScmBranch: UNKNOWN [INFO] [INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ tez-docs --- [INFO] [INFO] --- maven-jar-plugin:2.4:test-jar (default) @ tez-docs --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] tez ................................................ SUCCESS [ 1.985 s] [INFO] hadoop-shim ........................................ SUCCESS [ 2.796 s] [INFO] tez-api ............................................ SUCCESS [01:31 min] [INFO] tez-common ......................................... SUCCESS [ 6.802 s] [INFO] tez-runtime-internals .............................. SUCCESS [ 24.948 s] [INFO] tez-runtime-library ................................ FAILURE [03:14 min] [INFO] tez-mapreduce ...................................... SKIPPED [INFO] tez-examples ....................................... SKIPPED [INFO] tez-dag ............................................ SKIPPED [INFO] tez-tests .......................................... SKIPPED [INFO] tez-ext-service-tests .............................. SKIPPED [INFO] tez-ui ............................................. SUCCESS [05:27 min] [INFO] tez-plugins ........................................ SUCCESS [ 0.068 s] [INFO] tez-yarn-timeline-history .......................... SKIPPED [INFO] tez-yarn-timeline-history-with-acls ................ SKIPPED [INFO] tez-history-parser ................................. SKIPPED [INFO] tez-aux-services ................................... SKIPPED [INFO] tez-tools .......................................... SUCCESS [ 0.066 s] [INFO] tez-perf-analyzer .................................. SUCCESS [ 0.115 s] [INFO] tez-job-analyzer ................................... SKIPPED [INFO] tez-javadoc-tools .................................. SUCCESS [ 0.302 s] [INFO] hadoop-shim-impls .................................. SUCCESS [ 0.068 s] [INFO] hadoop-shim-2.7 .................................... SUCCESS [ 0.761 s] [INFO] tez-dist ........................................... SKIPPED [INFO] Tez ................................................ SUCCESS [ 0.061 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 10:51 min [INFO] Finished at: 2017-05-24T22:42:46+00:00 [INFO] Final Memory: 72M/1049M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14.1:test (default-test) on project tez-runtime-library: There are test failures. [ERROR] [ERROR] Please refer to <https://builds.apache.org/job/Tez-Build/ws/tez-runtime-library/target/surefire-reports> for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :tez-runtime-library Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results
