See
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/199/display/redirect?page=changes>
Changes:
[david.moravek] [BEAM-7561] HdfsFileSystems.match can match a directory.
[david.moravek] [BEAM-7561] Validate correct directory matching in
LocalFileSystemsTest
------------------------------------------
[...truncated 514.03 KB...]
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jackson/jackson-xc/1.9.13/e3480072bc95c202476ffa1de99ff7ee9149f29c/jackson-xc-1.9.13.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jackson-xc-1.9.13-SfanNbrjB0Xc9ezsJwkHIA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/log4j-1.2.17-BKQfCgaJhvD3NIXPUHwPQA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.directory.api/api-util/1.0.0-M20/a871abf060b3cf83fc6dc4d7e3d151fce50ac3cb/api-util-1.0.0-M20.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/api-util-1.0.0-M20-LFpnImZoggJL7N1kMBvkkg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.directory.server/apacheds-kerberos-codec/2.0.0-M15/1c16e4e477183641c5f0dd5cdecd27ec331bacb5/apacheds-kerberos-codec-2.0.0-M15.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/apacheds-kerberos-codec-2.0.0-M15-MRjiLqxE4VDDg98dQXdy9A.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.2.4/a60a5e993c98c864010053cb901b7eab25306568/gson-2.2.4.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/gson-2.2.4-L1T8JIB6TK1ylwEt2M6_PQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.auto.value/auto-value/1.4/6d1448fcd13074bd3658ef915022410b7c48343b/auto-value-1.4.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/auto-value-1.4-_7tYdqNvCowycFIDpBenFw.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.jcraft/jsch/0.1.42/a86104b0f2e0c0bab5b0df836065823a99b5e334/jsch-0.1.42.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jsch-0.1.42-dOqSBYAHe0wLURAagpKlKQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.zookeeper/zookeeper/3.4.6/1b2502e29da1ebaade2357cd1de35a855fa3755/zookeeper-3.4.6.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/zookeeper-3.4.6-fQHTF8cXJocliWz7gbGBUg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-common/2.7.3/9ffd994953fc4397830a1241a15c9121aebd9574/hadoop-common-2.7.3.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/hadoop-common-2.7.3-61HIhqWURE05eCWgDaK9Sw.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.sonatype.sisu.inject/cglib/2.2.1-v20090111/7ce5e983fd0e6c78346f4c9cbfa39d83049dda2/cglib-2.2.1-v20090111.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/cglib-2.2.1-v20090111-iK9ZMRZayL7KuEoVfJus4A.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.codehaus.jettison/jettison/1.1/1a01a2a1218fcf9faa2cc2a6ced025bdea687262/jettison-1.1.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jettison-1.1-_IDgqr1RbFRzkmLD1hgwOg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty/3.7.0.Final/7a8c35599c68c0bf383df74469aa3e03d9aca87/netty-3.7.0.Final.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/netty-3.7.0.Final-ye87LzfVgeUAqdbB78SraQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.2.4/3b7f38df6de5dd8b500e602ae8c2dd5ee446f883/httpcore-4.2.4.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/httpcore-4.2.4-bMuGIx2Ki5nFUbTd-Sbd0Q.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/xerces/xercesImpl/2.9.1/7bc7e49ddfe4fb5f193ed37ecc96c12292c8ceb6/xercesImpl-2.9.1.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/xercesImpl-2.9.1--Af4bX2dsl7b_HgqynyiqQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.inject/guice/3.0/9d84f15fe35e2c716a02979fb62f50a29f38aefa/guice-3.0.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/guice-3.0-yhx7o2aITPzSz7SNI5XEAA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/aopalliance/aopalliance/1.0/235ba8b489512805ac13a8f9ea77a1ca5ebe3e8/aopalliance-1.0.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/aopalliance-1.0-BBdwVOGA0J45mICO-gQBxw.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/jline/jline/0.9.94/99a18e9a44834afdebc467294e1138364c207402/jline-0.9.94.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jline-0.9.94-RiNclgJ3IG8A_iRxRDe8iQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.fusesource.leveldbjni/leveldbjni-all/1.8/707350a2eeb1fa2ed77a32ddb3893ed308e941db/leveldbjni-all-1.8.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/leveldbjni-all-1.8-aUTpvAPHk4ho5TyWcmrpFA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/asm/asm/3.1/c157def142714c544bdea2e6144645702adf7097/asm-3.1.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/asm-3.1-ubjS1Vb5RYqsjEY_1RH4bQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.tukaani/xz/1.5/9c64274b7dbb65288237216e3fae7877fd3f2bee/xz-1.5.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/xz-1.5-UQUOWVswjErsisMU9m4YvA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.jamesmurty.utils/java-xmlbuilder/0.4/ac5962e48cdee3a0a6e1f8e00fcb594747ac5aaf/java-xmlbuilder-0.4.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/java-xmlbuilder-0.4-D6R0ITpqAoLNkmT24N02WA.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.sun.xml.bind/jaxb-impl/2.2.3-1/56baae106392040a45a06d4a41099173425da1e6/jaxb-impl-2.2.3-1.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jaxb-impl-2.2.3-1-G2ief4fK8mFcD2pHgx0DQg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.htrace/htrace-core/3.1.0-incubating/f73606e7c9ede5802335c290bf47490ad6d51df3/htrace-core-3.1.0-incubating.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/htrace-core-3.1.0-incubating-xJpGYtaRoJ7tEOCjXdcymQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-hdfs/2.7.3/524d360c632c8cbe788b5411b00ed00770e52c3e/hadoop-hdfs-2.7.3.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/hadoop-hdfs-2.7.3-LZHu4msrmckte_sxLa7bfg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.14.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/beam-sdks-java-core-2.14.0-SNAPSHOT-95vJVNu6zDMyvE2UKHx7-g.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/xml-apis/xml-apis/1.3.04/90b215f48fe42776c8c7f6e3509ec54e84fd65ef/xml-apis-1.3.04.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/xml-apis-1.3.04-munCnkSX_DWj6t4ebdC76w.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/javax.xml.bind/jaxb-api/2.2.12/4c83805595b15acf41d71d49e3add7c0e85baaed/jaxb-api-2.2.12.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/jaxb-api-2.2.12-YiKXN-VwBR0qzkhZL699Tg.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/io.netty/netty-all/4.0.23.Final/294104aaf1781d6a56a07d561e792c5d0c95f45/netty-all-4.0.23.Final.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/netty-all-4.0.23.Final-RyWCbKe6NxPbZ0jujzkGww.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/it.unimi.dsi/fastutil/6.5.7/dec71174d0c8a20f355e5af8b59f25eb424c49d3/fastutil-6.5.7.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/fastutil-6.5.7-ke0rpg4VnGkMiiBmj4nUCQ.jar
Jun 19, 2019 12:40:45 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/home/jenkins/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-vendor-grpc-1_13_1/0.2/249a9a576fc6541d87972e2dc24cf11ea8129b8f/beam-vendor-grpc-1_13_1-0.2.jar
to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/beam-vendor-grpc-1_13_1-0.2-2fE94kZQ2ONeR3ea5r0PvA.jar
Jun 19, 2019 12:40:47 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 0 files cached, 361 files newly uploaded in 5
seconds
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read using Hadoop
InputFormat/Read(HadoopInputFormatBoundedSource) as step s1
Jun 19, 2019 12:40:47 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Collect read time as step s2
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Get values only/Values/Map as step s3
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Values as string as step s4
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s5
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s6
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s7
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s8
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) as step s9
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/View.AsIterable/CreateDataflowView as step
s10
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/CreateVoid/Read(CreateSource) as step s11
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/ProduceDefault as step s12
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Flatten.PCollections as step s13
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Window.Assign as step s14
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) as step
s15
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
as step s16
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign as step s17
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey as step s18
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map as
step s19
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/RewindowActuals/Window.Assign as step
s20
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map as step s21
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/RemoveActualsTriggering/Flatten.PCollections as step s22
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s23
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign as step
s24
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/RemoveDummyTriggering/Flatten.PCollections as step s25
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/FlattenDummyAndContents as step s26
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/NeverTrigger/Flatten.PCollections as
step s27
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupDummyAndContents as step s28
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s29
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s30
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s31
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s32
Jun 19, 2019 12:40:47 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s33
Jun 19, 2019 12:40:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/
Jun 19, 2019 12:40:48 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <74865 bytes, hash tpsPcrO0n8rH0HYEvmpwUQ> to
gs://temp-storage-for-perf-tests/hadoopformatioit0writeandreadusinghadoopformat-jenkins-0619124041-b9d7ecf6/output/results/staging/pipeline-tpsPcrO0n8rH0HYEvmpwUQ.pb
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_OUT
Dataflow SDK version: 2.14.0-SNAPSHOT
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_ERROR
Jun 19, 2019 12:40:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-19_05_40_48-4934196807129851263?project=apache-beam-testing
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_OUT
Submitted job: 2019-06-19_05_40_48-4934196807129851263
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT >
writeAndReadUsingHadoopFormat STANDARD_ERROR
Jun 19, 2019 12:40:49 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2019-06-19_05_40_48-4934196807129851263
Jun 19, 2019 12:40:49 PM
org.apache.beam.runners.dataflow.TestDataflowRunner run
INFO: Running Dataflow job 2019-06-19_05_40_48-4934196807129851263 with 1
expected assertions.
Jun 19, 2019 12:40:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:40:48.492Z: Autoscaling is enabled for job
2019-06-19_05_40_48-4934196807129851263. The number of workers will be between
1 and 1000.
Jun 19, 2019 12:40:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:40:48.530Z: Autoscaling was automatically enabled for
job 2019-06-19_05_40_48-4934196807129851263.
Jun 19, 2019 12:40:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2019-06-19T12:40:52.263Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
hadoopformatioit0writeandreadusinghadoopformat-jenkins-061-zrto. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Jun 19, 2019 12:40:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:40:52.851Z: Checking permissions granted to controller
Service Account.
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:01.701Z: Worker configuration: n1-standard-1 in
us-central1-a.
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.254Z: Expanding CoGroupByKey operations into
optimizable parts.
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.424Z: Expanding GroupByKey operations into
optimizable parts.
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.471Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.737Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.787Z: Elided trivial flatten
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.835Z: Elided trivial flatten
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.883Z: Elided trivial flatten
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.932Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:02.977Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$0/GroupGlobally/GroupDummyAndContents/Read
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.024Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.070Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.108Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.144Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.187Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.229Z: Unzipping flatten s26 for input
s21.org.apache.beam.sdk.values.PCollection.<init>:402#80783fbab681bb5c
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.277Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.325Z: Unzipping flatten s13 for input
s12.org.apache.beam.sdk.values.PCollection.<init>:402#c93a13e4ee1094aa
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.371Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Window.Into()/Window.Assign, through flatten Calculate
hashcode/Flatten.PCollections, into producer Calculate hashcode/ProduceDefault
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.408Z: Fusing consumer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign
Jun 19, 2019 12:41:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-19T12:41:03.457Z: Unzipping flatten s13-u44 for input
s14.output-c42
STDERR: Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2019-06-19 12:41:04,583 65febed2 MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-19 12:41:04,584 65febed2 MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2019-06-19 12:41:04,585 65febed2 MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/config-beam-performancetests-hadoopformat-199>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml>
--ignore-not-found
2019-06-19 12:41:05,158 65febed2 MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 903, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-19 12:41:05,159 65febed2 MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2019-06-19 12:41:05,160 65febed2 MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-06-19 12:41:05,160 65febed2 MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/runs/65febed2/pkb.log>
2019-06-19 12:41:05,160 65febed2 MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/runs/65febed2/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
