See
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/3866/display/redirect?page=changes>
Changes:
[Pablo Estrada] Adding display data to BQ File Loads transform
[sychen] Fix GroupIntoBathces.test_buffering_timer_in_fixed_window_streaming
[noreply] [BEAM-3736] Add CombineFn.setup and CombineFn.teardown to Python SDK
[noreply] [BEAM-11190] Fix grouping on categorical columns (#13256)
------------------------------------------
[...truncated 245.81 KB...]
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/xml/build/libs/beam-sdks-java-io-xml-2.26.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-xml-2.26.0-SNAPSHOT-tests-5BX-hind5tX4_cHq7a5Kw12IBdXkwFb9-_SfjsLbuQE.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT-JkJOM7wzyYaxbnuiZw5Kbo8S33pOml1HQ2LJuQZkXVI.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/hadoop-common/build/libs/beam-sdks-java-io-hadoop-common-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-hadoop-common-2.26.0-SNAPSHOT-w_3faoDyM4NM1-sF-LKOg1T4NT5lRE7UDlGQP81_IYI.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-core-construction-java-2.26.0-SNAPSHOT-kTU-KpFgDSgzDSez2sKX8ctgjbe-Js0Ms0gqyzEI-Mw.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT-gcsP8EcyYdo4vIPcYErUYhI3SRcME43KF5jcB5EODDc.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-tests-QMOSehchFoQJpRgX_kpR87Hl4-maxGwusDl4Z32Lryk.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-ySmkJHFsM__HZPrqBo3UTmQB_fe8004a4lcae_N9P_I.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT-tests.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-tests-es9R4wTfbgnSk1SxWg_Zv7SV2gcbowaDiy46ogCw7oo.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-runners-google-cloud-dataflow-java-legacy-****-2.26.0-SNAPSHOT-fH2h8iIrbYgSYZocznM279yVa7NhdHnd2Iz3mxzBb1k.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-fn-execution-2.26.0-SNAPSHOT-jikaYvhZUj_evBpbDGAmag8kzpWacYJZ6clgaxs0hSU.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/model/job-management/build/libs/beam-model-job-management-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-job-management-2.26.0-SNAPSHOT-oV0BLVhbvOAyPtppBZjKBfCfhUSARevW3ezCg0X8q9U.jar
Nov 05, 2020 12:09:41 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.26.0-SNAPSHOT.jar>
to
gs://dataflow-staging-us-central1-844138762903/temp/staging/beam-model-pipeline-2.26.0-SNAPSHOT-5sEDPDS8gRQPx1FaiZL-OFE12AGRUAsm4uI4DquZohM.jar
Nov 05, 2020 12:09:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 254 files cached, 29 files newly uploaded in
3 seconds
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Create xml records/Map as step s2
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Gather write start time as step s3
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step
s4
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s5
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s6
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum as step s7
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step
s8
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml files/WriteFiles/GatherTempFileResults/Add void
key/AddKeys/Map as step s9
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign as
step s10
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey as step s11
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable as step s12
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml files/WriteFiles/GatherTempFileResults/Drop
key/Values/Map as step s13
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml files/WriteFiles/GatherTempFileResults/Gather
bundles as step s14
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key as step s15
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s16
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s17
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s18
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map
as step s19
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml files/WriteFiles/FinalizeTempFileBundles/Finalize as
step s20
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key as step s21
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s22
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s23
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s24
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
as step s25
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Gather write end time as step s26
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Get file names/Values/Map as step s27
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find files/Match filepatterns as step s28
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find files/Reshuffle.ViaRandomKey/Pair with random key as step
s29
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s30
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step
s31
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as
step s32
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Find files/Reshuffle.ViaRandomKey/Values/Values/Map as step s33
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read matched files/ParDo(ToReadableFile) as step s34
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Gather read start time as step s35
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml files/ReadAllViaFileBasedSource/Split into ranges as
step s36
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml files/ReadAllViaFileBasedSource/Reshuffle/Pair with
random key as step s37
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml
files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
as step s38
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml
files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey as step s39
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml
files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/ExpandIterable as step s40
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml
files/ReadAllViaFileBasedSource/Reshuffle/Values/Values/Map as step s41
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read xml files/ReadAllViaFileBasedSource/Read ranges as step
s42
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Gather read end time as step s43
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Map xml records to strings/Map as step s44
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s45
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s46
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s47
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s48
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) as step s49
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/View.AsIterable/CreateDataflowView as step
s50
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/CreateVoid/Read(CreateSource) as step s51
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/ProduceDefault as step s52
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Flatten.PCollections as step s53
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step
s54
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s55
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s56
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s57
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as
step s58
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s59
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s60
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s61
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s62
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s63
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s64
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s65
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
as step s66
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey
as step s67
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
as step s68
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
as step s69
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
as step s70
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly
as step s71
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
as step s72
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView as step s73
Nov 05, 2020 12:09:44 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Delete test files as step s74
Nov 05, 2020 12:09:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://dataflow-staging-us-central1-844138762903/temp/staging/
Nov 05, 2020 12:09:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <300733 bytes, hash
350e0f418a92775413f6f5c09f1af6e6576ba3c0c0a8395da1be7cc2c10e5887> to
gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-NQ4PQYqSd1QT9vXAnxr25ldro8DAqDldob58wsEOWIc.pb
Nov 05, 2020 12:09:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.26.0-SNAPSHOT
Nov 05, 2020 12:09:48 AM
org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 403, performed 0 retries due to
IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP
framework says request can be retried, (caller responsible for retrying):
https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.xml.XmlIOIT > writeThenReadAll FAILED
java.lang.RuntimeException: Failed to create a workflow job:
(5e09e1c4bf677965): The workflow could not be created. Causes:
(5e09e1c4bf677c6e): Unable to get machine type information for machine type
n1-standard-1 in zone us-central1-f because of insufficient permissions. Please
refer to https://cloud.google.com/dataflow/access-control#creating_jobs and
make sure you have sufficient permissions.
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:1127)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:196)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:320)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:351)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:332)
at org.apache.beam.sdk.io.xml.XmlIOIT.writeThenReadAll(XmlIOIT.java:178)
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403
Forbidden
POST
https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "(5e09e1c4bf677965): The workflow could not be created.
Causes: (5e09e1c4bf677c6e): Unable to get machine type information for machine
type n1-standard-1 in zone us-central1-f because of insufficient permissions.
Please refer to https://cloud.google.com/dataflow/access-control#creating_jobs
and make sure you have sufficient permissions.",
"reason" : "forbidden"
} ],
"message" : "(5e09e1c4bf677965): The workflow could not be created.
Causes: (5e09e1c4bf677c6e): Unable to get machine type information for machine
type n1-standard-1 in zone us-central1-f because of insufficient permissions.
Please refer to https://cloud.google.com/dataflow/access-control#creating_jobs
and make sure you have sufficient permissions.",
"status" : "PERMISSION_DENIED"
}
at
com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:149)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:112)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:39)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:443)
at
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1108)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:541)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
at
org.apache.beam.runners.dataflow.DataflowClient.createJob(DataflowClient.java:62)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:1113)
... 5 more
1 test completed, 1 failed
Finished generating test XML results (0.024 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into:
<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for
':' Thread 9,5,main]) completed. Took 22.994 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1m 25s
92 actionable tasks: 55 executed, 37 from cache
Publishing build scan...
https://gradle.com/s/gskrbepkwp26o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
