See
<https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/969/display/redirect?page=changes>
Changes:
[github] Fix invalid formatting specifier in verify
[iemejia] [BEAM-9820] Upgrade Flink 1.9.x to 1.9.3
[github] [BEAM-9819] Relaxes the version range for httplib2 to <0.16.0. (#11522)
------------------------------------------
[...truncated 3.10 MB...]
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/e1ec3271-b1dd-472a-96ad-d143e4370d15,
shard=0, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@6c7fe5b,
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/counts-00000-of-00004
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/51ff37db-eae2-4518-8abd-a3fd3bd61a21,
shard=2, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@6c7fe5b,
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/counts-00002-of-00004
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/0221b55e-619e-4911-a204-536a1c5828f9,
shard=1, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@6c7fe5b,
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/counts-00001-of-00004
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
moveToOutputFiles
INFO: Will copy temporary file
FileResult{tempFilename=/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/9ee2133a-8847-4ad3-8a6a-1e7b4a2d9762,
shard=3, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@6c7fe5b,
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0,
onTimeIndex=0}} to final location
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/counts-00003-of-00004
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/9ee2133a-8847-4ad3-8a6a-1e7b4a2d9762
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/51ff37db-eae2-4518-8abd-a3fd3bd61a21
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/0221b55e-619e-4911-a204-536a1c5828f9
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
INFO: Will remove known temporary file
/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/e1ec3271-b1dd-472a-96ad-d143e4370d15
Apr 27, 2020 11:16:44 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation
removeTemporaryFiles
WARNING: Failed to match temporary files under:
[/tmp/groovy-generated-2635117893179012245-tmpdir/word-count-beam/.temp-beam-3f160d3d-9740-498d-be2d-60a0fbf446ff/].
grep Foundation counts*
counts-00002-of-00004:Foundation: 1
Verified Foundation: 1
[SUCCESS]
> Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-runners-google-cloud-dataflow-java/2.22.0-SNAPSHOT/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-g7Fm3jEiCx-r6gPYIfJa1A.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-runners-core-construction-java/2.22.0-SNAPSHOT/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TZNyJHNBF6P_-sED2Xf1tg.jar
Apr 27, 2020 11:16:45 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 191 files cached, 11 files newly uploaded in 3
seconds
Apr 27, 2020 11:16:45 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ReadLines/Read as step s1
Apr 27, 2020 11:16:46 AM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern gs://apache-beam-samples/shakespeare/* matched 44 files with
total size 5443510
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WordCount.CountWords/ParDo(ExtractWords) as step s2
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WordCount.CountWords/Count.PerElement/Init/Map as step s3
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey as step
s4
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues
as step s5
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding MapElements/Map as step s6
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign as step s7
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
as step s8
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step
s9
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step
s10
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum as step s11
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as
step s12
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
as step s13
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView as
step s14
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
as step s15
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
as step s16
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
as step s17
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize as step s18
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key as step s19
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s20
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s21
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s22
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
as step s23
Apr 27, 2020 11:16:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/
Apr 27, 2020 11:16:46 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <67452 bytes, hash 6YWteG29i60kWuUqkDL8HA> to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/pipeline-6YWteG29i60kWuUqkDL8HA.pb
Apr 27, 2020 11:16:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 11:16:47 AM
org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 403, performed 0 retries due to IOExceptions,
performed 0 retries due to unsuccessful status codes, HTTP framework says
request can be retried, (caller responsible for retrying):
https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/null/jobs.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
(default-cli) on project word-count-beam: An exception occured while executing
the Java class. Failed to create a workflow job: Permission denied on
'locations/null' (or it may not exist). 403 Forbidden
[ERROR] {
[ERROR] "code" : 403,
[ERROR] "errors" : [ {
[ERROR] "domain" : "global",
[ERROR] "message" : "Permission denied on 'locations/null' (or it may not
exist).",
[ERROR] "reason" : "forbidden"
[ERROR] } ],
[ERROR] "message" : "Permission denied on 'locations/null' (or it may not
exist).",
[ERROR] "status" : "PERMISSION_DENIED"
[ERROR] }
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Apr 27, 2020 11:16:39 AM
org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory
create
INFO: No stagingLocation provided, falling back to gcpTempLocation
Apr 27, 2020 11:16:40 AM org.apache.beam.runners.dataflow.DataflowRunner
fromOptions
INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from
the classpath: will stage 202 files. Enable logging at DEBUG level to see which
files will be staged.
Apr 27, 2020 11:16:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Apr 27, 2020 11:16:42 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 202 files from PipelineOptions.filesToStage to staging location
to prepare for execution.
Apr 27, 2020 11:16:43 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-model-pipeline/2.22.0-SNAPSHOT/beam-model-pipeline-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-model-pipeline-2.22.0-SNAPSHOT--401_8Sm69M8WiCXjqSTOA.jar
Apr 27, 2020 11:16:43 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-sdks-java-core/2.22.0-SNAPSHOT/beam-sdks-java-core-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-sdks-java-core-2.22.0-SNAPSHOT-TIqdeUwvOfcqkOdRrs8_Vg.jar
Apr 27, 2020 11:16:43 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/word-count-beam/target/classes
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/classes-qVkuAzQjIo-AQc-XfCKWEw.jar
Apr 27, 2020 11:16:43 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.22.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-sdks-java-io-google-cloud-platform-2.22.0-SNAPSHOT-vsDfpNCe1Xz3ti4odCnjrQ.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-sdks-java-expansion-service/2.22.0-SNAPSHOT/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-sdks-java-expansion-service-2.22.0-SNAPSHOT-iWJJKk8wD6u00-o7gtlV-w.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-model-job-management/2.22.0-SNAPSHOT/beam-model-job-management-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-model-job-management-2.22.0-SNAPSHOT-hMyA4QF5J0IVgcafkrplvw.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-sdks-java-extensions-google-cloud-platform-core/2.22.0-SNAPSHOT/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.22.0-SNAPSHOT-CS9dyJz3XcccTZ-Wx8YcNw.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-model-fn-execution/2.22.0-SNAPSHOT/beam-model-fn-execution-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-model-fn-execution-2.22.0-SNAPSHOT-hXm9hYnw4qxo2QtTiwjJSg.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-sdks-java-extensions-protobuf/2.22.0-SNAPSHOT/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-sdks-java-extensions-protobuf-2.22.0-SNAPSHOT-pWV7Dd5qRclxy7JYGB4HJg.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-runners-google-cloud-dataflow-java/2.22.0-SNAPSHOT/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-runners-google-cloud-dataflow-java-2.22.0-SNAPSHOT-g7Fm3jEiCx-r6gPYIfJa1A.jar
Apr 27, 2020 11:16:44 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
/tmp/groovy-generated-5238267072090216789-tmpdir/.m2/repository/org/apache/beam/beam-runners-core-construction-java/2.22.0-SNAPSHOT/beam-runners-core-construction-java-2.22.0-SNAPSHOT.jar
to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/beam-runners-core-construction-java-2.22.0-SNAPSHOT-TZNyJHNBF6P_-sED2Xf1tg.jar
Apr 27, 2020 11:16:45 AM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 191 files cached, 11 files newly uploaded in 3
seconds
Apr 27, 2020 11:16:45 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ReadLines/Read as step s1
Apr 27, 2020 11:16:46 AM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern gs://apache-beam-samples/shakespeare/* matched 44 files with
total size 5443510
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WordCount.CountWords/ParDo(ExtractWords) as step s2
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WordCount.CountWords/Count.PerElement/Init/Map as step s3
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey as step
s4
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues
as step s5
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding MapElements/Map as step s6
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign as step s7
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
as step s8
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step
s9
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step
s10
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum as step s11
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as
step s12
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
as step s13
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView as
step s14
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
as step s15
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
as step s16
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
as step s17
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize as step s18
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key as step s19
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
as step s20
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey
as step s21
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
as step s22
Apr 27, 2020 11:16:46 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
as step s23
Apr 27, 2020 11:16:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/
Apr 27, 2020 11:16:46 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <67452 bytes, hash 6YWteG29i60kWuUqkDL8HA> to
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/pipeline-6YWteG29i60kWuUqkDL8HA.pb
Apr 27, 2020 11:16:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.22.0-SNAPSHOT
Apr 27, 2020 11:16:47 AM
org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 403, performed 0 retries due to IOExceptions,
performed 0 retries due to unsuccessful status codes, HTTP framework says
request can be retried, (caller responsible for retrying):
https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/null/jobs.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java
(default-cli) on project word-count-beam: An exception occured while executing
the Java class. Failed to create a workflow job: Permission denied on
'locations/null' (or it may not exist). 403 Forbidden
[ERROR] {
[ERROR] "code" : 403,
[ERROR] "errors" : [ {
[ERROR] "domain" : "global",
[ERROR] "message" : "Permission denied on 'locations/null' (or it may not
exist).",
[ERROR] "reason" : "forbidden"
[ERROR] } ],
[ERROR] "message" : "Permission denied on 'locations/null' (or it may not
exist).",
[ERROR] "status" : "PERMISSION_DENIED"
[ERROR] }
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 16m 28s
8 actionable tasks: 7 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/ie4y6t7h3wp46
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
