See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/13/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-4594] Support state continuation tokens over the Fn API.
------------------------------------------
[...truncated 8.07 MB...]
Dec 17, 2018 11:41:54 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$13/RunChecks as step s35
Dec 17, 2018 11:41:54 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$13/VerifyAssertions/ParDo(DefaultConclude) as step s36
Dec 17, 2018 11:41:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-end-to-end-tests/flattentest0testflattenpcollectionsemptythenpardo-jenkins-1217114146-94e803ec/output/results/staging/
Dec 17, 2018 11:41:55 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <77886 bytes, hash ANajbVmdsA1ipmYhfOqwIw> to
gs://temp-storage-for-end-to-end-tests/flattentest0testflattenpcollectionsemptythenpardo-jenkins-1217114146-94e803ec/output/results/staging/pipeline-ANajbVmdsA1ipmYhfOqwIw.pb
org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsEmptyThenParDo STANDARD_OUT
Dataflow SDK version: 2.10.0-SNAPSHOT
org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsEmptyThenParDo STANDARD_ERROR
Dec 17, 2018 11:41:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-12-17_03_41_55-17546666855132132476?project=apache-beam-testing
org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsEmptyThenParDo STANDARD_OUT
Submitted job: 2018-12-17_03_41_55-17546666855132132476
org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsEmptyThenParDo STANDARD_ERROR
Dec 17, 2018 11:41:56 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2018-12-17_03_41_55-17546666855132132476
Dec 17, 2018 11:41:56 AM
org.apache.beam.runners.dataflow.TestDataflowRunner run
INFO: Running Dataflow job 2018-12-17_03_41_55-17546666855132132476 with 0
expected assertions.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:41:55.503Z: Autoscaling is enabled for job
2018-12-17_03_41_55-17546666855132132476. The number of workers will be between
1 and 1000.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:41:55.545Z: Autoscaling was automatically enabled for
job 2018-12-17_03_41_55-17546666855132132476.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-12-17T11:41:58.344Z: The workflow name is not a valid Cloud
Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring
will be labeled with this modified job name:
flattentest0testflattenpcollectionsemptythenpardo-jenkins--zrqf. For the best
monitoring experience, please name your job with a valid Cloud Label. For
details, see:
https://cloud.google.com/compute/docs/labeling-resources#restrictions
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:41:58.681Z: Checking permissions granted to controller
Service Account.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:02.653Z: Worker configuration: n1-standard-1 in
us-central1-b.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:03.229Z: Expanding CollectionToSingleton operations
into optimizable parts.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:03.576Z: Expanding CoGroupByKey operations into
optimizable parts.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:03.997Z: Expanding GroupByKey operations into
optimizable parts.
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.123Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.180Z: Elided trivial flatten
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.224Z: Elided trivial flatten
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.272Z: Elided trivial flatten
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.319Z: Unzipping flatten s29 for input
s17.org.apache.beam.sdk.values.PCollection.<init>:402#a4f9f304fed667ee
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.372Z: Fusing unzipped copy of
PAssert$13/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$13/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$13/GroupGlobally/KeyForDummy/AddKeys/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.478Z: Fusing consumer
PAssert$13/VerifyAssertions/ParDo(DefaultConclude) into PAssert$13/RunChecks
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.526Z: Unzipping flatten s29-u40 for input
s31-reify-value18-c38
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.568Z: Fusing unzipped copy of
PAssert$13/GroupGlobally/GroupDummyAndContents/Write, through flatten s29-u40,
into producer PAssert$13/GroupGlobally/GroupDummyAndContents/Reify
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.612Z: Fusing consumer
PAssert$13/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$13/GroupGlobally/GroupDummyAndContents/Read
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.660Z: Fusing consumer
PAssert$13/GroupGlobally/Values/Values/Map into
PAssert$13/GroupGlobally/GroupDummyAndContents/GroupByWindow
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.708Z: Fusing consumer PAssert$13/GetPane/Map into
PAssert$13/GroupGlobally/ParDo(Concat)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.750Z: Fusing consumer PAssert$13/RunChecks into
PAssert$13/GetPane/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.795Z: Fusing consumer
PAssert$13/GroupGlobally/ParDo(Concat) into
PAssert$13/GroupGlobally/Values/Values/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.847Z: Fusing consumer
PAssert$13/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$13/GroupGlobally/WindowIntoDummy/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.899Z: Fusing consumer
PAssert$13/GroupGlobally/GroupDummyAndContents/Write into
PAssert$13/GroupGlobally/GroupDummyAndContents/Reify
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.950Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:04.986Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.028Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$13/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.069Z: Fusing consumer
PAssert$13/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.113Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.160Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.206Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.260Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.305Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.340Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.388Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.446Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.491Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.530Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.575Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.620Z: Fusing consumer
PAssert$13/GroupGlobally/Window.Into()/Window.Assign into ParDo(Identity)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.666Z: Fusing consumer
PAssert$13/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$13/GroupGlobally/GatherAllOutputs/Values/Values/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.709Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.758Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.806Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
Flatten.PCollections/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.848Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$13/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.894Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.946Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$13/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:05.990Z: Fusing consumer
PAssert$13/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$13/GroupGlobally/Window.Into()/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.036Z: Fusing consumer
PAssert$13/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$13/GroupGlobally/RewindowActuals/Window.Assign
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.082Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.123Z: Fusing consumer
Flatten.PCollections/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into Flatten.PCollections/Create.Values/Read(CreateSource)/Impulse
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.173Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
into PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Impulse
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.211Z: Fusing consumer
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random key into
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.257Z: Fusing consumer ParDo(Identity) into
Flatten.PCollections/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.921Z: Executing operation
PAssert$13/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:06.989Z: Executing operation
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:07.035Z: Executing operation
PAssert$13/GroupGlobally/GroupDummyAndContents/Create
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:07.061Z: Starting 1 workers in us-central1-b...
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:07.080Z: Executing operation
Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:07.650Z: Executing operation
PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$13/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Dec 17, 2018 11:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:07.718Z: Executing operation
Flatten.PCollections/Create.Values/Read(CreateSource)/Impulse+Flatten.PCollections/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair
with random
key+Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Flatten.PCollections/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Dec 17, 2018 11:42:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:17.159Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Dec 17, 2018 11:42:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:50.994Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Dec 17, 2018 11:42:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:51.045Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
Dec 17, 2018 11:42:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:51.106Z: Workers have started successfully.
Dec 17, 2018 11:42:51 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T11:42:51.151Z: Workers have started successfully.
Dec 17, 2018 12:42:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2018-12-17T12:42:07.885Z: Workflow failed. Causes: The Dataflow job
appears to be stuck because no worker activity has been seen in the last 1h.
You can get help with Cloud Dataflow at
https://cloud.google.com/dataflow/support.
Dec 17, 2018 12:42:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T12:42:08.021Z: Cancel request is committed for workflow
job: 2018-12-17_03_41_55-17546666855132132476.
Dec 17, 2018 12:42:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T12:42:08.124Z: Cleaning up.
Dec 17, 2018 12:42:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T12:42:08.231Z: Stopping worker pool...
Dec 17, 2018 12:42:09 PM
org.apache.beam.runners.dataflow.TestDataflowRunner$ErrorMonitorMessagesHandler
process
INFO: Dataflow job 2018-12-17_03_41_55-17546666855132132476 threw
exception. Failure message was: Workflow failed. Causes: The Dataflow job
appears to be stuck because no worker activity has been seen in the last 1h.
You can get help with Cloud Dataflow at
https://cloud.google.com/dataflow/support.
Dec 17, 2018 12:43:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T12:43:48.944Z: Autoscaling: Reduced the number of workers
to 0 based on the rate of progress in the currently running step(s).
Dec 17, 2018 12:43:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-12-17T12:43:49.002Z: Worker pool stopped.
Dec 17, 2018 12:43:54 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-12-17_03_41_55-17546666855132132476 failed with status
FAILED.
Dec 17, 2018 12:43:54 PM
org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-12-17_03_41_55-17546666855132132476. Found 0 success, 0 failures out of 0
expected assertions.
Gradle Test Executor 112 finished executing tests.
> Task
> :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerExecutableStageTest
> FAILED
org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsEmptyThenParDo FAILED
java.lang.RuntimeException: Workflow failed. Causes: The Dataflow job
appears to be stuck because no worker activity has been seen in the last 1h.
You can get help with Cloud Dataflow at
https://cloud.google.com/dataflow/support.
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:350)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:331)
at
org.apache.beam.sdk.transforms.FlattenTest.testFlattenPCollectionsEmptyThenParDo(FlattenTest.java:224)
72 tests completed, 2 failed, 1 skipped
Finished generating test XML results (0.184 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerExecutableStageTest>
Generating HTML test report...
Finished generating test html results (0.235 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerExecutableStageTest>
:beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerExecutableStageTest
(Thread[Task worker for ':' Thread 115,5,main]) completed. Took 2 hrs 48 mins
19.554 secs.
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 115,5,main]) started.
> Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages FAILED
Caching disabled for task
':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not
been enabled for the task
Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not
up-to-date because:
Task has not declared any outputs despite executing actions.
Starting process 'command 'docker''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: docker rmi
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181217095230
Successfully started process 'command 'docker''
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181217095230
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3706c0f9df60ce85c38c5a9cab608375d17b48d9782c9682e9b1910e97e2dc2f
Starting process 'command 'gcloud''. Working directory:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java>
Command: gcloud --quiet container images delete --force-delete-tags
us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181217095230
Successfully started process 'command 'gcloud''
ERROR: (gcloud.container.images.delete)
[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181217095230] is not a
valid name.
:beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task
worker for ':' Thread 115,5,main]) completed. Took 1.108 secs.
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerExecutableStageTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerExecutableStageTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build.gradle'>
line: 457
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 51m 44s
75 actionable tasks: 69 executed, 5 from cache, 1 up-to-date
Publishing build scan...
https://gradle.com/s/chdwxtfi56ep2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
