See
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/7861/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-7006] Fix some race conditions in sdf splitting.
[github] [BEAM-6978] Disable SDF testing for Dataflow using portability. (#8261)
------------------------------------------
[...truncated 469.75 KB...]
root: INFO: 2019-04-09T18:42:51.004Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T18:42:51.053Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T18:42:51.101Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T18:42:51.149Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T18:42:51.196Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T18:42:51.227Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T18:42:51.274Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T18:42:51.321Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T18:42:51.360Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/InitializeWrite into
write/Write/WriteImpl/DoOnce/Map(decode)
root: INFO: 2019-04-09T18:42:51.409Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-09T18:42:51.458Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-09T18:42:51.506Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:51.554Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into
read/Read/Reshuffle/AddRandomKeys
root: INFO: 2019-04-09T18:42:51.594Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WindowInto(WindowIntoFn) into write/Write/WriteImpl/Pair
root: INFO: 2019-04-09T18:42:51.637Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Pair into write/Write/WriteImpl/WriteBundles/WriteBundles
root: INFO: 2019-04-09T18:42:51.680Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/AddRandomKeys into read/Read/Split
root: INFO: 2019-04-09T18:42:51.729Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/WriteBundles into format
root: INFO: 2019-04-09T18:42:51.779Z: JOB_MESSAGE_DETAILED: Fusing consumer
format into count
root: INFO: 2019-04-09T18:42:51.817Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:51.861Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:51.895Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2
root: INFO: 2019-04-09T18:42:51.937Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Reify into pair_with_one
root: INFO: 2019-04-09T18:42:51.988Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:52.036Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1
root: INFO: 2019-04-09T18:42:52.074Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:52.115Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:52.159Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0
root: INFO: 2019-04-09T18:42:52.207Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>) into
write/Write/WriteImpl/DoOnce/Impulse
root: INFO: 2019-04-09T18:42:52.255Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Split into read/Read/Impulse
root: INFO: 2019-04-09T18:42:52.308Z: JOB_MESSAGE_DETAILED: Fusing consumer
count into group/GroupByWindow
root: INFO: 2019-04-09T18:42:52.346Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:52.395Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey1
root: INFO: 2019-04-09T18:42:52.443Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read
root: INFO: 2019-04-09T18:42:52.491Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Extract into
write/Write/WriteImpl/GroupByKey/GroupByWindow
root: INFO: 2019-04-09T18:42:52.535Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/GroupByWindow into
write/Write/WriteImpl/GroupByKey/Read
root: INFO: 2019-04-09T18:42:52.567Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Write into
write/Write/WriteImpl/GroupByKey/Reify
root: INFO: 2019-04-09T18:42:52.615Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Reify into
write/Write/WriteImpl/WindowInto(WindowIntoFn)
root: INFO: 2019-04-09T18:42:52.663Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/GroupByWindow into group/Read
root: INFO: 2019-04-09T18:42:52.712Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:52.748Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Write into group/Reify
root: INFO: 2019-04-09T18:42:52.790Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/WriteBundles/MapToVoidKey0
root: INFO: 2019-04-09T18:42:52.833Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:52.880Z: JOB_MESSAGE_DETAILED: Fusing consumer
pair_with_one into split
root: INFO: 2019-04-09T18:42:52.923Z: JOB_MESSAGE_DETAILED: Fusing consumer
split into read/Read/ReadSplits
root: INFO: 2019-04-09T18:42:52.962Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:52.999Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/ReadSplits into read/Read/Reshuffle/RemoveRandomKeys
root: INFO: 2019-04-09T18:42:53.046Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/RemoveRandomKeys into
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
root: INFO: 2019-04-09T18:42:53.086Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T18:42:53.133Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey0
root: INFO: 2019-04-09T18:42:53.180Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify
root: INFO: 2019-04-09T18:42:53.224Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify into
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
root: INFO: 2019-04-09T18:42:53.265Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T18:42:53.308Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/Map(decode) into
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>)
root: INFO: 2019-04-09T18:42:53.343Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
root: INFO: 2019-04-09T18:42:53.399Z: JOB_MESSAGE_DEBUG: Workflow config is
missing a default resource spec.
root: INFO: 2019-04-09T18:42:53.436Z: JOB_MESSAGE_DEBUG: Adding StepResource
setup and teardown to workflow graph.
root: INFO: 2019-04-09T18:42:53.485Z: JOB_MESSAGE_DEBUG: Adding workflow start
and stop steps.
root: INFO: 2019-04-09T18:42:53.524Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-04-09T18:42:53.751Z: JOB_MESSAGE_DEBUG: Executing wait step
start120
root: INFO: 2019-04-09T18:42:53.854Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:53.901Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:53.913Z: JOB_MESSAGE_DEBUG: Starting worker pool
setup.
root: INFO: 2019-04-09T18:42:53.950Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:53.950Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-b...
root: INFO: 2019-04-09T18:42:53.993Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:54.037Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:54.092Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T18:42:54.144Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2019-04-09T18:42:54.190Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/GroupByKey/Create
root: INFO: 2019-04-09T18:42:54.237Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Create
root: INFO: 2019-04-09T18:42:54.282Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.327Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.393Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.444Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.487Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.540Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.585Z: JOB_MESSAGE_DEBUG: Value "group/Session"
materialized.
root: INFO: 2019-04-09T18:42:54.621Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/GroupByKey/Session" materialized.
root: INFO: 2019-04-09T18:42:54.646Z: JOB_MESSAGE_DEBUG: Value
"read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
root: INFO: 2019-04-09T18:42:54.696Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/DoOnce/Impulse+write/Write/WriteImpl/DoOnce/FlatMap(<lambda
at
core.py:2171>)+write/Write/WriteImpl/DoOnce/Map(decode)+write/Write/WriteImpl/InitializeWrite+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
root: INFO: 2019-04-09T18:42:54.740Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Impulse+read/Read/Split+read/Read/Reshuffle/AddRandomKeys+read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write
root: INFO: 2019-04-09T18:43:05.780Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 0 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-09T18:45:34.315Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-09T18:46:21.886Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 1 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-09T18:46:21.933Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-09T18:46:36.401Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-09T18:48:28.514Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/DoOnce/Map(decode).out" materialized.
root: INFO: 2019-04-09T18:48:28.552Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T18:48:28.598Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T18:48:28.631Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T18:48:28.678Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T18:48:28.718Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T18:48:28.774Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T18:48:30.423Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Close
root: INFO: 2019-04-09T18:48:30.514Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+read/Read/Reshuffle/RemoveRandomKeys+read/Read/ReadSplits+split+pair_with_one+group/Reify+group/Write
root: INFO: 2019-04-09T18:48:44.230Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T18:48:44.333Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T18:48:44.459Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T18:48:55.818Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T18:48:55.925Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T18:48:56.057Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T18:48:58.993Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T18:48:59.105Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T18:48:59.241Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T18:49:00.516Z: JOB_MESSAGE_BASIC: Executing operation
group/Close
root: INFO: 2019-04-09T18:49:00.656Z: JOB_MESSAGE_BASIC: Executing operation
group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
root: INFO: 2019-04-09T18:49:01.887Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T18:49:04.035Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T18:49:06.165Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T18:49:08.289Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T18:49:08.356Z: JOB_MESSAGE_DEBUG: Executing failure step
failure119
root: INFO: 2019-04-09T18:49:08.394Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes:
S20:group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
failed., A work item was attempted 4 times without success. Each time the
worker eventually lost contact with the service. The work item was attempted
on:
beamapp-jenkins-040918423-04091142-0mf7-harness-87nf,
beamapp-jenkins-040918423-04091142-0mf7-harness-87nf,
beamapp-jenkins-040918423-04091142-0mf7-harness-87nf,
beamapp-jenkins-040918423-04091142-0mf7-harness-87nf
root: INFO: 2019-04-09T18:49:08.562Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-04-09T18:49:09.013Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2019-04-09T18:49:09.065Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2019-04-09T18:54:32.430Z: JOB_MESSAGE_DETAILED: Autoscaling:
Reduced the number of workers to 0 based on the rate of progress in the
currently running step(s).
root: INFO: 2019-04-09T18:54:32.506Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-09T18:54:32.568Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2019-04-09T18:54:32.624Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2019-04-09_11_42_45-5248646047101830749 is in state
JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 34 tests in 4101.917s
FAILED (SKIP=1, errors=1)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_41-8885806751611057190?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_50_04-3802770085846081044?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_01_40-11655756810756149900?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_39-15613654427553891673?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_47_42-5355073160738150603?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_56_14-12156022580113957773?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_07_06-1306302567823332017?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_38-7527255932058879148?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_57_07-10545243818301823958?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_38-15662590309656845450?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_55_09-5575341866627541505?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_05_39-3534365431501166505?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_39-8118717860256896723?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_42_33-15643504843445771913?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_52_31-17583091611188688005?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_04_41-4439175919907142937?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_12_47-4355155633574006456?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_39-17302310131078175207?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_43_22-7968280384039019032?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_52_51-3338470786040679700?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_02_06-5558878772748100033?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_38-7840401104482636233?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_42_45-5248646047101830749?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_54_50-15626659619598170079?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_02_41-8791206692753454650?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_11_44-2807618161140221792?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_20_39-8176016653577514144?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_29_55-370960809826170114?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_32_38-6900097882579258198?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_43_04-8100044439198340006?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_11_55_22-16854331493152938199?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_12_05_43-12189783454641299593?project=apache-beam-testing.
> Task :beam-sdks-python:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/build.gradle'>
line: 229
* What went wrong:
Execution failed for task ':beam-sdks-python:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 12m 47s
6 actionable tasks: 6 executed
Publishing build scan...
https://gradle.com/s/4r5qcflrwx7zm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
