See
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/7855/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-5433] Deprecate environment url field.
[github] [BEAM-7011] Update Beam SDKs to use the StandardSideInputType enums.
------------------------------------------
[...truncated 470.28 KB...]
root: INFO: 2019-04-09T01:07:16.767Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T01:07:16.830Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T01:07:16.885Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T01:07:16.957Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-09T01:07:17.012Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T01:07:17.070Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T01:07:17.138Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T01:07:17.197Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-09T01:07:17.244Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/InitializeWrite into
write/Write/WriteImpl/DoOnce/Map(decode)
root: INFO: 2019-04-09T01:07:17.300Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-09T01:07:17.379Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-09T01:07:17.436Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:17.494Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into
read/Read/Reshuffle/AddRandomKeys
root: INFO: 2019-04-09T01:07:17.545Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WindowInto(WindowIntoFn) into write/Write/WriteImpl/Pair
root: INFO: 2019-04-09T01:07:17.590Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Pair into write/Write/WriteImpl/WriteBundles/WriteBundles
root: INFO: 2019-04-09T01:07:17.634Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/AddRandomKeys into read/Read/Split
root: INFO: 2019-04-09T01:07:17.687Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/WriteBundles into format
root: INFO: 2019-04-09T01:07:17.744Z: JOB_MESSAGE_DETAILED: Fusing consumer
format into count
root: INFO: 2019-04-09T01:07:17.797Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:17.866Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:17.908Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2
root: INFO: 2019-04-09T01:07:17.976Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Reify into pair_with_one
root: INFO: 2019-04-09T01:07:18.036Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:18.085Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1
root: INFO: 2019-04-09T01:07:18.145Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:18.196Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:18.460Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0
root: INFO: 2019-04-09T01:07:18.516Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>) into
write/Write/WriteImpl/DoOnce/Impulse
root: INFO: 2019-04-09T01:07:18.556Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Split into read/Read/Impulse
root: INFO: 2019-04-09T01:07:18.599Z: JOB_MESSAGE_DETAILED: Fusing consumer
count into group/GroupByWindow
root: INFO: 2019-04-09T01:07:18.662Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:18.706Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey1
root: INFO: 2019-04-09T01:07:18.759Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read
root: INFO: 2019-04-09T01:07:18.798Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Extract into
write/Write/WriteImpl/GroupByKey/GroupByWindow
root: INFO: 2019-04-09T01:07:18.845Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/GroupByWindow into
write/Write/WriteImpl/GroupByKey/Read
root: INFO: 2019-04-09T01:07:18.891Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Write into
write/Write/WriteImpl/GroupByKey/Reify
root: INFO: 2019-04-09T01:07:18.931Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Reify into
write/Write/WriteImpl/WindowInto(WindowIntoFn)
root: INFO: 2019-04-09T01:07:18.977Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/GroupByWindow into group/Read
root: INFO: 2019-04-09T01:07:19.030Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:19.076Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Write into group/Reify
root: INFO: 2019-04-09T01:07:19.127Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/WriteBundles/MapToVoidKey0
root: INFO: 2019-04-09T01:07:19.175Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:19.220Z: JOB_MESSAGE_DETAILED: Fusing consumer
pair_with_one into split
root: INFO: 2019-04-09T01:07:19.247Z: JOB_MESSAGE_DETAILED: Fusing consumer
split into read/Read/ReadSplits
root: INFO: 2019-04-09T01:07:19.293Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:19.347Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/ReadSplits into read/Read/Reshuffle/RemoveRandomKeys
root: INFO: 2019-04-09T01:07:19.395Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/RemoveRandomKeys into
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
root: INFO: 2019-04-09T01:07:19.445Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-09T01:07:19.506Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey0
root: INFO: 2019-04-09T01:07:19.563Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify
root: INFO: 2019-04-09T01:07:19.597Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify into
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
root: INFO: 2019-04-09T01:07:19.643Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-09T01:07:19.694Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/Map(decode) into
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>)
root: INFO: 2019-04-09T01:07:19.738Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
root: INFO: 2019-04-09T01:07:19.803Z: JOB_MESSAGE_DEBUG: Workflow config is
missing a default resource spec.
root: INFO: 2019-04-09T01:07:19.857Z: JOB_MESSAGE_DEBUG: Adding StepResource
setup and teardown to workflow graph.
root: INFO: 2019-04-09T01:07:19.899Z: JOB_MESSAGE_DEBUG: Adding workflow start
and stop steps.
root: INFO: 2019-04-09T01:07:19.946Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-04-09T01:07:20.193Z: JOB_MESSAGE_DEBUG: Executing wait step
start120
root: INFO: 2019-04-09T01:07:20.282Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.333Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.347Z: JOB_MESSAGE_DEBUG: Starting worker pool
setup.
root: INFO: 2019-04-09T01:07:20.381Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.390Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-a...
root: INFO: 2019-04-09T01:07:20.417Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.453Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.496Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-09T01:07:20.538Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2019-04-09T01:07:20.581Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/GroupByKey/Create
root: INFO: 2019-04-09T01:07:20.643Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Create
root: INFO: 2019-04-09T01:07:20.691Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.727Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.773Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.828Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.870Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.923Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-09T01:07:20.968Z: JOB_MESSAGE_DEBUG: Value "group/Session"
materialized.
root: INFO: 2019-04-09T01:07:21.030Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/GroupByKey/Session" materialized.
root: INFO: 2019-04-09T01:07:21.078Z: JOB_MESSAGE_DEBUG: Value
"read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
root: INFO: 2019-04-09T01:07:21.130Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/DoOnce/Impulse+write/Write/WriteImpl/DoOnce/FlatMap(<lambda
at
core.py:2171>)+write/Write/WriteImpl/DoOnce/Map(decode)+write/Write/WriteImpl/InitializeWrite+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
root: INFO: 2019-04-09T01:07:21.169Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Impulse+read/Read/Split+read/Read/Reshuffle/AddRandomKeys+read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write
root: INFO: 2019-04-09T01:07:34.993Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 0 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-09T01:09:05.313Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-09T01:09:41.869Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 1 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-09T01:09:41.924Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-09T01:10:07.424Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-09T01:11:55.875Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/DoOnce/Map(decode).out" materialized.
root: INFO: 2019-04-09T01:11:55.916Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T01:11:55.969Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T01:11:56.013Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-09T01:11:56.055Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T01:11:56.105Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T01:11:56.143Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-09T01:11:57.792Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Close
root: INFO: 2019-04-09T01:11:57.904Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+read/Read/Reshuffle/RemoveRandomKeys+read/Read/ReadSplits+split+pair_with_one+group/Reify+group/Write
root: INFO: 2019-04-09T01:12:15.483Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T01:12:15.592Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T01:12:15.703Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T01:12:17.093Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T01:12:17.196Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T01:12:17.322Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T01:12:19.906Z: JOB_MESSAGE_BASIC: Executing operation
group/Close
root: INFO: 2019-04-09T01:12:24.578Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-09T01:12:24.680Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-09T01:12:24.843Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-09T01:12:24.937Z: JOB_MESSAGE_BASIC: Executing operation
group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
root: INFO: 2019-04-09T01:12:26.170Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T01:12:27.325Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T01:12:29.474Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T01:12:31.622Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-09T01:12:31.684Z: JOB_MESSAGE_DEBUG: Executing failure step
failure119
root: INFO: 2019-04-09T01:12:31.724Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes:
S20:group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
failed., A work item was attempted 4 times without success. Each time the
worker eventually lost contact with the service. The work item was attempted
on:
beamapp-jenkins-040901070-04081807-xsym-harness-8crs,
beamapp-jenkins-040901070-04081807-xsym-harness-8crs,
beamapp-jenkins-040901070-04081807-xsym-harness-8crs,
beamapp-jenkins-040901070-04081807-xsym-harness-8crs
root: INFO: 2019-04-09T01:12:31.893Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-04-09T01:12:32.262Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2019-04-09T01:12:32.315Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2019-04-09T01:17:28.580Z: JOB_MESSAGE_DETAILED: Autoscaling:
Reduced the number of workers to 0 based on the rate of progress in the
currently running step(s).
root: INFO: 2019-04-09T01:17:28.690Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-09T01:17:28.750Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2019-04-09T01:17:28.812Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2019-04-08_18_07_11-1342017143600540529 is in state
JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 34 tests in 4243.533s
FAILED (SKIP=1, errors=1)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_57_01-16397932959358575026?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_13_45-6554092708741832365?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_25_22-16437560120793921616?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_59-16397932959358576775?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_21_12-47892905689280479?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_57_01-17849648980693031849?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_12_45-11429825053825812424?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_22_22-14359742662975840473?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_32_34-12068928409724678193?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_59-6272307450062988036?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_18_53-2819326403963972302?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_28_44-16357924976637611321?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_59-6737014563876811576?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_07_11-1342017143600540529?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_17_47-2476361922203450372?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_26_52-2022404083039730979?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_57-13502382306982474772?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_06_47-11650208256629291126?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_16_03-15860457297370597945?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_25_30-13776905748038723188?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_34_40-9917510246929079984?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_59-145184841038020189?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_08_32-5438596358778807864?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_17_53-5953150986100250406?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_26_24-10360565150577439892?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_38_11-8830773246660411375?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_47_32-7230574367907952429?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_57_03-16709530185752875699?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_56_59-4469052590458414182?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_08_25-2458469826163390302?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_20_32-12889687905106028391?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_30_57-6467035141249306438?project=apache-beam-testing.
> Task :beam-sdks-python:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/build.gradle'>
line: 229
* What went wrong:
Execution failed for task ':beam-sdks-python:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 15m 28s
6 actionable tasks: 6 executed
Publishing build scan...
https://gradle.com/s/bssqp5lbbvozs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
