See
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/7865/display/redirect?page=changes>
Changes:
[github] SamzaRunner: Remove the LinkedIn repo accidentally committed (#8259)
------------------------------------------
[...truncated 468.88 KB...]
root: INFO: 2019-04-10T01:54:16.873Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-10T01:54:16.922Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-10T01:54:16.972Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-10T01:54:17.015Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0 into
write/Write/WriteImpl/InitializeWrite
root: INFO: 2019-04-10T01:54:17.050Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-10T01:54:17.086Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-10T01:54:17.116Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-10T01:54:17.158Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1 into
write/Write/WriteImpl/Extract
root: INFO: 2019-04-10T01:54:17.202Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/InitializeWrite into
write/Write/WriteImpl/DoOnce/Map(decode)
root: INFO: 2019-04-10T01:54:17.252Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-10T01:54:17.303Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2 into
write/Write/WriteImpl/PreFinalize/PreFinalize
root: INFO: 2019-04-10T01:54:17.353Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:17.397Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps) into
read/Read/Reshuffle/AddRandomKeys
root: INFO: 2019-04-10T01:54:17.444Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WindowInto(WindowIntoFn) into write/Write/WriteImpl/Pair
root: INFO: 2019-04-10T01:54:17.497Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Pair into write/Write/WriteImpl/WriteBundles/WriteBundles
root: INFO: 2019-04-10T01:54:17.550Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/AddRandomKeys into read/Read/Split
root: INFO: 2019-04-10T01:54:17.595Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/WriteBundles into format
root: INFO: 2019-04-10T01:54:17.627Z: JOB_MESSAGE_DETAILED: Fusing consumer
format into count
root: INFO: 2019-04-10T01:54:17.681Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:17.729Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:17.772Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey2
root: INFO: 2019-04-10T01:54:17.817Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Reify into pair_with_one
root: INFO: 2019-04-10T01:54:17.868Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:17.925Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1
root: INFO: 2019-04-10T01:54:17.970Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:18.011Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:18.063Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0
root: INFO: 2019-04-10T01:54:18.113Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>) into
write/Write/WriteImpl/DoOnce/Impulse
root: INFO: 2019-04-10T01:54:18.159Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Split into read/Read/Impulse
root: INFO: 2019-04-10T01:54:18.202Z: JOB_MESSAGE_DETAILED: Fusing consumer
count into group/GroupByWindow
root: INFO: 2019-04-10T01:54:18.252Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:18.306Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey1
root: INFO: 2019-04-10T01:54:18.346Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read
root: INFO: 2019-04-10T01:54:18.384Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/Extract into
write/Write/WriteImpl/GroupByKey/GroupByWindow
root: INFO: 2019-04-10T01:54:18.421Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/GroupByWindow into
write/Write/WriteImpl/GroupByKey/Read
root: INFO: 2019-04-10T01:54:18.473Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Write into
write/Write/WriteImpl/GroupByKey/Reify
root: INFO: 2019-04-10T01:54:18.516Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/GroupByKey/Reify into
write/Write/WriteImpl/WindowInto(WindowIntoFn)
root: INFO: 2019-04-10T01:54:18.560Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/GroupByWindow into group/Read
root: INFO: 2019-04-10T01:54:18.613Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:18.660Z: JOB_MESSAGE_DETAILED: Fusing consumer
group/Write into group/Reify
root: INFO: 2019-04-10T01:54:18.694Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/WriteBundles/MapToVoidKey0
root: INFO: 2019-04-10T01:54:18.737Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:18.789Z: JOB_MESSAGE_DETAILED: Fusing consumer
pair_with_one into split
root: INFO: 2019-04-10T01:54:18.827Z: JOB_MESSAGE_DETAILED: Fusing consumer
split into read/Read/ReadSplits
root: INFO: 2019-04-10T01:54:18.869Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:18.922Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/ReadSplits into read/Read/Reshuffle/RemoveRandomKeys
root: INFO: 2019-04-10T01:54:18.969Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/RemoveRandomKeys into
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)
root: INFO: 2019-04-10T01:54:19.017Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/ToIsmRecordForMultimap
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read
root: INFO: 2019-04-10T01:54:19.054Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
into write/Write/WriteImpl/PreFinalize/MapToVoidKey0
root: INFO: 2019-04-10T01:54:19.106Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify
root: INFO: 2019-04-10T01:54:19.133Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify into
read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)
root: INFO: 2019-04-10T01:54:19.172Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
into
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey
root: INFO: 2019-04-10T01:54:19.219Z: JOB_MESSAGE_DETAILED: Fusing consumer
write/Write/WriteImpl/DoOnce/Map(decode) into
write/Write/WriteImpl/DoOnce/FlatMap(<lambda at core.py:2171>)
root: INFO: 2019-04-10T01:54:19.275Z: JOB_MESSAGE_DETAILED: Fusing consumer
read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps) into
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow
root: INFO: 2019-04-10T01:54:19.331Z: JOB_MESSAGE_DEBUG: Workflow config is
missing a default resource spec.
root: INFO: 2019-04-10T01:54:19.368Z: JOB_MESSAGE_DEBUG: Adding StepResource
setup and teardown to workflow graph.
root: INFO: 2019-04-10T01:54:19.412Z: JOB_MESSAGE_DEBUG: Adding workflow start
and stop steps.
root: INFO: 2019-04-10T01:54:19.472Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
root: INFO: 2019-04-10T01:54:19.735Z: JOB_MESSAGE_DEBUG: Executing wait step
start120
root: INFO: 2019-04-10T01:54:19.825Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:19.865Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:19.878Z: JOB_MESSAGE_DEBUG: Starting worker pool
setup.
root: INFO: 2019-04-10T01:54:19.911Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:19.932Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-a...
root: INFO: 2019-04-10T01:54:19.957Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:20.004Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:20.053Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Create
root: INFO: 2019-04-10T01:54:20.098Z: JOB_MESSAGE_BASIC: Executing operation
group/Create
root: INFO: 2019-04-10T01:54:20.134Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/GroupByKey/Create
root: INFO: 2019-04-10T01:54:20.162Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Create
root: INFO: 2019-04-10T01:54:20.212Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.258Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.310Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.360Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.402Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey1.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.452Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey2.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.507Z: JOB_MESSAGE_DEBUG: Value "group/Session"
materialized.
root: INFO: 2019-04-10T01:54:20.561Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/GroupByKey/Session" materialized.
root: INFO: 2019-04-10T01:54:20.616Z: JOB_MESSAGE_DEBUG: Value
"read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Session" materialized.
root: INFO: 2019-04-10T01:54:20.677Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/DoOnce/Impulse+write/Write/WriteImpl/DoOnce/FlatMap(<lambda
at
core.py:2171>)+write/Write/WriteImpl/DoOnce/Map(decode)+write/Write/WriteImpl/InitializeWrite+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/WriteBundles/MapToVoidKey0+write/Write/WriteImpl/PreFinalize/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/MapToVoidKey0+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/CreateIsmShardKeyAndSortKey+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Write
root: INFO: 2019-04-10T01:54:20.729Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Impulse+read/Read/Split+read/Read/Reshuffle/AddRandomKeys+read/Read/Reshuffle/ReshufflePerKey/Map(reify_timestamps)+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Reify+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Write
root: INFO: 2019-04-10T01:54:33.579Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 0 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-10T01:55:00.867Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-10T01:55:31.742Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised
the number of workers to 1 based on the rate of progress in the currently
running step(s).
root: INFO: 2019-04-10T01:55:31.785Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-10T01:56:01.925Z: JOB_MESSAGE_DETAILED: Workers have
started successfully.
root: INFO: 2019-04-10T01:57:59.305Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/DoOnce/Map(decode).out" materialized.
root: INFO: 2019-04-10T01:57:59.328Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-10T01:57:59.350Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-10T01:57:59.376Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Close
root: INFO: 2019-04-10T01:57:59.404Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-10T01:57:59.434Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-10T01:57:59.464Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/GroupByKeyHashAndSortByKeyAndWindow/Read+write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap
root: INFO: 2019-04-10T01:58:01.292Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Close
root: INFO: 2019-04-10T01:58:01.378Z: JOB_MESSAGE_BASIC: Executing operation
read/Read/Reshuffle/ReshufflePerKey/GroupByKey/Read+read/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+read/Read/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+read/Read/Reshuffle/RemoveRandomKeys+read/Read/ReadSplits+split+pair_with_one+group/Reify+group/Write
root: INFO: 2019-04-10T01:58:12.161Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-10T01:58:12.247Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-10T01:58:12.368Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/FinalizeWrite/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-10T01:58:23.385Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-10T01:58:23.460Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-10T01:58:23.559Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/WriteBundles/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-10T01:58:24.807Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/ToIsmRecordForMultimap.out0"
materialized.
root: INFO: 2019-04-10T01:58:24.896Z: JOB_MESSAGE_BASIC: Executing operation
write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize
root: INFO: 2019-04-10T01:58:25.006Z: JOB_MESSAGE_DEBUG: Value
"write/Write/WriteImpl/PreFinalize/_DataflowIterableSideInput(MapToVoidKey0.out.0)/Materialize.out0"
materialized.
root: INFO: 2019-04-10T01:58:27.579Z: JOB_MESSAGE_BASIC: Executing operation
group/Close
root: INFO: 2019-04-10T01:58:27.670Z: JOB_MESSAGE_BASIC: Executing operation
group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
root: INFO: 2019-04-10T01:58:29.593Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-10T01:58:31.732Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-10T01:58:31.840Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-10T01:58:33.966Z: JOB_MESSAGE_ERROR:
java.lang.IllegalArgumentException: This handler is only capable of dealing
with urn:beam:sideinput:materialization:multimap:0.1 materializations but was
asked to handle beam:side_input:multimap:v1 for PCollectionView with tag
side0-write/Write/WriteImpl/WriteBundles.
at
org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunction.java:506)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at
org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at
org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
root: INFO: 2019-04-10T01:58:34.006Z: JOB_MESSAGE_DEBUG: Executing failure step
failure119
root: INFO: 2019-04-10T01:58:34.038Z: JOB_MESSAGE_ERROR: Workflow failed.
Causes:
S20:group/Read+group/GroupByWindow+count+format+write/Write/WriteImpl/WriteBundles/WriteBundles+write/Write/WriteImpl/Pair+write/Write/WriteImpl/WindowInto(WindowIntoFn)+write/Write/WriteImpl/GroupByKey/Reify+write/Write/WriteImpl/GroupByKey/Write
failed., A work item was attempted 4 times without success. Each time the
worker eventually lost contact with the service. The work item was attempted
on:
beamapp-jenkins-041001540-04091854-wkiy-harness-n87b,
beamapp-jenkins-041001540-04091854-wkiy-harness-n87b,
beamapp-jenkins-041001540-04091854-wkiy-harness-n87b,
beamapp-jenkins-041001540-04091854-wkiy-harness-n87b
root: INFO: 2019-04-10T01:58:34.167Z: JOB_MESSAGE_DETAILED: Cleaning up.
root: INFO: 2019-04-10T01:58:34.501Z: JOB_MESSAGE_DEBUG: Starting worker pool
teardown.
root: INFO: 2019-04-10T01:58:34.529Z: JOB_MESSAGE_BASIC: Stopping worker pool...
root: INFO: 2019-04-10T02:02:39.204Z: JOB_MESSAGE_DETAILED: Autoscaling:
Reduced the number of workers to 0 based on the rate of progress in the
currently running step(s).
root: INFO: 2019-04-10T02:02:39.250Z: JOB_MESSAGE_DETAILED: Autoscaling: Would
further reduce the number of workers but reached the minimum number allowed for
the job.
root: INFO: 2019-04-10T02:02:39.318Z: JOB_MESSAGE_BASIC: Worker pool stopped.
root: INFO: 2019-04-10T02:02:39.371Z: JOB_MESSAGE_DEBUG: Tearing down pending
resources...
root: INFO: Job 2019-04-09_18_54_11-18387833607456038542 is in state
JOB_STATE_FAILED
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 34 tests in 4065.582s
FAILED (SKIP=1, errors=1)
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_03-1170196345206915199?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_01_23-15939584360570410707?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_13_53-3202457286445459685?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_00-12712498047651681714?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_07_27-3922913701174277943?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_17_27-405259747484638427?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_03-8499719583866827183?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_59_39-13869053309510212379?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_08_36-2731937702320304376?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_18_43-15876601892270197241?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_00-9940474581177994774?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_07_28-2953895501905330616?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_16_03-11743796062125850851?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_00-17166611814404134817?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_54_11-18387833607456038542?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_02_56-5545708486836415002?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_12_56-7466139399270906512?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_23_30-12168905126728480405?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_44_59-15625739874230922285?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_53_13-4178563073920929736?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_03_12-17409176548854152891?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_11_40-7817225223612937946?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_00-57201277802390646?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_54_56-149626389410848739?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_05_07-17576741032482766423?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_14_27-31937571718063591?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_24_13-5804796003133187022?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_33_28-13440651548310478775?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_43_13-13530565862097731773?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_45_00-3330069565970122887?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_18_56_25-13775836606781257099?project=apache-beam-testing.
Found:
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-09_19_09_42-6240272628056156432?project=apache-beam-testing.
> Task :beam-sdks-python:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/build.gradle'>
line: 229
* What went wrong:
Execution failed for task ':beam-sdks-python:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 12m 31s
6 actionable tasks: 6 executed
Publishing build scan...
https://gradle.com/s/6yf6lubsbc37w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
