See
<https://builds.apache.org/job/beam_PostCommit_XVR_Flink/887/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-8472] Get default GCP region from gcloud (Java)
[kcweaver] [BEAM-8512] Add integration tests for flink_runner.py
------------------------------------------
[...truncated 4.30 MB...]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (10/16).
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (10/16)
(09b0df55e9fbed03fa227000e2d6dabf) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (10/16)
(09b0df55e9fbed03fa227000e2d6dabf) [DEPLOYING]
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (10/16) (09b0df55e9fbed03fa227000e2d6dabf) [DEPLOYING].
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (10/16) (09b0df55e9fbed03fa227000e2d6dabf)
[DEPLOYING].
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (10/16)
(09b0df55e9fbed03fa227000e2d6dabf) switched from DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (10/16) (09b0df55e9fbed03fa227000e2d6dabf) switched from
DEPLOYING to RUNNING.
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (10/16)
(09b0df55e9fbed03fa227000e2d6dabf) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (10/16) (09b0df55e9fbed03fa227000e2d6dabf).
[DataSink (DiscardingOutput) (10/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (10/16)
(09b0df55e9fbed03fa227000e2d6dabf) [FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) 09b0df55e9fbed03fa227000e2d6dabf.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (10/16) (09b0df55e9fbed03fa227000e2d6dabf) switched from
RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (5/16)]
INFO org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (5/16) (6feae49387ddebf8d54f54da103c084d)
switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (5/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (5/16)
(6feae49387ddebf8d54f54da103c084d).
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (5/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (5/16) (6feae49387ddebf8d54f54da103c084d)
[FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4) switched from
CREATED to SCHEDULED.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4) switched from
SCHEDULED to DEPLOYING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying DataSink
(DiscardingOutput) (5/16) (attempt #0) to 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528
@ localhost (dataPort=-1)
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match})
6feae49387ddebf8d54f54da103c084d.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (5/16).
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match}) (5/16)
(6feae49387ddebf8d54f54da103c084d) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (5/16)
(08533fe02e50e21e17828765febcdfe4) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (5/16)
(08533fe02e50e21e17828765febcdfe4) [DEPLOYING]
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4) [DEPLOYING].
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4)
[DEPLOYING].
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (5/16)
(08533fe02e50e21e17828765febcdfe4) switched from DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4) switched from
DEPLOYING to RUNNING.
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (5/16)
(08533fe02e50e21e17828765febcdfe4) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4).
[DataSink (DiscardingOutput) (5/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (5/16)
(08533fe02e50e21e17828765febcdfe4) [FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) 08533fe02e50e21e17828765febcdfe4.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (5/16) (08533fe02e50e21e17828765febcdfe4) switched from
RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (1/16)]
INFO org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (1/16) (d2001eac1e7559a77b9324842723657b)
switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (1/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (1/16)
(d2001eac1e7559a77b9324842723657b).
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (1/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (1/16) (d2001eac1e7559a77b9324842723657b)
[FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match})
d2001eac1e7559a77b9324842723657b.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c) switched from
CREATED to SCHEDULED.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c) switched from
SCHEDULED to DEPLOYING.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying DataSink
(DiscardingOutput) (1/16) (attempt #0) to 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528
@ localhost (dataPort=-1)
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (1/16).
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match}) (1/16)
(d2001eac1e7559a77b9324842723657b) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (1/16)
(09fc70eb2005aad5b3bf36a9df71330c) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (1/16)
(09fc70eb2005aad5b3bf36a9df71330c) [DEPLOYING]
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c) [DEPLOYING].
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c)
[DEPLOYING].
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (1/16)
(09fc70eb2005aad5b3bf36a9df71330c) switched from DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c) switched from
DEPLOYING to RUNNING.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (3/16)]
INFO org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (3/16) (6157374cb7992e79fba0c3dd27c6e142)
switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (3/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (3/16)
(6157374cb7992e79fba0c3dd27c6e142).
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384) switched from
CREATED to SCHEDULED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (3/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (3/16) (6157374cb7992e79fba0c3dd27c6e142)
[FINISHED]
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384) switched from
SCHEDULED to DEPLOYING.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match})
6157374cb7992e79fba0c3dd27c6e142.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying DataSink
(DiscardingOutput) (3/16) (attempt #0) to 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528
@ localhost (dataPort=-1)
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (3/16).
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match}) (3/16)
(6157374cb7992e79fba0c3dd27c6e142) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (1/16)
(09fc70eb2005aad5b3bf36a9df71330c) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c).
[DataSink (DiscardingOutput) (1/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (1/16)
(09fc70eb2005aad5b3bf36a9df71330c) [FINISHED]
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) 09fc70eb2005aad5b3bf36a9df71330c.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (1/16) (09fc70eb2005aad5b3bf36a9df71330c) switched from
RUNNING to FINISHED.
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (3/16)
(f0ffb07679107876bf12f6af35c9f384) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (3/16)
(f0ffb07679107876bf12f6af35c9f384) [DEPLOYING]
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384) [DEPLOYING].
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384)
[DEPLOYING].
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (3/16)
(f0ffb07679107876bf12f6af35c9f384) switched from DEPLOYING to RUNNING.
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (3/16)
(f0ffb07679107876bf12f6af35c9f384) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384).
[DataSink (DiscardingOutput) (3/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (3/16)
(f0ffb07679107876bf12f6af35c9f384) [FINISHED]
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (16/16)]
INFO org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (16/16)
(906e1b2788930abd2332b618be379c77) switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (16/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (16/16)
(906e1b2788930abd2332b618be379c77).
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (16/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (16/16)
(906e1b2788930abd2332b618be379c77) [FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384) switched from
DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) f0ffb07679107876bf12f6af35c9f384.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07) switched from
CREATED to SCHEDULED.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match})
906e1b2788930abd2332b618be379c77.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07) switched from
SCHEDULED to DEPLOYING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying DataSink
(DiscardingOutput) (16/16) (attempt #0) to 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528
@ localhost (dataPort=-1)
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (3/16) (f0ffb07679107876bf12f6af35c9f384) switched from
RUNNING to FINISHED.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match}) (16/16)
(906e1b2788930abd2332b618be379c77) switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (2/16)]
INFO org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (2/16) (b3c167c487f64dfbe08b05c779c9eb02)
switched from RUNNING to FINISHED.
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (2/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (2/16)
(b3c167c487f64dfbe08b05c779c9eb02).
[MapPartition (MapPartition at [3]assert_that/{Group, Unkey, Match}) (2/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task MapPartition (MapPartition at
[3]assert_that/{Group, Unkey, Match}) (2/16) (b3c167c487f64dfbe08b05c779c9eb02)
[FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173) switched from
CREATED to SCHEDULED.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173) switched from
SCHEDULED to DEPLOYING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying DataSink
(DiscardingOutput) (2/16) (attempt #0) to 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528
@ localhost (dataPort=-1)
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (16/16).
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match})
b3c167c487f64dfbe08b05c779c9eb02.
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (16/16)
(12b335fc43a5b9a17e95d81efb483f07) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (16/16)
(12b335fc43a5b9a17e95d81efb483f07) [DEPLOYING]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [3]assert_that/{Group, Unkey, Match}) (2/16)
(b3c167c487f64dfbe08b05c779c9eb02) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07) [DEPLOYING].
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07)
[DEPLOYING].
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (16/16)
(12b335fc43a5b9a17e95d81efb483f07) switched from DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07) switched from
DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task DataSink
(DiscardingOutput) (2/16).
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (16/16)
(12b335fc43a5b9a17e95d81efb483f07) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07).
[DataSink (DiscardingOutput) (16/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (16/16)
(12b335fc43a5b9a17e95d81efb483f07) [FINISHED]
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (2/16)
(14e6edd10ed467ac0f914214a1364173) switched from CREATED to DEPLOYING.
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream leak
safety net for task DataSink (DiscardingOutput) (2/16)
(14e6edd10ed467ac0f914214a1364173) [DEPLOYING]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) 12b335fc43a5b9a17e95d81efb483f07.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (16/16) (12b335fc43a5b9a17e95d81efb483f07) switched from
RUNNING to FINISHED.
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173) [DEPLOYING].
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173)
[DEPLOYING].
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (2/16)
(14e6edd10ed467ac0f914214a1364173) switched from DEPLOYING to RUNNING.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173) switched from
DEPLOYING to RUNNING.
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput) (2/16)
(14e6edd10ed467ac0f914214a1364173) switched from RUNNING to FINISHED.
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173).
[DataSink (DiscardingOutput) (2/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task DataSink (DiscardingOutput) (2/16)
(14e6edd10ed467ac0f914214a1364173) [FINISHED]
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and
sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) 14e6edd10ed467ac0f914214a1364173.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (2/16) (14e6edd10ed467ac0f914214a1364173) switched from
RUNNING to FINISHED.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph - Job
BeamApp-root-1108224127-35fe3e81 (104898b5434370d5f594eea911426f7c) switched
from state RUNNING to FINISHED.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Job
104898b5434370d5f594eea911426f7c reached globally terminal state FINISHED.
[flink-runner-job-invoker] INFO
org.apache.flink.runtime.minicluster.MiniCluster - Shutting down Flink Mini
Cluster
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.jobmaster.JobMaster - Stopping the JobMaster for job
BeamApp-root-1108224127-35fe3e81(104898b5434370d5f594eea911426f7c).
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopping TaskExecutor
akka://flink/user/taskmanager_0.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Close ResourceManager
connection 73f8aab09a8d1a712d2c37c2f63c0655.
[flink-runner-job-invoker] INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shutting down rest
endpoint.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.JobLeaderService - Stop job leader
service.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Closing
TaskExecutor connection 9c4fe65b-4e3d-4d0e-a7b6-dac4bc732528 because: The
TaskExecutor is shutting down.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Suspending SlotPool.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.jobmaster.JobMaster - Close ResourceManager connection
73f8aab09a8d1a712d2c37c2f63c0655: JobManager is shutting down..
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Disconnect
job manager 8ee244f2a76f3c2c5dedcd2f417f434f@akka://flink/user/jobmanager_1 for
job 104898b5434370d5f594eea911426f7c from the resource manager.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager - Shutting
down TaskExecutorLocalStateStoresManager.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Stopping SlotPool.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.io.disk.FileChannelManagerImpl - FileChannelManager
removed spill file directory /tmp/flink-io-5f1fcc53-e867-4013-afbd-07ff840ff0a9
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.io.network.NettyShuffleEnvironment - Shutting down the
network environment and its components.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.io.disk.FileChannelManagerImpl - FileChannelManager
removed spill file directory
/tmp/flink-netty-shuffle-d35034e1-fd4b-40e4-935c-25d34a33392a
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.KvStateService - Shutting down the
kvState service and its components.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.JobLeaderService - Stop job leader
service.
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.filecache.FileCache - removed file cache directory
/tmp/flink-dist-cache-2d9d0cb3-365d-4ea4-9bda-5d4fd128a6c5
[flink-akka.actor.default-dispatcher-8] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopped TaskExecutor
akka://flink/user/taskmanager_0.
[ForkJoinPool.commonPool-worker-9] INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Removing cache
directory /tmp/flink-web-ui
[ForkJoinPool.commonPool-worker-9] INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shut down complete.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Shut down
cluster because application is in CANCELED, diagnostics
DispatcherResourceManagerComponent has been closed..
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping dispatcher
akka://flink/user/dispatcher.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping all
currently running jobs of dispatcher akka://flink/user/dispatcher.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl - Closing
the SlotManager.
[flink-akka.actor.default-dispatcher-11] INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl -
Suspending the SlotManager.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.rest.handler.legacy.backpressure.StackTraceSampleCoordinator
- Shutting down stack trace sample coordinator.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopped dispatcher
akka://flink/user/dispatcher.
[flink-akka.actor.default-dispatcher-9] INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.
[flink-metrics-2] INFO akka.remote.RemoteActorRefProvider$RemotingTerminator -
Shutting down remote daemon.
[flink-metrics-2] INFO akka.remote.RemoteActorRefProvider$RemotingTerminator -
Remote daemon shut down; proceeding with flushing remote transports.
[flink-metrics-2] INFO akka.remote.RemoteActorRefProvider$RemotingTerminator -
Remoting shut down.
[flink-metrics-2] INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService -
Stopping Akka RPC service.
[flink-metrics-2] INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService -
Stopped Akka RPC service.
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.blob.PermanentBlobCache - Shutting down BLOB cache
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.blob.TransientBlobCache - Shutting down BLOB cache
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.blob.BlobServer - Stopped BLOB server at 0.0.0.0:41765
[flink-akka.actor.default-dispatcher-5] INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.
[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - Execution finished in 46378
msecs
[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - Final accumulator values:
[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - __metricscontainers :
MetricQueryResults(Counters(ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/ExtractOutputs}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:385>)_18}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:385>)_18}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:390>)_22}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:389>)_21}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_30}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=external_4_AppliedPTransform_root/PerElement/PerElement:PairWithVoid_4}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=external_4_AppliedPTransform_root/PerElement/PerElement:PairWithVoid_4}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2467>)_26}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_9}: 12,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_41}: 300,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:390>)_22}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_32}: 759,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:1:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=external_4_AppliedPTransform_root/PerElement/PerElement:PairWithVoid_4}:
371,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:385>)_18}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:390>)_22}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_39}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_30}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:0:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_41}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=external_4_PCollection_PCollection_1}: 6,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/Map(decode)_16}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:389>)_21}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_17:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:390>)_22}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_30}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_29}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(unicode)_17}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/Map(decode)_16}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(unicode)_17}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:389>)_21}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:0:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_22}: 1,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/ExtractOutputs}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:0:0}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_2:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_10}: 12,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Precombine}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_11}: 12,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_12}: 12,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/pcollection:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_29}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_40}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/Map(decode)_16}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_40}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_29}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Precombine}:
0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_2:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_39}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/pcollection:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/pcollection_1:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_24:1}: 1,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_13}: 6,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/pcollection:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/pcollection:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_30}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/ExtractOutputs}:
0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_2:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_33}: 585,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_18}: 1,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_28}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_17}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_39}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_19}: 1,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=external_3_AppliedPTransform_root/Filter(<lambda at
expansion_service_test.py:62>)_3}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=external_3_AppliedPTransform_root/Filter(<lambda at
expansion_service_test.py:62>)_3}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_17:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Merge}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_28}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_27:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_40}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/Map(decode)_16}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=external_3_AppliedPTransform_root/Filter(<lambda at
expansion_service_test.py:62>)_3}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/pcollection_1:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_28}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_28}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Precombine}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/pcollection_1:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_17:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=external_3_AppliedPTransform_root/Filter(<lambda at
expansion_service_test.py:62>)_3}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2467>)_4}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_27}: 1,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Merge}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_29}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_28}: 1,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_40}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_17:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:1:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:385>)_18}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=external_4_AppliedPTransform_root/PerElement/PerElement:PairWithVoid_4}:
371,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Merge}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/pcollection_1:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_24:0}: 3,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_14}: 3,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_2}: 12,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_27:0}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_1}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_39}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_32}: 759,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_2:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_16}: 3,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_15}: 3,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(<lambda at external_test.py:389>)_21}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_41}: 300,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:0:0}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2467>)_4}:
462,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2467>)_26}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2467>)_26}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:1:0}: 0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Precombine}:
0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_24:1:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_33}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_30}: 1,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_41}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_33}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_33}: 585,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/ExtractOutputs}:
0,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=pcollection}: 6,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=pcollection_1}: 3,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(unicode)_17}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_29}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2467>)_26}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_27:0}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=pcollection_2}: 3,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_27:0}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_34}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_20}: 3,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Merge}:
0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_23}: 3,
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Map(unicode)_17}: 0,
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_21}: 3,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_32}: 0,
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_32}: 0,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2467>)_4}:
462,
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_Create/FlatMap(<lambda at core.py:2467>)_4}:
0)Distributions(48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=pcollection}: DistributionResult{sum=96, count=6, min=16,
max=16},
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_22}: DistributionResult{sum=17,
count=1, min=17, max=17},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_23}: DistributionResult{sum=63,
count=3, min=21, max=21},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_20}: DistributionResult{sum=54,
count=3, min=18, max=18},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_21}: DistributionResult{sum=57,
count=3, min=19, max=19},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_28}: DistributionResult{sum=41,
count=1, min=41, max=41},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_29}: DistributionResult{sum=33,
count=1, min=33, max=33},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_27}: DistributionResult{sum=58,
count=1, min=58, max=58},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=external_4_PCollection_PCollection_1}: DistributionResult{sum=90,
count=6, min=15, max=15},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=pcollection_2}: DistributionResult{sum=48, count=3, min=16,
max=16},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_13}: DistributionResult{sum=84,
count=6, min=14, max=14},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=pcollection_1}: DistributionResult{sum=72, count=3, min=21,
max=27},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_30}: DistributionResult{sum=14,
count=1, min=14, max=14},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_15}: DistributionResult{sum=51,
count=3, min=17, max=17},
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_19}: DistributionResult{sum=15,
count=1, min=15, max=15},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_16}: DistributionResult{sum=54,
count=3, min=18, max=18},
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_17}: DistributionResult{sum=13,
count=1, min=13, max=13},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_14}: DistributionResult{sum=45,
count=3, min=15, max=15},
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_18}: DistributionResult{sum=16,
count=1, min=16, max=16},
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_2}: DistributionResult{sum=165,
count=11, min=15, max=15},
14Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1}: DistributionResult{sum=13,
count=1, min=13, max=13},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_10}: DistributionResult{sum=168,
count=12, min=14, max=14},
ExternalTransform(beam:transforms:xlang:count)/PerElement/CombinePerKey(CountCombineFn)/Group.out/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_24:0}: DistributionResult{sum=72,
count=3, min=24, max=24},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_12}: DistributionResult{sum=168,
count=12, min=14, max=14},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_11}: DistributionResult{sum=168,
count=12, min=14, max=14},
48Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_9}: DistributionResult{sum=192,
count=12, min=16, max=16},
26assert_that/Create/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_24:1}: DistributionResult{sum=19,
count=1, min=19, max=19}))
[flink-runner-job-invoker] INFO
org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService -
Manifest at
/tmp/beam-artifact-staging/job_f3ebce16-e3f6-45bf-8b1c-501075aacb39/MANIFEST
has 0 artifact locations
[flink-runner-job-invoker] INFO
org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService
- Removed dir
/tmp/beam-artifact-staging/job_f3ebce16-e3f6-45bf-8b1c-501075aacb39/
ok
test_multi (apache_beam.transforms.external_test.ExternalTransformTest) ... ok
test_nested (apache_beam.transforms.external_test.ExternalTransformTest) ...
[pool-5-thread-1] INFO
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory - Closing
environment urn: "beam:env:docker:v1"
payload: "\n#apachebeam/python2.7_sdk:2.18.0.dev"
[grpc-default-executor-1] INFO sdk_worker.run - No more requests from control
plane
[grpc-default-executor-1] INFO sdk_worker.run - SDK Harness waiting for
in-flight requests to complete
[grpc-default-executor-0] WARN
org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown
endpoint.
[grpc-default-executor-1] INFO data_plane.close - Closing all cached grpc data
channels.
[grpc-default-executor-1] INFO sdk_worker.close - Closing all cached gRPC state
handlers.
[grpc-default-executor-1] INFO sdk_worker.run - Done consuming work.
[grpc-default-executor-1] INFO sdk_worker_main.main - Python sdk harness
exiting.
[grpc-default-executor-1] INFO
org.apache.beam.runners.fnexecution.logging.GrpcLoggingService - Logging client
hanged up.
[pool-5-thread-1] INFO
org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment -
Closing Docker container
db239bfc225289571de170874b78141fc8f9c34889b24016daf2ae1e3a0c0a3b. Logs:
2019/11/08 22:41:50 Initializing python harness: /opt/apache/beam/boot --id=1-1
--logging_endpoint=localhost:45799 --artifact_endpoint=localhost:41103
--provision_endpoint=localhost:40605 --control_endpoint=localhost:38925
2019/11/08 22:41:50 Installing setup packages ...
2019/11/08 22:41:50 Executing: python -m
apache_beam.runners.worker.sdk_worker_main
/usr/local/lib/python2.7/site-packages/apache_beam/__init__.py:84: UserWarning:
You are using Apache Beam with Python 2. New releases of Apache Beam will soon
support Python 3 only.
'You are using Apache Beam with Python 2. '
[pool-5-thread-1] WARN
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory - Error
cleaning up environment urn: "beam:env:docker:v1"
payload: "\n#apachebeam/python2.7_sdk:2.18.0.dev"
java.io.IOException: Received exit code 1 for command 'docker rm
db239bfc225289571de170874b78141fc8f9c34889b24016daf2ae1e3a0c0a3b'. stderr:
Error: No such container:
db239bfc225289571de170874b78141fc8f9c34889b24016daf2ae1e3a0c0a3b
at
org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:234)
at
org.apache.beam.runners.fnexecution.environment.DockerCommand.runShortCommand(DockerCommand.java:168)
at
org.apache.beam.runners.fnexecution.environment.DockerCommand.removeContainer(DockerCommand.java:163)
at
org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment.close(DockerContainerEnvironment.java:95)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.$closeResource(DefaultJobBundleFactory.java:349)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.close(DefaultJobBundleFactory.java:349)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.unref(DefaultJobBundleFactory.java:369)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient.access$700(DefaultJobBundleFactory.java:315)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.lambda$createEnvironmentCache$3(DefaultJobBundleFactory.java:154)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:1809)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3462)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3438)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3215)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4909)
at
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.close(DefaultJobBundleFactory.java:195)
at
org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext.close(DefaultExecutableStageContext.java:47)
at
org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.closeActual(ReferenceCountingExecutableStageContextFactory.java:208)
at
org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory$WrappedContext.access$200(ReferenceCountingExecutableStageContextFactory.java:184)
at
org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.release(ReferenceCountingExecutableStageContextFactory.java:173)
at
org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory.lambda$scheduleRelease$1(ReferenceCountingExecutableStageContextFactory.java:127)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
ok
test_payload (apache_beam.transforms.external_test.ExternalTransformTest) ... ok
test_pipeline_generation
(apache_beam.transforms.external_test.ExternalTransformTest) ... ok
test_simple (apache_beam.transforms.external_test.ExternalTransformTest) ... ok
----------------------------------------------------------------------
XML: nosetests-xlangValidateRunner.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 9 tests in 144.029s
OK (SKIP=1)
> Task :runners:flink:1.9:job-server:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 9397.
Stopping expansion service pid: 9398.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':runners:flink:1.9:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 17m 32s
112 actionable tasks: 92 executed, 17 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/6qv4l3o4k6tae
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
