See
<https://builds.apache.org/job/beam_PostCommit_Python37/2179/display/redirect?page=changes>
Changes:
[echauchot] Add GBK load tests script for spark structured streaming runner
[echauchot] Add CoGBK load tests script for spark structured streaming runner
[echauchot] Add Combine load tests script for spark structured streaming runner
[echauchot] Add Pardo load tests script for spark structured streaming runner
------------------------------------------
[...truncated 6.38 MB...]
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:coder:global_window:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:coder:windowed_value:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:coder:param_windowed_value:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:coder:state_backed_iterable:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities: "beam:coder:row:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:protocol:progress_reporting:v0"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:protocol:worker_status:v1"\n'
INFO:apache_beam.utils.subprocess_server:b'capabilities:
"beam:version:sdk_base:apache/beam_python3.7_sdk:2.22.0.dev"\n'
INFO:apache_beam.utils.subprocess_server:b'\n'
INFO:apache_beam.runners.worker.sdk_worker:No more requests from control plane
INFO:apache_beam.runners.worker.sdk_worker:SDK Harness waiting for in-flight
requests to complete
INFO:apache_beam.utils.subprocess_server:b'[MapPartition (MapPartition at
[1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)] WARN
org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown
endpoint.\n'
INFO:apache_beam.runners.worker.data_plane:Closing all cached grpc data
channels.
INFO:apache_beam.runners.worker.sdk_worker:Closing all cached gRPC state
handlers.
INFO:apache_beam.runners.worker.sdk_worker:Done consuming work.
INFO:apache_beam.utils.subprocess_server:b'[MapPartition (MapPartition at
[1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)] INFO
org.apache.flink.runtime.taskmanager.Task - MapPartition (MapPartition at
[1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)
(0284ce8aeb15a164c5538ba9f9f91942) switched from RUNNING to FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[MapPartition (MapPartition at
[1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
MapPartition (MapPartition at [1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)
(0284ce8aeb15a164c5538ba9f9f91942).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from
CREATED to SCHEDULED.\n'
INFO:apache_beam.utils.subprocess_server:b'[MapPartition (MapPartition at
[1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)] INFO
org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem streams are
closed for task MapPartition (MapPartition at [1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)
(0284ce8aeb15a164c5538ba9f9f91942) [FINISHED]\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task
and sending final execution state FINISHED to JobManager for task MapPartition
(MapPartition at [1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)
0284ce8aeb15a164c5538ba9f9f91942.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from
SCHEDULED to DEPLOYING.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying
DataSink (DiscardingOutput) (4/16) (attempt #0) to
4c3bd432-6f2d-42d6-b997-56948130f85a @ localhost (dataPort=-1)\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - MapPartition
(MapPartition at [1]read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)) (4/16)
(0284ce8aeb15a164c5538ba9f9f91942) switched from RUNNING to FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Received task
DataSink (DiscardingOutput) (4/16).\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput)
(4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from CREATED to DEPLOYING.\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Creating FileSystem stream
leak safety net for task DataSink (DiscardingOutput) (4/16)
(d0a90bfeaebb376673dc1a9325b92471) [DEPLOYING]\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Loading JAR files for task
DataSink (DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471)
[DEPLOYING].\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Registering task at network:
DataSink (DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471)
[DEPLOYING].\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput)
(4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from DEPLOYING to RUNNING.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from
DEPLOYING to RUNNING.\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - DataSink (DiscardingOutput)
(4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from RUNNING to FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for
DataSink (DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471).\n'
INFO:apache_beam.utils.subprocess_server:b'[DataSink (DiscardingOutput) (4/16)]
INFO org.apache.flink.runtime.taskmanager.Task - Ensuring all FileSystem
streams are closed for task DataSink (DiscardingOutput) (4/16)
(d0a90bfeaebb376673dc1a9325b92471) [FINISHED]\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task
and sending final execution state FINISHED to JobManager for task DataSink
(DiscardingOutput) (4/16) d0a90bfeaebb376673dc1a9325b92471.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - DataSink
(DiscardingOutput) (4/16) (d0a90bfeaebb376673dc1a9325b92471) switched from
RUNNING to FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job
BeamApp-jenkins-0416131229-e6afadfc (20187027da79fcaf5f43c0d7147c39e3) switched
from state RUNNING to FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-7]
INFO org.apache.flink.runtime.minicluster.MiniCluster - Shutting down Flink
Mini Cluster\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-7]
INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shutting
down rest endpoint.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Job
20187027da79fcaf5f43c0d7147c39e3 reached globally terminal state FINISHED.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopping
TaskExecutor akka://flink/user/taskmanager_0.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Close
ResourceManager connection f06fe908bfa0258fd43769a89d42efa3.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager -
Closing TaskExecutor connection 4c3bd432-6f2d-42d6-b997-56948130f85a because:
The TaskExecutor is shutting down.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.jobmaster.JobMaster - Stopping the JobMaster for
job BeamApp-jenkins-0416131229-e6afadfc(20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Suspending
SlotPool.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.jobmaster.JobMaster - Close ResourceManager
connection f06fe908bfa0258fd43769a89d42efa3: JobManager is shutting down..\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Stopping
SlotPool.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager -
Disconnect job manager
85e74aec11c3ad2e404fe7325fe24b01@akka://flink/user/jobmanager_1 for job
20187027da79fcaf5f43c0d7147c39e3 from the resource manager.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:5, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 02a689167a401b1940a699949efe25a1, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - Execution finished in 16495
msecs\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - Final accumulator values:\n'
INFO:apache_beam.utils.subprocess_server:b'[mini-cluster-io-thread-14] INFO
org.apache.flink.runtime.taskexecutor.TaskExecutor - JobManager for job
20187027da79fcaf5f43c0d7147c39e3 with leader id
85e74aec11c3ad2e404fe7325fe24b01 lost leadership.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-runner-job-invoker] INFO
org.apache.beam.runners.flink.FlinkPipelineRunner - __metricscontainers :
MetricQueryResults(Counters(69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_7:0}: 0, 73read with value
provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_6}: 1,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2714>)_19}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_9}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_19:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_26}: 0, 69read
with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_7:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0,
TAG=None}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:1:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_15}: 4, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_13}: 4,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_21}: 1, 73read with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_22}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_20}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 73read
with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15}: 0, 73read with value
provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_8}: 0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_4:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_5:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:0:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_12}: 4, 73read with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_7}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_4:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_4:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_19:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_19:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_25}: 0, 73read
with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_7:0}: 0, 73read with value
provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_7:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_7:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_7}: 1, 73read with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_7:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_5}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_33}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/Map(decode)_14}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_26}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/ProcessSizedElementsAndRestrictions0}:
0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_10}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_23}: 0, 68read with
value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_23}: 0, 68read with
value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:1:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:1:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2714>)_12}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/ProcessSizedElementsAndRestrictions0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/ProcessSizedElementsAndRestrictions0}:
0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_11}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_3}: 4, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_4}: 0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_2}: 4, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_26}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_1}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_19}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_33}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_4:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/Map(decode)_14}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_32}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0}:
0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:0:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:0:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/Map(decode)_14}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_21}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_14}: 1, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9}: 0,
69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_7:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_19:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 69read
with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/Map(decode)_14}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)_9}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0}:
0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_21}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_33}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_23}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_21}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_25}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/Map(decode)_21}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_25}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:0:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_32}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_32}:
0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_16:0}: 1,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Unkey_33}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_0_25}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 73read
with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_7:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 69read
with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_5:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_16:1:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_5:0}: 0, 69read with value
provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_5:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2714>)_19}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_9:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/pair_with_1_26}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/ProcessSizedElementsAndRestrictions0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_1/SplitAndSize0}: 1, 73read with
value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/ParDo(RemoveJsonFiles)_15}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 69read
with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2714>)_12}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_1/SplitAndSize0}: 1, 68read with
value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_16:1}: 4, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PCOLLECTION=ref_PCollection_PCollection_1/PairWithRestriction0}: 1, 69read
with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2714>)_12}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0}:
0, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:element_count:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0,
TAG=None}: 1,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/read/ref_PCollection_PCollection_1:0}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_34}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2714>)_19}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_34}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/ToVoidKey_23}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_32}:
0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_34}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:process_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22}: 0,
69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/_PassThroughThenCleanup/Create/FlatMap(<lambda at core.py:2714>)_12}: 0,
68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Create/FlatMap(<lambda at
core.py:2714>)_19}: 0,
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:ptransform_execution_time:total_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Match_34}: 0, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_22}: 0,
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=ref_AppliedPTransform_assert_that/Group/Flatten_27}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:finish_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_1/SplitAndSize0:0}: 0, 68read
with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:pardo_execution_time:start_bundle_msecs:v1
{PTRANSFORM=fn/write/ref_PCollection_PCollection_1/SplitAndSize0:0}:
0)Distributions(68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_13}: DistributionResult{sum=156,
count=4, min=36, max=45}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/PairWithRestriction0,
TAG=None}: DistributionResult{sum=685, count=1, min=685, max=685}, 68read with
value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_15}: DistributionResult{sum=180,
count=4, min=42, max=51},
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_16:0}: DistributionResult{sum=19,
count=1, min=19, max=19}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PTRANSFORM=ref_AppliedPTransform_read with value provider
query/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)_6/SplitAndSize0,
TAG=None}: DistributionResult{sum=953, count=1, min=953, max=953}, 68read with
value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1/PairWithRestriction0}:
DistributionResult{sum=685, count=1, min=685, max=685},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_19}: DistributionResult{sum=145,
count=1, min=145, max=145}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_4}: DistributionResult{sum=0,
count=0, min=9223372036854775807, max=-9223372036854775808}, 68read with value
provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_2}: DistributionResult{sum=152,
count=4, min=35, max=44}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_3}: DistributionResult{sum=152,
count=4, min=35, max=44}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1}: DistributionResult{sum=13,
count=1, min=13, max=13}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_12}: DistributionResult{sum=152,
count=4, min=35, max=44}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_16:1}: DistributionResult{sum=172,
count=4, min=40, max=49}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1/SplitAndSize0}:
DistributionResult{sum=953, count=1, min=953, max=953},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_20}: DistributionResult{sum=123,
count=1, min=123, max=123},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_21}: DistributionResult{sum=115,
count=1, min=115, max=115}, 73read with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_8}: DistributionResult{sum=0,
count=0, min=9223372036854775807, max=-9223372036854775808},
42assert_that/Group/GroupByKey/GroupByWindow.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_22}: DistributionResult{sum=14,
count=1, min=14, max=14}, 73read with value provider
query/_PassThroughThenCleanup/Create/Map(decode).None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_7}: DistributionResult{sum=15,
count=1, min=15, max=15}, 68read with value provider
query/Read/_SDFBoundedSourceWrapper/Impulse.None/SplitAndSize0/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1/SplitAndSize0}:
DistributionResult{sum=954, count=1, min=954, max=954},
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_14}: DistributionResult{sum=21,
count=1, min=21, max=21},
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_11}: DistributionResult{sum=15,
count=1, min=15, max=15}, 69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_6}: DistributionResult{sum=14,
count=1, min=14, max=14},
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_10}: DistributionResult{sum=15,
count=1, min=15, max=15}, 69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_7}: DistributionResult{sum=14,
count=1, min=14, max=14},
26assert_that/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_9}: DistributionResult{sum=13,
count=1, min=13, max=13}, 69read with value provider
query/_PassThroughThenCleanup/Create/Impulse.None/beam:env:external:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_5}: DistributionResult{sum=13,
count=1, min=13, max=13}))\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:14, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 6fc11d071f593ee60bf91296bc7d7eae, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:15, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 1bbb1df5b5225e4ffd04a73baa72ef49, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:2, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 3c38c6ac3222cdc5c69ae4a367722270, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:9, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 518694700fed95410398a5a045bc5796, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:12, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 27ef0929d77aa0b20a5941441efd3866, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:0, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: bf4ee81d9164b0c782533a45df0f29e4, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:3, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 8b27f0b5fcdf215b1615baf049689a14, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:6, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 96073c5c6516d9051a123f94d339a142, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:8, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 1de6a0e4fb29cc961e9e4cd5599277f2, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:7, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 5bdf1fe08b3a2ee7a845e1f1bc47636a, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:10, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 7e60a4836f38d68f168b3d4250b5edb4, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:4, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: ec423ea30db0abbd56a167ef13366009, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:13, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: c59ddf007c8528ff295f37b886ec6052, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:1, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 485f835bb50658150583a37cedeb001c, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl - Free slot
TaskSlot(index:11, state:ACTIVE, resource profile:
ResourceProfile{managedMemory=8.000mb (8388608 bytes), networkMemory=4.000mb
(4194304 bytes)}, allocationId: 9ed4dc8f31674e085968b3cc1e3cb653, jobId:
20187027da79fcaf5f43c0d7147c39e3).\n'
INFO:apache_beam.utils.subprocess_server:b'[ForkJoinPool.commonPool-worker-9]
INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Removing
cache directory /tmp/flink-web-ui\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-7]
INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Shut down
complete.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Close JobManager
connection for job 20187027da79fcaf5f43c0d7147c39e3.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Shut
down cluster because application is in CANCELED, diagnostics
DispatcherResourceManagerComponent has been closed..\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-7]
INFO
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent
- Closing components.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Close JobManager
connection for job 20187027da79fcaf5f43c0d7147c39e3.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Stop job leader
service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-7]
INFO org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess
- Stopping SessionDispatcherLeaderProcess.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager -
Shutting down TaskExecutorLocalStateStoresManager.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping
dispatcher akka://flink/user/dispatcher.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopping all
currently running jobs of dispatcher akka://flink/user/dispatcher.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl -
Closing the SlotManager.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-8]
INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl -
Suspending the SlotManager.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO
org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator
- Shutting down back pressure request coordinator.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Stopped
dispatcher akka://flink/user/dispatcher.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl -
FileChannelManager removed spill file directory
/tmp/flink-io-e00b0d57-34e4-4234-97f3-ac32259ecf96\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.io.network.NettyShuffleEnvironment - Shutting
down the network environment and its components.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl -
FileChannelManager removed spill file directory
/tmp/flink-netty-shuffle-eeb28b41-8ba7-441e-a1fe-e3626384db5b\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.KvStateService - Shutting down the
kvState service and its components.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.JobLeaderService - Stop job leader
service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.filecache.FileCache - removed file cache
directory /tmp/flink-dist-cache-503dc156-47d0-4276-b839-00e102ec312b\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.taskexecutor.TaskExecutor - Stopped TaskExecutor
akka://flink/user/taskmanager_0.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-4]
INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC
service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-metrics-2] INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote
daemon.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-metrics-2] INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut
down; proceeding with flushing remote transports.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-metrics-2] INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting shut down.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-metrics-2] INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-metrics-2] INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.blob.PermanentBlobCache - Shutting down BLOB
cache\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.blob.TransientBlobCache - Shutting down BLOB
cache\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.blob.BlobServer - Stopped BLOB server at
0.0.0.0:39841\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-akka.actor.default-dispatcher-6]
INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC
service.\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-runner-job-invoker] INFO
org.apache.beam.sdk.io.LocalFileSystem - Ignoring failed deletion of file
/tmp/beam-temp3qwesqst/artifactsuo570qks/job_7f3108c8-cf3e-4c12-8daa-384b67c069d3/
which already does not exist: {}\n'
INFO:apache_beam.utils.subprocess_server:b'java.nio.file.NoSuchFileException:
/tmp/beam-temp3qwesqst/artifactsuo570qks/job_7f3108c8-cf3e-4c12-8daa-384b67c069d3\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
java.nio.file.Files.delete(Files.java:1126)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.sdk.io.LocalFileSystem.delete(LocalFileSystem.java:189)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:386)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.artifact.BeamFileSystemLegacyArtifactStagingService.removeArtifacts(BeamFileSystemLegacyArtifactStagingService.java:115)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:63)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:245)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.setState(JobInvocation.java:246)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.access$200(JobInvocation.java:48)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:112)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:98)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n'
INFO:apache_beam.utils.subprocess_server:b'\tat
java.lang.Thread.run(Thread.java:748)\n'
INFO:apache_beam.utils.subprocess_server:b'[flink-runner-job-invoker] INFO
org.apache.beam.runners.fnexecution.artifact.BeamFileSystemLegacyArtifactStagingService
- Removed dir
/tmp/beam-temp3qwesqst/artifactsuo570qks/job_7f3108c8-cf3e-4c12-8daa-384b67c069d3/\n'
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
ok
test_native_source (apache_beam.io.gcp.bigquery_read_it_test.ReadTests) ...
SKIP: This test doesn't work on these runners: ['PortableRunner', 'FlinkRunner']
INFO:apache_beam.io.gcp.bigquery_read_it_test:Deleting dataset
python_read_table_15870427446652 in project apache-beam-testing
----------------------------------------------------------------------
XML: nosetests-postCommitIT-flink-py37.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 4 tests in 47.682s
OK (SKIP=2)
DEBUG:root:Sending SIGINT to job_server
DEBUG:root:Sending SIGINT to job_server
> Task :sdks:python:test-suites:portable:py37:postCommitPy37
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:19.931Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 1 to 5.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:22.545Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:22.580Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 1, though goal was 5. This could
be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:19.822Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:19.855Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 1, though goal was 5. This could
be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:20.460Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:24.043Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:21.101Z:
JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:21.787Z:
JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:25.523Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:25.538Z:
JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:26.153Z:
JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-c.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:23.742Z:
JOB_MESSAGE_DETAILED: BigQuery export job progress:
"dataflow_job_6204624738914719698" observed total of 1 exported files thus far.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:23.788Z:
JOB_MESSAGE_BASIC: BigQuery export job finished:
"dataflow_job_6204624738914719698"
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:27.494Z:
JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric
descriptors and Stackdriver will not create new Dataflow custom metrics for
this job. Each unique user-defined metric name (independent of the DoFn in
which it is defined) produces a new metric descriptor. To delete old / unused
metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:28.043Z:
JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric
descriptors and Stackdriver will not create new Dataflow custom metrics for
this job. Each unique user-defined metric name (independent of the DoFn in
which it is defined) produces a new metric descriptor. To delete old / unused
metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:30.433Z:
JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric
descriptors and Stackdriver will not create new Dataflow custom metrics for
this job. Each unique user-defined metric name (independent of the DoFn in
which it is defined) produces a new metric descriptor. To delete old / unused
metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:30.559Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 2 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:30.596Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 2, though goal was 5. This could
be a quota issue.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:36.002Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 5 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:38.770Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 5 based on
the rate of progress in the currently running stage(s).
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:48.491Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:12:48.490Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:54.389Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:54.428Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:53.803Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:53.835Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:55.275Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:55.322Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:52.547Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:13:52.586Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:14:03.515Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:14:03.554Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:14:05.409Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
INFO:apache_beam.runners.dataflow.dataflow_runner:2020-04-16T13:14:05.443Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
Traceback (most recent call last):
File "setup.py", line 325, in <module>
'mypy': generate_protos_first(mypy),
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/setuptools/__init__.py";,>
line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/commands.py";,>
line 158, in run
TestProgram(argv=argv, config=self.__config)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/core.py";,>
line 121, in __init__
**extra_args)
File "/usr/lib/python3.7/unittest/main.py", line 101, in __init__
self.runTests()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/core.py";,>
line 207, in runTests
result = self.testRunner.run(self.test)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py";,>
line 396, in run
timeout=nexttimeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in
_callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Terminated
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=efc0b701-5e61-4a27-9096-dad2272e588e,
currentDir=<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 7621
log file: /home/jenkins/.gradle/daemon/5.2.1/daemon-7621.out.log
----- Last 20 lines from daemon log file - daemon-7621.out.log -----
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/core.py";,>
line 121, in __init__
**extra_args)
File "/usr/lib/python3.7/unittest/main.py", line 101, in __init__
self.runTests()
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/core.py";,>
line 207, in runTests
result = self.testRunner.run(self.test)
File
"<https://builds.apache.org/job/beam_PostCommit_Python37/ws/src/build/gradleenv/-1734967052/lib/python3.7/site-packages/nose/plugins/multiprocess.py";,>
line 396, in run
timeout=nexttimeout)
File "<string>", line 2, in get
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in
_callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Terminated
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
