See
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/3445/display/redirect?page=changes>
Changes:
[heejong] [BEAM-11520] Stage extra PyPI dependencies with generated requirements
[heejong] raise exception for non-file type artifacts
------------------------------------------
[...truncated 61.02 MB...]
{
"@type":
"FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
"component_encodings": [],
"pipeline_proto_coder_id":
"ref_Coder_FastPrimitivesCoder_3"
},
{
"@type":
"FastPrimitivesCoder$eNprYE5OLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqYIfgYGhvi0xJycpMTk7HiwlkJ8pgVkJmfnpEJNYQGawlpbyJZUnKQHACYlLgM=",
"component_encodings": [],
"pipeline_proto_coder_id":
"ref_Coder_FastPrimitivesCoder_3"
}
],
"is_pair_like": true,
"pipeline_proto_coder_id": "ref_Coder_FastPrimitivesCoder_3"
},
{
"@type": "kind:global_window"
}
],
"is_wrapper": true
},
"output_name": "out",
"user_name": "write/BigQueryBatchFileLoads/Flatten.out"
}
],
"user_name": "write/BigQueryBatchFileLoads/Flatten"
}
}
],
"type": "JOB_TYPE_BATCH"
}
apache_beam.runners.dataflow.internal.apiclient: INFO: Create job: <Job
createTime: '2021-02-13T00:28:25.082153Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2021-02-12_16_28_23-1665012566232522298'
location: 'us-central1'
name: 'beamapp-jenkins-0213002815-204506'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2021-02-13T00:28:25.082153Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
apache_beam.runners.dataflow.internal.apiclient: INFO: Created job with id:
[2021-02-12_16_28_23-1665012566232522298]
apache_beam.runners.dataflow.internal.apiclient: INFO: Submitted job:
2021-02-12_16_28_23-1665012566232522298
apache_beam.runners.dataflow.internal.apiclient: INFO: To access the Dataflow
monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2021-02-12_16_28_23-1665012566232522298?project=apache-beam-testing
apache_beam.runners.dataflow.dataflow_runner: INFO: Job
2021-02-12_16_28_23-1665012566232522298 is in state JOB_STATE_RUNNING
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:27.260Z:
JOB_MESSAGE_DETAILED: Autoscaling is enabled for job
2021-02-12_16_28_23-1665012566232522298. The number of workers will be between
1 and 1000.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:27.401Z:
JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job
2021-02-12_16_28_23-1665012566232522298.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:31.823Z:
JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.575Z:
JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.603Z:
JOB_MESSAGE_DEBUG: Combiner lifting skipped for step
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables: GroupByKey not
followed by a combiner.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.666Z:
JOB_MESSAGE_DEBUG: Combiner lifting skipped for step
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations: GroupByKey not
followed by a combiner.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.700Z:
JOB_MESSAGE_DEBUG: Combiner lifting skipped for step
write/BigQueryBatchFileLoads/GroupShardedRows: GroupByKey not followed by a
combiner.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.733Z:
JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.798Z:
JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.910Z:
JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:33.958Z:
JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34Z:
JOB_MESSAGE_DETAILED: Unzipping flatten s23 for input s17.WrittenFiles
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.031Z:
JOB_MESSAGE_DETAILED: Fusing unzipped copy of
write/BigQueryBatchFileLoads/IdentityWorkaround, through flatten
write/BigQueryBatchFileLoads/DestinationFilesUnion, into producer
write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.060Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Write into
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.102Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/GroupByWindow into
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.140Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles) into
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/GroupByWindow
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.177Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/ParDo(TriggerLoadJobs)
into write/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.236Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/TriggerLoadJobsWithoutTempTables
into write/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.266Z:
JOB_MESSAGE_DETAILED: Unzipping flatten s23-u32 for input s24.None-c30
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.302Z:
JOB_MESSAGE_DETAILED: Fusing unzipped copy of
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify, through
flatten write/BigQueryBatchFileLoads/DestinationFilesUnion/Unzipped-1, into
producer write/BigQueryBatchFileLoads/IdentityWorkaround
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.330Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/IdentityWorkaround into
write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/WriteGroupedRecordsToFile
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.368Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify into
write/BigQueryBatchFileLoads/IdentityWorkaround
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.394Z:
JOB_MESSAGE_DETAILED: Fusing consumer
read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough) into
read/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.423Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RewindowIntoGlobal into
read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.458Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/AppendDestination into
write/BigQueryBatchFileLoads/RewindowIntoGlobal
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.492Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)
into write/BigQueryBatchFileLoads/AppendDestination
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.529Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/ParDo(_ShardDestinations) into
write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.563Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupShardedRows/Reify into
write/BigQueryBatchFileLoads/ParDo(_ShardDestinations)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.595Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupShardedRows/Write into
write/BigQueryBatchFileLoads/GroupShardedRows/Reify
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.665Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GroupShardedRows/GroupByWindow into
write/BigQueryBatchFileLoads/GroupShardedRows/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.724Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/DropShardNumber into
write/BigQueryBatchFileLoads/GroupShardedRows/GroupByWindow
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.757Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/WriteGroupedRecordsToFile
into write/BigQueryBatchFileLoads/DropShardNumber
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.792Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/LoadJobNamePrefix into
write/BigQueryBatchFileLoads/ImpulseSingleElementPC/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.826Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/CopyJobNamePrefix into
write/BigQueryBatchFileLoads/ImpulseSingleElementPC/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.864Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/GenerateFilePrefix into
write/BigQueryBatchFileLoads/ImpulseSingleElementPC/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.897Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/WaitForTempTableLoadJobs
into write/BigQueryBatchFileLoads/ImpulseMonitorLoadJobs/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.926Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/ParDo(TriggerCopyJobs) into
write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/WaitForTempTableLoadJobs
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.964Z:
JOB_MESSAGE_DETAILED: Fusing consumer read/MapFilesToRemove into
read/FilesToRemoveImpulse/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:34.993Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs into
write/BigQueryBatchFileLoads/ImpulseMonitorCopyJobs/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.014Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/PassTables into
write/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.048Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/AddUselessValue into
write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/PassTables
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.086Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Reify into
write/BigQueryBatchFileLoads/RemoveTempTables/AddUselessValue
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.114Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Write into
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Reify
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.148Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/GroupByWindow
into write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.173Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/GetTableNames into
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/GroupByWindow
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.202Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/RemoveTempTables/Delete into
write/BigQueryBatchFileLoads/RemoveTempTables/GetTableNames
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.232Z:
JOB_MESSAGE_DETAILED: Fusing consumer
write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/WaitForDestinationLoadJobs
into write/BigQueryBatchFileLoads/ImpulseMonitorDestLoadJobs/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.266Z:
JOB_MESSAGE_DETAILED: Fusing consumer
read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/ParDo(RemoveExtractedFiles)
into read/_PassThroughThenCleanup/Create/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.306Z:
JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.346Z:
JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.372Z:
JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.406Z:
JOB_MESSAGE_DEBUG: Assigning stage ids.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.566Z:
JOB_MESSAGE_DEBUG: Executing wait step start49
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.670Z:
JOB_MESSAGE_BASIC: Executing operation
read/FilesToRemoveImpulse/Read+read/MapFilesToRemove
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.696Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ImpulseSingleElementPC/Read+write/BigQueryBatchFileLoads/LoadJobNamePrefix+write/BigQueryBatchFileLoads/CopyJobNamePrefix+write/BigQueryBatchFileLoads/GenerateFilePrefix
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.721Z:
JOB_MESSAGE_DEBUG: Starting worker pool setup.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.733Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ImpulseEmptyPC/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.745Z:
JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.767Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ImpulseEmptyPC/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:28:35.840Z:
JOB_MESSAGE_DEBUG: Value "write/BigQueryBatchFileLoads/ImpulseEmptyPC/Read.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:29:05.212Z:
JOB_MESSAGE_BASIC: Your project already contains 100 Dataflow-created metric
descriptors, so new user metrics of the form custom.googleapis.com/* will not
be created. However, all user metrics are also available in the metric
dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics,
you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:29:12.959Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running stage(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:35:47.706Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:35:47.850Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.197Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ImpulseSingleElementPC/Read+write/BigQueryBatchFileLoads/LoadJobNamePrefix+write/BigQueryBatchFileLoads/CopyJobNamePrefix+write/BigQueryBatchFileLoads/GenerateFilePrefix
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.270Z:
JOB_MESSAGE_DEBUG: Value "write/BigQueryBatchFileLoads/LoadJobNamePrefix.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.303Z:
JOB_MESSAGE_DEBUG: Value "write/BigQueryBatchFileLoads/CopyJobNamePrefix.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.344Z:
JOB_MESSAGE_DEBUG: Value "write/BigQueryBatchFileLoads/GenerateFilePrefix.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.529Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/_UnpickledSideInput(LoadJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.582Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/_UnpickledSideInput(LoadJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.587Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/_UnpickledSideInput(LoadJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.609Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/_UnpickledSideInput(GenerateFilePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.628Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/_UnpickledSideInput(LoadJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.637Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/_UnpickledSideInput(GenerateFilePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.656Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/_UnpickledSideInput(GenerateFilePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.667Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/_UnpickledSideInput(CopyJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.692Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/_UnpickledSideInput(LoadJobNamePrefix.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.703Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/_UnpickledSideInput(GenerateFilePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.709Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/_UnpickledSideInput(CopyJobNamePrefix.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.725Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/_UnpickledSideInput(LoadJobNamePrefix.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.755Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/_UnpickledSideInput(GenerateFilePrefix.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.785Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/_UnpickledSideInput(CopyJobNamePrefix.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.814Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/_UnpickledSideInput(GenerateFilePrefix.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:08.861Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupShardedRows/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.189Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupShardedRows/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.275Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/GroupShardedRows/Session" materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.362Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.483Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.568Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Session"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:09.641Z:
JOB_MESSAGE_BASIC: Executing operation
read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+write/BigQueryBatchFileLoads/RewindowIntoGlobal+write/BigQueryBatchFileLoads/AppendDestination+write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)+write/BigQueryBatchFileLoads/IdentityWorkaround+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Write+write/BigQueryBatchFileLoads/ParDo(_ShardDestinations)+write/BigQueryBatchFileLoads/GroupShardedRows/Reify+write/BigQueryBatchFileLoads/GroupShardedRows/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:11.494Z:
JOB_MESSAGE_BASIC: Finished operation
read/FilesToRemoveImpulse/Read+read/MapFilesToRemove
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:11.585Z:
JOB_MESSAGE_DEBUG: Value "read/MapFilesToRemove.out" materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:11.663Z:
JOB_MESSAGE_BASIC: Executing operation
read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(MapFilesToRemove.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:11.718Z:
JOB_MESSAGE_BASIC: Finished operation
read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(MapFilesToRemove.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:11.798Z:
JOB_MESSAGE_DEBUG: Value
"read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(MapFilesToRemove.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.254Z:
JOB_MESSAGE_BASIC: Finished operation
read/Read+read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough)+write/BigQueryBatchFileLoads/RewindowIntoGlobal+write/BigQueryBatchFileLoads/AppendDestination+write/BigQueryBatchFileLoads/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)/ParDo(WriteRecordsToFile)+write/BigQueryBatchFileLoads/IdentityWorkaround+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Write+write/BigQueryBatchFileLoads/ParDo(_ShardDestinations)+write/BigQueryBatchFileLoads/GroupShardedRows/Reify+write/BigQueryBatchFileLoads/GroupShardedRows/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.314Z:
JOB_MESSAGE_DEBUG: Value
"read/_PassThroughThenCleanup/ParDo(PassThrough)/ParDo(PassThrough).cleanup_signal"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.337Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupShardedRows/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.374Z:
JOB_MESSAGE_BASIC: Executing operation
read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(ParDo(PassThrough).cleanup_signal.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.388Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupShardedRows/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.408Z:
JOB_MESSAGE_BASIC: Finished operation
read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(ParDo(PassThrough).cleanup_signal.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.440Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupShardedRows/Read+write/BigQueryBatchFileLoads/GroupShardedRows/GroupByWindow+write/BigQueryBatchFileLoads/DropShardNumber+write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/WriteGroupedRecordsToFile+write/BigQueryBatchFileLoads/IdentityWorkaround+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.487Z:
JOB_MESSAGE_DEBUG: Value
"read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/_UnpickledSideInput(ParDo(PassThrough).cleanup_signal.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:31.567Z:
JOB_MESSAGE_BASIC: Executing operation
read/_PassThroughThenCleanup/Create/Read+read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/ParDo(RemoveExtractedFiles)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:37.419Z:
JOB_MESSAGE_BASIC: Finished operation
read/_PassThroughThenCleanup/Create/Read+read/_PassThroughThenCleanup/ParDo(RemoveExtractedFiles)/ParDo(RemoveExtractedFiles)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:40.664Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupShardedRows/Read+write/BigQueryBatchFileLoads/GroupShardedRows/GroupByWindow+write/BigQueryBatchFileLoads/DropShardNumber+write/BigQueryBatchFileLoads/WriteGroupedRecordsToFile/WriteGroupedRecordsToFile+write/BigQueryBatchFileLoads/IdentityWorkaround+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Reify+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:40.719Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:40.768Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:40.816Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/GroupByWindow+write/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/ParDo(TriggerLoadJobs)+write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/TriggerLoadJobsWithoutTempTables
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.058Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/Read+write/BigQueryBatchFileLoads/GroupFilesByTableDestinations/GroupByWindow+write/BigQueryBatchFileLoads/ParDo(PartitionFiles)/ParDo(PartitionFiles)+write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs)/ParDo(TriggerLoadJobs)+write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/TriggerLoadJobsWithoutTempTables
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.113Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs).out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.138Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/TriggerLoadJobsWithTempTables/ParDo(TriggerLoadJobs).TemporaryTables"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.168Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.194Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/_UnpickledSideInput(ParDo(TriggerLoadJobs).out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.224Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/_UnpickledSideInput(ParDo(TriggerLoadJobs).TemporaryTables.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.239Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/_UnpickledSideInput(ParDo(TriggerLoadJobs).out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.242Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/_UnpickledSideInput(TriggerLoadJobsWithoutTempTables.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.261Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/_UnpickledSideInput(ParDo(TriggerLoadJobs).TemporaryTables.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.267Z:
JOB_MESSAGE_BASIC: Executing operation write/BigQueryBatchFileLoads/Flatten
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.287Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/_UnpickledSideInput(TriggerLoadJobsWithoutTempTables.out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.294Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/_UnpickledSideInput(ParDo(TriggerLoadJobs).out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.320Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/_UnpickledSideInput(ParDo(TriggerLoadJobs).TemporaryTables.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.322Z:
JOB_MESSAGE_BASIC: Finished operation write/BigQueryBatchFileLoads/Flatten
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.351Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/_UnpickledSideInput(TriggerLoadJobsWithoutTempTables.out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.376Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ImpulseMonitorLoadJobs/Read+write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/WaitForTempTableLoadJobs+write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/ParDo(TriggerCopyJobs)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.396Z:
JOB_MESSAGE_DEBUG: Value "write/BigQueryBatchFileLoads/Flatten.out"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:42:53.461Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ImpulseMonitorDestLoadJobs/Read+write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/WaitForDestinationLoadJobs
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.674Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ImpulseMonitorLoadJobs/Read+write/BigQueryBatchFileLoads/WaitForTempTableLoadJobs/WaitForTempTableLoadJobs+write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs)/ParDo(TriggerCopyJobs)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.731Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/ParDo(TriggerCopyJobs).out" materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.783Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/WaitForCopyJobs/_UnpickledSideInput(ParDo(TriggerCopyJobs).out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.823Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/WaitForCopyJobs/_UnpickledSideInput(ParDo(TriggerCopyJobs).out.0)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.865Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/WaitForCopyJobs/_UnpickledSideInput(ParDo(TriggerCopyJobs).out.0).output"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:02.919Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:03.061Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:03.112Z:
JOB_MESSAGE_DEBUG: Value
"write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Session"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:03.156Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/ImpulseMonitorCopyJobs/Read+write/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs+write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/PassTables+write/BigQueryBatchFileLoads/RemoveTempTables/AddUselessValue+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Reify+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:05.650Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ImpulseMonitorDestLoadJobs/Read+write/BigQueryBatchFileLoads/WaitForDestinationLoadJobs/WaitForDestinationLoadJobs
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:11.993Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/ImpulseMonitorCopyJobs/Read+write/BigQueryBatchFileLoads/WaitForCopyJobs/WaitForCopyJobs+write/BigQueryBatchFileLoads/RemoveTempTables/PassTables/PassTables+write/BigQueryBatchFileLoads/RemoveTempTables/AddUselessValue+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Reify+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:12.044Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:12.097Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:12.152Z:
JOB_MESSAGE_BASIC: Executing operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Read+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/GroupByWindow+write/BigQueryBatchFileLoads/RemoveTempTables/GetTableNames+write/BigQueryBatchFileLoads/RemoveTempTables/Delete
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:15.276Z:
JOB_MESSAGE_BASIC: Finished operation
write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/Read+write/BigQueryBatchFileLoads/RemoveTempTables/DeduplicateTables/GroupByWindow+write/BigQueryBatchFileLoads/RemoveTempTables/GetTableNames+write/BigQueryBatchFileLoads/RemoveTempTables/Delete
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:15.323Z:
JOB_MESSAGE_DEBUG: Executing success step success47
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:15.385Z:
JOB_MESSAGE_DETAILED: Cleaning up.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:15.427Z:
JOB_MESSAGE_DEBUG: Starting worker pool teardown.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2021-02-13T00:43:15.473Z:
JOB_MESSAGE_BASIC: Stopping worker pool...
apache_beam.runners.dataflow.dataflow_runner: WARNING: Timing out on waiting
for job 2021-02-12_16_28_23-1665012566232522298 after 903 seconds
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df-py37.xml
----------------------------------------------------------------------
XML:
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 68 tests in 5155.841s
FAILED (SKIP=6, errors=1, failures=1)
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 197
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:postCommitPy37IT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 118
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 29m 39s
213 actionable tasks: 153 executed, 56 from cache, 4 up-to-date
Gradle was unable to watch the file system for changes. The inotify watches
limit is too low.
Publishing build scan...
https://gradle.com/s/lx77jwi2w5p22
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
