See
<https://builds.apache.org/job/beam_PostCommit_Python2/2033/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-9565] Fix threading issue with WatermarkEstimatorsTest
------------------------------------------
[...truncated 11.16 MB...]
"output_name": "None",
"step_name": "s10"
},
"serialized_fn": "<string of 1508 bytes>",
"user_name": "WriteTeamScoreSums/ConvertToRow"
}
},
{
"kind": "ParallelWrite",
"name": "s12",
"properties": {
"create_disposition": "CREATE_IF_NEEDED",
"dataset": "hourly_team_score_it_dataset15849822606696",
"display_data": [],
"encoding": {
"@type": "kind:windowed_value",
"component_encodings": [
{
"@type":
"RowAsDictJsonCoder$eNprYEpOLEhMzkiNT0pNzNXLzNdLTy7QS8pMLyxNLaqML8nPzynmCsovdyx2yUwu8SrOz3POT0kt4ipk0GwsZKwtZErSAwBKpRfo",
"component_encodings": [],
"pipeline_proto_coder_id": "ref_Coder_RowAsDictJsonCoder_10"
},
{
"@type": "kind:global_window"
}
],
"is_wrapper": true
},
"format": "bigquery",
"parallel_input": {
"@type": "OutputReference",
"output_name": "None",
"step_name": "s11"
},
"project": "apache-beam-testing",
"schema": "{\"fields\": [{\"type\": \"INTEGER\", \"name\":
\"total_score\", \"mode\": \"NULLABLE\"}, {\"type\": \"STRING\", \"name\":
\"window_start\", \"mode\": \"NULLABLE\"}, {\"type\": \"STRING\", \"name\":
\"team\", \"mode\": \"NULLABLE\"}]}",
"table": "leader_board",
"user_name":
"WriteTeamScoreSums/WriteToBigQuery/WriteToBigQuery/NativeWrite",
"write_disposition": "WRITE_APPEND"
}
}
],
"type": "JOB_TYPE_BATCH"
}
apache_beam.runners.dataflow.internal.apiclient: INFO: Create job: <Job
createTime: u'2020-03-23T16:51:24.105476Z'
currentStateTime: u'1970-01-01T00:00:00Z'
id: u'2020-03-23_09_51_22-9079552427073905578'
location: u'us-central1'
name: u'beamapp-jenkins-0323165102-247839'
projectId: u'apache-beam-testing'
stageStates: []
startTime: u'2020-03-23T16:51:24.105476Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
apache_beam.runners.dataflow.internal.apiclient: INFO: Created job with id:
[2020-03-23_09_51_22-9079552427073905578]
apache_beam.runners.dataflow.internal.apiclient: INFO: To access the Dataflow
monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_22-9079552427073905578?project=apache-beam-testing
apache_beam.runners.dataflow.dataflow_runner: INFO: Job
2020-03-23_09_51_22-9079552427073905578 is in state JOB_STATE_RUNNING
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:22.821Z:
JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job
2020-03-23_09_51_22-9079552427073905578.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:22.821Z:
JOB_MESSAGE_DETAILED: Autoscaling is enabled for job
2020-03-23_09_51_22-9079552427073905578. The number of workers will be between
1 and 1000.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:26.378Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:27.205Z:
JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-c.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:27.936Z:
JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.058Z:
JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.084Z:
JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.152Z:
JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.206Z:
JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.226Z:
JOB_MESSAGE_DETAILED: Fusing consumer HourlyTeamScore/ParseGameEventFn into
ReadInputText/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.254Z:
JOB_MESSAGE_DETAILED: Fusing consumer HourlyTeamScore/FilterStartTime into
HourlyTeamScore/ParseGameEventFn
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.278Z:
JOB_MESSAGE_DETAILED: Fusing consumer HourlyTeamScore/FilterEndTime into
HourlyTeamScore/FilterStartTime
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.305Z:
JOB_MESSAGE_DETAILED: Fusing consumer HourlyTeamScore/AddEventTimestamps into
HourlyTeamScore/FilterEndTime
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.336Z:
JOB_MESSAGE_DETAILED: Fusing consumer HourlyTeamScore/FixedWindowsTeam into
HourlyTeamScore/AddEventTimestamps
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.369Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/Map(<lambda at hourly_team_score.py:145>)
into HourlyTeamScore/FixedWindowsTeam
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.404Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Partial
into HourlyTeamScore/ExtractAndSumScore/Map(<lambda at
hourly_team_score.py:145>)
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.578Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Reify into
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Partial
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.673Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Write into
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Reify
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.796Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine into
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Read
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:28.945Z:
JOB_MESSAGE_DETAILED: Fusing consumer
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Extract into
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.004Z:
JOB_MESSAGE_DETAILED: Fusing consumer TeamScoresDict into
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Extract
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.035Z:
JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/ConvertToRow into
TeamScoresDict
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.092Z:
JOB_MESSAGE_DETAILED: Fusing consumer
WriteTeamScoreSums/WriteToBigQuery/WriteToBigQuery/NativeWrite into
WriteTeamScoreSums/ConvertToRow
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.151Z:
JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.183Z:
JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.208Z:
JOB_MESSAGE_DEBUG: Adding workflow start and stop steps.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.231Z:
JOB_MESSAGE_DEBUG: Assigning stage ids.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.390Z:
JOB_MESSAGE_DEBUG: Executing wait step start22
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.456Z:
JOB_MESSAGE_BASIC: Executing operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.499Z:
JOB_MESSAGE_DEBUG: Starting worker pool setup.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.521Z:
JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-c...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.569Z:
JOB_MESSAGE_BASIC: Finished operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Create
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.638Z:
JOB_MESSAGE_DEBUG: Value
"HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Session"
materialized.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:29.702Z:
JOB_MESSAGE_BASIC: Executing operation
ReadInputText/Read+HourlyTeamScore/ParseGameEventFn+HourlyTeamScore/FilterStartTime+HourlyTeamScore/FilterEndTime+HourlyTeamScore/AddEventTimestamps+HourlyTeamScore/FixedWindowsTeam+HourlyTeamScore/ExtractAndSumScore/Map(<lambda
at
hourly_team_score.py:145>)+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Partial+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Reify+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:40.694Z:
JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric
descriptors and Stackdriver will not create new Dataflow custom metrics for
this job. Each unique user-defined metric name (independent of the DoFn in
which it is defined) produces a new metric descriptor. To delete old / unused
metric descriptors see
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:51:52.658Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 1 to 5.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:52:02.534Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:52:02.567Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 1, though goal was 5. This could
be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:52:13.384Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 4 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:52:13.409Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 4, though goal was 5. This could
be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:52:18.810Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 5 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:53:38.966Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:53:38.995Z:
JOB_MESSAGE_DETAILED: Workers have started successfully.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:57:29.358Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:36.113Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 5 to 64.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:42.043Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 11 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:42.068Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 11, though goal was 64. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:47.725Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 50 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:47.748Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 50, though goal was 64. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:53.147Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 62 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:53.173Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 62, though goal was 64. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:58.586Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 63 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:58:58.615Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 63, though goal was 64. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T16:59:04.080Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 64 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:03:29.359Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:04:36.254Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 64 to 92.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:04:42.082Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 84 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:04:42.148Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 84, though goal was 92. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:04:47.701Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 92 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:08:05.390Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 92 to 102.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:08:11.122Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 100 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:08:11.168Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 100, though goal was 102. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:08:16.617Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 102 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:09.811Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 102 to 113.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:15.727Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 110 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:15.760Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 110, though goal was 113. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:21.133Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 112 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:21.169Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 112, though goal was 113. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:29.359Z:
JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service
Account.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:09:32.098Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 113 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:06.754Z:
JOB_MESSAGE_BASIC: Autoscaling: Resizing worker pool from 113 to 128.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:12.658Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 125 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:12.693Z:
JOB_MESSAGE_DETAILED: Resized worker pool to 125, though goal was 128. This
could be a quota issue.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:18.128Z:
JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 128 based on
the rate of progress in the currently running step(s).
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:41.774Z:
JOB_MESSAGE_BASIC: Finished operation
ReadInputText/Read+HourlyTeamScore/ParseGameEventFn+HourlyTeamScore/FilterStartTime+HourlyTeamScore/FilterEndTime+HourlyTeamScore/AddEventTimestamps+HourlyTeamScore/FixedWindowsTeam+HourlyTeamScore/ExtractAndSumScore/Map(<lambda
at
hourly_team_score.py:145>)+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Partial+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Reify+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Write
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:41.884Z:
JOB_MESSAGE_BASIC: Executing operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:41.948Z:
JOB_MESSAGE_BASIC: Finished operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Close
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:11:42.020Z:
JOB_MESSAGE_BASIC: Executing operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Read+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Extract+TeamScoresDict+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/WriteToBigQuery/WriteToBigQuery/NativeWrite
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:20.894Z:
JOB_MESSAGE_BASIC: Executing BigQuery import job
"dataflow_job_7360605193662348259". You can check its status with the bq tool:
"bq show -j --project_id=apache-beam-testing dataflow_job_7360605193662348259".
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:31.391Z:
JOB_MESSAGE_BASIC: BigQuery import job "dataflow_job_7360605193662348259" done.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:32.040Z:
JOB_MESSAGE_BASIC: Finished operation
HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/Read+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine+HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Combine/Extract+TeamScoresDict+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/WriteToBigQuery/WriteToBigQuery/NativeWrite
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:32.139Z:
JOB_MESSAGE_DEBUG: Executing success step success20
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:32.308Z:
JOB_MESSAGE_DETAILED: Cleaning up.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:32.396Z:
JOB_MESSAGE_DEBUG: Starting worker pool teardown.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:12:32.449Z:
JOB_MESSAGE_BASIC: Stopping worker pool...
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:14:03.154Z:
JOB_MESSAGE_DETAILED: Autoscaling: Resized worker pool from 128 to 0.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:14:03.197Z:
JOB_MESSAGE_BASIC: Worker pool stopped.
apache_beam.runners.dataflow.dataflow_runner: INFO: 2020-03-23T17:14:03.272Z:
JOB_MESSAGE_DEBUG: Tearing down pending resources...
apache_beam.runners.dataflow.dataflow_runner: INFO: Job
2020-03-23_09_51_22-9079552427073905578 is in state JOB_STATE_DONE
apache_beam.io.gcp.tests.bigquery_matcher: INFO: Attempting to perform query
SELECT COUNT(*) FROM
`apache-beam-testing.hourly_team_score_it_dataset15849822606696.leader_board`
to BQ
google.auth.transport._http_client: DEBUG: Making request: GET
http://169.254.169.254
google.auth.transport._http_client: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/project/project-id
urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3,
connect=None, read=None, redirect=None, status=None)
google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
metadata.google.internal:80
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1"
200 144
google.auth.transport.requests: DEBUG: Making request: GET
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token
urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET
/computeMetadata/v1/instance/service-accounts/[email protected]/token
HTTP/1.1" 200 192
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1):
bigquery.googleapis.com:443
urllib3.connectionpool: DEBUG: https://bigquery.googleapis.com:443 "POST
/bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None
urllib3.connectionpool: DEBUG: https://bigquery.googleapis.com:443 "GET
/bigquery/v2/projects/apache-beam-testing/queries/751446a9-1d9e-44a3-a7bb-fc45eff916dc?location=US&maxResults=0
HTTP/1.1" 200 None
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
XML: nosetests-postCommitIT-df.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 58 tests in 4414.102s
FAILED (SKIP=7, errors=1)
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_22-9079552427073905578?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_15_40-9666469317242818194?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_24_21-17023161128569984063?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_32_10-13985271234268222278?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_40_31-10908127975222873962?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_48_49-16775306836672734659?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_56_35-1728061502525706968?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_28-14974151049762729363?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_05_30-8056386869418929019?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_14_20-5343366435902163140?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_22_48-16599447466239984591?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_31_35-4866979683194004758?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_39_45-15511413888030280563?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_54_28-8155223170794264048?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_06_39-656862629249991780?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_16_02-978767775898611935?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_25_30-9407394839629094529?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_34_45-9611878040360217314?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_42_44-15931549516798431885?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_22-6450321808503645178?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_09_52-13406880146655789023?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_19_04-17318963175414134542?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_26_44-4105604641000012954?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_34_34-1459887918511754691?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_42_11-11783396120028265051?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_23-17280660826133444992?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_00_13-17448926865026792061?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_09_52-774430465864758022?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_18_32-13429369950359630930?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_35_41-4878865978781628572?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_22-16728081634042092213?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_59_40-1733898178482306642?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_08_21-4293540177466461934?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_16_33-1958932073233082821?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_24_54-2069302617559755268?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_32_02-15129751918504992994?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_39_19-3469287094892206656?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_47_34-15679130204136827600?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_24-11552023901148894446?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_00_03-10464945414838846809?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_09_31-5108906555777972016?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_17_28-17117596239750855338?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_26_00-11752084871093962105?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_34_46-1342903174524794041?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_09_51_23-2284284509053966589?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_01_32-15768239938313834416?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_13_47-208488377572285502?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_22_17-17791451585646557278?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_29_52-17396698806401452840?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-23_10_37_53-3693622022237279149?project=apache-beam-testing
> Task :sdks:python:test-suites:dataflow:py2:postCommitIT FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'>
line: 85
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py2:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 15m 14s
126 actionable tasks: 102 executed, 21 from cache, 3 up-to-date
Publishing build scan...
https://gradle.com/s/3luehexvi6hc6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
