See <https://builds.apache.org/job/beam_PostCommit_Python2/966/display/redirect?page=changes>
Changes: [kcweaver] [BEAM-8028] add spark_runner.py [kcweaver] [BEAM-7224] add py wordcount tests for Spark ------------------------------------------ [...truncated 1.84 MB...] root: INFO: 2019-11-12T10:35:22.079Z: JOB_MESSAGE_DETAILED: Fusing consumer ParseGameEventFn into DecodeString root: INFO: 2019-11-12T10:35:22.082Z: JOB_MESSAGE_DETAILED: Fusing consumer AddEventTimestamps into ParseGameEventFn root: INFO: 2019-11-12T10:35:22.085Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateTeamScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>) into CalculateTeamScores/LeaderboardTeamFixedWindows root: INFO: 2019-11-12T10:35:22.088Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream into CalculateTeamScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>) root: INFO: 2019-11-12T10:35:22.090Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets into CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream root: INFO: 2019-11-12T10:35:22.094Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/Combine into CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets root: INFO: 2019-11-12T10:35:22.097Z: JOB_MESSAGE_DETAILED: Fusing consumer TeamScoresDict into CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/Combine root: INFO: 2019-11-12T10:35:22.102Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/ConvertToRow into TeamScoresDict root: INFO: 2019-11-12T10:35:22.107Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination into WriteTeamScoreSums/ConvertToRow root: INFO: 2019-11-12T10:35:22.109Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination root: INFO: 2019-11-12T10:35:22.111Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds root: INFO: 2019-11-12T10:35:22.114Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys root: INFO: 2019-11-12T10:35:22.116Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/WriteStream into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) root: INFO: 2019-11-12T10:35:22.119Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/ReadStream root: INFO: 2019-11-12T10:35:22.121Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets root: INFO: 2019-11-12T10:35:22.123Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) root: INFO: 2019-11-12T10:35:22.126Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) into WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys root: INFO: 2019-11-12T10:35:22.129Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateUserScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>) into CalculateUserScores/LeaderboardUserGlobalWindows root: INFO: 2019-11-12T10:35:22.131Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream into CalculateUserScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>) root: INFO: 2019-11-12T10:35:22.133Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets into CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream root: INFO: 2019-11-12T10:35:22.136Z: JOB_MESSAGE_DETAILED: Fusing consumer CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/Combine into CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets root: INFO: 2019-11-12T10:35:22.138Z: JOB_MESSAGE_DETAILED: Fusing consumer FormatUserScoreSums into CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/Combine root: INFO: 2019-11-12T10:35:22.140Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/ConvertToRow into FormatUserScoreSums root: INFO: 2019-11-12T10:35:22.142Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination into WriteUserScoreSums/ConvertToRow root: INFO: 2019-11-12T10:35:22.144Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination root: INFO: 2019-11-12T10:35:22.146Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds root: INFO: 2019-11-12T10:35:22.149Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys root: INFO: 2019-11-12T10:35:22.151Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/WriteStream into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps) root: INFO: 2019-11-12T10:35:22.154Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/ReadStream root: INFO: 2019-11-12T10:35:22.157Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets root: INFO: 2019-11-12T10:35:22.159Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps) root: INFO: 2019-11-12T10:35:22.161Z: JOB_MESSAGE_DETAILED: Fusing consumer WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) into WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys root: INFO: 2019-11-12T10:35:22.177Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph. root: INFO: 2019-11-12T10:35:22.231Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps. root: INFO: 2019-11-12T10:35:22.281Z: JOB_MESSAGE_DEBUG: Assigning stage ids. root: INFO: 2019-11-12T10:35:22.754Z: JOB_MESSAGE_DEBUG: Executing wait step start2 root: INFO: 2019-11-12T10:35:22.820Z: JOB_MESSAGE_DEBUG: Starting worker pool setup. root: INFO: 2019-11-12T10:35:22.827Z: JOB_MESSAGE_BASIC: Starting 1 workers... root: INFO: 2019-11-12T10:35:25.444Z: JOB_MESSAGE_BASIC: Executing operation CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream+CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets+CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/Combine+TeamScoresDict+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/WriteStream root: INFO: 2019-11-12T10:35:25.444Z: JOB_MESSAGE_BASIC: Executing operation WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/ReadStream+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) root: INFO: 2019-11-12T10:35:25.444Z: JOB_MESSAGE_BASIC: Executing operation ReadPubSub/Read+DecodeString+ParseGameEventFn+AddEventTimestamps+CalculateUserScores/LeaderboardUserGlobalWindows+CalculateTeamScores/LeaderboardTeamFixedWindows+CalculateTeamScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>)+CalculateTeamScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream+CalculateUserScores/ExtractAndSumScore/Map(<lambda at leader_board.py:151>)+CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/WriteStream root: INFO: 2019-11-12T10:35:25.454Z: JOB_MESSAGE_BASIC: Executing operation CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/ReadStream+CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/GroupByKey/MergeBuckets+CalculateUserScores/ExtractAndSumScore/CombinePerKey(sum)/Combine+FormatUserScoreSums+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)+WriteUserScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/WriteStream root: INFO: 2019-11-12T10:35:25.454Z: JOB_MESSAGE_BASIC: Executing operation WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/ReadStream+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/MergeBuckets+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys+WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn) root: INFO: 2019-11-12T10:35:50.817Z: JOB_MESSAGE_DEBUG: Executing input step topology_init_attach_disk_input_step root: INFO: 2019-11-12T10:35:50.822Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account. root: INFO: 2019-11-12T10:35:51.646Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-4 in us-central1-f. root: INFO: 2019-11-12T10:35:54.572Z: JOB_MESSAGE_WARNING: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete root: INFO: 2019-11-12T10:36:08.692Z: JOB_MESSAGE_DETAILED: Workers have started successfully. root: INFO: 2019-11-12T10:41:50.817Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account. root: WARNING: Timing out on waiting for job 2019-11-12_02_35_16-14430510704963818584 after 604 seconds root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_users` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/queries/ceaf879b-db52-4c4e-ae93-2cf8b18cd29c?timeoutMs=10000&location=US&maxResults=0 HTTP/1.1" 200 None urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/jobs/ceaf879b-db52-4c4e-ae93-2cf8b18cd29c?location=US HTTP/1.1" 200 None urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "GET /bigquery/v2/projects/apache-beam-testing/datasets/_7357fab0f784d2a7327ddbe81cdd1f4ca7e429cd/tables/anon3aec3d30_0edd_44d4_80d8_bd3a7f0cafdb/data HTTP/1.1" 200 None root: INFO: Read from given query (SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_users` WHERE total_score=5000 LIMIT 1), total rows 1 root: INFO: Generate checksum: de00231fe6730b972c0ff60a99988438911cda53 root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None root: WARNING: Retry with exponential backoff: waiting for 4.72777933568 seconds before retrying _query_with_retry because we caught exception: NotFound: 404 Not found: Table apache-beam-testing:leader_board_it_dataset1573554899589.leader_board_teams was not found in location US Traceback for above exception (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 206, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py";,> line 102, in _query_with_retry rows = query_job.result(timeout=60) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 2877, in result super(QueryJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 733, in result return super(_AsyncJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/future/polling.py";,> line 127, in result raise self._exception root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None root: WARNING: Retry with exponential backoff: waiting for 9.65171714922 seconds before retrying _query_with_retry because we caught exception: NotFound: 404 Not found: Table apache-beam-testing:leader_board_it_dataset1573554899589.leader_board_teams was not found in location US Traceback for above exception (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 206, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py";,> line 102, in _query_with_retry rows = query_job.result(timeout=60) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 2877, in result super(QueryJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 733, in result return super(_AsyncJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/future/polling.py";,> line 127, in result raise self._exception root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None root: WARNING: Retry with exponential backoff: waiting for 14.6870256662 seconds before retrying _query_with_retry because we caught exception: NotFound: 404 Not found: Table apache-beam-testing:leader_board_it_dataset1573554899589.leader_board_teams was not found in location US Traceback for above exception (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 206, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py";,> line 102, in _query_with_retry rows = query_job.result(timeout=60) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 2877, in result super(QueryJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 733, in result return super(_AsyncJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/future/polling.py";,> line 127, in result raise self._exception root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None root: WARNING: Retry with exponential backoff: waiting for 26.4901855857 seconds before retrying _query_with_retry because we caught exception: NotFound: 404 Not found: Table apache-beam-testing:leader_board_it_dataset1573554899589.leader_board_teams was not found in location US Traceback for above exception (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 206, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py";,> line 102, in _query_with_retry rows = query_job.result(timeout=60) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 2877, in result super(QueryJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 733, in result return super(_AsyncJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/future/polling.py";,> line 127, in result raise self._exception root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None root: WARNING: Retry with exponential backoff: waiting for 73.6282804662 seconds before retrying _query_with_retry because we caught exception: NotFound: 404 Not found: Table apache-beam-testing:leader_board_it_dataset1573554899589.leader_board_teams was not found in location US Traceback for above exception (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 206, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py";,> line 102, in _query_with_retry rows = query_job.result(timeout=60) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 2877, in result super(QueryJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/cloud/bigquery/job.py";,> line 733, in result return super(_AsyncJob, self).result(timeout=timeout) File "<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/build/gradleenv/-194514014/local/lib/python2.7/site-packages/google/api_core/future/polling.py";,> line 127, in result raise self._exception root: INFO: Attempting to perform query SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset1573554899589.leader_board_teams` WHERE total_score=5000 LIMIT 1 to BQ google.auth.transport._http_client: DEBUG: Making request: GET http://169.254.169.254 google.auth.transport._http_client: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/project/project-id urllib3.util.retry: DEBUG: Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None) google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): metadata.google.internal:80 urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/default/?recursive=true HTTP/1.1" 200 144 google.auth.transport.requests: DEBUG: Making request: GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token urllib3.connectionpool: DEBUG: http://metadata.google.internal:80 "GET /computeMetadata/v1/instance/service-accounts/[email protected]/token HTTP/1.1" 200 181 urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443 urllib3.connectionpool: DEBUG: https://www.googleapis.com:443 "POST /bigquery/v2/projects/apache-beam-testing/jobs HTTP/1.1" 200 None --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- XML: nosetests-postCommitIT-df.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 45 tests in 3421.318s FAILED (SKIP=4, errors=1) > Task :sdks:python:test-suites:dataflow:py2:postCommitIT FAILED FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':sdks:python:test-suites:portable:py2:portableWordCountFlinkRunnerBatch'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Python2/ws/src/sdks/python/test-suites/dataflow/py2/build.gradle'> line: 85 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py2:postCommitIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 0m 8s 119 actionable tasks: 108 executed, 8 from cache, 3 up-to-date Publishing build scan... https://gradle.com/s/ucmylrbh53w7g Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
