[
https://issues.apache.org/jira/browse/BEAM-8446?focusedWorklogId=333685&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-333685
]
ASF GitHub Bot logged work on BEAM-8446:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Oct/19 19:15
Start Date: 24/Oct/19 19:15
Worklog Time Spent: 10m
Work Description: ttanay commented on pull request #9858: [BEAM-8446]
Adding a test checking the wait for BQ jobs
URL: https://github.com/apache/beam/pull/9858#discussion_r338746705
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_file_loads_test.py
##########
@@ -424,6 +424,87 @@ def test_records_traverse_transform_with_mocks(self):
assert_that(jobs,
equal_to([job_reference]), label='CheckJobs')
+ @mock.patch('time.sleep')
+ def test_wait_for_job_completion(self, sleep_mock):
+ logging.getLogger().setLevel(logging.INFO)
Review comment:
Reducing log level to INFO outputs unneccesary stuff like:
```
INFO:root:==================== <function fix_side_input_pcoll_coders at
0x7fbbfb2d3730> ====================
INFO:root:==================== <function lift_combiners at 0x7fbbfb2d37b8>
====================
INFO:root:==================== <function expand_sdf at 0x7fbbfb2d3840>
====================
INFO:root:==================== <function expand_gbk at 0x7fbbfb2d38c8>
====================
INFO:root:==================== <function sink_flattens at 0x7fbbfb2d39d8>
====================
INFO:root:==================== <function greedily_fuse at 0x7fbbfb2d3a60>
====================
INFO:root:==================== <function read_to_impulse at 0x7fbbfb2d3ae8>
====================
INFO:root:==================== <function impulse_to_input at 0x7fbbfb2d3b70>
====================
INFO:root:==================== <function inject_timer_pcollections at
0x7fbbfb2d3d08> ====================
INFO:root:==================== <function sort_stages at 0x7fbbfb2d3d90>
====================
INFO:root:==================== <function window_pcollection_coders at
0x7fbbfb2d3e18> ====================
INFO:root:Creating state cache with size 100
INFO:root:Created Worker handler
<apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at
0x7fbbfb2927b8> for environment urn: "beam:env:embedded_python:v1"
INFO:root:Running
(((ref_AppliedPTransform_assert_that/Create/Read_9)+(ref_AppliedPTransform_assert_that/Group/pair_with_0_13))+(assert_that/Group/Flatten/Transcode/0))+(assert_that/Group/Flatten/Write/0)
INFO:root:Running
(ref_AppliedPTransform_job_ref/Read_3)+(ref_PCollection_PCollection_1/Write)
INFO:root:Running
((((((ref_AppliedPTransform_Create/Read_5)+(ref_AppliedPTransform_ParDo(WaitForBQJobs)_6))+(ref_AppliedPTransform_assert_that/WindowInto(WindowIntoFn)_10))+(ref_AppliedPTransform_assert_that/ToVoidKey_11))+(ref_AppliedPTransform_assert_that/Group/pair_with_1_14))+(assert_that/Group/Flatten/Transcode/1))+(assert_that/Group/Flatten/Write/1)
INFO:root:Job status: <Mock name='mock.status' id='140445338423704'>
INFO:root:Job status: <Mock name='mock.status' id='140445338424376'>
INFO:root:Job status: <Mock name='mock.status' id='140445338424992'>
INFO:root:Running
(assert_that/Group/Flatten/Read)+(assert_that/Group/GroupByKey/Write)
INFO:root:Running
(((assert_that/Group/GroupByKey/Read)+(ref_AppliedPTransform_assert_that/Group/Map(_merge_tagged_vals_under_key)_20))+(ref_AppliedPTransform_assert_that/Unkey_21))+(ref_AppliedPTransform_assert_that/Match_22)
```
Can this be removed if it's not being used in the test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 333685)
Time Spent: 4h 20m (was: 4h 10m)
> apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests.test_big_query_write_new_types
> is flaky
> ---------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8446
> URL: https://issues.apache.org/jira/browse/BEAM-8446
> Project: Beam
> Issue Type: New Feature
> Components: test-failures
> Reporter: Boyuan Zhang
> Assignee: Pablo Estrada
> Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> test_big_query_write_new_types appears to be flaky in
> beam_PostCommit_Python37 test suite.
> https://builds.apache.org/job/beam_PostCommit_Python37/733/
> https://builds.apache.org/job/beam_PostCommit_Python37/739/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
