[
https://issues.apache.org/jira/browse/BEAM-11476?focusedWorklogId=546323&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-546323
]
ASF GitHub Bot logged work on BEAM-11476:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Feb/21 00:07
Start Date: 03/Feb/21 00:07
Worklog Time Spent: 10m
Work Description: KevinGG commented on pull request #13881:
URL: https://github.com/apache/beam/pull/13881#issuecomment-772102411
If `cleanup` is invoked more than once, the extra invocation must have come
from the `ie.new_env()` when the current environment instance is not None. The
old code patches `cleanup` with the decorator. It seems that when running tests
on Jenkins, sometimes, even setting `ie._interactive_beam_env = None`
forcefully wouldn't guarantee the module property is `None` when the subsequent
statements in the same test method are executed.
In that case, the test should be resilient to global state of
`ie._interactive_beam_env`. By moving patches inside each test function after
`ie.new_env()`, the test mock is not affected by whether `ie.new_env()`
invoking `cleanup` or not. And the assertions can focus on testing the logic
within each test.
Couldn't reproduce the flakiness locally, but tested with 4 CPU cores
running concurrently for 100 flake runs:
```
pytest --flake-finder --flake-runs=100 -n 4
apache_beam/runners/interactive/interactive_environment_test.py
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 546323)
Time Spent: 20m (was: 10m)
> flaky test: test_track_user_pipeline_cleanup_non_inspectable_pipeline
> ---------------------------------------------------------------------
>
> Key: BEAM-11476
> URL: https://issues.apache.org/jira/browse/BEAM-11476
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Udi Meiri
> Assignee: Ning Kang
> Priority: P1
> Labels: currently-failing, flake
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Test name:
> apache_beam.runners.interactive.interactive_environment_test.InteractiveEnvironmentTest.test_track_user_pipeline_cleanup_non_inspectable_pipeline
> {code}
> Error Message
> AssertionError: Expected 'cleanup' to not have been called. Called 1 times.
> Stacktrace
> self =
> <apache_beam.runners.interactive.interactive_environment_test.InteractiveEnvironmentTest
> testMethod=test_track_user_pipeline_cleanup_non_inspectable_pipeline>
> mocked_cleanup = <MagicMock name='cleanup' id='140386685111432'>
> @patch(
> 'apache_beam.runners.interactive.interactive_environment'
> '.InteractiveEnvironment.cleanup')
> def test_track_user_pipeline_cleanup_non_inspectable_pipeline(
> self, mocked_cleanup):
> ie._interactive_beam_env = None
> ie.new_env()
> dummy_pipeline_1 = beam.Pipeline()
> dummy_pipeline_2 = beam.Pipeline()
> dummy_pipeline_3 = beam.Pipeline()
> dummy_pipeline_4 = beam.Pipeline()
> dummy_pcoll = dummy_pipeline_4 | beam.Create([1])
> dummy_pipeline_5 = beam.Pipeline()
> dummy_non_inspectable_pipeline = 'dummy'
> ie.current_env().watch(locals())
> from apache_beam.runners.interactive.background_caching_job import
> BackgroundCachingJob
> ie.current_env().set_background_caching_job(
> dummy_pipeline_1,
> BackgroundCachingJob(
> runner.PipelineResult(runner.PipelineState.DONE), limiters=[]))
> ie.current_env().set_test_stream_service_controller(dummy_pipeline_2,
> None)
> ie.current_env().set_cache_manager(
> cache.FileBasedCacheManager(), dummy_pipeline_3)
> ie.current_env().mark_pcollection_computed([dummy_pcoll])
> ie.current_env().set_cached_source_signature(
> dummy_non_inspectable_pipeline, None)
> ie.current_env().set_pipeline_result(
> dummy_pipeline_5,
> runner.PipelineResult(runner.PipelineState.RUNNING))
> > mocked_cleanup.assert_not_called()
> apache_beam/runners/interactive/interactive_environment_test.py:265:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> _mock_self = <MagicMock name='cleanup' id='140386685111432'>
> def assert_not_called(_mock_self):
> """assert that the mock was never called.
> """
> self = _mock_self
> if self.call_count != 0:
> msg = ("Expected '%s' to not have been called. Called %s times." %
> (self._mock_name or 'mock', self.call_count))
> > raise AssertionError(msg)
> E AssertionError: Expected 'cleanup' to not have been called.
> Called 1 times.
> /usr/lib/python3.7/unittest/mock.py:792: AssertionError
> {code}
> https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/3609/
> https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/3607/
> https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/3593/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
