[
https://issues.apache.org/jira/browse/FLINK-31069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688831#comment-17688831
]
Nathanael England commented on FLINK-31069:
-------------------------------------------
Once you get around/ignore this, a new class of issues has to be solved to get
the base tooling to work. I've opened FLINK-31073 to track my findings there.
> Pyflink 1.16.1 has unclosed resources at the end of unit tests
> --------------------------------------------------------------
>
> Key: FLINK-31069
> URL: https://issues.apache.org/jira/browse/FLINK-31069
> Project: Flink
> Issue Type: Bug
> Environment: Ubuntu 20.04
> Python 3.8.10
> Pyflink 1.16.1
> Reporter: Nathanael England
> Priority: Minor
>
> A simple pyflink unit test has unclosed resources at the end of the testing.
> A minimally reproducable example of this can be seen with the following in
> 1.16.1
> {code:python}
> from pyflink.testing import test_case_utils
> class InputBroadcastProcessFunctionTests(
> test_case_utils.PyFlinkStreamingTestCase):
> def test_nothing(self):
> pass
> {code}
> When `pytest.ini`is instructed to have `filterwarnings = errors`, the user is
> met with errors like the following
> {code:bash}
> $ pytest example_test.py
> ================================================================================================================================================
> test session starts
> ================================================================================================================================================
> platform linux -- Python 3.8.10, pytest-7.2.1, pluggy-1.0.0
> rootdir: /home/my_repo, configfile: pytest.ini
> plugins: forked-1.6.0, anyio-3.6.2, timeout-2.1.0, typeguard-2.13.3,
> xdist-2.5.0, rabbitmq-2.2.1, cov-4.0.0
> timeout: 60.0s
> timeout method: signal
> timeout func_only: False
> collected 1 item
>
>
>
> example_test.py E
>
>
> [100%]
> ======================================================================================================================================================
> ERRORS
> =======================================================================================================================================================
> _________________________________________________________________________________________________________________________
> ERROR at setup of InputBroadcastProcessFunctionTests.test_nothing
> _________________________________________________________________________________________________________________________
> cls = <class '_pytest.runner.CallInfo'>, func = <function
> call_runtest_hook.<locals>.<lambda> at 0x7f42530f0af0>, when = 'setup',
> reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
> @classmethod
> def from_call(
> cls,
> func: "Callable[[], TResult]",
> when: "Literal['collect', 'setup', 'call', 'teardown']",
> reraise: Optional[
> Union[Type[BaseException], Tuple[Type[BaseException], ...]]
> ] = None,
> ) -> "CallInfo[TResult]":
> """Call func, wrapping the result in a CallInfo.
>
> :param func:
> The function to call. Called without arguments.
> :param when:
> The phase in which the function is called.
> :param reraise:
> Exception or exceptions that shall propagate if raised by the
> function, instead of being wrapped in the CallInfo.
> """
> excinfo = None
> start = timing.time()
> precise_start = timing.perf_counter()
> try:
> > result: Optional[TResult] = func()
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/runner.py:339:
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/runner.py:260:
> in <lambda>
> lambda: ihook(item=item, **kwds), when=when, reraise=reraise
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pluggy/_hooks.py:265:
> in __call__
> return self._hookexec(self.name, self.get_hookimpls(), kwargs,
> firstresult)
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pluggy/_manager.py:80:
> in _hookexec
> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/unraisableexception.py:83:
> in pytest_runtest_setup
> yield from unraisable_exception_runtest_hook()
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
> with catch_unraisable_exception() as cm:
> yield
> if cm.unraisable:
> if cm.unraisable.err_msg is not None:
> err_msg = cm.unraisable.err_msg
> else:
> err_msg = "Exception ignored in"
> msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
> msg += "".join(
> traceback.format_exception(
> cm.unraisable.exc_type,
> cm.unraisable.exc_value,
> cm.unraisable.exc_traceback,
> )
> )
> > warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
> E pytest.PytestUnraisableExceptionWarning: Exception ignored
> in: <function Popen.__del__ at 0x7f427ce715e0>
> E
> E Traceback (most recent call last):
> E File "/usr/lib/python3.8/subprocess.py", line 946, in
> __del__
> E _warn("subprocess %s is still running" % self.pid,
> E ResourceWarning: subprocess 1906862 is still running
> dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/_pytest/unraisableexception.py:78:
> PytestUnraisableExceptionWarning
> -----------------------------------------------------------------------------------------------------------------------------------------------
> Captured stderr setup
> -----------------------------------------------------------------------------------------------------------------------------------------------
> 13:11:22.743 I test_case_utils.py@91: Using
> /home/my_repo/dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pyflink
> as FLINK_HOME...
> ------------------------------------------------------------------------------------------------------------------------------------------------
> Captured log setup
> -------------------------------------------------------------------------------------------------------------------------------------------------
> INFO root:test_case_utils.py:91 Using
> /home/my_repo/dist/export/python/virtualenvs/python-default/3.8.10/lib/python3.8/site-packages/pyflink
> as FLINK_HOME...
> ==============================================================================================================================================
> short test summary info
> ==============================================================================================================================================
> ERROR example_test.py::InputBroadcastProcessFunctionTests::test_nothing -
> pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function
> Popen.__del__ at 0x7f427ce715e0>
> =================================================================================================================================================
> 1 error in 1.22s
> ==================================================================================================================================================
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
