[
https://issues.apache.org/jira/browse/IMPALA-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18062379#comment-18062379
]
ASF subversion and git services commented on IMPALA-14776:
----------------------------------------------------------
Commit 79e2ab998c9c12259f5b671050e7dd610c0f9e16 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=79e2ab998 ]
IMPALA-14776 (part 2): Fix miscellaneous socket leaks
There are various sockets leaks that are not associated
with Impala clients:
- Users of urlopen don't auto-close the socket
- Direct use of sockets don't clean up the socket
- Various tests that spin up test servers (e.g. statestore tests)
don't close clients or server sockets.
This fixes those various locations either with manual clean up
or with-as constructs. For urlopen, this simplifies the import to
always use the Python 3 package location, dropping support for
Python 2.
Testing:
- Ran tests locally and verified that they stopped leaking
sockets
Change-Id: I8fd54fad18d6dbfb0f24ebd910f45cfed76c340a
Reviewed-on: http://gerrit.cloudera.org:8080/24061
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Yida Wu <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Fix pytest ResourceWarnings
> ---------------------------
>
> Key: IMPALA-14776
> URL: https://issues.apache.org/jira/browse/IMPALA-14776
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure, Test
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> After switching to pytest 6.2.5, it now outputs information about leaked
> resources (sockets, open files, thread pools):
> {noformat}
> 18:15:34
> authorization/test_ranger.py::TestRangerIndependent::test_grant_revoke_with_role["table_format:
> text/none" | "protocol: hs2" | "exec_option: {'abort_on_error': 1,
> 'batch_size': 0, 'disable_codegen': False, 'disable_codegen_rows_threshold':
> 0, 'exec_single_node_rows_threshold': 0, 'num_nodes': 0, 'test_replan': 1}"]
> 18:15:34
> /data0/jenkins/workspace/impala-cdw-master-staging-core-asan/repos/Impala/tests/common/impala_test_suite.py:951:
> ResourceWarning: unclosed <socket.socket fd=47,
> family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
> laddr=('127.0.0.1', 44380), raddr=('127.0.0.1', 21050)>{noformat}
> We should fix these warnings. In particular, tests that don't clean up
> clients can cause there to be orphaned HS2 sessions that trigger
> TestValidateMetrics.test_metrics_are_zero().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
