[
https://issues.apache.org/jira/browse/PROTON-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078390#comment-17078390
]
Jiri Daněk commented on PROTON-2111:
------------------------------------
I am thinking that this is a duplicate of PROTON-2116. I tried the reproducer
(using both objgraph and gc.collect(); print(len(gc.get_objects())) and I
observed that there is object count growth even in proton 0.27, but it is
nowhere as dramatic as in 0.28. Then I saw the issue is fixed in 0.30, and it
is also fixed in current master.
The test uses ssl. I think it is a good idea to have object leak test that uses
ssl. I'll clean up the test, decide whether to keep objgraph, use guppy (that
is what I have some experience with) or just the gc.get_objects, and send PR.
For that reason I am not closing this issue as duplicate just yet.
{noformat}
for v in 0.27.1 0.28.0 0.29.0 0.30.0; do
:(
echo $v
echo xxxxxxxxxxxxxxxxxxxxxx
TESTDIR=qpid-proton/python/tests/integration
INCLUDEDIR=qpid-proton/tests/py
PYTHONPATH=qpid-proton_${v}/build/install/lib64/proton/bindings/python:${INCLUDEDIR}
\
LD_LIBRARY_PATH=qpid-proton_${v}/build/install/lib64 \
python -m unittest discover -v -s $TESTDIR -p '*_test.py'
done
0.27.1
xxxxxxxxxxxxxxxxxxxxxx
test_send_message_ssl_no_object_leaks
(blocking_connection_fd_leak_test.BlockingConnectionFDLeakTests) ...
/home/jdanek/repos/qpid/qpid-proton/python/tests/integration/blocking_connection_fd_leak_test.py:148:
ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_INET,
type=SocketKind.SOCK_STREAM, proto=0, laddr=('::e092:8ad:e57f:0', 39407, 0,
4294967295)>
urls = "amqps://{0}:{1}".format(*broker.get_acceptor_sockname())
ResourceWarning: Enable tracemalloc to get the object allocation traceback
amqps://:::39407
function 8365 +8365
dict 4234 +4234
tuple 3157 +3157
list 2154 +2154
weakref 1927 +1927
builtin_function_or_method 1669 +1669
member_descriptor 1564 +1564
type 1289 +1289
wrapper_descriptor 1284 +1284
getset_descriptor 1230 +1230
len gc get objects 32694
len gc get objects 32711
len gc get objects 32728
len gc get objects 32745
len gc get objects 32762
==============================================================================
Type Old_ids Current_ids New_ids Count_Deltas
==============================================================================
function 0 8365 +8365 +8365
dict 0 4258 +4258 +4258
tuple 0 3151 +3151 +3151
list 0 2163 +2163 +2163
weakref 0 1925 +1925 +1925
builtin_function_or_method 0 1667 +1667 +1667
member_descriptor 0 1564 +1564 +1564
type 0 1289 +1289 +1289
wrapper_descriptor 0 1284 +1284 +1284
getset_descriptor 0 1230 +1230 +1230
==============================================================================
[<proton._transport.SSLDomain object at 0x7fe59f19e550>,
<proton._transport.SSLDomain object at 0x7fe59f19e7d0>]
[]
[<proton._transport.SSL object at 0x7fe59dbc5250>, <proton._transport.SSL
object at 0x7fe59dbc5390>, <proton._transport.SSL object at 0x7fe59dbc5550>,
<proton._transport.SSL object at 0x7fe59dbc5710>, <proton._transport.SSL object
at 0x7fe59dbc58d0>]
ok
----------------------------------------------------------------------
Ran 1 test in 3.262s
OK
0.28.0
xxxxxxxxxxxxxxxxxxxxxx
test_send_message_ssl_no_object_leaks
(blocking_connection_fd_leak_test.BlockingConnectionFDLeakTests) ...
amqps://127.0.0.1:37773
function 8423 +8423
dict 4250 +4250
tuple 3178 +3178
list 2158 +2158
weakref 1935 +1935
builtin_function_or_method 1562 +1562
member_descriptor 1553 +1553
type 1299 +1299
wrapper_descriptor 1284 +1284
getset_descriptor 1233 +1233
len gc get objects 32728
len gc get objects 32753
len gc get objects 32776
len gc get objects 32797
len gc get objects 32819
==============================================================================
Type Old_ids Current_ids New_ids Count_Deltas
==============================================================================
function 0 8423 +8423 +8423
dict 0 4290 +4290 +4290
tuple 0 3178 +3178 +3178
list 0 2166 +2166 +2166
weakref 0 1933 +1933 +1933
builtin_function_or_method 0 1560 +1560 +1560
member_descriptor 0 1553 +1553 +1553
type 0 1299 +1299 +1299
wrapper_descriptor 0 1284 +1284 +1284
getset_descriptor 0 1233 +1233 +1233
==============================================================================
[<proton._transport.SSLDomain object at 0x7fb9a1069550>,
<proton._transport.SSLDomain object at 0x7fb9a10697d0>,
<proton._transport.SSLDomain object at 0x7fb99f266790>,
<proton._transport.SSLDomain object at 0x7fb99f266ad0>,
<proton._transport.SSLDomain object at 0x7fb99f266350>,
<proton._transport.SSLDomain object at 0x7fb99f266cd0>,
<proton._transport.SSLDomain object at 0x7fb99f266410>,
<proton._transport.SSLDomain object at 0x7fb99f266c90>,
<proton._transport.SSLDomain object at 0x7fb99f266890>,
<proton._transport.SSLDomain object at 0x7fb99f2669d0>,
<proton._transport.SSLDomain object at 0x7fb99f266bd0>,
<proton._transport.SSLDomain object at 0x7fb99f275890>,
<proton._transport.SSLDomain object at 0x7fb99fab6c10>,
<proton._transport.SSLDomain object at 0x7fb99fab69d0>]
[<proton._reactor.Container object at 0x7fb99facaf50>,
<proton._reactor.Container object at 0x7fb99f266a10>,
<proton._reactor.Container object at 0x7fb99f266d50>,
<proton._reactor.Container object at 0x7fb99facae90>,
<proton._reactor.Container object at 0x7fb99fab7f10>,
<proton._reactor.Container object at 0x7fb99faba0d0>]
[<proton._transport.SSL object at 0x7fb99f266c10>, <proton._transport.SSL
object at 0x7fb99f2750d0>, <proton._transport.SSL object at 0x7fb99f266d90>,
<proton._transport.SSL object at 0x7fb99f2752d0>, <proton._transport.SSL object
at 0x7fb99f275210>, <proton._transport.SSL object at 0x7fb99f275510>,
<proton._transport.SSL object at 0x7fb99f275490>, <proton._transport.SSL object
at 0x7fb99f2757d0>, <proton._transport.SSL object at 0x7fb99f275610>,
<proton._transport.SSL object at 0x7fb99f275810>]
ok
----------------------------------------------------------------------
Ran 1 test in 0.600s
OK
0.29.0
xxxxxxxxxxxxxxxxxxxxxx
test_send_message_ssl_no_object_leaks
(blocking_connection_fd_leak_test.BlockingConnectionFDLeakTests) ...
amqps://127.0.0.1:39825
function 8423 +8423
dict 4250 +4250
tuple 3179 +3179
list 2158 +2158
weakref 1935 +1935
builtin_function_or_method 1562 +1562
member_descriptor 1553 +1553
type 1299 +1299
wrapper_descriptor 1284 +1284
getset_descriptor 1233 +1233
len gc get objects 32721
len gc get objects 32746
len gc get objects 32778
len gc get objects 32798
len gc get objects 32826
==============================================================================
Type Old_ids Current_ids New_ids Count_Deltas
==============================================================================
function 0 8423 +8423 +8423
dict 0 4290 +4290 +4290
tuple 0 3178 +3178 +3178
list 0 2166 +2166 +2166
weakref 0 1933 +1933 +1933
builtin_function_or_method 0 1560 +1560 +1560
member_descriptor 0 1553 +1553 +1553
type 0 1299 +1299 +1299
wrapper_descriptor 0 1284 +1284 +1284
getset_descriptor 0 1233 +1233 +1233
==============================================================================
[<proton._transport.SSLDomain object at 0x7f4278a4a550>,
<proton._transport.SSLDomain object at 0x7f4278a4a7d0>,
<proton._transport.SSLDomain object at 0x7f4276c4cad0>,
<proton._transport.SSLDomain object at 0x7f4276c4cc50>,
<proton._transport.SSLDomain object at 0x7f4276c4cb50>,
<proton._transport.SSLDomain object at 0x7f4276c4ca50>,
<proton._transport.SSLDomain object at 0x7f4276c4c610>,
<proton._transport.SSLDomain object at 0x7f4276c4cc10>,
<proton._transport.SSLDomain object at 0x7f4276c4cd50>,
<proton._transport.SSLDomain object at 0x7f4277fcf990>,
<proton._transport.SSLDomain object at 0x7f4276c4c450>,
<proton._transport.SSLDomain object at 0x7f4276c54950>,
<proton._transport.SSLDomain object at 0x7f427749c310>,
<proton._transport.SSLDomain object at 0x7f427749cb50>]
[<proton._reactor.Container object at 0x7f4276c4c950>,
<proton._reactor.Container object at 0x7f42774b3e90>,
<proton._reactor.Container object at 0x7f427749c990>,
<proton._reactor.Container object at 0x7f427749c890>,
<proton._reactor.Container object at 0x7f42774b3110>,
<proton._reactor.Container object at 0x7f427749eed0>]
[<proton._transport.SSL object at 0x7f4276c540d0>, <proton._transport.SSL
object at 0x7f4276c54250>, <proton._transport.SSL object at 0x7f4276c54090>,
<proton._transport.SSL object at 0x7f4276c54410>, <proton._transport.SSL object
at 0x7f4276c544d0>, <proton._transport.SSL object at 0x7f4276c4ca10>,
<proton._transport.SSL object at 0x7f4276c54650>, <proton._transport.SSL object
at 0x7f4276c549d0>, <proton._transport.SSL object at 0x7f4276c54a10>,
<proton._transport.SSL object at 0x7f4276c54c90>]
ok
----------------------------------------------------------------------
Ran 1 test in 0.475s
OK
0.30.0
xxxxxxxxxxxxxxxxxxxxxx
test_send_message_ssl_no_object_leaks
(blocking_connection_fd_leak_test.BlockingConnectionFDLeakTests) ...
amqps://127.0.0.1:33689
function 8451 +8451
dict 4255 +4255
tuple 3200 +3200
list 2158 +2158
weakref 1939 +1939
builtin_function_or_method 1563 +1563
member_descriptor 1553 +1553
type 1303 +1303
wrapper_descriptor 1284 +1284
getset_descriptor 1237 +1237
len gc get objects 32773
len gc get objects 32773
len gc get objects 32773
len gc get objects 32773
len gc get objects 32773
==============================================================================
Type Old_ids Current_ids New_ids Count_Deltas
==============================================================================
function 0 8451 +8451 +8451
dict 0 4245 +4245 +4245
tuple 0 3198 +3198 +3198
list 0 2156 +2156 +2156
weakref 0 1937 +1937 +1937
builtin_function_or_method 0 1561 +1561 +1561
member_descriptor 0 1553 +1553 +1553
type 0 1303 +1303 +1303
wrapper_descriptor 0 1284 +1284 +1284
getset_descriptor 0 1237 +1237 +1237
==============================================================================
[<proton._transport.SSLDomain object at 0x7fc9f7658550>,
<proton._transport.SSLDomain object at 0x7fc9f76587d0>,
<proton._transport.SSLDomain object at 0x7fc9f60bf850>,
<proton._transport.SSLDomain object at 0x7fc9f60bf610>]
[<proton._reactor.Container object at 0x7fc9f60bcf50>]
[]
ok
----------------------------------------------------------------------
Ran 1 test in 3.715s
OK
{noformat}
> python: memory leak on Container, SSL, and SSLDomain objects
> ------------------------------------------------------------
>
> Key: PROTON-2111
> URL: https://issues.apache.org/jira/browse/PROTON-2111
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
> Affects Versions: proton-c-0.28.0
> Reporter: Chenxiong Qi
> Assignee: Clifford Jansen
> Priority: Major
>
> I have an application using qpid.proton to interact with ActiveMQ broker to
> publish messages. Following is a fake script showing an example of the app
> works to send a message.
> {noformat}
> class SampleSender(proton.handlers.MessagingHandler):
> def __init__(self, msg_id, *args, **kwargs):
> super(SampleSender, self).__init__(*args, **kwargs)
> self.msg_id = msg_id
> def on_start(self, event):
> ssl_domain = proton.SSLDomain(proton.SSLDomain.MODE_CLIENT)
> ssl_domain.set_credentials(producer_config['certificate'],
> producer_config['private_key'],
> None)
> ssl_domain.set_trusted_ca_db(producer_config['trusted_certificates'])
> ssl_domain.set_peer_authentication(proton.SSLDomain.VERIFY_PEER)
> conn = event.container.connect(urls=producer_config['urls'],
> reconnect=False,
> ssl_domain=ssl_domain)
> event.container.create_sender(conn,
> target='topic://VirtualTopic.event')
> def on_sendable(self, event):
> msg = proton.Message(body={'msg-id': self.msg_id, 'name': 'python'})
> event.sender.send(msg)
> event.sender.close()
> event.connection.close()
> def send_msg(msg_id):
> container = proton.reactor.Container(SampleSender(msg_id))
> container.run()
> objgraph.show_growth()
> for i in range(3):
> send_msg(i + 1)
> new_ids = objgraph.get_new_ids()
> print(objgraph.at_addrs(new_ids['SSLDomain']))
> print(objgraph.at_addrs(new_ids['Container']))
> print(objgraph.at_addrs(new_ids['SSL']))
> {noformat}
> Each time to publish a message, a new Container object is created and a new
> handler object is created and passed to that container. After the code runs
> several times, Container, SSL and SSLDomain objects remain in memory and are
> not collected. I use objgraph to find out those objects.
> I'm using Fedora 30 and python-qpid-proton 0.28.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
