Re: proton 0.10 blocker
I'm fine going ahead with Gordon's fix. I don't have a lot of time to dig into the refcounting issue personally right now, but I'd at least leave the bug open until we have made it through a bit more testing. I have an uneasy feeling it (or something closely related) may pop up again if we push harder on testing. --Rafael On Mon, Jul 20, 2015 at 10:43 AM, Ken Giusti wrote: > +1 for using Gordon's fix for now - we can cut a beta and see if it holds > up. > > Since there's some unanswered questions regarding the patch's behavior, > I'll leave the bug open - drop the blocker status - and assign it over to > Rafi. He's better cerebrally equipped to understand the reference counting > implementation than I certainly am. > > > > - Original Message - > > From: "Robbie Gemmell" > > To: d...@qpid.apache.org, proton@qpid.apache.org > > Sent: Monday, July 20, 2015 12:03:06 PM > > Subject: Re: proton 0.10 blocker > > > > On 17 July 2015 at 23:32, Gordon Sim wrote: > > > On 07/17/2015 10:04 PM, Rafael Schloming wrote: > > >> > > >> On Fri, Jul 17, 2015 at 12:47 PM, Gordon Sim wrote: > > >> > > >>> On 07/17/2015 08:15 PM, Rafael Schloming wrote: > > >>> > > On Fri, Jul 17, 2015 at 11:56 AM, Gordon Sim > wrote: > > > > On 07/17/2015 07:17 PM, Rafael Schloming wrote: > > > > > > > > > Given this I believe the incref/decref pair is indeed running > into > > >> > > >> problems > > >> when being invoked from inside a finalizer. I'd be curious if an > > >> alternative fix would work. I suspect you could replace the > additional > > >> conditions you added to the if predicate with this: > > >> > > >> pn_refcount(endpoint) > 0 > > >> > > >> > > > If the refcount is *not* 0, what does the incref/decref sequence > > > accomplish? > > > > > > > > > I believe the answer to this is the same as the answer I just > posted on > > the > > other thread, i.e. the incref may trigger the incref hook (see > > pn_xxx_incref in engine.c), and this in turn may update the endpoint > > state > > and adjust the refcount accordingly. The decref may then end up > > finalizing > > the object. > > > > >>> > > >>> Right, understood now. > > >>> > > >>> Unfortunately replacing the additional conditions with just that > check on > > >>> the refcount doesn't prevent the crash though. > > >>> > > >> > > >> Doh, not the result I was hoping for. Does it yield the same stack > trace > > >> as > > >> before? > > > > > > > > > Yes > > > > > > > Given that and all who looked thinking the earlier proposal was safe, > > is it worth going with that change at least for now? Knocking things > > off the blocker list and getting an RC (or even just a beta) out would > > be really really good. > > > > Robbie > > > > -- > -K >
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user astitcher commented on the pull request: https://github.com/apache/qpid-proton/pull/47#issuecomment-123005675 I've raised [PROTON-951](https://issues.apache.org/jira/browse/PROTON-951) and [PROTON-952](https://issues.apache.org/jira/browse/PROTON-952) - which are related bugs to this discussion. [PROTON-952](https://issues.apache.org/jira/browse/PROTON-952) may be related to the python 2.6 tox issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (PROTON-951) Make install fails when building Proton-C with python 3.4
[
https://issues.apache.org/jira/browse/PROTON-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633996#comment-14633996
]
ASF GitHub Bot commented on PROTON-951:
---
Github user astitcher commented on the pull request:
https://github.com/apache/qpid-proton/pull/47#issuecomment-123005675
I've raised [PROTON-951](https://issues.apache.org/jira/browse/PROTON-951)
and [PROTON-952](https://issues.apache.org/jira/browse/PROTON-952) - which are
related bugs to this discussion.
[PROTON-952](https://issues.apache.org/jira/browse/PROTON-952) may be related
to the python 2.6 tox issue.
> Make install fails when building Proton-C with python 3.4
> -
>
> Key: PROTON-951
> URL: https://issues.apache.org/jira/browse/PROTON-951
> Project: Qpid Proton
> Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Andrew Stitcher
> Attachments: py34fail
>
>
> When building proton-C using a python 3.4 virtualenv make install fails:
> {quote}
> $ source ~/LocalPython/Python3/bin/activate
> $ cmake -DCMAKE_INSTALL_PREFIX=$PWD/install ../src
> $ cmake --build . --target install
> {quote}
> {quote}
> ...
> CMake Error at proton-c/bindings/python/cmake_install.cmake:106 (file):
> file INSTALL cannot find
> "/home/andrew/Work/proton/bld-python3/proton-c/bindings/python/cproton.pyc".
> Call Stack (most recent call first):
> proton-c/bindings/cmake_install.cmake:37 (include)
> proton-c/cmake_install.cmake:139 (include)
> cmake_install.cmake:50 (include)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-952) Building Proton with python 2.6 and python 3.4 on Travis CI finds and links wrong libpython
[
https://issues.apache.org/jira/browse/PROTON-952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633992#comment-14633992
]
Andrew Stitcher commented on PROTON-952:
Build log for python 3.4 (2.6 is similar)
{quote}
...
$ cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install
...
-- Build type is "RelWithDebInfo" (has debug symbols)
-- PN_VERSION: 0.10 (SNAPSHOT)
...
-- Found PythonInterp: /home/travis/virtualenv/python3.4.2/bin/python
...
-- Found PythonLibs: /usr/lib/libpython2.7.so
...
{quote}
So it finds the correct python executable but the incorrect libpython. This may
be a bug in the version of cmake being used (as I don't get this problem on my
fedora 21/22 developer boxes).
The failure symptoms are:
For Python 3.4 -
{quote}
...
$ ctest -V
...
test 2
Start 2: python-test
2: Test command: /home/travis/virtualenv/python3.4.2/bin/python
"/home/travis/build/astitcher/qpid-proton/proton-c/env.py"
"PATH=/home/travis/build/astitcher/qpid-proton/Build/proton-c:/home/travis/build/astitcher/qpid-proton/Build/tests/tools/apps/c:/home/travis/build/astitcher/qpid-proton/tests/tools/apps/python:/home/travis/.local/bin:/home/travis/virtualenv/python3.4.2/bin:/home/travis/bin:/home/travis/.local/bin:/home/travis/.gimme/versions/go1.4.1.linux.amd64/bin:/opt/python/2.7.9/bin:/opt/python/2.6.9/bin:/opt/python/3.4.2/bin:/opt/python/3.3.5/bin:/opt/python/3.2.5/bin:/opt/python/pypy-2.5.0/bin:/opt/python/pypy3-2.4.0/bin:/usr/local/phantomjs/bin:/home/travis/.nvm/v0.10.36/bin:./node_modules/.bin:/usr/local/maven-3.2.5/bin:/usr/local/clang-3.4/bin:/home/travis/.gimme/versions/go1.4.1.linux.amd64/bin:/opt/python/2.7.9/bin:/opt/python/2.6.9/bin:/opt/python/3.4.2/bin:/opt/python/3.3.5/bin:/opt/python/3.2.5/bin:/opt/python/pypy-2.5.0/bin:/opt/python/pypy3-2.4.0/bin:/usr/local/phantomjs/bin:./node_modules/.bin:/usr/local/maven-3.2.5/bin:/usr/local/clang-3.4/bin:/home/travis/.gimme/versions/go1.4.1.linux.amd64/bin:/home/travis/.rvm/gems/ruby-1.9.3-p551/bin:/home/travis/.rvm/gems/ruby-1.9.3-p551@global/bin:/home/travis/.rvm/rubies/ruby-1.9.3-p551/bin:/opt/python/2.7.9/bin:/opt/python/2.6.9/bin:/opt/python/3.4.2/bin:/opt/python/3.3.5/bin:/opt/python/3.2.5/bin:/opt/python/pypy-2.5.0/bin:/opt/python/pypy3-2.4.0/bin:/usr/local/phantomjs/bin:./node_modules/.bin:/usr/local/maven-3.2.5/bin:/usr/local/clang-3.4/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/travis/.rvm/bin:/home/travis/.rvm/bin:/home/travis/.rvm/bin"
"PYTHONPATH=/home/travis/build/astitcher/qpid-proton/tests/python:/home/travis/build/astitcher/qpid-proton/proton-c/bindings/python:/home/travis/build/astitcher/qpid-proton/Build/proton-c/bindings/python:/home/travis/build/astitcher/qpid-proton/Build/proton-c/bindings/python"
"PKG_CONFIG_PATH=/home/travis/build/astitcher/qpid-proton/Build/proton"
"CLASSPATH=/home/travis/build/astitcher/qpid-proton/Build/proton-j/proton-j.jar"
"SASLPASSWD=/usr/sbin/saslpasswd2" "VALGRIND=/usr/bin/valgrind"
"/home/travis/virtualenv/python3.4.2/bin/python"
"/home/travis/build/astitcher/qpid-proton/tests/python/proton-test"
2: Test timeout computed to be: 1500
2: Traceback (most recent call last):
2: File "/home/travis/build/astitcher/qpid-proton/tests/python/proton-test",
line 620, in
2: m = __import__(name, None, None, ["dummy"])
2: File
"/home/travis/build/astitcher/qpid-proton/tests/python/proton_tests/__init__.py",
line 20, in
2: import proton_tests.codec
2: File
"/home/travis/build/astitcher/qpid-proton/tests/python/proton_tests/codec.py",
line 21, in
2: from . import common
2: File
"/home/travis/build/astitcher/qpid-proton/tests/python/proton_tests/common.py",
line 26, in
2: from proton import Connection, Transport, SASL, Endpoint, Delivery, SSL
2: File
"/home/travis/build/astitcher/qpid-proton/proton-c/bindings/python/proton/__init__.py",
line 34, in
2: from cproton import *
2: File
"/home/travis/build/astitcher/qpid-proton/Build/proton-c/bindings/python/cproton.py",
line 26, in
2: _cproton = swig_import_helper()
2: File
"/home/travis/build/astitcher/qpid-proton/Build/proton-c/bindings/python/cproton.py",
line 22, in swig_import_helper
2: _mod = imp.load_module('_cproton', fp, pathname, description)
2: File "/home/travis/virtualenv/python3.4.2/lib/python3.4/imp.py", line 243,
in load_module
2: return load_dynamic(name, filename, file)
2: ImportError: dynamic module does not define init function (PyInit__cproton)
2/12 Test #2: python-test ..***Failed Required regular
expression not found.Regex=[Totals: .* 0 failed
] 0.24 sec
...
{quote}
For Python 2.6-
{quote}
...
$ ctest -V
...
test 2
Start 2: python-test
2: Test command: /home/travis/virtualenv/python2.6.9/bin/python
"/home/travis/build/astitcher/qpid-proton/proton-c/env.py"
"PATH=/home/travis/build/astitcher/qpid-proton/Build/proton-c:/home/travis/build/astit
[jira] [Commented] (PROTON-951) Make install fails when building Proton-C with python 3.4
[
https://issues.apache.org/jira/browse/PROTON-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633983#comment-14633983
]
Andrew Stitcher commented on PROTON-951:
This may not have been noticed before because running ctest runs all the tests
fine.
It seems to be a slight difference in the built artifacts for python 3.4
compared to python 2.7 (lack of .pyc being generated?)
> Make install fails when building Proton-C with python 3.4
> -
>
> Key: PROTON-951
> URL: https://issues.apache.org/jira/browse/PROTON-951
> Project: Qpid Proton
> Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Andrew Stitcher
> Attachments: py34fail
>
>
> When building proton-C using a python 3.4 virtualenv make install fails:
> {quote}
> $ source ~/LocalPython/Python3/bin/activate
> $ cmake -DCMAKE_INSTALL_PREFIX=$PWD/install ../src
> $ cmake --build . --target install
> {quote}
> {quote}
> ...
> CMake Error at proton-c/bindings/python/cmake_install.cmake:106 (file):
> file INSTALL cannot find
> "/home/andrew/Work/proton/bld-python3/proton-c/bindings/python/cproton.pyc".
> Call Stack (most recent call first):
> proton-c/bindings/cmake_install.cmake:37 (include)
> proton-c/cmake_install.cmake:139 (include)
> cmake_install.cmake:50 (include)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-952) Building Proton with python 2.6 and python 3.4 on Travis CI finds and links wrong libpython
Andrew Stitcher created PROTON-952: -- Summary: Building Proton with python 2.6 and python 3.4 on Travis CI finds and links wrong libpython Key: PROTON-952 URL: https://issues.apache.org/jira/browse/PROTON-952 Project: Qpid Proton Issue Type: Bug Affects Versions: 0.10 Environment: Travis CI build environment (highly customised Ubuntu) Reporter: Andrew Stitcher You can specify a specifiv version of python to use in the Traviis build process and that will set up the build environment to use a python virtualenv with that version of python. When using python 2.6 or 3.4 (and probably 3.3 too but I've not tested that) the cmake installed will find the python 2.7 libs and link them into the proton bindings extension. This seems to succeed at build time but seems to make the "python-test" fail to even start in both cases (though with slightly different symptoms). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-951) Make install fails when building Proton-C with python 3.4
[
https://issues.apache.org/jira/browse/PROTON-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Stitcher updated PROTON-951:
---
Attachment: py34fail
Attached Output from "make install"
> Make install fails when building Proton-C with python 3.4
> -
>
> Key: PROTON-951
> URL: https://issues.apache.org/jira/browse/PROTON-951
> Project: Qpid Proton
> Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Andrew Stitcher
> Attachments: py34fail
>
>
> When building proton-C using a python 3.4 virtualenv make install fails:
> {quote}
> $ source ~/LocalPython/Python3/bin/activate
> $ cmake -DCMAKE_INSTALL_PREFIX=$PWD/install ../src
> $ cmake --build . --target install
> {quote}
> {quote}
> ...
> CMake Error at proton-c/bindings/python/cmake_install.cmake:106 (file):
> file INSTALL cannot find
> "/home/andrew/Work/proton/bld-python3/proton-c/bindings/python/cproton.pyc".
> Call Stack (most recent call first):
> proton-c/bindings/cmake_install.cmake:37 (include)
> proton-c/cmake_install.cmake:139 (include)
> cmake_install.cmake:50 (include)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-951) Make install fails when building Proton-C with python 3.4
Andrew Stitcher created PROTON-951:
--
Summary: Make install fails when building Proton-C with python 3.4
Key: PROTON-951
URL: https://issues.apache.org/jira/browse/PROTON-951
Project: Qpid Proton
Issue Type: Bug
Affects Versions: 0.10
Reporter: Andrew Stitcher
When building proton-C using a python 3.4 virtualenv make install fails:
{quote}
$ source ~/LocalPython/Python3/bin/activate
$ cmake -DCMAKE_INSTALL_PREFIX=$PWD/install ../src
$ cmake --build . --target install
{quote}
{quote}
...
CMake Error at proton-c/bindings/python/cmake_install.cmake:106 (file):
file INSTALL cannot find
"/home/andrew/Work/proton/bld-python3/proton-c/bindings/python/cproton.pyc".
Call Stack (most recent call first):
proton-c/bindings/cmake_install.cmake:37 (include)
proton-c/cmake_install.cmake:139 (include)
cmake_install.cmake:50 (include)
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti reassigned PROTON-905: - Assignee: Rafael H. Schloming (was: Ken Giusti) > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Rafael H. Schloming >Priority: Critical > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-943) Bump library (.so) major version for proton-c libraries
[ https://issues.apache.org/jira/browse/PROTON-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633953#comment-14633953 ] ASF subversion and git services commented on PROTON-943: Commit 8f82638b7e8299c8c3544516e4b8a18d185d3325 in qpid-proton's branch refs/heads/master from [~kgiusti] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=8f82638 ] PROTON-943: bump libqpid-proton shared library major version for 0.10 release > Bump library (.so) major version for proton-c libraries > --- > > Key: PROTON-943 > URL: https://issues.apache.org/jira/browse/PROTON-943 > Project: Qpid Proton > Issue Type: Task > Components: proton-c >Reporter: Ted Ross >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > > 0.10 contains incompatible API changes. The library major versions need to > be incremented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-943) Bump library (.so) major version for proton-c libraries
[ https://issues.apache.org/jira/browse/PROTON-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti resolved PROTON-943. --- Resolution: Fixed Bumped to 3.0.0 > Bump library (.so) major version for proton-c libraries > --- > > Key: PROTON-943 > URL: https://issues.apache.org/jira/browse/PROTON-943 > Project: Qpid Proton > Issue Type: Task > Components: proton-c >Reporter: Ted Ross >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > > 0.10 contains incompatible API changes. The library major versions need to > be incremented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-943) Bump library (.so) major version for proton-c libraries
[ https://issues.apache.org/jira/browse/PROTON-943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti reassigned PROTON-943: - Assignee: Ken Giusti > Bump library (.so) major version for proton-c libraries > --- > > Key: PROTON-943 > URL: https://issues.apache.org/jira/browse/PROTON-943 > Project: Qpid Proton > Issue Type: Task > Components: proton-c >Reporter: Ted Ross >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > > 0.10 contains incompatible API changes. The library major versions need to > be incremented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633913#comment-14633913 ] Ken Giusti commented on PROTON-905: --- I've merged Gordon's work around, and have dropped the Blocker status. I'm leaving this open and assigning to Rafi as there is concern that there may be other issues involved that have not been addressed. > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Critical > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti updated PROTON-905: -- Priority: Critical (was: Blocker) > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Critical > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633911#comment-14633911 ] Ken Giusti commented on PROTON-905: --- Crash as captured by gsim: > #0 0x003d54635935 in raise () from /lib64/libc.so.6 > #1 0x003d546370e8 in abort () from /lib64/libc.so.6 > #2 0x003d5462e6a2 in __assert_fail_base () from /lib64/libc.so.6 > #3 0x003d5462e752 in __assert_fail () from /lib64/libc.so.6 > #4 0x7fef7d37a761 in pn_class_free (clazz=0x7fef7d5b4ec0, > object=0x3692060) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:120 > #5 0x7fef7d37aa12 in pn_free (object=) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:265 > #6 0x7fef7d386000 in pni_free_children (children=0x3690510, > freed=0x36905e0) at > /home/gordon/projects/proton-git/proton-c/src/engine/engine.c:454 > #7 0x7fef7d386d09 in pn_session_finalize (object=0x3690180) at > /home/gordon/projects/proton-git/proton-c/src/engine/engine.c:932 > #8 0x7fef7d37a64d in pn_class_decref (clazz=clazz@entry=0x7fef7d5b4e40, > object=object@entry=0x3690180) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:97 > #9 0x7fef7d37a70b in pn_class_free (clazz=0x7fef7d5b4e40, > object=0x3690180) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:122 > #10 0x7fef7d37aa12 in pn_free (object=) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:265 > #11 0x7fef7d386000 in pni_free_children (children=0x368e9c0, > freed=0x368ea90) at > /home/gordon/projects/proton-git/proton-c/src/engine/engine.c:454 > #12 0x7fef7d3866db in pn_connection_finalize (object=0x368e310) at > /home/gordon/projects/proton-git/proton-c/src/engine/engine.c:476 > #13 0x7fef7d37a64d in pn_class_decref (clazz=0x7fef7d5b4dc0, > object=object@entry=0x368e310) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:97 > #14 0x7fef7d37a9d2 in pn_decref (object=object@entry=0x368e310) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:255 > #15 0x7fef7d38bd08 in pn_transport_unbind > (transport=transport@entry=0x36831e0) at > /home/gordon/projects/proton-git/proton-c/src/transport/transport.c:733 > #16 0x7fef7d38bd88 in pn_transport_finalize (object=0x36831e0) at > /home/gordon/projects/proton-git/proton-c/src/transport/transport.c:602 > #17 0x7fef7d37a64d in pn_class_decref (clazz=0x7fef7d5b50c0, > object=0x36831e0) at > /home/gordon/projects/proton-git/proton-c/src/object/object.c:97 > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-905) Long-lived connections leak sessions and links
[ https://issues.apache.org/jira/browse/PROTON-905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633909#comment-14633909 ] ASF subversion and git services commented on PROTON-905: Commit d9ce3cfd0916ae3719cb39a83a6174c5f88b10bb in qpid-proton's branch refs/heads/master from [~gsim] [ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=d9ce3cf ] PROTON-905: fix to prevent crash with latest qpidd > Long-lived connections leak sessions and links > -- > > Key: PROTON-905 > URL: https://issues.apache.org/jira/browse/PROTON-905 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.9.1 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Blocker > Fix For: 0.10 > > Attachments: test-send.py > > > I found this issue while debugging a crash dump of qpidd. > Long lived connections do not free its sessions/link. > This only applies when NOT using the event model. The version of qpidd I > tested against (0.30) still uses the iterative model. Point to consider, I > don't know why this is the case. > Details: I have a test script that opens a single connection, then > continually creates sessions/links over that connection, sending one message > before closing and freeing the sessions/links. See attached. > Over time the qpidd run time consumes all memory on the system and is killed > by OOM. To be clear, I'm using drain to remove all sent messages - there is > no message build up. > On debugging this, I'm finding thousands of session objects on the > connections free sessions weakref list. Every one of those sessions has a > refcount of one. > Once the connection is finalized, all session objects are freed. But until > then, freed sessions continue to accumulate indefinitely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: proton 0.10 blocker
+1 for using Gordon's fix for now - we can cut a beta and see if it holds up. Since there's some unanswered questions regarding the patch's behavior, I'll leave the bug open - drop the blocker status - and assign it over to Rafi. He's better cerebrally equipped to understand the reference counting implementation than I certainly am. - Original Message - > From: "Robbie Gemmell" > To: d...@qpid.apache.org, proton@qpid.apache.org > Sent: Monday, July 20, 2015 12:03:06 PM > Subject: Re: proton 0.10 blocker > > On 17 July 2015 at 23:32, Gordon Sim wrote: > > On 07/17/2015 10:04 PM, Rafael Schloming wrote: > >> > >> On Fri, Jul 17, 2015 at 12:47 PM, Gordon Sim wrote: > >> > >>> On 07/17/2015 08:15 PM, Rafael Schloming wrote: > >>> > On Fri, Jul 17, 2015 at 11:56 AM, Gordon Sim wrote: > > On 07/17/2015 07:17 PM, Rafael Schloming wrote: > > > > > > Given this I believe the incref/decref pair is indeed running into > >> > >> problems > >> when being invoked from inside a finalizer. I'd be curious if an > >> alternative fix would work. I suspect you could replace the additional > >> conditions you added to the if predicate with this: > >> > >> pn_refcount(endpoint) > 0 > >> > >> > > If the refcount is *not* 0, what does the incref/decref sequence > > accomplish? > > > > > I believe the answer to this is the same as the answer I just posted on > the > other thread, i.e. the incref may trigger the incref hook (see > pn_xxx_incref in engine.c), and this in turn may update the endpoint > state > and adjust the refcount accordingly. The decref may then end up > finalizing > the object. > > >>> > >>> Right, understood now. > >>> > >>> Unfortunately replacing the additional conditions with just that check on > >>> the refcount doesn't prevent the crash though. > >>> > >> > >> Doh, not the result I was hoping for. Does it yield the same stack trace > >> as > >> before? > > > > > > Yes > > > > Given that and all who looked thinking the earlier proposal was safe, > is it worth going with that change at least for now? Knocking things > off the blocker list and getting an RC (or even just a beta) out would > be really really good. > > Robbie > -- -K
[jira] [Resolved] (PROTON-707) Valgrind 'invalid read' errors in proton_tests.message.LoadSaveTest tests
[ https://issues.apache.org/jira/browse/PROTON-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti resolved PROTON-707. --- Resolution: Fixed Fix Version/s: (was: 0.9) 0.10 > Valgrind 'invalid read' errors in proton_tests.message.LoadSaveTest tests > - > > Key: PROTON-707 > URL: https://issues.apache.org/jira/browse/PROTON-707 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.8 >Reporter: Ken Giusti > Fix For: 0.10 > > > Valgrind detects an invalid memory reference during these tests. > To reproduce: > $ valgrind -q --trace-children=yes > --suppressions=./proton_tests/valgrind.supp ./proton-test > "proton_tests.message.LoadSaveTest.*" > proton_tests.message.LoadSaveTest.testData .. > ==18459== Invalid read of size 1 > ==18459==at 0x4A0A534: memcpy@@GLIBC_2.14 (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==18459==by 0x30F0A8F1B7: PyString_FromStringAndSize (string3.h:51) > ==18459==by 0xF24F5C7: _wrap_pn_message_save (cprotonPYTHON_wrap.c:3303) > ==18459==by 0x30F0ADDC2D: PyEval_EvalFrameEx (ceval.c:4098) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADEBBC: PyEval_EvalCodeEx (ceval.c:3330) > ==18459==by 0x30F0ADD6A8: PyEval_EvalFrameEx (ceval.c:4194) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADEBBC: PyEval_EvalCodeEx (ceval.c:3330) > ==18459==by 0x30F0ADD6A8: PyEval_EvalFrameEx (ceval.c:4194) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459== Address 0x1055bb08 is 7 bytes after a block of size 17 alloc'd > ==18459==at 0x4A06409: malloc (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==18459==by 0xF24F500: _wrap_pn_message_save (cprotonPYTHON_wrap.c:4029) > ==18459==by 0x30F0ADDC2D: PyEval_EvalFrameEx (ceval.c:4098) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADEBBC: PyEval_EvalCodeEx (ceval.c:3330) > ==18459==by 0x30F0ADD6A8: PyEval_EvalFrameEx (ceval.c:4194) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADEBBC: PyEval_EvalCodeEx (ceval.c:3330) > ==18459==by 0x30F0ADD6A8: PyEval_EvalFrameEx (ceval.c:4194) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459==by 0x30F0ADD74B: PyEval_EvalFrameEx (ceval.c:4184) > ==18459== > pass -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-476) Support a user-context for SASL and SSL objects.
[ https://issues.apache.org/jira/browse/PROTON-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti closed PROTON-476. - Resolution: Won't Fix Fix Version/s: (was: 0.9) Probably no longer necessary - the new attachments interface allows multiple contexts to be associated with the transport (which is the parent of both the SASL and SSL objects). > Support a user-context for SASL and SSL objects. > > > Key: PROTON-476 > URL: https://issues.apache.org/jira/browse/PROTON-476 > Project: Qpid Proton > Issue Type: New Feature > Components: proton-c >Affects Versions: 0.5 >Reporter: Ken Giusti >Assignee: Ken Giusti > > For consistency and convenience, I'd like to add an API that would allow an > application-defined context be associated with the pn_sasl_t, > pn_ssl_domain_t, and pn_ssl_t objects. This api would follow the convention > used by other proton classes (eg, pn_connection_get_context(), > pn_connection_set_context()). ex: > void *pn_sasl_get_context(pn_sasl_t); > void pn_sasl_set_context(pn_sasl_t *, void *context); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-851) Implement pn_i_genuuid for platforms that not support uuid_generate
[ https://issues.apache.org/jira/browse/PROTON-851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti closed PROTON-851. - Resolution: Not A Problem Assignee: Ken Giusti 0.10 removes the need for a uuid generator. > Implement pn_i_genuuid for platforms that not support uuid_generate > --- > > Key: PROTON-851 > URL: https://issues.apache.org/jira/browse/PROTON-851 > Project: Qpid Proton > Issue Type: New Feature > Components: proton-c >Affects Versions: 0.9 > Environment: uclinux m68k >Reporter: Tomasz Nowicki >Assignee: Ken Giusti > Labels: patch > Fix For: 0.10 > > Attachments: proton-c-UUID.patch > > Original Estimate: 0h > Remaining Estimate: 0h > > Added pn_i_genuuid implementation for uclinux m68k (or any posix with no > uuid_generate). Method is using rand() to generate UUID. Define > USE_INTERNAL_UUID is introducedin CMAKE for proton-c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: proton 0.10 blocker
On 17 July 2015 at 23:32, Gordon Sim wrote: > On 07/17/2015 10:04 PM, Rafael Schloming wrote: >> >> On Fri, Jul 17, 2015 at 12:47 PM, Gordon Sim wrote: >> >>> On 07/17/2015 08:15 PM, Rafael Schloming wrote: >>> On Fri, Jul 17, 2015 at 11:56 AM, Gordon Sim wrote: On 07/17/2015 07:17 PM, Rafael Schloming wrote: > > > Given this I believe the incref/decref pair is indeed running into >> >> problems >> when being invoked from inside a finalizer. I'd be curious if an >> alternative fix would work. I suspect you could replace the additional >> conditions you added to the if predicate with this: >> >> pn_refcount(endpoint) > 0 >> >> > If the refcount is *not* 0, what does the incref/decref sequence > accomplish? > I believe the answer to this is the same as the answer I just posted on the other thread, i.e. the incref may trigger the incref hook (see pn_xxx_incref in engine.c), and this in turn may update the endpoint state and adjust the refcount accordingly. The decref may then end up finalizing the object. >>> >>> Right, understood now. >>> >>> Unfortunately replacing the additional conditions with just that check on >>> the refcount doesn't prevent the crash though. >>> >> >> Doh, not the result I was hoping for. Does it yield the same stack trace >> as >> before? > > > Yes > Given that and all who looked thinking the earlier proposal was safe, is it worth going with that change at least for now? Knocking things off the blocker list and getting an RC (or even just a beta) out would be really really good. Robbie
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user gemmellr commented on the pull request: https://github.com/apache/qpid-proton/pull/47#issuecomment-122924850 @astitcher ok great --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user astitcher commented on the pull request: https://github.com/apache/qpid-proton/pull/47#issuecomment-122922144 @gemmellr I'm going to try to characterise the python 3 build issue a bit better then raise a JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user gemmellr commented on the pull request: https://github.com/apache/qpid-proton/pull/47#issuecomment-122902052 @dnwe Yeah, I'm a bit lost with these bits I'm afraid hence the sugestion of mentioning any issues more directly on the list so those with a clue are aware something needs attention, as they likely won't have noticed these comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user dnwe commented on the pull request:
https://github.com/apache/qpid-proton/pull/47#issuecomment-122900045
@gemmellr the error message for py26 is quite odd. It seems like its
failing on the new "compile proton from python if libqpid-proton isn't
available to link against" that was added to the setup.py. I'm not sure why the
libqpid-proton build via cmake isn't considered usable though.
Here's the output log
```
Processing ./.tox/dist/python-qpid-proton-0.10.0.zip
Building wheels for collected packages: python-qpid-proton
Running setup.py bdist_wheel for python-qpid-proton
Complete output from command
/home/travis/build/apache/qpid-proton/proton-c/bindings/python/.tox/py26/bin/python2.6
-c "import
setuptools;__file__='/tmp/pip-zTErui-build/setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpTPU6sFpip-wheel-:
running bdist_wheel
running build
running build_ext
running configure
building '_cproton' extension
swigging cproton.i to cproton_wrap.c
swig -python -threads -o cproton_wrap.c cproton.i
cproton.i:394: Error: Unable to find 'proton/cproton.i'
error: command 'swig' failed with exit status 1
Failed building wheel for python-qpid-proton
Failed to build python-qpid-proton
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] qpid-proton pull request: NO-JIRA: Change travis configuration to ...
Github user gemmellr commented on the pull request: https://github.com/apache/qpid-proton/pull/47#issuecomment-122896223 Do the earlier comments mean that it no longer tests on 2.6 but did previously? Is that perhaps because the newer infrastructure doesn't include 2.6? or perhaps since the config is now specifying 2.7 explicitly? Possibly something worth mentioning more directly on the list. Similarly, is Ken aware of the issue you saw with Python 3.4? Might also be worth pointing out. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
