[jira] [Commented] (PROTON-804) the transport doesn't always shutdown its output properly
[ https://issues.apache.org/jira/browse/PROTON-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287347#comment-14287347 ] Rafael H. Schloming commented on PROTON-804: I believe this is a release blocker since if my understanding of the bug is correct, then any select loop that uses the transport can be trivially put into a busy loop using this bug. In fact it seems to be fairly easily triggered in a non malicious way if the connecting client dies at the right moment. (That is how I discovered it.) the transport doesn't always shutdown its output properly - Key: PROTON-804 URL: https://issues.apache.org/jira/browse/PROTON-804 Project: Qpid Proton Issue Type: Bug Reporter: Rafael H. Schloming Assignee: Andrew Stitcher Attachments: transport-test.patch I believe the issue is related to the autodetect code andrew added not too long ago. I've attached a patch that has a test case with a relatively minimal reproducer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
removing the proton driver API
I just noticed that dispatch seems to have it's own copy of driver.c now. I think that means the driver API is now dead code as messenger, the new reactor stuff, etc all use the newer selector API. Is anyone else using/aware of anyone using this code in anyway? I would like to at a minimum deprecate it for 0.9, and preferably remove it entirely if it is in fact currently unused. --Rafael
Re: removing the proton driver API
On Thu, Jan 22, 2015 at 10:50:22AM -0500, Rafael Schloming wrote: I just noticed that dispatch seems to have it's own copy of driver.c now. I think that means the driver API is now dead code as messenger, the new reactor stuff, etc all use the newer selector API. Is anyone else using/aware of anyone using this code in anyway? I would like to at a minimum deprecate it for 0.9, and preferably remove it entirely if it is in fact currently unused. Are there other APIs, like listener or connector, that'll be removed as well? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgps7d72Xxvta.pgp Description: PGP signature
[jira] [Resolved] (PROTON-800) [Windows C] Reactor test times out
[
https://issues.apache.org/jira/browse/PROTON-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cliff Jansen resolved PROTON-800.
-
Resolution: Fixed
Fix Version/s: 0.9
Assignee: Cliff Jansen
As per the other comments, plus creating a special iocp descriptor that just
holds a deadline for socket-less selectables.
[Windows C] Reactor test times out
--
Key: PROTON-800
URL: https://issues.apache.org/jira/browse/PROTON-800
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Affects Versions: 0.8
Environment: Windows Server 2012 R2, Visual Studio 2010 x64
Reporter: Chuck Rolke
Assignee: Cliff Jansen
Fix For: 0.9
ctest -VV times out on the reactor test.
Instrumenting and debugging shows the failure in test_reactor_acceptor
Running Debug instead of RelWithDebInfo crashes with an AccVio
{noformat}
qpid-protond.dll!pni_iocpdesc_map_get(iocp_t * iocp, unsigned __int64
s) Line 773 + 0xa bytes C++
qpid-protond.dll!pn_close(pn_io_t * io, unsigned __int64 socket) Line
350 + 0x16 bytes C++
qpid-protond.dll!pni_acceptor_finalize(pn_selectable_t * sel) Line 56
C++
qpid-protond.dll!pn_selectable_finalize(void * obj) Line 81C++
qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void *
object) Line 100 C++
qpid-protond.dll!pn_list_finalize(void * object) Line 205 C++
qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void *
object) Line 100 C++
qpid-protond.dll!pn_decref(void * object) Line 253 C++
qpid-protond.dll!pn_reactor_finalize(pn_reactor_t * reactor) Line 77
C++
qpid-protond.dll!pn_reactor_finalize_cast(void * object) Line 113 +
0x28 bytes C++
qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void *
object) Line 100 C++
qpid-protond.dll!pn_decref(void * object) Line 253 C++
qpid-protond.dll!pn_reactor_free(pn_reactor_t * reactor) Line 132
C++
c-reactor-tests.exe!test_reactor_acceptor() Line 171 C++
c-reactor-tests.exe!main(int argc, char * * argv) Line 446 C++
c-reactor-tests.exe!__tmainCRTStartup() Line 555 + 0x19 bytes C
c-reactor-tests.exe!mainCRTStartup() Line 371 C
kernel32.dll!7ff90cc913d2()
[Frames below may be incorrect and/or missing, no symbols loaded for
kernel32.dll]
ntdll.dll!7ff90e1003c4()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Ruby and the Engine APIs
The most important thing to get worked out for this is the memory
management semantics between C and Ruby. From what I can tell from your
branch, it looks like you haven't done that yet.
As I've said before, the first step to wrapping the engine API is to work
out a simple strategy for wrapping C objects from the host language. In the
case of python this is done with the wrapper.py mixin. It uses various
python hooks as well as elements of python's C extension API to allow C
objects to hold references to python objects and keep them alive according
to python's memory management semantics. Just translating this wrapper code
from python to ruby won't do much good as Ruby's memory management
semantics are different and it's C extension API is different.
The first step you should take before bothering to wrap any code is to
figure out how you can have C objects hold pointers to ruby objects without
either incurring memory leaks or seg faults. Judging from what you've done
in ruby.i:
+ static void pn_rbref_incref(void *object) { + // no reference counting
in Ruby + } + + static void pn_rbref_decref(void *object) { + // no
reference counting in Ruby + } + + static int pn_rbref_refcount(void
*object) { + return 1; + } + + static const pn_class_t
*pn_rbref_reify(void *object) { + return PN_RBREF; + }
I suspect if you were to actually try anything significant you would incur
either seg faults or memory leaks or both, depending on precisely how Ruby
GC works. The excerpt from your branch above is creating a class of pointer
that will lie to proton and tell it that it always has a reference when it
doesn't really, and yet also lie to the Ruby VM because it is keeping a
pointer to a Ruby object without any way for the Ruby GC mechanism to be
informed.
I recommend starting with a very simple test case for one object, i.e.
define a wrapper mixin for ruby and use it to wrap just one proton object,
e.g. Connection. Don't bother with wrapping any of the methods on
Connection, all you really want to do is verify that object
creation/deletion works properly. Once you've done that you can run
creation/deletion in a loop and verify that you don't leak memory. Before
this though, you'll need to read up on Ruby's C extension API in order to
figure out how to interface to it's memory management system so that the C
objects can hold pointers to Ruby objects and actually keep them alive, and
also free them when the C objects go away. Until you can do this and get
the basic loop test working with a simple object, there isn't really much
point in trying to wrap anything else.
--Rafael
On Wed, Jan 21, 2015 at 2:22 PM, Darryl L. Pierce dpie...@redhat.com
wrote:
I've been working on providing the low-level engine APIs in Ruby over
the past few weeks, and from what I can see I think I'm near to
completion regarding wrapping them.
However, I'm not sure if I'm missing something since I don't see a way
to use the code to create a connection. :D
Specifically, what I want to do next is create examples of working with
these low-level APIs, initially a simple send/receive example would be
best with a receiver that listens for a connection and a sender that
connects to it, all while being able to toggle tracing the
communication, etc. I see nothing existing for other languages to use as
a guide, so am a little stumped at the moment.
The work I've done to date is here:
https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: removing the proton driver API
Yes, those are part of the driver API. --Rafael On Thu, Jan 22, 2015 at 11:08 AM, Darryl L. Pierce dpie...@redhat.com wrote: On Thu, Jan 22, 2015 at 10:50:22AM -0500, Rafael Schloming wrote: I just noticed that dispatch seems to have it's own copy of driver.c now. I think that means the driver API is now dead code as messenger, the new reactor stuff, etc all use the newer selector API. Is anyone else using/aware of anyone using this code in anyway? I would like to at a minimum deprecate it for 0.9, and preferably remove it entirely if it is in fact currently unused. Are there other APIs, like listener or connector, that'll be removed as well? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/
[jira] [Commented] (PROTON-804) the transport doesn't always shutdown its output properly
[ https://issues.apache.org/jira/browse/PROTON-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287893#comment-14287893 ] Rafael H. Schloming commented on PROTON-804: rschloming: basically what happens is the triggering connection sends a sasl-init frame and the dies rschloming: and the server side of the connection reports a No valid protocol header found rschloming: from line 212 of transport.c rschloming: and then the output gets stuck in pn_io_layer_output_null rschloming: which just always returns 0 stitchedup: I doubt that would have worked before the autodetect code either (but the symptom would have been different) stitchedup: because the sasl code has no way to deal with premature failure rschloming: it's possible that your changes have simply made it more likely to occur if the eliminate pipelining on the client side rschloming: but I'm pretty sure it's a recent thing because it gets triggered very easily by a client that has a simple typo in it rschloming: i.e. just firing up and dying stitchedup: hmm, you should never get into the null layer after failure rschloming: yeah, I didn't think that looked right stitchedup: you will get there though rschloming: it doesn't seem to do that if you just connect and then die without sending any data at all stitchedup: because if the server tries to do output before autodetect has happened it can't know what to output rschloming: sure, but it seems like failure should force a given response rschloming: failure should basically force it to assume AMQP on the output side I woudl think rschloming: since it is sending an amqp close frame to indicate the error stitchedup: ok It would really help it you pasted the above info into the bug itself the nuget about pn_io_..._null would have actually been useful the transport doesn't always shutdown its output properly - Key: PROTON-804 URL: https://issues.apache.org/jira/browse/PROTON-804 Project: Qpid Proton Issue Type: Bug Reporter: Rafael H. Schloming Assignee: Andrew Stitcher Attachments: transport-test.patch I believe the issue is related to the autodetect code andrew added not too long ago. I've attached a patch that has a test case with a relatively minimal reproducer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
