[jira] [Created] (PROTON-1176) Core dump if reactor creation fails
Cliff Jansen created PROTON-1176: Summary: Core dump if reactor creation fails Key: PROTON-1176 URL: https://issues.apache.org/jira/browse/PROTON-1176 Project: Qpid Proton Issue Type: Bug Components: python-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen If a BlockingConnection fails creating a reactor from Proton-C, the wrapper constructor for the container fails on a null "impl" value, causing a core dump rather than an exception. This can happen if the process runs out of file descriptors and cannot allocate the self-pipes needed by the reactor. Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1319165 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1175) Document BlockingConnection resource cleanup
Cliff Jansen created PROTON-1175: Summary: Document BlockingConnection resource cleanup Key: PROTON-1175 URL: https://issues.apache.org/jira/browse/PROTON-1175 Project: Qpid Proton Issue Type: Improvement Components: python-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen The documentation is silent on resource cleanup for BlockingConnections. Currently if the BlockingConnection is not closed, the file descriptors and other reactor machinery are leaked. Standard Python techniques should be used to ensure resource clean up (try..finally blocks etc.) Only the sync_client example hints this might be desirable, let alone necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1057) Windows SChannel SSL test failure
[ https://issues.apache.org/jira/browse/PROTON-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153117#comment-15153117 ] Cliff Jansen commented on PROTON-1057: -- Further investigation reveals: - a simple test case is hard to create. Excluding all or most of the preceding tests makes the crash go away. - small changes to the executable make the crash go away: e.g. compiling schannel.c without optimization, or adding stack allocations to any of the suspect functions. - The crash appears to happen while unwinding the stack from a call to verify_peer() from within client_handshake(). > Windows SChannel SSL test failure > - > > Key: PROTON-1057 > URL: https://issues.apache.org/jira/browse/PROTON-1057 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.12.0 > Environment: Windows only. > Reporter: Cliff Jansen >Assignee: Cliff Jansen > Labels: windows > > This problem started since PROTON-1048 which did not touch the Proton-C > SChannel related code - it just tweaked the test infrastructure to use > alternate certificate formats for Windows. > The proton_tests.ssl.SslTest.test_server_hostname_authentication test > randomly crashes on Windows, presumably from some memory corruption > somewhere. It often runs fine. Seems to happen more frequently on some > systems than others. Not yet seen on a 64 bit build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1132) Timeout in valgrind test
[ https://issues.apache.org/jira/browse/PROTON-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1132. Resolution: Fixed > Timeout in valgrind test > > > Key: PROTON-1132 > URL: https://issues.apache.org/jira/browse/PROTON-1132 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.13.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > > Since PROTON-1115 was fixed some CI tests are failing with timeouts. Working > theory is that re-introducing valgrind just takes longer on some slower test > machinery. Will increase the timeout (doesn't slow down a successful run) > and close the JIRA if runs complete successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142931#comment-15142931 ] Cliff Jansen commented on PROTON-1071: -- bumped to 0.13. Still actively working this. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11.0 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.13.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen updated PROTON-1071: - Fix Version/s: (was: 0.12.0) 0.13.0 > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11.0 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.13.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1115) c++: memory leak in ssl examples
[ https://issues.apache.org/jira/browse/PROTON-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1115. Resolution: Fixed Fix Version/s: 0.13.0 > c++: memory leak in ssl examples > > > Key: PROTON-1115 > URL: https://issues.apache.org/jira/browse/PROTON-1115 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Alan Conway > Assignee: Cliff Jansen > Fix For: 0.13.0 > > > The ssl C++ examples are leaking memory, valgrind output for for ssl follows, > the ssl_client_certs example has the same leak. > Putting printfs in ssl_domain_impl shows it is constructed 34 times and freed > 21 times. This might explain the leak, but also seems like an unreasonably > large number of ssl_domains for such a simple test. > {code} > [aconway@wallace cpp (master $=)]$ pwd; valgrind --leak-check=full > ./ssl_client_cert 'amqps://127.0.0.1:19108/examples' > '/home/aconway/proton/examples/cpp/ssl_certs' > /home/aconway/proton/debug/examples/cpp > ==16788== Memcheck, a memory error detector > ==16788== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. > ==16788== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info > ==16788== Command: ./ssl_client_cert amqps://127.0.0.1:19108/examples > /home/aconway/proton/examples/cpp/ssl_certs > ==16788== > Inbound server connection connected via SSL. Protocol: TLSv1/SSLv3 > Inbound client certificate identity CN=test_client > Outgoing client connection connected via SSL. Server certificate identity > CN=test_server > Hello World! > ==16788== > ==16788== HEAP SUMMARY: > ==16788== in use at exit: 185,521 bytes in 3,585 blocks > ==16788== total heap usage: 33,114 allocs, 29,529 frees, 3,858,750 bytes > allocated > ==16788== > ==16788== 4,913 (200 direct, 4,713 indirect) bytes in 1 blocks are definitely > lost in loss record 775 of 780 > ==16788==at 0x4C28C50: malloc (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==16788==by 0x5615997: CRYPTO_malloc (mem.c:342) > ==16788==by 0x56ECA43: asn1_item_ex_combine_new (tasn_new.c:179) > ==16788==by 0x56EF4F0: ASN1_item_ex_d2i (tasn_dec.c:390) > ==16788==by 0x56EFACA: ASN1_item_d2i (tasn_dec.c:146) > ==16788==by 0x5356AA4: ssl3_get_server_certificate (s3_clnt.c:1228) > ==16788==by 0x535B921: ssl3_connect (s3_clnt.c:345) > ==16788==by 0x5365453: ssl23_get_server_hello (s23_clnt.c:799) > ==16788==by 0x5365453: ssl23_connect (s23_clnt.c:228) > ==16788==by 0x5365DE1: ssl23_read (s23_lib.c:133) > ==16788==by 0x53882D4: ssl_read (bio_ssl.c:165) > ==16788==by 0x56BE06B: BIO_read (bio_lib.c:210) > ==16788==by 0x5113EB5: process_input_ssl (openssl.c:917) > ==16788==by 0x50FBA9A: transport_consume (transport.c:1751) > ==16788==by 0x50FEFEF: pn_transport_process (transport.c:2860) > ==16788==by 0x5103E99: pni_connection_readable (connection.c:164) > ==16788==by 0x510DF7D: pn_selectable_readable (selectable.c:204) > ==16788==by 0x51052CE: pni_handle_quiesced (iohandler.c:41) > ==16788==by 0x5105505: pn_iodispatch (iohandler.c:103) > ==16788==by 0x51037B4: pn_handler_dispatch (handler.c:102) > ==16788==by 0x4E84386: proton::(anonymous > namespace)::override_handler::on_unhandled(proton::proton_event&) > (container_impl.cpp:107) > ==16788==by 0x4EA048F: > proton::proton_handler::on_reactor_quiesced(proton::proton_event&) > (proton_handler.cpp:32) > ==16788==by 0x4E9F838: > proton::proton_event::dispatch(proton::proton_handler&) (proton_event.cpp:105) > ==16788==by 0x4E84210: proton::(anonymous > namespace)::handler_context::dispatch(pn_handler_t*, pn_event_t*, > pn_event_type_t) (container_impl.cpp:71) > ==16788==by 0x51037B4: pn_handler_dispatch (handler.c:102) > ==16788==by 0x51030F4: pn_reactor_process (reactor.c:404) > ==16788==by 0x51034DF: pn_reactor_run (reactor.c:480) > ==16788==by 0x4EA1993: proton::reactor::run() (reactor.cpp:36) > ==16788==by 0x4E83F11: proton::container::run() (container.cpp:64) > ==16788==by 0x405B0D: main (ssl_client_cert.cpp:137) > ==16788== > ==16788== 4,913 (200 direct, 4,713 indirect) bytes in 1 blocks are definitely > lost in loss record 776 of 780 > ==16788==at 0x4C28C50: malloc (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==16788==by 0x5615997: CRYPTO_malloc (mem.c:342) > ==16788==by 0x56ECA43: asn1_item_ex_combine_new (tasn_new.c:179) >
[jira] [Commented] (PROTON-1129) C++ binding test failure with older python
[ https://issues.apache.org/jira/browse/PROTON-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136355#comment-15136355 ] Cliff Jansen commented on PROTON-1129: -- Multiple fixes for python versions < 2.7 unittest.setUpTest/tearDownTest (new in python 2.7) unittest.assertIn (new in python 2.7) Event.wait() (return arg changed in python 2.7) > C++ binding test failure with older python > -- > > Key: PROTON-1129 > URL: https://issues.apache.org/jira/browse/PROTON-1129 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.13.0 > Reporter: Cliff Jansen >Assignee: Cliff Jansen > > See errors like: > AttributeError: type object 'ContainerExampleTest' has no attribute 'broker' > Working theory: unittest.TestCase.setUpClass() not implemented until python > 2.7. The classmethod overrides are never called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1115) c++: memory leak in ssl examples
[ https://issues.apache.org/jira/browse/PROTON-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128272#comment-15128272 ] Cliff Jansen commented on PROTON-1115: -- Since PROTON-1119, I see exactly two ssl_domain_impl constructor calls and two matching delete()'s. The valgrind errors appear exactly the same. They appear to occur during IO activity as opposed to IO setup/configuration. I have checked in a temporary change to turn off valgrind for SSL so that CI stops blowing up everywhere. But I will leave this JIRA open pending further investigation. > c++: memory leak in ssl examples > > > Key: PROTON-1115 > URL: https://issues.apache.org/jira/browse/PROTON-1115 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Alan Conway > Assignee: Cliff Jansen > > The ssl C++ examples are leaking memory, valgrind output for for ssl follows, > the ssl_client_certs example has the same leak. > Putting printfs in ssl_domain_impl shows it is constructed 34 times and freed > 21 times. This might explain the leak, but also seems like an unreasonably > large number of ssl_domains for such a simple test. > {code} > [aconway@wallace cpp (master $=)]$ pwd; valgrind --leak-check=full > ./ssl_client_cert 'amqps://127.0.0.1:19108/examples' > '/home/aconway/proton/examples/cpp/ssl_certs' > /home/aconway/proton/debug/examples/cpp > ==16788== Memcheck, a memory error detector > ==16788== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. > ==16788== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info > ==16788== Command: ./ssl_client_cert amqps://127.0.0.1:19108/examples > /home/aconway/proton/examples/cpp/ssl_certs > ==16788== > Inbound server connection connected via SSL. Protocol: TLSv1/SSLv3 > Inbound client certificate identity CN=test_client > Outgoing client connection connected via SSL. Server certificate identity > CN=test_server > Hello World! > ==16788== > ==16788== HEAP SUMMARY: > ==16788== in use at exit: 185,521 bytes in 3,585 blocks > ==16788== total heap usage: 33,114 allocs, 29,529 frees, 3,858,750 bytes > allocated > ==16788== > ==16788== 4,913 (200 direct, 4,713 indirect) bytes in 1 blocks are definitely > lost in loss record 775 of 780 > ==16788==at 0x4C28C50: malloc (in > /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) > ==16788==by 0x5615997: CRYPTO_malloc (mem.c:342) > ==16788==by 0x56ECA43: asn1_item_ex_combine_new (tasn_new.c:179) > ==16788==by 0x56EF4F0: ASN1_item_ex_d2i (tasn_dec.c:390) > ==16788==by 0x56EFACA: ASN1_item_d2i (tasn_dec.c:146) > ==16788==by 0x5356AA4: ssl3_get_server_certificate (s3_clnt.c:1228) > ==16788==by 0x535B921: ssl3_connect (s3_clnt.c:345) > ==16788==by 0x5365453: ssl23_get_server_hello (s23_clnt.c:799) > ==16788==by 0x5365453: ssl23_connect (s23_clnt.c:228) > ==16788==by 0x5365DE1: ssl23_read (s23_lib.c:133) > ==16788==by 0x53882D4: ssl_read (bio_ssl.c:165) > ==16788==by 0x56BE06B: BIO_read (bio_lib.c:210) > ==16788==by 0x5113EB5: process_input_ssl (openssl.c:917) > ==16788==by 0x50FBA9A: transport_consume (transport.c:1751) > ==16788==by 0x50FEFEF: pn_transport_process (transport.c:2860) > ==16788==by 0x5103E99: pni_connection_readable (connection.c:164) > ==16788==by 0x510DF7D: pn_selectable_readable (selectable.c:204) > ==16788==by 0x51052CE: pni_handle_quiesced (iohandler.c:41) > ==16788==by 0x5105505: pn_iodispatch (iohandler.c:103) > ==16788==by 0x51037B4: pn_handler_dispatch (handler.c:102) > ==16788==by 0x4E84386: proton::(anonymous > namespace)::override_handler::on_unhandled(proton::proton_event&) > (container_impl.cpp:107) > ==16788==by 0x4EA048F: > proton::proton_handler::on_reactor_quiesced(proton::proton_event&) > (proton_handler.cpp:32) > ==16788==by 0x4E9F838: > proton::proton_event::dispatch(proton::proton_handler&) (proton_event.cpp:105) > ==16788==by 0x4E84210: proton::(anonymous > namespace)::handler_context::dispatch(pn_handler_t*, pn_event_t*, > pn_event_type_t) (container_impl.cpp:71) > ==16788==by 0x51037B4: pn_handler_dispatch (handler.c:102) > ==16788==by 0x51030F4: pn_reactor_process (reactor.c:404) > ==16788==by 0x51034DF: pn_reactor_run (reactor.c:480) > ==16788==by 0x4EA1993: proton::reactor::run() (reactor.cpp:36) > ==16788==by 0x4E83F11: proton::container::run() (container.cpp:64) > ==16788==by 0x405B0D: main (ssl_client_cert.cpp:137) > ==16788== > ==16788=
[jira] [Closed] (PROTON-1041) Add recurring timer example to the reactive C++ documentation
[ https://issues.apache.org/jira/browse/PROTON-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1041. Resolution: Fixed Fix Version/s: (was: 0.12.0) 0.13.0 > Add recurring timer example to the reactive C++ documentation > - > > Key: PROTON-1041 > URL: https://issues.apache.org/jira/browse/PROTON-1041 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.11.0, 0.12.0 >Reporter: Otavio Rodolfo Piske > Assignee: Cliff Jansen >Priority: Minor > Labels: patch > Fix For: 0.13.0 > > Attachments: recurring-timer-example.patch > > > The C++ documentation is missing the recurring_timer.cpp example in its > documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1027) Incorrectly handling of invalid addresses
[ https://issues.apache.org/jira/browse/PROTON-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1027. Resolution: Fixed Fixed by separate work in PROTON-1095. > Incorrectly handling of invalid addresses > - > > Key: PROTON-1027 > URL: https://issues.apache.org/jira/browse/PROTON-1027 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.11.0 > Environment: Fedora Linux 22, 64bit. > > Using built-in specs. > COLLECT_GCC=/usr/bin/gcc > COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/5.1.1/lto-wrapper > Target: x86_64-redhat-linux > Configured with: ../configure --enable-bootstrap > --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto --prefix=/usr > --mandir=/usr/share/man --infodir=/usr/share/info > --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared > --enable-threads=posix --enable-checking=release --enable-multilib > --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions > --enable-gnu-unique-object --enable-linker-build-id > --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array > --disable-libgcj --with-default-libstdcxx-abi=c++98 --with-isl > --enable-libmpx --enable-gnu-indirect-function --with-tune=generic > --with-arch_32=i686 --build=x86_64-redhat-linux > Thread model: posix > gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) > > Compiled with: > cmake -DCMAKE_INSTALL_PREFIX=/opt/devel/qpid-proton-0.11-SNAPSHOT > -DSYSINSTALL_BINDINGS=ON -DBUILD_PYTHON=OFF -DBUILD_PERL=OFF .. > --- >Reporter: Otavio Rodolfo Piske >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > The code seems to accept invalid combinations of hosts/IPs. > Having the QPid Proton source code compile, you can run one of the following > to reproduce the problem: > A. Try to connect to to a server with an invalid ip address: > ./server -a 355.355.355.412:5672/test.reactor.queue > server connected to amqp://355.355.355.412:5672/test.reactor.queue > B. Try to connect to an invalid server whose address cannot be resolved: > ./server -a host_does_not_exist:5672/test.reactor.queue > server connected to amqp://host_does_not_exist:5672/test.reactor.queue > C. Try to connect to a valid server using an invalid port: > ./server -a valid-address:5672/test.reactor.queue > server connected to amqp://valid-address:5672/test.reactor.queue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1119) C++ ssl_domain tracking uses unnecessary heap allocations
Cliff Jansen created PROTON-1119: Summary: C++ ssl_domain tracking uses unnecessary heap allocations Key: PROTON-1119 URL: https://issues.apache.org/jira/browse/PROTON-1119 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.11.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.13.0 As noted in PROTON-1115, there are more than a dozen unused ssl_domain connection option related allocations for even the simplest example program. While the actual underlying pn_ssl_domain_t object is lazily created, greater laziness can be achieved since it is pointless to track and count a known null pointer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1119) C++ ssl_domain tracking uses unnecessary heap allocations
[ https://issues.apache.org/jira/browse/PROTON-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1119. Resolution: Fixed > C++ ssl_domain tracking uses unnecessary heap allocations > - > > Key: PROTON-1119 > URL: https://issues.apache.org/jira/browse/PROTON-1119 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.11.1 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.13.0 > > > As noted in PROTON-1115, there are more than a dozen unused ssl_domain > connection option related allocations for even the simplest example program. > While the actual underlying pn_ssl_domain_t object is lazily created, greater > laziness can be achieved since it is pointless to track and count a known > null pointer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1103) C++ binding rename xxx_domain to ssl_xxx_options
[ https://issues.apache.org/jira/browse/PROTON-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1103. Resolution: Fixed > C++ binding rename xxx_domain to ssl_xxx_options > > > Key: PROTON-1103 > URL: https://issues.apache.org/jira/browse/PROTON-1103 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.11.1 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Drop use of "domain" to describe shared SSL configuration for multiple > connections. > client_domain -> ssl_client_options > server_domain -> ssl_server_options -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1102) C++ binding remove _t suffix for types
[ https://issues.apache.org/jira/browse/PROTON-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1102. Resolution: Fixed > C++ binding remove _t suffix for types > -- > > Key: PROTON-1102 > URL: https://issues.apache.org/jira/browse/PROTON-1102 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.11.1 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Not idiomatic. API improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1089) C++ binding link options
[ https://issues.apache.org/jira/browse/PROTON-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1089. Resolution: Fixed > C++ binding link options > > > Key: PROTON-1089 > URL: https://issues.apache.org/jira/browse/PROTON-1089 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.11.1 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Provide link options. > See https://reviews.apache.org/r/39222/ and subsequent improvements for the > connection options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1074) C++ cbinding SSL core dump
[ https://issues.apache.org/jira/browse/PROTON-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1074. Resolution: Fixed > C++ cbinding SSL core dump > -- > > Key: PROTON-1074 > URL: https://issues.apache.org/jira/browse/PROTON-1074 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > a8e9582 PROTON-1068: c++ remove counted_ptr use in ssl.hpp > This refactor removed the smart pointer ref counting. The underlying > pn_ssl_domain_t is not a proton ref counted object and is freed before use. > Restore the ref counting and add ssl_client_cert example to ctest suite > verification. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1117) Add link.detach method to C++ binding
Cliff Jansen created PROTON-1117: Summary: Add link.detach method to C++ binding Key: PROTON-1117 URL: https://issues.apache.org/jira/browse/PROTON-1117 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.11.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.13.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1102) C++ binding remove _t suffix for types
Cliff Jansen created PROTON-1102: Summary: C++ binding remove _t suffix for types Key: PROTON-1102 URL: https://issues.apache.org/jira/browse/PROTON-1102 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.11.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Not idiomatic. API improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1103) C++ binding rename xxx_domain to ssl_xxx_options
Cliff Jansen created PROTON-1103: Summary: C++ binding rename xxx_domain to ssl_xxx_options Key: PROTON-1103 URL: https://issues.apache.org/jira/browse/PROTON-1103 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.11.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Drop use of "domain" to describe shared SSL configuration for multiple connections. client_domain -> ssl_client_options server_domain -> ssl_server_options -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1076) C++ binding acceptor context
[ https://issues.apache.org/jira/browse/PROTON-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1076. Resolution: Fixed > C++ binding acceptor context > > > Key: PROTON-1076 > URL: https://issues.apache.org/jira/browse/PROTON-1076 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Use new acceptor context to configure incoming connections using new C > capability (PROTON-1054) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1048) Proton-C ssl tests on Windows with SChannel
[ https://issues.apache.org/jira/browse/PROTON-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1048. Resolution: Fixed > Proton-C ssl tests on Windows with SChannel > --- > > Key: PROTON-1048 > URL: https://issues.apache.org/jira/browse/PROTON-1048 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.12.0 > Environment: Windows with SChannel > Reporter: Cliff Jansen >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Create file based certificates (as opposed to registry certificates requiring > user intervention) for Windows SChannel in pkcs12 format from the existing > OpenSSL certificates and keys. > Update the test scripts to find the alternate certificates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1037) Add support for setting/getting message properties
[ https://issues.apache.org/jira/browse/PROTON-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1037. Resolution: Fixed > Add support for setting/getting message properties > -- > > Key: PROTON-1037 > URL: https://issues.apache.org/jira/browse/PROTON-1037 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.11.0 >Reporter: Otavio Rodolfo Piske > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Please add support for setting custom properties to a message in the C++ > reactive API -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1089) C++ binding link options
Cliff Jansen created PROTON-1089: Summary: C++ binding link options Key: PROTON-1089 URL: https://issues.apache.org/jira/browse/PROTON-1089 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.11.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Provide link options. See https://reviews.apache.org/r/39222/ and subsequent improvements for the connection options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1026) Invalid queue/destination causes a segmentation fault
[ https://issues.apache.org/jira/browse/PROTON-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1026. Resolution: Fixed > Invalid queue/destination causes a segmentation fault > - > > Key: PROTON-1026 > URL: https://issues.apache.org/jira/browse/PROTON-1026 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.11 > Environment: Fedora Linux 22, 64bit. > > Using built-in specs. > COLLECT_GCC=/usr/bin/gcc > COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/5.1.1/lto-wrapper > Target: x86_64-redhat-linux > Configured with: ../configure --enable-bootstrap > --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto --prefix=/usr > --mandir=/usr/share/man --infodir=/usr/share/info > --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared > --enable-threads=posix --enable-checking=release --enable-multilib > --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions > --enable-gnu-unique-object --enable-linker-build-id > --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array > --disable-libgcj --with-default-libstdcxx-abi=c++98 --with-isl > --enable-libmpx --enable-gnu-indirect-function --with-tune=generic > --with-arch_32=i686 --build=x86_64-redhat-linux > Thread model: posix > gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) > > Compiled with: > cmake -DCMAKE_INSTALL_PREFIX=/opt/devel/qpid-proton-0.11-SNAPSHOT > -DSYSINSTALL_BINDINGS=ON -DBUILD_PYTHON=OFF -DBUILD_PERL=OFF .. > --- >Reporter: Otavio Rodolfo Piske >Assignee: Cliff Jansen >Priority: Critical > Labels: crash > Fix For: 0.12.0 > > Attachments: proton-1026-backtrace.txt > > > Using an invalid path/destination in the address causes the code to crash > with SIGSEGV. > Having the QPid Proton code compiled, please use these teps to reproduce: > 1. Configure a broker > 2. Use the server example to send a message to a non-existent queue: > ./server -a server_address:5672/this_destination_does_no_exist -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1020) Typos in the error messages
[ https://issues.apache.org/jira/browse/PROTON-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1020. Resolution: Fixed > Typos in the error messages > --- > > Key: PROTON-1020 > URL: https://issues.apache.org/jira/browse/PROTON-1020 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.11 >Reporter: Otavio Rodolfo Piske > Assignee: Cliff Jansen >Priority: Minor > Fix For: 0.12.0 > > Attachments: proton-1020-error-message-typos.patch > > > Some error messages in the files messaging_event.cpp and proton_bits.cpp > contain typos that should be fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1076) C++ binding acceptor context
Cliff Jansen created PROTON-1076: Summary: C++ binding acceptor context Key: PROTON-1076 URL: https://issues.apache.org/jira/browse/PROTON-1076 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Use new acceptor context to configure incoming connections using new C capability (PROTON-1054) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1074) C++ cbinding SSL core dump
Cliff Jansen created PROTON-1074: Summary: C++ cbinding SSL core dump Key: PROTON-1074 URL: https://issues.apache.org/jira/browse/PROTON-1074 Project: Qpid Proton Issue Type: Bug Components: cpp-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 a8e9582 PROTON-1068: c++ remove counted_ptr use in ssl.hpp This refactor removed the smart pointer ref counting. The underlying pn_ssl_domain_t is not a proton ref counted object and is freed before use. Restore the ref counting and add ssl_client_cert example to ctest suite verification. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046607#comment-15046607 ] Cliff Jansen commented on PROTON-1071: -- There is no concept in Windows of a general select/poll that works on generic file descriptors/handles. You have a separate synchronous wait model for each file type. On top of that, you have a fancy proactor model (IO completion ports, IOCP) for asynchronous IO on a few file types: sockets and named pipes, but not console IO and anonymous pipes. The IOCP model is the preferred scalable IO mechanism for one or many threads. If using pn_selectable_set_fd(handle_xx) with IOCP, it cannot work for any handle_xx that is not IOCP aware. Well not without adding an extra thread that synchronously checks it (something that may have to be done for console IO). PROTON-688 was an attempt to find a useful subset of Posix and Windows capabilities (and other platforms, if identified) for performant IO (mostly looking at the dispatch router model). Threading consequences were one consideration, but so was mere achievability. The network (socket) IO plus pipe (interrupt mainly) IO were the prime candidates. Hence an "fd" in pn_selectable_set_fd is actually a pn_socket_t, not an int. An external socket may be used in this case, but also a pn_pipe object. In Posix, the pn_pipe is not an OS socket. In Windows, it currently is but may become something else, perhaps even an internal construct in future. Another difference to note, a Posix fd can move between different collections fds for poll/select. A Windows handle can be registered exactly once with a collection. It can never be moved to another selector. It is associated with a single IOCP port until it is closed. > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047552#comment-15047552 ] Cliff Jansen commented on PROTON-1071: -- I would suggest the following possibilities: 1. C API extension - Assert no IO thread safety in Proton, i.e. weaker than dispatch (except next line) - new pn_io_iterrupt(pn_io_t *io, handler); <- may be called on any thread - add interrupt event - rework EventInjector to use interrupt event 2. Make Windows pn_pipe thread safe on write side (formalize self-pipe trick) - Assert no IO thread safety in Proton, i.e. weaker than dispatch (with single exception) - pn_write on the write-side fd/handle of a pn_pipe may be called on any thread - rework EventInjector to use pn_pipe and pn_read/pn_write - Assume multi-threaded Proton applications will use their own IO and connection_engine 3. Strive for Posix level of thread safety - Make same thread safety guarantee on Windows io.c and selector.c as Posix via locking - rework EventInjector to use pn_pipe and pn_read/pn_write - Assume high performance applications will use their own IO and connection_engine - Fix API for pn_io_error and pn_io_wouldblock While not related to the issue of event injection, it is worth noting that the Windows IO could be broadened in each of the above scenarios to include filesystem handles (easy) and stdin (medium easy, but requiring an extra Proton-owned thread) and probably anything else (less easy). External unbound sockets are already supported with the caveat that once selected, they are forever bound to the Proton selector's completion port and cannot be used within IOCP elsewhere (another Proton pn_io_t or external IOCP-based selector). > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1052) SSL support in C++ reactor client
[ https://issues.apache.org/jira/browse/PROTON-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1052. Resolution: Fixed > SSL support in C++ reactor client > - > > Key: PROTON-1052 > URL: https://issues.apache.org/jira/browse/PROTON-1052 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Provide SSL configuration at container level and overrides per outgoing > connection and listener -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1053) SASL support in C++ reactor client
[ https://issues.apache.org/jira/browse/PROTON-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1053. Resolution: Fixed > SASL support in C++ reactor client > -- > > Key: PROTON-1053 > URL: https://issues.apache.org/jira/browse/PROTON-1053 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Affects Versions: 0.12.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > > Provide SASL operation and configuration at the container level and overrides > per outgoing connection and listener. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045851#comment-15045851 ] Cliff Jansen commented on PROTON-1071: -- This JIRA cuts across a number of Proton IO and threading issues. EventInjector as coded violates Proton's io model: the pipe needs to come from pn_pipe() and the reading and writing via pn_read() and pn_write(), all on the reactor's pn_io_t object. It works by fortuitous accident on Posix. Coding EventInjector to work this way for Posix is presumably simple. For Windows, it further requires tackling deferred thread safety, at least for pipes and the associated pn_io_t->selector (but that may pull in everything). As noted in PROTON-640, there is no thread safety in Proton io on Windows other than a weak guarantee that two separate threads may independently work on separate pn_io_t objects. PROTON-688 attempted to document the stronger but still limited thread-safety available in the Posix implementation of Proton io. This is the model of concurrency used by Qpid Dispatch Router and perhaps the model that the Windows implementation should strive to. There is an opposing point of view that pushing thread safety into Proton is counterproductive: applications know what threading/io model works best for their workload. Hence the rising interest in connection_engine. EventInjector seems like a pretty important use case for coordinating threads. Alternatively, a more limited (or dedicated) api extension, perhaps pn_reactor_inject() which does platform-specific thread-safe coordination of something to be collected and the pn_io_t->selector might be more clear in its intention (as opposed to knowing that pn_write/read/select work a certain way together). The Windows io.c and related code could be made to work at the same level of thread safety as Posix, but with obvious locking overhead. If the assumption is that an application needing high performance IO will just use the connection_engine and manage the IO itself, perhaps there is no need to care about the locking penalty. See also PROTON-889 regarding general thread safety outages in Proton on all platforms for pn_io_error() and pn_io_wouldblock(). > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-534) Rewrite driver.c to use OS-neutral io.c and selector.c
[ https://issues.apache.org/jira/browse/PROTON-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-534. --- Resolution: Won't Fix No longer applicable: driver.c deprecated. > Rewrite driver.c to use OS-neutral io.c and selector.c > -- > > Key: PROTON-534 > URL: https://issues.apache.org/jira/browse/PROTON-534 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.7 >Reporter: Cliff Jansen > > Driver.c no longer should require separate posix and windows implementations. > Thread safe issues need to be considered to come up with something more > elegant than the existing pn_driver_wait_1 (2 and 3) bits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-1071) EventInjector hangs on Windows
[ https://issues.apache.org/jira/browse/PROTON-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen reassigned PROTON-1071: Assignee: Cliff Jansen (was: Chuck Rolke) > EventInjector hangs on Windows > -- > > Key: PROTON-1071 > URL: https://issues.apache.org/jira/browse/PROTON-1071 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c, python-binding >Affects Versions: 0.11 > Environment: Windows >Reporter: Ken Giusti >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > I added a new reactor test that exercises the python-proton ApplicationEvent > and EventInjector classes: > proton_tests.reactor.ApplicationEventTest.test_application_events > See tests/python/proton_tests/reactor.py > This test passes on linux, but hangs when run on Windows. > Poking around a bit, I suspect the problem may be in the Windows selector > code. Description: > The EventInjector/ApplicationEvent classes provide a way to trigger events > from threads external to the reactor main loop. See > proton-c/bindings/python/proton/reactor.py. A pipe is used to wake up the > reactor when a new event is sent to the reactor (see reactor.py in the python > bindings). The EventInjector's trigger method puts the event on a queue and > writes to a pipe to wake up the reactor. The on_selectable_readable callback > in the EventInjector is called on the reactor thread to get the event off the > queue and clear the pipe. > On windows it appears as if the EventInjector selectable is made "readable" > even though nothing has been written to the pipe. This causes the os.read() > call in the on_selectable_readable() callback to hang. > Best I can tell the windows selector code doesn't work properly with a pipe. > The pn_selector_next() function is returning a read event on the pipe's read > descriptor even though the pipe is empty. But I'm not familiar with the > window's selector implementation, so this is a best guess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1069) Windows schannel ssl hang in shutdown sequence
Cliff Jansen created PROTON-1069: Summary: Windows schannel ssl hang in shutdown sequence Key: PROTON-1069 URL: https://issues.apache.org/jira/browse/PROTON-1069 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.11, 0.12.0 Environment: Windows TLS/SSL Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 If both parties of an SSL connection are using the proton schannel driver both can leave the corresponding transports waiting for additional bytes (that never come) to note the EOS status. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1069) Windows schannel ssl hang in shutdown sequence
[ https://issues.apache.org/jira/browse/PROTON-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1069. Resolution: Fixed > Windows schannel ssl hang in shutdown sequence > -- > > Key: PROTON-1069 > URL: https://issues.apache.org/jira/browse/PROTON-1069 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: 0.11, 0.12.0 > Environment: Windows TLS/SSL > Reporter: Cliff Jansen >Assignee: Cliff Jansen > Fix For: 0.12.0 > > > If both parties of an SSL connection are using the proton schannel driver > both can leave the corresponding transports waiting for additional bytes > (that never come) to note the EOS status. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-1066) Connection options for C++ binding
[ https://issues.apache.org/jira/browse/PROTON-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-1066. Resolution: Fixed Assignee: Cliff Jansen Fix Version/s: 0.12.0 > Connection options for C++ binding > -- > > Key: PROTON-1066 > URL: https://issues.apache.org/jira/browse/PROTON-1066 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding > Reporter: Cliff Jansen >Assignee: Cliff Jansen > Fix For: 0.12.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1066) Connection options for C++ binding
Cliff Jansen created PROTON-1066: Summary: Connection options for C++ binding Key: PROTON-1066 URL: https://issues.apache.org/jira/browse/PROTON-1066 Project: Qpid Proton Issue Type: Improvement Reporter: Cliff Jansen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1066) Connection options for C++ binding
[ https://issues.apache.org/jira/browse/PROTON-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15027839#comment-15027839 ] Cliff Jansen commented on PROTON-1066: -- Provide mechanism to configure variable numbers of connection options globally and per connection, including a reconnect mechanism. Note that many of the options can only be set before the OPEN frame, some can't be set until after the transport exists. See https://reviews.apache.org/r/39694/ and https://reviews.apache.org/r/39222/for background. The proposed implementation follows the revised mechanism by aconway. > Connection options for C++ binding > -- > > Key: PROTON-1066 > URL: https://issues.apache.org/jira/browse/PROTON-1066 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding > Reporter: Cliff Jansen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-1066) Connection options for C++ binding
[ https://issues.apache.org/jira/browse/PROTON-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen updated PROTON-1066: - Component/s: cpp-binding > Connection options for C++ binding > -- > > Key: PROTON-1066 > URL: https://issues.apache.org/jira/browse/PROTON-1066 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding > Reporter: Cliff Jansen > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1057) Windows SChannel SSL test failure
Cliff Jansen created PROTON-1057: Summary: Windows SChannel SSL test failure Key: PROTON-1057 URL: https://issues.apache.org/jira/browse/PROTON-1057 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.12.0 Environment: Windows only. Reporter: Cliff Jansen Assignee: Cliff Jansen This problem started since PROTON-1048 which did not touch the Proton-C SChannel related code - it just tweaked the test infrastructure to use alternate certificate formats for Windows. The proton_tests.ssl.SslTest.test_server_hostname_authentication test randomly crashes on Windows, presumably from some memory corruption somewhere. It often runs fine. Seems to happen more frequently on some systems than others. Not yet seen on a 64 bit build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-1054) Add acceptor context to incoming reactor connections.
[ https://issues.apache.org/jira/browse/PROTON-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen updated PROTON-1054: - Attachment: diff44 This patch provides a new call pn_connection_acceptor() that allows applications to find the acceptor that created the connection. Each connection holds a ref count to the acceptor. > Add acceptor context to incoming reactor connections. > - > > Key: PROTON-1054 > URL: https://issues.apache.org/jira/browse/PROTON-1054 > Project: Qpid Proton > Issue Type: Improvement > Components: proton-c >Affects Versions: 0.12.0 >Reporter: Cliff Jansen > Assignee: Cliff Jansen > Fix For: 0.12.0 > > Attachments: diff44 > > > The reactor's acceptor currently intercepts the readable callback and sets up > the incoming connection. The only generated event is PN_CONNECTION_INIT with > no context of the listener. > This prevents implementing separate connection options based on listening > interface/port. One solution is to have the acceptor add context to each > connection and provide a new API call to find the parent acceptor for a > connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1054) Add acceptor context to incoming reactor connections.
Cliff Jansen created PROTON-1054: Summary: Add acceptor context to incoming reactor connections. Key: PROTON-1054 URL: https://issues.apache.org/jira/browse/PROTON-1054 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.12.0 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 The reactor's acceptor currently intercepts the readable callback and sets up the incoming connection. The only generated event is PN_CONNECTION_INIT with no context of the listener. This prevents implementing separate connection options based on listening interface/port. One solution is to have the acceptor add context to each connection and provide a new API call to find the parent acceptor for a connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1053) SASL support in C++ reactor client
Cliff Jansen created PROTON-1053: Summary: SASL support in C++ reactor client Key: PROTON-1053 URL: https://issues.apache.org/jira/browse/PROTON-1053 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Provide SASL operation and configuration at the container level and overrides per outgoing connection and listener. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-865) C++ reactor client binding
[ https://issues.apache.org/jira/browse/PROTON-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-865. --- Resolution: Fixed Fix Version/s: 0.11 > C++ reactor client binding > -- > > Key: PROTON-865 > URL: https://issues.apache.org/jira/browse/PROTON-865 > Project: Qpid Proton > Issue Type: New Feature > Environment: C++ > Reporter: Cliff Jansen >Assignee: Cliff Jansen > Fix For: 0.11 > > > This JIRA tracks initial work on a C++ binding to the Proton reactor event > based model with an eye to providing an API very similar to the python > client. As stated on the Proton list, the broad goals are: > to make it easy to write simple programs > natural to build out more complicated ones > very similar API to the Python client > lean and performant > The initial introduction with accompanying HelloWorld can be found at > https://github.com/apache/qpid-proton/pull/18 > Ongoing work is proceeding in my github account in branch cpp-work > https://github.com/cliffjansen/qpid-proton/tree/cpp-work > commit 1453f57ca3f446450ef654505548f1947cb7a0f1 adds listener support, > exceptions and a logging interface. > The initial implementation will provide no thread safety guarantees, but the > bone structure should allow that to be added later with no API change. I > still hold out hope of eventually allowing multiple threads processing > separate connections concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1052) SSL support in C++ reactor client
Cliff Jansen created PROTON-1052: Summary: SSL support in C++ reactor client Key: PROTON-1052 URL: https://issues.apache.org/jira/browse/PROTON-1052 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Affects Versions: 0.12.0 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Provide SSL configuration at container level and overrides per outgoing connection and listener -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-1048) Proton-C ssl tests on Windows with SChannel
Cliff Jansen created PROTON-1048: Summary: Proton-C ssl tests on Windows with SChannel Key: PROTON-1048 URL: https://issues.apache.org/jira/browse/PROTON-1048 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.12.0 Environment: Windows with SChannel Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.12.0 Create file based certificates (as opposed to registry certificates requiring user intervention) for Windows SChannel in pkcs12 format from the existing OpenSSL certificates and keys. Update the test scripts to find the alternate certificates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Create new proton components: go-binding and cpp-binding
Sounds good. Thanks. On Thu, Oct 1, 2015 at 7:16 AM, Andrew Stitcherwrote: > On Wed, 2015-09-30 at 18:58 -0400, aconway wrote: >> I'd like to create these two components, who can edit the project? I >> can be owner for go-binding, cjansen should probably be owner for cpp >> -binding. > > Added with those suggested owners. > > Would the 2 of you like to be the default assignees (respectively) too? > > Andrew >
[jira] [Created] (PROTON-1002) Cancelled timer tasks poison task pool
Cliff Jansen created PROTON-1002: Summary: Cancelled timer tasks poison task pool Key: PROTON-1002 URL: https://issues.apache.org/jira/browse/PROTON-1002 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.10 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.10.1 A cancelled task remains in cancelled state when reused from the pool. The "new" task is scheduled pre-cancelled and has no effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-932) Document platform-specific usage of certificates and trust stores.
Cliff Jansen created PROTON-932: --- Summary: Document platform-specific usage of certificates and trust stores. Key: PROTON-932 URL: https://issues.apache.org/jira/browse/PROTON-932 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.9.1 Reporter: Cliff Jansen Assignee: Cliff Jansen Priority: Minor Fix For: 0.10 OpenSSL and SChannel have differing common usages for managing and specifying trust stores and certificates. Explain the differences and provide simple commands to reuse OpenSSL certificates and stores on Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-887) Windows SSL implementation needs working version of pn_ssl_get_remote_subject()
[ https://issues.apache.org/jira/browse/PROTON-887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-887. --- Resolution: Fixed Fix Version/s: 0.10 Windows SSL implementation needs working version of pn_ssl_get_remote_subject() --- Key: PROTON-887 URL: https://issues.apache.org/jira/browse/PROTON-887 Project: Qpid Proton Issue Type: Bug Reporter: Andrew Stitcher Assignee: Cliff Jansen Fix For: 0.10 Commit 894a463b introduced a new ssl API pn_ssl_get_remote_subject(). This has no Windows SChannel implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PROTON-887) Windows SSL implementation needs working version of pn_ssl_get_remote_subject()
[ https://issues.apache.org/jira/browse/PROTON-887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen reassigned PROTON-887: --- Assignee: Cliff Jansen Windows SSL implementation needs working version of pn_ssl_get_remote_subject() --- Key: PROTON-887 URL: https://issues.apache.org/jira/browse/PROTON-887 Project: Qpid Proton Issue Type: Bug Reporter: Andrew Stitcher Assignee: Cliff Jansen Fix For: 0.10 Commit 894a463b introduced a new ssl API pn_ssl_get_remote_subject(). This has no Windows SChannel implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-889) Thread safe pn_io_t
Cliff Jansen created PROTON-889: --- Summary: Thread safe pn_io_t Key: PROTON-889 URL: https://issues.apache.org/jira/browse/PROTON-889 Project: Qpid Proton Issue Type: Improvement Components: proton-c Reporter: Cliff Jansen It has been pointed out before that the pn_io API does not allow thread safe use of pn_error_t *pn_io_error(pn_io_t *io) bool pn_wouldblock(pn_io_t *io); For the moment, this JIRA serves as a reminder. I am hesitant to propose a specific solution without a solid outline of an actual implementation using it. We have the following use cases: - moderately scalable single threaded implementation using Proton's selector/selectable classes - custom selector/IO just using the Proton engine (i.e. Dispatch, Qpid C++ client/broker, no pn_io at all) - external loop with direct calls to pn_io methods The ultimate performance would come from the custom IO route optimized for the particular work load using engine primitives. But given that Proton is (at least mostly) thread safe on a per socket/connection basis (i.e. use by Dispatch), it would seem that additional parallelization could be achieved with various API changes (not just to pn_io methods). So, again with the disclaimer that an actual implementation should drive this, some options could be 6 new calls pn_io_(io, selectable, buf, len); (send/recv/write/read) pn_selectable_wouldblock(selectable); pn_selectable_error(selectable); 4 new calls pn_io_(io, socket, buf, len, bool *wouldblock, pn_error_t *err) Assuming the existing single threaded use cases should not be burdened with locking overhead, other related changes might include customized incref and decref functions per pn_class_t, external custom collectors (and surely more). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-887) Windows SSL implementation needs working version of pn_ssl_get_remote_subject()
[ https://issues.apache.org/jira/browse/PROTON-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546478#comment-14546478 ] Cliff Jansen commented on PROTON-887: - I took a quick look at this and believe that the existing trace statement in schannel.c CertNameToStr(...) mostly does what is needed here, although you may need to remove the XXX_FLAG modifier (or substitute another) to match openssl's punctuation. For the non-tracing case, CertNameToStr() would be called twice, first with a size of 0 to get a computed size, then with a malloc'ed buffer of that size. Windows SSL implementation needs working version of pn_ssl_get_remote_subject() --- Key: PROTON-887 URL: https://issues.apache.org/jira/browse/PROTON-887 Project: Qpid Proton Issue Type: Bug Reporter: Andrew Stitcher Commit 894a463b introduced a new ssl API pn_ssl_get_remote_subject(). This has no Windows SChannel implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-697) SChannel SSL/TLS support for Proton-c on Windows
[ https://issues.apache.org/jira/browse/PROTON-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-697. --- SChannel SSL/TLS support for Proton-c on Windows Key: PROTON-697 URL: https://issues.apache.org/jira/browse/PROTON-697 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 This JIRA tracks the progress of completing SChannel functionality in Proton beyond the start in PROTON-581. The target is Proton 0.9. This includes support for incoming connections client side certificates Windows registry and file based certificate stores Control over certificate name checking -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-865) C++ reactor client binding
Cliff Jansen created PROTON-865: --- Summary: C++ reactor client binding Key: PROTON-865 URL: https://issues.apache.org/jira/browse/PROTON-865 Project: Qpid Proton Issue Type: New Feature Environment: C++ Reporter: Cliff Jansen Assignee: Cliff Jansen This JIRA tracks initial work on a C++ binding to the Proton reactor event based model with an eye to providing an API very similar to the python client. As stated on the Proton list, the broad goals are: to make it easy to write simple programs natural to build out more complicated ones very similar API to the Python client lean and performant The initial introduction with accompanying HelloWorld can be found at https://github.com/apache/qpid-proton/pull/18 Ongoing work is proceeding in my github account in branch cpp-work https://github.com/cliffjansen/qpid-proton/tree/cpp-work commit 1453f57ca3f446450ef654505548f1947cb7a0f1 adds listener support, exceptions and a logging interface. The initial implementation will provide no thread safety guarantees, but the bone structure should allow that to be added later with no API change. I still hold out hope of eventually allowing multiple threads processing separate connections concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-847) Qpid Proton C Assertion trasport-output_pending = size failed
[ https://issues.apache.org/jira/browse/PROTON-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492625#comment-14492625 ] Cliff Jansen commented on PROTON-847: - Proton is not thread safe. Please try again single threaded and see if your errors persist. Qpid Proton C Assertion trasport-output_pending = size failed --- Key: PROTON-847 URL: https://issues.apache.org/jira/browse/PROTON-847 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.7 Environment: Debian linux, Proton C v 0.7 Reporter: Rajkumar Priority: Critical I am using Qpid proton C library v.7.0 for sending AMQPS messages to Azure cloud service bus. I am getting the following error. /home/debian/qpid/qpid-proton-0.7/proton-c/src/transport/transport.c:2110: pn_transport_pop: Assertion `transport-output_pending = size' failed. Can anyone tell why this error is coming? This error comes very frequently and not able to predict and no idea how to stop this ** I have created one messenger object and two thread is my program share that messenger object. If there is a time out occurs more than three times I free the messenger object and recreate new messenger object. ** First thread send message to cloud and second thread receives messages from the cloud. ** Is it a right way of doing or I need to create two messenger objects for sending and receiving messages? ** Or Do I need to create new messenger object every time while sending and receiving messages ? Please propose the design to avoid such kind of errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
C++ reactor based binding
I have started work on a C++ binding to the Proton reactor event based model with an eye to providing an API very similar to the python client. I have put together a skimpy skeleton of an implementation with enough corners cut to get a simple HelloWorld example to run, mostly to get myself acquainted with most of the moving parts. This can be seen at https://github.com/apache/qpid-proton/pull/18 I plan to get some JIRAs going on the topic and do my initial work in some branch on my github account. I welcome comments from anyone with ideas on the shape this should take and welcome any assistance anyone might wish to provide in it's implementation. The broad goals are: to make it easy to write simple programs natural to build out more complicated ones very similar API to the Python client lean and performant The choice of C++11 language features to include or leave out is still a question mark in my mind. I am currently trying to hedge my bets on this by introducing such features conservatively (and with plenty of notice). The downside is that C++11 language features are not available in older development systems, i.e. Visual Studio 2008 and gcc for RHEL 5. On the other hand, a non-trivial subset is available in VS2010 and most newer compilers, and this is after all a new C++ API on an Advanced message queuing protocol. Another big question in my mind is the desired level of multi threading support. Some event driven systems view threading as the devil's trickery and eschew thread safety guarantees and multi threading support entirely. Perhaps thread safety without multi threading support could be a valuable feature of the API, allowing the programmer to ignore the issue (at the expense of some locking overhead). Dispatch currently manages some significant multithreading use of Proton, but it does not use the reactor. Similar grained multi threading in the proposed C++ client looks quite possible with minimal locking overhead (you can set a pn_collector_t per connection and pn_event_t pools are already managed per collector... nice). Perhaps there are other thorny issues I haven't noticed plus there are remaining known outages in the pn_io_t interface that need fixing. If the API is implemented similar to previous qpid C++ client work, using Handles and PIMPL, presumably the implementation of multithreading could be deferred without affecting the API (unless the lack of Boost derails things somehow). Cliff
Re: Items for the 0.9 Release notes/announcement
I would like to highlight a functional change for Windows users now that the SChannel based SSL processing can work the same as for OpenSSL: Important Windows SSL/TLS functionality change: The Proton SSL/TLS module for 0.8 using the native Microsoft SChannel libraries was not configurable and successful handshakes resulted solely from having the appropriate CA certificate in the official Windows Trusted Root CA store. It is now possible to specify alternate trusted root CA databases and to turn off certificate checking altogether, using the same Proton API conventions as for OpenSSL on Posix systems. In particular, Proton applications in Windows will not check server certificates at all in 0.9 unless the capability is explicitly enabled using the pn_ssl_domain_set_trusted_ca_db() function. To use the system Trusted Root CA store: pn_ssl_domain_set_trusted_ca_db(d, sys:root) Or to use a file based PKCS#12 certificate store: pn_ssl_domain_set_trusted_ca_db(d, mycerts.p12) On Wed, Mar 25, 2015 at 8:25 AM, Ken Giusti kgiu...@redhat.com wrote: How about: New Reactor API I think that's a pretty big deal, no? ;) - Original Message - From: Rafael Schloming r...@alum.mit.edu To: proton@qpid.apache.org Sent: Tuesday, March 24, 2015 3:16:52 PM Subject: Items for the 0.9 Release notes/announcement I'm trying to put together a relatively complete set of changes and release notes for 0.9. If there is any particular feature be it new or some existing behaviour change that is worthy of being mentioned in the release notes or release announcement, please follow up with a suitable blurb on this thread. --Rafael -- -K
[jira] [Resolved] (PROTON-742) Proton Windows SChannel buffer copy error
[ https://issues.apache.org/jira/browse/PROTON-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-742. - Resolution: Fixed Proton Windows SChannel buffer copy error - Key: PROTON-742 URL: https://issues.apache.org/jira/browse/PROTON-742 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows with SChannel Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 Attachments: PROTON-742-0.patch The double buffering used to preserve partially decrypted frames spanning multiple TLS records sometimes copied too much or too little data, resulting in spurious framing errors or hangs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-818) Reactor C soak tests
[ https://issues.apache.org/jira/browse/PROTON-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-818. - Resolution: Fixed Fix Version/s: 0.9 Initial versions created. Should be able to provide additional tests, like star topology, ssl etc. Reactor C soak tests Key: PROTON-818 URL: https://issues.apache.org/jira/browse/PROTON-818 Project: Qpid Proton Issue Type: Test Components: proton-c Affects Versions: 0.9 Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 Provide analogous programs to msgr-send and msgr-recv that can extend the soak tests to reactor sample programs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-658) driver.c implemented with selectors and selectables
[ https://issues.apache.org/jira/browse/PROTON-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-658. --- Resolution: Won't Fix Fix Version/s: (was: 0.9) This was 95% completed, with unresolved MP bugs if tested with dispatch. The driver model has since been removed from proton completely, so this is now moot. driver.c implemented with selectors and selectables --- Key: PROTON-658 URL: https://issues.apache.org/jira/browse/PROTON-658 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.7 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Provide a reworked driver.c using the selector and selectable primitives. Required on the Windows platform to integrate with ctest suite and the new completion port IO model. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-697) SChannel SSL/TLS support for Proton-c on Windows
[ https://issues.apache.org/jira/browse/PROTON-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-697. - Resolution: Fixed Fix Version/s: 0.9 completed except: Session resume. Server side should just work as is (but not tested), client side needs to provide and track session IDs to reuse the saved credential. Arbitrary system/registry or pkcs12 file stores may now be used to provided trusted CAs if revocation list processing is not required. This will work for most self-signed certificate scenarios. If full revocation list processing is required to authenticate, Proton will fail the certification unless the root CA is also located in systems official Trusted Root CA store. BEHAVIOR CHANGE: Specifying alternate trusted CA certificates, or none at all, was not possible in Proton 0.8, but is now and follows the original conventions as for OpenSSL. In particular, in Proton 0.8, authentication followed defaults as would exist in a browser. Now, in Proton 0.9, if no root CA database is specified, the server certificate will not be validated in any way. To continue validating the server credential as in Proton 0.8, an application must now specify the trusted certificate database (just as it would for OpenSSL) using the pn_ssl_domain_set_trusted_ca_db() method, either using sys:root or an other trusted CA database. SChannel SSL/TLS support for Proton-c on Windows Key: PROTON-697 URL: https://issues.apache.org/jira/browse/PROTON-697 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 This JIRA tracks the progress of completing SChannel functionality in Proton beyond the start in PROTON-581. The target is Proton 0.9. This includes support for incoming connections client side certificates Windows registry and file based certificate stores Control over certificate name checking -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-840) Fix Windows components to use transport logging
Cliff Jansen created PROTON-840: --- Summary: Fix Windows components to use transport logging Key: PROTON-840 URL: https://issues.apache.org/jira/browse/PROTON-840 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8, 0.9 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Posix implemented new centralized logging. Windows needs to catch up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-578) proton-c: windows/io.c prints Unknown error for all winsock errors
[ https://issues.apache.org/jira/browse/PROTON-578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-578. - Resolution: Fixed Fix Version/s: 0.9 Mostly fixed as suggested. PROTON-840 tracks further improvements and parity with the Posix implementation. proton-c: windows/io.c prints Unknown error for all winsock errors Key: PROTON-578 URL: https://issues.apache.org/jira/browse/PROTON-578 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.7 Reporter: Dominic Evans Assignee: Cliff Jansen Fix For: 0.9 Attachments: 10_fix_winsock_error_code_printing.patch The code in windows/io.c is attempting to call strerror on Winsock WSA error codes returned by WSAGetLastError(). However, these codes don't map to Unix errno codes, so will always output an Unknown error msg. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-563) Generated Proton.sln doesn't compile
[ https://issues.apache.org/jira/browse/PROTON-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-563. - Resolution: Fixed Fix Version/s: 0.9 I believe this has been fixed. At least for my testing. Please reopen if that is not the case. Generated Proton.sln doesn't compile Key: PROTON-563 URL: https://issues.apache.org/jira/browse/PROTON-563 Project: Qpid Proton Issue Type: Bug Environment: Visual Studio 2012 Reporter: Paolo Patierno Assignee: Cliff Jansen Fix For: 0.9 Following README file for compiling under Windows and using Visual Studio 2012, the ALL_BUILD project doesn't compile. The error is IntelliSense: identifier strncasecmp is undefined c:\qpid-proton-0.6\proton-c\src\ssl\openssl.c 208 13 qpid-proton -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-837) C reactor event samples, including external loop (libuv)
Cliff Jansen created PROTON-837: --- Summary: C reactor event samples, including external loop (libuv) Key: PROTON-837 URL: https://issues.apache.org/jira/browse/PROTON-837 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.9 Reporter: Cliff Jansen Assignee: Cliff Jansen Provide sample programs to use the reactor event model. Also provide an optional external loop driver to demonstrate the use of external event/io loops with Proton. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: I think that's a blocker...
The Proton IOCP implementation provides a reactor capability over the proactor API, which mostly works as you expect. I would remind folks of the following Windows-isms compared to POSIX: Windows was late to the tcp/ip party (championing LAN Manager) and 3rd parties provided tcp/ip stacks for a long time. A socket was an alien entity that was supposed to use different APIs from other file handle things. This persists to this day and is why you can't mix a Windows pipe and socket in a single Windows selection mechanism. Hence the documented restrictions on pn_pipe(). Well you can mix them in IOCP, because that came along after Microsoft was in the tcp/ip game. But that has other consequences: a socket doesn't have fluid usage in IOCP since it can be bound only once. Once selected within Proton, it cannot be used in another IOCP context elsewhere. Proton can support external loops provide its own loop allow applications to completely manage IO themselves (i.e. qpid cpp) Proton appears to be close to supporting a Dispatch-level of concurrency. Is the above enough, or do we have to find extensions to these that are useful to developers but don't stretch the differences between Windows and POSIX beyond possibility? On Thu, Feb 26, 2015 at 12:22 PM, Andrew Stitcher astitc...@redhat.com wrote: On Thu, 2015-02-26 at 15:09 -0500, Rafael Schloming wrote: ... It sounds like one way or another we need at least some design changes. I don't think it's workable to have overlapping/close but distinct semantics for the API on different platforms (e.g. you can move sockets on one platform but not on another). I'm starting to think we either need one platform to precisely and fully emulate the semantics of the other platform, or they both need to implement some interface that is slightly higher level and can better accommodate the differences. I may have misremembered, but I think the essential platform difference here is that Windows IOCP really tries to implement a Proactor type of pattern rather than the reactor type of pattern that we are using in Proton. [Proactor is where fundamentally the system calls back a processing function on a thread that you give it] In the qpid implementation I had to effectively make the exposed interface a proactor type interface in all platforms, to bridge the gap. I'm not sure this is workable in a context where the user can supply their own event loop. Andrew
Re: I think that's a blocker...
A pn_io_t is heavyweight in Windows, because it has an opposite usage pattern and moves a lot of kernel stuff into user space compared to POSIX. The quoted documentation was my attempt to capture the Dispatch usage pattern, which I assumed would be typical of an application trying to spread proton engine use between threads: basically single access to pn_selector_select() via a condition variable, and no more than one thread working on a given selectable (using proton engine encoding/decoding etc., not just io). In the end, we could just add a zillion locks into the Windows code and make it look like it is as thread safe as the POSIX counterpart (which has implicit safety when it does in the kernel what Windows is doing in user space), but that would defeat using IO completion ports at all. The documentation was my attempt of balancing performance with sophisticated proton usage on multiple platforms. Note that there is only one pn_selector_t allowed per pn_io_t (a very strong Windows completion port requirement, and sockets are bound to a single completion port for life). On Wed, Feb 25, 2015 at 8:52 AM, Rafael Schloming r...@alum.mit.edu wrote: On Wed, Feb 25, 2015 at 10:49 AM, Ted Ross tr...@redhat.com wrote: Would it be safe to assume that any operations on driver-io are not thread safe? Dispatch is a multi-threaded application. It looks to me as though io-error is a resource shared across the threads in an unsafe way. Interesting... so this is what the docs say: /** * A ::pn_io_t manages IO for a group of pn_socket_t handles. A * pn_io_t object may have zero or one pn_selector_t selectors * associated with it (see ::pn_io_selector()). If one is associated, * all the pn_socket_t handles managed by a pn_io_t must use that * pn_selector_t instance. * * The pn_io_t interface is single-threaded. All methods are intended * to be used by one thread at a time, except that multiple threads * may use: * * ::pn_write() * ::pn_send() * ::pn_recv() * ::pn_close() * ::pn_selector_select() * * provided at most one thread is calling ::pn_selector_select() and * the other threads are operating on separate pn_socket_t handles. */ I think this has been somewhat modified by the constraints from the windows implementation, and I'm not sure I understand completely what the constraints are there, or entirely what is being described above, but on the posix front, the pn_io_t is little more than just a holder for an error slot, and you should have one of these per thread. It shouldn't be a problem to use send/recv/etc from multiple threads though so long as you pass in the pn_io_t from the current thread. --Rafael
Re: [jira] [Commented] (PROTON-824) Windows fails testIdleTimeout with assert p.conn.remote_condition
Looking at the driver and frame traces, it appears that the Windows IOCP code is dropping an error condition on close or is just taking longer to report the closed condition than its Linux counterpart. I am taking a closer look and will report back. Cliff On Mon, Feb 23, 2015 at 7:29 AM, Chuck Rolke (JIRA) j...@apache.org wrote: [ https://issues.apache.org/jira/browse/PROTON-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1495#comment-1495 ] Chuck Rolke commented on PROTON-824: Here's what's happening. All of the Program's reactor callbacks are firing and each is passing its assertions. But when the main program continues from Reactor(p).run() then p.conn does not have a remote_condition any more. Any thoughts? {noformat} 1: start 1: INFO: on_reactor_init 1: INFO: on_connection_remote_open 1: INFO: on_timer_task 1: INFO: on_connection_remote_close 1: INFO:self.conn : proton.Connection 0x2704610 ~ 0x27be2f0 1: INFO:self.conn.remote_condition : Condition('amqp:resource-limit-exceeded', 'local-idle-timeout expired') 1: INFO: p.conn : proton.Connection 0x2704610 ~ 0x27be2f0 1: INFO: p.conn.remote_condition : None 1: proton_tests.engine.AAAServerTest.testIdleTimeout ... proton_tests.engine.AAAServerTest.testIdleTimeout ... fail 1: Error during test: Traceback (most recent call last): 1: File D:/Users/crolke/git/rh-qpid-proton/tests/python/proton-test, line 355, in run 1: phase() 1: File D:\Users\crolke\git\rh-qpid-proton\tests\python\proton_tests\engine.py, line 1894, in testIdleTimeout 1: assert p.conn.remote_condition 1: AssertionError {noformat} Windows fails testIdleTimeout with assert p.conn.remote_condition - Key: PROTON-824 URL: https://issues.apache.org/jira/browse/PROTON-824 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.9 Environment: Windows Server 2008 or 2012 Visual studio 2010, x86 Reporter: Chuck Rolke {noformat} 1: proton_tests.engine.ServerTest.testIdleTimeout . fail 1: Error during test: Traceback (most recent call last): 1: File D:/Users/crolke/git/rh-qpid-proton/tests/python/proton-test, line 355, in run 1: phase() 1: File D:\Users\crolke\git\rh-qpid-proton\tests\python\proton_tests\engine.py, line 1919 (or so), in testIdleTimeout 1: assert p.conn.remote_condition 1: AssertionError {noformat} Playing with Program explicit timeout (trying 10 instead of 3) gets the test to pass sometimes. It passes sometimes with 3 as well but normally fails. In debugging this it looks like there as no synchronization between what a test will show through print statements and what the proton library shows through PN_TRACE_FRM statements. Are there any hints to lining these up? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-818) Reactor C soak tests
Cliff Jansen created PROTON-818: --- Summary: Reactor C soak tests Key: PROTON-818 URL: https://issues.apache.org/jira/browse/PROTON-818 Project: Qpid Proton Issue Type: Test Components: proton-c Affects Versions: 0.9 Reporter: Cliff Jansen Assignee: Cliff Jansen Provide analogous programs to msgr-send and msgr-recv that can extend the soak tests to reactor sample programs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-800) [Windows C] Reactor test times out
[ https://issues.apache.org/jira/browse/PROTON-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-800. - Resolution: Fixed Fix Version/s: 0.9 Assignee: Cliff Jansen As per the other comments, plus creating a special iocp descriptor that just holds a deadline for socket-less selectables. [Windows C] Reactor test times out -- Key: PROTON-800 URL: https://issues.apache.org/jira/browse/PROTON-800 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Server 2012 R2, Visual Studio 2010 x64 Reporter: Chuck Rolke Assignee: Cliff Jansen Fix For: 0.9 ctest -VV times out on the reactor test. Instrumenting and debugging shows the failure in test_reactor_acceptor Running Debug instead of RelWithDebInfo crashes with an AccVio {noformat} qpid-protond.dll!pni_iocpdesc_map_get(iocp_t * iocp, unsigned __int64 s) Line 773 + 0xa bytes C++ qpid-protond.dll!pn_close(pn_io_t * io, unsigned __int64 socket) Line 350 + 0x16 bytes C++ qpid-protond.dll!pni_acceptor_finalize(pn_selectable_t * sel) Line 56 C++ qpid-protond.dll!pn_selectable_finalize(void * obj) Line 81C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_list_finalize(void * object) Line 205 C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_finalize(pn_reactor_t * reactor) Line 77 C++ qpid-protond.dll!pn_reactor_finalize_cast(void * object) Line 113 + 0x28 bytes C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_free(pn_reactor_t * reactor) Line 132 C++ c-reactor-tests.exe!test_reactor_acceptor() Line 171 C++ c-reactor-tests.exe!main(int argc, char * * argv) Line 446 C++ c-reactor-tests.exe!__tmainCRTStartup() Line 555 + 0x19 bytes C c-reactor-tests.exe!mainCRTStartup() Line 371 C kernel32.dll!7ff90cc913d2() [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll] ntdll.dll!7ff90e1003c4() {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-800) [Windows C] Reactor test times out
[ https://issues.apache.org/jira/browse/PROTON-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283275#comment-14283275 ] Cliff Jansen commented on PROTON-800: - The new reactor code uses selectors/selectables differently, namely: Assign a socket-less selectable into the selector and set the selectable's socket at some future date. This means that completion port enlistment can happen at any time, which the current selector code is not ready for. I have a tentative fix but first have to get the Windows code compiling again against Friday's (separate) API changes. All coming soon... [Windows C] Reactor test times out -- Key: PROTON-800 URL: https://issues.apache.org/jira/browse/PROTON-800 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Server 2012 R2, Visual Studio 2010 x64 Reporter: Chuck Rolke ctest -VV times out on the reactor test. Instrumenting and debugging shows the failure in test_reactor_acceptor Running Debug instead of RelWithDebInfo crashes with an AccVio {noformat} qpid-protond.dll!pni_iocpdesc_map_get(iocp_t * iocp, unsigned __int64 s) Line 773 + 0xa bytes C++ qpid-protond.dll!pn_close(pn_io_t * io, unsigned __int64 socket) Line 350 + 0x16 bytes C++ qpid-protond.dll!pni_acceptor_finalize(pn_selectable_t * sel) Line 56 C++ qpid-protond.dll!pn_selectable_finalize(void * obj) Line 81C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_list_finalize(void * object) Line 205 C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_finalize(pn_reactor_t * reactor) Line 77 C++ qpid-protond.dll!pn_reactor_finalize_cast(void * object) Line 113 + 0x28 bytes C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_free(pn_reactor_t * reactor) Line 132 C++ c-reactor-tests.exe!test_reactor_acceptor() Line 171 C++ c-reactor-tests.exe!main(int argc, char * * argv) Line 446 C++ c-reactor-tests.exe!__tmainCRTStartup() Line 555 + 0x19 bytes C c-reactor-tests.exe!mainCRTStartup() Line 371 C kernel32.dll!7ff90cc913d2() [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll] ntdll.dll!7ff90e1003c4() {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-800) [Windows C] Reactor test times out
[ https://issues.apache.org/jira/browse/PROTON-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278397#comment-14278397 ] Cliff Jansen commented on PROTON-800: - The memory fault in the stack trace occurs from pn_io_t use (pn_close) after the io has been released. It can be made to go away simply by moving the first two lines: pn_decref(reactor-selector); pn_decref(reactor-io); pn_decref(reactor-collector); pn_decref(reactor-handler); pn_decref(reactor-children); to the end of pn_reactor_finalize(...) so that the pn_io_t is the last to be (possibly) freed. This matches the use in messenger where created objects are released in reverse order. So question #1 is does the overall API imply that pn_io_t objects, which are created first because so much else depends on them, should be freed/decref_ed last? The answer to this has deep implications on the Windows io code which has to do many Linuxy kernel things in user space on cleanup. The Windows IO code assumes the answer is yes. If that is wrong, it needs to do extra ref counting to know when the IO really is being shut down. On a completely separate note, fixing the above issue does not fix this JIRA, it just moves the problem a small distance. It still hangs waiting for some IO event in a pn_selector_select call. Sadly it is not related to 363ea5031d26bc19cd12e113e8b365d398120dea, but I will fix that anyway separate from this JIRA. Thanks very much for the heads up in any case. Investigation continues... [Windows C] Reactor test times out -- Key: PROTON-800 URL: https://issues.apache.org/jira/browse/PROTON-800 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Server 2012 R2, Visual Studio 2010 x64 Reporter: Chuck Rolke ctest -VV times out on the reactor test. Instrumenting and debugging shows the failure in test_reactor_acceptor Running Debug instead of RelWithDebInfo crashes with an AccVio {noformat} qpid-protond.dll!pni_iocpdesc_map_get(iocp_t * iocp, unsigned __int64 s) Line 773 + 0xa bytes C++ qpid-protond.dll!pn_close(pn_io_t * io, unsigned __int64 socket) Line 350 + 0x16 bytes C++ qpid-protond.dll!pni_acceptor_finalize(pn_selectable_t * sel) Line 56 C++ qpid-protond.dll!pn_selectable_finalize(void * obj) Line 81C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_list_finalize(void * object) Line 205 C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_finalize(pn_reactor_t * reactor) Line 77 C++ qpid-protond.dll!pn_reactor_finalize_cast(void * object) Line 113 + 0x28 bytes C++ qpid-protond.dll!pn_class_decref(const pn_class_t * clazz, void * object) Line 100 C++ qpid-protond.dll!pn_decref(void * object) Line 253 C++ qpid-protond.dll!pn_reactor_free(pn_reactor_t * reactor) Line 132 C++ c-reactor-tests.exe!test_reactor_acceptor() Line 171 C++ c-reactor-tests.exe!main(int argc, char * * argv) Line 446 C++ c-reactor-tests.exe!__tmainCRTStartup() Line 555 + 0x19 bytes C c-reactor-tests.exe!mainCRTStartup() Line 371 C kernel32.dll!7ff90cc913d2() [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll] ntdll.dll!7ff90e1003c4() {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-798) Hang in Windows SSL negotiation
Cliff Jansen created PROTON-798: --- Summary: Hang in Windows SSL negotiation Key: PROTON-798 URL: https://issues.apache.org/jira/browse/PROTON-798 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.9 Environment: Windows Schannel Reporter: Cliff Jansen Assignee: Cliff Jansen If a client closes a connection part way through protocol negotiation the propagating close can be missed causing a hang (and busy loop). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-798) Hang in Windows SSL negotiation
[ https://issues.apache.org/jira/browse/PROTON-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-798. - Resolution: Fixed Fix Version/s: 0.9 Hang in Windows SSL negotiation --- Key: PROTON-798 URL: https://issues.apache.org/jira/browse/PROTON-798 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.9 Environment: Windows Schannel Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 If a client closes a connection part way through protocol negotiation the propagating close can be missed causing a hang (and busy loop). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-794) [Windows] Visual Studio 2008 compile error
[ https://issues.apache.org/jira/browse/PROTON-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271107#comment-14271107 ] Cliff Jansen commented on PROTON-794: - This also fails on Linux if using BUILD_WITH_CXX in cmake. The compiler can be mollified with a double cast, first to a size_t or ssize_t, then to the enum, but check with Andrew if that passes the C++ smell test. Alternatively the enum could be stored and retrieved as a pointer offset. [Windows] Visual Studio 2008 compile error -- Key: PROTON-794 URL: https://issues.apache.org/jira/browse/PROTON-794 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.9 Environment: Windows Server 2008 32 bit Visual Studio 2008 Reporter: Chuck Rolke {noformat} 13-- Rebuild All started: Project: c-reactor-tests, Configuration: RelWithDebInfo Win32 -- 13Deleting intermediate and output files for project 'c-reactor-tests', configuration 'RelWithDebInfo|Win32' 13Building Custom Rule C:/Hudson/jobs/rh_qpid_proton-trunk/workspace/rh-qpid-proton/proton-c/src/tests/CMakeLists.txt 13CMake does not need to re-run because C:\Hudson\jobs\rh_qpid_proton-trunk\workspace\rh-qpid-proton\build\proton-c\src\tests\CMakeFiles\generate.stamp is up-to-date. 13Compiling... 13reactor.c 13..\..\..\..\proton-c\src\tests\reactor.c(80) : error C2440: 'type cast' : cannot convert from 'void *' to 'pn_event_type_t' 13There is no context in which this conversion is possible {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Request for Proton code review: SSL certificate stores (Windows and more)
Following Andrew's lead, I've posted a pull request for review feedback on an extended usage pattern to the Proton API to specify alternate certificate containers, for Windows in this case, but it should also apply to alternate SSL/TLS implementations such as NSS or Java keystores. https://github.com/apache/qpid-proton/pull/2 I expect a wider audience than just Windows users may be interested in the changes in ssl.h and I am especially interested on feedback there. Some may prefer an outright API change or addition. I, of course, also welcome any additional eyes on the Windows-specific code too. Cliff
[jira] [Created] (PROTON-742) Proton Windows SChannel buffer copy error
Cliff Jansen created PROTON-742: --- Summary: Proton Windows SChannel buffer copy error Key: PROTON-742 URL: https://issues.apache.org/jira/browse/PROTON-742 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows with SChannel Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.9 The double buffering used to preserve partially decrypted frames spanning multiple TLS records sometimes copied too much or too little data, resulting in spurious framing errors or hangs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: VOTE: Release Proton 0.8 RC5 as 0.8 final
[ x ] Yes, release Proton 0.8 RC5 as 0.8 final Tested on Windows: 32/64 bit, XP/Win8.1 On Monday, October 27, 2014, Rafael Schloming r...@alum.mit.edu wrote: Hi Everyone, Sorry for the delay, there seemed to be some kind of Nexus outage today, so I was unable to generate the java binaries until just now. I've posted RC5 in the usual places. The only difference from RC4 is a one line delta that replaces the assertion failure when we receive out-of-sequence ids with a connection shutdown error. Please have a look and register your vote. Source code can be found here: http://people.apache.org/~rhs/qpid-proton-0.8rc5/ Java binaries are here: https://repository.apache.org/content/repositories/orgapacheqpid-1021 [ ] Yes, release Proton 0.8 RC5 as 0.8 final [ ] No, because ... --Rafael
[jira] [Resolved] (PROTON-719) Disable SSL v3 for Windows SChannel
[ https://issues.apache.org/jira/browse/PROTON-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-719. - Resolution: Fixed Disable SSL v3 for Windows SChannel --- Key: PROTON-719 URL: https://issues.apache.org/jira/browse/PROTON-719 Project: Qpid Proton Issue Type: Bug Components: proton-j Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 Windows advisory: https://technet.microsoft.com/en-us/library/security/3009008.aspx See especially part 3: Disable SSL 3.0 in Windows, but note that a similar registry setting exists for CLIENT. Schannel works differently from openssl: SChannel can override default protocols (in registry), but cannot override enabled protocols (also in registry). A user or global administrator can force AMQP 1.0 SChannel connections to succeed during protocol negotiations over SSLv3 despite Proton's best efforts. Possible solutions on Windows: 1. always fail after the fact if an SSLv3 connection has actually been established 2. succeed for SSLV3 if registry allows it, but log a warning 3. succeed for SSLV3 only if registry allows it and env variable PROTON_SSLV3_UNSAFE=override_by_user Since SSLv3 is not considered secure, and there are no known legacy AMQP 1.0 that are unable to provide TLS1.0 or above, #1 seems to provide the greatest security without known inconvenience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-719) Disable SSL v3 for Windows SChannel
Cliff Jansen created PROTON-719: --- Summary: Disable SSL v3 for Windows SChannel Key: PROTON-719 URL: https://issues.apache.org/jira/browse/PROTON-719 Project: Qpid Proton Issue Type: Bug Components: proton-j Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 Windows advisory: https://technet.microsoft.com/en-us/library/security/3009008.aspx See especially part 3: Disable SSL 3.0 in Windows, but note that a similar registry setting exists for CLIENT. Schannel works differently from openssl: SChannel can override default protocols (in registry), but cannot override enabled protocols (also in registry). A user or global administrator can force AMQP 1.0 SChannel connections to succeed during protocol negotiations over SSLv3 despite Proton's best efforts. Possible solutions on Windows: 1. always fail after the fact if an SSLv3 connection has actually been established 2. succeed for SSLV3 if registry allows it, but log a warning 3. succeed for SSLV3 only if registry allows it and env variable PROTON_SSLV3_UNSAFE=override_by_user Since SSLv3 is not considered secure, and there are no known legacy AMQP 1.0 that are unable to provide TLS1.0 or above, #1 seems to provide the greatest security without known inconvenience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Proton 0.8 RC1
Windows now works with ctest for 0.8. Yay! The rc1 version of qpid-proton-0.8.tar.gz works for me (build and ctest) on: XP32bit compile python27 cmake28 VS2010 server VM WIN7 32bit compile python25 cmake28 vs2008 laptop WIN8 64bit compile python27 cmake30 vs2013 server VM Cliff On Mon, Sep 29, 2014 at 2:28 PM, Rafael Schloming r...@alum.mit.edu wrote: Hi Everyone, I've created 0.8 RC1 and posted it in the usual locations. The source can be found here: http://people.apache.org/~rhs/qpid-proton-0.8rc1/ Java binaries are available here: https://repository.apache.org/content/repositories/orgapacheqpid-1015/ Please check it out and follow up with email and/or JIRAs as appropriate. Thanks, ---Rafael
[jira] [Created] (PROTON-702) Windows proton-c selector can forget timer events
Cliff Jansen created PROTON-702: --- Summary: Windows proton-c selector can forget timer events Key: PROTON-702 URL: https://issues.apache.org/jira/browse/PROTON-702 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 A pn_selector_update to the identical deadline value will turn off the timer notification. This causes several ctest failures. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-702) Windows proton-c selector can forget timer events
[ https://issues.apache.org/jira/browse/PROTON-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-702. - Resolution: Fixed Windows proton-c selector can forget timer events - Key: PROTON-702 URL: https://issues.apache.org/jira/browse/PROTON-702 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 A pn_selector_update to the identical deadline value will turn off the timer notification. This causes several ctest failures. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-697) SChannel SSL/TLS support for Proton-c on Windows
[ https://issues.apache.org/jira/browse/PROTON-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150692#comment-14150692 ] Cliff Jansen commented on PROTON-697: - Reminder from https://reviews.apache.org/r/26013: Kenneth Giusti (Sept. 25, 2014, 1:20 p.m.) One caveat: it appears as if there's a lot of code duplicated between the openssl.c and schannel.c. Just for maintenance's sake, would it be possible to put the common code into a common file and better abstract the boundary between the two implementations? Probably a lot of work - something to postpone until after 0.8 IMHO SChannel SSL/TLS support for Proton-c on Windows Key: PROTON-697 URL: https://issues.apache.org/jira/browse/PROTON-697 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen This JIRA tracks the progress of completing SChannel functionality in Proton beyond the start in PROTON-581. The target is Proton 0.9. This includes support for incoming connections client side certificates Windows registry and file based certificate stores Control over certificate name checking -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-668) Document Proton-c IO restrictions for 0.8 release
[ https://issues.apache.org/jira/browse/PROTON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-668. - Resolution: Fixed from https://reviews.apache.org/r/26075: This describes my best guess of balancing convenience for the developer versus performance. Ultimately, with the exception of stickiness between completion ports and sockets, which prevents moving a socket between or out of a pn_io_t, any Posix behavior can probably be mimicked in Windows, but at a performance penalty. With these restrictions, third party event loops are supported. Proton event loops can also accommodate external non-sockets if that is useful (if dispensed with, that makes Windows code easier). Multi-threaded IO as currently used in Dispatch is also supported. It is worth noting Bozo's points that the interface doesn't always make sense for multithreaded ops: pn_io_error() and pn_io_wouldblock(). Finally, it should be noted that the Windows code has some way to go to satisfy the above, but understanding the intended supported capabilities needs to come first. Document Proton-c IO restrictions for 0.8 release - Key: PROTON-668 URL: https://issues.apache.org/jira/browse/PROTON-668 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Reporter: Cliff Jansen Assignee: Cliff Jansen Priority: Blocker Fix For: 0.8 Proton is designed to provide an efficient IO layer that functions without imposing a threading model on the application. Applications may (1) roll their own IO and just use the Proton engine, (2) use all Proton primitives, (3) use some Proton primitives augmented by an external event loop. Case (1) is unrelated to this JIRA. The others may be restated: Scenario 2: Proton event loop: a proton selector manages socket events for all sockets placed in the selector, all associated sockets use pn_io_xxx() calls. Sockets outside the selector are unmanaged and passed through to the OS socket function unchanged. Scenario 3: Third party event loop (no proton selector involved), all sockets are treated as for unmanaged in scenario 2. Scenario 4, 5...: Others to support? The problem: The Proton Posix pattern for efficient IO is: tell me when your (OS) buffer is ready for io transfer (in or out) Whereas the normal Windows pattern is somewhat reversed (IO completion ports): tell me when you are done transferring data (to or from) my (user space) buffer The current Windows IOCP implementation (PROTON-640) tries to make the latter look like the former with some constraints. There should be documentation specifying reasonable limits on Proton usage that may be falsely implied by the API but do not translate efficiently to Windows. Assuming that future Windows implementations may adopt more aggressive performance strategies (especially on the read side), I would propose something along the lines of: a socket may only ever be used with a single pn_io_t in its lifetime exception: a socket from pn_accept() is not yet associated with any pn_io_t and its first use can be with any pn_io_t (or never with a pn_io_t at all) send/recv/close may not be intermixed with similar non-Proton OS calls (otherwise: out of order or lost data) a socket can move once from an external loop to a proton loop, but never the other way pn_pipe() values can only be used with pn_read and pn_write and pn_selector_select, they cannot participate in an external event loop. Furthermore, there is no thread safety except: threads may do concurrent pn_io_xxx() calls as long as no two are simultaneous on the same socket (where xxx is send/recv/read/write) pn_selector_select() is thread safe against pn_read/pn_write/pn_send/pn_recv, but the outcome of the select is indeterminate. pn_selector_select() must be interrupted and restarted at any time when other simultaneous IO may affect the outcome. calls on different pn_io_t objects do not interact and are thread safe. If it is desirable for a socket to be used in an external loop after being used in a Proton loop, we would need some sort of blocking calls along the lines of: pn_io_flush() pn_io_drain() which would be no-ops on Posix but would unwind outstanding completions on Windows. Early criticism of any of the above assumptions would be greatly appreciated. I will try to reword the above, or its evolution into the existing documentation for 0.8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-701) Windows ctest fixes for proton-c
Cliff Jansen created PROTON-701: --- Summary: Windows ctest fixes for proton-c Key: PROTON-701 URL: https://issues.apache.org/jira/browse/PROTON-701 Project: Qpid Proton Issue Type: Bug Components: proton-c, python-binding Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Priority: Minor Fix For: 0.8 Provide fixes related to ctest interaction with Windows proton-c for 0.8 as time allows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PROTON-697) SChannel SSL/TLS support for Proton-c on Windows
Cliff Jansen created PROTON-697: --- Summary: SChannel SSL/TLS support for Proton-c on Windows Key: PROTON-697 URL: https://issues.apache.org/jira/browse/PROTON-697 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen This JIRA tracks the progress of completing SChannel functionality in Proton beyond the start in PROTON-581. The target is Proton 0.9. This includes support for incoming connections client side certificates Windows registry and file based certificate stores Control over certificate name checking -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-581) SSL/TLS support for Proton-c on Windows
[ https://issues.apache.org/jira/browse/PROTON-581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147236#comment-14147236 ] Cliff Jansen commented on PROTON-581: - See https://reviews.apache.org/r/26013 for the current best patch. This implementation will not complete in time for Proton 0.8, so it is being reduced in scope to just handle client connections without client side-certificates. See PROTON-697 for progress on the full version targeted for Proton 0.9. SSL/TLS support for Proton-c on Windows --- Key: PROTON-581 URL: https://issues.apache.org/jira/browse/PROTON-581 Project: Qpid Proton Issue Type: New Feature Components: proton-c Affects Versions: 0.6, 0.7 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Attachments: schannel_0.patch Provide a Microsoft SChannel based SSL/TLS layer for Proton-c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PROTON-690) Fix Windows components for new Proton object/class refcounting mechanisms
[ https://issues.apache.org/jira/browse/PROTON-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen resolved PROTON-690. - Resolution: Fixed Fixed: r1625894 Fix Windows components for new Proton object/class refcounting mechanisms - Key: PROTON-690 URL: https://issues.apache.org/jira/browse/PROTON-690 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 Windows IOCP functionality relies on the very handy refcounting capabilities of the maps and lists. The usage conventions have change considerably and broken the Windows IO at many levels. Update for these semantic changes and r1625220's switch from pn_new() to calloc(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (PROTON-684) Restrict IOCP enlistment to pn_selector_t usage scenarios in Windows
[ https://issues.apache.org/jira/browse/PROTON-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen closed PROTON-684. --- Resolution: Fixed Restrict IOCP enlistment to pn_selector_t usage scenarios in Windows Key: PROTON-684 URL: https://issues.apache.org/jira/browse/PROTON-684 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.8 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Priority: Blocker Fix For: 0.8 Attachments: PN-684-0.patch Prevent IOCP enlistemnt interfering with external event loops. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: 0.8 release prep
I would like someone with better Python/SWIG expertise to take a look at PROTON-687, at least to confirm if the python swig wrapper for pn_delivery_set_context (and other similar borrowed references) are the culprits, or the victims of a bug elsewhere. I would like to update PROTON-581 (Schannel ssl for windows) for changes in the SSL internal api and get it checked in. I would also like to take a closer look at PROTON-660 (openssl on Windows) or have someone else review it for rc1. On Thu, Sep 18, 2014 at 8:32 AM, Clebert Suconic csuco...@redhat.com wrote: I have a commit that I would like to send towards a contrib component: https://github.com/clebertsuconic/qpid-proton/commit/ab423a90f6b1e6d06c84615c5724497a3f9e8533 The current code is always encoding it as Bytes, and I need the ProtonJ as an output on the case of the JMS component. On Sep 18, 2014, at 11:15 AM, Miguel da Rocha Correia Lima mcorreial...@landix.com.br wrote: Hi Rafael, The PROTON-661 was useful for us. We are working with this patch without any error. We supose that it can be useful for others. Miguel da Rocha Correia Lima miguel at landix dot com dot br Em 18/09/2014 11:55, Rafael Schloming escreveu: Hi Everyone, I'd like to put out a 0.8 RC soon. I will be going through JIRA shortly in order to triage any remaining issues. If there are any particular issues that you feel strongly should be included in 0.8, please follow up on this thread and bring them to my attention. --Rafael
[jira] [Created] (PROTON-687) Memory corruption in proton-test
Cliff Jansen created PROTON-687: --- Summary: Memory corruption in proton-test Key: PROTON-687 URL: https://issues.apache.org/jira/browse/PROTON-687 Project: Qpid Proton Issue Type: Bug Components: proton-c, python-binding Affects Versions: 0.8 Reporter: Cliff Jansen proton-test will fail with memory violations. Deallocated memory is accessed, usually with benign results but less often on Windows. The attached crude patch allows the bug to be discovered on Linux as well. It is not clear to me if the problem is with the swig wrapper, or whether this just exposes a reference counting bug in proton-c. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-687) Memory corruption in proton-test
[ https://issues.apache.org/jira/browse/PROTON-687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cliff Jansen updated PROTON-687: Attachment: segv.patch Memory corruption in proton-test Key: PROTON-687 URL: https://issues.apache.org/jira/browse/PROTON-687 Project: Qpid Proton Issue Type: Bug Components: proton-c, python-binding Affects Versions: 0.8 Reporter: Cliff Jansen Attachments: segv.patch proton-test will fail with memory violations. Deallocated memory is accessed, usually with benign results but less often on Windows. The attached crude patch allows the bug to be discovered on Linux as well. It is not clear to me if the problem is with the swig wrapper, or whether this just exposes a reference counting bug in proton-c. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-668) Document Proton-c IO restrictions for 0.8 release
[ https://issues.apache.org/jira/browse/PROTON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131213#comment-14131213 ] Cliff Jansen commented on PROTON-668: - Additional notes based on comments to the Proton mailing list: a socket may only ever be used with a single pn_io_t in its lifetime exception: a socket from pn_accept() is not yet associated with any pn_io_t and its first use can be with any pn_io_t (or never with a pn_io_t at all) I'm not sure if it's implied, but pn_connect() and pn_listen() also need to support 'third party event loop'. Specifically, pn_connect() has to remain non-blocking (we get to know about the connect error later in the external event loop) Yes third party loop support is implied for pn_listen and pn_connect. The exception comment is about moving a socket between two pn_io_t instances (or determining first use). The third party loop is merely not allowed to pass any given socket to more that one pn_io_t. threads may do concurrent pn_io_xx) calls as long as no two are simultaneous on the same socket (where xxx is send/recv/read/write) This will break pn_io_error() and pn_io_wouldblock() as they are defined now. There is certainly no error or blocking state kept per socket, or per thread. I am describing the existing multi-threaded use in driver.c (heavily used by Dispatch) and assuming others may wish to do the same on Windows. pn_io_error and pn_wouldblock are conspicuously absent in driver.c (old and new). I have a feeling you don't really want/need to expose the pn_pipe(), but add a pn_selector_interrupt() and a mechanism of querying that for the caller of pn_selector_select() especially as you want to implement it completely differently on windows. Yes, the pipe is an awkward difference between Windows and Posix thinking, where the latter sees pipes and sockets as close kin but Windows sees them as very distant relatives. Ditching the pn_pipe() call altogether in favour of a pn_selector_interrupt() is cleanest and most performant from the Windows IOCP perspective. However, as long as a pn_pipe is only accessed via pn_read and pn_write the scope for optimization reduces any performance penalty to a minimum. So I am OK keeping things as they are but would not resist change. Document Proton-c IO restrictions for 0.8 release - Key: PROTON-668 URL: https://issues.apache.org/jira/browse/PROTON-668 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.8 Reporter: Cliff Jansen Assignee: Cliff Jansen Priority: Blocker Fix For: 0.8 Proton is designed to provide an efficient IO layer that functions without imposing a threading model on the application. Applications may (1) roll their own IO and just use the Proton engine, (2) use all Proton primitives, (3) use some Proton primitives augmented by an external event loop. Case (1) is unrelated to this JIRA. The others may be restated: Scenario 2: Proton event loop: a proton selector manages socket events for all sockets placed in the selector, all associated sockets use pn_io_xxx() calls. Sockets outside the selector are unmanaged and passed through to the OS socket function unchanged. Scenario 3: Third party event loop (no proton selector involved), all sockets are treated as for unmanaged in scenario 2. Scenario 4, 5...: Others to support? The problem: The Proton Posix pattern for efficient IO is: tell me when your (OS) buffer is ready for io transfer (in or out) Whereas the normal Windows pattern is somewhat reversed (IO completion ports): tell me when you are done transferring data (to or from) my (user space) buffer The current Windows IOCP implementation (PROTON-640) tries to make the latter look like the former with some constraints. There should be documentation specifying reasonable limits on Proton usage that may be falsely implied by the API but do not translate efficiently to Windows. Assuming that future Windows implementations may adopt more aggressive performance strategies (especially on the read side), I would propose something along the lines of: a socket may only ever be used with a single pn_io_t in its lifetime exception: a socket from pn_accept() is not yet associated with any pn_io_t and its first use can be with any pn_io_t (or never with a pn_io_t at all) send/recv/close may not be intermixed with similar non-Proton OS calls (otherwise: out of order or lost data) a socket can move once from an external loop to a proton loop, but never the other way pn_pipe() values can only be used with pn_read and pn_write and pn_selector_select, they cannot participate in an external event loop
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Hi Bozzo, Please take a look at PROTON-668 and confirm none of the assumptions are counter to your usage, especially whether you fit scenario 3 and do not have problems with the proposed restriction to pn_pipe. If this is OK for you, I think I am close to a fix that will work for you. Cliff On Wed, Sep 10, 2014 at 4:51 AM, Bozo Dragojevic bo...@digiverse.si wrote: Hi Cliff, I agree that the current extra API call is kludgy if not downright ugly. I noticed today you already added a new API call pn_io_selector(), which has separate windows and posix implementations. For short term 'fix', I'd propose to hide my hack inthere -- make calling pn_io_selector() turn on the iocp on windows. Bozzo This would not impact the API and would restore the On 9. 09. 14 07:02, Cliff Jansen wrote: Ho Bozo, Thank you for comments and the suggested patch. I would prefer a solution that did not have a special Windows-only-sometimes call pn_io_no_iocp(). It seems to me anyway that there is another class of sockets that are pulled into an IOCP context too early, so that a separate solution is required that should fix your problem too. Basically, a more lazy enlistment strategy should leave you outside IOCP and fix the other issue too. Based on your problem and how Dispatch strives for multithreaded performance, I think I have a better handle on what Proton should be providing for small to medium-large scalability. I am going to try to define what is a sensible intersection of Windows and Posix capabilities to be supported by the proton io/selector/selectable classes in a separate documentation JIRA and try to get a fix for you ASAP, probably in yet another JIRA. Cliff
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Ho Bozo, Thank you for comments and the suggested patch. I would prefer a solution that did not have a special Windows-only-sometimes call pn_io_no_iocp(). It seems to me anyway that there is another class of sockets that are pulled into an IOCP context too early, so that a separate solution is required that should fix your problem too. Basically, a more lazy enlistment strategy should leave you outside IOCP and fix the other issue too. Based on your problem and how Dispatch strives for multithreaded performance, I think I have a better handle on what Proton should be providing for small to medium-large scalability. I am going to try to define what is a sensible intersection of Windows and Posix capabilities to be supported by the proton io/selector/selectable classes in a separate documentation JIRA and try to get a fix for you ASAP, probably in yet another JIRA. Cliff On Fri, Sep 5, 2014 at 3:18 AM, Bozo Dragojevic bo...@digiverse.si wrote: On 5. 09. 14 11:53, Bozo Dragojevic wrote: The patch is a bit rough - not sure I really like the name pn_io_no_iocp() - missing non-windows stub for pn_io_no_iocp() - it most likely handles the selector access wrong (see XXX comment) I missed this bit (blush) diff --git a/proton-c/src/windows/io.c b/proton-c/src/windows/io.c index f9c85b4..b5fc7b2 100644 --- a/proton-c/src/windows/io.c +++ b/proton-c/src/windows/io.c @@ -263,6 +263,14 @@ pn_socket_t pn_connect(pn_io_t *io, const char *hostarg, const char *port) if (io-iocp != NULL) { sock = pni_iocp_begin_connect(io-iocp, sock, addr, io-error); } else { +if (connect(sock, addr-ai_addr, addr-ai_addrlen) != 0) { + if (WSAGetLastError() != WSAEWOULDBLOCK) { +pni_win32_error(io-error, connect, WSAGetLastError()); +freeaddrinfo(addr); +closesocket(sock); +return INVALID_SOCKET; + } +} freeaddrinfo(addr); } return sock; Bozzo
Re: [jira] [Commented] (PROTON-640) IO completion port Windows IO for Proton
Hi, I am not sure what you are trying to do with xdispatch. If you are using and external loop and managing your own IO, you can bypass the low level Proton IO primitives and access the proton engine directly. This is how Qpid uses Proton. If you were using the driver.c mechanism, please take a look at PROTON-658. This and the main messenger loop show how the selectors/selectables can be used within Proton. pn_selector_select() is pretty central to the completion port model on windows. You may find that if you use Windows accept and connect instead of pn_accept and pn_connect you can keep your descriptors completely outside the completion port model and use things as you did. If not, perhaps we can find another way to isolate your sockets from the completion port implementation. The Windows completion port implementation is designed to provide a highly scalable throughput in a constrained (mostly) single thread model, but remain lightweight for small clients. Central to that is the integration with selectors and selectables. But it can't be everything to everybody. On Thu, Sep 4, 2014 at 8:33 AM, Bozo Dragojevic bo...@digiverse.si wrote: Hi! Just a headsup. This commit breaks pn_accept() for me, it fails all the time. I'm not using selectors but drive the socket with libdispatch (from xdispatch) with a custom driver. If I revert this commit the trunk is useable. I'll dig deeper what is going on, I was focusing on getting SSL to work on windows. Cheers, Bozzo On 4. 09. 14 04:12, ASF subversion and git services (JIRA) wrote: [ https://issues.apache.org/jira/browse/PROTON-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120849#comment-14120849 ] ASF subversion and git services commented on PROTON-640: Commit 1622394 from cliffjan...@apache.org in branch 'proton/trunk' [ https://svn.apache.org/r1622394 ] PROTON-640 Windows IO completion port implementation for pn_io and pn_selectable classes IO completion port Windows IO for Proton Key: PROTON-640 URL: https://issues.apache.org/jira/browse/PROTON-640 Project: Qpid Proton Issue Type: Improvement Components: proton-c Affects Versions: 0.7 Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.8 Provide a native IO completion port layer similar to the C++ QPID version for Proton. -- This message was sent by Atlassian JIRA (v6.3.4#6332)