Re: inconsistent proton library names?
I've not looked at the branch lately (only just back from vacation), but I would very much hope that there would be nothing preventing having both the JNI and native-Java libraries in the classpath, and allowing for explicit creation of the desired implementation of Connection / Messenger / whatever (which I'd probably suggest be done via a factory rather explicit construction, but that's just personal taste). I would hope the Service Loader would only affect the implementation created by *default* from a factory -- Rob On 4 January 2013 22:54, Phil Harvey p...@philharveyonline.com wrote: The in-progress code on the jni branch does not currently allow this, although is no technical barrier to doing it. We just haven't yet decided on the nicest api for allowing the application to choose the implementation it wants. The ability to mix implementations within a jvm will certainly be nice when writing interoperability tests. Phil On Jan 4, 2013 9:16 PM, Rafael Schloming r...@alum.mit.edu wrote: Does that mean you won't be able to use both the C and Java implementation simultaneously within a single JVM? --Rafael On Fri, Jan 4, 2013 at 4:02 PM, Phil Harvey p...@philharveyonline.com wrote: Ditto for Java. From the developer's point of view, they'll simply be using the Java interfaces in proton-api such as Connection [1]. Our current intention is that the choice of whether to use the pure Java implementations or the proton-c-via-Swig-via-JNI one will be made using a factory instantiated by a java.util.ServiceLoader. The decision will therefore depend on your runtime classpath. Client code will not have a build time dependency on the Swig/JNI layer. [1] http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-j/proton-api/src/main/java/org/apache/qpid/proton/engine/Connection.java On 4 January 2013 20:40, Darryl L. Pierce dpie...@redhat.com wrote: On Fri, Jan 04, 2013 at 03:32:44PM -0500, Rafael Schloming wrote: Given what little I know of loading JNI stuff, that seems to make sense for Java. FWIW, the python and ruby bindings don't ever actually expose the name of the C extension library since in both cases we have the so-called buttercream frosting layer that wraps the raw C extension module. I would hope we'd have something similar for perl and Java so that these names shouldn't ever be visible to users. Per does. It uses qpid::proton namespace for the Message and Messenger classes. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/
Re: inconsistent proton library names?
Hi Rob, I believe we're thinking along the same lines. The ServiceLoader approach does indeed only affect which implementation you get by default. We will also allow the client to explicitly choose their implementation if they wish, and there will be no problem with both implmentations being used in the same proccess (this will be handy for writing interoperability tests). Phil On 7 January 2013 08:37, Rob Godfrey rob.j.godf...@gmail.com wrote: I've not looked at the branch lately (only just back from vacation), but I would very much hope that there would be nothing preventing having both the JNI and native-Java libraries in the classpath, and allowing for explicit creation of the desired implementation of Connection / Messenger / whatever (which I'd probably suggest be done via a factory rather explicit construction, but that's just personal taste). I would hope the Service Loader would only affect the implementation created by *default* from a factory -- Rob On 4 January 2013 22:54, Phil Harvey p...@philharveyonline.com wrote: The in-progress code on the jni branch does not currently allow this, although is no technical barrier to doing it. We just haven't yet decided on the nicest api for allowing the application to choose the implementation it wants. The ability to mix implementations within a jvm will certainly be nice when writing interoperability tests. Phil On Jan 4, 2013 9:16 PM, Rafael Schloming r...@alum.mit.edu wrote: Does that mean you won't be able to use both the C and Java implementation simultaneously within a single JVM? --Rafael On Fri, Jan 4, 2013 at 4:02 PM, Phil Harvey p...@philharveyonline.com wrote: Ditto for Java. From the developer's point of view, they'll simply be using the Java interfaces in proton-api such as Connection [1]. Our current intention is that the choice of whether to use the pure Java implementations or the proton-c-via-Swig-via-JNI one will be made using a factory instantiated by a java.util.ServiceLoader. The decision will therefore depend on your runtime classpath. Client code will not have a build time dependency on the Swig/JNI layer. [1] http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-j/proton-api/src/main/java/org/apache/qpid/proton/engine/Connection.java On 4 January 2013 20:40, Darryl L. Pierce dpie...@redhat.com wrote: On Fri, Jan 04, 2013 at 03:32:44PM -0500, Rafael Schloming wrote: Given what little I know of loading JNI stuff, that seems to make sense for Java. FWIW, the python and ruby bindings don't ever actually expose the name of the C extension library since in both cases we have the so-called buttercream frosting layer that wraps the raw C extension module. I would hope we'd have something similar for perl and Java so that these names shouldn't ever be visible to users. Per does. It uses qpid::proton namespace for the Message and Messenger classes. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/
Re: Updating versions (was: 0.3 RC1)
On Fri, Jan 04, 2013 at 03:56:54PM -0500, Rafael Schloming wrote: Not difficult, no. We just have to pass it through a similar filter as would be done when using autoconf. For development purposes the version doesn't matter, just when we're creating those artifacts. So we could change release.sh to take the version number that's passed in and use it, though that's duplicating what's in the root CMakeLists.txt already. It just needs to be something we don't have to pass into the build system for packaging, for example. I think if we were to consolidate it into one place we'd pick a file in svn and have both release.sh and CMakeLists.txt pull it out of there and nix the release.sh parameter. That way the svn number encoded in the release will always point to a source tree that builds with the correct version. +1 That sounds good to me. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpDAFGk8AqyB.pgp Description: PGP signature
Re: inconsistent proton library names?
On Fri, Jan 04, 2013 at 03:49:57PM -0500, Rafael Schloming wrote: Compared to the other bindings, it seems inconsistent for the former to state its Perl-ness in its name, and for the latter to state its Swig-ness. Thoughts? Negative on Perl. The raw Perl extension is named cproton_perl because otherwise Cmake fails. That's because we already have a target named cproton and changing Perl to that will cause a name collision. And, anyway, the output of Cmake isn't used to distribution the Perl language bindings. Those are distributed as the perl.i file, Makefile.PL, various license and doc files, and the qpid::proton namespace classes. When that's installed, a proper library is generated for Perl to use. Hmm, this doesn't seem to quite be the case. The cmake build does in fact install the swig generated files, i.e. libcproton_perl.so, and cproton_perl.pm, however it does not install the qpid::proton namespace classes. Sorry, I wasn't referring to what the Cmake system installs there. I was referring to what a packaging system would consume when creating the binaries that are installed on a system. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpgmE9Gpd4j0.pgp Description: PGP signature
[jira] [Resolved] (PROTON-171) after connection close, proton-c SSL decryption fails when left-over bytes passed to input()
[ https://issues.apache.org/jira/browse/PROTON-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti resolved PROTON-171. --- Resolution: Fixed Fix Version/s: 0.3 http://svn.apache.org/viewvc?view=revisionrevision=1430010 after connection close, proton-c SSL decryption fails when left-over bytes passed to input() Key: PROTON-171 URL: https://issues.apache.org/jira/browse/PROTON-171 Project: Qpid Proton Issue Type: Bug Components: proton-c Reporter: Philip Harvey Assignee: Ken Giusti Fix For: 0.3 We've been working on the Java SSL implementation and are seeing a test fail against proton-c but that works against proton-j. We believe this is due to a bug in proton-c. One of the scenarios we wanted to cover in our testing was the case where the Transport input method leaves left-overs, e.g. when you call server.input() with 100 bytes of input, but it only accepts 20, as indicated by its return value. For example, we expect this to happen if the preceding client.output() call is told to write to a buffer sized such that its output contains a trailing *fragment* of an SSL packet, which input() won't be able to decipher. We therefore modified the pump method in proton/tests/proton_tests/ssl.py to handle this case. In its loop, it now captures the bytes left over after calling input(), and prepends them to the input() invocation in the next iteration. The buffer size is now a parameter so individual tests can exercise the packet fragmenting behaviour described above. We made the following change: --- diff --git a/tests/proton_tests/ssl.py b/tests/proton_tests/ssl.py index 8567b1b..237c3da 100644 --- a/tests/proton_tests/ssl.py +++ b/tests/proton_tests/ssl.py @@ -43,13 +43,32 @@ class SslTest(common.Test): self.t_client = None self.t_server = None -def _pump(self): +def _pump(self, buffer_size=1024): + +Make the transport send up to buffer_size bytes (this will be the AMQP +header and open frame) returning a buffer containing the bytes +sent. Transport is stateful so this will return 0 when it has +no more frames to send. +TODO this function is duplicated in sasl.py. Should be moved to a common place. + +out_client_leftover_by_server = +out_server_leftover_by_client = +i=0 while True: -out_client = self.t_client.output(1024) -out_server = self.t_server.output(1024) -if out_client: self.t_server.input(out_client) -if out_server: self.t_client.input(out_server) + +out_client = out_client_leftover_by_server + self.t_client.output(buffer_size) +out_server = out_server_leftover_by_client + self.t_server.output(buffer_size) + +if out_client: +number_server_consumed = self.t_server.input(out_client) +out_client_leftover_by_server = out_client[number_server_consumed:] # if it consumed everything then this is empty + +if out_server: +number_client_consumed = self.t_client.input(out_server) +out_server_leftover_by_client = out_server[number_client_consumed:] # if it consumed everything then this is empty + if not out_client and not out_server: break +i=i+1 def _testpath(self, file): Set the full path to the certificate,keyfile, etc. for the test. --- Several ssl tests now fail when run against proton-c, all with the same error. This surprised us because we hadn't started playing with the buffer size yet - we were still using the default of 1024. For example, test_server_authentication gives this output: --- proton_tests.ssl.SslTest.test_server_authentication .[0xa2ca208:0] ERROR[-2] SSL Failure: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac fail Error during test: Traceback (most recent call last): File ./tests/proton-test, line 331, in run phase() File /home/phil/dev/proton/tests/proton_tests/ssl.py, line 166, in test_server_authentication self._pump() File /home/phil/dev/proton/tests/proton_tests/ssl.py, line 63, in _pump number_server_consumed = self.t_server.input(out_client) File /home/phil/dev/proton/proton-c/bindings/python/proton.py, line 2141, in input return self._check(n) File /home/phil/dev/proton/proton-c/bindings/python/proton.py, line 2115, in _check raise exc([%s]: %s % (err,
