Re: Changing the Proton build system to accommodate jni bindings
On 21 January 2013 18:05, Rafael Schloming r...@alum.mit.edu wrote: On Mon, Jan 21, 2013 at 9:33 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: Ummm... it's a dependency... you're familiar with those, yeah? The same way that the Qpid JMS clients depend on a JMS API jar, for which the source is readily available from another source. The JNI binding would build if the dependency was installed. The same way I believe the SSL code in the core of proton-c builds if the dependency for it is installed. That's not really a proper analogy. Again the JMS interfaces are defined outside of qpid. We don't release them, and we depend only on a well defined version of them, we don't share a release cycle with them. If the JMS API was something that we developed/defined right alongside the impl and was part of the same release process, we would certainly not be allowed to release without the source. This releasing without the source is a complete red herring and you know it. The source is released in whichever scheme we settle upon. If you want an example of dependencies within the qpid project, how did the AMQP 1.0 work on the C++ broker get released for 0.20? Did all the proton source get released with the C++ Broker / client? In the future are you expecting every part of the Qpid project which depends on proton to include its full source? If yes then how is the source tree going to work - is everything to be a subdirectory of proton-c? I agree that having the source for the version of the Java API included in the source release bundle is advantageous. But if the collective decision is that we have taken a religious position that the source tarballs can only be svn exports of subdirectories of our source tree, then my preference would be to use separated dependencies over duplication in the repository. Personally I would think that having a more flexible policy on constructing the release source tarballs would make a lot more sense. -- Rob --Rafael
Re: Changing the Proton build system to accommodate jni bindings
On Tue, Jan 22, 2013 at 4:22 AM, Rob Godfrey rob.j.godf...@gmail.comwrote: On 21 January 2013 18:05, Rafael Schloming r...@alum.mit.edu wrote: On Mon, Jan 21, 2013 at 9:33 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: Ummm... it's a dependency... you're familiar with those, yeah? The same way that the Qpid JMS clients depend on a JMS API jar, for which the source is readily available from another source. The JNI binding would build if the dependency was installed. The same way I believe the SSL code in the core of proton-c builds if the dependency for it is installed. That's not really a proper analogy. Again the JMS interfaces are defined outside of qpid. We don't release them, and we depend only on a well defined version of them, we don't share a release cycle with them. If the JMS API was something that we developed/defined right alongside the impl and was part of the same release process, we would certainly not be allowed to release without the source. This releasing without the source is a complete red herring and you know it. The source is released in whichever scheme we settle upon. If you want an example of dependencies within the qpid project, how did the AMQP 1.0 work on the C++ broker get released for 0.20? Did all the proton source get released with the C++ Broker / client? In the future are you expecting every part of the Qpid project which depends on proton to include its full source? If yes then how is the source tree going to work - is everything to be a subdirectory of proton-c? Again that's not really the same. If the Java API where on a separate (staggered) release cycle and the dependency was on a specific version, then that would be the same, but for what we're discussing, it really isn't. Proton and the cpp broker live under different trunks and branch/release separately, as far as I know this is not what you're proposing for the Java API, it is to live under the same trunk and branch/release together. I agree that having the source for the version of the Java API included in the source release bundle is advantageous. But if the collective decision is that we have taken a religious position that the source tarballs can only be svn exports of subdirectories of our source tree, then my preference would be to use separated dependencies over duplication in the repository. Personally I would think that having a more flexible policy on constructing the release source tarballs would make a lot more sense. You can call it religious if you like, but I don't think there is anything invalid about wanting to keep a simple mapping between release artifacts and proton developer environment. In the past we have had quite direct experience of exactly this factor contributing to very poor out of the box experience for users. Correct me if I'm wrong, but I believe you yourself have actually advocated (or at least agreed with) this position in the past. That said, I don't think I'm asking for us to be entirely inflexible in that regard. There really are two opposing concerns here, one being the user experience for our release artifacts, and the other being the convenience of the development process for proton developers. All I'm asking is that we recognize that there is a real tradeoff and be willing to explore options that might preserve user experience albeit at a hopefully minor cost to developer convenience. For any other aspect of software engineering this would be a no-brainer, you start from the user requirements/user experience and work your way backwards to the simplest solution that achieves this, however this proposal and the related requirements JIRA make zero mention of any *user* requirements merely developer requirements. This may be ok for Java where all your users will get stuff via jars and the source tarball is mostly a formality, but for C the situation is different. --Rafael
Re: Changing the Proton build system to accommodate jni bindings
It sounds like we're still a little way away from reaching a consensus. As a step towards this, I would like to clarify the relative priority of the various requirements that have come up. I've therefore created a page on the wiki that lists them, with a child page briefly describing the various proposals. https://cwiki.apache.org/confluence/display/qpid/Proton+build+system+requirements What are people's views on the relative priority of these requirements? Are there any I've missed? I think answering these questions is a prerequisite for agreeing the technical solution. Phil On 22 January 2013 13:34, Rob Godfrey rob.j.godf...@gmail.com wrote: On 22 January 2013 13:47, Rafael Schloming r...@alum.mit.edu wrote: On Tue, Jan 22, 2013 at 4:22 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: On 21 January 2013 18:05, Rafael Schloming r...@alum.mit.edu wrote: On Mon, Jan 21, 2013 at 9:33 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: Ummm... it's a dependency... you're familiar with those, yeah? The same way that the Qpid JMS clients depend on a JMS API jar, for which the source is readily available from another source. The JNI binding would build if the dependency was installed. The same way I believe the SSL code in the core of proton-c builds if the dependency for it is installed. That's not really a proper analogy. Again the JMS interfaces are defined outside of qpid. We don't release them, and we depend only on a well defined version of them, we don't share a release cycle with them. If the JMS API was something that we developed/defined right alongside the impl and was part of the same release process, we would certainly not be allowed to release without the source. This releasing without the source is a complete red herring and you know it. The source is released in whichever scheme we settle upon. If you want an example of dependencies within the qpid project, how did the AMQP 1.0 work on the C++ broker get released for 0.20? Did all the proton source get released with the C++ Broker / client? In the future are you expecting every part of the Qpid project which depends on proton to include its full source? If yes then how is the source tree going to work - is everything to be a subdirectory of proton-c? Again that's not really the same. If the Java API where on a separate (staggered) release cycle and the dependency was on a specific version, then that would be the same, but for what we're discussing, it really isn't. Proton and the cpp broker live under different trunks and branch/release separately, as far as I know this is not what you're proposing for the Java API, it is to live under the same trunk and branch/release together. The point was that the source code doesn't need to be in the same tarball let alone the same subdirectory in source control. If one considers that the Java API is a dependency then whether it is released concurrently or not with the JNI binding is moot. I've already said that it is preferable to have the source within the same tarball for the source release, but if needs be then I can live with the strict dependency view of things. I agree that having the source for the version of the Java API included in the source release bundle is advantageous. But if the collective decision is that we have taken a religious position that the source tarballs can only be svn exports of subdirectories of our source tree, then my preference would be to use separated dependencies over duplication in the repository. Personally I would think that having a more flexible policy on constructing the release source tarballs would make a lot more sense. You can call it religious if you like, but I don't think there is anything invalid about wanting to keep a simple mapping between release artifacts and proton developer environment. In the past we have had quite direct experience of exactly this factor contributing to very poor out of the box experience for users. Correct me if I'm wrong, but I believe you yourself have actually advocated (or at least agreed with) this position in the past. That said, I don't think I'm asking for us to be entirely inflexible in that regard. There really are two opposing concerns here, one being the user experience for our release artifacts, and the other being the convenience of the development process for proton developers. I actually think there are three perspectives here. The user experience of our release artefacts, the committer experience of working on the checkedout codebase, and the release manager view of preparing the release aretfacts from the source control. All I'm asking is that we recognize that there is a real tradeoff and be willing to explore options that might preserve
Re: Changing the Proton build system to accommodate jni bindings
Thanks for posting this, I think it's a very useful step. I'd suggest adding another Stakeholder -- someone testing a release artifact. Rob makes a good point that the release manager is a distinct view, but I think the desire to minimize deltas between the svn tree and the release artifacts is most directly motivated by my experience *testing* release artifacts. I remember going through qpid releases in the old days and having the very unpleasant experience of trying to remember from 8 or 10 months ago how exactly stuff worked in the release artifact as compared to the build tree. I very much like the fact that with a simple export I can be highly confident that my experience of stuff working in my checkout translates well to the release artifacts and testing them is a very familiar, quick, and easy process. Strictly speaking I think the requirement from a release management perspective is purely that we can produce releases at the rate we need, so it has to be quick and easy and robust to different environments, but I wouldn't say the export thing is a requirement of the release manager per/se. As many have pointed out we already use a script for this and it can remap things quite easily. I have more thoughts on the release process, especially as it is somewhat expanded now to produce java binaries and will need to expand more to include windows stuff, however I need to run an errand at the moment. I'll post and/or comment on the page later though. --Rafael I very much like the fact that our current release artifacts are trivial On Tue, Jan 22, 2013 at 11:43 AM, Phil Harvey p...@philharveyonline.comwrote: It sounds like we're still a little way away from reaching a consensus. As a step towards this, I would like to clarify the relative priority of the various requirements that have come up. I've therefore created a page on the wiki that lists them, with a child page briefly describing the various proposals. https://cwiki.apache.org/confluence/display/qpid/Proton+build+system+requirements What are people's views on the relative priority of these requirements? Are there any I've missed? I think answering these questions is a prerequisite for agreeing the technical solution. Phil On 22 January 2013 13:34, Rob Godfrey rob.j.godf...@gmail.com wrote: On 22 January 2013 13:47, Rafael Schloming r...@alum.mit.edu wrote: On Tue, Jan 22, 2013 at 4:22 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: On 21 January 2013 18:05, Rafael Schloming r...@alum.mit.edu wrote: On Mon, Jan 21, 2013 at 9:33 AM, Rob Godfrey rob.j.godf...@gmail.com wrote: Ummm... it's a dependency... you're familiar with those, yeah? The same way that the Qpid JMS clients depend on a JMS API jar, for which the source is readily available from another source. The JNI binding would build if the dependency was installed. The same way I believe the SSL code in the core of proton-c builds if the dependency for it is installed. That's not really a proper analogy. Again the JMS interfaces are defined outside of qpid. We don't release them, and we depend only on a well defined version of them, we don't share a release cycle with them. If the JMS API was something that we developed/defined right alongside the impl and was part of the same release process, we would certainly not be allowed to release without the source. This releasing without the source is a complete red herring and you know it. The source is released in whichever scheme we settle upon. If you want an example of dependencies within the qpid project, how did the AMQP 1.0 work on the C++ broker get released for 0.20? Did all the proton source get released with the C++ Broker / client? In the future are you expecting every part of the Qpid project which depends on proton to include its full source? If yes then how is the source tree going to work - is everything to be a subdirectory of proton-c? Again that's not really the same. If the Java API where on a separate (staggered) release cycle and the dependency was on a specific version, then that would be the same, but for what we're discussing, it really isn't. Proton and the cpp broker live under different trunks and branch/release separately, as far as I know this is not what you're proposing for the Java API, it is to live under the same trunk and branch/release together. The point was that the source code doesn't need to be in the same tarball let alone the same subdirectory in source control. If one considers that the Java API is a dependency then whether it is released concurrently or not with the JNI binding is moot. I've already said that it is preferable to have the source within the same tarball for the source release, but if needs be then I
RE: Is Proton a lightweight alternative to Qpid?
Ted,
This is a follow up of our earlier discussion. After building Proton using
cmake as described, I copied and placed those four files into another directory:
~/myproject/proton/proton.py
~/myproject/proton/cproton.py
~/myproject/proton/_cproton.so
~/myproject/proton/libqpid-proton.so.1.0.0
~/myproject/proton/libqpid-proton.so.1
I found that I also needed a link libqpid-proton.so.1 to point to
libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build
directory, when I try to import proton.py, it spits out:
import proton
Traceback (most recent call last):
File stdin, line 1, in module
File proton.py, line 33, in module
from cproton import *
File cproton.py, line 26, in module
_cproton = swig_import_helper()
File cproton.py, line 22, in swig_import_helper
_mod = imp.load_module('_cproton', fp, pathname, description)
ImportError: libqpid-proton.so.1: cannot open shared object file: No such file
or directory
Is there something in cmake that is adding absolute paths to the library files?
When I built again, importing proton worked fine. Is there a cmake variable
that I need to turn off to remove any references to the build directory? The
reason I need to do this is because when I copy it over to a clean install of
the same Linux version, it spits out that import error.
Thanks,
Taylor
From: Eagy, Taylor [te...@blackbirdtech.com]
Sent: Wednesday, January 16, 2013 12:13 PM
To: proton@qpid.apache.org
Subject: RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted,
Thanks for your help on this. I'm excited to see that proton is getting a
Windows port since I wasn't able to build it in VS2012 successfully.
Thanks,
Taylor
From: Rafael Schloming [r...@alum.mit.edu]
Sent: Tuesday, January 15, 2013 8:13 PM
To: proton@qpid.apache.org
Subject: Re: Is Proton a lightweight alternative to Qpid?
If you run cmake this way you can build the minimal code needed for just
the proton library and its python bindings:
cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF
-DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree
A quick test on my system shows that a make install based on the above
build works out to about 1.4MB. Stripping out header files and some package
config stuff would get you down to about 1.2MB if you want to go super
barebones.
--Rafael
On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote:
Taylor,
You need the following files:
proton.py (from proton-c/bindings/python)
cproton.py (from $BUILD/bindings/python)
_cproton.so(from $BUILD/bindings/python)
libqpid-proton.so (from $BUILD)
-Ted
On 01/15/2013 03:35 PM, Eagy, Taylor wrote:
Ted,
Proton is more lightweight and the systems that it runs on won't have
Java installed. While I would prefer a more Pythonic portable solution, as
long as Proton-c builds within 5MB, then it should work. However, I'm
getting a bunch of undefined reference messages from pythonPYTHON_wrap.c
when trying to make install it. So if I just want to use the p2p messaging
between Python processes, what are the minimum amount of files that I need
to create a Python queue server to handle the queues between processes?
(i.e. proton.py, cproton.py, etc)
Thanks,
Taylor
