Re: Is Proton a lightweight alternative to Qpid?
A couple of things...
I believe cmake places an 'rpath' on the libraries when they are built.
This is removed when you run 'make install' and the libraries are copied
to their installed location.
The .so and .so.1 files are soft links to the .so.1.0.0 file. Depending
on how you copy them, they may act in unexpected ways.
You may need to set the environment variable LD_LIBRARY_PATH to your
local directory so the loader can find the library file if it's not in a
standard install location.
I hope this helps,
-Ted
On 01/22/2013 06:02 PM, Eagy, Taylor wrote:
Ted,
This is a follow up of our earlier discussion. After building Proton using
cmake as described, I copied and placed those four files into another directory:
~/myproject/proton/proton.py
~/myproject/proton/cproton.py
~/myproject/proton/_cproton.so
~/myproject/proton/libqpid-proton.so.1.0.0
~/myproject/proton/libqpid-proton.so.1
I found that I also needed a link libqpid-proton.so.1 to point to
libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build
directory, when I try to import proton.py, it spits out:
import proton
Traceback (most recent call last):
File stdin, line 1, in module
File proton.py, line 33, in module
from cproton import *
File cproton.py, line 26, in module
_cproton = swig_import_helper()
File cproton.py, line 22, in swig_import_helper
_mod = imp.load_module('_cproton', fp, pathname, description)
ImportError: libqpid-proton.so.1: cannot open shared object file: No such file
or directory
Is there something in cmake that is adding absolute paths to the library files?
When I built again, importing proton worked fine. Is there a cmake variable
that I need to turn off to remove any references to the build directory? The
reason I need to do this is because when I copy it over to a clean install of
the same Linux version, it spits out that import error.
Thanks,
Taylor
From: Eagy, Taylor [te...@blackbirdtech.com]
Sent: Wednesday, January 16, 2013 12:13 PM
To: proton@qpid.apache.org
Subject: RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted,
Thanks for your help on this. I'm excited to see that proton is getting a
Windows port since I wasn't able to build it in VS2012 successfully.
Thanks,
Taylor
From: Rafael Schloming [r...@alum.mit.edu]
Sent: Tuesday, January 15, 2013 8:13 PM
To: proton@qpid.apache.org
Subject: Re: Is Proton a lightweight alternative to Qpid?
If you run cmake this way you can build the minimal code needed for just
the proton library and its python bindings:
cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF
-DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree
A quick test on my system shows that a make install based on the above
build works out to about 1.4MB. Stripping out header files and some package
config stuff would get you down to about 1.2MB if you want to go super
barebones.
--Rafael
On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote:
Taylor,
You need the following files:
proton.py (from proton-c/bindings/python)
cproton.py (from $BUILD/bindings/python)
_cproton.so(from $BUILD/bindings/python)
libqpid-proton.so (from $BUILD)
-Ted
On 01/15/2013 03:35 PM, Eagy, Taylor wrote:
Ted,
Proton is more lightweight and the systems that it runs on won't have
Java installed. While I would prefer a more Pythonic portable solution, as
long as Proton-c builds within 5MB, then it should work. However, I'm
getting a bunch of undefined reference messages from pythonPYTHON_wrap.c
when trying to make install it. So if I just want to use the p2p messaging
between Python processes, what are the minimum amount of files that I need
to create a Python queue server to handle the queues between processes?
(i.e. proton.py, cproton.py, etc)
Thanks,
Taylor
RE: Is Proton a lightweight alternative to Qpid?
Thanks a lot Ted. It's working now. I appreciate all your help.
Thanks,
Taylor
From: Ted Ross [tr...@redhat.com]
Sent: Wednesday, January 23, 2013 8:26 AM
To: proton@qpid.apache.org
Subject: Re: Is Proton a lightweight alternative to Qpid?
A couple of things...
I believe cmake places an 'rpath' on the libraries when they are built.
This is removed when you run 'make install' and the libraries are copied
to their installed location.
The .so and .so.1 files are soft links to the .so.1.0.0 file. Depending
on how you copy them, they may act in unexpected ways.
You may need to set the environment variable LD_LIBRARY_PATH to your
local directory so the loader can find the library file if it's not in a
standard install location.
I hope this helps,
-Ted
On 01/22/2013 06:02 PM, Eagy, Taylor wrote:
Ted,
This is a follow up of our earlier discussion. After building Proton using
cmake as described, I copied and placed those four files into another
directory:
~/myproject/proton/proton.py
~/myproject/proton/cproton.py
~/myproject/proton/_cproton.so
~/myproject/proton/libqpid-proton.so.1.0.0
~/myproject/proton/libqpid-proton.so.1
I found that I also needed a link libqpid-proton.so.1 to point to
libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build
directory, when I try to import proton.py, it spits out:
import proton
Traceback (most recent call last):
File stdin, line 1, in module
File proton.py, line 33, in module
from cproton import *
File cproton.py, line 26, in module
_cproton = swig_import_helper()
File cproton.py, line 22, in swig_import_helper
_mod = imp.load_module('_cproton', fp, pathname, description)
ImportError: libqpid-proton.so.1: cannot open shared object file: No such
file or directory
Is there something in cmake that is adding absolute paths to the library
files? When I built again, importing proton worked fine. Is there a cmake
variable that I need to turn off to remove any references to the build
directory? The reason I need to do this is because when I copy it over to a
clean install of the same Linux version, it spits out that import error.
Thanks,
Taylor
From: Eagy, Taylor [te...@blackbirdtech.com]
Sent: Wednesday, January 16, 2013 12:13 PM
To: proton@qpid.apache.org
Subject: RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted,
Thanks for your help on this. I'm excited to see that proton is getting a
Windows port since I wasn't able to build it in VS2012 successfully.
Thanks,
Taylor
From: Rafael Schloming [r...@alum.mit.edu]
Sent: Tuesday, January 15, 2013 8:13 PM
To: proton@qpid.apache.org
Subject: Re: Is Proton a lightweight alternative to Qpid?
If you run cmake this way you can build the minimal code needed for just
the proton library and its python bindings:
cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF
-DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree
A quick test on my system shows that a make install based on the above
build works out to about 1.4MB. Stripping out header files and some package
config stuff would get you down to about 1.2MB if you want to go super
barebones.
--Rafael
On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote:
Taylor,
You need the following files:
proton.py (from proton-c/bindings/python)
cproton.py (from $BUILD/bindings/python)
_cproton.so(from $BUILD/bindings/python)
libqpid-proton.so (from $BUILD)
-Ted
On 01/15/2013 03:35 PM, Eagy, Taylor wrote:
Ted,
Proton is more lightweight and the systems that it runs on won't have
Java installed. While I would prefer a more Pythonic portable solution, as
long as Proton-c builds within 5MB, then it should work. However, I'm
getting a bunch of undefined reference messages from pythonPYTHON_wrap.c
when trying to make install it. So if I just want to use the p2p messaging
between Python processes, what are the minimum amount of files that I need
to create a Python queue server to handle the queues between processes?
(i.e. proton.py, cproton.py, etc)
Thanks,
Taylor
RE: Is Proton a lightweight alternative to Qpid?
Ted,
This is a follow up of our earlier discussion. After building Proton using
cmake as described, I copied and placed those four files into another directory:
~/myproject/proton/proton.py
~/myproject/proton/cproton.py
~/myproject/proton/_cproton.so
~/myproject/proton/libqpid-proton.so.1.0.0
~/myproject/proton/libqpid-proton.so.1
I found that I also needed a link libqpid-proton.so.1 to point to
libqpid-proton.so.1.0.0. However, after I removed the qpid-proton-c-0.2/build
directory, when I try to import proton.py, it spits out:
import proton
Traceback (most recent call last):
File stdin, line 1, in module
File proton.py, line 33, in module
from cproton import *
File cproton.py, line 26, in module
_cproton = swig_import_helper()
File cproton.py, line 22, in swig_import_helper
_mod = imp.load_module('_cproton', fp, pathname, description)
ImportError: libqpid-proton.so.1: cannot open shared object file: No such file
or directory
Is there something in cmake that is adding absolute paths to the library files?
When I built again, importing proton worked fine. Is there a cmake variable
that I need to turn off to remove any references to the build directory? The
reason I need to do this is because when I copy it over to a clean install of
the same Linux version, it spits out that import error.
Thanks,
Taylor
From: Eagy, Taylor [te...@blackbirdtech.com]
Sent: Wednesday, January 16, 2013 12:13 PM
To: proton@qpid.apache.org
Subject: RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted,
Thanks for your help on this. I'm excited to see that proton is getting a
Windows port since I wasn't able to build it in VS2012 successfully.
Thanks,
Taylor
From: Rafael Schloming [r...@alum.mit.edu]
Sent: Tuesday, January 15, 2013 8:13 PM
To: proton@qpid.apache.org
Subject: Re: Is Proton a lightweight alternative to Qpid?
If you run cmake this way you can build the minimal code needed for just
the proton library and its python bindings:
cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF
-DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree
A quick test on my system shows that a make install based on the above
build works out to about 1.4MB. Stripping out header files and some package
config stuff would get you down to about 1.2MB if you want to go super
barebones.
--Rafael
On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote:
Taylor,
You need the following files:
proton.py (from proton-c/bindings/python)
cproton.py (from $BUILD/bindings/python)
_cproton.so(from $BUILD/bindings/python)
libqpid-proton.so (from $BUILD)
-Ted
On 01/15/2013 03:35 PM, Eagy, Taylor wrote:
Ted,
Proton is more lightweight and the systems that it runs on won't have
Java installed. While I would prefer a more Pythonic portable solution, as
long as Proton-c builds within 5MB, then it should work. However, I'm
getting a bunch of undefined reference messages from pythonPYTHON_wrap.c
when trying to make install it. So if I just want to use the p2p messaging
between Python processes, what are the minimum amount of files that I need
to create a Python queue server to handle the queues between processes?
(i.e. proton.py, cproton.py, etc)
Thanks,
Taylor
RE: Is Proton a lightweight alternative to Qpid?
Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
RE: Is Proton a lightweight alternative to Qpid?
@qpid.apache.org Subject: RE: Is Proton a lightweight alternative to Qpid? Rafael and Ted, Thanks for your help on this. I'm excited to see that proton is getting a Windows port since I wasn't able to build it in VS2012 successfully. Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:13 PM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? If you run cmake this way you can build the minimal code needed for just the proton library and its python bindings: cmake -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_PYTHON=ON -DBUILD_PHP=OFF -DBUILD_PERL=OFF -DBUILD_RUBY=OFF path_to_src_tree A quick test on my system shows that a make install based on the above build works out to about 1.4MB. Stripping out header files and some package config stuff would get you down to about 1.2MB if you want to go super barebones. --Rafael On Tue, Jan 15, 2013 at 5:38 PM, Ted Ross tr...@redhat.com wrote: Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
Re: Is Proton a lightweight alternative to Qpid?
On Mon, Jan 14, 2013 at 11:14 AM, Eagy, Taylor te...@blackbirdtech.comwrote: Hi guys, I've been using Qpid for the past several months and I really like it. However, I've mainly just been using it to pass messages between several Python processes running on the same machine, so using Qpid is probably overkill. Then I noticed Proton and got excited. Ideally I'm looking for a fast, lightweight, and portable queueing library preferrably in Python. Are there any roadmap plans to create a Proton Python broker/engine? I looked at RabbitMQ, but read the performance wasn't as good as Qpid. I was looking into the examples and noticed you could implement your own queueing server in Python, but it's not exactly portable since I'd have to build it for Linux and Windows. Hi, To answer the question in your subject first, Proton isn't an alternative to Qpid per/se, rather it's a component of Qpid. The latest release of the cpp broker uses proton to provide AMQP 1.0 support, and we plan to use it in future releases of the Java broker also. That said, Proton is definitely intended to be used independently of either the cpp or Java broker, and can be used without either (i.e. peer to peer), so in that sense the answer to your question is yes. There is definitely interest in building a lightweight queuing component that works well with Proton and can be flexibly deployed in a variety of topologies, and even dynamically/transparently redeployed at runtime. A python prototype has been discussed as a starting point for some of this work, and there has been other work ongoing both in terms of re-factoring the cpp broker and in terms of prototyping new servers that may ultimately contribute. Why do you think your own queuing server built in python wouldn't be portable? Can you describe a little bit more about your scenario, e.g. do you need persistence, transactions, etc ...? Do you have any particular performance requirements? Can you describe your messaging topology at all? --Rafael
Re: Is Proton a lightweight alternative to Qpid?
Taylor, Another possibility to look at is using ActiveMQ as your broker. They've added an AMQP transport (based on Proton-J) to their project recently. -Ted On 01/15/2013 12:37 PM, Eagy, Taylor wrote: Rafael, Thanks for responding. The only reason why I said it wouldn't be portable is because when I saw the CMake files I thought I'd have to build it to bind C functions from the engine to Python. However, if I can perform the p2p messaging just using the proton.py then that would work. My scenario is the following: I have three Python modules running. One is a web service that takes incoming requests and places the request data as a message on an incoming queue, then another Python service listens on the incoming queue and processes the data in the messages. After it finishes processing the data it passes the results as another message onto an outgoing queue which is then grabbed by the last Python service and sent back. Previously I was using the Java Qpid broker because I need persistence enabled so that if the Qpid broker or one of these Python services were to fail during a queue transaction, the services, when restarted, would be able to pickup the durable message and continue the data flow. While it works great with the Qpid broker, the main issue is that the Qpid broker is just too heavy. I need a fast lightweight version that still offers the basic persistence (I was using Derby store) and ideally written in Python since all of my code is in Python. I don't have any specific performance requirements other than saying the faster the better. Right now all of the queueing is done locally between these services, but I would like to have the ability to extend it easily to work over networked machines which I know Proton can do. Do you think it's overkill to use something like Qpid or Proton in my scenario? Thanks, Taylor From: Rafael Schloming [r...@alum.mit.edu] Sent: Tuesday, January 15, 2013 8:44 AM To: proton@qpid.apache.org Subject: Re: Is Proton a lightweight alternative to Qpid? On Mon, Jan 14, 2013 at 11:14 AM, Eagy, Taylor te...@blackbirdtech.comwrote: Hi guys, I've been using Qpid for the past several months and I really like it. However, I've mainly just been using it to pass messages between several Python processes running on the same machine, so using Qpid is probably overkill. Then I noticed Proton and got excited. Ideally I'm looking for a fast, lightweight, and portable queueing library preferrably in Python. Are there any roadmap plans to create a Proton Python broker/engine? I looked at RabbitMQ, but read the performance wasn't as good as Qpid. I was looking into the examples and noticed you could implement your own queueing server in Python, but it's not exactly portable since I'd have to build it for Linux and Windows. Hi, To answer the question in your subject first, Proton isn't an alternative to Qpid per/se, rather it's a component of Qpid. The latest release of the cpp broker uses proton to provide AMQP 1.0 support, and we plan to use it in future releases of the Java broker also. That said, Proton is definitely intended to be used independently of either the cpp or Java broker, and can be used without either (i.e. peer to peer), so in that sense the answer to your question is yes. There is definitely interest in building a lightweight queuing component that works well with Proton and can be flexibly deployed in a variety of topologies, and even dynamically/transparently redeployed at runtime. A python prototype has been discussed as a starting point for some of this work, and there has been other work ongoing both in terms of re-factoring the cpp broker and in terms of prototyping new servers that may ultimately contribute. Why do you think your own queuing server built in python wouldn't be portable? Can you describe a little bit more about your scenario, e.g. do you need persistence, transactions, etc ...? Do you have any particular performance requirements? Can you describe your messaging topology at all? --Rafael
Re: Is Proton a lightweight alternative to Qpid?
Taylor, You need the following files: proton.py (from proton-c/bindings/python) cproton.py (from $BUILD/bindings/python) _cproton.so(from $BUILD/bindings/python) libqpid-proton.so (from $BUILD) -Ted On 01/15/2013 03:35 PM, Eagy, Taylor wrote: Ted, Proton is more lightweight and the systems that it runs on won't have Java installed. While I would prefer a more Pythonic portable solution, as long as Proton-c builds within 5MB, then it should work. However, I'm getting a bunch of undefined reference messages from pythonPYTHON_wrap.c when trying to make install it. So if I just want to use the p2p messaging between Python processes, what are the minimum amount of files that I need to create a Python queue server to handle the queues between processes? (i.e. proton.py, cproton.py, etc) Thanks, Taylor
Is Proton a lightweight alternative to Qpid?
Hi guys, I've been using Qpid for the past several months and I really like it. However, I've mainly just been using it to pass messages between several Python processes running on the same machine, so using Qpid is probably overkill. Then I noticed Proton and got excited. Ideally I'm looking for a fast, lightweight, and portable queueing library preferrably in Python. Are there any roadmap plans to create a Proton Python broker/engine? I looked at RabbitMQ, but read the performance wasn't as good as Qpid. I was looking into the examples and noticed you could implement your own queueing server in Python, but it's not exactly portable since I'd have to build it for Linux and Windows. Thanks, TJ
