Re: soap over QPid?
Hi, Basically you need to send your SOPA/REST messages embedded in a Qpid(JMS) message and a queue/topic on the broker acts as an endpoint. [1] explains how to use the Qpid Java broker with Apache Axis2 invoke Web services using it's JMS transport for an example. [1] http://wso2.org/library/3663 Danushka On Wed, Nov 9, 2011 at 1:49 AM, Daryoush Mehrtash wrote: > I am looking into possibility of proxy-ing SOAP and/or REST interface over > messaging (QPID). Any suggestions? Good starting point? > > Thanks > > Daryoush >
Re: Java Broker - creating and closing too many connections causes memory leak?
Hi Praveen,
AFAIK what you see is a known issue in the Java broker. I ran into the same
issue when I used the Java broker in a messaging product based on the Java
broker. Our team has submitted a couple of patches too but then the code
base was in the middle of some major refactoring. I think Robbie is the
right person to elaborate on this.
Thanks,
Danushka
On Sat, Oct 29, 2011 at 12:31 AM, Praveen M wrote:
> Hi,
>
> I ran the following test a few times and I ran into a OOM exception. Can
> someone please tell me if i'm doing something wrong or if this could
> potentially be a bug? Thanks.
>
> 1) Create 1000 queues.
> 2) Create 4 consumers -> each consumer uses it's own connection
> 3) In a loop do the following,
>
> for each queues 1..1000 {
>create producer(create new connection) for queue |i|
>produce 1 message for that queue.
>close producer (close connection)
> }
>
> 4) Wait till all messages are consumed by the 4 consumers and then close
> all
> 4 consumers.
> 5) Re-Run the test without a broker restart
>
> Each time the test ran, it created and closed 1004 connections (with at
> most
> 5 connections living in parallel...we close the earlier queue's producer's
> connection before creating a producer for the next queue).
>
> Also, I checked for any stray threads on the client side due to any
> unclosed
> connection, and it looks like everything was closed fine (I didn't see any
> stray threads).
>
> On the broker side, I saw that the memory usage creeps up each time i
> re-run
> the test, and eventually hits a OOM. (Running the above test about 5 times
> causes a OOM )
>
> I tried profiling the broker to see what's holding on to all that memory
> and
> saw a certain class's(org.apache.qpid.transport.Method[]) instance count
> grow up each time my test was run and it was holding the maximum memory in
> the broker.
>
> The number of instances of org.apache.qpid.transport.Method[] were trending
> as follows:
>
> After first test - 2008
> After second test - 4016
> ... and so on.
>
> There seems to be a direct relation between the number of those objects and
> the connections created and closed. ~ Why are these objects not freed on
> closing the connection??
> I'm using the Java Broker out of trunk and using it with a
> DerbyMessageStore
> for the test.
>
> Or is there something fundamentally wrong by the way I run the test, I
> guess
> opening and closing so many connections isn't a great idea (maybe I should
> be dealing with sessions). But still, I see that this can be a potential
> problem in a really long running broker system(where we cannot afford to
> restart the broker periodically) which can have clients joining/leaving at
> regular intervals.
>
>
> Thank you for all the effort answering our questions,
> --
> -Praveen
>
Re: Qpid Java Broker performance lower than expected
Hi Robbie, I did not notice that the BDB store was faster than the Derby store when I checked some time back. Thanks, Danushka On Wed, Oct 26, 2011 at 5:07 AM, Robbie Gemmell wrote: > Hi Vinay, > > I havent done any performance benchmarking of the Derby store to know > what a representative number would actually be, but I will try to take > a look at some point. I havent actually used QpidBench, so can I ask > if there were any specific command(s) you ran so I can try the same > scenarios? > > We havent paid much attention to performance of the Java broker for a > while unfortunately because we have been working on various other > issues such agetting memory usage under control and sorting out > correctness issues etc since adding a newer protocol version and doing > some significant refactorings and reimplementations, but as we reach > the light at the end of the tunnel on those it is something which > should move further up the priority list. > > It is worth nothing that there is also a BDB persistent store for the > Java broker that you might want to look at, as I would expect it to be > faster. It has recently been moved into the main repo, but is still an > optional module which you need to explicitly ask for to be built > (because BDB itself uses the Sleepycat Licence, which invokes > restrictions upon distribution that mean it is not Apache Licence > compatible). You can build the store module and include it (but not > BDB itself) in the broker binary release bundle by using the following > build command: > > ant build release-bin -Dmodules.opt=bdbstore -Ddownload-bdb=true > > You will find that downloads the bdb je jar into > qpid/java/lib/bdbstore, and then creates a broker binary release in > qpid/java/broker/release which includes the additional store module. > You can make the BDB je jar available to the broker by creating a > lib\opt subdir and copying the je jar into it, where it will get > picked up automatically assuming you are using Java 6+. You can then > use org.apache.qpid.server.store.berkeleydb.BDBMessageStore as the > store class config instead of the other stores. > > Robbie > > On 24 October 2011 16:25, vipun wrote: > > Hi, > > I'm collecting performance figures for QPID Java based broker. The > results > > which i got after running the QpidBench program are a little lower than > > expected. My machine which is a quad core, 8GB RAM with Windows 7 gives a > > message throughput of around 400 messages when both producer and consumer > > client instances are active. > > > > Qpid Java broker is configured to run over Derby and messaging is in > > persistent mode. I was expecting somewhere around 1000 atleast going by > the > > following blog which does comparisons between different messaging > providers. > > > > http://bhavin.directi.com/rabbitmq-vs-apache-activemq-vs-apache-qpid/ > > > > Do you think, the figures from my tests are correct, or what are the > > expected performance results, or are there any tweaks which need to be > done > > for performance gains. I am running out of trunk. > > > > Thanks & Regards > > Vinay > > > > -- > > View this message in context: > http://apache-qpid-users.2158936.n2.nabble.com/Qpid-Java-Broker-performance-lower-than-expected-tp6925405p6925405.html > > Sent from the Apache Qpid users mailing list archive at Nabble.com. > > > > - > > Apache Qpid - AMQP Messaging Implementation > > Project: http://qpid.apache.org > > Use/Interact: mailto:users-subscr...@qpid.apache.org > > > > > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:users-subscr...@qpid.apache.org > >
Re: Java Broker with DerbyMessageStore running out of memory with 20K Queues
Hi Praveen,
Do you notice the same behavior even when you run the broker without Derby
message store?. AFAIK this has nothing to do with the persistence storage
you use.
Thanks,
Danushka
On Wed, Oct 12, 2011 at 4:14 AM, Praveen M wrote:
> Hi,
>
> I'm an apache qpid newbie and am trying to benchmark Qpid Java Broker to
> see
> if it could be use for one of my usecase.
>
> My UseCase requires the ability to create atleast 20K persistent queues and
> have them all running in parallel.
>
> I am using the DerbyMessageStore as I understand that the default
> MemoryMessageQueue is not persistant across broker restarts.
>
> I'm running the broker with a heap of 4GB and options QPID_OPTS set to
> -Damqj.read_write_pool_size=32 -Dmax_prefetch=1
>
>
> My test does the following:
>
> 1) Creates a queue and registers a listener on that queue. I do this upto
> 20K times for 20K distinct queues. I create the queues with the following
> option
>create: always , node : {type : queue, durable : true}}
>- this step goes quite fine. I was monitoring the memory usage during
> this step and it almost always stayed stable around 500-800MB
> 2) I produce messages for the queues (one message for each queues) and the
> messages are consumed by the registered handlers in step 1.
>- When this step starts, the memory usage just shoots up and exhausts my
> 4GB memory all together.
>
>
> Can someone please help me explaining why I am seeing this kind of a
> behavior?
>
> Also, Can you please point out if I'm missing some setting or doing
> something completely wrong/stupid?
>
>
> Thanks,
> --
> -Praveen
>
Re: Clustering Support
Failover on client side? Thanks, Danushka On Wed, Aug 3, 2011 at 12:12 AM, Carl Trieloff wrote: > On 08/02/2011 02:40 PM, prasanth baireddy wrote: > > Hi, > > Iam using apache qpid(java) 0.10 in my application.The scenario is the > > application should support connecting to 100 applications at the same > time > > for message exchanges. > > Can apche qpid provide clustering support for this.Please suugest ur > ideas > > on the above issue > > Thanks in Advance > > > > should do based on limited details provided. > > Carl. > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:users-subscr...@qpid.apache.org > >
Re: Session auto acknowledge
Have you created your JMS session with ack mode Session.CLIENT_ACKNOWLEDGE? Thanks, Danushka On Mon, Jul 25, 2011 at 4:13 PM, Emir Ibrahimbegovic < emir.ibrahimbego...@gmail.com> wrote: > Hello, > > I'm trying to download messages from my QPID queue without acknowledging > them. Even though I don't acknowledge them, they still get downloaded from > the queue. How can I check is my QPID queue setup with auto acknowledge(is > there something like that)? > > Thanks, > E >
Re: [ANNOUNCE] Apache Qpid 0.10 released
Hi Robbie, The download link seems to be broken. Thanks, Danushka On Tue, May 3, 2011 at 9:54 PM, Robbie Gemmell wrote: > The Apache Qpid community is pleased to announce the immediate > availability of Apache Qpid 0.10. > > Apache Qpid (http://qpid.apache.org) is a cross-platform enterprise > messaging solution which implements the Advanced Message Queuing > Protocol (AMQP, http://www.amqp.org). It provides brokers written in > C++ and Java, along with clients for C++, Java JMS, .Net, Python, and > Ruby. > > Qpid 0.10 is available from our website: > > http://qpid.apache.org/download.cgi > > The 0.10 release has undergone a lot of bug fixing and feature > enhancement. We recommend that all users upgrade. A few changes of > note: > > - The C++ broker now supports priority-ordered message queuing > - The C++ broker and client now implement producer flow control > - The Java JMS client is now available via Maven > > A comprehensive list of features, improvements, and bugs resolved in > the 0.10 release is available here: > > http://qpid.apache.org/release_notes_0.10.html > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:dev-subscr...@qpid.apache.org > >
Getting ACL to Work with Java Broker
Hi devs, I have configured the broker has explained in [1] but I do not see the clients being checked for authorization other than virtualhost-level permissions. I am using a trunk build. Please let me know if the doc is up-to-date and possible goofs that I may have made. [1] - http://qpid.apache.org/java-xml-acls.html Thanks, Danushka
Re: Without Ruby ...
Hi Anil, I am newbie to qpid, apologize for stupid questions. Did not pull the files from Source Repository, downloaded M4 Release ( http://www.apache.org/dist/qpid/M4/qpid-cpp-M4.tar.gz ) I need just C++ client., Is python and Ruby dependency only from client standpoint ? This is a bug in the release artifact. Ruby is required for code generation and what you have downloaded already includes generated source. You have two options to get rid of this. 1. Install Ruby on your system :). This includes minimum hassle. 2. Open your configure.ac and comment out this block of rules. AC_CHECK_PROG([RUBY], [ruby], [ruby]) test -n "$RUBY" && generate=yes test -z "$RUBY" && AC_MSG_ERROR([Missing ruby installation (try "yum install ruby").]) Then run autoconf & automake to generate your new configure script. Now run ./configure. Danushka -- Danushka Menikkumbura Technical Lead, WSO2 Inc. blog : http://danushka-menikkumbura.blogspot.com/ http://wso2.com/ - "The Open Source SOA Company" - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: notes on building and testing M4 for Windows
How about adding a wiki page on the qpid.apache.org site for this? It sounds like it would be very useful. +1. Danushka -- Danushka Menikkumbura Technical Lead, WSO2 Inc. blog : http://danushka-menikkumbura.blogspot.com/ http://wso2.com/ - "The Open Source SOA Company" - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: c++ broker build for ubuntu?
Hi, Please refer to the INSTALL file in the C++ root directory for build instructions, dependencies, etc. You need to have C++ source and the specs to build SVN source. Cheers, Danushka I am attempting to install qpid on an ubuntu machine. Since ubuntu is a very popular distribution, perhaps the devs could add it to the ubuntu repo so it can be installed as easily as fedora? (perhaps one of the non-redhat employees :) ) The wiki pages are quite confusing for someone trying to build it themselves. I installed the following packages: apt-get install libboost-dev uuid-dev pkg-config build-essential make autoconf automake help2man libtool doxygen graphviz ruby (it took a little googling to find the right packages for libboost, uuid, etc.) The next step on this page (http://qpid.apache.org/qpid-cpp-build-how-to.html) says that I need to check out Java source...notice that the web page file name refers to "cpp-build-how-to" and it is linked as such from another page. Yet the bread-crumb-trail at the top of the page puts it under java: Apache Qpid > Index > Qpid Java Documentation > Qpid Java How To > Qpid Java Build How To > Building > Qpid Cpp Build How To I'm checking out recommended source code now (I think it is going to retrieve the full trunk, not just c++). Will post feedback if there are further hiccups. -- Danushka Menikkumbura Technical Lead, WSO2 Inc. blog : http://danushka-menikkumbura.blogspot.com/ http://wso2.com/ - "The Open Source SOA Company" - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: Compiling qpid in Windows
Hey Josep, I spent like a week or so racking my brains trying to run the C++ Examples of qpid on Windows, but the farthest I could get was exactly where you did. It seems like there are some changes that need to be made above the project generation, otherwise it's quite unlikely that we manage to make this last version work on Windows. There are lots of recursive dependencies among the qpid classes that always get you to compilation errors. did anyone make these Examples run? is there anyone who knows if any of the least older versions of Qpid did work on Windows? We've been asked by our client to have a demo of the software as soon as possible running under Windows and this is delaying our work so much. Josep, if you manage to make it work, please let me know. regards, Errors related to symbol redefinitions is a common pitfall in using static libs. Just try changing the library link order and see. Another option is to get the compiler not to load conflicting symbols from subsequent libraries. But that is really painful if it complains about so many conflicts. I am working on the DLLs. I might be able to have something usable somewhere mid next week. Danushka -- Danushka Menikkumbura Technical Lead, WSO2 Inc. blog : http://danushka-menikkumbura.blogspot.com/ http://wso2.com/ - "The Open Source SOA Company" - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: How to run the Examples on visual studio?
I'm trying to make qpid work using Microsoft Visual Studio 9.0. I've managed to start the broker after several hours. At the moment it's running and listening to port 5672. Once I've achieved this, I've been trying to run any of the examples that come with qpid, but so far didn't manage to do it. It looks like everything has been thought to be built on a Linux system. I'm not being able to run neither the examples under the /Examples directory nor the tests under /src/tests directory. Can anybody help me out with this? Perhaps I'm not using Visual Studio properly. Is anyone running the examples with this framework? By the way, I'm using the version of qpid that can be found here: http://www.apache.org/dist/qpid/M4/qpid-cpp-M4.tar.gz Thanks in advance for your help. Marc I think you have to create a VS solution for examples, yourself. Danushka -- Danushka Menikkumbura Technical Lead, WSO2 Inc. blog : http://danushka-menikkumbura.blogspot.com/ http://wso2.com/ - "The Open Source SOA Company" - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
