Re: Java Broker - creating and closing too many connections causes memory leak?
Thanks a lot Robbie for the prompt reply and looking into this.
Praveen
On Fri, Oct 28, 2011 at 7:41 PM, Robbie Gemmell wrote:
> This isn't a current known issue since work was previously undertaken
> to fix similar problems, so this sounds like a new bug or possibly
> something that has been reintroduced. The Method[] objects are part of
> the underlying transport layer backing each JMS Session, so they will
> directly correlate to the number of Connections and Sessions you are
> creating, although there appear to have been exactly twice what you
> indicated you are using per run which is stange. They should be
> getting cleared up, but something is obviously holding onto them.
>
> Opening and closing a connection and session per message is of course
> rather inefficient, but it should of course still work. I'll add this
> to my list of things to look at.
>
> Robbie
>
> On 28 October 2011 20:01, Praveen M wrote:
> > Hi,
> >
> > I ran the following test a few times and I ran into a OOM exception. Can
> > someone please tell me if i'm doing something wrong or if this could
> > potentially be a bug? Thanks.
> >
> > 1) Create 1000 queues.
> > 2) Create 4 consumers -> each consumer uses it's own connection
> > 3) In a loop do the following,
> >
> > for each queues 1..1000 {
> >create producer(create new connection) for queue |i|
> >produce 1 message for that queue.
> >close producer (close connection)
> > }
> >
> > 4) Wait till all messages are consumed by the 4 consumers and then close
> all
> > 4 consumers.
> > 5) Re-Run the test without a broker restart
> >
> > Each time the test ran, it created and closed 1004 connections (with at
> most
> > 5 connections living in parallel...we close the earlier queue's
> producer's
> > connection before creating a producer for the next queue).
> >
> > Also, I checked for any stray threads on the client side due to any
> unclosed
> > connection, and it looks like everything was closed fine (I didn't see
> any
> > stray threads).
> >
> > On the broker side, I saw that the memory usage creeps up each time i
> re-run
> > the test, and eventually hits a OOM. (Running the above test about 5
> times
> > causes a OOM )
> >
> > I tried profiling the broker to see what's holding on to all that memory
> and
> > saw a certain class's(org.apache.qpid.transport.Method[]) instance count
> > grow up each time my test was run and it was holding the maximum memory
> in
> > the broker.
> >
> > The number of instances of org.apache.qpid.transport.Method[] were
> trending
> > as follows:
> >
> > After first test - 2008
> > After second test - 4016
> > ... and so on.
> >
> > There seems to be a direct relation between the number of those objects
> and
> > the connections created and closed. ~ Why are these objects not freed on
> > closing the connection??
> > I'm using the Java Broker out of trunk and using it with a
> DerbyMessageStore
> > for the test.
> >
> > Or is there something fundamentally wrong by the way I run the test, I
> guess
> > opening and closing so many connections isn't a great idea (maybe I
> should
> > be dealing with sessions). But still, I see that this can be a potential
> > problem in a really long running broker system(where we cannot afford to
> > restart the broker periodically) which can have clients joining/leaving
> at
> > regular intervals.
> >
> >
> > Thank you for all the effort answering our questions,
> > --
> > -Praveen
> >
>
> -
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscr...@qpid.apache.org
>
>
--
-Praveen
Re: Java Broker - creating and closing too many connections causes memory leak?
This isn't a current known issue since work was previously undertaken
to fix similar problems, so this sounds like a new bug or possibly
something that has been reintroduced. The Method[] objects are part of
the underlying transport layer backing each JMS Session, so they will
directly correlate to the number of Connections and Sessions you are
creating, although there appear to have been exactly twice what you
indicated you are using per run which is stange. They should be
getting cleared up, but something is obviously holding onto them.
Opening and closing a connection and session per message is of course
rather inefficient, but it should of course still work. I'll add this
to my list of things to look at.
Robbie
On 28 October 2011 20:01, Praveen M wrote:
> Hi,
>
> I ran the following test a few times and I ran into a OOM exception. Can
> someone please tell me if i'm doing something wrong or if this could
> potentially be a bug? Thanks.
>
> 1) Create 1000 queues.
> 2) Create 4 consumers -> each consumer uses it's own connection
> 3) In a loop do the following,
>
> for each queues 1..1000 {
> create producer(create new connection) for queue |i|
> produce 1 message for that queue.
> close producer (close connection)
> }
>
> 4) Wait till all messages are consumed by the 4 consumers and then close all
> 4 consumers.
> 5) Re-Run the test without a broker restart
>
> Each time the test ran, it created and closed 1004 connections (with at most
> 5 connections living in parallel...we close the earlier queue's producer's
> connection before creating a producer for the next queue).
>
> Also, I checked for any stray threads on the client side due to any unclosed
> connection, and it looks like everything was closed fine (I didn't see any
> stray threads).
>
> On the broker side, I saw that the memory usage creeps up each time i re-run
> the test, and eventually hits a OOM. (Running the above test about 5 times
> causes a OOM )
>
> I tried profiling the broker to see what's holding on to all that memory and
> saw a certain class's(org.apache.qpid.transport.Method[]) instance count
> grow up each time my test was run and it was holding the maximum memory in
> the broker.
>
> The number of instances of org.apache.qpid.transport.Method[] were trending
> as follows:
>
> After first test - 2008
> After second test - 4016
> ... and so on.
>
> There seems to be a direct relation between the number of those objects and
> the connections created and closed. ~ Why are these objects not freed on
> closing the connection??
> I'm using the Java Broker out of trunk and using it with a DerbyMessageStore
> for the test.
>
> Or is there something fundamentally wrong by the way I run the test, I guess
> opening and closing so many connections isn't a great idea (maybe I should
> be dealing with sessions). But still, I see that this can be a potential
> problem in a really long running broker system(where we cannot afford to
> restart the broker periodically) which can have clients joining/leaving at
> regular intervals.
>
>
> Thank you for all the effort answering our questions,
> --
> -Praveen
>
-
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscr...@qpid.apache.org
Re: configure: error: db4-devel package missing. Installing BDB with Qpid C++ broker on ubuntu?
aww..my bad..ignore this mail. For curious ppl, I installed sudo apt-get install libdb4.8++-dev (the C++ library) and it worked. :) On Fri, Oct 28, 2011 at 4:54 PM, Praveen M wrote: > Hi, > > I get the following error when I try to install BDB on ubuntu with a C++ > qpid broker that I've built. > > configure: error: db4-devel package missing. Please ensure both db4 and > db4-devel are installed. (hint: "yum install db4-devel" should do it...) > > I'm following the installation procedure of BSD listed there, > > https://cwiki.apache.org/qpid/faq.html#FAQ-Persistence > > I run into the above issue when i do > > ./configure --with-qpid-checkout=../qpid/ > > I built the qpid C++ broker out of trunk, and now am trying to install BDB > to use with it. > > I installed BDB when i first got this issue by using > > sudo apt-get install libdb4.8-dev > > Is the above the right thing to do? Am I missing something? > > Any help will be great. > > Thank you, > -- > -Praveen > -- -Praveen
configure: error: db4-devel package missing. Installing BDB with Qpid C++ broker on ubuntu?
Hi, I get the following error when I try to install BDB on ubuntu with a C++ qpid broker that I've built. configure: error: db4-devel package missing. Please ensure both db4 and db4-devel are installed. (hint: "yum install db4-devel" should do it...) I'm following the installation procedure of BSD listed there, https://cwiki.apache.org/qpid/faq.html#FAQ-Persistence I run into the above issue when i do ./configure --with-qpid-checkout=../qpid/ I built the qpid C++ broker out of trunk, and now am trying to install BDB to use with it. I installed BDB when i first got this issue by using sudo apt-get install libdb4.8-dev Is the above the right thing to do? Am I missing something? Any help will be great. Thank you, -- -Praveen
How can you do at-least-one in federated broker?
I have a distributed QPID broker scenario that need to make sure messages
are received at-least-once in the remote clients. I am trying to see if I
have to configure the brokers or not.
On the source broker I have:
qpid-config -a SRC_BROKER add queue SRC_QUEUE
On Destination broker I set up the route and queues to read from
> qpid-route queue add DEST_BROKER SRC_BROKER amq.fanout SRC_QUEUE
> qpid-config -a DEST_BROKER add queue DEST_QUEUE
> qpid-config -a DEST_BROKER bind amq.fanout DEST_QUEUE
I like to run the spout and drain apps in the python example.
spout SRC_QUEUE
drain "DEST_QUEUE; {link:{reliability:"at-least-once"}}"
Questions:
a) Do I have to do anything on the "spout" to make sure the message has
at-least-once reliability>
b) Do I have to configure the Destination borker, or the route to make sure
all messages are sent to the broker.
Thanks
Daryoush
Re: Java Broker - creating and closing too many connections causes memory leak?
Hi Praveen,
AFAIK what you see is a known issue in the Java broker. I ran into the same
issue when I used the Java broker in a messaging product based on the Java
broker. Our team has submitted a couple of patches too but then the code
base was in the middle of some major refactoring. I think Robbie is the
right person to elaborate on this.
Thanks,
Danushka
On Sat, Oct 29, 2011 at 12:31 AM, Praveen M wrote:
> Hi,
>
> I ran the following test a few times and I ran into a OOM exception. Can
> someone please tell me if i'm doing something wrong or if this could
> potentially be a bug? Thanks.
>
> 1) Create 1000 queues.
> 2) Create 4 consumers -> each consumer uses it's own connection
> 3) In a loop do the following,
>
> for each queues 1..1000 {
>create producer(create new connection) for queue |i|
>produce 1 message for that queue.
>close producer (close connection)
> }
>
> 4) Wait till all messages are consumed by the 4 consumers and then close
> all
> 4 consumers.
> 5) Re-Run the test without a broker restart
>
> Each time the test ran, it created and closed 1004 connections (with at
> most
> 5 connections living in parallel...we close the earlier queue's producer's
> connection before creating a producer for the next queue).
>
> Also, I checked for any stray threads on the client side due to any
> unclosed
> connection, and it looks like everything was closed fine (I didn't see any
> stray threads).
>
> On the broker side, I saw that the memory usage creeps up each time i
> re-run
> the test, and eventually hits a OOM. (Running the above test about 5 times
> causes a OOM )
>
> I tried profiling the broker to see what's holding on to all that memory
> and
> saw a certain class's(org.apache.qpid.transport.Method[]) instance count
> grow up each time my test was run and it was holding the maximum memory in
> the broker.
>
> The number of instances of org.apache.qpid.transport.Method[] were trending
> as follows:
>
> After first test - 2008
> After second test - 4016
> ... and so on.
>
> There seems to be a direct relation between the number of those objects and
> the connections created and closed. ~ Why are these objects not freed on
> closing the connection??
> I'm using the Java Broker out of trunk and using it with a
> DerbyMessageStore
> for the test.
>
> Or is there something fundamentally wrong by the way I run the test, I
> guess
> opening and closing so many connections isn't a great idea (maybe I should
> be dealing with sessions). But still, I see that this can be a potential
> problem in a really long running broker system(where we cannot afford to
> restart the broker periodically) which can have clients joining/leaving at
> regular intervals.
>
>
> Thank you for all the effort answering our questions,
> --
> -Praveen
>
Re: Downloading the Built version of qpidc-0.12
FYI: Qpid is now in officially in Debian sid/wheezy. Two packages to go: the persistence module and qpid-tools. Am 28.10.2011 um 13:17 schrieb Gordon Sim: > On 10/26/2011 09:55 PM, Daniel Mounessa wrote: >> Is there a built version for Linux that I could download and use or do I >> need to download the source code and rebuild the qpidd? > > What distribution of linux are you using? For Fedora you should be able to > get qpid via yum. Cajus Pollmeier has produced debian squeeze packages, > available at http://apt.gonicus.de/. Other than that you will need to build > your own. > > - > Apache Qpid - AMQP Messaging Implementation > Project: http://qpid.apache.org > Use/Interact: mailto:users-subscr...@qpid.apache.org > - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
Java Broker - creating and closing too many connections causes memory leak?
Hi,
I ran the following test a few times and I ran into a OOM exception. Can
someone please tell me if i'm doing something wrong or if this could
potentially be a bug? Thanks.
1) Create 1000 queues.
2) Create 4 consumers -> each consumer uses it's own connection
3) In a loop do the following,
for each queues 1..1000 {
create producer(create new connection) for queue |i|
produce 1 message for that queue.
close producer (close connection)
}
4) Wait till all messages are consumed by the 4 consumers and then close all
4 consumers.
5) Re-Run the test without a broker restart
Each time the test ran, it created and closed 1004 connections (with at most
5 connections living in parallel...we close the earlier queue's producer's
connection before creating a producer for the next queue).
Also, I checked for any stray threads on the client side due to any unclosed
connection, and it looks like everything was closed fine (I didn't see any
stray threads).
On the broker side, I saw that the memory usage creeps up each time i re-run
the test, and eventually hits a OOM. (Running the above test about 5 times
causes a OOM )
I tried profiling the broker to see what's holding on to all that memory and
saw a certain class's(org.apache.qpid.transport.Method[]) instance count
grow up each time my test was run and it was holding the maximum memory in
the broker.
The number of instances of org.apache.qpid.transport.Method[] were trending
as follows:
After first test - 2008
After second test - 4016
... and so on.
There seems to be a direct relation between the number of those objects and
the connections created and closed. ~ Why are these objects not freed on
closing the connection??
I'm using the Java Broker out of trunk and using it with a DerbyMessageStore
for the test.
Or is there something fundamentally wrong by the way I run the test, I guess
opening and closing so many connections isn't a great idea (maybe I should
be dealing with sessions). But still, I see that this can be a potential
problem in a really long running broker system(where we cannot afford to
restart the broker periodically) which can have clients joining/leaving at
regular intervals.
Thank you for all the effort answering our questions,
--
-Praveen
Re: Downloading the Built version of qpidc-0.12
On 10/26/2011 09:55 PM, Daniel Mounessa wrote: Is there a built version for Linux that I could download and use or do I need to download the source code and rebuild the qpidd? What distribution of linux are you using? For Fedora you should be able to get qpid via yum. Cajus Pollmeier has produced debian squeeze packages, available at http://apt.gonicus.de/. Other than that you will need to build your own. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:users-subscr...@qpid.apache.org
