I didn't. I'd appreciate if you could do that, as you have all the details.

http://issues.apache.org/cayenne/

Thanks
Andrus


On Mar 15, 2007, at 3:45 PM, Ayhan Kondoz wrote:

Hi Andrus,

did you log this as a bug? And do you need anything else from me?

At the moment I added a cronjob to restart the WebService every night which prevents this bug from causing any harm but I really would like to fix it properly and remove the cronjob.

Thanx
Ayhan Kondoz

-----Ursprüngliche Nachricht-----
Von: Andrus Adamchik [mailto:[EMAIL PROTECTED]
Gesendet: Mittwoch, 21. Februar 2007 15:20
An: [email protected]
Betreff: Re: AW: AW: possible bug / memory leak in DispatchQueue and
EventManager?

30 is good. If you had a retained DataContext somewhere you'd get the
same number as the number of stuck invocations.

1 million of Invocations is bad.

I guess this needs to be logged as a possible bug to investigate,
with as many details as possible on the JVM version, etc.

Andrus


On Feb 21, 2007, at 4:07 PM, Ayhan Kondoz wrote:
Yes seems live there are DataContext instances.

http://img73.imageshack.us/img73/3068/memta6.jpg

I started the GC a few times before I took this screenshot.
As you can see there are still 30 DataContext instances. No idea
why thought...


-----Ursprüngliche Nachricht-----
Von: Andrus Adamchik [mailto:[EMAIL PROTECTED]
Gesendet: Mittwoch, 21. Februar 2007 14:49
An: [email protected]
Betreff: Re: AW: possible bug / memory leak in DispatchQueue and
EventManager?

Do you see DataContext instances in the memory profile? I wonder how many of those do you have, as those are the only Cayenne objects that
have hard reference to the listeners.

Andrus


On Feb 21, 2007, at 3:37 PM, Ayhan Kondoz wrote:

Hi,

the JVM has 1 GB of memory and i already tried to run the GC
manually but it doesn't make a difference. I can start the GC from
the profiler tool but the number of instances does not change.
I guess that there are some other references to this objects so
that the weak ref. does not expire.

And there are no EventManager exception in the log files.


ayhan

-----Ursprüngliche Nachricht-----
Von: Andrus Adamchik [mailto:[EMAIL PROTECTED]
Gesendet: Mittwoch, 21. Februar 2007 14:17
An: [email protected]
Betreff: Re: possible bug / memory leak in DispatchQueue and
EventManager?

Seems like your assessment of the EventManager leaking is correct.

Now the cause is not that clear. A shot in the dark - this is
due to
a combination of lots of spare memory in JVM (so weak references
are
not collected fast enough) and slow custom 'equals' and 'hashCode'
methods in invocation.

How much heap size do you have in your server? Also can you try to
add a request filter that would do 'System.gc()' on every few
thousands requests, and see if it makes any difference.

Also - do you see any EventManager exceptions in the logs?

Andrus


On Feb 21, 2007, at 10:55 AM, Ayhan Kondoz wrote:
Hi,



i think i found a possible bug / memory leak within the
DispatchQueue and EventManager.





First a little bit about my setup:



I have 3 servers. Each server runs an axis service. The service
uses cayenne 1.2.1 to connect to a database. It reads customer and
account information from the DB etc..

The servers are using cayenne's shared caching with javagroups as
the messaging service so that changes made from one server are
dispatched to the other servers.



The avarage connections per second is somewhere around 4-5.



However I have a very strange problem with this setup so I startet to search for the reason. The problem is that with nearly constant and unchanging usage the load of each server increases over time. To further test this I created a test server with a similar setup.
There I created a test program that creates totally constant
usage.
But even with the unchanging usage the load of the server is
increasing until the cpu load is so high that the requests can not
be processed anymore.



I installed a java profiler to trying to pinpoint the location of
this error and this is what I found out.



I let the server run for 24 hours and then stopped the program
which creates the test usage.

But even while the server was idle there where still a lot of
instances in the java heap after the GC run.



http://img206.imageshack.us/img206/9769/memorysy5.jpg



Please note the HashMap, WeakReference and Invocation counts. I
pressume that the $ObjectProvider_*** is cayenne aswell but I am
not sure.



Now the following image shows the cpu profile with incoming
connections.



http://img339.imageshack.us/img339/2441/cpuci0.jpg



As you can see 58% of the cpu time is used within HashSet.add().



So when I consider the two facts i think that there might be a
possible problem with the EventManager. The first table tells us
that there are over 1 million instances of HashSet's and cayenne
Invocations. So it seems like the set's within the DispatchQueue
are not recycled properly so that the object count rises over time
which result in extremly high processtime when trying to add new
HashSet's.





Thanks

Ayhan







Ayhan Kondoz



Software-Entwicklung



---------------------------------------------------------------- --
--
--
------------

Telefon:    +49 (0) 40 513 06 616

Telefax:    +49 (0) 40 513 06 998 616

E-Mail:     [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]
ag.de>

Website: http://www.freenet.de <http://www.freenet.de/> ; http://
www.freenet-ag.de <http://www.freenet-ag.de/>

---------------------------------------------------------------- --
--
--
------------

freenet.de AG

Deelbögenkamp 4c

22297 Hamburg

---------------------------------------------------------------- --
--
--
------------

Vorsitzender des Aufsichtsrates: Prof. Dr. Helmut Thoma

Vorstand: Eckhard Spoerr (Vors.),
Axel Krieger, Stephan Esch, Eric Berger

Amtsgericht Hamburg HRB 74048





Diese Information ist ausschließlich für die adressierte Person
oder Organisation bestimmt und könnte vertrauliches und/oder
privilegiertes Material enthalten. Personen oder Organisationen,
für die diese Information nicht bestimmt ist, ist es nicht
gestattet, diese zu lesen, erneut zu übertragen, zu verbreiten,
anderweitig zu verwenden oder sich durch sie veranlasst zu sehen,
Maßnahmen irgendeiner Art zu ergreifen. Sollten Sie diese
Nachricht
irrtümlich erhalten haben, bitten wir Sie, sich mit dem
Absender in
Verbindung zu setzen und das Material von Ihrem Computer zu
löschen.



The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential
and/or
privileged material. Any review, retransmission, dissemination or
other use of, or taking of any action in reliance upon, this
information by persons or entities other than the intended
recipient is prohibited. If you received this in error, please
contact the sender and delete the material from any computer.










Reply via email to