Did you use SocketFactory ?
http://river.apache.org/user-guide-socketfactories.html
Gr. Sim
On 13-01-12 13:28, Bryan Thompson wrote:
Peter,
The DGC Threads are in the server JVM. They are nearly all for the exported
Futures.
Thanks,
Bryan
-----Original Message-----
From: Peter Jones [mailto:[email protected]]
Sent: Friday, January 13, 2012 12:04 AM
To: [email protected]
Cc: [email protected]
Subject: Re: DGC threads issue
Bryan,
DGC threads should not be per exported object. Generally
speaking, they tend to be per endpoint (at which there are
one or more remote objects exported). Are you using any sort
of custom endpoint implementation? Problems like this can
occur when an endpoint implementation doesn't implement
Object.equals and hashCode appropriately, so the expected
batching of threads (and communication) per endpoint does not occur.
It might help to see, from a thread dump, exactly which DGC
threads are causing this problem. And they are in the server
JVM (with all the exported remote objects), not the remote
callers' JVM(s)?
-- Peter
On Jan 12, 2012, at 3:45 PM, Tom Hobbs wrote:
Hi Bryan,
Sorry that no one got back to you about this. I'm afraid
that I don't
know the answer to your question, I've copied the dev list
into this
email in case someone who monitors that list (but not this one) has
any ideas.
Best regards,
Tom
On Thu, Jan 12, 2012 at 2:29 PM, Bryan Thompson
<[email protected]> wrote:
Just to follow up on this thread myself. I modified the
pattern to return a "thick" future rather than a proxy for
the future. This caused the RMI call to wait on the server
until the future was done and then sent back the outcome.
This "fixed" the DGC memory/thread leak by reducing the
number of exported proxies drammatically.
In terms of best practices, is distributed DGC simply not
useful for exported objects with short life spans? Can it
only be used with proxies for relatively long lived services?
Thanks,
Bryan
-----Original Message-----
From: Bryan Thompson
Sent: Tuesday, January 03, 2012 12:06 PM
To: [email protected]
Subject: DGC threads issue
Hello,
Background:
I am seeing what would appear to be one DGC thread allocated per
exported object. This is using River 2.2 and Sun JDK 1.6.0_17.
Relevant configuration parameters are below.
I am observing problems with the DGC threads not being
retired on a
timely basis. The exported objects are proxies for Futures which
are being executed on the service. The code pattern is such that
the proxied Future goes out of lexical scope quite
quickly. E.g.,
rmiCallReturningProxyForFuture().get().
Under a modest load, a large number of such Futures are exported
which results in a large number of long lived DGC threads. This
turns into a problem for the JVM due to the stack allocation per
thread. Presumably this is not good for other reasons as well
(e.g., scheduling).
I have tried to override the leaseValue and checkInterval
defaults
per the configuration options below. I suspect that the lease
interval is somehow not being obeyed, which is presumably
a problem
on my end. However, I can verify that the configuration
values are
in fact showing up in
System.getProperties() for at least some of the JVMs
involved (the
one which drives the workload and the one that I am
monitoring with
the large number of DGC lease threads).
Some questions:
Is this one-thread-per-exported proxy the expected
behavior when DGC
is requested for the exported object?
The DGC lease checker threads appear to expire ~14 - 15 minutes
after I terminate the process which was originating the RMI
requests. This is close the sum of the default
leaseValue (10m) and
checkInterval (5m) parameters, but maybe there is some
other timeout
which is controlling this? If this is the sum of those
parameters,
why would the DGC lease threads live until the sum of
those values?
I thought that the lease would expire after the leaseValue (10m
default).
Can the issue I am observing be caused by a low heap
pressure on the
JVM to which the RMI proxies were exported? If it fails
to GC those
proxies, even though they are reachable, could that cause DGC to
continue to retain those proxies on the JVM which exported them?
Is there any way to configure DGC to use a thread pool or to have
the leases managed by a single thread?
Is it possible that there is an interaction with the
useNIO option?
Relevant options that I am using include:
-Dcom.sun.jini.jeri.tcp.useNIO=true
-Djava.rmi.dgc.leaseValue=30000
-Dsun.rmi.dgc.checkInterval=15000
-Dsun.rmi.transport.tcp.connectionPool=true
Thanks in advance,
Bryan
--
QCG, Software voor het MKB, 071-5890970, http://www.qcg.nl
Quality Consultancy Group b.v., Leiderdorp, Kvk Den Haag: 28088397
