Tim Ellison wrote:
Rana Dasgupta wrote:
On 2/5/07, Tim Ellison <[EMAIL PROTECTED]> wrote:
I don't see the distinction. There is still a 1:1 between the Java
object and the native resource. If you run out of file handles then it
is potentially as bad as running out of native memory, the disparity in
size between the Java resource and the native resource is immaterial
isn't it?
There is a distinction. One could directallocate only a few buffers and
exhaust the C heap ( if I understand the api correctly ) if the api
implementation is not eager about releasing back unused resources. One is
unlikely to hit a limit with a few outstanding fd's.
My point was that we could potentially run out of any OS resource being
held onto by a Java object before running out of Java heap for those
objects. Maybe file handles was a poor example.
I think the file handle mention was my fault, but it came from a
concrete example: Dacapo xalan in the 2006-10 release, which would fail
for exactly this reason if the heap was large enough.
I don't think that we need to expand the scope of the problem to do
all types of resource accounting in the JVM or in an underlying
platform layer. I don't see the spec providing any basic guidance in
this area, it falls in the category of failsafe JVM's which is an
interesting area, but IMHO beyond scope at this point. My suggestion
is that we treat this as a specific nio api implementation problem
for which we can think of adding some VM/GC support if it helps.
We may have to disagree here then, I think we should indeed be
considering a general solution for detecting early when these Java
'handle' objects are collectable -- then we can apply it to direct
buffers or other handle types.
My point was that by adding a single GC -> VM interface function, all
types of native resources could be addressed in a simple extensible way.
The subsequent policy and mechanism decisions then become an ongoing
design process for the VM and classlib native implementers, but once
this interface function is added, GC can support freeing of native
resources by whatever mechanisms the VM chooses to implement.
The extent of the support is based on how widely this fails in real
usage scenarios. After all the doc seems to say that the directbuffer
is best chosen for long lived large buffers only, those that will
need infrequent collection, implying weak GC dependance.
Yep, while we have a test case that demonstrates the difference between
Harmony and other implementations, a failing application would be more
compelling.
One way to do this could be to treat these as a special class of Java
objects with all allocation on the Java heap, as Xiao Feng suggests, and
let GC decide on handling the resource exhaustion.
I suggest that would have it's own problems though. The memory would
have to be contiguous and pinned, and may be large. We could end up
with a fragmented heap quite quickly.
I strongly disagree with special-casing any of these resources. To me,
supporting pinned allocation for direct buffers is orthogonal to
providing a way for the VM to request GC when certain native resources
are at risk of depletion.
The second is for the api to ask the GC some specific questions about
unreachable object instances to eagerly decide on deallocation. That
requires a private interface.
right, which is an optimization of the suggestion of using the public
System.gc() API
Surely a well-written Java application releases native resources either
explicitly or through a reference type or finalizer ? Yes, it's nice to
tolerate badly written apps but is that the first-order concern here ?
The third is for the api implementation to do most of the
housekeeping ( hence my suggestion about max memory ) without relying
on platform hooks, and heuristically decide on when to invoke GC.
This will fail sometimes, but that may be OK. The solution can be
made more efficient ( than finalizers etc. ) by making the interface
private.
What heuristic would you suggest beyond the malloc failure?
% (native)heap usage, with some kind of 'damping' heuristic to prevent
thrashing.
The last would be fore the api to add platform hooks to help with its
housekeeping. But I don't believe that this is possible for this
solution to be perfect( imagine several VM instances running on a
client machine ), and forces a dependency on a third platform
dependant piece, if it is to be kept independent of classlib and VM.
Which 'platform hooks' do you mean? (not sure if you meant they are in
Harmony at the moment or are speaking hypothetically).
Regards,
Tim
--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/
