andrewglowacki commented on issue #2331: URL: https://github.com/apache/accumulo/issues/2331#issuecomment-957004905
@dlmarion exactly there are many flavors of OOM error. @ctubbsii OOM errors are recoverable in certain situations. 1. The one I see regularly is the 'direct memory' flavor. This one occurs when you try to allocate a direct byte buffer, but you've reached your pre-configured direct memory limit. This happens because direct memory is only freed after DBBs are GC'd. This is unlike normal resources where you can call close() and the resource is freed immediately. DBBs are used in Hadoop (and Thrift apparently) to improve I/O performance. Therefore, as long as you wait a little while, direct memory OOM errors will eventually go away and they don't have anything to do with normal JVM memory. 2. The other case is less exact but I still encounter it frequently. Suppose you have N threads each executing a high memory consuming operation local to the thread. Then suppose you run out of memory or reach the GC overhead limit in one or more of them, provoking a OOM error. In this situation, once the thread is terminated, the GC can run, and the JVM will have plenty of memory free again. Either way, it doesn't matter who caused the OOM error. It is up to the library maintainer to ensure resources are properly freed *in all cases*. The JVM itself does this with the try-with-resources construct. It doesn't stop to verify whether or not the exception that's unrolling the stack is a Error. It just closes the resource it's managing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
