ctubbsii commented on issue #1689: URL: https://github.com/apache/accumulo/issues/1689#issuecomment-696407230
I'm pretty confident in saying that I don't think this is an Accumulo memory manager bug at all. Perhaps this is a hardware bug (it seems like some sort of memory/storage corruption). Perhaps it can be mitigated within the classloader, through detection of rapid change events? However, given the direction we're headed regarding the classloader changes (see mailing list for discussion about that), I'm not sure what changes to the existing classloader, if any, would be worth it at this time. Related issue: we should really stop trying to catch `Throwable`... and limit ourselves to catching `Exception`, at most (preferably more narrow than that, if possible). In this code, and in lots of other places, it looks like we could have caught a `Throwable` (which includes `Error`s), assumed we could handle it, and possibly only logged (or failed to log) and moved on, when in fact, we have no chance of recovery (such as the case with `Error`s like `OutOfMemoryError`). According [to Oracle's troubleshooting guide](https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html), this may be thrown when there's not enough native memory to load a class.... which is a situation that seems like it could cause the reloading behavior you saw, as well as other memory corruption issues, if we didn't just exit immediately and instead try to "handle" it and continue after hitting an `OutOfMemoryError`. Were there low-memory issues on the machine(s) where you saw this occur? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
