dlmarion opened a new pull request, #3161: URL: https://github.com/apache/accumulo/pull/3161
Accumulo has had a [GarbageCollectionLogger](https://github.com/apache/accumulo/blob/main/server/base/src/main/java/org/apache/accumulo/server/GarbageCollectionLogger.java) for a long time. Currently it runs every 5 seconds in the server processes and logs a message that memory is low when the amount of free memory is less than 5% of the heap. This PR modifies the GarbageCollectionLogger to optionally take a more active role. This PR: 1. Renames the GarbageCollectionLogger to LowMemoryDetector. 2. Adds a volatile boolean to the LowMemoryDetector that it sets to true when it detects that memory is low and it's not operating in a passive manner. 3. Adds new properties for the server processes to set whether the LowMemoryDetector is active or passive, it's check interval, and the free memory threshold. 4. Modifies the `lookup` and `nextBatch` methods in `TabletBase` such that: a. if low memory at the entrance to the method, it waits until that condition no longer exists b. if low memory while aggregating results, it returns the batch of results early (similar to time limit or batch limit being exceeded) 5. Modified the LowMemoryDetector so that iterators that aggregate KV's or create a lot of garbage can use this information to return early if possible. a. Added `isRunningLowOnMemory()` to SKVI interface, default implementation returns `false` b. Modiifed WrappingIterator to implement `isRunningLowOnMemory()` using the LowMemoryDetector. 6. Added two new metrics that are incremented when the start of a scan is paused or when the scan returns early due to low memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
