Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/958#discussion_r140939480
--- Diff:
exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java ---
@@ -22,16 +22,55 @@
import javax.annotation.concurrent.ThreadSafe;
import org.apache.drill.exec.exception.OutOfMemoryException;
+import org.apache.drill.exec.util.AssertionUtil;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
/**
* Provides a concurrent way to manage account for memory usage without
locking. Used as basis for Allocators. All
* operations are threadsafe (except for close).
*/
@ThreadSafe
-class Accountant implements AutoCloseable {
- // private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(Accountant.class);
+@VisibleForTesting
+public class Accountant implements AutoCloseable {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(Accountant.class);
+
+ // See DRILL-5808
+ // Allow a "grace margin" above the allocator limit for those operators
+ // that are trying to stay within the limit, but are hindered by the
+ // current power-of-two allocations and random vector doublings. Instead
+ // of failing the query, the accountant allows an excess allocation
within
+ // the grace, but issues a warning to the log.
+ //
+ // Grace is allowed only when requested, and then only in production
+ // code, or if enabled in debug code by setting the following system
+ // property.
+
+ public static final String STRICT_ALLOCATOR =
"drill.memory.strict.allocator";
+
+ // Allow clients to allocate beyond the limit by this factor.
+ // If the limit is 10K, then with a margin of 1, the system
+ // will allow a grace of another 10K for a hard limit of 20K.
+ // Warnings are issued for allocations that use the grace.
+
+ public static final int GRACE_MARGIN = 1;
+
+ // In large-memory allocators, don't allow more than 100 MB
+ // grace.
+
+ public static final int MAX_GRACE = 100 * 1024 * 1024;
+
+ // Use the strictness set in a system property. If not set, then
+ // default to strict in debug mode, lenient in production mode.
+
+ public static final boolean STRICT =
System.getProperty(STRICT_ALLOCATOR) == null
--- End diff --
This is a constant, hence the caps. It may be a bit confusing since it is
set as an expression, but that is perfectly valid Java.
---
