szlta commented on a change in pull request #1951:
URL: https://github.com/apache/hive/pull/1951#discussion_r571973578
##########
File path:
llap-server/src/java/org/apache/hadoop/hive/llap/cache/LowLevelLrfuCachePolicy.java
##########
@@ -836,13 +912,33 @@ public String description() {
* @return long array with LRFU stats
*/
public long[] getUsageStats() {
- long dataOnHeap = 0L; // all non-meta related buffers on min-heap
- long dataOnList = 0L; // all non-meta related buffers on eviction list
- long metaOnHeap = 0L; // meta data buffers on min-heap
- long metaOnList = 0L; // meta data buffers on eviction list
- long listSize = 0L; // number of entries on eviction list
- long lockedData = 0L; // number of bytes in locked data buffers
- long lockedMeta = 0L; // number of bytes in locked metadata buffers
+ long dataOnHeap = 0L; // all non-meta related buffers on min-heap
+ long dataOnList = 0L; // all non-meta related buffers on eviction
list
+ long metaOnHeap = 0L; // meta data buffers on min-heap
+ long metaOnList = 0L; // meta data buffers on eviction list
+ long listSize = 0L; // number of entries on eviction list
+ long lockedData = 0L; // number of bytes in locked data buffers
+ long lockedMeta = 0L; // number of bytes in locked metadata buffers
+ long bpWrapCount = 0L; // number of buffers in BP wrapper
threadlocals
+ long bpWrapDistinct = 0L; // number of distinct buffers in BP wrapper
threadlocals
+ long bpWrapData = 0L; // number of bytes stored in BP wrapper data
buffers
+ long bpWrapMeta = 0L; // number of bytes stored in BP wrapper
metadata buffers
+
+ // Using set to produce result of distinct buffers only
+ // (same buffer may be present in multiple thread local bp wrappers, or
even inside heap/list, but ultimately
+ // it uses the same cache space)
+ Set<LlapCacheableBuffer> bpWrapperBuffers = new HashSet<>();
+ for (BPWrapper bpWrapper : bpWrappers.values()) {
+ bpWrapper.lock.lock();
Review comment:
It is only called when someone (e.g. cluster admin) asks for memory
stats through web UI. Hence it is okay to block all structures for these cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
