abstractdog commented on a change in pull request #2812:
URL: https://github.com/apache/hive/pull/2812#discussion_r758468637
##########
File path:
llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskCommunicator.java
##########
@@ -1163,4 +1165,15 @@ private QueryIdentifierProto
constructQueryIdentifierProto(int dagIdentifier) {
public String getAmHostString() {
return amHost;
}
+
+ /**
+ * Overrides TezTaskCommunicatorImpl.getTotalUsedMemory in order to provide
correct aggregated memory usage.
+ * In LLAP, every container reports the whole used heap of the daemon
they're running in, so we need to consider
+ * every usedMemory once per daemon.
+ * @return
+ */
+ @Override
+ public long getTotalUsedMemory() {
+ return pingedNodeMap.values().stream().mapToLong(c -> c.usedMemory).sum();
Review comment:
thanks for the comments @pgaref
```
Sum does not provide much value here
```
I see, the sum is just one information (we need at least the number of
daemons additionally to make it have a meaning), but the percentage looks more
useful
I was already thinking about percentage, what should be the value of 100%? I
mean, usedMemory is heap usage in daemons, what do you think we should consider
as the "whole" memory?
I found adjustedExecutorMemory which would make sense:
https://github.com/apache/hive/blob/1046f41ea36ab3c8b036481128ba9b76dda2882a/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java#L227
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
