zhangbutao commented on code in PR #4935:
URL: https://github.com/apache/hive/pull/4935#discussion_r1422399411
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java:
##########
@@ -71,7 +71,7 @@ public class SplitGrouper {
// TODO This needs to be looked at. Map of Map to Map... Made concurrent for
now since split generation
// can happen in parallel.
- private static final Map<Map<Path, PartitionDesc>, Map<Path, PartitionDesc>>
cache =
+ private final Map<Map<Path, PartitionDesc>, Map<Path, PartitionDesc>> cache =
Review Comment:
if removing static, multiple SplitGrouper instances will have each own
cache; I am afraid of this will consume much memory of LLAP coordinator (Not
sure i understand correctlly).
From the jstack, it seems that thread is blocked at
`org.apache.hadoop.hive.common.CopyOnFirstWriteProperties::equals`, which was
introduced to resolve high memory usage about `Properties `, HIVE-16079.
Maybe we can find more clues on `CopyOnFirstWriteProperties`?
Just my subjective thought, for your information.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
