The problem is that the memory is untracked so it won't be accounted against the query.
You can look at process RSS or the process memory consumption on the /memz to see the total memory consumed including untracked allocations. The last time I ran into something like this I used a script from stackoverflow to graph the memory consumption: https://issues.cloudera.org/browse/IMPALA-2940 http://stackoverflow.com/questions/7998302/graphing-a-processs-memory-usage On Wed, Mar 1, 2017 at 4:21 PM, Marcel Kornacker (Code Review) < [email protected]> wrote: > Marcel Kornacker has posted comments on this change. > > Change subject: IMPALA-4899: Fix parquet table writer dictionary leak > ...................................................................... > > > Patch Set 4: > > (1 comment) > > http://gerrit.cloudera.org:8080/#/c/6181/4/be/src/exec/hdfs-table-sink.h > File be/src/exec/hdfs-table-sink.h: > > Line 184: typedef std::pair<std::unique_ptr<OutputPartition>, > std::vector<int32_t>> PartitionPair; > > please include updates to the class comments (of the affected classes) > that > to expand on that: your commit message explains the intention behind the > unique_ptrs. however, that's not visible from the perspective of the next > person looking at this code, so it's better to have this directly in the > code. > > > -- > To view, visit http://gerrit.cloudera.org:8080/6181 > To unsubscribe, visit http://gerrit.cloudera.org:8080/settings > > Gerrit-MessageType: comment > Gerrit-Change-Id: I06e354086ad24071d4fbf823f25f5df23933688f > Gerrit-PatchSet: 4 > Gerrit-Project: Impala-ASF > Gerrit-Branch: master > Gerrit-Owner: Joe McDonnell <[email protected]> > Gerrit-Reviewer: Joe McDonnell <[email protected]> > Gerrit-Reviewer: Lars Volker <[email protected]> > Gerrit-Reviewer: Marcel Kornacker <[email protected]> > Gerrit-Reviewer: Tim Armstrong <[email protected]> > Gerrit-HasComments: Yes >
