Hello Internal Jenkins, Todd Lipcon, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/2659
to look at the new patch set (#2).
Change subject: IMPALA-3238: avoid log spam for very large hash tables
......................................................................
IMPALA-3238: avoid log spam for very large hash tables
Before this change, every time the limit was hit in ConsumeMemory() was
hit, a warning message with stack trace was logged. The streaming preagg
calls this once per partition per batch, resulting in an enormous volume
of logging.
The fix is to log the warning only once for each client (i.e. each exec
node).
We need a very large data set to reproduce this error. To test it
manually I lowered the threshold to 1 << 19, ran 'select distinct
l_orderkey from tpch_20_parquet', and confirmed that only two warning
messages per query appeared in the logs (one per agg node).
Change-Id: I366ce1148e83ac2eb71535d552bc68cb08388eec
---
M be/src/runtime/buffered-block-mgr.cc
1 file changed, 13 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/59/2659/2
--
To view, visit http://gerrit.cloudera.org:8080/2659
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I366ce1148e83ac2eb71535d552bc68cb08388eec
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Henry Robinson <[email protected]>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
