Tim Armstrong has uploaded a new change for review. http://gerrit.cloudera.org:8080/2659
Change subject: IMPALA-3238: avoid log spam for very large hash tables ...................................................................... IMPALA-3238: avoid log spam for very large hash tables Before this change, every time the limit was hit in ConsumeMemory() was hit, a warning message with stack trace was logged. The streaming preagg calls this once per partition per batch, resulting in an enormous volume of logging. The fix is to log the warning only once for each client (i.e. each exec node). We need a very large data set to reproduce this error. To test it manually I lowered the threshold to 1 << 19, ran 'select distinct l_orderkey from tpch_20_parquet', and confirmed that only two warning messages per query appeared in the logs (one per agg node). Change-Id: I366ce1148e83ac2eb71535d552bc68cb08388eec --- M be/src/runtime/buffered-block-mgr.cc 1 file changed, 13 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/59/2659/1 -- To view, visit http://gerrit.cloudera.org:8080/2659 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I366ce1148e83ac2eb71535d552bc68cb08388eec Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Tim Armstrong <[email protected]>
