Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10058 )
Change subject: IMPALA-6847: work around high memory estimates for AC ...................................................................... Patch Set 12: > (3 comments) > I also really like the ideas in patch 8 and looks like a good > direction to steer our solution towards in IMPALA-6460. It was > essentially decoupling the 2 functions performed by mem_limit ( 1. > to enforce a runtime limit and 2. to be used in admission control > for admitting and accounting of memory) and would have made impala > more stable, but the fact that a query option was used to override > mem_limit's functionality and had overlapping functionality with > mem_estimate (acting as an upper limit for it) was making it > confusing. Just for historical context, the current way we use MEM_LIMIT as the "reservation" for admission control (i.e. #2) was put in place as a placeholder until the backend actually implements real memory reservations. Now that we have real memory reservations for the most hungry operators, we want to leverage that (with additional heuristics) rather than MEM_LIMIT for admission. But without memory reservations in the backend, MEM_LIMIT seems the most conservative approach for stability. -- To view, visit http://gerrit.cloudera.org:8080/10058 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10058 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 17 Apr 2018 16:58:14 +0000 Gerrit-HasComments: No
