Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/10058 )
Change subject: IMPALA-6847: work around high memory estimates for AC ...................................................................... Patch Set 12: > > (3 comments) > > > I also really like the ideas in patch 8 and looks like a good > > direction to steer our solution towards in IMPALA-6460. It was > > essentially decoupling the 2 functions performed by mem_limit ( > 1. > > to enforce a runtime limit and 2. to be used in admission control > > for admitting and accounting of memory) and would have made > impala > > more stable, but the fact that a query option was used to > override > > mem_limit's functionality and had overlapping functionality with > > mem_estimate (acting as an upper limit for it) was making it > > confusing. > > Just for historical context, the current way we use MEM_LIMIT as > the "reservation" for admission control (i.e. #2) was put in place > as a placeholder until the backend actually implements real memory > reservations. Now that we have real memory reservations for the > most hungry operators, we want to leverage that (with additional > heuristics) rather than MEM_LIMIT for admission. But without > memory reservations in the backend, MEM_LIMIT seems the most > conservative approach for stability. Makes sense! Thanks for the context -- To view, visit http://gerrit.cloudera.org:8080/10058 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia5fc32a507ad0f00f564dfe4f954a829ac55d14e Gerrit-Change-Number: 10058 Gerrit-PatchSet: 12 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 17 Apr 2018 18:12:07 +0000 Gerrit-HasComments: No
