Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10988 )
Change subject: IMPALA-7297: soft memory limit for reservation increases ...................................................................... Patch Set 5: I think you have it right. The 80/20 heuristic has been around since Impala 2.0. I don't think there's anything special about the numbers but it's proven difficult to change without negatively impacting some queries (either worse performance or higher memory requirements) One additional bit of context is that long-term we want to move most or all execution memory under reservations, so this is a bit of a stopgap. I also have a follow-on patch that makes HDFS + Kudu scans check the soft limit to decide whether to start or stop scanner threads. HDFS scans use a mix of reserved and non-reserved memory. Kudu uses all non-reserved memory. So that's a case where the alternative of reducing the reserved memory doesn't help us. -- To view, visit http://gerrit.cloudera.org:8080/10988 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I39dcf2dd870a59c8b8cda4488fe41ce936d715ac Gerrit-Change-Number: 10988 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Comment-Date: Thu, 26 Jul 2018 22:59:47 +0000 Gerrit-HasComments: No
