Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21484 )
Change subject: IMPALA-12800: Add null slots cache ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/21484/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21484/3//COMMIT_MSG@18 PS3, Line 18: Adds 'use_null_slots_cache' query option to disable caching. Documents : the new option. > Any downside to control this via backend flag instead? I think we want this My thinking was queries fall into 3 categories: 1. hand-written query with small number of expressions, so this cache will be pretty small 2. computer-generated query with large number of very similar expressions, where this cache helps a lot 3. query with lots of unique expressions, where this cache could use a lot of extra memory (preserving the copied expressions) with no benefit I expect 3 to be pretty rare, but could happen, so I wanted a knob to be able to turn off for specific queries. http://gerrit.cloudera.org:8080/#/c/21484/3/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/21484/3/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@541 PS3, Line 541: // Null slots cache - for expressions with slots replaced by nulls - to reduce : // backend expression evaluation when query contains many similar expressions. : private final Map<Expr, Boolean> isTrueWithNullSlotsCache = new HashMap<>(); : : // Track null slots cache hits. : private int isTrueWithNullSlotsCacheHits = 0; : : // Track null slots cache misses. : private int isTrueWithNullSlotsCacheMisses = 0; > It is probably worth to contain these into its own class. Sure, I can wrap this up in a private class. I thought about mirroring https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L383-L390, but that seemed excessive. -- To view, visit http://gerrit.cloudera.org:8080/21484 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib63f5553284f21f775d2097b6c5d6bbb63699acd Gerrit-Change-Number: 21484 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Thu, 06 Jun 2024 21:01:54 +0000 Gerrit-HasComments: Yes
