Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17132 )
Change subject: IMPALA-10492: Lower default MAX_CNF_EXPRS query option ...................................................................... IMPALA-10492: Lower default MAX_CNF_EXPRS query option MAX_CNF_EXPRS was set to unlimited by default. The CNF rewrite can lead to significant frontend memory usage and eventually OutOfMemory for a complex query that contain many predicates. We need to lower the default value to avoid this memory problem while maintaining performance for our TPC-DS and TPC-H workloads. We investigate the maximum number of CNF expressions in TPC-DS and TPC-H by printing out the final value of 'numCnfExprs_' from ConvertToCNFRule.java to the query profile. We found 5 queries that applies CNF rewrite rules as follow: | Query | numCnfExprs_ | |-----------+--------------| | TPCDS-Q13 | 168 | | TPCDS-Q85 | 100 | | TPCDS-Q48 | 34 | | TPCH-Q19 | 124 | | TPCH-Q7 | 3 | This patch lower the default value from unlimited to 200 based on the result above. Testing: - Manually verify that MAX_CNF_EXPRS 200 is enough for our TPC-DS and TPC-H worloads. - Pass core tests. Change-Id: I7ca3d0e094ac01c24a046c25d6a1b56bf134faa8 Reviewed-on: http://gerrit.cloudera.org:8080/17132 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift 2 files changed, 2 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7ca3d0e094ac01c24a046c25d6a1b56bf134faa8 Gerrit-Change-Number: 17132 Gerrit-PatchSet: 5 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: David Rorke <dro...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
