Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11001 )
Change subject: IMPALA-7234: Improve memory estimates produced by the Planner ...................................................................... IMPALA-7234: Improve memory estimates produced by the Planner Previously, the planner used the getMajorityFormat to estimate the memory requirements of its partitions. Additionally, before IMPALA-6625 was merged, the majority format for a multi-format table with no numerical majority was calculated using a HashMap, thus producing non deterministic results. This change ensures that the memory estimate is deterministic and always based on partition that has the maximum memory requirement. Testing: Ran all PlannerTests. Also, modified plans of scans with multiple partitions to ensure that the memory estimate produced corresponds to the partition with the maximum requirement. Change-Id: I0666ae3d45fbd8615d3fa9a8626ebd29cf94fb4b Reviewed-on: http://gerrit.cloudera.org:8080/11001 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test 8 files changed, 70 insertions(+), 78 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11001 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0666ae3d45fbd8615d3fa9a8626ebd29cf94fb4b Gerrit-Change-Number: 11001 Gerrit-PatchSet: 10 Gerrit-Owner: Pooja Nilangekar <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Pooja Nilangekar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
