[
https://issues.apache.org/jira/browse/IMPALA-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655903#comment-16655903
]
ASF subversion and git services commented on IMPALA-7689:
---------------------------------------------------------
Commit 5af5456a2d95a43ce63f4e364ff0b9631729bb1a in impala's branch
refs/heads/master from Bharath Vissapragada
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=5af5456 ]
IMPALA-7689: Reduce per column per partition stats estimate size
With the improvements in the incremental stats memory representation
(IMPALA-7424), the per column per partition stats estimate should be
reduced to account for the compressed memory footprint. Doing some
experiments on various test tables, I see the size is down by 50-70%.
This patch reduces the size estimate by 50% (conservative). Ideally we
don't need to estimate on the Catalog server during serialization since
we can compute the byte sizes by looping through all the partitions.
However this patch retains the current logic to keep it consistent with
"compute incremental stats" analysis.
Change-Id: I347b41d9b298d7cd73ec812692172e0511415eee
Reviewed-on: http://gerrit.cloudera.org:8080/11706
Reviewed-by: Bharath Vissapragada <bhara...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Improve size estimate for incremental stats
> -------------------------------------------
>
> Key: IMPALA-7689
> URL: https://issues.apache.org/jira/browse/IMPALA-7689
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 3.1.0
> Reporter: Vuk Ercegovac
> Assignee: bharath v
> Priority: Major
>
> After compressing incremental stats, their size estimate is not too
> conservative. We use that size estimate to block the functionality (see the
> corresponding expr in analysis and serialization in catalogd), so without
> adjusting the estimate, the functionality will be blocked unnecessarily.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
