Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/7999 )
Change subject: [DOCS] Tighten up advice about first COMPUTE INCREMENTAL STATS ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/7999/2/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/7999/2/docs/shared/impala_common.xml@1244 PS2, Line 1244: 2 GB, a serious error can occur. If only a limited number of partitions are actively being > Done. "Serious error" was my compromise I always used for MySQL, where the Sorry I just realized the ask here. Yea we do need some numbers like #partitions #files and #blocks to come up with an estimate of a particular table's contribution to the memory footprint. Roughly something of the order of #partitions * 2kB + #files * 750B + # file_blocks * 300B The problem is that there is no easy way a user can get these numbers for a given table. Similarly for incremental stats we can use the HMS dump to get an estimate (or directly connect to the HMS db) select sum(length(PARTITION_PARAMS.PARAM_KEY) + length(PARTITION_PARAMS.PARAM_VALUE)), PARTITIONS.TBL_ID from PARTITIONS, PARTITION_PARAMS where PARTITIONS.PART_ID = PARTITION_PARAMS.PART_ID and PARTITION_PARAMS.PARAM_KEY LIKE 'impala_intermediate%' group by PARTITIONS.TBL_ID; But in most serious setups, end users can't do this. IMPALA-4870 aims to expose most of this data (WIP). May be we can revisit this doc once it is done. -- To view, visit http://gerrit.cloudera.org:8080/7999 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia53a6518ce5541e5c9a2cd896856ce042a599b03 Gerrit-Change-Number: 7999 Gerrit-PatchSet: 2 Gerrit-Owner: John Russell <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Greg Rahn <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Russell <[email protected]> Gerrit-Reviewer: Mostafa Mokhtar <[email protected]> Gerrit-Reviewer: Silvius Rus <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Fri, 06 Oct 2017 23:46:04 +0000 Gerrit-HasComments: Yes
