Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/14037 )
Change subject: IMPALA-8839: Remove COLUMN_STATS_ACCURATE from properties ...................................................................... Patch Set 5: (2 comments) submit patch set 6 http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/14037/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3762 PS3, Line 3762: table.getDb().getName(), table.getName()); : } : } > You could get the write id of INSERT the same way as we get the transaction I got the conclusion of the same writeid by testing as the following: This list the write ID in partitions table after 1. Hive did compute stats for the whole table (analyze table insertonly_part_colstats compute statistics for columns;) . 2. Impala insert a row with column stats accurate remove. 3. Hive compute stats for the whole table again. You can see the write ID in partitions increased only 1 each time which means no waste of write id number. And it consists with hive's insert statement. See the last select on partitions(it is after hive insert a row to the 2010-01-01 partiton: HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274; PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID | WRITE_ID ---------+-------------+------------------+---------------+-------+--------+---------- 15991 | 1565289739 | 0 | ds=2010-01-01 | 19250 | 3274 | 18 15992 | 1565289739 | 0 | ds=2010-01-02 | 19251 | 3274 | 18 (2 rows) HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274; PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID | WRITE_ID ---------+-------------+------------------+---------------+-------+--------+---------- 15991 | 1565289739 | 0 | ds=2010-01-01 | 19250 | 3274 | 19 15992 | 1565289739 | 0 | ds=2010-01-02 | 19251 | 3274 | 18 (2 rows) HMS_home_yongzhi_Impala_cdp=> select * from "PARTITION_PARAMS" where "PART_ID"=15991; PART_ID | PARAM_KEY | PARAM_VALUE ---------+-----------------------+------------- 15991 | transient_lastDdlTime | 1565633500 15991 | numFiles | 4 15991 | totalSize | 8 15991 | numRows | 4 15991 | rawDataSize | 4 (5 rows) HMS_home_yongzhi_Impala_cdp=> select * from "TBLS" where "TBL_NAME"='insertonly_part_colstats'; HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274; PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID | WRITE_ID ---------+-------------+------------------+---------------+-------+--------+---------- 15991 | 1565289739 | 0 | ds=2010-01-01 | 19250 | 3274 | 20 15992 | 1565289739 | 0 | ds=2010-01-02 | 19251 | 3274 | 20 (2 rows) HMS_home_yongzhi_Impala_cdp=> select * from "PARTITIONS" where "TBL_ID"=3274; PART_ID | CREATE_TIME | LAST_ACCESS_TIME | PART_NAME | SD_ID | TBL_ID | WRITE_ID ---------+-------------+------------------+---------------+-------+--------+---------- 15991 | 1565289739 | 0 | ds=2010-01-01 | 19250 | 3274 | 21 15992 | 1565289739 | 0 | ds=2010-01-02 | 19251 | 3274 | 20 (2 rows) http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/14037/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3718 PS5, Line 3718: if (update.isSetTransaction_id()) { : transactionId = update.getTransaction_id(); : } > nit: fits single line Done -- To view, visit http://gerrit.cloudera.org:8080/14037 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I13f4a77022a7112e10a07314359f927eae083deb Gerrit-Change-Number: 14037 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen <yc...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Yongzhi Chen <yc...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Mon, 12 Aug 2019 18:43:52 +0000 Gerrit-HasComments: Yes
