zhangqianqiong has posted comments on this change. ( http://gerrit.cloudera.org:8080/21845 )
Change subject: IMPALA-13385: Compact the response of ddl operations ...................................................................... Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/21845/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21845/3//COMMIT_MSG@9 PS3, Line 9: present > nit: present Done http://gerrit.cloudera.org:8080/#/c/21845/3//COMMIT_MSG@20 PS3, Line 20: In an experiment with a hive : table containing 160 partitions, I observed that using TCompactProtocol : can reduce the serialized data size by 34.4% compared to the previous : method. > We have a test scenario described in https://github.com/apache/impala/blob/ I used the scale_test_metadata script to construct the 1k_col_tbl table, and the test results for this table are as follows: Before compaction, the serialized size was 408M, and after compaction, it was 369M, representing only a 9.5% reduction in data size. The primary reason for this minimal optimization is that nearly all columns in this table are of the string type, and TCompactProtocol does not optimize string types. Therefore, the optimization effect on the 1k_col_tbl table is not significant. http://gerrit.cloudera.org:8080/#/c/21845/1/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/21845/1/fe/src/main/java/org/apache/impala/service/JniCatalog.java@263 PS1, Line 263: private byte[] execAndSerializeWithCompaction(String methodName, > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/21845/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/21845/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java@263 PS3, Line 263: private byte[] execAndSerializeWithCompaction(String methodName, > line too long (91 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/21845 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idea9313c7f1f1596f3620e60b08a99efc7fa0466 Gerrit-Change-Number: 21845 Gerrit-PatchSet: 5 Gerrit-Owner: zhangqianqiong <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: zhangqianqiong <[email protected]> Gerrit-Comment-Date: Mon, 30 Sep 2024 10:46:01 +0000 Gerrit-HasComments: Yes
