zhangqianqiong has posted comments on this change. ( http://gerrit.cloudera.org:8080/21845 )
Change subject: IMPALA-13385: Compact the response of ddl operations ...................................................................... Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/21845/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21845/3//COMMIT_MSG@20 PS3, Line 20: In an experiment with a hive : table containing 160 partitions, I observed that using TCompactProtocol : can reduce the serialized data size by 34.4% compared to the previous : method. > I used the scale_test_metadata script to construct the 1k_col_tbl table, an I conducted another test. Since all data columns in the 1k_col_tbl table, except for the id column, were empty, these columns occupied almost no space during serialization. To address this, I modified the partition column containing data from string type to int type and rebuilt the data. The test results showed that the serialized size before compaction was 89M, and after compaction, it was reduced to 49M, representing a 45% reduction in data size. http://gerrit.cloudera.org:8080/#/c/21845/2/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/21845/2/fe/src/main/java/org/apache/impala/service/JniCatalog.java@263 PS2, Line 263: private byte[] execAndSerializeWithCompaction(String methodName, > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/21845/4/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/21845/4/fe/src/main/java/org/apache/impala/service/JniCatalog.java@263 PS4, Line 263: private byte[] execAndSerializeWithCompaction(String methodName, > line too long (91 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/21845 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idea9313c7f1f1596f3620e60b08a99efc7fa0466 Gerrit-Change-Number: 21845 Gerrit-PatchSet: 5 Gerrit-Owner: zhangqianqiong <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: zhangqianqiong <[email protected]> Gerrit-Comment-Date: Mon, 30 Sep 2024 10:56:05 +0000 Gerrit-HasComments: Yes
