Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21845 )
Change subject: IMPALA-13385: Compact the response of ddl operations ...................................................................... Patch Set 6: (3 comments) http://gerrit.cloudera.org:8080/#/c/21845/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21845/6//COMMIT_MSG@9 PS6, Line 9: At present, when catalogd responds to DDL operations, it sends the entire : table object. This is only true in the legacy catalog mode. In local-catalog mode, the response just contains the invalidations (similar to db/table names). http://gerrit.cloudera.org:8080/#/c/21845/6//COMMIT_MSG@16 PS6, Line 16: the following exception: `java.long.OutOfMemoryError: Requested array size : exceeds VM limit`. > Does the change solve the problem completely (when compact protocol is enou I think we need to verify this. The OOM of exceeding JVM array limit is thrown when growing the array in Java. Here the byte array is generated in cpp codes. Maybe it can survive. http://gerrit.cloudera.org:8080/#/c/21845/6//COMMIT_MSG@23 PS6, Line 23: 1. I built the 1k_col_tbl table using the scale_test_metadata script. > Did you also look at the CPU effect of this? My assumption is that compact We should avoid hitting OutOfMemoryError. Even if our code explicitly catches it, the related JVM options like -XX:+ExitOnOutOfMemoryError, -XX:OnOutOfMemoryError=cmd, will be executed, leading the process being killed. -- To view, visit http://gerrit.cloudera.org:8080/21845 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idea9313c7f1f1596f3620e60b08a99efc7fa0466 Gerrit-Change-Number: 21845 Gerrit-PatchSet: 6 Gerrit-Owner: zhangqianqiong <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: zhangqianqiong <[email protected]> Gerrit-Comment-Date: Mon, 24 Feb 2025 04:36:53 +0000 Gerrit-HasComments: Yes
