Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/19198 )
Change subject: IMPALA-11478: Cleanup JniCatalog ...................................................................... Patch Set 4: (8 comments) http://gerrit.cloudera.org:8080/#/c/19198/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19198/3//COMMIT_MSG@13 PS3, Line 13: Frontend test added as JniCatalogOpTest. > We are lack of tests for thread annotations and logs. It'd be nice if you c I checked manually the exceptions and the thread dump for getDbs and getCatalogObject methods Thread dump shows the correct name during calls: Thread-18 [getting databases]" #62 prio=5 os_prio=0 tid=0x000000000dd77800 nid=0x24a19 runnable [0x00007f8bba89b000] I1122 10:36:07.223536 154715 JniUtil.java:161] getDbs request: getting databases I1122 10:36:16.965569 154715 JniUtil.java:165] Finished getDbs request: getting databases. Time spent: 9s742ms Too slow and large error message remained the same (forced with debugger, so the durations are off a bit): W1122 10:52:32.269219 174059 JniUtil.java:188] Response too large and too slow: size=7115 (6.95KB), duration=35392ms (35s392ms), method: getDbs, request: TGetDbsParams() W1122 10:55:05.165840 174059 JniUtil.java:188] Response too large and too slow: size=7115 (6.95KB), duration=1404300ms (23m24s), method: getDbs, request: TGetDbsParams() OOM, Runtime exception logs are resembling with the previous solution: E1122 11:36:28.131889 226354 JniUtil.java:171] Error in getting thrift catalog object of TABLE:tpch_nested_orc_def.part. Time spent: I1122 11:36:28.132845 226354 jni-util.cc:288] java.lang.OutOfMemoryError: aa at org.apache.impala.service.JniCatalog.lambda$getCatalogObject$7(JniCatalog.java:347) at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:64) at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:86) at org.apache.impala.service.JniCatalog.getCatalogObject(JniCatalog.java:344) E1122 11:41:34.900446 232524 JniUtil.java:171] Error in getting thrift catalog object of TABLE:tpch_nested_orc_def.part. Time spent: I1122 11:41:34.900624 232524 jni-util.cc:288] org.apache.impala.common.ImpalaRuntimeException: aa at org.apache.impala.service.JniCatalog.lambda$getCatalogObject$7(JniCatalog.java:348) at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:64) at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:86) at org.apache.impala.service.JniCatalog.getCatalogObject(JniCatalog.java:345) http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java@20 PS3, Line 20: import static org.apache.impala.service.JniCatalogOp.execOp; > Please avoid star import Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java@191 PS3, Line 191: v > nit: remove DOT here Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java@294 PS3, Line 294: params > Shouldn't this be "params"? Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalog.java@316 PS3, Line 316: params > params? Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalogOp.java File fe/src/main/java/org/apache/impala/service/JniCatalogOp.java: http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalogOp.java@20 PS3, Line 20: com.google.common.base.Preco > We use "com.google.common.base.Preconditions" instead: Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalogOp.java@38 PS3, Line 38: : /** : * Executes @operand in an annotated thread environme > Could you add some commends about the parameters? especially the return typ Done http://gerrit.cloudera.org:8080/#/c/19198/3/fe/src/main/java/org/apache/impala/service/JniCatalogOp.java@57 PS3, Line 57: throws ImpalaException, TException { : Preconditions.checkNotNull(operand); : Preconditions.checkNotNull(finalClause); > It'd be nice if move these into JniUtil#logResponse(). Moved logging logic to a separate class. -- To view, visit http://gerrit.cloudera.org:8080/19198 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1932172e2d13a7aab2336661c18befb4407ec9ab Gerrit-Change-Number: 19198 Gerrit-PatchSet: 4 Gerrit-Owner: Peter Rozsa <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Comment-Date: Tue, 22 Nov 2022 10:59:23 +0000 Gerrit-HasComments: Yes
