Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19282 )
Change subject: IMPALA-11753: CatalogD OOMkilled due to natively allocated memory ...................................................................... IMPALA-11753: CatalogD OOMkilled due to natively allocated memory CatalogD can be OOMKilled due to too much natively allocated memory. The bug is due to a misuse of a Java compression API: https://bugs.openjdk.org/browse/JDK-8257032 The problem is that we create our own Deflater object and pass it to the constructor of DeflaterOutputStream: https://github.com/apache/impala/blob/84fa6d210d3966e5ece8b4ac84ff8bd8780dec4e/fe/src/main/java/org/apache/impala/util/CompressionUtil.java#L47 This means that Java's DeflaterOutputStream won't assume ownership on the Deflater, and won't invoke its end() method: * https://github.com/openjdk/jdk/blob/a249a52501f3cd7d4fbe5293d14ac8d0d6ffcc69/src/java.base/share/classes/java/util/zip/DeflaterOutputStream.java#L144 * https://github.com/openjdk/jdk/blob/a249a52501f3cd7d4fbe5293d14ac8d0d6ffcc69/src/java.base/share/classes/java/util/zip/DeflaterOutputStream.java#L246-L247 The Deflater's methods are implemented in C and allocate native memory. This means that until the GC doesn't destroy the unreachable Deflater objects they can consume quite much native memory. In some scenarios it can even result in OOMKills by the kernel. The fix is to override the DeflaterOutputStream's close() method so it invokes end() on the Deflater object. Change-Id: I663a21f60871e32d2d0100ea03d92fd8ab460691 Reviewed-on: http://gerrit.cloudera.org:8080/19282 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/util/CompressionUtil.java 1 file changed, 12 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19282 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I663a21f60871e32d2d0100ea03d92fd8ab460691 Gerrit-Change-Number: 19282 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
