Brian Hulette created BEAM-12220:
------------------------------------
Summary: ZipFiles.zipDirectory leaks native JVM memory
Key: BEAM-12220
URL: https://issues.apache.org/jira/browse/BEAM-12220
Project: Beam
Issue Type: Bug
Components: runner-core, runner-dataflow, sdk-java-core
Affects Versions: 2.22.0
Reporter: Brian Hulette
Assignee: Brian Hulette
Noticed this in a long-running JVM process starting multiple Dataflow
pipelines. Heap usage slowly rose as observed by {{jcmd <pid> VM.native_memory
summary.diff}}, but a heap dump did not show a consistent amount of usage.
Followed [this
guide|https://github.com/jeffgriffith/native-jvm-leaks/blob/master/README.md]
for identify native memory leaks, which pointed me to java.util.ZipFile.
We use it in
[runners.core-construction.PipelineResources|https://github.com/apache/beam/blob/8e6695643b69da1ab83c9aad16a913cf6e442823/runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/resources/PipelineResources.java#L132]
via
[sdk.util.ZipFiles|https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java]
Closing the ZipOutputStream
[here|https://github.com/apache/beam/blob/47cfbcb63f4d0642d26106485bc6fdb894da3086/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java#L226]
seems to alleviate the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
