[
https://issues.apache.org/jira/browse/BEAM-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17137917#comment-17137917
]
Thomas Weise edited comment on BEAM-5640 at 6/16/20, 8:46 PM:
--------------------------------------------------------------
No longer an issue for us after moving to py3. If anyone else using the package
runs into this issue, please re-open.
was (Author: thw):
No longer an issue after moving to py3.
> Portable python sdk worker leaks memory when PyOpenSSL package is present
> -------------------------------------------------------------------------
>
> Key: BEAM-5640
> URL: https://issues.apache.org/jira/browse/BEAM-5640
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Micah Wylde
> Priority: P3
> Fix For: Not applicable
>
>
> When PyOpenSSL package is installed on a system (e.g., in a virtualenv) the
> python sdk_worker process leaks memory. I've validated this when using the
> flink portable runner in streaming mode, but it may occur in other
> configurations as well. The leak is pretty significant, amounting to tens of
> MBs/sec.
> I've put together a reproduction for the issue
> [here|https://github.com/mwylde/beam/tree/micah_memory_leak]. That branch
> includes a flink streaming data source that generates data, as well as a
> python pipeline that demonstrates the issue.
> To reproduce:
> {code:java}
> check out the branch:
> $ git clone [email protected]:mwylde/beam.git
> $ git checkout micah_memory_leak
> build the python docker container with pyopenssl installed:
> $ cd beam
> $ ./gradlew :beam-sdks-python-container:docker
> start the job server with embedded flink cluster:
> $ ./gradlew runShadow
> run the pipeline:
> $ ./gradlew :beam-sdks-python:streamingLeak{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
