[
https://issues.apache.org/jira/browse/BEAM-5640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123093#comment-17123093
]
Beam JIRA Bot commented on BEAM-5640:
-------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it
has been labeled "stale-P2". If this issue is still affecting you, we care!
Please comment and remove the label. Otherwise, in 14 days the issue will be
moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed
explanation of what these priorities mean.
> Portable python sdk worker leaks memory when PyOpenSSL package is present
> -------------------------------------------------------------------------
>
> Key: BEAM-5640
> URL: https://issues.apache.org/jira/browse/BEAM-5640
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Micah Wylde
> Priority: P2
> Labels: stale-P2
>
> When PyOpenSSL package is installed on a system (e.g., in a virtualenv) the
> python sdk_worker process leaks memory. I've validated this when using the
> flink portable runner in streaming mode, but it may occur in other
> configurations as well. The leak is pretty significant, amounting to tens of
> MBs/sec.
> I've put together a reproduction for the issue
> [here|https://github.com/mwylde/beam/tree/micah_memory_leak]. That branch
> includes a flink streaming data source that generates data, as well as a
> python pipeline that demonstrates the issue.
> To reproduce:
> {code:java}
> check out the branch:
> $ git clone [email protected]:mwylde/beam.git
> $ git checkout micah_memory_leak
> build the python docker container with pyopenssl installed:
> $ cd beam
> $ ./gradlew :beam-sdks-python-container:docker
> start the job server with embedded flink cluster:
> $ ./gradlew runShadow
> run the pipeline:
> $ ./gradlew :beam-sdks-python:streamingLeak{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
