[ https://issues.apache.org/jira/browse/BEAM-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik resolved BEAM-7848. ----------------------------- Fix Version/s: Not applicable Resolution: Fixed > Add possibility to manage quantity of instances (threads) per worker in > Python SDK > ---------------------------------------------------------------------------------- > > Key: BEAM-7848 > URL: https://issues.apache.org/jira/browse/BEAM-7848 > Project: Beam > Issue Type: Improvement > Components: runner-dataflow > Environment: Python SDK > ApacheBeam version==2.13.0 > worker_type==n1-standard-4 > Reporter: Severyn Parkhomenko > Priority: Major > Fix For: Not applicable > > Attachments: Selection_042.png > > > I'm developing a streaming pipeline with big memory consumption in one of the > PTransforms. > After some period after starting this pipeline fails without any specific > logs (see attachment file) > It looks like, that it happens because of OutOfMemory. > It would be great to set a limit of threads that will be used in a single > worker to control memory load. > I found such option in JAVA SDK (--_numberOfWorkerHarnessThreads_), but in > Python SDK it is absent -- This message was sent by Atlassian Jira (v8.3.2#803003)