Re: Limit the number of DoFn instances per worker?

2017-10-18 Thread Lukasz Cwik
ng well for your usecase/pipeline. Also provide a >>> description of your pipeline and some job ids (if possible). >>> >>> >>> On Mon, Oct 16, 2017 at 6:26 PM, Rafal Wojdyla <r...@spotify.com> wrote: >>> >>>> Hi. >>>> To answe

Re: Limit the number of DoFn instances per worker?

2017-10-18 Thread Raghu Angadi
, you >>> could use `numberOfWorkerHarnessThreads`. See more here >>> <https://github.com/spotify/scio/wiki/FAQ#how-do-i-control-concurrency-number-of-dofn-threads-in-dataflow-workers>. >>> That said, I'm not gonna comment whether that is a good remedy for your >>&g

Re: Limit the number of DoFn instances per worker?

2017-10-17 Thread Lukasz Cwik
ent whether that is a good remedy for your > actual problem. > - rav > > On Mon, Oct 16, 2017 at 8:48 PM, Derek Hao Hu <phoenixin...@gmail.com> > wrote: > >> Hi, >> >> ​Is there an easy way to limit the number of DoFn instances per worker? >> >>