Re: Access to a shared resource within a mapper

Hi Fabian, I didn't realize you meant that lazy val should be inside RichMapFunction implementation, it makes sense. That's what I ended up doing already. Thanks! Timur On Mon, Apr 25, 2016 at 3:34 AM, Fabian Hueske wrote: > Hi Timur, > > a TaskManager may run as many

Re: Access to a shared resource within a mapper

Hi Timur, a TaskManager may run as many subtasks of a Map operator as it has slots. Each subtask of an operator runs in a different thread. Each parallel subtask of a Map operator has its own MapFunction object, so it should be possible to use a lazy val. However, you should not use static

Re: Access to a shared resource within a mapper

Actually, a follow-up question: is map function single-threaded (within one task manager, that is). If it's not then lazy initialization wont' work, is it right? On Fri, Apr 22, 2016 at 11:50 AM, Stephan Ewen wrote: > You may also be able to initialize the client only in the

Re: Access to a shared resource within a mapper

Outstanding! Thanks, Aljoscha. On Fri, Apr 22, 2016 at 2:06 AM, Aljoscha Krettek wrote: > Hi, > you could use a RichMapFunction that has an open method: > > data.map(new RichMapFunction[...]() { > def open(): () = { > // initialize client > } > > def map(input:

Re: Access to a shared resource within a mapper

Hi, you could use a RichMapFunction that has an open method: data.map(new RichMapFunction[...]() { def open(): () = { // initialize client } def map(input: INT): OUT = { // use client } } the open() method is called before any elements are passed to the function. The counterpart