Re: Passing parameter to DoFn in Python

[OrielResearch Eila Arich-Landkof]

great! thanks

On Tue, Apr 10, 2018 at 7:31 PM, Robert Bradshaw 
wrote:

> Yes, DoFns are normal Python classes. To do this you would write
>
> class SampleFn(beam.DoFn):
> def __init__(self, samplePath):
> self.samplePath = samplePath
>
> def process(self, element):
> # use self.samplePath here, will get to remote workers via pickling
>
> On Tue, Apr 10, 2018 at 4:27 PM OrielResearch Eila Arich-Landkof <
> e...@orielresearch.org> wrote:
>
>> Hi all,
>>
>> Is it possible to pass a string parameter with DoFn function and what
>> would be the syntax?
>>
>> The call should look something like that:
>> beam.ParDo(SampleFn(samplePath))
>>
>> how would the class definition be updated?
>> class SampleFn(beam.DoFn):
>>   def process(self,element):
>>
>>
>> Thanks,
>>
>> --
>> Eila
>> www.orielresearch.org
>> https://www.meetup.com/Deep-Learning-In-Production/
>>
>


-- 
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/

Re: Passing parameter to DoFn in Python

[Robert Bradshaw]

Yes, DoFns are normal Python classes. To do this you would write

class SampleFn(beam.DoFn):
def __init__(self, samplePath):
self.samplePath = samplePath

def process(self, element):
# use self.samplePath here, will get to remote workers via pickling

On Tue, Apr 10, 2018 at 4:27 PM OrielResearch Eila Arich-Landkof <
e...@orielresearch.org> wrote:

> Hi all,
>
> Is it possible to pass a string parameter with DoFn function and what
> would be the syntax?
>
> The call should look something like that:
> beam.ParDo(SampleFn(samplePath))
>
> how would the class definition be updated?
> class SampleFn(beam.DoFn):
>   def process(self,element):
>
>
> Thanks,
>
> --
> Eila
> www.orielresearch.org
> https://www.meetup.com/Deep-Learning-In-Production/
>

Passing parameter to DoFn in Python

[OrielResearch Eila Arich-Landkof]

Hi all,

Is it possible to pass a string parameter with DoFn function and what would
be the syntax?

The call should look something like that:
beam.ParDo(SampleFn(samplePath))

how would the class definition be updated?
class SampleFn(beam.DoFn):
  def process(self,element):


Thanks,

-- 
Eila
www.orielresearch.org
https://www.meetup.com/Deep-Learning-In-Production/