Re: Accessing configuration in RichFunction

2016-01-18 Thread Christian Kreutzfeldt
Hi Robert, using the constructor is actually the selected way. Using the existing lifecycle method was an idea to integrate it more with the existing framework design ;-) Best Christian 2016-01-18 13:38 GMT+01:00 Robert Metzger : > Hi Christian, > > I think the DataStream API does not allow y

Re: Accessing configuration in RichFunction

2016-01-18 Thread Robert Metzger
Hi Christian, I think the DataStream API does not allow you to pass any parameter to the open(Configuration) method. That method is only used in the DataSet (Batch) API, and its use is discouraged. A much better option to pass a Configuration into your function is as follows: Configuration mapC

Re: Accessing configuration in RichFunction

2016-01-18 Thread Christian Kreutzfeldt
Hi Max, maybe I explained it a bit mistakable ;-) I have a stream-based application which contains a RichFilterFunction implementation. The parent provides a lifecycle method open (open(Configuration)) which receives a Configuration object as input. I would like to use this call to pass options i

Re: Accessing configuration in RichFunction

2016-01-18 Thread Maximilian Michels
Hi Christian, For your implementation, would it suffice to pass a Configuration with your RichFilterFunction? You said the global job parameters are not passed on to your user function? Can you confirm this is a bug? Cheers, Max On Wed, Jan 13, 2016 at 10:59 AM, Christian Kreutzfeldt wrote: > H

Re: Accessing configuration in RichFunction

2016-01-13 Thread Christian Kreutzfeldt
Hi Fabian, thanks for your quick response. I just figured out that I forgot to mention a small but probably relevant detail: I am working with the streaming api. Although there is a way to access the overall job settings, I need a solution to "reduce" the view on configuration options available o

Re: Accessing configuration in RichFunction

2016-01-13 Thread Fabian Hueske
Hi Christian, the open method is called by the Flink workers when the parallel tasks are initialized. The configuration parameter is the configuration object of the operator. You can set parameters in the operator config as follows: DataSet text = ... DataSet wc = text.flatMap(new Tokenizer()).ge

Accessing configuration in RichFunction

2016-01-13 Thread Christian Kreutzfeldt
Hi While working on a RichFilterFunction implementation I was wondering, if there is a much better way to access configuration options read from file during startup. Actually, I am using getRuntimeContext().getExecutionConfig().getGlobalJobParameters() to get access to my settings. Reason for tha