Re: Configuring UDFs with user defined parameters

2015-09-14 Thread Stephan Ewen
Yes, exactly... On Tue, Sep 15, 2015 at 1:15 AM, Matthias J. Sax wrote: > Thanks, now I understand how to do it: > > We just use > > env.getConfig().setGlobalJobParameters(new StromConfig()); > > with "StormConfig extends GlobalJobParameters" > > We can access this configuration in SourceFunctio

Re: Configuring UDFs with user defined parameters

2015-09-14 Thread Matthias J. Sax
Thanks, now I understand how to do it: We just use > env.getConfig().setGlobalJobParameters(new StromConfig()); with "StormConfig extends GlobalJobParameters" We can access this configuration in SourceFunction via > getRuntimeContext().getExecutionConfig().getGlobalJobParameters() and in Stream

Re: Configuring UDFs with user defined parameters

2015-09-14 Thread Fabian Hueske
Ah, here's the discussion I was looking for :-) I think Stephan refers to ExecutionConfig.setGlobalJobParameters(). 2015-09-15 0:25 GMT+02:00 Matthias J. Sax : > It might sound stupid. But how could such a configuration be set? > > StreamExecutionEnvironment only offerst ".getConfig()" > > -Matth

Re: Configuring UDFs with user defined parameters

2015-09-14 Thread Matthias J. Sax
It might sound stupid. But how could such a configuration be set? StreamExecutionEnvironment only offerst ".getConfig()" -Matthias On 09/07/2015 03:05 PM, Stephan Ewen wrote: > The JobConfig is a system level config. Would be nice to not expose them to > the user-level unless necessary. > > Wha

Re: Configuring UDFs with user defined parameters

2015-09-07 Thread Stephan Ewen
The JobConfig is a system level config. Would be nice to not expose them to the user-level unless necessary. What about using the ExecutionConfig, where you can add shared user-level parameters? On Mon, Sep 7, 2015 at 1:39 PM, Matthias J. Sax wrote: > Thanks for the input. > > However, I doubt

Re: Configuring UDFs with user defined parameters

2015-09-07 Thread Matthias J. Sax
Thanks for the input. However, I doubt that a member variable approach is feasible, because when the Storm topology is translated into a Flink program (in `FlinkBuilder.createTopology()`) the Storm configuration is not available yet. And adding the configuration later to each operator would be cum

Re: Configuring UDFs with user defined parameters

2015-09-06 Thread Aljoscha Krettek
Hi, I think the possibility to use a Configuration object is a legacy from the past where the API was a bit closer to how Hadoop works. In my opinion this is not necessary anymore since User Code objects can just contain configuration settings in fields. The feature for the Storm API could probabl