Re: Customization of execution environment

2020-07-31 Thread Aljoscha Krettek
I agree! My long-term goal is that a Configuration is the basis of truth and that the programmatic setter methods and everything else just modify the underlying configuration. We have made big steps in at least allowing to configure most (if not all) StreamExecutionEnvironment and

Re: Customization of execution environment

2020-07-30 Thread Flavio Pompermaier
That's fine and it's basically what I do as well..I was arguing that it's bad (IMHO) that you could access the config from the BatchTableEnvironment (via bte.getConfig().getConfiguration()). You legitimately think that you are customizing the env but that's illusory. You should not be able to set

Re: Customization of execution environment

2020-07-30 Thread Arvid Heise
I'm not entirely sure, if I completely understand the interaction of BTE and ExecEnv, but I'd create it this way Configuration conf = new Configuration(); conf.setInteger(TaskManagerOptions.NUM_TASK_SLOTS, PARALLELISM); ExecutionEnvironment env =

Re: Customization of execution environment

2020-07-29 Thread Robert Metzger
Hi Flavio, I think the recommended approach is as follows: (then you don't need to create to environments) final Configuration conf = new Configuration(); conf.setLong(...) env = new LocalEnvironment(conf); I agree that in theory it would be nicer if the configuration returned was editable, but

Customization of execution environment

2020-07-28 Thread Flavio Pompermaier
Hi to all, migrating to Flink 1.11 I've tried to customize the exec env in this way: ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); BatchTableEnvironment bte = BatchTableEnvironment.create(env); final Configuration conf = bte.getConfig().getConfiguration();