Hi Andrey,
Thanks for your detailed answer, and I have created a JIRA issue to discuss
it [1].
Please check the description and help me to fill the details, like
component/s, since
I'm not sure where it should be put. Thank you very much.
Best,
Tony Wei
[1]
Hi Tony,
File system factories are class-loaded in running JVMs of task executors.
That is why their configured objects are shared by different Flink jobs.
It is not possible to change their options per created file system and per job
at the moment.
This could be changed, e.g. for s3, by
It might be difficult as you the task manager and job manager are pre-started
in a session mode.
It seems that flink http server will always use the configuration that you
specified when you start your flink cluster, i.e. start-cluster.sh, I don't
find a way to override it.
--
Sent from:
Hi yinhua,
Our flink version is 1.6.0.
Best,
Tony Wei
yinhua.dai 於 2018年11月27日 週二 下午2:32寫道：
> Which flink version are you using.
> I know how it works in yarn, but not very clear with standalone mode.
>
>
>
> --
> Sent from:
>
Which flink version are you using.
I know how it works in yarn, but not very clear with standalone mode.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi yinhua,
I didn't try this yet, but I didn't see this option in both flink cli tool
and rest api either.
Could you please provide more details about how to use this option to
submit flink
application?
BTW, we are using standalone session cluster, not yarn session cluster. And
I need
to submit
Did you try "-Dkey=value"?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Hi,
Is there anyone can answer me?
Thanks,
Tony Wei
Tony Wei 於 2018年11月20日 週二 下午7:39寫道：
> Hi,
>
> Is there any way to provide s3.access-key and s3.secret-key in flink
> application, instead of setting
> them in flink-conf.yaml?
>
> In our use case, my team provide a flink standalone cluster
Hi,
Is there any way to provide s3.access-key and s3.secret-key in flink
application, instead of setting
them in flink-conf.yaml?
In our use case, my team provide a flink standalone cluster to users.
However, we don't want to let
each user use the same s3 bucket as filesystem to store
