Hi Richard, You can use dynamic properties to add your environmental variables.
Set jobmanager env: e.g. -Dcontainerized.master.env.GOOGLE_APPLICATION_CREDENTIALS=xyz Set taskmanager env: e.g. -Dcontainerized.taskmanager.env.GOOGLE_APPLICATION_CREDENTIALS=xyz Best Regards, Jiayi Liao Original Message Sender: Richard Deurwaarder<rich...@xeli.eu> Recipient: user<user@flink.apache.org> Date: Tuesday, Sep 24, 2019 23:01 Subject: Setting environment variables of the taskmanagers (yarn) Hello, We have our flink job (1.8.0) running on our hadoop 2.7 cluster with yarn. We would like to add the GCS connector to use GCS rather than HDFS. Following the documentation of the GCS connector[1] we have to specify which credentials we want to use and there are two ways of doing this: * Edit core-site.xml * Set an environment variable: GOOGLE_APPLICATION_CREDENTIALS Because we're on a company shared hadoop cluster we do not want to change the cluster wide core-site.xml. This leaves me with two options: 1. Create a custom core-site.xml and use --yarnship to send it to all the taskmanager contains. If I do this, to what value should I set fs.hdfs.hadoopconf[2] in flink-conf ? 2. The second option would be to set an environment variable, however because the taskmanagers are started via yarn I'm having trouble figuring out how to make sure this environment variable is set for each yarn container / taskmanager. I would appreciate any help you can provide. Thank you, Richard [1] https://github.com/GoogleCloudPlatform/bigdata-interop/blob/master/gcs/INSTALL.md#configure-hadoop [2] https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#hdfs
