[ 
https://issues.apache.org/jira/browse/FLINK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruguo Yu updated FLINK-20682:
-----------------------------
    Description: 
{{Current, we submit flink job to yarn with run-application target and need to 
specify some configuration related to hadoop, because we use distributed 
filesystem similar to Ali oss to storage resources, in this case, we will pass 
special configuration option and set them to hadoopConfiguration.}}

{{In order to solve such problems, we can provide a configuration option 
prefixed with "flink.hadoop."(such as -Dflink.hadoop.xxx=yyy), and then take it 
into H}}{{adoopConfiguration.}}

{{A simple implementation code is as follows:}}{{}}
{code:java}
module: flink-filesystems/flink-hadoop-fs
class: org.apache.flink.runtime.util.HadoopUtils

//代码占位符
public static Configuration 
getHadoopConfiguration(org.apache.flink.configuration.Configuration 
flinkConfiguration) {
   ......
   // Copy any "flink.hadoop.xxx=yyy" flink configuration to hadoop 
configuration as "xxx=yyy"
   for (String key : flinkConfiguration.keySet()) {
      if (key.startsWith("flink.hadoop.")) {
         result.set(key.substring("flink.hadoop.".length()),
            flinkConfiguration.getString(key, null));
      }
   }
   return result;
}{code}

  was:
{{Current, we submit flink job to yarn with run-application target and need to 
specify some configuration related to hadoop, because we use distributed 
filesystem similar to Ali oss to storage resources, in this case, we will pass 
special configuration option and set them to hadoopConfiguration.}}

{{In order to solve such problems, we can provide a configuration option 
prefixed with "flink.hadoop."(such as -Dflink.hadoop.x.y.z), and then take it 
into H}}{{adoopConfiguration.}}

{{A simple implementation code is as follows:}}{{}}
{code:java}
module: flink-filesystems/flink-hadoop-fs
class: org.apache.flink.runtime.util.HadoopUtils

//代码占位符
public static Configuration 
getHadoopConfiguration(org.apache.flink.configuration.Configuration 
flinkConfiguration) {
   ......
   // Copy any "flink.hadoop.xxx=yyy" flink configuration to hadoop 
configuration as "xxx=yyy"
   for (String key : flinkConfiguration.keySet()) {
      if (key.startsWith("flink.hadoop.")) {
         result.set(key.substring("flink.hadoop.".length()),
            flinkConfiguration.getString(key, null));
      }
   }
   return result;
}{code}


> Add configuration options related to hadoop
> -------------------------------------------
>
>                 Key: FLINK-20682
>                 URL: https://issues.apache.org/jira/browse/FLINK-20682
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / YARN
>    Affects Versions: 1.12.0
>            Reporter: Ruguo Yu
>            Priority: Major
>              Labels: pull-requests-available
>             Fix For: 1.13.0, 1.12.1
>
>
> {{Current, we submit flink job to yarn with run-application target and need 
> to specify some configuration related to hadoop, because we use distributed 
> filesystem similar to Ali oss to storage resources, in this case, we will 
> pass special configuration option and set them to hadoopConfiguration.}}
> {{In order to solve such problems, we can provide a configuration option 
> prefixed with "flink.hadoop."(such as -Dflink.hadoop.xxx=yyy), and then take 
> it into H}}{{adoopConfiguration.}}
> {{A simple implementation code is as follows:}}{{}}
> {code:java}
> module: flink-filesystems/flink-hadoop-fs
> class: org.apache.flink.runtime.util.HadoopUtils
> //代码占位符
> public static Configuration 
> getHadoopConfiguration(org.apache.flink.configuration.Configuration 
> flinkConfiguration) {
>    ......
>    // Copy any "flink.hadoop.xxx=yyy" flink configuration to hadoop 
> configuration as "xxx=yyy"
>    for (String key : flinkConfiguration.keySet()) {
>       if (key.startsWith("flink.hadoop.")) {
>          result.set(key.substring("flink.hadoop.".length()),
>             flinkConfiguration.getString(key, null));
>       }
>    }
>    return result;
> }{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to