[
https://issues.apache.org/jira/browse/FLINK-34114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jonasjc updated FLINK-34114:
----------------------------
Description:
When run job by K8s or Yarn Application Mode use
org.apache.flink.configuration.GlobalConfiguration#loadYAMLResource method
parse config。
{code:java}
private static Configuration loadYAMLResource(File file) {
final Configuration config = new Configuration();
try (BufferedReader reader =
new BufferedReader(new InputStreamReader(new
FileInputStream(file)))) {
String line;
int lineNo = 0;
while ((line = reader.readLine()) != null) {
lineNo++;
// 1. check for comments
String[] comments = line.split("#", 2);
String conf = comments[0].trim();
// 2. get key and value
if (conf.length() > 0) {
String[] kv = conf.split(": ", 2);
// skip line with no valid key-value pair
if (kv.length == 1) {
LOG.warn(
"Error while trying to split key and value in
configuration file "
+ file
+ ":"
+ lineNo
+ ": Line is not a key-value pair (missing
space after ':'?)");
continue;
}
String key = kv[0].trim();
String value = kv[1].trim();
// sanity check
if (key.length() == 0 || value.length() == 0) {
LOG.warn(
"Error after splitting key and value in
configuration file "
+ file
+ ":"
+ lineNo
+ ": Key or value was empty");
continue;
}
config.setString(key, value);
}
}
} catch (IOException e) {
throw new RuntimeException("Error parsing YAML configuration.", e);
}
return config;
} {code}
if config value contains '#',ofenly by $internal.application.program-args,the
following code is not correct
{code:java}
line.split("#", 2) {code}
To fix this,we can just skip the line when start with '#'
> Parse error while $internal.application.program-args contains '#' in Yarn/K8s
> Application Mode
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-34114
> URL: https://issues.apache.org/jira/browse/FLINK-34114
> Project: Flink
> Issue Type: Bug
> Components: API / Core
> Affects Versions: 1.18.0
> Reporter: jonasjc
> Priority: Major
> Fix For: 1.19.0
>
>
> When run job by K8s or Yarn Application Mode use
> org.apache.flink.configuration.GlobalConfiguration#loadYAMLResource method
> parse config。
>
> {code:java}
> private static Configuration loadYAMLResource(File file) {
> final Configuration config = new Configuration();
> try (BufferedReader reader =
> new BufferedReader(new InputStreamReader(new
> FileInputStream(file)))) {
> String line;
> int lineNo = 0;
> while ((line = reader.readLine()) != null) {
> lineNo++;
> // 1. check for comments
> String[] comments = line.split("#", 2);
> String conf = comments[0].trim();
> // 2. get key and value
> if (conf.length() > 0) {
> String[] kv = conf.split(": ", 2);
> // skip line with no valid key-value pair
> if (kv.length == 1) {
> LOG.warn(
> "Error while trying to split key and value in
> configuration file "
> + file
> + ":"
> + lineNo
> + ": Line is not a key-value pair
> (missing space after ':'?)");
> continue;
> }
> String key = kv[0].trim();
> String value = kv[1].trim();
> // sanity check
> if (key.length() == 0 || value.length() == 0) {
> LOG.warn(
> "Error after splitting key and value in
> configuration file "
> + file
> + ":"
> + lineNo
> + ": Key or value was empty");
> continue;
> }
> config.setString(key, value);
> }
> }
> } catch (IOException e) {
> throw new RuntimeException("Error parsing YAML configuration.", e);
> }
> return config;
> } {code}
> if config value contains '#',ofenly by
> $internal.application.program-args,the following code is not correct
> {code:java}
> line.split("#", 2) {code}
> To fix this,we can just skip the line when start with '#'
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
