[
https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869254#comment-17869254
]
yiheng tang edited comment on FLINK-35898 at 7/29/24 7:38 AM:
--------------------------------------------------------------
If my log.file named as "/data08/yarn/userlogs/application
1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log",
the command will be incorrectly truncated.
was (Author: JIRAUSER306337):
If my log.file named as "/data08/yarn/userlogs/application
1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log",
the command is incorrectly truncated.
> When running Flink on Yarn, if the log file path contains a space, the
> startup will fail.
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-35898
> URL: https://issues.apache.org/jira/browse/FLINK-35898
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.20.0
> Reporter: yiheng tang
> Priority: Minor
> Attachments: image-2024-07-25-21-51-50-072.png,
> image-2024-07-25-21-52-26-973.png
>
>
> We use Yarn to launch Flink jobs and try to add custom parameters to
> configure the log output format in the YarnLogConfigUtil.getLog4jCommand
> method. However, when the parameter value contains spaces, it causes the
> startup to fail.
> The start command in the Yarn container is as follows, omitting some
> irrelevant parameters.
> {code:java}
> /bin/bash -c "$JAVA_HOME/bin/java
> -Dlog.file="/data08/yarn/userlogs/application_1719804259110_16060/container_e43_1719804259110_16060_01_000001/jobmanager.log"
> -Dlog4j.configuration=file.properties
> -Dlog4j.configurationFile=file.properties
> -Dlog4j2.isThreadContextMapInheritable=true
> -Dlog.layout.pattern="%d{yyyy-MM-dd HH:mm} %-5p %-60c %-60t %x - %m%n %ex"
> -Dlog.level=INFO" {code}
> We discovered that when the value of log.layout.pattern contains spaces, the
> command is incorrectly truncated, causing the content after the space to be
> unrecognized, and therefore the Flink job cannot be launched in the Yarn
> container.
> {code:java}
> {HH:mm} %-5p %-60c %-60t %x - %m%n %ex" -Dlog.level=INFO"{code}
> Therefore, we suggest using single quotes (') instead of double quotes (") to
> wrap the values of parameters in the {{getLogBackCommand}} method and
> {{getLog4jCommand}} method of the {{YarnLogConfigUtil}} class.
> {code:java}
> private static String getLogBackCommand(final String logConfigFilePath) {
> final boolean hasLogback =
> logConfigFilePath.endsWith(CONFIG_FILE_LOGBACK_NAME);
> if (!hasLogback) {
> return "";
> } return new StringBuilder(
> "-Dlog.file=\""
> + ApplicationConstants.LOG_DIR_EXPANSION_VAR
> + "/jobmanager.log\"")
> .append(" -Dlogback.configurationFile=file:" +
> CONFIG_FILE_LOGBACK_NAME)
> .toString();
> } private static String getLog4jCommand(final String
> logConfigFilePath) {
> final boolean hasLog4j =
> logConfigFilePath.endsWith(CONFIG_FILE_LOG4J_NAME);
> if (!hasLog4j) {
> return "";
> } return new StringBuilder(
> "-Dlog.file=\""
> + ApplicationConstants.LOG_DIR_EXPANSION_VAR
> + "/jobmanager.log\"")
> .append(" -Dlog4j.configuration=file:" +
> CONFIG_FILE_LOG4J_NAME)
> .append(" -Dlog4j.configurationFile=file:" +
> CONFIG_FILE_LOG4J_NAME)
> .toString();
> }
> } {code}
> ---
> !image-2024-07-25-21-52-26-973.png|width=843,height=506!!image-2024-07-25-21-51-50-072.png|width=417,height=218!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
