Felix Dreissig created FLINK-7503: ------------------------------------- Summary: Config options taskmanager.log.path and jobmanager.web.log.path are misleading, if not broken Key: FLINK-7503 URL: https://issues.apache.org/jira/browse/FLINK-7503 Project: Flink Issue Type: Bug Affects Versions: 1.3.2 Reporter: Felix Dreissig Priority: Minor
Setting config option {{taskmanager.log.path}} to the absolute directory of the logs caused the TaskManager logs to be unavailable through the JobManager web UI in our setup: The web UI said "Fetching TaskManager log failed.", JobManager and TaskManager logs see below. By grepping through the source code, I found that {{taskmanager.log.path}} (resp. {{TASK_MANAGER_LOG_PATH_KEY}}) outside of tests only ever gets used by {{TaskManager.handleRequestTaskManagerLog()}}, but not for writing the logs. Which makes sense, because writing them is (solely, as far as I can tell) handled by Log4j. Documentation on {{taskmanager.log.path}} is rather sparse and just says "The config parameter defining the taskmanager log file location". Apart from not telling what the value is supposed to look like (absolute/relative path), this also doesn't say how the option is supposed to be used. A similar case is {{jobmanager.web.log.path}}, which isn't even documented (no idea how it ended up in our config) and apart from tests is only used in {{WebMonitorUtils.LogFileLocation.find()}}. For production deployments, it normally won't even be picked up there since {{log.file}} is always set when Flink is launched through the start script. However, this option caused no issues for us so far. ---- JobManager log: {code} ERROR org.apache.flink.runtime.webmonitor.handlers.TaskManagerLogHandler - Fetching TaskManager log failed. java.util.NoSuchElementException: None.get at scala.None$.get(Option.scala:347) at scala.None$.get(Option.scala:345) at org.apache.flink.runtime.webmonitor.handlers.TaskManagerLogHandler$2.apply(TaskManagerLogHandler.java:200) at org.apache.flink.runtime.webmonitor.handlers.TaskManagerLogHandler$2.apply(TaskManagerLogHandler.java:197) {code} TaskManager log: {code} ERROR akka.actor.OneForOneStrategy - /var/log/flink (Is a directory) java.io.FileNotFoundException: /var/log/flink (Is a directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.apache.flink.runtime.taskmanager.TaskManager.org$apache$flink$runtime$taskmanager$TaskManager$$handleRequestTaskManagerLog(TaskManager.scala:840) at org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$handleMessage$1.applyOrElse(TaskManager.scala:337) {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)