[jira] [Commented] (MAPREDUCE-4778) Fair scheduler event log is only written if directory exists on HDFS

[Wing Yew Poon (JIRA)]

    [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492800#comment-13492800
 ] 

Wing Yew Poon commented on MAPREDUCE-4778:
------------------------------------------

In FairSchedulerEventLog.java, we have:

{noformat}
  boolean init(Configuration conf, String jobtrackerHostname) {
    try {
      logDir = conf.get("mapred.fairscheduler.eventlog.location",
          new File(System.getProperty("hadoop.log.dir")).getAbsolutePath()
          + File.separator + "fairscheduler");
      Path logDirPath = new Path(logDir);
      FileSystem fs = logDirPath.getFileSystem(conf);
      if (!fs.exists(logDirPath)) {
        if (!fs.mkdirs(logDirPath)) {
          throw new IOException(
              "Mkdirs failed to create " + logDirPath.toString());
        }
      }
      String username = System.getProperty("user.name");
      logFile = String.format("%s%shadoop-%s-fairscheduler-%s.log",
          logDir, File.separator, username, jobtrackerHostname);
      logDisabled = false;
      PatternLayout layout = new PatternLayout("%d{ISO8601}\t%m%n");
      appender = new DailyRollingFileAppender(layout, logFile, "'.'yyyy-MM-dd")\
;
      appender.activateOptions();
      LOG.info("Initialized fair scheduler event log, logging to " + logFile);
    } catch (IOException e) {
      LOG.error(
          "Failed to initialize fair scheduler event log. Disabling it.", e);
      logDisabled = true;
    }
    return !(logDisabled);
  }
{noformat}

The

{noformat}
      Path logDirPath = new Path(logDir);
      FileSystem fs = logDirPath.getFileSystem(conf);
      if (!fs.exists(logDirPath)) {
        if (!fs.mkdirs(logDirPath)) {
          throw new IOException(
              "Mkdirs failed to create " + logDirPath.toString());
        }
      }
{noformat}

appears to be unnecessary.
                
> Fair scheduler event log is only written if directory exists on HDFS
> --------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4778
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4778
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker, scheduler
>    Affects Versions: 1.1.0
>            Reporter: Sandy Ryza
>
> The fair scheduler event log is supposed to be written to the local 
> filesystem, at {hadoop.log.dir}/fairscheduler.  The event log will not be 
> written unless this directory exists on HDFS.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira