[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899228#comment-15899228 ]
Sean Owen commented on SPARK-19829: ----------------------------------- Why wouldn't log4j be a good solution here? its purpose is managing logs and can define rolling appenders. > The log about driver should support rolling like executor > --------------------------------------------------------- > > Key: SPARK-19829 > URL: https://issues.apache.org/jira/browse/SPARK-19829 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 3.0.0 > Reporter: hustfxj > Priority: Minor > > We should rollback the log of the driver , or the log maybe large!!! > {code:title=DriverRunner.java|borderStyle=solid} > // modify the runDriver > private def runDriver(builder: ProcessBuilder, baseDir: File, supervise: > Boolean): Int = { > builder.directory(baseDir) > def initialize(process: Process): Unit = { > // Redirect stdout and stderr to files------ the old code > // val stdout = new File(baseDir, "stdout") > // CommandUtils.redirectStream(process.getInputStream, stdout) > // > // val stderr = new File(baseDir, "stderr") > // val formattedCommand = builder.command.asScala.mkString("\"", "\" > \"", "\"") > // val header = "Launch Command: %s\n%s\n\n".format(formattedCommand, > "=" * 40) > // Files.append(header, stderr, StandardCharsets.UTF_8) > // CommandUtils.redirectStream(process.getErrorStream, stderr) > // Redirect its stdout and stderr to files---------support rolling > val stdout = new File(baseDir, "stdout") > stdoutAppender = FileAppender(process.getInputStream, stdout, conf) > val stderr = new File(baseDir, "stderr") > val formattedCommand = builder.command.asScala.mkString("\"", "\" \"", > "\"") > val header = "Launch Command: %s\n%s\n\n".format(formattedCommand, "=" > * 40) > Files.append(header, stderr, StandardCharsets.UTF_8) > stderrAppender = FileAppender(process.getErrorStream, stderr, conf) > } > runCommandWithRetry(ProcessBuilderLike(builder), initialize, supervise) > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org