vanzin commented on a change in pull request #27208: [SPARK-30481][CORE]
Integrate event log compactor into Spark History Server
URL: https://github.com/apache/spark/pull/27208#discussion_r369677061
##########
File path: docs/configuration.md
##########
@@ -1023,6 +1023,26 @@ Apart from these, the following properties are also
available, and may be useful
The max size of event log file before it's rolled over.
</td>
</tr>
+<tr>
+ <td><code>spark.history.fs.eventLog.rolling.maxFilesToRetain</code></td>
+ <td>Int.MaxValue</td>
+ <td>
+ The maximum number of event log files which will be retained as
non-compacted. By default,
Review comment:
Spark currently doesn't have a "release notes" document in the docs dir, but
I'm wondering if we should mention somewhere that this is a new feature and may
not be completely stable (i.e. use with caution, it may delete more data than
you expect, cause some UI issues we haven't thought about, etc). There are
migration guides, but that doesn't seem like the right place...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
