Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22752#discussion_r226420000
--- Diff: core/src/main/scala/org/apache/spark/deploy/history/config.scala
---
@@ -64,4 +64,11 @@ private[spark] object config {
.bytesConf(ByteUnit.BYTE)
.createWithDefaultString("1m")
+ val IN_PROGRESS_ABSOLUTE_LENGTH_CHECK =
+ ConfigBuilder("spark.history.fs.inProgressAbsoluteLengthCheck.enabled")
--- End diff --
How much overhead are we talking about?
That thread is not really in any performance critical path, and in general
people won't have so many running apps that this should become a problem... but
that kinda depends on how bad this new call is.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
