Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/20532#discussion_r166818413
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -228,14 +231,23 @@ private[spark] class EventLoggingListener(
logEvent(event, flushLogger = true)
}
+ // Only sampled logging when fraction greater than zero, fraction equals
1 means logging all
override def onBlockUpdated(event: SparkListenerBlockUpdated): Unit = {
- if (shouldLogBlockUpdates) {
- logEvent(event, flushLogger = true)
+ if (blockUpdateSampleFraction > 0) {
--- End diff --
Think it again. Seems check `>0` could avoid generate a random number when
it is configured to 0.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
