Github user squito commented on the issue:
https://github.com/apache/spark/pull/22109
this looks reasonable, but now I'm wondering whether this will only affect
the driver. Couldn't it also effect the executors? Executors might get
created as soon as there is a
[`schedulerBackend`](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L493)
and they can register as soon as there is a blockManagerMaster, which is done
as part of the [SparkEnv
creation](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L424)
-- both happen before the EventLoggingListener is created. so we'd need to do
something different for that case, probably have the listenerBus buffer up
events until we tell it all "primimary" listeners are registered.
I suppose that's an extremely unlikely race, so I might be overthinking
this. I do recall seeing cases where the executors didn't have info either,
but I doubt this was the cause.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
