ajithme commented on a change in pull request #24438: [SPARK-23626][CORE]
DAGScheduler blocked due to JobSubmitted event
URL: https://github.com/apache/spark/pull/24438#discussion_r277928327
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1309,4 +1309,11 @@ package object config {
.bytesConf(ByteUnit.BYTE)
.createOptional
+ private[spark] val EAGER_RDD_PARTITION_EVALUATE =
+ ConfigBuilder("spark.rdd.eager.partition")
Review comment:
Thanks @srowen I did some testing and these seems to solve the problem but i
was skeptic of the change so kept it configurable so that user(admin) can
change back if this introduces unwanted behavior. I did not intend to make this
configuration public. DAGScheduler is still a grey area for me, would like to
know any feedback for this change. If it checks all the boxes, i am too +1 for
not making this configurable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
