tgravescs commented on a change in pull request #28053: [SPARK-29153][CORE]Add
ability to merge resource profiles within a stage with Stage Level Scheduling
URL: https://github.com/apache/spark/pull/28053#discussion_r400238749
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1814,4 +1814,8 @@ package object config {
.bytesConf(ByteUnit.BYTE)
.createOptional
+ private[spark] val RESOURCE_PROFILE_MERGE_CONFLICTS =
+ ConfigBuilder("spark.scheduler.resourceProfile.mergeConflicts")
+ .booleanConf
Review comment:
our normal format for differentiating driver and executor configs is
spark.driver. and spark.executor., so I am not in favor or changing that to be
spark.resource.driver spark.resource.executor as I think that is not consistent.
I'm fine with changing to spark.driver.resource. and
spark.executor.resource. or if not specific spark.resource.
I'm not sure what you would call spark.driver.resourcesFile that makes sense
though, unless you call it. spark.driver.resource.resourcesFile which seems
like just extra but is consistent among the feature.
Similar here, spark.scheduler is a prefix that tells you what component is
affects, if we want to call this
spark.scheduler.resource.resourceProfileMergeConflicts or something like that
I'd be ok with that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
