[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29395: [3.0][SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-13 Thread GitBox


dongjoon-hyun commented on a change in pull request #29395:
URL: https://github.com/apache/spark/pull/29395#discussion_r470395517



##
File path: core/src/main/scala/org/apache/spark/internal/config/Tests.scala
##
@@ -61,4 +61,19 @@ private[spark] object Tests {
 .version("3.0.0")
 .intConf
 .createWithDefault(2)
+
+  val RESOURCES_WARNING_TESTING = 
ConfigBuilder("spark.resources.warnings.testing")
+.version("3.1.0")

Review comment:
   This should be `3.0.1` when it comes to `branch-3.0`, @Ngone51 .





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29395: [3.0][SPARK-32518][CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-13 Thread GitBox


dongjoon-hyun commented on a change in pull request #29395:
URL: https://github.com/apache/spark/pull/29395#discussion_r470395606



##
File path: core/src/main/scala/org/apache/spark/internal/config/Tests.scala
##
@@ -61,4 +61,19 @@ private[spark] object Tests {
 .version("3.0.0")
 .intConf
 .createWithDefault(2)
+
+  val RESOURCES_WARNING_TESTING = 
ConfigBuilder("spark.resources.warnings.testing")
+.version("3.1.0")
+.booleanConf
+.createWithDefault(false)
+
+  // This configuration is used for unit tests to allow skipping the task cpus 
to cores validation
+  // to allow emulating standalone mode behavior while running in local mode. 
Standalone mode
+  // by default doesn't specify a number of executor cores, it just uses all 
the ones available
+  // on the host.
+  val SKIP_VALIDATE_CORES_TESTING =
+  ConfigBuilder("spark.testing.skipValidateCores")
+.version("3.1.0")
+.booleanConf
+.createWithDefault(false)

Review comment:
   ditto. This should be `3.0.1` when it comes to `branch-3.0`, @Ngone51 .
   Also, after merging this, please update `master` branch consistently.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org