tgravescs commented on code in PR #37938:
URL: https://github.com/apache/spark/pull/37938#discussion_r975386173
##########
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:
##########
@@ -129,6 +130,9 @@ public class YarnShuffleService extends AuxiliaryService {
// Whether failure during service initialization should stop the NM.
@VisibleForTesting
static final String STOP_ON_FAILURE_KEY = "spark.yarn.shuffle.stopOnFailure";
+
+ @VisibleForTesting
+ static final String INTEGRATION_TESTING =
"spark.yarn.shuffle.integrationTesting";
Review Comment:
we have other configs. just for testing, like
spark.dynamicAllocation.testing, I think we should keep name similar so please
rename to "spark.yarn.shuffle.testing"
##########
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:
##########
@@ -237,6 +241,10 @@ protected void serviceInit(Configuration externalConf)
throws Exception {
boolean stopOnFailure = _conf.getBoolean(STOP_ON_FAILURE_KEY,
DEFAULT_STOP_ON_FAILURE);
+ if (_recoveryPath == null && _conf.getBoolean(INTEGRATION_TESTING, false))
{
Review Comment:
I think its fine to use separate config, we do that in other places as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
