Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18706#discussion_r129161633
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -243,7 +243,6 @@ public void
initializeApplication(ApplicationInitializationContext context) {
String appId = context.getApplicationId().toString();
try {
ByteBuffer shuffleSecret = context.getApplicationDataForService();
- logger.info("Initializing application {}", appId);
--- End diff --
Yep, here's an edited portion of the log (without this patch, so notice the
duplicate registration messages too):
```
2017-07-24 14:38:11,839 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing container
container_1500931912895_0001_01_000002
2017-07-24 14:38:11,839 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing application
application_1500931912895_0001
2017-07-24 14:38:11,864 INFO
org.apache.spark.network.sasl.ShuffleSecretManager: Registered shuffle secret
for application application_1500931912895_0001
2017-07-24 14:38:13,506 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing container
container_1500931912895_0001_01_000008
2017-07-24 14:38:13,506 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing application
application_1500931912895_0001
2017-07-24 14:38:13,507 INFO
org.apache.spark.network.sasl.ShuffleSecretManager: Registered shuffle secret
for application application_1500931912895_0001
2017-07-24 14:38:13,542 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing container
container_1500931912895_0001_01_000007
2017-07-24 14:38:13,542 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing application
application_1500931912895_0001
2017-07-24 14:38:13,542 INFO
org.apache.spark.network.sasl.ShuffleSecretManager: Registered shuffle secret
for application application_1500931912895_0001
2017-07-24 14:38:14,393 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing container
container_1500931912895_0001_01_000010
2017-07-24 14:38:14,394 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing application
application_1500931912895_0001
2017-07-24 14:38:14,394 INFO
org.apache.spark.network.sasl.ShuffleSecretManager: Registered shuffle secret
for application application_1500931912895_0001
2017-07-24 14:38:14,394 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing container
container_1500931912895_0001_01_000011
2017-07-24 14:38:14,394 INFO
org.apache.spark.network.yarn.YarnShuffleService: Initializing application
application_1500931912895_0001
2017-07-24 14:38:14,395 INFO
org.apache.spark.network.sasl.ShuffleSecretManager: Registered shuffle secret
for application application_1500931912895_0001
2017-07-24 14:39:13,454 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping container
container_1500931912895_0001_01_000007
2017-07-24 14:39:13,512 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping container
container_1500931912895_0001_01_000011
2017-07-24 14:39:13,614 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping container
container_1500931912895_0001_01_000008
2017-07-24 14:39:13,614 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping container
container_1500931912895_0001_01_000010
2017-07-24 14:39:13,616 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping container
container_1500931912895_0001_01_000002
2017-07-24 14:39:13,627 INFO
org.apache.spark.network.yarn.YarnShuffleService: Stopping application
application_1500931912895_0001
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
