Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/1758#issuecomment-51391446
I captured some logs from the passing and failing commits:
https://gist.github.com/41aeeafddbc4c1770d7b
To obtain these, I ran `sbt/sbt "core/test:test-only
*ConnectionManagerSuite"` after modifying
`core/src/test/resources/log4j.properties` to use a DEBUG log level.
One big difference that jumps out to me is the handler for security
negotiation messages. In the failing test case, I see
```java
14/08/06 13:07:01.390 DEBUG ConnectionManager: This is security neg message
14/08/06 13:07:01.391 DEBUG ConnectionManager: Client handleAuth for id:
joshs-mbp_58889_1
14/08/06 13:07:01.392 ERROR ConnectionManager: Error handling sasl client
authentication
javax.security.sasl.SaslException: DIGEST-MD5: Digest-challenge format
violation: algorithm directive missing
at
com.sun.security.sasl.digest.DigestMD5Client.processChallenge(DigestMD5Client.java:296)
at
com.sun.security.sasl.digest.DigestMD5Client.evaluateChallenge(DigestMD5Client.java:225)
at
org.apache.spark.SparkSaslClient.saslResponse(SparkSaslClient.scala:84)
at
org.apache.spark.network.ConnectionManager.handleClientAuthentication(ConnectionManager.scala:533)
at
org.apache.spark.network.ConnectionManager.handleAuthentication(ConnectionManager.scala:610)
at
org.apache.spark.network.ConnectionManager.org$apache$spark$network$ConnectionManager$$handleMessage(ConnectionManager.scala:639)
at
org.apache.spark.network.ConnectionManager$$anon$9.run(ConnectionManager.scala:511)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```
while in the passing test case I see
```java
14/08/06 13:09:38.269 DEBUG ConnectionManager: This is security neg message
14/08/06 13:09:38.270 DEBUG ConnectionManager: Server handleAuth for id:
joshs-mbp_58931_1
14/08/06 13:09:38.271 DEBUG ConnectionManager: saslContext not established
14/08/06 13:09:38.271 DEBUG ConnectionManager: Creating sasl Server
14/08/06 13:09:38.273 DEBUG ConnectionManager: Server sasl not completed:
joshs-mbp_58932_1
14/08/06 13:09:38.273 DEBUG SecurityMessage: message total size is : 150
14/08/06 13:09:38.273 INFO ConnectionManager: creating new sending
connection for security! joshs-mbp_58932_2
14/08/06 13:09:38.273 DEBUG SendingConnection: Added [BufferMessage(id = 6,
size = 150)] to outbox for sending to [ConnectionManagerId(joshs-mbp,58931)]
14/08/06 13:09:38.273 DEBUG ConnectionManager: Selector selected 0 of 2 keys
14/08/06 13:09:38.273 INFO SendingConnection: Initiating connection to
[joshs-mbp/192.168.1.245:58931]
14/08/06 13:09:38.273 DEBUG ConnectionManager: After handleAuth result was
true, returning
14/08/06 13:09:38.274 DEBUG ConnectionManager: Handling delay is 6 ms
```
If we only have a single ConnectionManager initiating connections, then
shouldn't the logs always show `Server handleAuth for ...` followed by `Client
handleAuth for ...`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
