LuciferYang opened a new pull request, #36876: URL: https://github.com/apache/spark/pull/36876
### What changes were proposed in this pull request? This PR aims to use `Utils.localHostNameForURI` instead of `Utils.localCanonicalHostName` in the following suites which changed in https://github.com/apache/spark/pull/36866 - `MasterSuite` - `MasterWebUISuite` - `RocksDBBackendHistoryServerSuite` ### Why are the changes needed? These test cases fails when we run with `SPARK_LOCAL_IP=::1` and `-Djava.net.preferIPv6Addresses=true` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GA - Manual test: 1. `export SPARK_LOCAL_IP=::1` ``` echo $SPARK_LOCAL_IP ::1 ``` 2. add `-Djava.net.preferIPv6Addresses=true` to MAVEN_OPTS, for example: ``` diff --git a/pom.xml b/pom.xml index 1ce3b43faf..3356622985 100644 --- a/pom.xml +++ b/pom.xml @@ -2943,7 +2943,7 @@ <include>**/*Suite.java</include> </includes> <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory> - <argLine>-ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=${CodeCacheSize} ${extraJavaTestArgs} -Dio.netty.tryReflectionSetAccessible=true</argLine> + <argLine>-ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=${CodeCacheSize} ${extraJavaTestArgs} -Dio.netty.tryReflectionSetAccessible=true -Djava.net.preferIPv6Addresses=true</argLine> <environmentVariables> <!-- Setting SPARK_DIST_CLASSPATH is a simple way to make sure any child processes ``` 3. maven test `RocksDBBackendHistoryServerSuite`, `MasterSuite` and `MasterWebUISuite` ``` mvn clean install -DskipTests -pl core -am mvn clean test -pl core -Dtest=none -DwildcardSuites=org.apache.spark.deploy.history.RocksDBBackendHistoryServerSuite mvn clean test -pl core -Dtest=none -DwildcardSuites=org.apache.spark.deploy.master.MasterSuite mvn clean test -pl core -Dtest=none -DwildcardSuites=org.apache.spark.deploy.master.ui.MasterWebUISuite ``` **Before** RocksDBBackendHistoryServerSuite: ``` - Redirect to the root page when accessed to /history/ *** FAILED *** java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:613) at java.net.Socket.connect(Socket.java:561) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ... Run completed in 31 seconds, 745 milliseconds. Total number of tests run: 73 Suites: completed 2, aborted 0 Tests: succeeded 3, failed 70, canceled 0, ignored 0, pending 0 *** 70 TESTS FAILED *** ``` MasterSuite: ``` - master/worker web ui available behind front-end reverseProxy *** FAILED *** The code passed to eventually never returned normally. Attempted 487 times over 50.079685917 seconds. Last failure message: Connection refused (Connection refused). (MasterSuite.scala:405) Run completed in 3 minutes, 48 seconds. Total number of tests run: 32 Suites: completed 2, aborted 0 Tests: succeeded 29, failed 3, canceled 0, ignored 0, pending 0 *** 3 TESTS FAILED *** ``` MasterWebUISuite: ``` - Kill multiple hosts *** FAILED *** java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:613) at java.net.Socket.connect(Socket.java:561) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ... Run completed in 7 seconds, 83 milliseconds. Total number of tests run: 4 Suites: completed 2, aborted 0 Tests: succeeded 0, failed 4, canceled 0, ignored 0, pending 0 *** 4 TESTS FAILED *** ``` **After** RocksDBBackendHistoryServerSuite: ``` Run completed in 38 seconds, 205 milliseconds. Total number of tests run: 73 Suites: completed 2, aborted 0 Tests: succeeded 73, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` MasterSuite: ``` Run completed in 1 minute, 10 seconds. Total number of tests run: 32 Suites: completed 2, aborted 0 Tests: succeeded 32, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` MasterWebUISuite: ``` Run completed in 6 seconds, 330 milliseconds. Total number of tests run: 4 Suites: completed 2, aborted 0 Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
