Jose Luis López created HADOOP-19978:
----------------------------------------
Summary: Upgrade maven-surefire-plugin to 3.6
Key: HADOOP-19978
URL: https://issues.apache.org/jira/browse/HADOOP-19978
Project: Hadoop Common
Issue Type: Improvement
Components: common, test
Reporter: Jose Luis López
h3. Why
HADOOP-19964 reimplements TimedOutTestsListener as a JUnit Platform
TestExecutionListener, so a timed-out test prints a thread dump again. The
{{listener}} provider property that eight poms already carry binds to it only
once Surefire accepts platform listeners there — SUREFIRE-1639, shipped in
maven-surefire-plugin 3.6.0. Until Hadoop moves to that version the listener
is dormant.
h3. What blocks the upgrade
Bumping {{maven-surefire-plugin.version}} from 3.5.3 to 3.6.0 turns 5 of 7 CI
test jobs red: common, mr, hdfs-rbf, hdfs-other, hdfs-slow. Controlled against
another branch built the same morning on the same runners with 3.5.3, which
was 7/7 green. Every failure is one fault — the forked JVM preferring IPv6:
{code:java}
TestNameNodeRpcServer.testNamenodeRpcBindAny expected: <0.0.0.0> but was:
<0:0:0:0:0:0:0:0>
TestHSAdminServer not a valid host:port authority: 0:0:0:0:0:0:0:0:33745
TestBlockRecovery2 not a valid host:port authority: 0:0:0:0:0:0:0:0:39677
TestDNS ArrayIndexOutOfBounds Index 3 out of bounds for length 1
TestWebHdfsTokens Connect localhost:44787: Connection refused
{code}
hadoop-project.xml sets {{java.net.preferIPv4Stack=true}} in surefire's
{{{}systemPropertyVariables{}}}. Those are applied by the forked booter, not as
{{-D}} at JVM startup — in 3.5.3 and 3.6.0 alike. Class-load traces
({{{}-Xlog:class+load{}}}) show SystemPropertyManager running at index 699 under
3.5.3 and 736 under 3.6.0; in that gap 3.6.0 loads {{sun.nio.ch.IOUtil}} and
{{{}sun.nio.ch.SocketChannelImpl{}}}, where 3.5.3 loads no networking classes at
all. 3.6.0 initialises the fork's networking before the property lands, so the
flag is read too late.
Reported upstream as apache/maven-surefire#3456.
h3. Changes
* Bump {{maven-surefire-plugin.version}} to 3.6.0.
* Move {{-Djava.net.preferIPv4Stack=true}} into
{{maven-surefire-plugin.argLine}} so it is a real JVM startup argument. All
22 {{<argLine>}} overrides in the tree already interpolate that property, and
maven-failsafe-plugin shares it, so it propagates everywhere. No
property-based fix can work: Surefire initialises networking before applying
any of them.
* Add hadoop-common's test-jar in test scope to hadoop-yarn-services-core and
hadoop-yarn-applications-catalog-webapp. Both inherit the {{listener}}
property from hadoop-yarn but depend only on the main jar, so with the
listener live Surefire cannot resolve the class and the fork dies before any
test runs. Verified: without the dependency the module reports "Tests run: 0"
and the fork terminates; with it, tests pass.
h3. Notes
Only {{java.net.preferIPv4Stack}} is affected. Checking the other
systemPropertyVariables against the same criterion — is anything initialised
before the properties are applied? — {{{}java.security.egd{}}},
{{{}java.security.krb5.conf{}}}, {{{}hadoop.log.dir{}}}, {{{}hadoop.tmp.dir{}}},
{{test.build.*}} and {{require.test.libhadoop}} are all read later and are
unaffected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
