[
https://issues.apache.org/jira/browse/MAPREDUCE-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shilun Fan updated MAPREDUCE-7545:
----------------------------------
Fix Version/s: 3.5.1
3.6.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> TaskId.hashCode depends on the JVM
> ----------------------------------
>
> Key: MAPREDUCE-7545
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7545
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 3.6.0
> Reporter: Jose Luis López
> Assignee: Jose Luis López
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.1, 3.6.0
>
>
> TaskId.hashCode() mixes in getTaskType().hashCode(). TaskType is an enum, and
> an
> enum's hashCode() is its identity hash. HotSpot draws identity hashes from a
> sequence that shifts with whatever asked for one earlier in the JVM: which
> jars
> are on the classpath, which classes are initialized. So every HashMap keyed by
> TaskId iterates in an order that depends on the environment, not on the tasks.
> TestRuntimeEstimators depends on that order. It builds five slow maps and
> expects three speculations to win. In some orders only one does, and all three
> tests then fail every time on that environment:
> TestRuntimeEstimators.testLegacyEstimator:254->coreTestEstimator:247
> We got the wrong number of successful speculations. ==> expected: <3> but
> was: <1>
> Seen on GitHub Actions (Java 17) on the HADOOP-19970 branch, twice, with the
> surefire reruns failing identically, while the same commit passes locally.
> Reproduced locally by consuming four extra identity hashes before the test
> starts (new Object().hashCode() in a static block), with no other change.
> Fix: hash the task type's ordinal instead of the enum itself, so the hash is
> the same in every JVM. A test in TestIds pins it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
