[
https://issues.apache.org/jira/browse/MAPREDUCE-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18119219#comment-18119219
]
ASF GitHub Bot commented on MAPREDUCE-7545:
-------------------------------------------
joseluisll commented on PR #8754:
URL: https://github.com/apache/hadoop/pull/8754#issuecomment-5833951294
@slfan1989 Would you mind reviewing this one? It's a one-line change plus a
unit test.
TestRuntimeEstimators has started failing on several of my open PRs. None of
those PRs cause it — TaskId.hashCode() varies between JVMs, so the iteration
order of maps keyed by TaskId does too, and the test is sensitive to that
order.
> TaskId.hashCode depends on the JVM
> ----------------------------------
>
> Key: MAPREDUCE-7545
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7545
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Jose Luis López
> Assignee: Jose Luis López
> Priority: Major
> Labels: pull-request-available
>
> TaskId.hashCode() mixes in getTaskType().hashCode(). TaskType is an enum, and
> an
> enum's hashCode() is its identity hash. HotSpot draws identity hashes from a
> sequence that shifts with whatever asked for one earlier in the JVM: which
> jars
> are on the classpath, which classes are initialized. So every HashMap keyed by
> TaskId iterates in an order that depends on the environment, not on the tasks.
> TestRuntimeEstimators depends on that order. It builds five slow maps and
> expects three speculations to win. In some orders only one does, and all three
> tests then fail every time on that environment:
> TestRuntimeEstimators.testLegacyEstimator:254->coreTestEstimator:247
> We got the wrong number of successful speculations. ==> expected: <3> but
> was: <1>
> Seen on GitHub Actions (Java 17) on the HADOOP-19970 branch, twice, with the
> surefire reruns failing identically, while the same commit passes locally.
> Reproduced locally by consuming four extra identity hashes before the test
> starts (new Object().hashCode() in a static block), with no other change.
> Fix: hash the task type's ordinal instead of the enum itself, so the hash is
> the same in every JVM. A test in TestIds pins it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
