[
https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152185#comment-15152185
]
Sean Owen commented on SPARK-13371:
-----------------------------------
Heh, I just fired up IntelliJ, which flags this code, and looked for all other
instances of *just this problem*. There are 24. [~gq] do you mind if I open a
PR to fix all of them? You still get credit.
[~joshrosen] I might run a much bigger sweep with IJ and open a PR or two to
fix clear bugs. We talked a long time ago about the value of fixing merely
"suboptimal" code or style issues. You rightly said it wasn't usually worth it
because of merge conflicts. For 2.x, do you think it could be worth some forms
of that cleanup?
> TaskSetManager.dequeueSpeculativeTask compares Option[String] and String
> directly.
> ----------------------------------------------------------------------------------
>
> Key: SPARK-13371
> URL: https://issues.apache.org/jira/browse/SPARK-13371
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 1.5.2, 1.6.0
> Reporter: Guoqiang Li
>
> {noformat}
> TaskSetManager.dequeueSpeculativeTask compares Option[String] and String
> directly.
> {noformat}
> Ths code:
> https://github.com/apache/spark/blob/87abcf7df921a5937fdb2bae8bfb30bfabc4970a/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala#L344
> {code}
> if (TaskLocality.isAllowed(locality, TaskLocality.RACK_LOCAL)) {
> for (rack <- sched.getRackForHost(host)) {
> for (index <- speculatableTasks if canRunOnHost(index)) {
> val racks =
> tasks(index).preferredLocations.map(_.host).map(sched.getRackForHost)
> // racks: Seq[Option[String]] and rack: String
> if (racks.contains(rack)) {
> speculatableTasks -= index
> return Some((index, TaskLocality.RACK_LOCAL))
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
