[
https://issues.apache.org/jira/browse/HIVE-13066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152252#comment-15152252
]
Rui Li commented on HIVE-13066:
-------------------------------
I'm not able to reproduce the issue. But I tried to make the task fail if the
output file already exists. And speculative task fails as expected, which means
it's possible that speculative tasks write to the same file as original tasks
and corrupt the results. Therefore I think we should append the real attempt ID
for each task, if that doesn't cause too much trouble.
Meanwhile we still need a way to verify the root cause of the issue. Any ideas?
> Hive on Spark gives incorrect results when speculation is on
> ------------------------------------------------------------
>
> Key: HIVE-13066
> URL: https://issues.apache.org/jira/browse/HIVE-13066
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Rui Li
> Assignee: Rui Li
>
> The issue is reported by users. One possible reason is that we always append
> 0 as the attempt ID for each task so that hive won't be able to distinguish
> between speculative tasks and original ones.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
