[
https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059890#comment-17059890
]
Takeshi Yamamuro commented on SPARK-17495:
------------------------------------------
Ah, I see. I've looked over the merged PRs and, in the branch-2.4 (the master,
too), it seems the hive hash function is not used yet in
partitioning/bucketing. So, we need more work to use the hash there. This Jira
just targets the implementation of the hive-simulated hash, so I think we can
close this now.
> Hive hash implementation
> ------------------------
>
> Key: SPARK-17495
> URL: https://issues.apache.org/jira/browse/SPARK-17495
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Tejas Patil
> Assignee: Tejas Patil
> Priority: Minor
> Fix For: 2.2.0
>
>
> Spark internally uses Murmur3Hash for partitioning. This is different from
> the one used by Hive. For queries which use bucketing this leads to different
> results if one tries the same query on both engines. For us, we want users to
> have backward compatibility to that one can switch parts of applications
> across the engines without observing regressions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
