[
https://issues.apache.org/jira/browse/HIVE-1721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063501#comment-13063501
]
Siying Dong commented on HIVE-1721:
-----------------------------------
Andrew, what do you mean by "the filter could be built in parallel with an MR
job"? Our initial plan was to only build filter based on smaller tables and
apply the filter against the big table to reduce data to be shuffled.
For the syntax, the plan is to use syntax like MAPJOIN. We can do something
like SELECT /*+ BLOOMFILTER(t1) +*/ ... FROM t1 JOIN t2 ...
> use bloom filters to improve the performance of joins
> -----------------------------------------------------
>
> Key: HIVE-1721
> URL: https://issues.apache.org/jira/browse/HIVE-1721
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: J. Andrew Key
> Labels: optimization
>
> In case of map-joins, it is likely that the big table will not find many
> matching rows from the small table.
> Currently, we perform a hash-map lookup for every row in the big table, which
> can be pretty expensive.
> It might be useful to try out a bloom-filter containing all the elements in
> the small table.
> Each element from the big table is first searched in the bloom filter, and
> only in case of a positive match,
> the small table hash table is explored.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
