[
https://issues.apache.org/jira/browse/HIVE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045578#comment-13045578
]
[email protected] commented on HIVE-2036:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/#review773
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java
<https://reviews.apache.org/r/857/#comment1666>
Update Javadoc and param name, including an explanation of what handler is
supposed to do when multiple indexes are passed in.
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
<https://reviews.apache.org/r/857/#comment1675>
I'm confused by the logic here. You are throwing together all of the
columns for all of the indexes, but we need to keep them segregated, don't we?
Each subquery should only contain references to the columns relevant to the
corresponding index.
(But the partitioning predicates need to be applied to each index.)
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java
<https://reviews.apache.org/r/857/#comment1668>
Why is this public instead of private?
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java
<https://reviews.apache.org/r/857/#comment1667>
Use HiveUtils.unparseIdentifier
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/HiveBitmapIndexInputFormat.java
<https://reviews.apache.org/r/857/#comment1669>
Why do we need this class at all? The superclass already uses
hive.index.blockfilter.file by default.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
<https://reviews.apache.org/r/857/#comment1672>
Seems like we should only be looking at the indexes on the table accessed
by this table scan. (This comment is retroactive to the original version of
the file.)
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
<https://reviews.apache.org/r/857/#comment1673>
Seems like the costing comment below applies to this too.
ql/src/test/queries/clientpositive/index_bitmap3.q
<https://reviews.apache.org/r/857/#comment1670>
Why do we need this setting at all? (I'm not sure why it was there in the
original version of the file.)
- John
On 2011-06-06 21:37:38, Syed Albiz wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/857/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-06 21:37:38)
bq.
bq.
bq. Review request for hive and John Sichi.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Add support for generating index queries to support automatic usage of
bitmap indexes. This required changing the interface to the IndexHandlers to
support accepting queries on multiple indexes. The compact indexes were
modified to use this new interface as well, although no functional changes were
made to how they work. Only supports AND predicates right now, but it should be
possibly to extend the BitmapQuery interface defined in this patch to easily
support OR predicates as well. Currently benchmarking these changes on a test
cluster.
bq.
bq.
bq. This addresses bug HIVE-2036.
bq. https://issues.apache.org/jira/browse/HIVE-2036
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845
bq. ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java
e5ee183
bq.
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
af9d7b1
bq. ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java
PRE-CREATION
bq. ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java
PRE-CREATION
bq. ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java
PRE-CREATION
bq.
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/HiveBitmapIndexInputFormat.java
PRE-CREATION
bq.
ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java
56e7609
bq. ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
d64e88b
bq.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
268560d
bq.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
0873e1a
bq. ql/src/test/queries/clientpositive/index_bitmap3.q 508eb94
bq. ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION
bq. ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/857/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Passes unit tests, additional testcase to test automatic bitmap indexing
index_bitmap_auto.q was also added to the TestCliDriver suite. Currently
benchmarking changes on a test cluster.
bq.
bq.
bq. Thanks,
bq.
bq. Syed
bq.
bq.
> Update bitmap indexes for automatic usage
> -----------------------------------------
>
> Key: HIVE-2036
> URL: https://issues.apache.org/jira/browse/HIVE-2036
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
> Affects Versions: 0.8.0
> Reporter: Russell Melick
> Assignee: Syed S. Albiz
> Attachments: HIVE-2036.1.patch
>
>
> HIVE-1644 will provide automatic usage of indexes, and HIVE-1803 adds bitmap
> index support. The bitmap code will need to be extended after it is
> committed to enable automatic use of indexing. Most work will be focused in
> the BitmapIndexHandler, which needs to generate the re-entrant QL index
> query. There may also be significant work in the IndexPredicateAnalyzer to
> support predicates with OR's, instead of just AND's as it is currently.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
