[jira] [Commented] (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13023513#comment-13023513 ] jirapos...@reviews.apache.org commented on HIVE-1644: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/558/ --- (Updated 2011-04-23 06:41:49.007802) Review request for hive. Changes --- HIVE-1644.16.patch Summary --- Review request for HIVE-1644.12.patch This addresses bug HIVE-1644. https://issues.apache.org/jira/browse/HIVE-1644 Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2cdaeb6 conf/hive-default.xml 79ea477 ql/src/java/org/apache/hadoop/hive/ql/Driver.java ca337a8 ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 69ee03b ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java c02d90b ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexHandler.java dd0186d ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java 411b78f ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexQueryContext.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 50db44c ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 6162676 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexWhereResolver.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 0ae9fa2 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcCtx.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 374e123 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java c41bb32 ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 73391e9 ql/src/test/queries/clientpositive/index_auto.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_file_format.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_multiple.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_partitioned.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_test_if_used.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_unused.q PRE-CREATION ql/src/test/results/clientpositive/index_auto.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_file_format.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_multiple.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_partitioned.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_test_if_used.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_unused.q.out PRE-CREATION Diff: https://reviews.apache.org/r/558/diff Testing --- Thanks, Russell use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.10.patch, HIVE-1644.11.patch, HIVE-1644.12.patch, HIVE-1644.13.patch, HIVE-1644.14.patch, HIVE-1644.15.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch, HIVE-1644.5.patch, HIVE-1644.6.patch, HIVE-1644.7.patch, HIVE-1644.8.patch, HIVE-1644.9.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13023514#comment-13023514 ] jirapos...@reviews.apache.org commented on HIVE-1644: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/558/#review533 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java https://reviews.apache.org/r/558/#comment1110 I thought this might what caused the original table to be used, instead of the stale index. By adding the index table, we keep the original table around. However, clearing the inputs before adding the index table didn't change anything. ql/src/test/results/clientpositive/index_auto_test_if_used.q.out https://reviews.apache.org/r/558/#comment1109 We shouldn't be seeing this output. We're still generating the right plan, but something is wrong when we run it. - Russell On 2011-04-23 06:41:49, Russell Melick wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/558/ bq. --- bq. bq. (Updated 2011-04-23 06:41:49) bq. bq. bq. Review request for hive. bq. bq. bq. Summary bq. --- bq. bq. Review request for HIVE-1644.12.patch bq. bq. bq. This addresses bug HIVE-1644. bq. https://issues.apache.org/jira/browse/HIVE-1644 bq. bq. bq. Diffs bq. - bq. bq.common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2cdaeb6 bq.conf/hive-default.xml 79ea477 bq.ql/src/java/org/apache/hadoop/hive/ql/Driver.java ca337a8 bq.ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 69ee03b bq.ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java c02d90b bq.ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexHandler.java dd0186d bq.ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java 411b78f bq.ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexQueryContext.java PRE-CREATION bq. ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446 bq.ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 50db44c bq.ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 6162676 bq. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexWhereResolver.java PRE-CREATION bq. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 0ae9fa2 bq. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcCtx.java PRE-CREATION bq. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java PRE-CREATION bq. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java PRE-CREATION bq.ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 374e123 bq.ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java c41bb32 bq.ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 73391e9 bq.ql/src/test/queries/clientpositive/index_auto.q PRE-CREATION bq.ql/src/test/queries/clientpositive/index_auto_file_format.q PRE-CREATION bq.ql/src/test/queries/clientpositive/index_auto_multiple.q PRE-CREATION bq.ql/src/test/queries/clientpositive/index_auto_partitioned.q PRE-CREATION bq.ql/src/test/queries/clientpositive/index_auto_test_if_used.q PRE-CREATION bq.ql/src/test/queries/clientpositive/index_auto_unused.q PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto.q.out PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto_file_format.q.out PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto_multiple.q.out PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto_partitioned.q.out PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto_test_if_used.q.out PRE-CREATION bq.ql/src/test/results/clientpositive/index_auto_unused.q.out PRE-CREATION bq. bq. Diff: https://reviews.apache.org/r/558/diff bq. bq. bq. Testing bq. --- bq. bq. bq. Thanks, bq. bq. Russell bq. bq. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.10.patch, HIVE-1644.11.patch, HIVE-1644.12.patch,
[jira] [Updated] (HIVE-1644) use filter pushdown for automatically accessing indexes
[ https://issues.apache.org/jira/browse/HIVE-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Melick updated HIVE-1644: - Attachment: HIVE-1644.16.patch Patch 16 One last problem. It doesn't appear the index is actually used. I tried looking a little bit at the hadoop logs, but I don't really know what I'm looking for. I left a comment on the reviewboard. use filter pushdown for automatically accessing indexes --- Key: HIVE-1644 URL: https://issues.apache.org/jira/browse/HIVE-1644 Project: Hive Issue Type: Improvement Components: Indexing Affects Versions: 0.8.0 Reporter: John Sichi Assignee: Russell Melick Attachments: HIVE-1644.1.patch, HIVE-1644.10.patch, HIVE-1644.11.patch, HIVE-1644.12.patch, HIVE-1644.13.patch, HIVE-1644.14.patch, HIVE-1644.15.patch, HIVE-1644.16.patch, HIVE-1644.2.patch, HIVE-1644.3.patch, HIVE-1644.4.patch, HIVE-1644.5.patch, HIVE-1644.6.patch, HIVE-1644.7.patch, HIVE-1644.8.patch, HIVE-1644.9.patch HIVE-1226 provides utilities for analyzing filters which have been pushed down to a table scan. The next step is to use these for selecting available indexes and generating access plans for those indexes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-1644 Use filter pushdown for automatically accessing indexes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/558/#review533 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java https://reviews.apache.org/r/558/#comment1110 I thought this might what caused the original table to be used, instead of the stale index. By adding the index table, we keep the original table around. However, clearing the inputs before adding the index table didn't change anything. ql/src/test/results/clientpositive/index_auto_test_if_used.q.out https://reviews.apache.org/r/558/#comment1109 We shouldn't be seeing this output. We're still generating the right plan, but something is wrong when we run it. - Russell On 2011-04-23 06:41:49, Russell Melick wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/558/ --- (Updated 2011-04-23 06:41:49) Review request for hive. Summary --- Review request for HIVE-1644.12.patch This addresses bug HIVE-1644. https://issues.apache.org/jira/browse/HIVE-1644 Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2cdaeb6 conf/hive-default.xml 79ea477 ql/src/java/org/apache/hadoop/hive/ql/Driver.java ca337a8 ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 69ee03b ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java c02d90b ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexHandler.java dd0186d ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java 411b78f ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexQueryContext.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 50db44c ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 6162676 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexWhereResolver.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 0ae9fa2 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcCtx.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 374e123 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java c41bb32 ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 73391e9 ql/src/test/queries/clientpositive/index_auto.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_file_format.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_multiple.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_partitioned.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_test_if_used.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_unused.q PRE-CREATION ql/src/test/results/clientpositive/index_auto.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_file_format.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_multiple.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_partitioned.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_test_if_used.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_unused.q.out PRE-CREATION Diff: https://reviews.apache.org/r/558/diff Testing --- Thanks, Russell
Re: Review Request: HIVE-1644 Use filter pushdown for automatically accessing indexes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/558/ --- (Updated 2011-04-23 06:41:49.007802) Review request for hive. Changes --- HIVE-1644.16.patch Summary --- Review request for HIVE-1644.12.patch This addresses bug HIVE-1644. https://issues.apache.org/jira/browse/HIVE-1644 Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 2cdaeb6 conf/hive-default.xml 79ea477 ql/src/java/org/apache/hadoop/hive/ql/Driver.java ca337a8 ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 69ee03b ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java c02d90b ql/src/java/org/apache/hadoop/hive/ql/index/AbstractIndexHandler.java dd0186d ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java 411b78f ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexQueryContext.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 1f01446 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 50db44c ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java 6162676 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexWhereResolver.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 0ae9fa2 ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcCtx.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 374e123 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java c41bb32 ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 73391e9 ql/src/test/queries/clientpositive/index_auto.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_file_format.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_multiple.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_partitioned.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_test_if_used.q PRE-CREATION ql/src/test/queries/clientpositive/index_auto_unused.q PRE-CREATION ql/src/test/results/clientpositive/index_auto.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_file_format.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_multiple.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_partitioned.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_test_if_used.q.out PRE-CREATION ql/src/test/results/clientpositive/index_auto_unused.q.out PRE-CREATION Diff: https://reviews.apache.org/r/558/diff Testing --- Thanks, Russell
[jira] [Commented] (HIVE-1803) Implement bitmap indexing in Hive
[
https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13023519#comment-13023519
]
John Sichi commented on HIVE-1803:
--
Everything passed...eeexcept TestParse, where I got 44 failures. Example diff
below; somehow the output seems to have gotten reordered, and it's in the area
of virtual columns. Best is to find out why and fix it, otherwise make sure
it's deterministic and then update .q.out.
{noformat}
[junit] diff -b -I'\(\(java version=.* class=java.beans.XMLDecoder\)\|
\(string.*/tmp/.*/string\)\|\(stringfile:.*/string\)\|\(stringpfile:.*
/string\)\|\(string[0-9]\{10\}/string\)\|\(string/.*/warehouse/.*/strin
g\)\)' /data/users/jsichi/open/test-trunk/build/ql/test/logs/positive/case_sens
itivity.q.xml /data/users/jsichi/open/test-trunk/ql/src/test/results/compiler/pl
an/case_sensitivity.q.xml
[junit] 1209c1209
[junit] stringINPUT__FILE__NAME/string
[junit] ---
[junit] stringBLOCK__OFFSET__INSIDE__FILE/string
[junit] 1215c1215,1219
[junit] object idref=PrimitiveTypeInfo0/
[junit] ---
[junit] object class=org.apache.hadoop.hive.serde2.ty
peinfo.PrimitiveTypeInfo
[junit] void property=typeName
[junit] stringbigint/string
[junit] /void
[junit] /object
[junit] 1225c1229
[junit] stringBLOCK__OFFSET__INSIDE__FILE/string
[junit] ---
[junit] stringINPUT__FILE__NAME/string
[junit] 1231,1235c1235
[junit] object class=org.apache.hadoop.hive.serde2.ty
peinfo.PrimitiveTypeInfo
[junit] void property=typeName
[junit] stringbigint/string
{noformat}
Implement bitmap indexing in Hive
-
Key: HIVE-1803
URL: https://issues.apache.org/jira/browse/HIVE-1803
Project: Hive
Issue Type: New Feature
Components: Indexing
Reporter: Marquis Wang
Assignee: Marquis Wang
Attachments: HIVE-1803.1.patch, HIVE-1803.10.patch,
HIVE-1803.11.patch, HIVE-1803.12.patch, HIVE-1803.13.patch,
HIVE-1803.2.patch, HIVE-1803.3.patch, HIVE-1803.4.patch, HIVE-1803.5.patch,
HIVE-1803.6.patch, HIVE-1803.7.patch, HIVE-1803.8.patch, HIVE-1803.9.patch,
JavaEWAH_20110304.zip, bitmap_index_1.png, bitmap_index_2.png, javaewah.jar,
javaewah.jar, unit-tests.2.patch, unit-tests.3.patch, unit-tests.patch
Implement bitmap index handler to complement compact indexing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: Removed finalizePartition() from the patch
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/648/ --- Review request for hive and Carl Steinbach. Summary --- 1. Removed finalizePartition(). Will file separate jira for it. 2. Added container objects for different event types. 3. Changed MetaStoreEventListener from interface to abstract class. 4. Modifications to allow a list of listeners instead of just one. This addresses bug HIVE-2038. https://issues.apache.org/jira/browse/HIVE-2038 Diffs - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096112 trunk/conf/hive-default.xml 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/CreateDatabaseEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/CreateTableEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropDatabaseEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropTableEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/ListenerEvent.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java PRE-CREATION Diff: https://reviews.apache.org/r/648/diff Testing --- Thanks, Ashutosh
[jira] [Commented] (HIVE-2038) Metastore listener
[ https://issues.apache.org/jira/browse/HIVE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13023521#comment-13023521 ] jirapos...@reviews.apache.org commented on HIVE-2038: - --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/648/ --- Review request for hive and Carl Steinbach. Summary --- 1. Removed finalizePartition(). Will file separate jira for it. 2. Added container objects for different event types. 3. Changed MetaStoreEventListener from interface to abstract class. 4. Modifications to allow a list of listeners instead of just one. This addresses bug HIVE-2038. https://issues.apache.org/jira/browse/HIVE-2038 Diffs - trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096112 trunk/conf/hive-default.xml 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreEventListener.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1096112 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/CreateDatabaseEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/CreateTableEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropDatabaseEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropTableEvent.java PRE-CREATION trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/ListenerEvent.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyListener.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java PRE-CREATION Diff: https://reviews.apache.org/r/648/diff Testing --- Thanks, Ashutosh Metastore listener -- Key: HIVE-2038 URL: https://issues.apache.org/jira/browse/HIVE-2038 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.8.0 Attachments: hive-2038.patch, metastore_listener.patch, metastore_listener.patch, metastore_listener.patch Provide to way to observe changes happening on Metastore -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.7.0-h0.20 #84
See https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/84/ -- [...truncated 26904 lines...] [junit] Loading data to table default.srcbucket2 [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/srcbucket23.txt' INTO TABLE srcbucket2 [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@srcbucket2 [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' INTO TABLE src [junit] PREHOOK: type: LOAD [junit] Copying data from https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Copying file: https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt [junit] Loading data to table default.src [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' INTO TABLE src [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src [junit] OK [junit] Copying file: https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt' INTO TABLE src1 [junit] PREHOOK: type: LOAD [junit] Copying data from https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt [junit] Loading data to table default.src1 [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv3.txt' INTO TABLE src1 [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src1 [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq' INTO TABLE src_sequencefile [junit] PREHOOK: type: LOAD [junit] Copying data from https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq [junit] Copying file: https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq [junit] Loading data to table default.src_sequencefile [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.seq' INTO TABLE src_sequencefile [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_sequencefile [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq' INTO TABLE src_thrift [junit] PREHOOK: type: LOAD [junit] Copying data from https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq [junit] Copying file: https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq [junit] Loading data to table default.src_thrift [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/complex.seq' INTO TABLE src_thrift [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_thrift [junit] OK [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt' INTO TABLE src_json [junit] PREHOOK: type: LOAD [junit] Copying data from https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt [junit] Copying file: https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt [junit] Loading data to table default.src_json [junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/json.txt' INTO TABLE src_json [junit] POSTHOOK: type: LOAD [junit] POSTHOOK: Output: default@src_json [junit] OK [junit] diff https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/test/logs/negative/wrong_distinct1.q.out https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/ql/src/test/results/compiler/errors/wrong_distinct1.q.out [junit] Done query: wrong_distinct1.q [junit] Hive history file=https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/tmp/hive_job_log_hudson_201104231210_983765139.txt [junit] Begin query: wrong_distinct2.q [junit] Hive history file=https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/build/ql/tmp/hive_job_log_hudson_201104231210_1282196077.txt [junit] PREHOOK: query: LOAD DATA LOCAL INPATH 'https://builds.apache.org/hudson/job/Hive-0.7.0-h0.20/ws/hive/data/files/kv1.txt' OVERWRITE INTO TABLE srcpart PARTITION (ds='2008-04-08',hr='11') [junit] PREHOOK: type: LOAD [junit] Copying data from
Jenkins build is back to normal : Hive-trunk-h0.20 #687
See https://builds.apache.org/hudson/job/Hive-trunk-h0.20/687/
[jira] [Updated] (HIVE-1803) Implement bitmap indexing in Hive
[
https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marquis Wang updated HIVE-1803:
---
Attachment: HIVE-1803.14.patch
HIVE-1803.14.patch
The issue with the last patch was the order in which
VirtualColumn.getRegistry().iterator() was returning. The old code stored the
virtual column registry as a HashMap, so I added the columns to the registry in
the order the HashMap would have returned them.
This patch fixes that. I'm still seeing errors in groupby1.q through
groupby6.q. It looks like various numbers are returning wrong, but it doesn't
appear to be related to the virtual columns. I can't really tell whether there
is a pattern to it.
can you take a look?
{noformat}
[junit] stringCNTR_NAME_GBY_28_NUM_INPUT_ROWS/string
[junit] 1345c1341
[junit] stringCNTR_NAME_GBY_4_NUM_OUTPUT_ROWS/string
[junit] ---
[junit] stringCNTR_NAME_GBY_28_NUM_OUTPUT_ROWS/string
[junit] 1348c1344
[junit] stringCNTR_NAME_GBY_4_TIME_TAKEN/string
[junit] ---
[junit] stringCNTR_NAME_GBY_28_TIME_TAKEN/string
[junit] 1351c1347
[junit] stringCNTR_NAME_GBY_4_FATAL_ERROR/string
[junit] ---
[junit] stringCNTR_NAME_GBY_28_FATAL_ERROR/string
{/noformat}
Implement bitmap indexing in Hive
-
Key: HIVE-1803
URL: https://issues.apache.org/jira/browse/HIVE-1803
Project: Hive
Issue Type: New Feature
Components: Indexing
Reporter: Marquis Wang
Assignee: Marquis Wang
Attachments: HIVE-1803.1.patch, HIVE-1803.10.patch,
HIVE-1803.11.patch, HIVE-1803.12.patch, HIVE-1803.13.patch,
HIVE-1803.14.patch, HIVE-1803.14.patch, HIVE-1803.2.patch, HIVE-1803.3.patch,
HIVE-1803.4.patch, HIVE-1803.5.patch, HIVE-1803.6.patch, HIVE-1803.7.patch,
HIVE-1803.8.patch, HIVE-1803.9.patch, JavaEWAH_20110304.zip,
bitmap_index_1.png, bitmap_index_2.png, javaewah.jar, javaewah.jar,
unit-tests.2.patch, unit-tests.3.patch, unit-tests.patch
Implement bitmap index handler to complement compact indexing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1803) Implement bitmap indexing in Hive
[ https://issues.apache.org/jira/browse/HIVE-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13023634#comment-13023634 ] John Sichi commented on HIVE-1803: -- The number comes from the operator ID; I'm not sure why those would have shifted, but if it's consistent, then give me a patch that just updates the .q.out files and if that passes for me, we'll call it a day. Implement bitmap indexing in Hive - Key: HIVE-1803 URL: https://issues.apache.org/jira/browse/HIVE-1803 Project: Hive Issue Type: New Feature Components: Indexing Reporter: Marquis Wang Assignee: Marquis Wang Attachments: HIVE-1803.1.patch, HIVE-1803.10.patch, HIVE-1803.11.patch, HIVE-1803.12.patch, HIVE-1803.13.patch, HIVE-1803.14.patch, HIVE-1803.14.patch, HIVE-1803.2.patch, HIVE-1803.3.patch, HIVE-1803.4.patch, HIVE-1803.5.patch, HIVE-1803.6.patch, HIVE-1803.7.patch, HIVE-1803.8.patch, HIVE-1803.9.patch, JavaEWAH_20110304.zip, bitmap_index_1.png, bitmap_index_2.png, javaewah.jar, javaewah.jar, unit-tests.2.patch, unit-tests.3.patch, unit-tests.patch Implement bitmap index handler to complement compact indexing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
