Re: Review Request: HIVE-1803: Implement bitmap indexing in Hive (new review starting from patch 6)

2011-03-22 Thread Marquis Wang


 On None, John Sichi wrote:
  ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexInputFormat.java,
   line 45
  https://reviews.apache.org/r/481/diff/1/?file=13771#file13771line45
 
  I'm confused about how the backwards compatibility works for the index 
  filename property...who uses this property name?

The property name is set on the command line when the index query is run (see 
the index_compact.q tests). This String is how the class knows where the index 
filename is stored.


 On None, John Sichi wrote:
  ql/build.xml, line 187
  https://reviews.apache.org/r/481/diff/1/?file=13758#file13758line187
 
  Why do you need to unpack the .jar?  And why to json/classes?

I was getting java.lang.NoClassDefFoundError: javaewah/EWAHCompressedBitmap 
errors at runtime without unpacking it. I guess I forgot to change the 
destination to something else when I copied that line. Is unpacking the .jar 
unnecessary? I'm not really familiar with how ivy(?) handles these libraries.


- Marquis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/481/#review315
---


On 2011-03-08 16:27:50, John Sichi wrote:
 
 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/481/
 ---
 
 (Updated 2011-03-08 16:27:50)
 
 
 Review request for hive.
 
 
 Summary
 ---
 
 Review board was giving me grief trying to update the old patch, so I'm 
 creating a fresh review request for HIVE-1803.6
 
 
 This addresses bug HIVE-1803.
 https://issues.apache.org/jira/browse/HIVE-1803
 
 
 Diffs
 -
 
   lib/README 1c2f0b1 
   lib/javaewah-0.2.jar PRE-CREATION 
   ql/build.xml 50c604e 
   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ba222f3 
   ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java ff74f08 
   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java 308d985 
   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexResult.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexedInputFormat.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeTask.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeWork.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/TableBasedIndexHandler.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectInput.java 
 PRE-CREATION 
   ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectOutput.java 
 PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 
 1f01446 
   
 ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexInputFormat.java
  6c320c5 
   
 ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexResult.java
  0c9ccea 
   
 ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeTask.java
  eac168f 
   
 ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeWork.java
  26beb4e 
   ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 
 391e5de 
   ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 77220a1 
   ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 30714b8 
   
 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapOp.java
  PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFEWAHBitmap.java 
 PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapAnd.java
  PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java
  PRE-CREATION 
   
 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapOr.java 
 PRE-CREATION 
   ql/src/test/queries/clientpositive/index_bitmap.q PRE-CREATION 
   ql/src/test/queries/clientpositive/index_bitmap1.q PRE-CREATION 
   ql/src/test/queries/clientpositive/index_bitmap2.q PRE-CREATION 
   ql/src/test/queries/clientpositive/index_bitmap3.q PRE-CREATION 
   ql/src/test/queries/clientpositive/index_compact.q 6547a52 
   ql/src/test/queries/clientpositive/index_compact_1.q 6d59353 
   ql/src/test/queries/clientpositive/index_compact_2.q 358b5e9 
   ql/src/test/queries/clientpositive/index_compact_3.q ee8abda 
   ql/src/test/queries/clientpositive/udf_bitmap_and.q PRE-CREATION 
   ql/src/test/queries/clientpositive/udf_bitmap_or.q PRE-CREATION 
   ql/src/test/results/clientpositive/index_bitmap.q.out PRE-CREATION 
   ql/src/test/results/clientpositive/index_bitmap1.q.out PRE-CREATION 
   ql/src/test/results/clientpositive/index_bitmap2.q.out PRE-CREATION 
   

Review Request: HIVE-1803: Implement bitmap indexing in Hive (new review starting from patch 6)

2011-03-08 Thread John Sichi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/481/
---

Review request for hive.


Summary
---

Review board was giving me grief trying to update the old patch, so I'm 
creating a fresh review request for HIVE-1803.6


This addresses bug HIVE-1803.
https://issues.apache.org/jira/browse/HIVE-1803


Diffs
-

  lib/README 1c2f0b1 
  lib/javaewah-0.2.jar PRE-CREATION 
  ql/build.xml 50c604e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java ba222f3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java ff74f08 
  ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndex.java 308d985 
  ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexResult.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexedInputFormat.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeTask.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/IndexMetadataChangeWork.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/TableBasedIndexHandler.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectInput.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapObjectOutput.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java 
1f01446 
  
ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexInputFormat.java
 6c320c5 
  
ql/src/java/org/apache/hadoop/hive/ql/index/compact/HiveCompactIndexResult.java 
0c9ccea 
  
ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeTask.java
 eac168f 
  
ql/src/java/org/apache/hadoop/hive/ql/index/compact/IndexMetadataChangeWork.java
 26beb4e 
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java 
391e5de 
  ql/src/java/org/apache/hadoop/hive/ql/io/IOContext.java 77220a1 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 30714b8 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFEWAHBitmapOp.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFEWAHBitmap.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapAnd.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapEmpty.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFEWAHBitmapOr.java 
PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap2.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_bitmap3.q PRE-CREATION 
  ql/src/test/queries/clientpositive/index_compact.q 6547a52 
  ql/src/test/queries/clientpositive/index_compact_1.q 6d59353 
  ql/src/test/queries/clientpositive/index_compact_2.q 358b5e9 
  ql/src/test/queries/clientpositive/index_compact_3.q ee8abda 
  ql/src/test/queries/clientpositive/udf_bitmap_and.q PRE-CREATION 
  ql/src/test/queries/clientpositive/udf_bitmap_or.q PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap2.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/index_bitmap3.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/udf_bitmap_and.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/udf_bitmap_or.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/481/diff


Testing
---


Thanks,

John