Hi, Couple of questions if I may:
1. Is there a plan to support Filters which are derived from the where clause? 2. Many people tend to place multiple fields concatenated in the row-key, or even one. If there's no place to define the rowkey schema, how can the storage engine use a where clause on the rowkey fields to minimize the amount of information scan? 3. Does Drill have any where to define a schema of the fields in HBase in advance, so there won't be any need for casting? On Sun, Mar 9, 2014 at 10:02 PM, Steven Phillips <[email protected]>wrote: > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java, > line 79 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512198#file512198line79> > > > > > > Is is possible to get columns back without ROW_KEY? In this case > it won't be added. > > The approach I am taking here is to include a column for the row key in > these cases: > > The column "row_key" is included as one of the selected columns > > No columns are selected, in which case all columns are returned, including > one for "row_key" > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java, > line 167 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512198#file512198line167> > > > > > > When will this be null? > > In the case where we are not including the "row_key" column in the output, > the rowKeyVector will be null. > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java, > line 266 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512198#file512198line266> > > > > > > Are we only supporting VarBinary type from HBase? > > HBase does not natively support any types other than VarBinary. To specify > a type for a column, a user will have to cast the column to the appropriate > type. > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java, > line 286 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512198#file512198line286> > > > > > > Remove, and don't need to clean up ResultScanner? > > This bug is already fixed in my branch, and I will updating the patch > shortly. > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java, > line 99 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512222#file512222line99> > > > > > > Is this a fix? > > Yes, that is a fix, but it's not really part of this patch, I think there > should be a separate Jira for that. > > > > On March 9, 2014, 4:10 p.m., Timothy Chen wrote: > > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseGroupScan.java, > line 138 > > > < > https://reviews.apache.org/r/18841/diff/1/?file=512197#file512197line138> > > > > > > Seems a bit odd to have a class variable but populate it every get > call. I think either check null or just make it return a new list everytime > as it seems to do that already > > You are right, but I believe this method will only be called once. But I > should make the code more clear. > > > - Steven > > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18841/#review36603 > ----------------------------------------------------------- > > > On March 6, 2014, 9:24 a.m., Steven Phillips wrote: > > > > ----------------------------------------------------------- > > This is an automatically generated e-mail. To reply, visit: > > https://reviews.apache.org/r/18841/ > > ----------------------------------------------------------- > > > > (Updated March 6, 2014, 9:24 a.m.) > > > > > > Review request for drill. > > > > > > Bugs: DRILL-88 > > https://issues.apache.org/jira/browse/DRILL-88 > > > > > > Repository: drill-git > > > > > > Description > > ------- > > > > DRILL-88 HBase storage engine > > > > > > Start HBase storage engine. > > > > > > Diffs > > ----- > > > > contrib/storage-hbase/pom.xml 7860ea5e7bbf884b4590021ed3024d25b9d8ed7d > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseGroupScan.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseScanBatchCreator.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseSchemaFactory.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePlugin.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePluginConfig.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseSubScan.java > PRE-CREATION > > > contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HTableReadEntry.java > PRE-CREATION > > contrib/storage-hbase/src/main/resources/drill-module.conf PRE-CREATION > > > contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseRecordReaderTest.java > PRE-CREATION > > > contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseTestsSuite.java > PRE-CREATION > > > contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestTableGenerator.java > PRE-CREATION > > contrib/storage-hbase/src/test/resources/hbase-site.xml PRE-CREATION > > > contrib/storage-hbase/src/test/resources/hbase/hbase_scan_screen_physical.json > PRE-CREATION > > > contrib/storage-hbase/src/test/resources/hbase/hbase_scan_screen_physical_column_select.json > PRE-CREATION > > > contrib/storage-hbase/src/test/resources/hbase/hbase_scan_screen_physical_family_select.json > PRE-CREATION > > contrib/storage-hbase/src/test/resources/logback.xml PRE-CREATION > > distribution/pom.xml ffbac0f3a9851fc6d05c9d4b0d5b029558ce267b > > distribution/src/assemble/bin.xml > 4e37c6a9053e827cc800b3e2ebeb491c3a2c67fb > > distribution/src/resources/drill-config.sh > 7be5badea76771f5d9a20cc7bb674be6d277eba5 > > distribution/src/resources/runbit > 85daf8742ab559fe6588a2fd013ea2117e55bbd3 > > distribution/src/resources/sqlline > c526f03213cd438efce36332ca04f8dc1b4a390b > > distribution/src/resources/submit_plan > 3c9184a00762a965e0006a1a296908b7796cf8eb > > exec/java-exec/src/main/codegen/templates/NullableValueVectors.java > 051c62d213451fd9973c47b052cc9fc0ee188d6a > > exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java > 5cd83afecc24a1fba3d5b566d565d460235e5e63 > > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java > f76e59a6ec3efd897e083ea4a755971a4fedcf13 > > exec/java-exec/src/test/resources/sender/hash_exchange.json > fc45ec19b196ec8d61e5971cf3338ae93015638c > > > > Diff: https://reviews.apache.org/r/18841/diff/ > > > > > > Testing > > ------- > > > > > > Thanks, > > > > Steven Phillips > > > > > >
