[
https://issues.apache.org/jira/browse/DRILL-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khurram Faraaz updated DRILL-3849:
----------------------------------
Fix Version/s: 1.2.0
> IOB Exception : ORDER BY ROW_KEY over date_epoch_be encoded data
> ----------------------------------------------------------------
>
> Key: DRILL-3849
> URL: https://issues.apache.org/jira/browse/DRILL-3849
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.2.0
> Environment: 4 node cluster CentOS
> Reporter: Khurram Faraaz
> Assignee: Smidth Panchamia
> Fix For: 1.2.0
>
>
> Order by ROW_KEY that has DATE type data results in IOB Exception where
> queried from Drill. Note that there is a duplicate in ROW_KEY.
> {code}
> Drill master commitID : b9afcf8f
> [root@centos-01 rowkey_filter_pushdown]# hbase version
> 2015-09-29 04:50:04,510 INFO [main] util.VersionInfo: HBase
> 0.98.12-mapr-1506-SNAPSHOT
> 2015-09-29 04:50:04,511 INFO [main] util.VersionInfo: Subversion
> git://5e8a86c5195a/root/opensource/mapr-hbase-0.98/dl/mapr-hbase-0.98 -r
> 34eede8b74911af09661867c07440888b648c3ba
> 2015-09-29 04:50:04,511 INFO [main] util.VersionInfo: Compiled by root on
> Tue Jul 21 00:24:58 UTC 2015
> {code}
> A regular select without order by on row_key gives correct results.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT
> CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be') AS RK,
> CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T;
> +-------------+-------+
> | RK | cf1 |
> +-------------+-------+
> | 2014-08-25 | val1 |
> | 2015-09-26 | val5 |
> | 2015-09-28 | val4 |
> | 2015-09-29 | val3 |
> | 2015-09-29 | val2 |
> +-------------+-------+
> 5 rows selected (0.79 seconds)
> {code}
> However when encoding 'date_epoch_be' is used in the query and ORDER BY is
> used on ROW_KEY we see the IOB Exception.
> {code}
> 0: jdbc:drill:schema=dfs.tmp> SELECT
> CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be') AS RK,
> CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T ORDER BY
> ROW_KEY;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133,
> cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133,
> unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx:
> 0, cap: 392/392)))).slice(89, 64)
> Fragment 2:0
> [Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
> (state=,code=0)
> 0: jdbc:drill:schema=dfs.tmp>
> {code}
> Stack trace from drillbit.log for the failing query
> {code}
> 2015-09-29 04:33:04,295 [29f5eb81-5696-49fd-b86e-788ed103fec9:frag:2:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133,
> unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped:
> UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap:
> 392/392)))).slice(89, 64)
> Fragment 2:0
> [Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133,
> unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped:
> UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap:
> 392/392)))).slice(89, 64)
> Fragment 2:0
> [Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
> ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.lang.IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133,
> cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133,
> unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx:
> 0, cap: 392/392)))).slice(89, 64)
> at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:115)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
> at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:105)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
> at io.netty.buffer.DrillBuf.slice(DrillBuf.java:348)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
> at
> org.apache.drill.exec.record.RecordBatchLoader.load(RecordBatchLoader.java:102)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.next(UnorderedReceiverBatch.java:186)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:145)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:258)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:252)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_45]
> at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
> ~[hadoop-common-2.5.1-mapr-1503.jar:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:252)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 4 common frames omitted
> {code}
> Snippet used to insert data into base table
> {code}
> Date dt1= new Date(1408924800000L);
> byte[] rowKey1 = ByteBuffer.allocate(8)
> .putLong(dt1.getTime()).array();
> // getTime() will convert date into a `long'
> Put p = new Put(rowKey1);
>
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
> table.put(p);
> Date dt2= new Date(1443500086094L);
> byte[] rowKey2 = ByteBuffer.allocate(8)
> .putLong(dt2.getTime()).array();
> // getTime() will convert date into a `long'
> p = new Put(rowKey2);
>
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val2"));
> table.put(p);
> Date dt3= new Date(1443491446094L);
> byte[] rowKey3 = ByteBuffer.allocate(8)
> .putLong(dt3.getTime()).array();
> // getTime() will convert date into a `long'
> p = new Put(rowKey3);
>
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val3"));
> table.put(p);
> Date dt4= new Date(1443413686094L);
> byte[] rowKey4 = ByteBuffer.allocate(8)
> .putLong(dt4.getTime()).array();
> // getTime() will convert date into a `long'
> p = new Put(rowKey4);
>
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val4"));
> table.put(p);
> Date dt5= new Date(1443240886094L);
> byte[] rowKey5 = ByteBuffer.allocate(8)
> .putLong(dt5.getTime()).array();
> // getTime() will convert date into a `long'
> p = new Put(rowKey5);
>
> p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val5"));
> table.put(p);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
