Khurram Faraaz created DRILL-3849:
-------------------------------------
Summary: IOB Exception : ORDER BY ROW_KEY over date_epoch_be
encoded data
Key: DRILL-3849
URL: https://issues.apache.org/jira/browse/DRILL-3849
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 1.2.0
Environment: 4 node cluster CentOS
Reporter: Khurram Faraaz
Assignee: Smidth Panchamia
Order by ROW_KEY that has DATE type data results in IOB Exception where queried
from Drill. Note that there is a duplicate in ROW_KEY.
{code}
Drill master commitID : b9afcf8f
[root@centos-01 rowkey_filter_pushdown]# hbase version
2015-09-29 04:50:04,510 INFO [main] util.VersionInfo: HBase
0.98.12-mapr-1506-SNAPSHOT
2015-09-29 04:50:04,511 INFO [main] util.VersionInfo: Subversion
git://5e8a86c5195a/root/opensource/mapr-hbase-0.98/dl/mapr-hbase-0.98 -r
34eede8b74911af09661867c07440888b648c3ba
2015-09-29 04:50:04,511 INFO [main] util.VersionInfo: Compiled by root on Tue
Jul 21 00:24:58 UTC 2015
{code}
A regular select without order by on row_key gives correct results.
{code}
0: jdbc:drill:schema=dfs.tmp> SELECT
CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be') AS RK,
CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T;
+-------------+-------+
| RK | cf1 |
+-------------+-------+
| 2014-08-25 | val1 |
| 2015-09-26 | val5 |
| 2015-09-28 | val4 |
| 2015-09-29 | val3 |
| 2015-09-29 | val2 |
+-------------+-------+
5 rows selected (0.79 seconds)
{code}
However when encoding 'date_epoch_be' is used in the query and ORDER BY is used
on ROW_KEY we see the IOB Exception.
{code}
0: jdbc:drill:schema=dfs.tmp> SELECT
CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be') AS RK,
CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T ORDER BY ROW_KEY;
Error: SYSTEM ERROR: IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133,
cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133,
unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0,
cap: 392/392)))).slice(89, 64)
Fragment 2:0
[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
(state=,code=0)
0: jdbc:drill:schema=dfs.tmp>
{code}
Stack trace from drillbit.log for the failing query
{code}
2015-09-29 04:33:04,295 [29f5eb81-5696-49fd-b86e-788ed103fec9:frag:2:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IndexOutOfBoundsException:
DrillBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0,
widx: 133, cap: 133/133, unwrapped:
UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap:
392/392)))).slice(89, 64)
Fragment 2:0
[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133,
unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped:
UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap:
392/392)))).slice(89, 64)
Fragment 2:0
[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.lang.IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133,
cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133,
unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0,
cap: 392/392)))).slice(89, 64)
at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:115)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:105)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.slice(DrillBuf.java:348)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
at
org.apache.drill.exec.record.RecordBatchLoader.load(RecordBatchLoader.java:102)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.next(UnorderedReceiverBatch.java:186)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:145)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:258)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:252)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.7.0_45]
at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
~[hadoop-common-2.5.1-mapr-1503.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:252)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
... 4 common frames omitted
{code}
Snippet used to insert data into base table
{code}
Date dt1= new Date(1408924800000L);
byte[] rowKey1 = ByteBuffer.allocate(8) .putLong(dt1.getTime()).array();
// getTime() will convert date into a `long'
Put p = new Put(rowKey1);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
table.put(p);
Date dt2= new Date(1443500086094L);
byte[] rowKey2 = ByteBuffer.allocate(8) .putLong(dt2.getTime()).array();
// getTime() will convert date into a `long'
p = new Put(rowKey2);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val2"));
table.put(p);
Date dt3= new Date(1443491446094L);
byte[] rowKey3 = ByteBuffer.allocate(8) .putLong(dt3.getTime()).array();
// getTime() will convert date into a `long'
p = new Put(rowKey3);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val3"));
table.put(p);
Date dt4= new Date(1443413686094L);
byte[] rowKey4 = ByteBuffer.allocate(8) .putLong(dt4.getTime()).array();
// getTime() will convert date into a `long'
p = new Put(rowKey4);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val4"));
table.put(p);
Date dt5= new Date(1443240886094L);
byte[] rowKey5 = ByteBuffer.allocate(8) .putLong(dt5.getTime()).array();
// getTime() will convert date into a `long'
p = new Put(rowKey5);
p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val5"));
table.put(p);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
