Arina Ielchiieva created DRILL-5767:
---------------------------------------
Summary: Fix error when querying data with the same column names
as implicit columns
Key: DRILL-5767
URL: https://issues.apache.org/jira/browse/DRILL-5767
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.11.0
Reporter: Arina Ielchiieva
Fix For: 1.12.0
Data file {{suffix.csvh}}:
{noformat}
id, suffix
1, json
2, parquet
{noformat}
The data file contains [one of four implicit
columns|https://drill.apache.org/docs/querying-a-file-system-introduction/],
namely suffix. User may override implicit column name using system / session
property ({{ALTER session SET `drill.exec.storage.implicit.suffix.column.label`
= 'appendix'}}).
But currently select from such data set returns an error:
{noformat}
2017-09-04 10:24:58,946 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
26530034-9c9a-05ed-1ae3-2e086b49f196: select * from
dfs.`D:\drill\files\suffix.csvh`
2017-09-04 10:24:58,963 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-09-04 10:24:58,964 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,964 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,964 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,976 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-09-04 10:24:58,977 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,977 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,977 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:24:58,994 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:58,995 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:59,000 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:24:59,001 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
localhost. Skipping affinity to that host.
2017-09-04 10:24:59,001 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Time: 0ms total, 0.297648ms avg, 0ms max.
2017-09-04 10:24:59,001 [26530034-9c9a-05ed-1ae3-2e086b49f196:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Earliest start: 0.428000 ?s, Latest start: 0.428000 ?s, Average
start: 0.428000 ?s .
2017-09-04 10:24:59,015 [26530034-9c9a-05ed-1ae3-2e086b49f196:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 26530034-9c9a-05ed-1ae3-2e086b49f196:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2017-09-04 10:24:59,015 [26530034-9c9a-05ed-1ae3-2e086b49f196:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 26530034-9c9a-05ed-1ae3-2e086b49f196:0:0:
State to report: RUNNING
2017-09-04 10:24:59,017 [26530034-9c9a-05ed-1ae3-2e086b49f196:frag:0:0] INFO
o.a.d.e.s.e.t.c.CompliantTextRecordReader - User Error Occurred: Error
processing input: , line=2, char=22. Content parsed: [ ] (Error processing
input: , line=2, char=22. Content parsed: [ ])
org.apache.drill.common.exceptions.UserException: DATA_READ ERROR: Error
processing input: , line=2, char=22. Content parsed: [ ]
Failure while reading file file:/D:/drill/files/suffix.csvh. Happened at or
shortly before byte position 22.
[Error Id: 77052479-abff-41bc-bc1c-87a230b207e7 ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.CompliantTextRecordReader.next(CompliantTextRecordReader.java:218)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:180)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:133)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
[na:1.7.0_71]
at javax.security.auth.Subject.doAs(Subject.java:415) [na:1.7.0_71]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: com.univocity.parsers.common.TextParsingException: Error processing
input: , line=2, char=22. Content parsed: [ ]
at
org.apache.drill.exec.store.easy.text.compliant.TextReader.handleException(TextReader.java:480)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.TextReader.parseNext(TextReader.java:389)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.CompliantTextRecordReader.next(CompliantTextRecordReader.java:212)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
... 19 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: index: 0, length: 4 (expected:
range(0, 0))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:123)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:147)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:520)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at
org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:396)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.VarCharVector$Mutator.setSafe(VarCharVector.java:615)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.FieldVarCharOutput.endField(FieldVarCharOutput.java:193)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.TextReader.parseField(TextReader.java:325)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.TextReader.parseRecord(TextReader.java:141)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.text.compliant.TextReader.parseNext(TextReader.java:370)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
... 20 common frames omitted
2017-09-04 10:24:59,017 [26530034-9c9a-05ed-1ae3-2e086b49f196:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 26530034-9c9a-05ed-1ae3-2e086b49f196:0:0:
State change requested RUNNING --> FAILED
2017-09-04 10:24:59,017 [26530034-9c9a-05ed-1ae3-2e086b49f196:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 26530034-9c9a-05ed-1ae3-2e086b49f196:0:0:
State change requested FAILED --> FINISHED
2017-09-04 10:24:59,023 [Client-1] INFO o.a.d.j.i.DrillCursor$ResultsListener
- [#17] Query failed:
org.apache.drill.common.exceptions.UserRemoteException: DATA_READ ERROR: Error
processing input: , line=2, char=22. Content parsed: [ ]
Failure while reading file file:/D:/drill/files/suffix.csvh. Happened at or
shortly before byte position 22.
Fragment 0:0
[Error Id: 77052479-abff-41bc-bc1c-87a230b207e7 on ArinaAsus:31010]
at
org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:368)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:90)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:274)
[drill-rpc-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:244)
[drill-rpc-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
[netty-handler-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
2017-09-04 10:24:59,044 [BitServer-4] WARN o.a.drill.exec.work.foreman.Foreman
- Dropping request to move to COMPLETED state as query is already at FAILED
state (which is terminal).
2017-09-04 10:24:59,045 [BitServer-4] WARN o.a.d.e.w.b.ControlMessageHandler -
Dropping request to cancel fragment. 26530034-9c9a-05ed-1ae3-2e086b49f196:0:0
does not exist.
{noformat}
The same is for parquet file:
{noformat}
2017-09-04 10:26:01,562 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
2652fff5-c89a-3b8c-e77e-ec01d0f939ac: select * from t
2017-09-04 10:26:01,579 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-09-04 10:26:01,580 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,580 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,580 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,593 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 7 classes for
org.apache.drill.exec.store.dfs.FormatPlugin took 0ms
2017-09-04 10:26:01,593 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,593 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,593 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.c.s.persistence.ScanResult - loading 8 classes for
org.apache.drill.common.logical.FormatPluginConfig took 0ms
2017-09-04 10:26:01,609 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:26:01,609 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatuses() took 0 ms,
numFiles: 1
2017-09-04 10:26:01,615 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 0 ms to get file statuses
2017-09-04 10:26:01,616 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1
using 1 threads. Time: 0ms total, 0.841625ms avg, 0ms max.
2017-09-04 10:26:01,616 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of 1
using 1 threads. Earliest start: 0.855000 ?s, Latest start: 0.855000 ?s,
Average start: 0.855000 ?s .
2017-09-04 10:26:01,616 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Took 1 ms to read file metadata
2017-09-04 10:26:01,629 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 2652fff5-c89a-3b8c-e77e-ec01d0f939ac:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2017-09-04 10:26:01,630 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 2652fff5-c89a-3b8c-e77e-ec01d0f939ac:0:0:
State to report: RUNNING
2017-09-04 10:26:01,632 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:frag:0:0] ERROR
o.a.d.exec.physical.impl.ScanBatch - SYSTEM ERROR: IndexOutOfBoundsException:
index: 0, length: 4 (expected: range(0, 0))
[Error Id: c57b2845-ea49-472b-b42b-dc9653405801 ]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
[Error Id: c57b2845-ea49-472b-b42b-dc9653405801 ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:550)
~[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:249)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:133)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:105)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:95)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
[na:1.7.0_71]
at javax.security.auth.Subject.doAs(Subject.java:415) [na:1.7.0_71]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error in
parquet record reader.
Message:
Hadoop path:
/tmp/f8e5598d-7dad-4ed6-8a42-49f9946c2624/b96b1dd4-ac66-4caf-91d6-c0d2ac063c96/0_0_0.parquet
Total records read: 0
Row group index: 0
Records in row group: 2
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message root {
required binary id (UTF8);
required binary suffix (UTF8);
}
, metadata: {drill.version=1.12.0-SNAPSHOT, drill-writer.version=2}}, blocks:
[BlockMetaData{2, 98 [ColumnMetaData{SNAPPY [id] BINARY [PLAIN, BIT_PACKED],
4}, ColumnMetaData{SNAPPY [suffix] BINARY [PLAIN, BIT_PACKED], 43}]}]}
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.handleException(ParquetRecordReader.java:272)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:299)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:180)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
... 18 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: index: 0, length: 4 (expected:
range(0, 0))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:123)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:147)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:520)
~[drill-memory-base-1.12.0-SNAPSHOT.jar:4.0.27.Final]
at
org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:396)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.vector.VarCharVector$Mutator.setValueLengthSafe(VarCharVector.java:652)
~[vector-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.VarLengthValuesColumn.readAndStoreValueSizeInformation(VarLengthValuesColumn.java:100)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.VarLengthColumn.processPageData(VarLengthColumn.java:49)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.determineSize(ColumnReader.java:191)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.VarLenBinaryReader.determineSizesSerial(VarLenBinaryReader.java:95)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.VarLenBinaryReader.readFields(VarLenBinaryReader.java:76)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.BatchReader$VariableWidthReader.readRecords(BatchReader.java:166)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.BatchReader.readBatch(BatchReader.java:42)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.next(ParquetRecordReader.java:297)
~[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
... 19 common frames omitted
2017-09-04 10:26:01,632 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 2652fff5-c89a-3b8c-e77e-ec01d0f939ac:0:0:
State change requested RUNNING --> FAILED
2017-09-04 10:26:01,633 [2652fff5-c89a-3b8c-e77e-ec01d0f939ac:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 2652fff5-c89a-3b8c-e77e-ec01d0f939ac:0:0:
State change requested FAILED --> FINISHED
2017-09-04 10:26:01,640 [Client-1] INFO o.a.d.j.i.DrillCursor$ResultsListener
- [#18] Query failed:
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
Fragment 0:0
[Error Id: c57b2845-ea49-472b-b42b-dc9653405801 on ArinaAsus:31010]
at
org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:368)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:90)
[drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:274)
[drill-rpc-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:244)
[drill-rpc-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
[netty-handler-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
2017-09-04 10:26:01,940 [BitServer-4] WARN o.a.drill.exec.work.foreman.Foreman
- Dropping request to move to COMPLETED state as query is already at FAILED
state (which is terminal).
2017-09-04 10:26:01,941 [BitServer-4] WARN o.a.d.e.w.b.ControlMessageHandler -
Dropping request to cancel fragment. 2652fff5-c89a-3b8c-e77e-ec01d0f939ac:0:0
does not exist.
{noformat}
On Apache Drill 1.9 both queries return the data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
