"Failure while running fragment."
Here's an example doing it the way you suggest. (Last 100 lines of
sqlline.log are also attached.) It sounds like you think the multi-file
query should actually work?
Thanks
0: jdbc:drill:zk=local> select * from dfs.`/data` limit 5;
+------------+
| columns |
+------------+
<first 5 rows of data from first file>
Query failed: Query failed: Failure while running fragment., refCnt: 0,
decrement: 1 [ 29674d0f-bec3-4175-b1df-3afb68c6e76f on
sandbox.hortonworks.com:31010 ]
[ 29674d0f-bec3-4175-b1df-3afb68c6e76f on sandbox.hortonworks.com:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
On Sun, Feb 8, 2015 at 3:21 PM, Neeraja Rentachintala <
[email protected]> wrote:
> What is the error that you are seeing?
> Can you simply point it to the directory (without *.csv) to see if it
> helps.
>
>
> On Sun, Feb 8, 2015 at 10:33 AM, Minnow Noir <[email protected]> wrote:
>
> > I'm trying to do ad-hoc exploration/analysis over multiple files without
> > having to concatenate them. New files show up on a regular basis, and
> > creating large, redundant concatenated files seems inelegant for data
> > exploration. I've tried the obvious (... from dfs.`/dir/*.csv` but that
> > only returns lines from the first file it finds, and then an error for
> the
> > next file.
> >
> > Is there any current way to do this?
> >
> >
> > Thanks
> >
>
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
[netty-common-4.0.24.Final.jar:4.0.24.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-02-08 20:26:18,888 [BitClient-2] INFO o.a.drill.exec.rpc.data.DataClient
- Channel closed between local 0.0.0.0/0.0.0.0:53549 and remote
sandbox.hortonworks.com/10.0.2.15:31012
2015-02-08 20:26:18,889 [2b283994-bc45-a24e-db33-fccbe35962e1:frag:0:0] WARN
o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
fragment
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at
io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:101)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:820)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at
io.netty.buffer.UnsafeDirectLittleEndian.release(UnsafeDirectLittleEndian.java:25)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.work.batch.UnlimitedRawBatchBuffer.kill(UnlimitedRawBatchBuffer.java:127)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.kill(UnorderedReceiverBatch.java:107)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.kill(IteratorValidatorBatchIterator.java:86)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:98)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:114)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-02-08 20:26:18,890 [2b283994-bc45-a24e-db33-fccbe35962e1:frag:0:0] ERROR
o.a.drill.exec.ops.FragmentContext - Fragment Context received failure.
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at
io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:101)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:820)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at
io.netty.buffer.UnsafeDirectLittleEndian.release(UnsafeDirectLittleEndian.java:25)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.work.batch.UnlimitedRawBatchBuffer.kill(UnlimitedRawBatchBuffer.java:127)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.kill(UnorderedReceiverBatch.java:107)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.kill(IteratorValidatorBatchIterator.java:86)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:98)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:114)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-02-08 20:26:18,891 [2b283994-bc45-a24e-db33-fccbe35962e1:frag:0:0] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
29674d0f-bec3-4175-b1df-3afb68c6e76f: Failure while running fragment.
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at
io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:101)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at io.netty.buffer.WrappedByteBuf.release(WrappedByteBuf.java:820)
~[netty-buffer-4.0.24.Final.jar:4.0.24.Final]
at
io.netty.buffer.UnsafeDirectLittleEndian.release(UnsafeDirectLittleEndian.java:25)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.release(DrillBuf.java:206)
~[drill-java-exec-0.7.0-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.work.batch.UnlimitedRawBatchBuffer.kill(UnlimitedRawBatchBuffer.java:127)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.kill(UnorderedReceiverBatch.java:107)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.kill(IteratorValidatorBatchIterator.java:86)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:98)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:97)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:114)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-02-08 20:26:18,912 [2b283994-bc45-a24e-db33-fccbe35962e1:frag:0:0] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
11f7e25f-941f-4515-81a2-fe702e5d8c5b: Query failed: Failure while running
fragment., refCnt: 0, decrement: 1 [ 29674d0f-bec3-4175-b1df-3afb68c6e76f on
sandbox.hortonworks.com:31010 ]
[ 29674d0f-bec3-4175-b1df-3afb68c6e76f on sandbox.hortonworks.com:31010 ]
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running fragment.,
refCnt: 0, decrement: 1 [ 29674d0f-bec3-4175-b1df-3afb68c6e76f on
sandbox.hortonworks.com:31010 ]
[ 29674d0f-bec3-4175-b1df-3afb68c6e76f on sandbox.hortonworks.com:31010 ]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:93)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:151)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:113)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:109)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:166)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:129)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
