sure, I can try disabling sync parquet reader. Will this however, impact the performance of queries on parquet data ?
On Thu, Oct 12, 2017 at 9:39 AM, Kunal Khatua <[email protected]> wrote: > If this resolves the issue, could you share some additional details, such > as the metadata of the Parquet files, the OS, etc.? Details describing the > setup is also very helpful in identifying what could be the cause of the > error. > > We had observed some similar DATA_READ errors in the early iterations of > the Async Parquet reader, but those have been resolved. I'm presuming > you're already on the latest (i.e. Apache Drill 1.11.0) > > -----Original Message----- > From: Arjun kr [mailto:[email protected]] > Sent: Wednesday, October 11, 2017 6:52 PM > To: [email protected] > Subject: Re: Exception while reading parquet data > > > Can you try disabling async parquet reader to see if problem gets resolved. > > > alter session set `store.parquet.reader.pagereader.async`=false; > > Thanks, > > Arjun > > > ________________________________ > From: PROJJWAL SAHA <[email protected]> > Sent: Wednesday, October 11, 2017 2:20 PM > To: [email protected] > Subject: Exception while reading parquet data > > I get below exception when querying parquet data on Oracle Storage Cloud > service. > Any pointers on what does this point to ? > > Regards, > Projjwal > > > ERROR o.a.d.e.u.f.BufferedDirectBufInputStream - Error reading from > stream part-00006-25a9ae4b-fd9e-4770-b17e-9a29b270a4c2.parquet. Error was > : null > 2017-10-09 09:42:18,516 [scan-2] INFO o.a.d.e.s.p.c.AsyncPageReader - > User Error Occurred: Exception occurred while reading from disk. > (java.lang.IndexOutOfBoundsException) > org.apache.drill.common.exceptions.UserException: DATA_READ ERROR: > Exception occurred while reading from disk. > > File: > /data25GB/storereturns/part-00006-25a9ae4b-fd9e-4770-b17e- > 9a29b270a4c2.parquet > Column: sr_return_time_sk > Row Group Start: 479751 > > [Error Id: 10680bb8-d1d6-43a1-b5e0-ef15bd8a9406 ] at > org.apache.drill.common.exceptions.UserException$ > Builder.build(UserException.java:550) > ~[drill-common-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader. > handleAndThrowException(AsyncPageReader.java:185) > [drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.store.parquet.columnreaders. > AsyncPageReader.access$700(AsyncPageReader.java:82) > [drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$ > AsyncPageReaderTask.call(AsyncPageReader.java:461) > [drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$ > AsyncPageReaderTask.call(AsyncPageReader.java:381) > [drill-java-exec-1.11.0.jar:1.11.0] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_121] at > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > [na:1.8.0_121] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] Caused by: > java.io.IOException: java.lang.IndexOutOfBoundsException > at > org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream. > getNextBlock(BufferedDirectBufInputStream.java:185) > ~[drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream. > readInternal(BufferedDirectBufInputStream.java:212) > ~[drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream.read( > BufferedDirectBufInputStream.java:277) > ~[drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.util.filereader.DirectBufInputStream.getNext( > DirectBufInputStream.java:111) > ~[drill-java-exec-1.11.0.jar:1.11.0] > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader$ > AsyncPageReaderTask.call(AsyncPageReader.java:421) > [drill-java-exec-1.11.0.jar:1.11.0] > ... 5 common frames omitted > Caused by: java.lang.IndexOutOfBoundsException: null at > java.nio.Buffer.checkBounds(Buffer.java:567) ~[na:1.8.0_121] at > java.nio.ByteBuffer.put(ByteBuffer.java:827) ~[na:1.8.0_121] at > java.nio.DirectByteBuffer.put(DirectByteBuffer.java:379) ~[na:1.8.0_121] > at > org.apache.parquet.hadoop.util.CompatibilityUtil.getBuf( > CompatibilityUtil.java:110) > ~[parquet-hadoop-1.8.1-drill-r0.jar:1.8.1-drill-r0] > at > org.apache.drill.exec.util.filereader.BufferedDirectBufInputStream. > getNextBlock(BufferedDirectBufInputStream.java:182) > ~[drill-java-exec-1.11.0.jar:1.11.0] > ... 9 common frames omitted > 2017-10-09 09:42:20,533 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3] > INFO o.a.d.e.w.fragment.FragmentExecutor - > 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State change requested > AWAITING_ALLOCATION --> RUNNING > 2017-10-09 09:42:20,533 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3] > INFO o.a.d.e.w.f.FragmentStatusReporter - > 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State to report: RUNNING > 2017-10-09 09:42:20,534 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3] > INFO o.a.d.e.w.fragment.FragmentExecutor - > 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State change requested RUNNING > --> CANCELLATION_REQUESTED > 2017-10-09 09:42:20,534 [26248359-2fc8-d177-c3a6-507f6857e0ea:frag:2:3] > INFO o.a.d.e.w.f.FragmentStatusReporter - > 26248359-2fc8-d177-c3a6-507f6857e0ea:2:3: State to report: > CANCELLATION_REQUESTED >
