Hello everyone ,
Please I am having issues querying some parquet files generated using scoop1 in
Apache Drill.
I checked the logs and I see the following exception everywhere
Aug 7, 2018 4:28:09 PM WARNING: org.apache.parquet.CorruptStatistics: Ignoring
statistics because created_by could not be parsed (see PARQUET-251): parquet-mr
(build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb)
org.apache.parquet.VersionParser$VersionParseException: Could not parse
created_by: parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb) using
format: (.+) version ((.*) )?\(build ?(.*)\)
at org.apache.parquet.VersionParser.parse(VersionParser.java:112)
at
org.apache.parquet.CorruptStatistics.shouldIgnoreStatistics(CorruptStatistics.java:66)
at
org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetStatistics(ParquetMetadataConverter.java:264)
at
org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetMetadata(ParquetMetadataConverter.java:568)
at
org.apache.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata(ParquetMetadataConverter.java:545)
at
org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:455)
at
org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:412)
at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:435)
at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633)
at
org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3(Metadata.java:428)
at
org.apache.drill.exec.store.parquet.Metadata.access$100(Metadata.java:96)
at
org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:364)
at
org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:352)
at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:56)
at
org.apache.drill.exec.store.TimedRunnable$LatchedRunnable.run(TimedRunnable.java:98)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Please what can I do to mitigate against this
________________________________
Peter Edike
Senior Software Engineer
Research and Development, ENG
Engineering
[cid:[email protected]]
Office NO:
Mobile NO:
Email: [email protected]<mailto:[email protected]>
Interswitch Limited
1648C Oko-Awo Street, Victoria Island Lagos
Customer Contact Centre 0700-9065000
? http://www.interswitchgroup.com<http://www.interswitchgroup.com/>
[cid:[email protected]]<https://www.quickteller.com/loan-request>
This e-mail and all attachments transmitted with it remain the property of
Interswitch Limited , the information contained herein are private
confidential and intended solely for the use of the addressee. If you have
received this e-mail in error, kindly notify the sender. If you are not the
addressee, you should not disseminate, distribute or copy this e-mail. Kindly
notify Interswitch immediately by email if you have received this email in
error and delete this email and any attachment from your system Emails cannot
be guaranteed to be secure or error free as the message and any attachments
could be intercepted, corrupted, lost, delayed, incomplete or amended. the
contents of this email or its attachments have been scanned for all viruses and
all reasonable measures have been taken to ensure that no viruses are present.
Interswitch Limited and its subsidiaries do not accept liability for damage
caused by this email or any attachments.This message has been marked as
CONFIDENTIAL on Tuesday, August 7, 2018 @ 5:10:31 PM
