Hello everyone , Please I am having issues querying some parquet files generated using scoop1 in Apache Drill.
I checked the logs and I see the following exception everywhere Aug 7, 2018 4:28:09 PM WARNING: org.apache.parquet.CorruptStatistics: Ignoring statistics because created_by could not be parsed (see PARQUET-251): parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb) org.apache.parquet.VersionParser$VersionParseException: Could not parse created_by: parquet-mr (build 6aa21f8776625b5fa6b18059cfebe7549f2e00cb) using format: (.+) version ((.*) )?\(build ?(.*)\) at org.apache.parquet.VersionParser.parse(VersionParser.java:112) at org.apache.parquet.CorruptStatistics.shouldIgnoreStatistics(CorruptStatistics.java:66) at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetStatistics(ParquetMetadataConverter.java:264) at org.apache.parquet.format.converter.ParquetMetadataConverter.fromParquetMetadata(ParquetMetadataConverter.java:568) at org.apache.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata(ParquetMetadataConverter.java:545) at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:455) at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:412) at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:435) at org.apache.drill.exec.store.parquet.Metadata$1.run(Metadata.java:428) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633) at org.apache.drill.exec.store.parquet.Metadata.getParquetFileMetadata_v3(Metadata.java:428) at org.apache.drill.exec.store.parquet.Metadata.access$100(Metadata.java:96) at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:364) at org.apache.drill.exec.store.parquet.Metadata$MetadataGatherer.runInner(Metadata.java:352) at org.apache.drill.exec.store.TimedRunnable.run(TimedRunnable.java:56) at org.apache.drill.exec.store.TimedRunnable$LatchedRunnable.run(TimedRunnable.java:98) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Please what can I do to mitigate against this ________________________________ Peter Edike Senior Software Engineer Research and Development, ENG Engineering [cid:image002.png@01D42E71.8D117260] Office NO: Mobile NO: Email: peter.ed...@interswitchgroup.com<mailto:peter.ed...@interswitchgroup.com> Interswitch Limited 1648C Oko-Awo Street, Victoria Island Lagos Customer Contact Centre 0700-9065000 ? http://www.interswitchgroup.com<http://www.interswitchgroup.com/> [cid:image003.png@01D42E71.8D117260]<https://www.quickteller.com/loan-request> This e-mail and all attachments transmitted with it remain the property of Interswitch Limited , the information contained herein are private confidential and intended solely for the use of the addressee. If you have received this e-mail in error, kindly notify the sender. If you are not the addressee, you should not disseminate, distribute or copy this e-mail. Kindly notify Interswitch immediately by email if you have received this email in error and delete this email and any attachment from your system Emails cannot be guaranteed to be secure or error free as the message and any attachments could be intercepted, corrupted, lost, delayed, incomplete or amended. the contents of this email or its attachments have been scanned for all viruses and all reasonable measures have been taken to ensure that no viruses are present. Interswitch Limited and its subsidiaries do not accept liability for damage caused by this email or any attachments.This message has been marked as CONFIDENTIAL on Tuesday, August 7, 2018 @ 5:10:31 PM