[
https://issues.apache.org/jira/browse/DRILL-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16489988#comment-16489988
]
ASF GitHub Bot commented on DRILL-6353:
---------------------------------------
parthchandra commented on a change in pull request #1259: DRILL-6353: Upgrade
Parquet MR dependencies
URL: https://github.com/apache/drill/pull/1259#discussion_r190758787
##########
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetMetadataCache.java
##########
@@ -737,6 +738,7 @@ public void testBooleanPartitionPruning() throws Exception
{
}
}
+ @Ignore
Review comment:
Hmm, we need to take a look at this. For a period of two years, files
written by tools using Parquet libraries were writing incorrect statistics, but
because Drill had its own build where we had fixed the issue (we found the
issue in the first place), files written by Drill were correct. A very large
number of Drill users use the Parquet files produced by Drill and it was
decided that we cannot penalize them. We provided a migration tool to users to
tag files produced by Drill. The tool added information in the extra metadata
in Parquet files to indicate the file was written by Drill and stats from these
files should be allowed.
AFAIK, this should be in the current build of Drill Parquet as well as in
the Parquet library v 1.8.2 and above.
Do you know if the stats corruption that affects these tests is something
that was fixed in a version after 1.8.2?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Upgrade Parquet MR dependencies
> -------------------------------
>
> Key: DRILL-6353
> URL: https://issues.apache.org/jira/browse/DRILL-6353
> Project: Apache Drill
> Issue Type: Task
> Reporter: Vlad Rozov
> Assignee: Vlad Rozov
> Priority: Major
> Fix For: 1.14.0
>
>
> Upgrade from a custom build {{1.8.1-drill-r0}} to Apache release {{1.10.0}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
