[
https://issues.apache.org/jira/browse/DRILL-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420082#comment-16420082
]
Paul Rogers commented on DRILL-6223:
------------------------------------
[~sachouche], agree completely that we should fix bugs that can be fixed. The
rub is that most of the schema change "bugs" are actually inherent ambiguities
in the input data. For those that are not ambiguous, then there is a single
answer, and so it is a bug that can be fixed. But, if file A has an INT and
file B has a VARCHAR, there is no a priori way to know which is "correct" (or
if, say, both should really be converted to DECIMAL.)
This is a challenging assignment because of these ambiguities. I wrestled with
the issues with JSON and in the scan operators; there is a long list of cases
for which there is no good answer without a schema.
On the other hand, if the user presents a collection of files, all with the
same schema, then Drill should certainly work as there is no ambiguity.
Looking forward to reviewing the code.
> Drill fails on Schema changes
> ------------------------------
>
> Key: DRILL-6223
> URL: https://issues.apache.org/jira/browse/DRILL-6223
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.10.0, 1.12.0
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Fix For: 1.14.0
>
>
> Drill Query Failing when selecting all columns from a Complex Nested Data
> File (Parquet) Set). There are differences in Schema among the files:
> * The Parquet files exhibit differences both at the first level and within
> nested data types
> * A select * will not cause an exception but using a limit clause will
> * Note also this issue seems to happen only when multiple Drillbit minor
> fragments are involved (concurrency higher than one)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
