[
https://issues.apache.org/jira/browse/TEZ-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mustafa Iman resolved TEZ-4177.
-------------------------------
Resolution: Invalid
> Improve error message for external orc table
> --------------------------------------------
>
> Key: TEZ-4177
> URL: https://issues.apache.org/jira/browse/TEZ-4177
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Mustafa Iman
> Assignee: Mustafa Iman
> Priority: Major
>
> Since there is no schema validation for external tables, users may face
> various errors if their orc data and external table schema does not match. If
> orc schema has fewer columns than projection OrcEncodedDataConsumer may
> receive an incomplete TypeDescription array which will manifest itself as
> NullPointerException later.
> We can at least verify that OrcEncodedDataConsumer gets enough
> TypeDescriptions. If assertion fails, user sees there is something wrong with
> the schema and hopefully resolves the problem quickly. If there are enough
> columns in the file but the schema of the query does not match, user
> generally sees a ClassCastException. If there are enough columns and types
> accidentally match, there is nothing we can do as this is an external table.
> We have seen this when trying to use a managed table as external table
> location. Although user facing schemas are the same, managed table has acid
> related metadata. I am adding a q file demonstrating NullPointerException
> with TestMiniLlapLocalCliDriver and the output after the fix. I haven't added
> this to precommit tests as it is hard to assert the exception message from
> mini driver framework and effectively it is just changing the error.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
