Mustafa Iman created HIVE-23455:

             Summary: Improve error message for external orc table
                 Key: HIVE-23455
             Project: Hive
          Issue Type: Improvement
            Reporter: Mustafa Iman
            Assignee: Mustafa Iman

Since there is no schema validation for external tables, users may face various 
errors if their orc data and external table schema does not match. If orc 
schema has fewer columns than projection OrcEncodedDataConsumer may receive an 
incomplete TypeDescription array which will manifest itself as 
NullPointerException later.

We can at least verify that OrcEncodedDataConsumer gets enough 
TypeDescriptions. If assertion fails, user sees there is something wrong with 
the schema and hopefully resolves the problem quickly. If there are enough 
columns in the file but the schema of the query does not match, user generally 
sees a ClassCastException. If there are enough columns and types accidentally 
match, there is nothing we can do as this is an external table.

We have seen this when trying to use a managed table as external table 
location. Although user facing schemas are the same, managed table has acid 
related metadata. I am adding a q file demonstrating NullPointerException with 
TestMiniLlapLocalCliDriver and the output after the fix. I haven't added this 
to precommit tests as it is hard to assert the exception message from mini 
driver framework and effectively it is just changing the error.

This message was sent by Atlassian Jira

Reply via email to