Mark Grover created SQOOP-2161:
----------------------------------
Summary: Incremental append on to Hive Parquet tables doesn't work
Key: SQOOP-2161
URL: https://issues.apache.org/jira/browse/SQOOP-2161
Project: Sqoop
Issue Type: Bug
Components: hive-integration
Affects Versions: 1.4.5
Reporter: Mark Grover
I have some code that does incremental append to a Hive parquet table. I am
able to get the first time automatic table creation in Hive to work, but
subsequent appends return errors. Also, ideally, I would like to create the
Hive table explicitly myself and only just do appends to it.
More concretely, the code I am using is at
https://gist.github.com/markgrover/86f54663ece0943bc8ed
I am also attaching two verbose error logs. Each of them contain, at the top,
the command ran.
new_pre_created_verbose.log contains the error which occurs if I try to import
data into an empty Hive parquet table that have been created in Hive. I ran the
create table statement in the above gist and it matches the source schema
one-to-one.
In order to get past the above error, I don't run the hive create table command
explicitly. Sqoop then successfully creates the table and adds data to it.
However, on the next run when I want to append more data, I get another error
which is detailed in append_pre_created_verbose.log
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
