Vandana Yadav created CARBONDATA-2007:
-----------------------------------------
Summary: Unexpected behavior while inserting data into a
partitioned table
Key: CARBONDATA-2007
URL: https://issues.apache.org/jira/browse/CARBONDATA-2007
Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 1.3.0
Environment: spark 2.1
Reporter: Vandana Yadav
Attachments: 2000_UniqData.csv
Unexpected behavior while inserting data into a partitioned table
Steps to reproduce
1) Create hive table:
a) CREATE TABLE uniqdata_hive (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1
int)ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
b) Load data:
LOAD DATA LOCAL INPATH
'/home/knoldus/Desktop/csv/TestData/Data/uniqdata/2000_UniqData.csv' into table
UNIQDATA_HIVE;
2) create Partoitioned table:
a) CREATE TABLE uniqdata_bigint (CUST_ID int,CUST_NAME
String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1
int) partitioned by (BIGINT_COLUMN1 bigint) STORED BY
'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
b) CREATE TABLE uniqdata_double (CUST_ID int,CUST_NAME
String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1
bigint,BIGINT_COLUMN2 bigint, DECIMAL_COLUMN2 decimal(36,10), Double_COLUMN2
double, INTEGER_COLUMN1 int) partitioned by (Double_COLUMN1 double) STORED BY
'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB");
3) Load data into the partitioned table from hive table:
a) insert into uniqdata_bigint partition(BIGINT_COLUMN1)select * from
uniqdata_hive limit 5;
output:
+---------+--+
| Result |
+---------+--+
+---------+--+
b) select * from uniqdata_bigint order by cust_id;
output:
+----------+------------------+----------------------------+------------------------+-------+-----------------+------------------+------------------+-----------------+-----------------+------------------+-----------------+--+
| cust_id | cust_name | active_emui_version | dob
| doj | bigint_column2 | decimal_column1 | decimal_column2 |
double_column1 | double_column2 | integer_column1 | bigint_column1 |
+----------+------------------+----------------------------+------------------------+-------+-----------------+------------------+------------------+-----------------+-----------------+------------------+-----------------+--+
| NULL | | | NULL
| NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL |
| NULL | | | 1970-01-01
11:00:03.0 | NULL | NULL | NULL | NULL |
NULL | NULL | NULL | NULL |
| NULL | | ACTIVE_EMUI_VERSION_00000 | NULL
| NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL |
| NULL | CUST_NAME_00000 | | NULL
| NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL |
| 8999 | | | NULL
| NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL |
+----------+------------------+----------------------------+------------------------+-------+-----------------+------------------+------------------+-----------------+-----------------+------------------+-----------------+--+
5 rows selected (0.769 seconds)
c)insert into uniqdata_double partition(Double_COLUMN1)select * from
uniqdata_hive limit 5;
Expected Output: data should be inserted successfully into the table.
Actual Output:
Error: org.apache.spark.sql.AnalysisException: Cannot insert into table
`newcarbon`.`uniqdata_double` because the number of columns are different: need
11 columns, but query has 12 columns.; (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
