[
https://issues.apache.org/jira/browse/HIVE-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21428:
--------------------------------------
Summary: field delimiter of serde set for partition is not getting
respected when schema evolution is enabled (was: field delimiter of serde set
for partition is not getting respected when vectorization is enabled)
> field delimiter of serde set for partition is not getting respected when
> schema evolution is enabled
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-21428
> URL: https://issues.apache.org/jira/browse/HIVE-21428
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.1.1
> Reporter: Ganesha Shreedhara
> Priority: Major
>
>
> *Steps to reproduce:*
> create external table src (c1 string, c2, string, c3 string) partitioned by
> (part string)
> location '/tmp/src';
>
>
> echo "d1\td2" >> data.txt;
> hadoop dfs -put data.txt /tmp/src/part=part1/;
>
> MSCK REPAIR TABLE src;
>
> ALTER TABLE src PARTITION (part='part1')
> SET SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> WITH SERDEPROPERTIES ('columns'='c1,c2', 'column.types' ='string,string',
> 'field.delim'='\t');
>
> create table dest (c1 string, c2 string, c3 string, c4 string);
> insert overwrite table dest select * from src;
> select * from dest;
>
> *Result* (wrong)*:*
> d1 d2 NULL NULL part1
>
> set hive.vectorized.execution.enabled=false;
> insert overwrite table dest select * from src;
> select * from dest;
>
> *Result* (Correct)*:*
> d1 d2 NULL part1
>
> This is because "d1\td2" is getting considered as single column because the
> filed delimiter used by deserialiser is *^A* instead of *\t* which is set at
> partition level.
> It is working fine if I alter the field delimiter of serde for the entire
> table.
> So, looks like serde properties in TableDesc is taking precedence over serde
> properties in PartitionDesc.
> This issue is not there in 2.x versions.
>
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
