[
https://issues.apache.org/jira/browse/CARBONDATA-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajantha Bhat resolved CARBONDATA-4282.
--------------------------------------
Fix Version/s: 2.3.0
Resolution: Fixed
> Issues with table having complex columns related to long string, SI, local
> dictionary
> -------------------------------------------------------------------------------------
>
> Key: CARBONDATA-4282
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4282
> Project: CarbonData
> Issue Type: Bug
> Reporter: SHREELEKHYA GAMPA
> Priority: Major
> Fix For: 2.3.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *1. Insert/load fails after alter add complex column if table contains long
> string columns.*
> [Steps] :-
> DROP TABLE IF EXISTS alter_com;
> CREATE TABLE alter_com(intfield int,EDUCATED string ,rankk string ) STORED AS
> carbondata
> TBLPROPERTIES('inverted_index'='intField','sort_columns'='intField','TABLE_BLOCKSIZE'=
> '256
> MB','TABLE_BLOCKLET_SIZE'='8','SORT_SCOPE'='no_sort','COLUMN_META_CACHE'='rankk','carbon.column.compressor'='gzip','long_string_columns'='rankk','table_page_size_inmb'='1');
> insert into alter_com values(1,'cse','xi'); select * from alter_com limit 1;
> ALTER TABLE alter_com ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>,
> map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5
> Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>,
> map8 map<string,struct<d:int, s:struct<im:string>>>);
> ALTER TABLE alter_com SET TBLPROPERTIES('long_string_columns'='EDUCATED');
> insert into alter_com values(1,'ece','x', map(1,2),map(3,2.34),
> map(1.23,'hello'),map('abc','def'),
> map(true,'2017-02-01'),map('time','2018-02-01
> 02:00:00.0'),map('ph',array(1,2)),
> map('a',named_struct('d',23,'s',named_struct('im','sh'))));
> [Expected Result] :- insert/load should be success after alter add map column
> ,if table contains long string columns
> *2. create index on array of complex column (map/struct) throws null pointer
> exception instead of correct error message.*
> [Steps] :-
> drop table if exists strarmap1; create table strarmap1(id int,name string,str
> struct<intval:int,tm:timestamp,map1:map<int,string>>,arr
> array<map<string,timestamp>>) stored as carbondata
> tblproperties('inverted_index'='name','sort_columns'='name','TABLE_BLOCKSIZE'=
> '256 MB','TABLE_BLOCKLET_SIZE'='8','CACHE_LEVEL'='BLOCKLET'); load data
> inpath 'hdfs://hacluster/chetan/strarmap1.csv' into table strarmap1
> options('fileheader'='id,name,str,arr','COMPLEX_DELIMITER_LEVEL_3'='#','COMPLEX_DELIMITER_LEVEL_2'='$','COMPLEX_DELIMITER_LEVEL_1'='&','BAD_RECORDS_ACTION'='FORCE');
> CREATE INDEX index2 ON TABLE strarmap1 (arr) as 'carbondata'
> properties('sort_scope'='global_sort','global_sort_partitions'='3');
> [Expected Result] :- create index on array of map(string,timestamp) should
> thrown correct validation error message.
> [Actual Issue]:- create index on array of map(string,timestamp) throws null
> pointer exception instead of correct error message
> *3. alter table property local dictionary inlcude/exclude with newly added
> map column is failing.*
> [Steps] :-
> drop table if exists strarmap1; create table strarmap1(id int,name string,str
> struct<intval:int,tm:timestamp,map1:map<int,string>>,arr
> array<map<string,timestamp>>) stored as carbondata
> tblproperties('inverted_index'='name','sort_columns'='name','local_dictionary_enable'='false','local_dictionary_include'='map1','local_dictionary_exclude'='str,arr','local_dictionary_threshold'='1000');
> ALTER TABLE strarmap1 ADD COLUMNS(map1 Map<short,int>, map2 Map<long,double>,
> map3 Map<decimal(3,2),string>, map4 Map<char(5),varchar(50)>, map5
> Map<boolean,date>,map6 Map<string,timestamp>,map7 map<string,array<string>>,
> map8 map<string,struct<d:int, s:struct<im:string>>>); ALTER TABLE strarmap1
> SET
> TBLPROPERTIES('LOCAL_DICTIONARY_ENABLE'='true','local_dictionary_include'='map4','local_dictionary_threshold'='1000');
> [Expected Result] :- alter table property local dictionary inlcude/exclude
> with newly added map column should be success
> [Actual Issue]:- alter table property local dictionary inlcude/exclude with
> newly added map column is failing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
