[
https://issues.apache.org/jira/browse/CARBONDATA-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SHREELEKHYA GAMPA updated CARBONDATA-4143:
--
Description:
To enable to run UT with index server using flag {{useIndexServer.}}
excluded some of the test cases to not run with index server.
added test case with prepriming.
To Fix below issues:
1. With index server enabled, select query gives incorrect result with SI when
parent and child table segments are not in sync.
queries to execute:
0: jdbc:hive2://dggphisprb50622:22550/> create table test (c1 string,c2 int,c3
string,c5 string) STORED AS carbondata;
+-+
| Result |
+-+
+-+
No rows selected (0.564 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> load data inpath
'hdfs://hacluster/chetan/dest.csv' into table test;
+-+
| Segment ID |
+-+
| 0 |
+-+
1 row selected (1.764 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> create index index_test on table test
(c3) AS 'carbondata';
+-+
| Result |
+-+
+-+
No rows selected (2.412 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> load data inpath
'hdfs://hacluster/chetan/dest.csv' into table test;
+-+
| Segment ID |
+-+
| 1 |
+-+
1 row selected (2.839 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
+-+-+-+--+
| c1 | c2 | c3 | c5 |
+-+-+-+--+
| d | 4 | dd | ddd |
| d | 4 | dd | ddd |
+-+-+-+--+
2 rows selected (3.452 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> delete from table index_test where
segment.ID in(1);
+-+
| Result |
+-+
+-+
No rows selected (0.413 seconds)
0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
+-+-+-+--+
| c1 | c2 | c3 | c5 |
+-+-+-+--+
| d | 4 | dd | ddd |
+-+-+-+--+
1 row selected (3.262 seconds)
0: jdbc:hive2://dggphisprb50622:22550/>
Expected: to return 2 rows.
2. When reindex is triggered, if stale files are present in the segment
directory the segment file is being written with incorrect file names. (both
valid index and stale mergeindex file names). As a result, duplicate data is
present in SI table but there is no error/incorrect query results.
was:
To enable to run UT with index server using flag {{useIndexServer.}}
excluded some of the test cases to not run with index server.
added test case with prepriming.
> UT with index server
>
>
> Key: CARBONDATA-4143
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4143
> Project: CarbonData
> Issue Type: Improvement
>Reporter: SHREELEKHYA GAMPA
>Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> To enable to run UT with index server using flag {{useIndexServer.}}
> excluded some of the test cases to not run with index server.
> added test case with prepriming.
> To Fix below issues:
> 1. With index server enabled, select query gives incorrect result with SI
> when parent and child table segments are not in sync.
> queries to execute:
> 0: jdbc:hive2://dggphisprb50622:22550/> create table test (c1 string,c2
> int,c3 string,c5 string) STORED AS carbondata;
> +-+
> | Result |
> +-+
> +-+
> No rows selected (0.564 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> load data inpath
> 'hdfs://hacluster/chetan/dest.csv' into table test;
> +-+
> | Segment ID |
> +-+
> | 0 |
> +-+
> 1 row selected (1.764 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> create index index_test on table test
> (c3) AS 'carbondata';
> +-+
> | Result |
> +-+
> +-+
> No rows selected (2.412 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> load data inpath
> 'hdfs://hacluster/chetan/dest.csv' into table test;
> +-+
> | Segment ID |
> +-+
> | 1 |
> +-+
> 1 row selected (2.839 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
> +-+-+-+--+
> | c1 | c2 | c3 | c5 |
> +-+-+-+--+
> | d | 4 | dd | ddd |
> | d | 4 | dd | ddd |
> +-+-+-+--+
> 2 rows selected (3.452 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> delete from table index_test where
> segment.ID in(1);
> +-+
> | Result |
> +-+
> +-+
> No rows selected (0.413 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/> select * from test where c3='dd';
> +-+-+-+--+
> | c1 | c2 | c3 | c5 |
> +-+-+-+--+
> | d | 4 | dd | ddd |
> +-+-+-+--+
> 1 row selected (3.262 seconds)
> 0: jdbc:hive2://dggphisprb50622:22550/>
> Expected: to return 2 rows.
> 2. When reindex is triggered, if stale files are present in the segment
>
