[
https://issues.apache.org/jira/browse/CARBONDATA-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227384#comment-17227384
]
Prasanna Ravichandran edited comment on CARBONDATA-4029 at 11/6/20, 12:47 PM:
------------------------------------------------------------------------------
Working fine and passed with the newly created SDK segments. Some old SDK
segments with future timestamp will be considered invalid segment because of
some other scenarios in delete/update. Old SDK files with future timestamp
values, cannot be fixed.
was (Author: prasanna ravichandran):
Working fine and passed with the newly created SDK segments. Some old SDK
segments with future timestamp will be considered as consider invalid segment
because of some other scenarios in delete/update. Old SDK files with future
timestamp values, cannot be fixed.
> After delete in the table which has Alter-added SDK segments, then the
> count(*) is 0.
> -------------------------------------------------------------------------------------
>
> Key: CARBONDATA-4029
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4029
> Project: CarbonData
> Issue Type: Bug
> Affects Versions: 2.0.0
> Environment: 3 node FI cluster
> Reporter: Prasanna Ravichandran
> Priority: Minor
> Attachments: Primitive.rar
>
>
> Do delete on a table which has alter added SDK segments. then the count* is
> 0. Even count* will be 0 even any number of SDK segments are added after it.
> Test queries:
> drop table if exists external_primitive;
> create table external_primitive (id int, name string, rank smallint, salary
> double, active boolean, dob date, doj timestamp, city string, dept string)
> stored as carbondata;
> --before executing the below alter add segment-place the attached SDK files
> in hdfs at /sdkfiles/primitive2 folder;
> alter table external_primitive add segment
> options('path'='hdfs://hacluster/sdkfiles/primitive2','format'='carbon');select
> * from external_primitive;
> delete from external_primitive where id =2;select * from external_primitive;
> Console output:
> /> drop table if exists external_primitive;
> +---------+
> | Result |
> +---------+
> +---------+
> No rows selected (1.586 seconds)
> /> create table external_primitive (id int, name string, rank smallint,
> salary double, active boolean, dob date, doj timestamp, city string, dept
> string) stored as carbondata;
> +---------+
> | Result |
> +---------+
> +---------+
> No rows selected (0.774 seconds)
> /> alter table external_primitive add segment
> options('path'='hdfs://hacluster/sdkfiles/primitive2','format'='carbon');select
> * from external_primitive;
> +---------+
> | Result |
> +---------+
> +---------+
> No rows selected (1.077 seconds)
> INFO : Execution ID: 320
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> | id | name | rank | salary | active | dob | doj | city | dept |
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> | 1 | AAA | 3 | 3444345.66 | true | 1979-12-09 | 2011-02-10 01:00:20.0 | Pune
> | IT |
> | 2 | BBB | 2 | 543124.66 | false | 1987-02-19 | 2017-01-01 12:00:20.0 |
> Bangalore | DATA |
> | 3 | CCC | 1 | 787878.888 | false | 1982-05-12 | 2015-12-01 02:20:20.0 |
> Pune | DATA |
> | 4 | DDD | 1 | 99999.24 | true | 1981-04-09 | 2000-01-15 07:00:20.0 | Delhi
> | MAINS |
> | 5 | EEE | 3 | 545656.99 | true | 1987-12-09 | 2017-11-25 04:00:20.0 | Delhi
> | IT |
> | 6 | FFF | 2 | 768678.0 | false | 1987-12-20 | 2017-01-10 05:00:20.0 |
> Bangalore | DATA |
> | 7 | GGG | 3 | 765665.0 | true | 1983-06-12 | 2017-01-01 02:00:20.0 | Pune |
> IT |
> | 8 | HHH | 2 | 567567.66 | false | 1979-01-12 | 1995-01-01 12:00:20.0 |
> Bangalore | DATA |
> | 9 | III | 2 | 787878.767 | true | 1985-02-19 | 2005-08-15 01:00:20.0 | Pune
> | DATA |
> | 10 | JJJ | 3 | 887877.14 | true | 2000-05-19 | 2016-10-10 12:00:20.0 |
> Bangalore | MAINS |
> | 18 | | 3 | 7.86786786787E9 | true | 1980-10-05 | 1995-10-07 22:00:20.0 |
> Bangalore | IT |
> | 19 | | 2 | 5464545.33 | true | 1986-06-06 | 2008-08-15 01:00:20.0 | Delhi |
> DATA |
> | 20 | NULL | 3 | 7867867.34 | true | 2000-05-01 | 2014-01-18 12:00:20.0 |
> Bangalore | MAINS |
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> 13 rows selected (2.458 seconds)
> /> delete from external_primitive where id =2;select * from
> external_primitive;
> INFO : Execution ID: 322
> +--------------------+
> | Deleted Row Count |
> +--------------------+
> | 1 |
> +--------------------+
> 1 row selected (3.723 seconds)
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> | id | name | rank | salary | active | dob | doj | city | dept |
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> No rows selected (1.531 seconds)
> /> alter table external_primitive add segment
> options('path'='hdfs://hacluster/sdkfiles/primitive3','format'='carbon');select
> * from external_primitive;
> +---------+
> | Result |
> +---------+
> +---------+
> No rows selected (0.766 seconds)
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> | id | name | rank | salary | active | dob | doj | city | dept |
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> +-----+-------+-------+---------+---------+------+------+-------+-------+
> No rows selected (1.439 seconds)
> /> select count(*) from external_primitive;
> INFO : Execution ID: 335
> +-----------+
> | count(1) |
> +-----------+
> | 0 |
> +-----------+
> 1 row selected (1.278 seconds)
> />
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
