[
https://issues.apache.org/jira/browse/CARBONDATA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akash R Nilugal updated CARBONDATA-2021:
----------------------------------------
Description:
when delete is success and update is failed while writing status file then a
stale carbon data file is created.
so removing that file on clean up . and also not considering that one during
query.
when the update operation is running and the user stops it abruptly,
then the carbon data file will be remained in the store .
so extra data is coming.
during the next update the clean up of the files need to be handled.
and in query also new data file should be excluded.
test({color:#008000}"overwrite whole partition table with empty data"{color}) {
sql({color:#008000}"create table partitionLoadTable(name string, age int)
PARTITIONED BY(address string) stored by 'carbondata'"{color})
sql({color:#008000}"insert into partitionLoadTable select
'abc',4,'def'"{color})
sql({color:#008000}"insert into partitionLoadTable select
'abd',5,'xyz'"{color})
sql({color:#008000}"create table noLoadTable (name string, age int, address
string) stored by 'carbondata'"{color})
sql({color:#008000}"insert overwrite table partitionLoadTable select * from
noLoadTable"{color})
checkAnswer(sql({color:#008000}"select * from partitionLoadTable"{color}),
sql({color:#008000}"select * from noLoadTable"{color}))
}
was:
when delete is success and update is failed while writing status file then a
stale carbon data file is created.
so removing that file on clean up . and also not considering that one during
query.
when the update operation is running and the user stops it abruptly,
then the carbon data file will be remained in the store .
so extra data is coming.
during the next update the clean up of the files need to be handled.
and in query also new data file should be excluded.
> when delete is success and update is failed while writing status file then a
> stale carbon data file is created.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-2021
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2021
> Project: CarbonData
> Issue Type: Bug
> Reporter: Akash R Nilugal
> Assignee: Akash R Nilugal
> Priority: Minor
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> when delete is success and update is failed while writing status file then a
> stale carbon data file is created.
> so removing that file on clean up . and also not considering that one during
> query.
> when the update operation is running and the user stops it abruptly,
> then the carbon data file will be remained in the store .
> so extra data is coming.
> during the next update the clean up of the files need to be handled.
> and in query also new data file should be excluded.
>
> test({color:#008000}"overwrite whole partition table with empty data"{color})
> {
> sql({color:#008000}"create table partitionLoadTable(name string, age int)
> PARTITIONED BY(address string) stored by 'carbondata'"{color})
> sql({color:#008000}"insert into partitionLoadTable select
> 'abc',4,'def'"{color})
> sql({color:#008000}"insert into partitionLoadTable select
> 'abd',5,'xyz'"{color})
> sql({color:#008000}"create table noLoadTable (name string, age int, address
> string) stored by 'carbondata'"{color})
> sql({color:#008000}"insert overwrite table partitionLoadTable select * from
> noLoadTable"{color})
> checkAnswer(sql({color:#008000}"select * from partitionLoadTable"{color}),
> sql({color:#008000}"select * from noLoadTable"{color}))
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
