Zhichao Zhang created CARBONDATA-2258:
------------------------------------------
Summary: Separate visible and invisible segments info into two
files to reduce the size of tablestatus file.
Key: CARBONDATA-2258
URL: https://issues.apache.org/jira/browse/CARBONDATA-2258
Project: CarbonData
Issue Type: Improvement
Components: core
Affects Versions: 1.4.0, 1.3.2
Reporter: Zhichao Zhang
Assignee: Zhichao Zhang
Fix For: 1.4.0, 1.3.2
The size of the tablestatus file is getting larger, there are many places will
scan this file and it will impact the performance of reading this file.
According to the discussion on
[thread|http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/The-size-of-the-tablestatus-file-is-getting-larger-does-it-impact-the-performance-of-reading-this-fi-td41941.html],
it can *append* the
invisible segment list to the file called 'tablestatus.history' when execute
command 'CLEAN FILES FOR TABLE' (in method
'SegmentStatusManager.deleteLoadsAndUpdateMetadata') every time, separate
visible and invisible segments into two files(tablestatus file and
tablestatus.history file).
If later it needs to support listing all segments(include visible and
invisible) list when execute 'SHOW SEGMENTS FOR TABLE', it just need to read
from two files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
