vikramahuja1001 commented on a change in pull request #3917:
URL: https://github.com/apache/carbondata/pull/3917#discussion_r510659146
##########
File path:
core/src/main/java/org/apache/carbondata/core/metadata/SegmentFileStore.java
##########
@@ -1105,28 +1109,79 @@ public static void cleanSegments(CarbonTable table,
List<PartitionSpec> partitio
* @throws IOException
*/
public static void deleteSegment(String tablePath, Segment segment,
- List<PartitionSpec> partitionSpecs,
- SegmentUpdateStatusManager updateStatusManager) throws Exception {
+ List<PartitionSpec> partitionSpecs, SegmentUpdateStatusManager
updateStatusManager,
+ SegmentStatus segmentStatus, Boolean isPartitionTable, String timeStamp)
+ throws Exception {
SegmentFileStore fileStore = new SegmentFileStore(tablePath,
segment.getSegmentFileName());
List<String> indexOrMergeFiles =
fileStore.readIndexFiles(SegmentStatus.SUCCESS, true,
FileFactory.getConfiguration());
+ List<String> filesToDelete = new ArrayList<>();
Map<String, List<String>> indexFilesMap = fileStore.getIndexFilesMap();
for (Map.Entry<String, List<String>> entry : indexFilesMap.entrySet()) {
- FileFactory.deleteFile(entry.getKey());
+ // Move the file to the trash folder in case the segment status is
insert in progress
+ if (segmentStatus == SegmentStatus.INSERT_IN_PROGRESS) {
+ if (!isPartitionTable) {
+ TrashUtil.moveDataToTrashFolderByFile(tablePath, entry.getKey(),
timeStamp +
+ CarbonCommonConstants.FILE_SEPARATOR +
CarbonCommonConstants.LOAD_FOLDER + segment
+ .getSegmentNo());
+ } else {
+ TrashUtil.moveDataToTrashFolderByFile(tablePath, entry.getKey(),
timeStamp +
Review comment:
For normal table, we do like: timestamp/Segment_#, there is no use of
having Fact and Part0 folders in trash
For partition table, we do like: timestamp/Segment_#/partition_folder, the
segment number is added so as the recovery can be segment wise.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
