Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1674#discussion_r158055612
--- Diff:
core/src/main/java/org/apache/carbondata/core/metadata/PartitionMapFileStore.java
---
@@ -176,13 +179,87 @@ public PartitionMapper readPartitionMap(String
partitionMapPath) {
public void readAllPartitionsOfSegment(String segmentPath) {
CarbonFile[] partitionFiles = getPartitionFiles(segmentPath);
if (partitionFiles != null && partitionFiles.length > 0) {
+ partionedSegment = true;
for (CarbonFile file : partitionFiles) {
PartitionMapper partitionMapper =
readPartitionMap(file.getAbsolutePath());
partitionMap.putAll(partitionMapper.getPartitionMap());
}
}
}
+ public boolean isPartionedSegment() {
+ return partionedSegment;
+ }
+
+ /**
+ * Drops the partitions from the partition mapper file of the segment
and writes to a new file.
+ * @param segmentPath
+ * @param partitionsToDrop
+ * @param uniqueId
+ * @throws IOException
+ */
+ public void dropPartitions(String segmentPath, List<String>
partitionsToDrop, String uniqueId)
--- End diff --
1.For clean files jira CARBONDATA-1863 is already created as part of a
design.
2. Now the partitionmap is created using the loadtime stamp. And it will
not delete while dropping the partition. But while reading we make sure that we
read the latest file using the latest timestamp. And also load status is
updated with new timestamp so all the cache will be cleared from the driver.
---
