Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1674#discussion_r158022717
--- Diff:
core/src/main/java/org/apache/carbondata/core/metadata/PartitionMapFileStore.java
---
@@ -176,13 +179,87 @@ public PartitionMapper readPartitionMap(String
partitionMapPath) {
public void readAllPartitionsOfSegment(String segmentPath) {
CarbonFile[] partitionFiles = getPartitionFiles(segmentPath);
if (partitionFiles != null && partitionFiles.length > 0) {
+ partionedSegment = true;
for (CarbonFile file : partitionFiles) {
PartitionMapper partitionMapper =
readPartitionMap(file.getAbsolutePath());
partitionMap.putAll(partitionMapper.getPartitionMap());
}
}
}
+ public boolean isPartionedSegment() {
+ return partionedSegment;
+ }
+
+ /**
+ * Drops the partitions from the partition mapper file of the segment
and writes to a new file.
+ * @param segmentPath
+ * @param partitionsToDrop
+ * @param uniqueId
+ * @throws IOException
+ */
+ public void dropPartitions(String segmentPath, List<String>
partitionsToDrop, String uniqueId)
--- End diff --
1) clean files needs to be handled to delete partition drop.
2) Reading paritionmap should adhere to segUpdatetimestamp/transactionid
from tablestatus file and drop parition should create mapfile with
segUpdatetimestamp and update it later in segmentstatus. After max query time
during cleanup old files can be deleted.
This will avoid temporary state reading during query.
3) Drop partition done in one driver should also drop related partitions in
cache in other driver. So there should be way to drop using
segUpdatetimestamp/transactionid.
4) Same thing applicable to InsertOverwrite scenario also.
---
