Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1674#discussion_r157829858
--- Diff:
core/src/main/java/org/apache/carbondata/core/metadata/PartitionMapFileStore.java
---
@@ -176,13 +179,87 @@ public PartitionMapper readPartitionMap(String
partitionMapPath) {
public void readAllPartitionsOfSegment(String segmentPath) {
CarbonFile[] partitionFiles = getPartitionFiles(segmentPath);
if (partitionFiles != null && partitionFiles.length > 0) {
+ partionedSegment = true;
for (CarbonFile file : partitionFiles) {
PartitionMapper partitionMapper =
readPartitionMap(file.getAbsolutePath());
partitionMap.putAll(partitionMapper.getPartitionMap());
}
}
}
+ public boolean isPartionedSegment() {
+ return partionedSegment;
+ }
+
+ /**
+ * Drops the partitions from the partition mapper file of the segment
and writes to a new file.
+ * @param segmentPath
+ * @param partitionsToDrop
+ * @param uniqueId
+ * @throws IOException
+ */
+ public void dropPartitions(String segmentPath, List<String>
partitionsToDrop, String uniqueId)
+ throws IOException {
+ readAllPartitionsOfSegment(segmentPath);
+ List<String> indexesToDrop = new ArrayList<>();
+ for (Map.Entry<String, List<String>> entry: partitionMap.entrySet()) {
+ for (String partition: partitionsToDrop) {
+ if (entry.getValue().contains(partition)) {
+ indexesToDrop.add(entry.getKey());
+ }
+ }
+ }
+ if (indexesToDrop.size() > 0) {
+ // Remove the indexes from partition map
+ for (String indexToDrop : indexesToDrop) {
+ partitionMap.remove(indexToDrop);
+ }
+ PartitionMapper mapper = new PartitionMapper();
+ mapper.setPartitionMap(partitionMap);
+ String path = segmentPath + "/" + uniqueId +
CarbonTablePath.PARTITION_MAP_EXT;
+ writePartitionFile(mapper, path);
+ }
+ }
+
+ /**
+ * It deletes the old partition mapper files in case of success. And in
case of failure it removes
+ * the old new file.
+ * @param segmentPath
+ * @param uniqueId
+ * @param success
+ */
+ public void commitPartitions(String segmentPath, final String uniqueId,
boolean success) {
+ CarbonFile carbonFile = FileFactory.getCarbonFile(segmentPath);
--- End diff --
Locks are not taken while dropping partitions in Hive metastore, don't we
require to take it?
---
