nsivabalan commented on a change in pull request #4840:
URL: https://github.com/apache/hudi/pull/4840#discussion_r815308544
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java
##########
@@ -182,14 +182,30 @@ public abstract void preCompact(
.withOperationField(config.allowOperationMetadataField())
.withPartition(operation.getPartitionPath())
.build();
- if (!scanner.iterator().hasNext()) {
- scanner.close();
- return new ArrayList<>();
- }
Option<HoodieBaseFile> oldDataFileOpt =
operation.getBaseFile(metaClient.getBasePath(),
operation.getPartitionPath());
+ // Considering following scenario: if all log blocks in this fileSlice is
rollback, it returns an empty scanner.
+ // But in this case, we need to give it a base file. Otherwise, it will
lose base file in following fileSlice.
+ if (!scanner.iterator().hasNext()) {
+ if (!oldDataFileOpt.isPresent()) {
+ scanner.close();
+ return new ArrayList<>();
+ } else {
+ // TODO: we may directly rename original parquet file if there is not
evolution/devolution of schema
+ /*
+ TaskContextSupplier taskContextSupplier =
hoodieCopyOnWriteTable.getTaskContextSupplier();
+ String newFileName = FSUtils.makeDataFileName(instantTime,
+
FSUtils.makeWriteToken(taskContextSupplier.getPartitionIdSupplier().get(),
taskContextSupplier.getStageIdSupplier().get(),
taskContextSupplier.getAttemptIdSupplier().get()),
+ operation.getFileId(),
hoodieCopyOnWriteTable.getBaseFileExtension());
+ Path oldFilePath = new Path(oldDataFileOpt.get().getPath());
+ Path newFilePath = new Path(oldFilePath.getParent(), newFileName);
+ FileUtil.copy(fs,oldFilePath, fs, newFilePath, false, fs.getConf());
+ */
+ }
Review comment:
I intentionally left it there if we ever want to fix that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
