yihua commented on code in PR #13746:
URL: https://github.com/apache/hudi/pull/13746#discussion_r2298211940
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java:
##########
@@ -85,29 +60,21 @@ public HoodieCreateHandle(HoodieWriteConfig config, String
instantTime, HoodieTa
public HoodieCreateHandle(HoodieWriteConfig config, String instantTime,
HoodieTable<T, I, K, O> hoodieTable,
String partitionPath, String fileId,
Option<Schema> overriddenSchema,
TaskContextSupplier taskContextSupplier, boolean
preserveMetadata) {
- super(config, instantTime, partitionPath, fileId, hoodieTable,
overriddenSchema,
- taskContextSupplier, preserveMetadata);
- writeStatus.setFileId(fileId);
- writeStatus.setPartitionPath(partitionPath);
- writeStatus.setStat(new HoodieWriteStat());
-
- this.path = makeNewPath(partitionPath);
-
+ super(config, instantTime, hoodieTable, partitionPath, fileId,
overriddenSchema, taskContextSupplier, preserveMetadata);
+ this.logger = LoggerFactory.getLogger(HoodieCreateHandle.class);
try {
- HoodiePartitionMetadata partitionMetadata = new
HoodiePartitionMetadata(storage, instantTime,
- new StoragePath(config.getBasePath()),
- FSUtils.constructAbsolutePath(config.getBasePath(), partitionPath),
- hoodieTable.getPartitionMetafileFormat());
- partitionMetadata.trySave();
- createMarkerFile(partitionPath,
- FSUtils.makeBaseFileName(this.instantTime, this.writeToken,
this.fileId, hoodieTable.getBaseFileExtension()));
- this.fileWriter =
- HoodieFileWriterFactory.getFileWriter(instantTime, path,
hoodieTable.getStorage(), config,
- writeSchemaWithMetaFields, this.taskContextSupplier,
config.getRecordMerger().getRecordType());
+ this.fileWriter = initializeFileWriter();
} catch (IOException e) {
throw new HoodieInsertException("Failed to initialize
HoodieStorageWriter for path " + path, e);
}
- LOG.info("New CreateHandle for partition :" + partitionPath + " with
fileId " + fileId);
+ createPartitionMetadataAndMarkerFile();
Review Comment:
To maintain the ordering, it would be better to call
`createPartitionMetadataAndMarkerFile()` first and then `this.fileWriter =
initializeFileWriter();`, to avoid possible file writer implementation that
creates a file on storage during writer initialization, violating the sequence
in the create handle (marker should be created before data file creation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
