yihua commented on code in PR #19717:
URL: https://github.com/apache/hudi/pull/19717#discussion_r4020777955
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -435,8 +453,16 @@ private void initializeMetadataPartition(
Indexer indexer,
String dataTableInstantTime,
Map<String, List<FileInfo>> partitionToAllFilesMap,
- Lazy<List<FileSliceAndPartition>> lazyMergedFileSlices) throws
IOException {
- String instantTimeForPartition =
generateUniqueInstantTime(dataTableInstantTime);
+ Lazy<List<FileSliceAndPartition>> lazyMergedFileSlices,
+ Option<String> requestedIndexPartitionOpt) throws IOException {
+ // A requested partition initializes under a fresh solo-family instant,
never the indexing
+ // action's own instant. The action's completion applies its data commit
to the metadata table
+ // too, and finding that instant already completed there is treated as a
partially applied
+ // earlier commit: it is rolled back and re-applied, destroying the
initialization records
+ // while leaving the file groups. The solo family is the established shape
for
+ // metadata-table-only bootstrap commits and survives that reconciliation.
+ String instantTimeForPartition = requestedIndexPartitionOpt.isPresent()
+ ? generateUniqueSoloInstantTime() :
generateUniqueInstantTime(dataTableInstantTime);
Review Comment:
Addressed by dropping the solo instant altogether rather than extending the
exemption: the requested partition bootstraps under the indexing action's own
instant again, so isIndexingCommit covers it for both eager rollback and the
heartbeat cleanup, and the BaseHoodieTableServiceClient change is reverted.
Verified the PR's tests pass at the action instant.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -435,8 +453,16 @@ private void initializeMetadataPartition(
Indexer indexer,
String dataTableInstantTime,
Map<String, List<FileInfo>> partitionToAllFilesMap,
- Lazy<List<FileSliceAndPartition>> lazyMergedFileSlices) throws
IOException {
- String instantTimeForPartition =
generateUniqueInstantTime(dataTableInstantTime);
+ Lazy<List<FileSliceAndPartition>> lazyMergedFileSlices,
+ Option<String> requestedIndexPartition) throws IOException {
+ // A requested partition initializes under a fresh solo-family instant,
never the indexing
+ // action's own instant. The action's completion applies its data commit
to the metadata table
+ // too, and finding that instant already completed there is treated as a
partially applied
+ // earlier commit: it is rolled back and re-applied, destroying the
initialization records
+ // while leaving the file groups. The solo family is the established shape
for
+ // metadata-table-only bootstrap commits and survives that reconciliation.
+ String instantTimeForPartition = requestedIndexPartition.isPresent()
+ ? generateUniqueSoloInstantTime() :
generateUniqueInstantTime(dataTableInstantTime);
Review Comment:
Done, the requested partition now takes the action instant like every other
type, so this distinction is gone. The solo instant turned out not to be
load-bearing for the PR's tests, and keeping the action instant preserves the
HUDI-5733 exemption.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
