tarun11Mavani commented on code in PR #19776:
URL: https://github.com/apache/pinot/pull/19776#discussion_r4215303306
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/SplitSegmentCommitter.java:
##########
@@ -77,6 +92,15 @@ public SegmentCompletionProtocol.Response commit(
}
_params.withSegmentLocation(segmentLocation);
+ if (_uploadMetadataTar && _segmentUploader.isMetadataTarUploadSupported()
+ && !isPeerSegmentLocation(segmentLocation)) {
+ String metadataTarLocation =
+ uploadMetadataTarQuietly(segmentBuildDescriptor.getMetadataFiles(),
_params.getSegmentName());
Review Comment:
The metadata tar follows what the segment tar does. `uploadSegment` writes
the segment to `<seg>.tmp.<uuid>` before `segmentCommitEndWithMetadata`, and
the controller moves it into place only on commit success. The metadata tar is
uploaded to `<seg>.metadata.tar.gz.tmp.<uuid>` and the controller moves it at
the same step.
A failed commit leaves only a tmp object. `deleteTmpSegments` already sweeps
those, and the final-name sidecar is never written. The controller needs the
tmp location in the commit-end request, so uploading after commit end would
need a new protocol step.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
