xiangfu0 commented on code in PR #18813:
URL: https://github.com/apache/pinot/pull/18813#discussion_r3446248750
##########
pinot-plugins/pinot-minion-tasks/pinot-minion-builtin-tasks/src/main/java/org/apache/pinot/plugin/minion/tasks/BaseSingleSegmentConversionExecutor.java:
##########
@@ -210,19 +209,17 @@ public SegmentConversionResult
executeTask(PinotTaskConfig pinotTaskConfig)
throw new UnsupportedOperationException("Unrecognized push mode: "
+ pushType);
}
} catch (Exception e) {
- uploadSuccessful = false;
_minionMetrics.addMeteredTableValue(tableNameWithType,
MinionMeter.SEGMENT_UPLOAD_FAIL_COUNT, 1L);
LOGGER.error("Segment upload failed for segment {}, table {}",
segmentName, tableNameWithType, e);
_eventObserver.notifyTaskError(_pinotTaskConfig, e);
- }
- if (!FileUtils.deleteQuietly(convertedTarredSegmentFile)) {
- LOGGER.warn("Failed to delete tarred converted segment: {}",
convertedTarredSegmentFile.getAbsolutePath());
- }
-
- if (uploadSuccessful) {
- LOGGER.info("Done executing {} on table: {}, segment: {}", taskType,
tableNameWithType, segmentName);
+ throw e;
Review Comment:
In METADATA push mode this rethrow turns controller-side push failures into
task retries, but the retry comes back through `moveSegmentToOutputPinotFS()`
with the same `<segment>.tar.gz` target. `overwriteOutput` is normally left
false in `MinionTaskUtils.getPushTaskConfig()`, so if the first attempt already
copied the tar before failing, the retry now dies on `Output file already
exists` before it can resend metadata. That makes transient metadata-push
failures sticky instead of self-healing. Can we make the staged tar idempotent
across retries or clean it up before rethrowing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
