havefunatcode commented on code in PR #20160:
URL: https://github.com/apache/pinot/pull/20160#discussion_r4488095386
##########
pinot-spi/src/main/java/org/apache/pinot/spi/ingestion/batch/spec/PushJobSpec.java:
##########
@@ -57,6 +57,14 @@ public class PushJobSpec implements Serializable {
/// recursively.
private String _pushFileNamePattern;
+ /// Applicable for URI push type.
+ /// If true, and `pushFileNamePattern` is configured, the files matched by
the pattern are pushed even without a
+ /// segment archive extension, e.g. segments stored in the deep store such
as committed REALTIME segments.
+ /// Directories, segment metadata files and temporary segment files
(`<segment>.tmp.<uuid>`) are always skipped.
+ /// When `outputDirURI` points to the deep store of another table, keep
`cleanUpOutputDir` false, otherwise the pushed
Review Comment:
Good point, it now fails fast. The standalone, Hadoop and Spark URI push
runners call `SegmentPushUtils.validateUriPushJobSpec()` in `init()`, which
throws if both are true, before anything is pushed or deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
