xiangfu0 commented on code in PR #19947:
URL: https://github.com/apache/pinot/pull/19947#discussion_r4321649585
##########
pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark-3/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark3/SparkSegmentMetadataPushJobRunner.java:
##########
@@ -107,6 +105,9 @@ public void run()
// Collect files to process
List<String> segmentsToPush = getSegmentsToPush(outputDirFS, outputDirURI);
+ if (segmentsToPush.isEmpty()) {
+ return;
+ }
Review Comment:
Fixed in
[c411786740](https://github.com/apache/pinot/commit/c411786740978a9ca97f3746dcaf42e66ebaf7a7).
The runner determines consistent-push mode before the empty-selection return,
and empty REFRESH snapshots complete preUpload/postUpload without upload or
Spark dispatch. Regression coverage includes automatic, default, and parallel
push settings, plus a non-consistent empty selection.
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/SegmentPushUtils.java:
##########
@@ -684,17 +702,20 @@ public static Map<String, String>
getSegmentUriToTarPathMap(URI outputDirURI, Pu
}
}
- URI uri = URI.create(file);
+ URI uri = "file".equals(outputDirURI.getScheme()) && new
File(file).isAbsolute()
+ ? new File(file).toURI() : URI.create(file);
Review Comment:
Fixed in
[c411786740](https://github.com/apache/pinot/commit/c411786740978a9ca97f3746dcaf42e66ebaf7a7)
by guarding outputDirURI before reading its scheme. Added a regression test
for a null output directory with a nonempty segmentUriPrefix, preserving the
generated download URI and original local archive path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
