jineshparakh commented on code in PR #17713:
URL: https://github.com/apache/pinot/pull/17713#discussion_r2815109094
##########
pinot-plugins/pinot-file-system/pinot-gcs/src/main/java/org/apache/pinot/plugin/filesystem/GcsPinotFS.java:
##########
@@ -153,8 +153,11 @@ public boolean deleteBatch(List<URI> segmentUris, boolean
forceDelete)
if (existsDirectoryOrBucket(gcsUri)) {
result &= delete(gcsUri, forceDelete);
} else {
- blobIds.add(getBlob(gcsUri).getBlobId());
- if (blobIds.size() >= DELETE_BATCH_LIMIT || !iterator.hasNext()) {
+ Blob blob = getBlob(gcsUri);
+ if (blob != null) {
+ blobIds.add(blob.getBlobId());
+ }
+ if (blobIds.size() >= DELETE_BATCH_LIMIT || (!blobIds.isEmpty() &&
!iterator.hasNext())) {
Review Comment:
@xiangfu0 @shounakmk219
Quick question on the testing side. The existing `GcsPinotFSTest` is purely
an integration test that requires real GCS credentials
(`GOOGLE_APPLICATION_CREDENTIALS, GCP_PROJECT, GCS_BUCKET`) and skips all tests
if those aren't set. Does Pinot CI actually have GCS credentials configured, or
are these tests effectively never run in CI?
There are no mock-based unit tests in the GCS module today, unlike S3 which
uses `S3MockContainer`. Given that this is a small, focused change (a null
check + a condition guard), do you think introducing a full Mockito-based test
setup for `GcsPinotFS` is warranted here, or would that be better suited as a
broader test infrastructure improvement in a separate effort?
Happy to add tests if needed, just want to make sure we're aligned on the
right approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
