xiangfu0 commented on code in PR #18560:
URL: https://github.com/apache/pinot/pull/18560#discussion_r3288266869
##########
pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionGroupMetadataFetcher.java:
##########
@@ -152,9 +158,11 @@ private Boolean fetchMultipleStreams()
_exception = e;
return Boolean.FALSE;
} catch (Exception e) {
- LOGGER.warn("Could not get StreamMetadata for topic {}", topicName, e);
- _exception = e;
- throw e;
+ // In a multi-topic table, a single inaccessible topic (e.g. deleted
from Kafka) must not
+ // block metadata fetching for the remaining healthy topics. Log and
skip — the successful
+ // topics are already in _streamMetadataList and will be returned to
the caller.
+ LOGGER.warn("Could not get StreamMetadata for topic {}, skipping in
multi-topic fetch", topicName, e);
Review Comment:
Swallowing every non-`TransientConsumerException` here turns "could not
fetch metadata" into "this topic has no live partitions" for the controller.
`ensureAllPartitionsConsuming()` treats a missing partition in the fetched
metadata as end-of-life and stops creating the next CONSUMING segment, so
generic provider/auth/runtime failures can silently halt ingestion for the
affected topic. This needs a typed topic-deleted/not-found path, or the failure
must still be propagated so the controller does not take the end-of-life branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
