dclim commented on a change in pull request #7428: Add errors and state to
stream supervisor status API endpoint
URL: https://github.com/apache/incubator-druid/pull/7428#discussion_r281372717
##########
File path:
extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/indexing/kafka/KafkaRecordSupplier.java
##########
@@ -122,37 +126,44 @@ public void seekToLatest(Set<StreamPartition<Integer>>
partitions)
@Override
public Long getLatestSequenceNumber(StreamPartition<Integer> partition)
{
- Long currPos = consumer.position(new TopicPartition(partition.getStream(),
partition.getPartitionId()));
+ Long currPos = getPosition(partition);
seekToLatest(Collections.singleton(partition));
- Long nextPos = consumer.position(new TopicPartition(partition.getStream(),
partition.getPartitionId()));
+ Long nextPos = getPosition(partition);
seek(partition, currPos);
return nextPos;
}
@Override
public Long getEarliestSequenceNumber(StreamPartition<Integer> partition)
{
- Long currPos = consumer.position(new TopicPartition(partition.getStream(),
partition.getPartitionId()));
+ Long currPos = getPosition(partition);
seekToEarliest(Collections.singleton(partition));
- Long nextPos = consumer.position(new TopicPartition(partition.getStream(),
partition.getPartitionId()));
+ Long nextPos = getPosition(partition);
seek(partition, currPos);
return nextPos;
}
@Override
public Long getPosition(StreamPartition<Integer> partition)
{
- return consumer.position(new TopicPartition(partition.getStream(),
partition.getPartitionId()));
+ return wrapExceptions(() -> consumer.position(new TopicPartition(
+ partition.getStream(),
+ partition.getPartitionId()
+ )));
}
@Override
public Set<Integer> getPartitionIds(String stream)
{
- List<PartitionInfo> partitions = consumer.partitionsFor(stream);
- if (partitions == null) {
- throw new ISE("Topic [%s] is not found in KafkaConsumer's list of
topics", stream);
- }
- return
partitions.stream().map(PartitionInfo::partition).collect(Collectors.toSet());
+ return wrapExceptions(() -> {
+ // use consumer.listTopics() instead of partitionsFor() to force a
remote call so we can detect stream issues
+ Map<String, List<PartitionInfo>> topics = consumer.listTopics();
Review comment:
Hah nice, thanks for pointing that PR out. In that case, I can go back to
using `partitionsFor`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
