junrao commented on code in PR #14385:
URL: https://github.com/apache/kafka/pull/14385#discussion_r1327614488
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java:
##########
@@ -667,6 +683,35 @@ private static <K, V> ClusterResourceListeners
configureClusterResourceListeners
return clusterResourceListeners;
}
+ /**
+ *
+ * Indicates if the consumer is using the Kafka-based offset management
strategy,
+ * according to config {@link CommonClientConfigs#GROUP_ID_CONFIG}
+ */
+ private boolean isCommittedOffsetsManagementEnabled() {
+ return groupId.isPresent();
+ }
+
+ /**
+ * Refresh the committed offsets for provided partitions.
+ *
+ * @param timer Timer bounding how long this method can block
+ * @return true iff the operation completed within the timeout
+ */
+ private boolean refreshCommittedOffsetsIfNeeded(Timer timer) {
Review Comment:
Could we add the new param to the javadoc?
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java:
##########
@@ -141,4 +146,36 @@ public static <K, V> List<ConsumerInterceptor<K, V>>
configuredConsumerIntercept
return (List<ConsumerInterceptor<K, V>>)
ClientUtils.configuredInterceptors(config,
ConsumerConfig.INTERCEPTOR_CLASSES_CONFIG, ConsumerInterceptor.class);
}
+ public static boolean refreshCommittedOffsets(final Map<TopicPartition,
OffsetAndMetadata> offsets,
Review Comment:
Could we add a description of the method?
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java:
##########
@@ -667,6 +683,35 @@ private static <K, V> ClusterResourceListeners
configureClusterResourceListeners
return clusterResourceListeners;
}
+ /**
+ *
+ * Indicates if the consumer is using the Kafka-based offset management
strategy,
+ * according to config {@link CommonClientConfigs#GROUP_ID_CONFIG}
+ */
+ private boolean isCommittedOffsetsManagementEnabled() {
+ return groupId.isPresent();
+ }
+
+ /**
+ * Refresh the committed offsets for provided partitions.
+ *
+ * @param timer Timer bounding how long this method can block
+ * @return true iff the operation completed within the timeout
+ */
+ private boolean refreshCommittedOffsetsIfNeeded(Timer timer) {
+ final Set<TopicPartition> initializingPartitions =
subscriptions.initializingPartitions();
+
+ log.debug("Refreshing committed offsets for partitions {}",
initializingPartitions);
+ try {
+ final Map<TopicPartition, OffsetAndMetadata> offsets =
eventHandler.addAndGet(new OffsetFetchApplicationEvent(initializingPartitions),
timer);
Review Comment:
We block the `poll` call here to initialize the offset, but when the offset
is reset, we handle the offsetReset asynchronously without blocking the `poll`
call. Should we do these consistently? Does the existing consumer have the same
behavior?
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java:
##########
@@ -667,6 +683,35 @@ private static <K, V> ClusterResourceListeners
configureClusterResourceListeners
return clusterResourceListeners;
}
+ /**
+ *
+ * Indicates if the consumer is using the Kafka-based offset management
strategy,
+ * according to config {@link CommonClientConfigs#GROUP_ID_CONFIG}
+ */
+ private boolean isCommittedOffsetsManagementEnabled() {
+ return groupId.isPresent();
+ }
+
+ /**
+ * Refresh the committed offsets for provided partitions.
+ *
+ * @param timer Timer bounding how long this method can block
+ * @return true iff the operation completed within the timeout
+ */
+ private boolean refreshCommittedOffsetsIfNeeded(Timer timer) {
+ final Set<TopicPartition> initializingPartitions =
subscriptions.initializingPartitions();
+
+ log.debug("Refreshing committed offsets for partitions {}",
initializingPartitions);
+ try {
+ final Map<TopicPartition, OffsetAndMetadata> offsets =
eventHandler.addAndGet(new OffsetFetchApplicationEvent(initializingPartitions),
timer);
+ return ConsumerUtils.refreshCommittedOffsets(offsets,
this.metadata, this.subscriptions);
+ } catch (org.apache.kafka.common.errors.TimeoutException e) {
+ log.error("Couldn't refresh committed offsets before timeout
expired");
+ return false;
+ }
+ }
Review Comment:
At this point, it's still possible that not all partitions' position have
been initialized. Should we return false in that case?
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java:
##########
@@ -667,6 +683,35 @@ private static <K, V> ClusterResourceListeners
configureClusterResourceListeners
return clusterResourceListeners;
}
+ /**
+ *
+ * Indicates if the consumer is using the Kafka-based offset management
strategy,
+ * according to config {@link CommonClientConfigs#GROUP_ID_CONFIG}
+ */
+ private boolean isCommittedOffsetsManagementEnabled() {
+ return groupId.isPresent();
+ }
+
+ /**
+ * Refresh the committed offsets for provided partitions.
Review Comment:
There are no provided partitions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
