adixitconfluent commented on code in PR #17539:
URL: https://github.com/apache/kafka/pull/17539#discussion_r1806725239
##########
core/src/main/java/kafka/server/share/DelayedShareFetch.java:
##########
@@ -204,6 +187,58 @@ Map<TopicIdPartition, FetchRequest.PartitionData>
acquirablePartitions() {
return topicPartitionData;
}
+ /**
+ * Prepare partitions fetch data structure for acquirable partitions in
the share fetch request satisfying minBytes criteria.
+ */
+ Map<TopicIdPartition, FetchPartitionData>
replicaManagerFetchData(Map<TopicIdPartition, FetchRequest.PartitionData>
topicPartitionData,
+ boolean
hasRequestTimedOut) {
+ log.trace("Fetchable share partitions data: {} with groupId: {} fetch
params: {}", topicPartitionData,
+ shareFetchData.groupId(), shareFetchData.fetchParams());
+ Map<TopicIdPartition, FetchPartitionData>
replicaManagerFetchSatisfyingMinBytes = new HashMap<>();
+ Map<TopicIdPartition, FetchPartitionData> responseData = new
HashMap<>();
+ try {
+ Seq<Tuple2<TopicIdPartition, LogReadResult>> responseLogResult =
replicaManager.readFromLog(
+ shareFetchData.fetchParams(),
+ CollectionConverters.asScala(
+ topicPartitionData.entrySet().stream().map(entry ->
+ new Tuple2<>(entry.getKey(),
entry.getValue())).collect(Collectors.toList())
+ ),
+ QuotaFactory.UnboundedQuota$.MODULE$,
+ true);
+
+ AtomicInteger accumulatedBytes = new AtomicInteger(0);
+
+ responseLogResult.foreach(tpLogResult -> {
+ TopicIdPartition topicIdPartition = tpLogResult._1();
+ LogReadResult logResult = tpLogResult._2();
+ FetchPartitionData fetchPartitionData =
logResult.toFetchPartitionData(false);
+ responseData.put(topicIdPartition, fetchPartitionData);
+
accumulatedBytes.addAndGet(logResult.info().records.sizeInBytes());
+ return BoxedUnit.UNIT;
+ });
+ log.trace("Data successfully retrieved by replica manager: {}",
responseData);
+
+ if (accumulatedBytes.get() >=
shareFetchData.fetchParams().minBytes)
+ replicaManagerFetchSatisfyingMinBytes = responseData;
+ } catch (Exception e) {
+ log.error("Error processing delayed share fetch request", e);
+ } finally {
+ // The case when we cannot satisfy the share fetch requests
because the response has lesser data than minBytes
+ // and the call is coming from tryComplete, hence we want to
release partitions lock so that the next
+ // tryComplete/onComplete call complete successfully.
+ if (replicaManagerFetchSatisfyingMinBytes.isEmpty() &&
!hasRequestTimedOut) {
+ // Releasing the lock to move ahead with the next request in
queue.
+ releasePartitionLocks(shareFetchData.groupId(),
topicPartitionData.keySet());
+ }
+ }
+ // This call is coming from onComplete, hence we return the response
data irrespective of whether minBytes is
+ // satisfied or not.
+ if (hasRequestTimedOut)
+ return responseData;
Review Comment:
Now that I think again, I should return a map with key as topic partition
and value as `fetchPartitionData` object containing 0 records and since we have
not been able to satisfy all the fetch request criterias. @junrao your thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
