divijvaidya commented on code in PR #14329:
URL: https://github.com/apache/kafka/pull/14329#discussion_r1315675510
##########
core/src/main/java/kafka/log/remote/RemoteLogManager.java:
##########
@@ -349,41 +350,46 @@ public void onLeadershipChange(Set<Partition>
partitionsBecomeLeader,
}
/**
- * Deletes the internal topic partition info if delete flag is set as true.
+ * Stop the remote-log-manager task for the given partitions. And, calls
the
+ * {@link RemoteLogMetadataManager#onStopPartitions(Set)} when {@link
StopPartition#deleteLocalLog()} is true.
+ * Deletes the partitions from the remote storage when {@link
StopPartition#deleteRemoteLog()} is true.
*
- * @param topicPartitions topic partitions that needs to be stopped.
- * @param delete flag to indicate whether the given topic
partitions to be deleted or not.
+ * @param stopPartitions topic partitions that needs to be stopped.
* @param errorHandler callback to handle any errors while stopping the
partitions.
*/
- public void stopPartitions(Set<TopicPartition> topicPartitions,
- boolean delete,
+ public void stopPartitions(Set<StopPartition> stopPartitions,
BiConsumer<TopicPartition, Throwable>
errorHandler) {
- LOGGER.debug("Stopping {} partitions, delete: {}",
topicPartitions.size(), delete);
- Set<TopicIdPartition> topicIdPartitions = topicPartitions.stream()
- .filter(topicIdByPartitionMap::containsKey)
- .map(tp -> new TopicIdPartition(topicIdByPartitionMap.get(tp),
tp))
- .collect(Collectors.toSet());
-
- topicIdPartitions.forEach(tpId -> {
+ LOGGER.debug("Stop partitions: {}", stopPartitions);
+ for (StopPartition stopPartition: stopPartitions) {
+ TopicPartition tp = stopPartition.topicPartition();
try {
- RLMTaskWithFuture task = leaderOrFollowerTasks.remove(tpId);
- if (task != null) {
- LOGGER.info("Cancelling the RLM task for tpId: {}", tpId);
- task.cancel();
- }
- if (delete) {
- LOGGER.info("Deleting the remote log segments task for
partition: {}", tpId);
- deleteRemoteLogPartition(tpId);
+ if (topicIdByPartitionMap.containsKey(tp)) {
+ TopicIdPartition tpId = new
TopicIdPartition(topicIdByPartitionMap.get(tp), tp);
+ RLMTaskWithFuture task =
leaderOrFollowerTasks.remove(tpId);
+ if (task != null) {
+ LOGGER.info("Cancelling the RLM task for tpId: {}",
tpId);
+ task.cancel();
+ }
+ if (stopPartition.deleteRemoteLog()) {
+ LOGGER.info("Deleting the remote log segments task for
partition: {}", tpId);
+ deleteRemoteLogPartition(tpId);
+ }
}
Review Comment:
good point. So we are assuming that if a topic is in `topicIdByPartitionMap`
then it has active archival otherwise not. This is not a good source of truth
IMO since we are going to replace it with metadata cache in another PR (why
have two caches for tp <-> tpId mapping).
Nevertheless, let's add a comment for now here and maybe create an
improvement JIRA to not call stopPArtitions for internal or non-tiered topics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
