numinnex commented on code in PR #2958:
URL: https://github.com/apache/iggy/pull/2958#discussion_r2969481232
##########
core/server/src/shard/system/segments.rs:
##########
@@ -172,6 +182,16 @@ impl IggyShard {
continue;
}
+ // Same barrier as delete_expired_segments_for_partition: do not
+ // delete segments containing unconsumed messages.
+ let min_committed = Self::min_committed_offset(
+ &partition.consumer_offsets,
+ &partition.consumer_group_offsets,
+ );
+ if min_committed.is_some_and(|barrier| first.end_offset > barrier)
{
+ continue;
+ }
+
Review Comment:
Do we really want to prevent deleting expired segments that have registered
consumers via committed offsets ? I am not sure whether it is an correct
behavior + it potentially could lead in some pathological cases, where we have
a zombie consumer who doesn't consume anything for a long time), that prevents
the reclamation mechanism from deleting expired segments.
##########
core/server/src/shard/system/segments.rs:
##########
@@ -172,6 +182,16 @@ impl IggyShard {
continue;
}
+ // Same barrier as delete_expired_segments_for_partition: do not
+ // delete segments containing unconsumed messages.
+ let min_committed = Self::min_committed_offset(
+ &partition.consumer_offsets,
+ &partition.consumer_group_offsets,
+ );
+ if min_committed.is_some_and(|barrier| first.end_offset > barrier)
{
+ continue;
+ }
+
Review Comment:
Do we really want to prevent deleting expired segments that have registered
consumers via committed offsets ? I am not sure whether it is an correct
behavior + it potentially could lead in some pathological cases, where we have
a zombie consumer (who doesn't consume anything for a long time), that prevents
the reclamation mechanism from deleting expired segments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
