IgorBerman commented on code in PR #16362:
URL: https://github.com/apache/druid/pull/16362#discussion_r1591166270
##########
indexing-service/src/test/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTaskTest.java:
##########
@@ -880,7 +846,7 @@ public void testKillBatchSizeThree() throws Exception
Assert.assertEquals(Collections.emptyList(), observedUnusedSegments);
Assert.assertEquals(
- new KillTaskReport.Stats(4, 3, 4),
+ new KillTaskReport.Stats(4, 3, 0),
Review Comment:
@kfaraz regarding your comment about locks and inter-connection, I have few
questions/comments if you permit me
1. API markAsUsed/markAsUnused MUST NOT happen concurrently with a kill task
this one is interesting. Why markAsUsed/Unused not taking any locks?
2. API markAsUsed/markAsUnused MUST NOT happen concurrently with an APPEND
job.
if we using concurrent locks, this may happen already, isn't it? Once again,
markAsUsed/markAsUnused not using any locks(at least I haven't found it)
3. Overall it seems to me that markAsUsed/markAsUnused not playing "nice"
with concurrent locks currently, am I right?
4. kill should PROBABLY NOT happen concurrently with a REPLACE job in case
the REPLACE job wants to upgrade a segment that the kill is trying to get rid
of. Having the auto-kill buffer period mostly safeguards against such
situations but it is still a possibility.
if one uses kill job with REPLACE lock(default for concurrent locks),
shouldn't 2 REPLACE jobs be mutually exclusive? Maybe we shouldn't permit any
other lock besides REPLACE?
5. kill CAN happen concurrently with an APPEND job, because the append job
itself has the information of all the segments that it needs to commit.
Deleting any other segments from the system should not have any effect on it
(except maybe killing segments of the version to which we are trying to append
data, but that is again safeguarded by the auto-kill buffer period)
if we trying to append data to some segments, shouldn't they be 'used'? I
mean kill job should delete only unused segments, isn't it? Or I'm missing
something
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
