[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-14349: -- Source Control Link: https://github.com/shichao-an/cassandra/commits/14349-trunk Resolution: Fixed Status: Resolved (was: Ready to Commit) > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Low > Fix For: 4.0-rc2, 3.11.10 > > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-14349: -- Fix Version/s: 3.11.10 4.0-rc2 > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Low > Fix For: 3.11.10, 4.0-rc2 > > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14349: Status: Ready to Commit (was: Patch Available) > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-14349: Reviewer: Joshua McKenzie > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14349) Untracked CDC segment files are not deleted after replay
[ https://issues.apache.org/jira/browse/CASSANDRA-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate McCall updated CASSANDRA-14349: Status: Patch Available (was: Open) > Untracked CDC segment files are not deleted after replay > > > Key: CASSANDRA-14349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14349 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Shichao An >Assignee: Shichao An >Priority: Minor > > When CDC is enabled, a hard link to each commit log file will be created in > cdc_raw directory. Those commit logs with CDC mutations will also have cdc > index files created along with the hard links; these are intended for the > consumer to handle and clean them up. > However, if we don't produce any CDC traffic, those hard links in cdc_raw > will be never cleaned up (because hard links will still be created, without > the index files), whereas the real original commit logs are correctly deleted > after replay during process startup. This will results in many untracked hard > links in cdc_raw if we restart the cassandra process many times. I am able to > use CCM to reproduce it in trunk version which has the CASSANDRA-12148 > changes. > This seems a bug in handleReplayedSegment of the commit log segment manager > which neglects to take care of CDC commit logs. I will attach a patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
