[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13068887#comment-13068887 ] Hudson commented on CASSANDRA-2521: --- Integrated in Cassandra #967 (See [https://builds.apache.org/job/Cassandra/967/]) Use reference counting to delete sstables instead of relying on the GC patch by slebresne; reviewed by jbellis for CASSANDRA-2521 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1149085 Files : * /cassandra/trunk/test/unit/org/apache/cassandra/io/sstable/SSTableUtils.java * /cassandra/trunk/src/java/org/apache/cassandra/service/AntiEntropyService.java * /cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/trunk/src/java/org/apache/cassandra/db/DataTracker.java * /cassandra/trunk/src/java/org/apache/cassandra/db/Table.java * /cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java * /cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java * /cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOutSession.java * /cassandra/trunk/test/unit/org/apache/cassandra/streaming/SerializationsTest.java * /cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java * /cassandra/trunk/src/java/org/apache/cassandra/io/util/BufferedSegmentedFile.java * /cassandra/trunk/src/java/org/apache/cassandra/io/util/SegmentedFile.java * /cassandra/trunk/src/java/org/apache/cassandra/io/util/MmappedSegmentedFile.java * /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableDeletingTask.java * /cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java * /cassandra/trunk/CHANGES.txt * /cassandra/trunk/src/java/org/apache/cassandra/streaming/PendingFile.java * /cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamInSession.java * /cassandra/trunk/src/java/org/apache/cassandra/streaming/StreamOut.java * /cassandra/trunk/src/java/org/apache/cassandra/service/GCInspector.java * /cassandra/trunk/test/unit/org/apache/cassandra/streaming/StreamingTransferTest.java * /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableReader.java * /cassandra/trunk/src/java/org/apache/cassandra/io/sstable/SSTableDeletingReference.java Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt, 2521-v5.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061761#comment-13061761 ] Jeff Kesselman commented on CASSANDRA-2521: --- Just a note: while relying on phantom references or finalizer for your cleanup is incorrect, it is reasonable to use them to do a verification that your cleanup has occurred on those objects that do get collected. This is with the caveat however that PhantomReferences always add something to your gc costs so you might want to make it switchable. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13059908#comment-13059908
]
Terje Marthinussen commented on CASSANDRA-2521:
---
I found an error in how the patch had gotten merged in our code.
After fixing that, I see no Compacted files around that I cannot explain.
This found the the error:
{noformat}
public void releaseReference()
{
if (holdReferences.decrementAndGet() == 0 isCompacted.get())
{
// Force finalizing mmapping if necessary
ifile.cleanup();
dfile.cleanup();
deletingTask.schedule();
}
assert holdReferences.get() = 0 : Reference counter +
holdReferences.get() + for + dfile.path;
}
{noformat}
that assert turned quite useful so a recommended addon.
Been doing a lot of updating, repairing, querying since then on a few system
here ranging from 3 nodes with 100-200 million docs to 12 nodes with more than
a billion. Nothing abnormal so far. Things that made us run out of disk before
or sent nodes into full GC land, now works.
I would still like to see that full GC is not called if disk runs past 90%
usage as and mmap is not used. No point halting the world if not needed?
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch,
0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch,
0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057295#comment-13057295 ] Terje Marthinussen commented on CASSANDRA-2521: --- I have not found any further major issues here, but I think there is still situations where files are deleted late but they do seem to go away. Not sure if we are missing something in terms of reference counting and GC delete it eventually or it is just a delayed free or delete for some reason, but it does not happen too often. Will see try to add some debug logging and see what I find. I was looking at the code though and I am wondering about one segment. I have not had time to actually test this, but in submitUserDefined() there is a finally statement removing References for sstables but I could not immediately see where there are References acquired for all the sstables that needs to be freed there? I am sure it's just me missing something, but anyway... Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057305#comment-13057305 ] Sylvain Lebresne commented on CASSANDRA-2521: - bq. Not sure if we are missing something in terms of reference counting and GC delete it eventually If you don't use mmap, the GC shouldn't do anything. So if it is deleted eventually, it would indicate some place where we last decrement is delayed longer that it needs somehow. It'd be interesting if you find more. bq. in submitUserDefined() there is a finally statement removing References for sstables but I could not immediately see where there are References acquired It's in the lookupSSTables (a private method used by submitUserDefined). I admit it's not super clean but It felt like the simplest way to do this in a thread safe manner without holding unneeded references for too long. bq. Only thing missing beyond that is to get this into 0.8. I really don't think this would be reasonable. This is not a trivial change by any mean, nor does it fixes a regression. Which is not saying it doesn't make life much easier. But I'm really uncomfortable pushing that to 0.8. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057614#comment-13057614 ] Terje Marthinussen commented on CASSANDRA-2521: --- In releaseReference(), if holdReferences for some reason gets less than 0, maybe the code should do an assert or throw an exception (or anything else that gives a stack trace)? Should help debugging for some error scenarios with reference mismatches. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056406#comment-13056406
]
Sylvain Lebresne commented on CASSANDRA-2521:
-
bq. but I guess these may be results of references acquired which are not freed
as the streaming fills up the disk and fails.
Yes, until we have CASSANDRA-2433 (rebased with this), we don't detect failing
streaming and thus never delete the files (until restart). Which btw make me
say that we better have CASSANDRA-2433 if we have this ticket. But that was the
plan anyway.
bq. there are no less but 53 tmp files. A lot of concurrent streams here!
Though it is not related to this ticket, I'll note that CASSANDRA-2816 only
stagger the merkle tree creation, not the streaming. That is, the streaming
will be staggered to some extends, but if the streaming part is much longer
than the merkle tree creation one, you will still have lots of concurrent
stream going on. But -tmp files also includes the compaction that are going on,
and failed repair leaves -tmp file around, which could also help explaining
there large number. In any case, this is not related to the issue at hand :)
{quote}
However I noticed this in the log:
INFO [Thread-185] 2011-06-28 05:01:15,390 StorageService.java (line 2083)
requesting GC to free disk space
I guess we can get rid of that?
{quote}
In some cases (mmap with non-sun jvm at least) we are still relying on the GC
to free space.
Terje, if you can confirm that you didn't saw something utterly wrong with the
last patch (related to that patch, no repair), I'll commit it. I think having
it in trunk quicker will help with having more testing quicker. And given that
we don't want to have bugs in our force unmapping it'll be a good thing. In
particular, could be good to have someone try that on windows.
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch,
0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch,
0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056645#comment-13056645 ] Terje Marthinussen commented on CASSANDRA-2521: --- Latest iteration of this patch look great in my case, but please note again, I have not tested mmap. Maybe you can make the full GC trigger conditional if mmap is used? Only thing missing beyond that is to get this into 0.8. Disk use is very significantly improved in some cases and I think this could easily be the biggest improvement you can do in 0.8.2 in terms of operation friendliness. Regarding the above mentioned tmp tables, there is definitely issues there, but don't think it is related to this patch so feel free to merge. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055887#comment-13055887 ] Jonathan Ellis commented on CASSANDRA-2521: --- +1 from me in principle, suggest waiting for Terje's test before committing tho :) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055888#comment-13055888 ] Jonathan Ellis commented on CASSANDRA-2521: --- (suggest adding a comment for the markCompacted acquire/release code on commit.) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056239#comment-13056239 ] Terje Marthinussen commented on CASSANDRA-2521: --- First impression is good. I merged this with 2816 last night and tested overnight with a dataset that normally uses 32-40% disk space when compacted. Calling repair on a handful of nodes, I would normally have a good chance of seeing a node or two with 90%+ full disk and regular full GC's. With these patches, disk use seem to peak in the 55-65% disk area which is a very significant improvement. However, unfortunately, I started repairs on 5 of 12 nodes and one of them has gone crazy however and filled the disk 100%. :( I did happen to start repair twice on this node by accident. I somewhat do not think that is the real problem however, but I am not sure. Will make a ticket anyway on adding functionality to prevent repairs from getting started twice on the same CF and I will test more to see if this happens again. However I noticed this in the log: INFO [Thread-185] 2011-06-28 05:01:15,390 StorageService.java (line 2083) requesting GC to free disk space I guess we can get rid of that? Should be nothing good from calling full GC there anymore? If anything, maybe we should consider a variation here where cassandra instead aborts all repairs and compactions and clean up everything before trying again with smaller amounts of data in each compaction/repair attempt? Another ticket on that maybe? Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056263#comment-13056263 ] Terje Marthinussen commented on CASSANDRA-2521: --- Looking a bit more on files and logs from the node with full disk. I suspect things mainly went wrong due to 2 repairs (if they actually both ran) 1. Seems there are some Compacted files around which does not get cleaned up, but I guess these may be results of references acquired which are not freed as the streaming fills up the disk and fails. 2. there are no less but 53 -tmp- files. A lot of concurrent streams here! I still think it may be a good idea if we could detect that we were heading for full disk and abort everything before things crashed. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13056268#comment-13056268 ] Jonathan Ellis commented on CASSANDRA-2521: --- bq. I still think it may be a good idea if we could detect that we were heading for full disk and abort everything before things crashed That's a good feature to have, but out of scope here. :) CASSANDRA-809 is open as a general deal better with running out of space ticket, but we can open more specific ones. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt, 2521-v4.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055026#comment-13055026 ] Terje Marthinussen commented on CASSANDRA-2521: --- Did another repair overnight, one minor compaction included some 20 small sstables, all of them remains as well as a few from other compactions and the files from the repairs described before. Definitely something fishy here. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13055106#comment-13055106 ] Terje Marthinussen commented on CASSANDRA-2521: --- As for the last version of this patch, a quick look tonight shows access problems with markCurrentViewReferenced() which is called from outside cfs. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054844#comment-13054844 ] Terje Marthinussen commented on CASSANDRA-2521: --- I have not tested Jonathan's updated version, but the version before seems to work good so far. Repair still keep data around while running, but that may be solved with CASSANDRA-2816. Other than that, sstables in general seems to disappear within a minute of being freed from compaction. Except for when running repair, I have to actively hunt for -Compacted files to find them now. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054859#comment-13054859 ] Sylvain Lebresne commented on CASSANDRA-2521: - bq. Repair still keep data around while running, but that may be solved with CASSANDRA-2816. This is the expected behavior. Or more precisely, it all depends on what we're talking about. When repair is working with a sstable, we cannot and we should not delete it. Even if they have been compacted, if we're in the middle of streaming them, we should not delete it, there is nothing we can do about it. Now, I've tried to make it so that as soon as a file is not needed by repair anymore, it is deleted and thus if what you're talking about is sstable that get kept around forever, then it is a bug in the patch. To have a -Compacted file around don't mean the sstable can be deleted and it will never be. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054904#comment-13054904 ] Terje Marthinussen commented on CASSANDRA-2521: --- Yes, no problem that files stay for a while during repair, but I believe currently, files are not deleted until the entire repair is finished even if they have completed streaming, and I suppose that is not needed? Regardless, I believe CASSANDRA-2816 may fix this? Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054906#comment-13054906 ] Terje Marthinussen commented on CASSANDRA-2521: --- When that is said, I have to actually verify that nothing is deleted until the repair is completed. I am pretty sure this happened with GC doing the cleanup, but I have actually not verified that with this patch. I just observe that I do not see anything get deleted until it finishes, but just because I don't see that with some occasional ls's does not mean it does not happen. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054941#comment-13054941 ] Terje Marthinussen commented on CASSANDRA-2521: --- Hm... there may actually be something here. I am getting some *Compacted which does not seem to go away, not even after 7 hours after finished. 3 Compactions 2 of them have left 1 sstable, one has left 2. I also happened to run 3 repairs, but one compaction starts just after one of those repairs finished (the other 2 starts and finished during the repair. This is running 2 as minimum compaction threshold. The compaction which left 2 files, left them after a compaction of 3 files. The 2 other *Compacted was made after compactions of 2 files The system is under reasonably heavy input of data and sstables are added, compacted and removed all the time. Just these which do not get removed and the oldest is 8 hours now. Note, I do not use mmap. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054949#comment-13054949 ] Terje Marthinussen commented on CASSANDRA-2521: --- Hm, forget that. All 3 compactions happens during the repairs. This test is also using CASSANDRA-2433 and there are no errors reported during the repairs. Only one of the leftover sstables is listed in the log entries for stream context metadata Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054286#comment-13054286 ] Peter Schuller commented on CASSANDRA-2521: --- For the record, if a subsequent mapping overlaps with the one that got unmapped, it's possible reads give randomly selected data from some other file, or writes corrupting some other file. So failure cases are not limited to crashes (which would be preferable in this case). Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch, 0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch, 2521-v3.txt http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050287#comment-13050287 ] Sylvain Lebresne commented on CASSANDRA-2521: - Actually I started working on this yesterday evening and I think I'm almost done. So re-assigning to myself for now :) Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050309#comment-13050309 ] Terje Marthinussen commented on CASSANDRA-2521: --- Fabulous! If you backport that to 0.8 (or if it is back-portable), I will volunteer to give it a decent doze of real life testing asap. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050551#comment-13050551 ] Jonathan Ellis commented on CASSANDRA-2521: --- bq. the deletion of a file won't work until the mmapping is finalized (aka, GC, Where art thou), at least on windows Ugh, totally forgot about that. Probably one of the reasons we went with might as well use a phantom reference in the first place. Linux does allow the delete but I suspect the space doesn't actually get freed up until the munmap still. So not really an improvement there either. Is it possible to do mmap/munmap via JNA for libc-based systems? I think I'd rather fall back to non-mmap'd i/o on windows, than continue to hack around the GC like this. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050560#comment-13050560
]
T Jake Luciani commented on CASSANDRA-2521:
---
We could use this trick:
{noformat}
try {
// Manually free the old buffer using undocumented unsafe APIs.
// If this fails, we'll drop the reference and hope GC finds it
// eventually.
Object cleaner = buf.getClass().getMethod(cleaner).invoke(buf);
cleaner.getClass().getMethod(clean).invoke(cleaner);
} catch (Exception e) {
// Perhaps a non-sun-derived JVM - contributions welcome
LOG.warn(Couldn't realloc bytebuffer, e);
}
{noformat}
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050597#comment-13050597 ] Ryan King commented on CASSANDRA-2521: -- +1 for less hacking around the GC Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050611#comment-13050611 ] Peter Schuller commented on CASSANDRA-2521: --- @jbellis The *unlink* is fine (just like with files that are opened w/o mmap), but the actual deletion has to be postponed for fundamental reasons: How would the userland application be informed of an attempted access? You could segfault of course, or similar delivery of an asynchronous event, but that's not very useful to most applications. This closely relates to why Java doesn't allow forced munmap to begin with; having to do some kind of synchronization to allow safe access to munmapped memory in a way that doesn't violate the Java sandbox, would be very costly given that a large point of mmap is to utilize the CPU's MMU in the non-faulting case for a minimum of overhead. (I'm very much +1 on not using the GC for external resources like sstables (for trunk).) @jbellis And yes we can do our own mmap:ing with JNA as an alternative, but given that tjake's solution should work for Sun JVM:s even without JNA that seems preferable to me. Longer term support for non-hotspot JVM:s seems like a lesser concern. Cassandra isn't really something you want to run on an arbitrary JVM anyway. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[
https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050637#comment-13050637
]
Jonathan Ellis commented on CASSANDRA-2521:
---
bq. We could use this trick
It's not obvious that this works with mapped buffers (as opposed to normal
malloc'd direct buffers) but it looks like it does:
DirectByteBuffer constructor, used via reflection to create MappedByteBuffer
objects (!)
{code}
// For memory-mapped buffers -- invoked by FileChannelImpl via reflection
protected DirectByteBuffer(int cap, long addr, Runnable unmapper) {
super(-1, 0, cap, cap, true);
address = addr;
viewedBuffer = null;
cleaner = Cleaner.create(this, unmapper);
}
{code}
Here's the unmapper that gets passed, from FCI:
{code}
private static class Unmapper
implements Runnable
{
private long address;
private long size;
private Unmapper(long address, long size) {
assert (address != 0);
this.address = address;
this.size = size;
}
public void run() {
if (address == 0)
return;
unmap0(address, size);
address = 0;
}
}
{code}
(And of course Cleaner.clean just calls run on its Runnable, with a second
layer of only-run-this-once wrapping.)
Move away from Phantom References for Compaction/Memtable
-
Key: CASSANDRA-2521
URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Chris Goffinet
Assignee: Sylvain Lebresne
Fix For: 1.0
Attachments:
0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch
http://wiki.apache.org/cassandra/MemtableSSTable
Let's move to using reference counting instead of relying on GC to be called
in StorageService.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050773#comment-13050773 ] Terje Marthinussen commented on CASSANDRA-2521: --- We stopped using mmap a while back. Benchmarks was highly inconclusive about any performance benefits and the operational benefits of seeing how much memory the darned thing actually uses and general improved stabilty has so far easily outweighed any teorethical lower singel digit performance benefit we could find in static benchmarks and on more realistic use we so far don't really see any performance issues. Personally I have no problems with a patch with that limitation. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050203#comment-13050203 ] Jeff Kesselman commented on CASSANDRA-2521: --- Relying on either finalizers or phantom references for your own clean up is not correct. These mechanisms are not triggered when an object becomes a candidate for collection, but only when the space is actually reclaimed. The time between these two states is up to the gc implementation and can be infinite in that the gc is never *required* to collect any given object unless failure to collect that object would result in an Out Of Memory condition. GCs often only partially collect, choosing the easiest objects to clean up first and going no further if no more space is required. A null gc that never collects space is actually perfectly legal under the java VM specification and has actually been used by IBM in the past for certain operational environments. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
[ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13050204#comment-13050204 ] Jeff Kesselman commented on CASSANDRA-2521: --- It should also be noted that any object with either a finalizer or a phantom reference to it cannot be cleaned up in the eden by the hotspot generational collector and is forced to proceed to the next generation where gc is significantly more expensive. Whether this has any real effect on your gc times depends on the lifetime of the object in question. Creating a lot of short lived objects with finalizers or phantom references is bad. Creating middle to long lived objects with finalizers or phantom references is less of a problem. Move away from Phantom References for Compaction/Memtable - Key: CASSANDRA-2521 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521 Project: Cassandra Issue Type: Improvement Reporter: Chris Goffinet Assignee: Sylvain Lebresne Fix For: 1.0 http://wiki.apache.org/cassandra/MemtableSSTable Let's move to using reference counting instead of relying on GC to be called in StorageService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
