[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104637#comment-13104637 ] Yang Yang commented on CASSANDRA-3179: -- with 1.0.0 HEAD, no more SEGV after a night of stress tests. thanks JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103419#comment-13103419 ] Yang Yang commented on CASSANDRA-3179: -- btw, as I was trying to going through the code to understand the problem, I found it seems that the MMapedSegmentedFile.cleanup() code could be skipped in some sequences of mixed read and compact: read increases the refcount, then DataTracker.replaceCompactedSSTables()--- SSTableReader.releaseReference() could see a refcount of 2, and not call the dfile.cleanup() it's not a big problem ( code seems to run fine with the cleanup() lines commented out), but that kind of thwarts the purpose of adding them in JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103960#comment-13103960 ] Brandon Williams commented on CASSANDRA-3179: - I'm unable to discern any read performance difference between the performance test and v2 patches. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103985#comment-13103985 ] Jonathan Ellis commented on CASSANDRA-3179: --- bq. SSTableReader.releaseReference() could see a refcount of 2, and not call the dfile.cleanup() That means a read thread has a reference still, so cleanup will run when THAT thread releases it. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104035#comment-13104035 ] Yang Yang commented on CASSANDRA-3179: -- thanks. unfortunately I got the SEGV again, after updating to the latest version. I tried to confirm so ran it again, so far seen 2 times. INFO 14:49:57,806 Compacting Minor: [SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h -76-Data.db'), SSTableReader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-77-Data.db'), SSTableR eader(path='/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-79-Data.db'), SSTableReader(path='/usr/scratc h/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-78-Data.db')] 9135.963: [GC 9135.963: [ParNew: 2550601K-133528K(2764800K), 1.2821340 secs] 5163440K-2756799K(5836800K) icms_dc=0 , 1.2824040 secs] [Times: user=1.06 sys=0.04, real=1.28 secs] INFO 14:50:07,017 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/measuredSession-h-88-Data.db ( 21435483 bytes) INFO 14:50:07,018 Writing Memtable-session_limit_filter@839347940(21205810/281849841 serialized/live bytes, 286565 ops) INFO 14:50:15,987 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/session_limit_filter-h-88-Data .db (23326502 bytes) INFO 14:50:15,999 Writing Memtable-ad_impression_session@100750437(21205662/296528055 serialized/live bytes, 286563 ops) 9154.456: [GC 9154.456: [ParNew: 2591128K-125067K(2764800K), 0.2846910 secs] 5214399K-2756607K(5836800K) icms_dc=0 , 0.2849470 secs] [Times: user=0.71 sys=0.01, real=0.29 secs] INFO 14:50:26,220 Completed flushing /usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/ad_impression_session-h-89-Data.db (52154466 bytes) INFO 14:50:26,221 Writing Memtable-ad_ip_agent@1095480823(21205662/286042295 serialized/live bytes, 286563 ops) INFO 14:50:26,463 Compacted to [/usr/scratch/yyang/cass/lib/cassandra/data/testBudget_items/IpFilter-h-80-Data.db,]. 94,782,508 to 83,577,108 (~88% of original) bytes for 6,368 keys at 2.783688MBPS. Time: 28,633ms. INFO 14:50:26,464 CF Total Bytes Compacted: 3,494,202,400 # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2b7b9c88, pid=6461, tid=1253022016 # # JRE version: 7.0-b147 # Java VM: Java HotSpot(TM) 64-Bit Server VM (21.0-b17 mixed mode linux-amd64 compressed oops) # Problematic frame: # J org.apache.cassandra.io.util.AbstractDataInput.readByte()B # # Core dump written. Default location: /usr/scratch/yyang/whisky/core or core.6461 attaching the err file separately JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104055#comment-13104055 ] Brandon Williams commented on CASSANDRA-3179: - Can you reproduce w/o compressed oops? JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104061#comment-13104061 ] Yang Yang commented on CASSANDRA-3179: -- let me try ... On Tue, Sep 13, 2011 at 3:47 PM, Brandon Williams (JIRA) JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104106#comment-13104106 ] Yang Yang commented on CASSANDRA-3179: -- confirmed with the -XX:-UseCompressedOops arg, has generated 1 SEGV so far btw, it would be really helpful to add a simple dumb stress test testcase, some bugs are not easily exposed with the unit tests. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104110#comment-13104110 ] Brandon Williams commented on CASSANDRA-3179: - What process are you following to reproduce? I can't do it with compaction+reads, but I'm also not using JRE 7. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104119#comment-13104119 ] Yang Yang commented on CASSANDRA-3179: -- I'm just writing + reading on the cassandra server, and doing this on the side: while : ;do nodetool flush ; sleep 20;done (disclaimer: the cassandra server is a modified one, where I have a thread that takes in client request through a custom AVRO server, and then call the storageProxy.batch_mutate() and StorageProxy.get() directly. it should not be materially different from the pure cassandra server as far as this bug is concerned, since it's basically just swapping out the thrift server ) On Tue, Sep 13, 2011 at 5:42 PM, Brandon Williams (JIRA) JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104140#comment-13104140 ] Jonathan Ellis commented on CASSANDRA-3179: --- You're sure you're on 1.0.0 = r1170342? JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104141#comment-13104141 ] Yang Yang commented on CASSANDRA-3179: -- very interesting, I switched from mmap mode to standard, and go the following errors on compaction, I believe they follow the same path as the mmap route, but since it gave an exception here, instead of siliently SEGV, this could provide a useful hint to what caused the SEGV Caused by: java.nio.channels.ClosedChannelException at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268) at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640) at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356) at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79) ... 21 more ERROR 01:19:22,415 Fatal exception in thread Thread[ReadStage:246,5,main] java.lang.RuntimeException: java.lang.RuntimeException: error reading 1 of 1 at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1165) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.RuntimeException: error reading 1 of 1 at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:83) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107) at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:194) at org.apache.cassandra.utils.MergeIterator.init(MergeIterator.java:47) at org.apache.cassandra.utils.MergeIterator$ManyToOne.init(MergeIterator.java:142) at org.apache.cassandra.utils.MergeIterator.get(MergeIterator.java:66) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:96) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:249) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:61) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1276) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1171) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1127) at org.apache.cassandra.db.Table.getRow(Table.java:388) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:61) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:694) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1161) ... 3 more Caused by: java.nio.channels.ClosedChannelException at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:268) at java.io.RandomAccessFile.readByte(RandomAccessFile.java:640) at org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:356) at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:367) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:87) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:82) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:72) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:36) at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79) ... 21 more INFO 01:19:23,738 Compacted to [/mnt/cass/lib/cassandra/data/testBudget_items/measuredSession-h-10-Data.db,]. 16,840,264 to 16,840,036 (~99% of original) bytes for 22,531 keys at 3.247707MBPS. Time: 4,945ms. INFO 01:19:23,738 CF Total Bytes Compacted:
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104143#comment-13104143 ] Yang Yang commented on CASSANDRA-3179: -- sorry for the misunderstanding. I applied the v2 patch on 4dbda612f49c97fd5e3f66e7875a20ec9a0dc829 (Aug 26 version) I could test the latest one if you think that's going to be different. On Tue, Sep 13, 2011 at 6:22 PM, Jonathan Ellis (JIRA) j...@apache.org wrote: JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104148#comment-13104148 ] Jonathan Ellis commented on CASSANDRA-3179: --- Please test 1.0.0 branch to make sure we are doing the same thing. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Jonathan Ellis Fix For: 1.0.0 Attachments: 3179-performance-test.txt, 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid6461.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13102746#comment-13102746 ] Brandon Williams commented on CASSANDRA-3179: - I am unable to repro without compaction running. JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13102777#comment-13102777 ] Jonathan Ellis commented on CASSANDRA-3179: --- The problem is that our whole zero copy mmap'd read path is broken by unmap sstables as soon as they are no longer referenced. Specifically, we release references during reads in CollationController or getRangeSlice as soon as we have generated CF objects from the sstables, but we will continue to reference the buffer contents later on when we send the results back to the user or the coordinator (or, as in the 7031 log, just use the results internally). JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3179) JVM segfaults
[ https://issues.apache.org/jira/browse/CASSANDRA-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13103174#comment-13103174 ] Brandon Williams commented on CASSANDRA-3179: - +1 JVM segfaults - Key: CASSANDRA-3179 URL: https://issues.apache.org/jira/browse/CASSANDRA-3179 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: java version 1.6.0_26 Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode) Reporter: Brandon Williams Assignee: Sylvain Lebresne Fix For: 1.0.0 Attachments: 3179-v2.txt, 3179.txt, hs_err_pid12074.log, hs_err_pid28971.log, hs_err_pid7031.log Both with and without compressed OOPs enabled. Seems to mostly happen during compaction+reads. I'll attach some hs_err files shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
