[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273734#comment-14273734 ] Leonid Shalupov commented on CASSANDRA-8535: Nope, it's not about 8544. 8544 is on our current production, 2.1.1. This bug is in 2.1-branch, after 2.1.1. I've seen it in production, so it's not test-only. I'll try to reproduce it and report. java.lang.RuntimeException: Failed to rename XXX to YYY --- Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov Assignee: Joshua McKenzie Fix For: 2.1.3 {code} java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db to build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: java.nio.file.FileSystemException: build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db - build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45] at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) ~[na:1.7.0_45] at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45] at org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166) ~[main/:na] ... 18 common frames omitted {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264791#comment-14264791 ] Leonid Shalupov commented on CASSANDRA-8535: This is ticket is about both of them, I see the issue easily reproduced in production and in tests. It's far easier to reproduce the issue in tests, so I published the stack trace from there. I've applied https://issues.apache.org/jira/secure/attachment/12689356/8399_v2.txt, tests are still failing (right from the start, I do not need to wait). What else can I do? It would be nice to get it fixed in 2.1 (not 3.0) java.lang.RuntimeException: Failed to rename XXX to YYY --- Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov Assignee: Joshua McKenzie {code} java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db to build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: java.nio.file.FileSystemException: build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db - build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45] at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) ~[na:1.7.0_45] at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45] at org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166) ~[main/:na] ... 18 common frames omitted {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8544) Cassandra could not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
[ https://issues.apache.org/jira/browse/CASSANDRA-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264804#comment-14264804 ] Leonid Shalupov commented on CASSANDRA-8544: I've seen it two times locally, one time it was reported by the customer of JetBrains Upsource. It always with compactions_in_progress files, see exception below: {quote} 14:16:38.232 [MemtableFlushWriter:4] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:4,5,main] org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException: C:\Tools\Upsource\data\cassandra\data\system\compactions_in_progress-55080a b05d9c388690a4acb25fe1f77b\system-compactions_in_progress-tmp-ka-13253-Statistics.db: Der Prozess kann nicht auf die Datei zugreifen, da sie von ein em anderen Prozess verwendet wird. at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:135) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:121) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:113) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.io.sstable.SSTableWriter.abort(SSTableWriter.java:355) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.io.sstable.SSTableWriter.abort(SSTableWriter.java:333) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:381) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:312) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.1.jar:2.1.1] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.1.jar:2.1.1] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1053) ~[cassandra-all-2.1.1.jar:2.1.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_71] {quote} This particular exception seems to be fixed by turning off AV and Windows Search, but it's no good to fail on the next start. How are you resolving / working around this? Does it work after another attempt to restart? No workarounds attempted. It doesn't work after another restart. Attaching a patch that should print out some more details on the NPE if you can reproduce it consistently. Unfortunately I can't reproduce it consistently, so no luck here. Cassandra could not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers --- Key: CASSANDRA-8544 URL: https://issues.apache.org/jira/browse/CASSANDRA-8544 Project: Cassandra Issue Type: Bug Environment: Windows Reporter: Leonid Shalupov Assignee: Joshua McKenzie Labels: windows Fix For: 2.1.3 Attachments: 8544_show_npe.txt It happens sometimes after restarts caused by undeletable files under Windows. {quote} Caused by: java.lang.NullPointerException at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:579) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:232) at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:377) at com.jetbrains.cassandra.service.CassandraServiceMain.start(CassandraServiceMain.java:81) ... 6 more {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264791#comment-14264791 ] Leonid Shalupov edited comment on CASSANDRA-8535 at 1/5/15 5:41 PM: This ticket is about both of them, I see the issue easily reproduced in production and in tests. It's far easier to reproduce the issue in tests, so I published the stack trace from there. I've applied https://issues.apache.org/jira/secure/attachment/12689356/8399_v2.txt, tests are still failing (right from the start, I do not need to wait). What else can I do? It would be nice to get it fixed in 2.1 (not 3.0) was (Author: shalupov): This is ticket is about both of them, I see the issue easily reproduced in production and in tests. It's far easier to reproduce the issue in tests, so I published the stack trace from there. I've applied https://issues.apache.org/jira/secure/attachment/12689356/8399_v2.txt, tests are still failing (right from the start, I do not need to wait). What else can I do? It would be nice to get it fixed in 2.1 (not 3.0) java.lang.RuntimeException: Failed to rename XXX to YYY --- Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov Assignee: Joshua McKenzie {code} java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db to build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: java.nio.file.FileSystemException: build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db - build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45] at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) ~[na:1.7.0_45] at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45] at org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166) ~[main/:na] ... 18 common
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264819#comment-14264819 ] Leonid Shalupov commented on CASSANDRA-8325: untested_8325.patch works for me Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt, untested_8325.patch See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260187#comment-14260187 ] Leonid Shalupov commented on CASSANDRA-8325: unsafeCopy1.txt patch fails in Unsafe.getLong with slightly different stack trace. BTW, it's reproducible with just ant clean test-all on FreeBSD {quote} Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j sun.misc.Unsafe.getLong(Ljava/lang/Object;J)J+0 j org.apache.cassandra.utils.FastByteOperations$UnsafeOperations.compareTo(Ljava/lang/Object;JILjava/lang/Object;JI)I+50 j org.apache.cassandra.utils.FastByteOperations$UnsafeOperations.compareTo(Ljava/lang/Object;JILjava/nio/ByteBuffer;)I+84 j org.apache.cassandra.utils.FastByteOperations$UnsafeOperations.compareTo(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I+57 j org.apache.cassandra.utils.FastByteOperations$UnsafeOperations.compare(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I+2 j org.apache.cassandra.utils.FastByteOperations.compareUnsigned(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I+5 j org.apache.cassandra.utils.ByteBufferUtil.compareUnsigned(Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I+2 j org.apache.cassandra.db.DecoratedKey.compareTo(Lorg/apache/cassandra/db/RowPosition;)I+52 J 1215 C1 org.apache.cassandra.db.DecoratedKey.compareTo(Ljava/lang/Object;)I (9 bytes) @ 0x000804262b8c [0x000804262940+0x24c] J 1214 C1 java.util.concurrent.ConcurrentSkipListMap.cpr(Ljava/util/Comparator;Ljava/lang/Object;Ljava/lang/Object;)I (26 bytes) @ 0x00080426258c [0x000804262260+0x32c] j java.util.concurrent.ConcurrentSkipListMap.doGet(Ljava/lang/Object;)Ljava/lang/Object;+103 j java.util.concurrent.ConcurrentSkipListMap.get(Ljava/lang/Object;)Ljava/lang/Object;+2 j org.apache.cassandra.db.Memtable.getColumnFamily(Lorg/apache/cassandra/db/DecoratedKey;)Lorg/apache/cassandra/db/ColumnFamily;+5 j org.apache.cassandra.db.CollationController.collectAllData(Z)Lorg/apache/cassandra/db/ColumnFamily;+133 j org.apache.cassandra.db.CollationController.getTopLevelColumns(Z)Lorg/apache/cassandra/db/ColumnFamily;+39 j org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(Lorg/apache/cassandra/db/filter/QueryFilter;I)Lorg/apache/cassandra/db/ColumnFamily;+40 j org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(Lorg/apache/cassandra/db/filter/QueryFilter;)Lorg/apache/cassandra/db/ColumnFamily;+141 j org.apache.cassandra.schema.LegacySchemaTables.readSchemaPartitionForKeyspace(Ljava/lang/String;Lorg/apache/cassandra/db/DecoratedKey;)Lorg/apache/cassandra/db/Row;+19 j org.apache.cassandra.schema.LegacySchemaTables.readSchemaPartitionForKeyspace(Ljava/lang/String;Ljava/lang/String;)Lorg/apache/cassandra/db/Row;+15 j org.apache.cassandra.schema.LegacySchemaTables.readSchemaForKeyspaces(Ljava/lang/String;Ljava/util/Set;)Ljava/util/Map;+38 j org.apache.cassandra.schema.LegacySchemaTables.mergeSchema(Ljava/util/Collection;Z)V+146 ... {quote} Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8544) Cassandra could not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers
Leonid Shalupov created CASSANDRA-8544: -- Summary: Cassandra could not start with NPE in ColumnFamilyStore.removeUnfinishedCompactionLeftovers Key: CASSANDRA-8544 URL: https://issues.apache.org/jira/browse/CASSANDRA-8544 Project: Cassandra Issue Type: Bug Environment: Windows Reporter: Leonid Shalupov It happens sometimes after restarts caused by undeletable files under Windows. {quote} Caused by: java.lang.NullPointerException at org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:579) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:232) at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:377) at com.jetbrains.cassandra.service.CassandraServiceMain.start(CassandraServiceMain.java:81) ... 6 more {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260206#comment-14260206 ] Leonid Shalupov commented on CASSANDRA-8325: I'm ready to try other patches :) Since I've set up the environment it would not take much time. Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log, unsafeCopy1.txt See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
Leonid Shalupov created CASSANDRA-8535: -- Summary: java.lang.RuntimeException: Failed to rename XXX to YYY Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov {code} [22:11:59] : [junit] java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db to build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] [22:11:59] : [junit]at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] [22:11:59] : [junit]at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] [22:11:59] : [junit]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] [22:11:59] : [junit]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] [22:11:59] : [junit]at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257449#comment-14257449 ] Leonid Shalupov commented on CASSANDRA-8535: Background: after experiencing various issues like CASSANDRA-8019 on 2.1.1 we decided to try latest cassandra-2.1 branch. It failed with exception I quoted in description. This time I've set up continuous integration for Cassandra on our server: https://teamcity.jetbrains.com/project.html?projectId=ApacheCassandraguest=1 to reproduce it (feel free to use it, just write me) I run ant test-all there and nothing more. And build failed with this exception: https://teamcity.jetbrains.com/viewLog.html?tab=buildLogbuildTypeId=ApacheCassandra_TestsWindowsbuildId=365436guest=1 java.lang.RuntimeException: Failed to rename XXX to YYY --- Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov {code} [22:11:59] : [junit] java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db to build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] [22:11:59] : [junit]at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] [22:11:59] : [junit]at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] [22:11:59] : [junit]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[jira] [Updated] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonid Shalupov updated CASSANDRA-8535: --- Description: {code} java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db to build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: java.nio.file.FileSystemException: build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db - build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45] at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) ~[na:1.7.0_45] at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45] at org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166) ~[main/:na] ... 18 common frames omitted {code} was: {code} [22:11:59] : [junit] java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db to build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] [22:11:59] : [junit]at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[ https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14257469#comment-14257469 ] Leonid Shalupov commented on CASSANDRA-8535: It failed with org.apache.cassandra.cache.AutoSavingCacheTest testSerializeAndLoadKeyCache test first, then org.apache.cassandra.config.DefsTest dropKS and followed by many others. Please read the log, it's filled with them: https://teamcity.jetbrains.com/downloadBuildLog.html?buildId=365436guest=1 java.lang.RuntimeException: Failed to rename XXX to YYY --- Key: CASSANDRA-8535 URL: https://issues.apache.org/jira/browse/CASSANDRA-8535 Project: Cassandra Issue Type: Bug Environment: Windows 2008 X64 Reporter: Leonid Shalupov {code} java.lang.RuntimeException: Failed to rename build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db to build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226) ~[main/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] Caused by: java.nio.file.FileSystemException: build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db - build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) ~[na:1.7.0_45] at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) ~[na:1.7.0_45] at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) ~[na:1.7.0_45] at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45] at org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184) ~[main/:na] at org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166) ~[main/:na] ... 18 common frames omitted {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224318#comment-14224318 ] Leonid Shalupov commented on CASSANDRA-8325: Tried with loop and without it. unsafe.getByte(l) always ok, returns value consistent with putByte unsafe.getByte(null, l) always fails with {code} # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00080243d1f0, pid=39405, tid=34384930816 # # JRE version: OpenJDK Runtime Environment (7.0-b14) (build 1.7.0_71-b14) # Java VM: OpenJDK 64-Bit Server VM (24.71-b01 mixed mode bsd-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x83d1f0] JVM_handle_bsd_signal+0x113470 # # Core dump written. Default location: /cores/core or core.39405 # # An error report file with more information is saved as: # /usr/home/upsource/hs_err_pid39405.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp # {code} Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224892#comment-14224892 ] Leonid Shalupov commented on CASSANDRA-8325: I think it's more of are documented NOT to be the same Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219184#comment-14219184 ] Leonid Shalupov commented on CASSANDRA-8325: {code} public class A { public static void main(String[] args) throws Exception { java.lang.reflect.Field f = sun.misc.Unsafe.class.getDeclaredField(theUnsafe); f.setAccessible(true); sun.misc.Unsafe unsafe = (sun.misc.Unsafe) f.get(null); long l = unsafe.allocateMemory(900L*1024*1024); System.err.println(l); } } {code} $ javac A.java $ java A 36679188480 Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14216287#comment-14216287 ] Leonid Shalupov commented on CASSANDRA-8325: - It always crashes, no random occurrences - 2.0.x was ok (do not remember exact minor version) - I'll try to run it under linux compat Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
Leonid Shalupov created CASSANDRA-8325: -- Summary: Cassandra 2.1.x fails to start on FreeBSD (JVM crash) Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214527#comment-14214527 ] Leonid Shalupov commented on CASSANDRA-8325: Related thread in FreeBSD mailing list: http://lists.freebsd.org/pipermail/freebsd-stable/2014-October/080834.html {quote} Looks like it might be cassandra bug, as it's calling sun.misc.Unsafe.getByte() which can crash the JVM with bad addresses. {quote} Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonid Shalupov updated CASSANDRA-8325: --- Attachment: system.log Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214543#comment-14214543 ] Leonid Shalupov edited comment on CASSANDRA-8325 at 11/17/14 11:09 AM: --- STR: 1. Get FreeBSD 10.0, install openjdk via pkg install openjdk 7 2. Download and unpack apache-cassandra-2.1.2-bin.tar.gz 3. Run ./cassandra in apache-cassandra-2.1.2/bin 4. In couple of seconds Cassandra will crash was (Author: shalupov): STR: 1. Get FreeBSD 10.1, install openjdk via pkg install openjdk 7 2. Download and unpack apache-cassandra-2.1.2-bin.tar.gz 3. Run ./cassandra in apache-cassandra-2.1.2/bin 4. In couple of seconds Cassandra will crash Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8325) Cassandra 2.1.x fails to start on FreeBSD (JVM crash)
[ https://issues.apache.org/jira/browse/CASSANDRA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214543#comment-14214543 ] Leonid Shalupov commented on CASSANDRA-8325: STR: 1. Get FreeBSD 10.1, install openjdk via pkg install openjdk 7 2. Download and unpack apache-cassandra-2.1.2-bin.tar.gz 3. Run ./cassandra in apache-cassandra-2.1.2/bin 4. In couple of seconds Cassandra will crash Cassandra 2.1.x fails to start on FreeBSD (JVM crash) - Key: CASSANDRA-8325 URL: https://issues.apache.org/jira/browse/CASSANDRA-8325 Project: Cassandra Issue Type: Bug Environment: FreeBSD 10.0 with openjdk version 1.7.0_71, 64-Bit Server VM Reporter: Leonid Shalupov Attachments: hs_err_pid1856.log, system.log See attached error file after JVM crash {quote} FreeBSD xxx.intellij.net 10.0-RELEASE FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 {quote} {quote} % java -version openjdk version 1.7.0_71 OpenJDK Runtime Environment (build 1.7.0_71-b14) OpenJDK 64-Bit Server VM (build 24.71-b01, mixed mode) {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099920#comment-14099920 ] Leonid Shalupov commented on CASSANDRA-7754: Another one: 01:04:42.171 [MemtableFlushWriter:4] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:4,5,main] org.apache.cassandra.io.FSWriteError: java.io.FileNotFoundException: /xxx/data/ideace/ideaindex-3457657025a211e4af4261d16c1df9f6/ideace-ideaindex-tmp-ka-35-CompressionInfo.db (No such file or directory) at org.apache.cassandra.io.compress.CompressedSequentialWriter.close(CompressedSequentialWriter.java:264) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.FileUtils.closeQuietly(FileUtils.java:222) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.abort(SSTableWriter.java:347) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.abort(SSTableWriter.java:331) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:375) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65] Caused by: java.io.FileNotFoundException: /xxx/data/ideace/ideaindex-3457657025a211e4af4261d16c1df9f6/ideace-ideaindex-tmp-ka-35-CompressionInfo.db (No such file or directory) at java.io.FileOutputStream.open(Native Method) ~[na:1.7.0_65] at java.io.FileOutputStream.init(FileOutputStream.java:221) ~[na:1.7.0_65] at java.io.FileOutputStream.init(FileOutputStream.java:110) ~[na:1.7.0_65] at org.apache.cassandra.io.compress.CompressionMetadata$Writer.close(CompressionMetadata.java:351) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.compress.CompressedSequentialWriter.close(CompressedSequentialWriter.java:260) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 12 common frames omitted 01:04:42.479 [MemtableFlushWriter:4] ERROR o.a.cassandra.service.StorageService - Stopping gossiper 01:04:44.480 [MemtableFlushWriter:4] ERROR o.a.cassandra.service.StorageService - Stopping RPC server 01:04:44.481 [MemtableFlushWriter:4] ERROR o.a.cassandra.service.StorageService - Stopping native transport FileNotFoundException in MemtableFlushWriter Key: CASSANDRA-7754 URL: https://issues.apache.org/jira/browse/CASSANDRA-7754 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov Exception in cassandra logs, after upgrade to 2.1: [MemtableFlushWriter:91] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:91,5,main] java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5]
[jira] [Updated] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonid Shalupov updated CASSANDRA-7754: --- Priority: Critical (was: Major) FileNotFoundException in MemtableFlushWriter Key: CASSANDRA-7754 URL: https://issues.apache.org/jira/browse/CASSANDRA-7754 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov Priority: Critical Exception in cassandra logs, after upgrade to 2.1: [MemtableFlushWriter:91] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:91,5,main] java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65] Caused by: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_65] at java.io.RandomAccessFile.init(RandomAccessFile.java:241) ~[na:1.7.0_65] at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:71) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 14 common frames omitted -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099923#comment-14099923 ] Leonid Shalupov commented on CASSANDRA-7754: Found on WARN level, seems to be related: 01:04:44.399 [SharedPool-Worker-3] DEBUG o.a.c.service.FileCacheService - Evicting cold readers for /xxx/data/ideace/content-38cb3a5025a211e4af4261d16c1df9f6/ideace-content-ka-16-Data.db 01:04:44.426 [SharedPool-Worker-3] WARN o.a.c.c.AbstractTracingAwareExecutorService - Uncaught exception on thread Thread[SharedPool-Worker-3,10,main]: {} java.lang.RuntimeException: java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/data/ideace/content-38cb3a5025a211e4af4261d16c1df9f6/ideace-content-ka-16-Data.db (No such file or directory) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2068) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_65] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/data/ideace/content-38cb3a5025a211e4af4261d16c1df9f6/ideace-content-ka-16-Data.db (No such file or directory) at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:47) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:59) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:40) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1689) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.createFileDataInput(SSTableNamesIterator.java:93) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.readIndexedColumns(SSTableNamesIterator.java:209) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.read(SSTableNamesIterator.java:142) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.init(SSTableNamesIterator.java:59) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:89) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.collectTimeOrderedData(CollationController.java:125) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:59) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1873) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1681) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:345) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1393) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2065) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 4 common frames omitted Caused by: java.io.FileNotFoundException: /xxx/data/ideace/content-38cb3a5025a211e4af4261d16c1df9f6/ideace-content-ka-16-Data.db (No such file or directory) at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_65] at java.io.RandomAccessFile.init(RandomAccessFile.java:241) ~[na:1.7.0_65] at org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:58) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at
[jira] [Commented] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099955#comment-14099955 ] Leonid Shalupov commented on CASSANDRA-7754: Thanks! I've found a race condition with cleanup of the data directory in my own code. FileNotFoundException in MemtableFlushWriter Key: CASSANDRA-7754 URL: https://issues.apache.org/jira/browse/CASSANDRA-7754 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov Priority: Critical Exception in cassandra logs, after upgrade to 2.1: [MemtableFlushWriter:91] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:91,5,main] java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65] Caused by: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_65] at java.io.RandomAccessFile.init(RandomAccessFile.java:241) ~[na:1.7.0_65] at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:71) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 14 common frames omitted -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
Leonid Shalupov created CASSANDRA-7754: -- Summary: FileNotFoundException in MemtableFlushWriter Key: CASSANDRA-7754 URL: https://issues.apache.org/jira/browse/CASSANDRA-7754 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov Exception in cassandra logs, after upgrade to 2.1: [MemtableFlushWriter:91] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:91,5,main] java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65] Caused by: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_65] at java.io.RandomAccessFile.init(RandomAccessFile.java:241) ~[na:1.7.0_65] at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:71) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 14 common frames omitted -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7754) FileNotFoundException in MemtableFlushWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093890#comment-14093890 ] Leonid Shalupov commented on CASSANDRA-7754: Another one: java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/data/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-tmp-ka-147-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_65] Caused by: java.io.FileNotFoundException: /xxx/data/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-tmp-ka-147-Index.db (No such file or directory) at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_65] at java.io.RandomAccessFile.init(RandomAccessFile.java:241) ~[na:1.7.0_65] at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:71) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] ... 14 common frames omitted FileNotFoundException in MemtableFlushWriter Key: CASSANDRA-7754 URL: https://issues.apache.org/jira/browse/CASSANDRA-7754 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov Exception in cassandra logs, after upgrade to 2.1: [MemtableFlushWriter:91] ERROR o.a.c.service.CassandraDaemon - Exception in thread Thread[MemtableFlushWriter:91,5,main] java.lang.RuntimeException: java.io.FileNotFoundException: /xxx/cassandra/data/system/batchlog-0290003c977e397cac3efdfdc01d626b/system-batchlog-tmp-ka-186-Index.db (No such file or directory) at org.apache.cassandra.io.util.SequentialWriter.init(SequentialWriter.java:75) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:104) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:99) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.init(SSTableWriter.java:550) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.sstable.SSTableWriter.init(SSTableWriter.java:134) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:383) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:330) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
[jira] [Created] (CASSANDRA-7756) NullPointerException in getTotalBufferSize
Leonid Shalupov created CASSANDRA-7756: -- Summary: NullPointerException in getTotalBufferSize Key: CASSANDRA-7756 URL: https://issues.apache.org/jira/browse/CASSANDRA-7756 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov 18:59:50.499 [SharedPool-Worker-1] WARN o.apache.cassandra.io.util.FileUtils - Failed closing /xxx/cassandra/data/pr1407782307/trigramindexcounter-d2817030218611e4b65c619763d48c52/pr1407782307-trigramindexcounter-ka-1-Data.db - chunk length 65536, data length 8199819. java.lang.NullPointerException: null at org.apache.cassandra.io.util.RandomAccessReader.getTotalBufferSize(RandomAccessReader.java:157) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.compress.CompressedRandomAccessReader.getTotalBufferSize(CompressedRandomAccessReader.java:159) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.FileCacheService.sizeInBytes(FileCacheService.java:186) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.FileCacheService.put(FileCacheService.java:150) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.PoolingSegmentedFile.recycle(PoolingSegmentedFile.java:50) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.RandomAccessReader.close(RandomAccessReader.java:230) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.FileUtils.closeQuietly(FileUtils.java:222) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.init(SSTableNamesIterator.java:69) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:89) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:261) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:59) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1873) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1681) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:345) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.getCurrentValuesFromCFS(CounterMutation.java:274) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.getCurrentValues(CounterMutation.java:241) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.processModifications(CounterMutation.java:209) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.apply(CounterMutation.java:136) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1116) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2065) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_65] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7756) NullPointerException in getTotalBufferSize
[ https://issues.apache.org/jira/browse/CASSANDRA-7756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094166#comment-14094166 ] Leonid Shalupov commented on CASSANDRA-7756: Reproduced in 2.1.0-rc5, so https://github.com/apache/cassandra/commit/900f29c7f7e1d563c4b0c63eae0da8877766813f does not fix it NullPointerException in getTotalBufferSize -- Key: CASSANDRA-7756 URL: https://issues.apache.org/jira/browse/CASSANDRA-7756 Project: Cassandra Issue Type: Bug Environment: Linux, OpenJDK 1.7 Reporter: Leonid Shalupov 18:59:50.499 [SharedPool-Worker-1] WARN o.apache.cassandra.io.util.FileUtils - Failed closing /xxx/cassandra/data/pr1407782307/trigramindexcounter-d2817030218611e4b65c619763d48c52/pr1407782307-trigramindexcounter-ka-1-Data.db - chunk length 65536, data length 8199819. java.lang.NullPointerException: null at org.apache.cassandra.io.util.RandomAccessReader.getTotalBufferSize(RandomAccessReader.java:157) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.compress.CompressedRandomAccessReader.getTotalBufferSize(CompressedRandomAccessReader.java:159) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.FileCacheService.sizeInBytes(FileCacheService.java:186) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.FileCacheService.put(FileCacheService.java:150) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.PoolingSegmentedFile.recycle(PoolingSegmentedFile.java:50) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.RandomAccessReader.close(RandomAccessReader.java:230) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.io.util.FileUtils.closeQuietly(FileUtils.java:222) ~[cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.columniterator.SSTableNamesIterator.init(SSTableNamesIterator.java:69) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.NamesQueryFilter.getSSTableColumnIterator(NamesQueryFilter.java:89) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:261) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:59) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1873) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1681) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:345) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.getCurrentValuesFromCFS(CounterMutation.java:274) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.getCurrentValues(CounterMutation.java:241) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.processModifications(CounterMutation.java:209) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.db.CounterMutation.apply(CounterMutation.java:136) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1116) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2065) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_65] at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:163) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103) [cassandra-all-2.1.0-rc5.jar:2.1.0-rc5] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
[ https://issues.apache.org/jira/browse/CASSANDRA-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13606355#comment-13606355 ] Leonid Shalupov commented on CASSANDRA-4687: Things in this issue happens without dropping or truncating keyspaces or column families. Is it really a duplicate? Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) --- Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Fix For: 1.2.4 Attachments: 4687-debugging.txt Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
[ https://issues.apache.org/jira/browse/CASSANDRA-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567501#comment-13567501 ] Leonid Shalupov commented on CASSANDRA-4687: User on SO has the same problem: http://stackoverflow.com/questions/14131558/cql3-select-succeeds-for-some-keys-but-fails-for-others/14611903 Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) --- Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Assignee: Pavel Yaskevich Fix For: 1.2.2 Attachments: 4687-debugging.txt Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5052) cassandra-cli should escape keyspace name
Leonid Shalupov created CASSANDRA-5052: -- Summary: cassandra-cli should escape keyspace name Key: CASSANDRA-5052 URL: https://issues.apache.org/jira/browse/CASSANDRA-5052 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.7 Reporter: Leonid Shalupov show schema yields use __someKeyspace, expecting use '__someKeyspace'. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
[ https://issues.apache.org/jira/browse/CASSANDRA-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461105#comment-13461105 ] Leonid Shalupov commented on CASSANDRA-4687: It's different in my case: {code} [INFO ] 21 сен 2012 13:18:25 CacheService - Initializing key cache with capacity of 50 MBs. [INFO ] 21 сен 2012 13:18:25 CacheService - Scheduling key cache save to each 14400 seconds (going to save all keys). [INFO ] 21 сен 2012 13:18:25 CacheService - Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider [INFO ] 21 сен 2012 13:18:25 CacheService - Scheduling row cache save to each 0 seconds (going to save all keys). {code} Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) --- Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.5 Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Assignee: Pavel Yaskevich Priority: Critical Fix For: 1.1.6 Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
[ https://issues.apache.org/jira/browse/CASSANDRA-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461107#comment-13461107 ] Leonid Shalupov commented on CASSANDRA-4687: Unfortunately, I can't reproduce it every time now like some weeks ago, when I did git bisect. Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) --- Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.5 Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Assignee: Pavel Yaskevich Priority: Critical Fix For: 1.1.6 Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
[ https://issues.apache.org/jira/browse/CASSANDRA-4687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461115#comment-13461115 ] Leonid Shalupov commented on CASSANDRA-4687: Additional information: - it happens after flushing - they appear in pairs in different ReadStage threads Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) --- Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.5 Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Assignee: Pavel Yaskevich Priority: Critical Fix For: 1.1.6 Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4687) Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk)
Leonid Shalupov created CASSANDRA-4687: -- Summary: Exception: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) Key: CASSANDRA-4687 URL: https://issues.apache.org/jira/browse/CASSANDRA-4687 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.5 Environment: CentOS 6.3 64-bit, Oracle JRE 1.6.0.33 64-bit, single node cluster Reporter: Leonid Shalupov Priority: Critical Under heavy write load sometimes cassandra fails with assertion error. git bisect leads to commit 295aedb278e7a495213241b66bc46d763fd4ce66. works fine if global key/row caches disabled in code. {quote} java.lang.AssertionError: DecoratedKey(xxx, yyy) != DecoratedKey(zzz, kkk) in /var/lib/cassandra/data/...-he-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1345) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1207) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1142) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:819) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1253) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) {quote} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira