[jira] [Created] (CASSANDRA-9812) Handle corrupted files during startup.
Andreas Schnitzerling created CASSANDRA-9812:
Summary: Handle corrupted files during startup.
Key: CASSANDRA-9812
URL: https://issues.apache.org/jira/browse/CASSANDRA-9812
Project: Cassandra
Issue Type: New Feature
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
This ticket is relying to CASSANDRA-9686 (refer for details). Here the
conclusion: In our company we cannot avoid power-cut of the nodes (unexpected
and for tests). We need a behavior, which keeps the nodes online even on
finding corrupted files during startup. One idea was copy and scrub corrupted
files. [~Stefania] wrote:
{code}
Yes a disk corruption due to a power cut could explain it. I don't think we
should delete corrupt sstables though, but we could maybe move them somewhere
else - where they wouldn't be automatically loaded. Then the scrub tool could
copy the fixed version back into the right folder, but this is kind of opposite
of what it does at the moment (save a backup and then fix the original).
{code}
This could avoid stopping the nodes and keep the cluster running. We need that
behavior only on startup of the nodes, not during runtime. The only cause seems
to be power-cut. The nodes are configured to start C* as a service.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627911#comment-14627911
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
I filed a new ticket Handle corrupted files during startup. (CASSANDRA-9812).
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.1.9, 2.2.0, 3.0 beta 1
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626108#comment-14626108
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
I just want the same behavior as in prior versions of C*. Means not to stop on
corrupted files since I have replication and regular repair of important
keyspaces. So I have to use IGNORE to reach that since I'm not at work everyday
and C* is used everyday.
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.1.9, 2.2.0, 3.0 beta 1
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624561#comment-14624561
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
After applying all patches, the server stops now on corrupted files with
several exceptions. LEAK ERROR I liked more since the server kept on running.
Can we not move those files to another directory since the hard-drive is not
corrupted? Is it necessary to stop on policy stop for these errors?
{code:title=system.log}
WARN [main] 2015-07-13 13:43:49,838 CLibrary.java:72 - JNA link failure, one
or more native method will be unavailable.
WARN [main] 2015-07-13 13:43:49,838 StartupChecks.java:148 - 32bit JVM
detected. It is recommended to run Cassandra on a 64bit JVM for better
performance.
WARN [main] 2015-07-13 13:43:50,048 SigarLibrary.java:167 - Cassandra server
running in degraded mode. Is swap disabled? : false, Address space adequate? :
true, nofile limit adequate? : true, nproc limit adequate? : true
ERROR [SSTableBatchOpen:1] 2015-07-13 13:44:07,959 SSTableReader.java:503 -
Cannot read sstable
D:\Programme\Cassandra\data\data\logdata\onlinedata\la-115-big=[Data.db,
Index.db, Filter.db, CompressionInfo.db, Statistics.db, TOC.txt,
Digest.adler32]; file system error, skipping table
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with
0 chunks encountered: java.io.DataInputStream@18340c2
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:695)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:656)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:450)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:356)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:493)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@18340c2
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
... 15 common frames omitted
ERROR [SSTableBatchOpen:1] 2015-07-13 13:44:07,959 FileUtils.java:464 - Exiting
forcefully due to file system exception on startup, disk failure policy stop
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with
0 chunks encountered: java.io.DataInputStream@18340c2
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc2-SNAPSHOT.jar:2.2.0-rc2-SNAPSHOT]
at
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624687#comment-14624687
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
Would-be nice if I could decide that behavior with a parameter and not ignoring
all in general. If I'm not in my company for some days or on weekend, I cannot
inspect the files. I have to rely on the cluster, even on hard reset of the
nodes.
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.1.x, 2.2.x
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618399#comment-14618399
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
I applied the patch. The only thing disturbing me, is that everytime, I restart
C*, I get long stacktraces as well starting with:{code}ERROR
[SSTableBatchOpen:1] (stacktrace){code}. The error-one-liner {code}ERROR
[Reference-Reaper:1] LEAK DETECTED + Filename{code} should be enough not to
blow up log since corrupted file is not handled by the user yet. Moving the
corrupted files somewhere else would-be a good idea too for my opinion.
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.2.x
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9694:
-
Attachment: system.log.2a.zip
system.log.1a.zip
I made the same test (incl. data, commitlog and saved_caches from 2.1.7) w/
actual 2.2.0-rc2 of today + patches of CASSANDRA-9686.
12:13 logged in via cql as super-user
12:16 logged in via cql as standard-user and tested SELECT on allowed and
restricted tables.
12:18 nodetool stopdaemon.
Result is in system.log.1a.zip + system.log.2a.zip
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc2
Attachments: 9694.txt, system.log.1.zip, system.log.1a.zip,
system.log.2..zip, system.log.2a.zip, system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at
[jira] [Commented] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613087#comment-14613087
]
Andreas Schnitzerling commented on CASSANDRA-9694:
--
I disabled auth and everything else is working correctly. I tested it for 20
hours continuously running.
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc2
Attachments: 9694.txt, system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9694:
-
Attachment: system.log.2..zip
system.log.1.zip
Here the steps:
1. copied 2.2.0 instance w/o commitlog, data, saved_caches
2. created data dir
3. copied (backed up) 2.1.7 data (user-ks + system + system_auth +
system_traces) into data (except user-CF onlinedata which is the bigest
containing 13 GB of data)
4. changed log-level to DEBUG
5. enabled auth
6. started cassandra
7. after 8 minutes nodetool stopdaemon
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc2
Attachments: 9694.txt, system.log.1.zip, system.log.2..zip,
system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
[jira] [Commented] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611726#comment-14611726
]
Andreas Schnitzerling commented on CASSANDRA-9694:
--
During normal operation (importing data) I still get these errors.
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Attachments: system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611681#comment-14611681
]
Andreas Schnitzerling commented on CASSANDRA-9686:
--
I checked the 2.1.7 instance, which I copied before upgrading. There is the
same issue and I have an idea: Our computers are in 3 different laboratories
for electronic engineers. C* is running there as a backround-job. We have
regular emergency-tests once a month in the morning. They switch off power and
computers are not shut-down. I think I cannot change the process there and in
laboratories it can always happen during electronic-tests, that fuses are
triggered. That can happen anywhere since not everybody uses power-backup. For
my opinion, invalid files like here should be deleted - maybe with a warning in
the log - especially if we don't loose data like here for temporary
process-infos.
{code:title=system.log v2.1.7}
ERROR [SSTableBatchOpen:1] 2015-06-24 14:12:02,033 CassandraDaemon.java:223 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with
0 chunks encountered: java.io.DataInputStream@1a32dcf
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:205)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:127)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:721)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:676)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:482)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:381)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:519)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1a32dcf
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:183)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
... 15 common frames omitted
ERROR [SSTableBatchOpen:1] 2015-06-24 14:12:02,123 CassandraDaemon.java:223 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with
0 chunks encountered: java.io.DataInputStream@11ca50a
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:205)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:127)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
at
[jira] [Commented] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611731#comment-14611731
]
Andreas Schnitzerling commented on CASSANDRA-9694:
--
I made a test: I stopped C*, renamed the system_auth folder and started C*
again. Result: I can still login as a created user and I got an exception.
{code:title=system.log}
WARN [GossipTasks:1] 2015-07-02 11:46:03,060 FailureDetector.java:245 - Not
marking nodes down due to local pause of 172056754011 50
ERROR [Thrift:1] 2015-07-02 11:48:54,008 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at org.apache.cassandra.auth.RolesCache.getRoles(RolesCache.java:70)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.cassandra.auth.Roles.hasSuperuserStatus(Roles.java:51)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.isSuper(AuthenticatedUser.java:71)
[jira] [Updated] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9694:
-
Attachment: system_exception.log
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Attachments: system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611898#comment-14611898
]
Andreas Schnitzerling commented on CASSANDRA-9694:
--
I wiped system_auth after the same errors occurred (before and after wiping, no
effect of wiping).
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc2
Attachments: 9694.txt, system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9695) repair problem
[
https://issues.apache.org/jira/browse/CASSANDRA-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611885#comment-14611885
]
Andreas Schnitzerling commented on CASSANDRA-9695:
--
Thanks for the clarification! Since we regular have limitations in function in
mixed-version-clusters there should be always an info about such limitations in
the NEWS.TXT for example. See as well CASSANDRA-9694.
repair problem
--
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth',
{
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Commented] (CASSANDRA-9694) system_auth not upgraded
[
https://issues.apache.org/jira/browse/CASSANDRA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611874#comment-14611874
]
Andreas Schnitzerling commented on CASSANDRA-9694:
--
Thanks for the clarification! Since behavior is known and documented (later),
it should be enough to write an informative WARN one-liner into the log and not
blow up with stack-traces...?
system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Sam Tunnicliffe
Fix For: 2.2.0 rc2
Attachments: 9694.txt, system_exception.log
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were
not upgraded automatically. I upgraded them (I needed 2 times per table
because of CASSANDRA-9566). After upgrading the system_auth tables I could
login via cql using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9687) Wrong partitioner after upgrading sstables (secondary indexes are not handled correctly after CASSANDRA-6962)
[
https://issues.apache.org/jira/browse/CASSANDRA-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9687:
-
Attachment: la-1-big.zip
C* is always generating la-1-big.* which cause C* stopping on the next start
although I delete them before start. No old matching jb-1 files in the
folder. The jb-1642 files - including la-1642-big-Index.db in the same folder
- are accepted.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-07-02 15:39:08,888 SSTableReader.java:432 -
Cannot open D:\Programme\Cassandra\data\data\nieste\niesteinverters\la-1-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system
partitioner org.apache.cassandra.dht.Murmur3Partitioner. Note that the default
partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you will need
to edit that to match your old partitioner if upgrading.
{code}
Wrong partitioner after upgrading sstables (secondary indexes are not handled
correctly after CASSANDRA-6962)
-
Key: CASSANDRA-9687
URL: https://issues.apache.org/jira/browse/CASSANDRA-9687
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Yuki Morishita
Priority: Critical
Fix For: 2.2.0 rc2
Attachments: la-1-big.zip, la-540-big-CompressionInfo.db,
la-540-big-Data.db, la-540-big-Digest.adler32, la-540-big-Filter.db,
la-540-big-Index.db, la-540-big-Statistics.db, la-540-big-Summary.db,
la-540-big-TOC.txt, nieste-niesteinverters-jb-540-CompressionInfo.db,
nieste-niesteinverters-jb-540-Data.db,
nieste-niesteinverters-jb-540-Filter.db,
nieste-niesteinverters-jb-540-Index.db,
nieste-niesteinverters-jb-540-Statistics.db,
nieste-niesteinverters-jb-540-Summary.db,
nieste-niesteinverters-jb-540-TOC.txt, system.log, system.zip
After upgrading one of 15 nodes from 2.1.7 to 2.2.0.rc1, C* upgrades
automatic sstables. After restart of C*, some of these new generated sstables
are not accepted anymore and C* crashes. If I delete the affected sstables,
C* starts again.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-30 13:08:54,861 SSTableReader.java:432 -
Cannot open
D:\Programme\Cassandra\data\data\nieste\niesteinverters\la-540-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system
partitioner org.apache.cassandra.dht.Murmur3Partitioner. Note that the
default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
{code}
{code:title=schema}
CREATE TABLE niesteinverters (
id bigint,
comment maptimestamp, text,
creation_time timestamp,
fk_ncom bigint,
last_event timestamp,
last_filesize int,
last_onl_data timestamp,
last_time timestamp,
ncom_hist maptimestamp, bigint,
version int,
PRIMARY KEY ((id))
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='Table for niesteinverters
(niesteplants-niestecoms-niesteinverters)' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
CREATE INDEX niesteinvertersniestecomsIndex ON niesteinverters (fk_ncom);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9686:
-
Attachment: cassandra.bat
cassandra.yaml
sstable_activity.zip
compactions_in_progress.zip
I made a copy of the C* 2.1.7 instance before upgrading. So I attached the
files before upgrade, which cause the leak errors after upgrade.
The upgrade-steps:
- I copied the C* instance.
- I replaced the folders except the data, commitlog and saved_caches folders.
- I put data, commitlog and saved_caches folders into a new parent data-folder.
- I made changes in cassandra.bat. Mainly limited RAM to 1 GB and jmxremote.
- I changed necessary preferences in cassandra.yaml via diff.
- I reinstalled the C*-service.
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.2.x
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9696) nodetool stopdaemon exception
Andreas Schnitzerling created CASSANDRA-9696:
Summary: nodetool stopdaemon exception
Key: CASSANDRA-9696
URL: https://issues.apache.org/jira/browse/CASSANDRA-9696
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: cassandra.bat, cassandra.yaml
nodetool stopdaemon produces an exception. I use the default-locations
(uncommented in config) for that dirs like explained in cassandra.yaml.
{code:title=win-console}
%CASSANDRA_HOME%\binnodetool stopdaemon
Starting NodeTool
error: commitlog_directory is missing and -Dcassandra.storagedir is not set
-- StackTrace --
org.apache.cassandra.exceptions.ConfigurationException: commitlog_directory is
missing and -Dcassandra.storagedir is not
set
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:494)
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:111)
at
org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:54)
at
org.apache.cassandra.tools.nodetool.StopDaemon.execute(StopDaemon.java:37)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:239)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:153)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9696) nodetool stopdaemon exception
[
https://issues.apache.org/jira/browse/CASSANDRA-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9696:
-
Description:
nodetool stopdaemon produces an exception. I use the default-locations
(uncommented in config) for that dirs like explained in cassandra.yaml. Anyway
- C* is stopping.
{code:title=win-console}
%CASSANDRA_HOME%\binnodetool stopdaemon
Starting NodeTool
error: commitlog_directory is missing and -Dcassandra.storagedir is not set
-- StackTrace --
org.apache.cassandra.exceptions.ConfigurationException: commitlog_directory is
missing and -Dcassandra.storagedir is not
set
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:494)
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:111)
at
org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:54)
at
org.apache.cassandra.tools.nodetool.StopDaemon.execute(StopDaemon.java:37)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:239)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:153)
{code}
was:
nodetool stopdaemon produces an exception. I use the default-locations
(uncommented in config) for that dirs like explained in cassandra.yaml.
{code:title=win-console}
%CASSANDRA_HOME%\binnodetool stopdaemon
Starting NodeTool
error: commitlog_directory is missing and -Dcassandra.storagedir is not set
-- StackTrace --
org.apache.cassandra.exceptions.ConfigurationException: commitlog_directory is
missing and -Dcassandra.storagedir is not
set
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:494)
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:111)
at
org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:54)
at
org.apache.cassandra.tools.nodetool.StopDaemon.execute(StopDaemon.java:37)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:239)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:153)
{code}
nodetool stopdaemon exception
-
Key: CASSANDRA-9696
URL: https://issues.apache.org/jira/browse/CASSANDRA-9696
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: cassandra.bat, cassandra.yaml
nodetool stopdaemon produces an exception. I use the default-locations
(uncommented in config) for that dirs like explained in cassandra.yaml.
Anyway - C* is stopping.
{code:title=win-console}
%CASSANDRA_HOME%\binnodetool stopdaemon
Starting NodeTool
error: commitlog_directory is missing and -Dcassandra.storagedir is not set
-- StackTrace --
org.apache.cassandra.exceptions.ConfigurationException: commitlog_directory
is missing and -Dcassandra.storagedir is not
set
at
org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:494)
at
org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:111)
at
org.apache.cassandra.utils.JVMStabilityInspector.inspectThrowable(JVMStabilityInspector.java:54)
at
org.apache.cassandra.tools.nodetool.StopDaemon.execute(StopDaemon.java:37)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:239)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:153)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9695) repair problem
Andreas Schnitzerling created CASSANDRA-9695:
Summary: repair problem
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth', {
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Created] (CASSANDRA-9694) system_auth not upgraded
Andreas Schnitzerling created CASSANDRA-9694:
Summary: system_auth not upgraded
Key: CASSANDRA-9694
URL: https://issues.apache.org/jira/browse/CASSANDRA-9694
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
After upgrading Authorization-Exceptions occur. I checked the system_auth
keyspace and have seen, that tables users, credentials and permissions were not
upgraded automatically. I upgraded them (I needed 2 times per table because of
CASSANDRA-9566). After upgrading the system_auth tables I could login via cql
using different users.
{code:title=system.log}
WARN [Thrift:14] 2015-07-01 11:38:57,748 CassandraAuthorizer.java:91 -
CassandraAuthorizer failed to authorize #User updateprog for keyspace
logdata
ERROR [Thrift:14] 2015-07-01 11:41:26,210 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.statements.SelectStatement.checkAccess(SelectStatement.java:143)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:222)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1891)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4588)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4572)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9695) repair problem
[
https://issues.apache.org/jira/browse/CASSANDRA-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9695:
-
Attachment: system.zip
Whole system.log of 2.2.0-rc1.
repair problem
--
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth',
{
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Updated] (CASSANDRA-9687) Wrong partitioner after upgrading sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9687:
-
Attachment: system.zip
Whole system.log of 2.2.0-rc1
Wrong partitioner after upgrading sstables
--
Key: CASSANDRA-9687
URL: https://issues.apache.org/jira/browse/CASSANDRA-9687
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Branimir Lambov
Priority: Critical
Fix For: 2.2.0 rc2
Attachments: la-540-big-CompressionInfo.db, la-540-big-Data.db,
la-540-big-Digest.adler32, la-540-big-Filter.db, la-540-big-Index.db,
la-540-big-Statistics.db, la-540-big-Summary.db, la-540-big-TOC.txt,
nieste-niesteinverters-jb-540-CompressionInfo.db,
nieste-niesteinverters-jb-540-Data.db,
nieste-niesteinverters-jb-540-Filter.db,
nieste-niesteinverters-jb-540-Index.db,
nieste-niesteinverters-jb-540-Statistics.db,
nieste-niesteinverters-jb-540-Summary.db,
nieste-niesteinverters-jb-540-TOC.txt, system.log, system.zip
After upgrading one of 15 nodes from 2.1.7 to 2.2.0.rc1, C* upgrades
automatic sstables. After restart of C*, some of these new generated sstables
are not accepted anymore and C* crashes. If I delete the affected sstables,
C* starts again.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-30 13:08:54,861 SSTableReader.java:432 -
Cannot open
D:\Programme\Cassandra\data\data\nieste\niesteinverters\la-540-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system
partitioner org.apache.cassandra.dht.Murmur3Partitioner. Note that the
default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
{code}
{code:title=schema}
CREATE TABLE niesteinverters (
id bigint,
comment maptimestamp, text,
creation_time timestamp,
fk_ncom bigint,
last_event timestamp,
last_filesize int,
last_onl_data timestamp,
last_time timestamp,
ncom_hist maptimestamp, bigint,
version int,
PRIMARY KEY ((id))
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='Table for niesteinverters
(niesteplants-niestecoms-niesteinverters)' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
CREATE INDEX niesteinvertersniestecomsIndex ON niesteinverters (fk_ncom);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9695) repair problem
[
https://issues.apache.org/jira/browse/CASSANDRA-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610435#comment-14610435
]
Andreas Schnitzerling commented on CASSANDRA-9695:
--
Before 2.2.0 no such problems under same conditions. I didn'd see overload in
task-manager as well. Is repair of 2.2.0 validated in an 2.1.x cluster?
repair problem
--
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth',
{
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Updated] (CASSANDRA-9695) repair problem
[
https://issues.apache.org/jira/browse/CASSANDRA-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9695:
-
Attachment: (was: system.zip)
repair problem
--
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth',
{
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Issue Comment Deleted] (CASSANDRA-9695) repair problem
[
https://issues.apache.org/jira/browse/CASSANDRA-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9695:
-
Comment: was deleted
(was: Whole system.log of 2.2.0-rc1.)
repair problem
--
Key: CASSANDRA-9695
URL: https://issues.apache.org/jira/browse/CASSANDRA-9695
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system-repair.log
Exception during repair of system_auth. Other keyspaces affected as well.
Cluster: 14xv2.1.7 + 1xv2.2.0-rc1.
{code:title=system.log}
ERROR [Thread-4] 2015-07-01 14:18:32,953 SystemDistributedKeyspace.java:203 -
Error executing query INSERT INTO system_distributed.parent_repair_history
(parent_id, keyspace_name, columnfamily_names, requested_ranges, started_at)
VALUES (491ad8e0-1feb-11e5-8830-9b845260997e,'system_auth',
{
'role_permissions','resource_role_permissons_index','roles','users','credentials','permissions','role_members'
}, {
[jira] [Created] (CASSANDRA-9685) UncheckedExecutionException in CustomTThreadPoolServer
Andreas Schnitzerling created CASSANDRA-9685:
Summary: UncheckedExecutionException in CustomTThreadPoolServer
Key: CASSANDRA-9685
URL: https://issues.apache.org/jira/browse/CASSANDRA-9685
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system.log
I have a cluster with 15 nodes running C* 2.1.7 and upgraded one to 2.2.0-rc1.
{code:title=system.log}
ERROR [Thrift:11] 2015-06-30 11:13:52,295 CustomTThreadPoolServer.java:223 -
Error occurred during processing of message.
com.google.common.util.concurrent.UncheckedExecutionException:
com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out -
received only 0 responses.
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
~[guava-16.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
~[guava-16.0.jar:na]
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
~[guava-16.0.jar:na]
at
org.apache.cassandra.auth.PermissionsCache.getPermissions(PermissionsCache.java:72)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.auth.AuthenticatedUser.getPermissions(AuthenticatedUser.java:104)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.authorize(ClientState.java:362)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.checkPermissionOnResourceChain(ClientState.java:295)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.ensureHasPermission(ClientState.java:272)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasAccess(ClientState.java:259)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.service.ClientState.hasColumnFamilyAccess(ClientState.java:243)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.createMutationList(CassandraServer.java:815)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.CassandraServer.batch_mutate(CassandraServer.java:950)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3996)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra.java:3980)
~[apache-cassandra-thrift-2.2.0-rc1.jar:2.2.0-rc1]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.2.jar:0.9.2]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:204)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
Andreas Schnitzerling created CASSANDRA-9686:
Summary: FSReadError and LEAK DETECTED after upgrading
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with
0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@3e547f)
to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9687) Wrong partitioner after upgrading sstables
Andreas Schnitzerling created CASSANDRA-9687:
Summary: Wrong partitioner after upgrading sstables
Key: CASSANDRA-9687
URL: https://issues.apache.org/jira/browse/CASSANDRA-9687
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: la-540-big-CompressionInfo.db, la-540-big-Data.db,
la-540-big-Digest.adler32, la-540-big-Filter.db, la-540-big-Index.db,
la-540-big-Statistics.db, la-540-big-Summary.db, la-540-big-TOC.txt, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0.rc1, C* upgrades automatic
sstables. After restart of C*, some of these new generated sstables are not
accepted anymore and C* crashes. If I delete the affected sstables, C* starts
again.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-30 13:08:54,861 SSTableReader.java:432 -
Cannot open D:\Programme\Cassandra\data\data\nieste\niesteinverters\la-540-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system
partitioner org.apache.cassandra.dht.Murmur3Partitioner. Note that the default
partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you will need
to edit that to match your old partitioner if upgrading.
{code}
{code:title=schema}
CREATE TABLE niesteinverters (
id bigint,
comment maptimestamp, text,
creation_time timestamp,
fk_ncom bigint,
last_event timestamp,
last_filesize int,
last_onl_data timestamp,
last_time timestamp,
ncom_hist maptimestamp, bigint,
version int,
PRIMARY KEY ((id))
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='Table for niesteinverters
(niesteplants-niestecoms-niesteinverters)' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
CREATE INDEX niesteinvertersniestecomsIndex ON niesteinverters (fk_ncom);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9687) Wrong partitioner after upgrading sstables
[
https://issues.apache.org/jira/browse/CASSANDRA-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-9687:
-
Attachment: nieste-niesteinverters-jb-540-TOC.txt
nieste-niesteinverters-jb-540-Summary.db
nieste-niesteinverters-jb-540-Statistics.db
nieste-niesteinverters-jb-540-Index.db
nieste-niesteinverters-jb-540-Filter.db
nieste-niesteinverters-jb-540-Data.db
nieste-niesteinverters-jb-540-CompressionInfo.db
That should be the matching files before upgrading.
Wrong partitioner after upgrading sstables
--
Key: CASSANDRA-9687
URL: https://issues.apache.org/jira/browse/CASSANDRA-9687
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: la-540-big-CompressionInfo.db, la-540-big-Data.db,
la-540-big-Digest.adler32, la-540-big-Filter.db, la-540-big-Index.db,
la-540-big-Statistics.db, la-540-big-Summary.db, la-540-big-TOC.txt,
nieste-niesteinverters-jb-540-CompressionInfo.db,
nieste-niesteinverters-jb-540-Data.db,
nieste-niesteinverters-jb-540-Filter.db,
nieste-niesteinverters-jb-540-Index.db,
nieste-niesteinverters-jb-540-Statistics.db,
nieste-niesteinverters-jb-540-Summary.db,
nieste-niesteinverters-jb-540-TOC.txt, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0.rc1, C* upgrades
automatic sstables. After restart of C*, some of these new generated sstables
are not accepted anymore and C* crashes. If I delete the affected sstables,
C* starts again.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-30 13:08:54,861 SSTableReader.java:432 -
Cannot open
D:\Programme\Cassandra\data\data\nieste\niesteinverters\la-540-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system
partitioner org.apache.cassandra.dht.Murmur3Partitioner. Note that the
default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
{code}
{code:title=schema}
CREATE TABLE niesteinverters (
id bigint,
comment maptimestamp, text,
creation_time timestamp,
fk_ncom bigint,
last_event timestamp,
last_filesize int,
last_onl_data timestamp,
last_time timestamp,
ncom_hist maptimestamp, bigint,
version int,
PRIMARY KEY ((id))
) WITH
bloom_filter_fp_chance=0.01 AND
caching='{keys:ALL, rows_per_partition:NONE}' AND
comment='Table for niesteinverters
(niesteplants-niestecoms-niesteinverters)' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'LeveledCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
CREATE INDEX niesteinvertersniestecomsIndex ON niesteinverters (fk_ncom);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593219#comment-14593219
]
Andreas Schnitzerling commented on CASSANDRA-8535:
--
It is not helping always. In some circumstances it's crashing again.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593421#comment-14593421
]
Andreas Schnitzerling commented on CASSANDRA-8535:
--
I try to fix it with 2 finalizers. One in SequentialWriter and one in
RandomAccessReader. Latter helped in C* 2.0.x. One node I run w/ both
finalizers and file-trace + referenceCount and one I use only the
SequentialWriter-finalizer.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589635#comment-14589635
]
Andreas Schnitzerling edited comment on CASSANDRA-8535 at 6/19/15 8:59 AM:
---
On Windows 7, files and directories cannot be renamed, if they're still open,
so we need to close them before:
{code:title=src/java/org/apache/cassandra/io/sstable/SSTableWriter.java}
private PairDescriptor, StatsMetadata close(FinishType type, long repairedAt)
Descriptor descriptor = this.descriptor;
if (type.isFinal)
{
dataFile.writeFullChecksum(descriptor);
writeMetadata(descriptor, metadataComponents);
// save the table of components
SSTable.appendTOC(descriptor, components);
+++ dataFile.close();
descriptor = rename(descriptor, components);
}
{code}
I'm using Win7-32bit and got the same stack trace. So 32bit or 64bit is not the
issue.
was (Author: andie78):
On Windows 7, files and directories cannot be renamed, if they're still open,
so we need to close them before:
{code:title=src/java/org/apache/cassandra/io/sstable/SSTableWriter.java}
private PairDescriptor, StatsMetadata close(FinishType type, long repairedAt)
Descriptor descriptor = this.descriptor;
if (type.isFinal)
{
dataFile.writeFullChecksum(descriptor);
writeMetadata(descriptor, metadataComponents);
// save the table of components
SSTable.appendTOC(descriptor, components);
+++ dataFile.close();
descriptor = rename(descriptor, components);
}
{code}
Can somebody check/test and maybe commit it? Thx.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593561#comment-14593561
]
Andreas Schnitzerling commented on CASSANDRA-8535:
--
I used my (necessary) finalizer-patch from 2.0 in 2.1.7-tentative (today
cloned) as well. 2.1.6-release crashes but 2.1.7 of today seems to work now
with the finalizer(s). At Monday I will check the file-traces and look for
exceptions.
Actually it looks like, 2.1.6/7 needs only a finalizer in SequentialWriter to
work w/o crash in windows. Monday I know more after some time w/ my patch.
Has 2.2 improved io-resource-management or why should it be better for windows
now? For my belief, cassandra has leaks in filehandles and windows seems to be
strict compared to linux. Why else finalizers help?
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589635#comment-14589635
]
Andreas Schnitzerling edited comment on CASSANDRA-8535 at 6/17/15 11:12 AM:
On Windows 7, files and directories cannot be renamed, if they're still open,
so we need to close them before:
{code:title=src/java/org/apache/cassandra/io/sstable/SSTableWriter.java}
private PairDescriptor, StatsMetadata close(FinishType type, long repairedAt)
Descriptor descriptor = this.descriptor;
if (type.isFinal)
{
dataFile.writeFullChecksum(descriptor);
writeMetadata(descriptor, metadataComponents);
// save the table of components
SSTable.appendTOC(descriptor, components);
+++ dataFile.close();
descriptor = rename(descriptor, components);
}
{code}
Can somebody check/test and maybe commit it? Thx.
was (Author: andie78):
On Windows 7, files and directories cannot be renamed, if they're still open,
so we need to close them before:
{code:title=\src\java\org\apache\cassandra\io\sstable\SSTableWriter.java}
private PairDescriptor, StatsMetadata close(FinishType type, long repairedAt)
Descriptor descriptor = this.descriptor;
if (type.isFinal)
{
dataFile.writeFullChecksum(descriptor);
writeMetadata(descriptor, metadataComponents);
// save the table of components
SSTable.appendTOC(descriptor, components);
+++ dataFile.close();
descriptor = rename(descriptor, components);
}
{code}
Can somebody check/test and maybe commit it? Thx.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589635#comment-14589635
]
Andreas Schnitzerling commented on CASSANDRA-8535:
--
On Windows 7, files and directories cannot be renamed, if they're still open,
so we need to close them before:
{code:title=\src\java\org\apache\cassandra\io\sstable\SSTableWriter.java}
private PairDescriptor, StatsMetadata close(FinishType type, long repairedAt)
Descriptor descriptor = this.descriptor;
if (type.isFinal)
{
dataFile.writeFullChecksum(descriptor);
writeMetadata(descriptor, metadataComponents);
// save the table of components
SSTable.appendTOC(descriptor, components);
+++ dataFile.close();
descriptor = rename(descriptor, components);
}
{code}
Can somebody check/test and maybe commit it? Thx.
java.lang.RuntimeException: Failed to rename XXX to YYY
---
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
Assignee: Joshua McKenzie
Labels: Windows
Fix For: 2.1.5
Attachments: 8535_v1.txt, 8535_v2.txt, 8535_v3.txt
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
-
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another
process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
--
This message was sent by Atlassian
[jira] [Commented] (CASSANDRA-5383) Windows 7 deleting/renaming files problem
[
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589504#comment-14589504
]
Andreas Schnitzerling commented on CASSANDRA-5383:
--
I put it manually into the source:
/src/org/apache/cassandra/io/util/RandomAccessReader.java
Put (overwrite) the finalizer there.
To avoid unnecessary logs after patching u can un-comment the warning
(logger.error) in /sstable/SSTableDeletingTask.java
To build u need maven (I use 3.0.5) and ant (1.9.2) and an internet-connection.
I have to use ant on a dedicated server in front of the proxy since I couldn't
get proxy-settings working in ant.
After the modifications u can type ant jar if u just need the patched jar(s).
ant release builds u the compressed bin-package.
Windows 7 deleting/renaming files problem
-
Key: CASSANDRA-5383
URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
Project: Cassandra
Issue Type: Bug
Components: Tests
Affects Versions: 2.0 beta 1
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
Labels: qa-resolved
Fix For: 2.0.2
Attachments:
0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch,
0001-CASSANDRA-5383-v2.patch,
0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch,
5383-v3.patch, 5383_patch_v2_system.log, cant_move_file_patch.log,
test_log.5383.patch_v2.log.txt, v2+cant_move_file_patch.log
Two unit tests are failing on Windows 7 due to errors in renaming/deleting
files:
org.apache.cassandra.db.ColumnFamilyStoreTest:
{code}
[junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
[junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
13.904 sec
[junit]
[junit] - Standard Error -
[junit] ERROR 13:06:46,058 Unable to delete
build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
(it will be removed on server restart; we'll also retry after GC)
[junit] ERROR 13:06:48,508 Fatal exception in thread
Thread[NonPeriodicTasks:1,5,main]
[junit] java.lang.RuntimeException: Tried to hard link to file that does
not exist
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
[junit] at
org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
[junit] at
org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
[junit] at
org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
[junit] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
[junit] at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
[junit] at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit] at java.lang.Thread.run(Thread.java:662)
[junit] - ---
[junit] Testcase:
testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
Caused an ERROR
[junit] Failed to rename
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
to
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
[junit] java.lang.RuntimeException: Failed to rename
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
to
build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
[junit] at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
[junit] at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
[junit] at
org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
[junit] at
org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
[junit] at
org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
[junit]
[junit]
[junit] Testcase:
testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
Caused an ERROR
[junit] java.io.IOException: Failed to delete
[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580341#comment-14580341 ] Andreas Schnitzerling commented on CASSANDRA-8072: -- Hello, I made following steps: I decom a 2.0.15 node with 128 vnodes and tried to bootstrap 2.1.6-R on the same node w/ 256 vnodes in write survey mode to test. 2.1.6 doesn't bootstrap because of the unable gossib exception but the old 2.0.15 does it w/o problems. Even if i use cassandra.yaml from 2.0.15 (deletetd properties invalid for 2.1.6) it doesn't start. I have 14 nodes 2.0.15 running on Windows 7. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams Fix For: 2.1.x, 2.0.x Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2 When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps.  I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[
https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580341#comment-14580341
]
Andreas Schnitzerling edited comment on CASSANDRA-8072 at 6/10/15 10:15 AM:
Hello, I made following steps: I decom a 2.0.15 node with 128 vnodes and tried
to bootstrap 2.1.6-R on the same node w/ 256 vnodes in write survey mode to
test. 2.1.6 doesn't bootstrap because of the unable gossib exception but the
old 2.0.15 does it w/o problems. Even if i use cassandra.yaml from 2.0.15
(deleted properties invalid for 2.1.6) it doesn't start. I have 14 nodes 2.0.15
running on Windows 7.
{panel:title=system.log}
ERROR [main] 2015-06-10 12:03:22,200 CassandraDaemon.java:553 - Exception
encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307)
~[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:530)
~[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:774)
~[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:711)
~[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:602)
~[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:394)
[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:536)
[apache-cassandra-2.1.6.jar:2.1.6]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:625)
[apache-cassandra-2.1.6.jar:2.1.6]
WARN [StorageServiceShutdownHook] 2015-06-10 12:03:22,200 Gossiper.java:1418 -
No local state or state is in silent shutdown, not announcing shutdown
INFO [StorageServiceShutdownHook] 2015-06-10 12:03:22,200
MessagingService.java:708 - Waiting for messaging service to quiesce
INFO [ACCEPT-PC5771/10.2.0.61] 2015-06-10 12:03:22,200
MessagingService.java:958 - MessagingService has terminated the accept() thread
{panel}
was (Author: andie78):
Hello, I made following steps: I decom a 2.0.15 node with 128 vnodes and tried
to bootstrap 2.1.6-R on the same node w/ 256 vnodes in write survey mode to
test. 2.1.6 doesn't bootstrap because of the unable gossib exception but the
old 2.0.15 does it w/o problems. Even if i use cassandra.yaml from 2.0.15
(deletetd properties invalid for 2.1.6) it doesn't start. I have 14 nodes
2.0.15 running on Windows 7.
Exception during startup: Unable to gossip with any seeds
-
Key: CASSANDRA-8072
URL: https://issues.apache.org/jira/browse/CASSANDRA-8072
Project: Cassandra
Issue Type: Bug
Reporter: Ryan Springer
Assignee: Brandon Williams
Fix For: 2.1.x, 2.0.x
Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2,
cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2,
cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2,
casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2
When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster
in either ec2 or locally, an error occurs sometimes with one of the nodes
refusing to start C*. The error in the /var/log/cassandra/system.log is:
ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513)
Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200)
at
org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444)
at
org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:609)
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:502)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java
(line 1279) Announcing shutdown
INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326
MessagingService.java (line 701) Waiting for messaging service to quiesce
INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327
MessagingService.java (line 941) MessagingService has terminated the accept()
[jira] [Commented] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580409#comment-14580409 ] Andreas Schnitzerling commented on CASSANDRA-8072: -- I tried severel times to switch between 2.0.15 and 2.1.6 (start bootstr at 2.0.15, stop, copy data to 2.1.6) but 2.1.6 don't start probably. One time, 2.1.6 have seen only other nodes, not himself. Now I deleted all again on that node, removed the node with nodetool and now the bootstrap w/ 2.0.15 works well in write survey mode. I'll wait until bootstrap finished and then I try to upgrade the same node to 2.1.6 w/ write survey as well. Seems like 2.1.6 is not backward compatible bootstrapping to a 2.0.x cluster. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams Fix For: 2.1.x, 2.0.x Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2 When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps.  I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580409#comment-14580409 ] Andreas Schnitzerling edited comment on CASSANDRA-8072 at 6/10/15 11:16 AM: I tried severel times to switch between 2.0.15 and 2.1.6 (start bootstr at 2.0.15, stop, copy data to 2.1.6) but 2.1.6 doesn't start probably. One time, 2.1.6 had seen only other nodes, not himself. Now I deleted all data again on that node, removed the node with nodetool and now the bootstrap w/ 2.0.15 works well in write survey mode. I'll wait until bootstrap finished and then I try to upgrade the same node to 2.1.6 w/ write survey as well. Seems like 2.1.6 is not backward compatible bootstrapping to a 2.0.x cluster. was (Author: andie78): I tried severel times to switch between 2.0.15 and 2.1.6 (start bootstr at 2.0.15, stop, copy data to 2.1.6) but 2.1.6 don't start probably. One time, 2.1.6 have seen only other nodes, not himself. Now I deleted all again on that node, removed the node with nodetool and now the bootstrap w/ 2.0.15 works well in write survey mode. I'll wait until bootstrap finished and then I try to upgrade the same node to 2.1.6 w/ write survey as well. Seems like 2.1.6 is not backward compatible bootstrapping to a 2.0.x cluster. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams Fix For: 2.1.x, 2.0.x Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2 When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps.  I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8072) Exception during startup: Unable to gossip with any seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580409#comment-14580409 ] Andreas Schnitzerling edited comment on CASSANDRA-8072 at 6/10/15 11:17 AM: I tried severel times to switch between 2.0.15 and 2.1.6 (start bootstr at 2.0.15, stop, copy data to 2.1.6) but 2.1.6 doesn't start probably. One time, 2.1.6 had seen only other nodes, not himself. Now I deleted all data again on that node, removed the node with nodetool and now the bootstrap w/ 2.0.15 works well in write survey mode. I'll wait until bootstrap finished and then I try to upgrade the same node to 2.1.6 w/ write survey as well. Seems like 2.1.6 is not backward compatible bootstrapping to a 2.0.x cluster. Can u confirm that behavior? was (Author: andie78): I tried severel times to switch between 2.0.15 and 2.1.6 (start bootstr at 2.0.15, stop, copy data to 2.1.6) but 2.1.6 doesn't start probably. One time, 2.1.6 had seen only other nodes, not himself. Now I deleted all data again on that node, removed the node with nodetool and now the bootstrap w/ 2.0.15 works well in write survey mode. I'll wait until bootstrap finished and then I try to upgrade the same node to 2.1.6 w/ write survey as well. Seems like 2.1.6 is not backward compatible bootstrapping to a 2.0.x cluster. Exception during startup: Unable to gossip with any seeds - Key: CASSANDRA-8072 URL: https://issues.apache.org/jira/browse/CASSANDRA-8072 Project: Cassandra Issue Type: Bug Reporter: Ryan Springer Assignee: Brandon Williams Fix For: 2.1.x, 2.0.x Attachments: cas-dev-dt-01-uw1-cassandra-seed01_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra-seed02_logs.tar.bz2, cas-dev-dt-01-uw1-cassandra02_logs.tar.bz2, casandra-system-log-with-assert-patch.log, trace_logs.tar.bz2 When Opscenter 4.1.4 or 5.0.1 tries to provision a 2-node DSC 2.0.10 cluster in either ec2 or locally, an error occurs sometimes with one of the nodes refusing to start C*. The error in the /var/log/cassandra/system.log is: ERROR [main] 2014-10-06 15:54:52,292 CassandraDaemon.java (line 513) Exception encountered during startup java.lang.RuntimeException: Unable to gossip with any seeds at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1200) at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:444) at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:655) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:609) at org.apache.cassandra.service.StorageService.initServer(StorageService.java:502) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585) INFO [StorageServiceShutdownHook] 2014-10-06 15:54:52,326 Gossiper.java (line 1279) Announcing shutdown INFO [StorageServiceShutdownHook] 2014-10-06 15:54:54,326 MessagingService.java (line 701) Waiting for messaging service to quiesce INFO [ACCEPT-localhost/127.0.0.1] 2014-10-06 15:54:54,327 MessagingService.java (line 941) MessagingService has terminated the accept() thread This errors does not always occur when provisioning a 2-node cluster, but probably around half of the time on only one of the nodes. I haven't been able to reproduce this error with DSC 2.0.9, and there have been no code or definition file changes in Opscenter. I can reproduce locally with the above steps.  I'm happy to test any proposed fixes since I'm the only person able to reproduce reliably so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227764#comment-14227764
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
Confirmed. I testet 2.1.2 empty w/ 256MB (! for test) JVM. Result: No Error.
After that I put 1GB new generated sstables from 2.0.11 (incl. system), started
upgradesstables, restarted and still no error. So 32-bit with low mem is still
supported in 2.1... :-)
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Fix For: 2.1.3
Attachments: cassandra.bat, cassandra.yaml,
logdata-onlinedata-ka-196504-CompressionInfo.zip, printChunkOffsetErrors.txt,
system-compactions_in_progress-ka-47594-CompressionInfo.zip,
system-sstable_activity-jb-25-Filter.zip, system.log, system_AssertionTest.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8333) Streaming Error during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8333:
-
Description:
During repair, connections are closing and throwing exceptions. CPU is running
on 100%, when error occurs. My test-configuration is one node w/ 2.1.2 and 11
nodes w/ 2.0.11. If I make repair either on 2.1 or 2.0 I get such an error. But
if I have 2.0 everywhere istalled, no error. Seems to be incompatibility
between 2.0 and 2.1.
{panel:title=system.log}
ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 -
[Stream #0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
durch den Hostcomputer abgebrochen
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
at
org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:326)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
ERROR [AntiEntropySessions:1] 2014-11-18 12:28:34,948 RepairSession.java:303 -
[repair #e10d0240-6f15-11e4-bc5c-5fe413b6852c] session completed with the
following error
org.apache.cassandra.exceptions.RepairException: [repair
#e10d0240-6f15-11e4-bc5c-5fe413b6852c on logdata/onlinedata,
(-143721749331492309,-139544903266258032]] Sync failed between /10.9.9.241 and
/10.6.8.212
at
org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:223)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:389)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
Since in windows only parallel repair is possible, is there a way to throttle
CPU-consumption? I reduced rpc_X_threads to 4 and concurrent_reads/writes to 4.
But no change. On other nodes is C* 2.0.10 and nothing in their system.log.
was:
During repair, connections are closing and throwing exceptions. CPU is running
on 100%, when error occurs.
{panel:title=system.log}
ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 -
[Stream #0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
durch den Hostcomputer abgebrochen
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
at
org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:326)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
ERROR [AntiEntropySessions:1]
[jira] [Updated] (CASSANDRA-8333) Streaming Error during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8333:
-
Description:
During repair, connections are closing and throwing exceptions. CPU is running
on 100%, when error occurs. My test-configuration is one node w/ 2.1.2 and 11
nodes w/ 2.0.11. If I make repair either on 2.1 or 2.0 I get such an error. But
if I have 2.0 everywhere istalled, no error. 2.0 nodes make endless repair in
that circumstance. Seems to be incompatibility between 2.0 and 2.1.
{panel:title=system.log}
ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 -
[Stream #0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
durch den Hostcomputer abgebrochen
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
at
org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:326)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
ERROR [AntiEntropySessions:1] 2014-11-18 12:28:34,948 RepairSession.java:303 -
[repair #e10d0240-6f15-11e4-bc5c-5fe413b6852c] session completed with the
following error
org.apache.cassandra.exceptions.RepairException: [repair
#e10d0240-6f15-11e4-bc5c-5fe413b6852c on logdata/onlinedata,
(-143721749331492309,-139544903266258032]] Sync failed between /10.9.9.241 and
/10.6.8.212
at
org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:223)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:389)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
Since in windows only parallel repair is possible, is there a way to throttle
CPU-consumption? I reduced rpc_X_threads to 4 and concurrent_reads/writes to 4.
But no change. On other nodes is C* 2.0.10 and nothing in their system.log.
was:
During repair, connections are closing and throwing exceptions. CPU is running
on 100%, when error occurs. My test-configuration is one node w/ 2.1.2 and 11
nodes w/ 2.0.11. If I make repair either on 2.1 or 2.0 I get such an error. But
if I have 2.0 everywhere istalled, no error. Seems to be incompatibility
between 2.0 and 2.1.
{panel:title=system.log}
ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 -
[Stream #0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
durch den Hostcomputer abgebrochen
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
at
org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225896#comment-14225896
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
Hello, we are not the same persons :-). It is happening at an existing dataset
from 2.0.10. Actually I'm downgrading my 3 of 12 nodes back to 2.0.1x because I
really need a stable system now. In my expirience 2.0 and 2.1 are not full
compatible (cql metadata like tables are not shown in mixed cluster and repair
is failure-prone as well between 2.0 and 2.1). Since yst we have 40 new devices
under test and today additionally 36. All their data I need to import to my
cluster. I will upgrade in a few months after some releases. As far I know,
there is a C* mode for testing w/o (completly) joining the cluster? How do I
configurate it?
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml,
printChunkOffsetErrors.txt, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225907#comment-14225907
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
I have another, offtopic question: If I make repair on every node w/ option
-pr, is it the same effect like w/o -pr on one node? What is neccessary to sync
replicas as well? (f.e. always w/o -pr or if -pr then repair on every
node...)(FYI: I need to use option -par since Im using windows). Thx.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml,
printChunkOffsetErrors.txt, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: logdata-onlinedata-ka-196504-CompressionInfo.zip
system-sstable_activity-jb-25-Filter.zip
system-compactions_in_progress-ka-47594-CompressionInfo.zip
system_AssertionTest.log
The patch found some files. I attached them including the log.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml,
logdata-onlinedata-ka-196504-CompressionInfo.zip, printChunkOffsetErrors.txt,
system-compactions_in_progress-ka-47594-CompressionInfo.zip,
system-sstable_activity-jb-25-Filter.zip, system.log, system_AssertionTest.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222995#comment-14222995
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
I managed to start 2.1.2 w/finalizer-patch and -Xms1229M -Xmx1229M^. Same
error. Windows Task-Manager shows only 308MB used after that error. So one node
w/ 1GB jvm set no error (my post before) and another same node w/1,2GB jvm set
throws that error.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217575#comment-14217575
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
On one node I can run with 1GB and no memory problems. I think I can find more
of that nodes. So it's well running on 32-bit. I think, it should be supported
and I hope, my other issues not relying to memory will not be rejected because
of 32-bit. Thanks.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8067) NullPointerException in KeyCacheSerializer
[
https://issues.apache.org/jira/browse/CASSANDRA-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217774#comment-14217774
]
Andreas Schnitzerling commented on CASSANDRA-8067:
--
Today I got that failure during nodetool upgradesstables after upgrading from
2.0.10 to 2.1.2 w/ finalizer-patch CASSANDRA-6283
{noformat}
ERROR [CompactionExecutor:65] 2014-11-19 09:56:02,453 CassandraDaemon.java:153
- Exception in thread Thread[CompactionExecutor:65,1,main]
java.lang.NullPointerException: null
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:274)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1088)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{noformat}
NullPointerException in KeyCacheSerializer
--
Key: CASSANDRA-8067
URL: https://issues.apache.org/jira/browse/CASSANDRA-8067
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Eric Leleu
Fix For: 2.1.1
Hi,
I have this stack trace in the logs of Cassandra server (v2.1)
{code}
ERROR [CompactionExecutor:14] 2014-10-06 23:32:02,098
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:14,1,main]
java.lang.NullPointerException: null
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:225)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1061)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
Source) ~[na:1.7.0]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0]
at java.lang.Thread.run(Unknown Source) [na:1.7.0]
{code}
It may not be critical because this error occured in the AutoSavingCache.
However the line 475 is about the CFMetaData so it may hide bigger issue...
{code}
474 CFMetaData cfm =
Schema.instance.getCFMetaData(key.desc.ksname, key.desc.cfname);
475 cfm.comparator.rowIndexEntrySerializer().serialize(entry,
out);
{code}
Regards,
Eric
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8333) Streaming Error during repair
Andreas Schnitzerling created CASSANDRA-8333:
Summary: Streaming Error during repair
Key: CASSANDRA-8333
URL: https://issues.apache.org/jira/browse/CASSANDRA-8333
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Attachments: system.log
During repair, connections are closing and throwing exceptions. CPU is running
on 100%, when error occurs.
{panel:title=system.log}
ERROR [STREAM-OUT-/10.6.8.212] 2014-11-18 12:28:34,948 StreamSession.java:472 -
[Stream #0866dc80-6f16-11e4-bc5c-5fe413b6852c] Streaming error occurred
java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert
durch den Hostcomputer abgebrochen
at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[na:1.7.0_55]
at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
~[na:1.7.0_55]
at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_55]
at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[na:1.7.0_55]
at
org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:346)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:326)
[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
ERROR [AntiEntropySessions:1] 2014-11-18 12:28:34,948 RepairSession.java:303 -
[repair #e10d0240-6f15-11e4-bc5c-5fe413b6852c] session completed with the
following error
org.apache.cassandra.exceptions.RepairException: [repair
#e10d0240-6f15-11e4-bc5c-5fe413b6852c on logdata/onlinedata,
(-143721749331492309,-139544903266258032]] Sync failed between /10.9.9.241 and
/10.6.8.212
at
org.apache.cassandra.repair.RepairSession.syncComplete(RepairSession.java:223)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:389)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:126)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[apache-cassandra-2.1.2-SNAPSHOT.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
Since in windows only parallel repair is possible, is there a way to throttle
CPU-consumption? I reduced rpc_X_threads to 4 and concurrent_reads/writes to 4.
But no change. On other nodes is C* 2.0.10 and nothing in their system.log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212032#comment-14212032
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
It's only occuring at the start of cassandra. Not anymore during operation.
That says me, cassandra 2.1 fits to 512MB. There seems to be a bug during
initialization compared to 2.0.X.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212101#comment-14212101
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
We cannot install more than 3GB of RAM on 32-bit-system (adressing
limitations). We need 32-bit version because of old (only available) drivers
for our laboratory equipment. With 768MB configured, C* doesn't start anymore.
One reason of my decission for C* was to use available ressources and not to
buy more computers.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14209528#comment-14209528
]
Andreas Schnitzerling commented on CASSANDRA-8192:
--
I upgraded to 2.1.2 w/finalizer-patch for CASSANDRA-6283 and tried now the
low-environment settings mentioned in CASSANDRA-8070. They didn't change the
situation. With 2.0.10 were not any exceptions at the same configuration. Is it
better to downgrade back again to avoid any exceptions? Here is the output of
nodetool info one minute after start and after 9 Memory-Exceptions:
{panel:title=system.log}
Starting NodeTool
ID : c45a5633-84a0-4196-b2d2-8b3d7060654a
Gossip active: true
Thrift active: true
Native Transport active: true
Load : 69,45 GB
Generation No: 1415869749
Uptime (seconds) : 85
Heap Memory (MB) : 87,62 / 499,25
Data Center : dc
Rack : rack
Exceptions : 9
Key Cache: entries 189, size 15,09 KB, capacity 24 MB, 68 hits, 265
requests, 0,257 recent hit rate, 14400 save
period in seconds
Row Cache: entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
requests, NaN recent hit rate, 0 save period in
seconds
Counter Cache: entries 0, size 0 bytes, capacity 12 MB, 0 hits, 0 requests,
NaN recent hit rate, 7200 save period in
seconds
Token: (invoke with -T/--tokens to see all 256 tokens)
{panel}
Does it mean, that only 88MB of 499MB are used?
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: (was: system.log)
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.yaml
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8067) NullPointerException in KeyCacheSerializer
[
https://issues.apache.org/jira/browse/CASSANDRA-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206159#comment-14206159
]
Andreas Schnitzerling commented on CASSANDRA-8067:
--
I still have the same issue running since a few weeks w/ 2.1.1 and I already
upgraded the sstables on that node. There are a lot in my system.log. In 2.0.10
there was not such an issue. I still have most nodes running on 2.0.10. I'm
curious for Yuki's report, if upgrading the whole cluster fixes the problem.
You can see my system.log at CASSANDRA-8192.
NullPointerException in KeyCacheSerializer
--
Key: CASSANDRA-8067
URL: https://issues.apache.org/jira/browse/CASSANDRA-8067
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Eric Leleu
Fix For: 2.1.1
Hi,
I have this stack trace in the logs of Cassandra server (v2.1)
{code}
ERROR [CompactionExecutor:14] 2014-10-06 23:32:02,098
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:14,1,main]
java.lang.NullPointerException: null
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:475)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:463)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:225)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$11.run(CompactionManager.java:1061)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
Source) ~[na:1.7.0]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0]
at java.lang.Thread.run(Unknown Source) [na:1.7.0]
{code}
It may not be critical because this error occured in the AutoSavingCache.
However the line 475 is about the CFMetaData so it may hide bigger issue...
{code}
474 CFMetaData cfm =
Schema.instance.getCFMetaData(key.desc.ksname, key.desc.cfname);
475 cfm.comparator.rowIndexEntrySerializer().serialize(entry,
out);
{code}
Regards,
Eric
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: system.log
Other issues in that system.log: CASSANDRA-6283, CASSANDRA-8067, CASSANDRA-8069
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: cassandra.bat
I don't have powershell-permissions.
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.bat, cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8070) OutOfMemoryError in OptionalTasks
[
https://issues.apache.org/jira/browse/CASSANDRA-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206191#comment-14206191
]
Andreas Schnitzerling commented on CASSANDRA-8070:
--
I dream as well from better environment ;-) . In our laboratories we need to
use 32 bit because of old driver software. And that limits as well to 3GB.
Since we have a lot of those machines, we like to use that ressources to save
and process data. For our application it's enough (calculate eventcounters,
show some graphics, sometimes I run spark-jobs).
OutOfMemoryError in OptionalTasks
-
Key: CASSANDRA-8070
URL: https://issues.apache.org/jira/browse/CASSANDRA-8070
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [OptionalTasks:1] 2014-10-02 09:26:42,821 CassandraDaemon.java:166 -
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at com.yammer.metrics.stats.Snapshot.init(Snapshot.java:30)
~[metrics-core-2.2.0.jar:na]
at
com.yammer.metrics.stats.ExponentiallyDecayingSample.getSnapshot(ExponentiallyDecayingSample.java:131)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Histogram.getSnapshot(Histogram.java:180)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Timer.getSnapshot(Timer.java:183)
~[metrics-core-2.2.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:340)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
[na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source) [na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: (was: system.log)
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8192) AssertionError in Memory.java
[
https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8192:
-
Attachment: system.log
cassandra.yaml
I attached the system.log since start of 2.1.1 (continuously running) and
cassandra.yaml .
AssertionError in Memory.java
-
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Attachments: cassandra.yaml, system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and
CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8192) AssertionError in Memory.java
Andreas Schnitzerling created CASSANDRA-8192:
Summary: AssertionError in Memory.java
Key: CASSANDRA-8192
URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during
start up.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:135)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438)
~[apache-cassandra-2.1.1.jar:2.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
{panel}
In the attached log you can still see as well CASSANDRA-8069 and CASSANDRA-6283.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8071) OutOfMemoryError in CompactionExecutor
[
https://issues.apache.org/jira/browse/CASSANDRA-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163474#comment-14163474
]
Andreas Schnitzerling commented on CASSANDRA-8071:
--
Invalid config (cold_reads_to_omit) I corrected already after
documentation-studies and I put it to the CF-Config (cql). JNA I don't need and
it's just a warning. I even didn't use JNA at C* 2.0.10 and there are no such
errors. Missing JNA shouldn't be an issue for those errors.
OutOfMemoryError in CompactionExecutor
--
Key: CASSANDRA-8071
URL: https://issues.apache.org/jira/browse/CASSANDRA-8071
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [CompactionExecutor:18] 2014-10-02 09:26:44,431
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:18,1,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:79)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1874)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1660)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1672)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:272)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:278)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:148)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:468)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8071) OutOfMemoryError in CompactionExecutor
[
https://issues.apache.org/jira/browse/CASSANDRA-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163474#comment-14163474
]
Andreas Schnitzerling edited comment on CASSANDRA-8071 at 10/8/14 1:31 PM:
---
Invalid config (cold_reads_to_omit) I corrected already after
documentation-studies and I put it to the CF-Config (cql). JNA I don't need and
it's just a warning. I even didn't use JNA at C* 2.0.10 and there are no such
errors. Missing JNA shouldn't be an issue for those errors. Cassandra is
running.
was (Author: andie78):
Invalid config (cold_reads_to_omit) I corrected already after
documentation-studies and I put it to the CF-Config (cql). JNA I don't need and
it's just a warning. I even didn't use JNA at C* 2.0.10 and there are no such
errors. Missing JNA shouldn't be an issue for those errors.
OutOfMemoryError in CompactionExecutor
--
Key: CASSANDRA-8071
URL: https://issues.apache.org/jira/browse/CASSANDRA-8071
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [CompactionExecutor:18] 2014-10-02 09:26:44,431
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:18,1,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:79)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1874)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1660)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1672)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:272)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:278)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:148)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:468)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8071) OutOfMemoryError in CompactionExecutor
[
https://issues.apache.org/jira/browse/CASSANDRA-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163474#comment-14163474
]
Andreas Schnitzerling edited comment on CASSANDRA-8071 at 10/8/14 1:32 PM:
---
Invalid config (cold_reads_to_omit) I corrected already after
documentation-studies and I put it to the CF-Config (cql). JNA I don't need and
it's just a warning. I even don't use JNA at C* 2.0.10 and there are no such
errors. Missing JNA shouldn't be an issue. Cassandra is running.
was (Author: andie78):
Invalid config (cold_reads_to_omit) I corrected already after
documentation-studies and I put it to the CF-Config (cql). JNA I don't need and
it's just a warning. I even didn't use JNA at C* 2.0.10 and there are no such
errors. Missing JNA shouldn't be an issue for those errors. Cassandra is
running.
OutOfMemoryError in CompactionExecutor
--
Key: CASSANDRA-8071
URL: https://issues.apache.org/jira/browse/CASSANDRA-8071
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [CompactionExecutor:18] 2014-10-02 09:26:44,431
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:18,1,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:79)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1874)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1660)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1672)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:272)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:278)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:148)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:468)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8071) OutOfMemoryError in CompactionExecutor
[
https://issues.apache.org/jira/browse/CASSANDRA-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163499#comment-14163499
]
Andreas Schnitzerling commented on CASSANDRA-8071:
--
Actually it's running since a few days under load (importing logfiles). The
last time the failure occurred was at Oct 2th (system.log). Didn't get that
failure anymore. I still get the other failures, mostly NullPointerException
from KeyCacheSerializer in CompactionExecutor CASSANDRA-8067 .
OutOfMemoryError in CompactionExecutor
--
Key: CASSANDRA-8071
URL: https://issues.apache.org/jira/browse/CASSANDRA-8071
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [CompactionExecutor:18] 2014-10-02 09:26:44,431
CassandraDaemon.java:166 - Exception in thread
Thread[CompactionExecutor:18,1,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:79)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1874)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1660)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1672)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:272)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:278)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:148)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:468)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8009) nodetools -h host status is showing wrong load
[
https://issues.apache.org/jira/browse/CASSANDRA-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161740#comment-14161740
]
Andreas Schnitzerling commented on CASSANDRA-8009:
--
Hello,
I'm importing Gigabytes of logdata. I have 12 nodes, 2 running with C*
2.1-release and 10 with C* 2.0.10. Because of a failure in importing process
before, I delete the data and write again. I do not replace all data, so I
wouldn't delete the whole CF. The interesting part is, that the space of files
is growing in the same amount like on the 2.0.10-nodes but nodetool status is
showing me less and less load compared to the other nodes. I expect negative
values in the future like I read somewhere else with C* 2.1. Here is the output
of nodetool status:
UN 10.9.9.69 67,19 GB 256 11,1%
UN 10.6.8.107 62,06 GB 256 10,1%
UN 10.7.8.144 32,13 GB 128 4,3%
UN 10.6.8.92,47 GB256 10,3%
UN 10.6.8.127 4,5 GB 256 10,7%
UN 10.6.8.212 60,88 GB 256 10,5%
UN 10.7.8.124 33,68 GB 128 3,8%
UN 10.7.8.140 60,58 GB 128 5,7%
UN 10.9.9.151 58,58 GB 256 10,0%
UN 10.9.9.241 54,84 GB 256 9,7%
UN 10.7.8.138 35,03 GB 128 4,6%
UN 10.9.9.240 58,83 GB 256 9,3%
Nodes 10.6.8.9 and 10.6.8.127 (both C* 2.1, rest C* 2.0.10) should have
something between 54GB and 61GB like the other nodes with 256 tokens.
I hope there's no impact to the data...
Env: Windows-7 32bit, Java 7 (1.7.0_67)
nodetools -h host status is showing wrong load
--
Key: CASSANDRA-8009
URL: https://issues.apache.org/jira/browse/CASSANDRA-8009
Project: Cassandra
Issue Type: Bug
Components: Core, Tools
Environment: Cassandra: Cassandra 2.1.0
OS: Red Hat Enterprise Linux Server release 6.5
Java: Java(TM) SE Runtime Environment (build 1.7.0_55-b13)
Java: Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)
Reporter: leandro moreira
Fix For: 2.1.0
When I use *nodetool status* it shows me a load very different from *df -h*
outuput.
{quote}
$nodetool -h anyofmyhost status
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 1 1.67 TB256 23.2% 6216e547-xxx RAC1
UN 2 1.86 TB256 24.3% 835c566d-xxx RAC1
[user@anyofmyhost ~]$ df -h
FilesystemSize Used Avail Use% Mounted on
...
/dev/sda6 1.6T 70G 1.5T 5% /opt
...
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8069) IllegalArgumentException in SSTableBatchOpen
Andreas Schnitzerling created CASSANDRA-8069:
Summary: IllegalArgumentException in SSTableBatchOpen
Key: CASSANDRA-8069
URL: https://issues.apache.org/jira/browse/CASSANDRA-8069
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception on start and
during operation.
{panel:title=system.log}
ERROR [SSTableBatchOpen:1] 2014-10-01 10:43:27,088 CassandraDaemon.java:166 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.IllegalArgumentException: null
at org.apache.cassandra.io.util.Memory.allocate(Memory.java:61)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:172)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:125)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:757)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:716)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:394)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:294)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:430)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8070) OutOfMemoryError in OptionalTasks
Andreas Schnitzerling created CASSANDRA-8070:
Summary: OutOfMemoryError in OptionalTasks
Key: CASSANDRA-8070
URL: https://issues.apache.org/jira/browse/CASSANDRA-8070
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception on start and
during operation.
{panel:title=system.log}
ERROR [OptionalTasks:1] 2014-10-02 09:26:42,821 CassandraDaemon.java:166 -
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at com.yammer.metrics.stats.Snapshot.init(Snapshot.java:30)
~[metrics-core-2.2.0.jar:na]
at
com.yammer.metrics.stats.ExponentiallyDecayingSample.getSnapshot(ExponentiallyDecayingSample.java:131)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Histogram.getSnapshot(Histogram.java:180)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Timer.getSnapshot(Timer.java:183)
~[metrics-core-2.2.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:340)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
[na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source) [na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8071) OutOfMemoryError in CompactionExecutor
Andreas Schnitzerling created CASSANDRA-8071:
Summary: OutOfMemoryError in CompactionExecutor
Key: CASSANDRA-8071
URL: https://issues.apache.org/jira/browse/CASSANDRA-8071
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [CompactionExecutor:18] 2014-10-02 09:26:44,431 CassandraDaemon.java:166
- Exception in thread Thread[CompactionExecutor:18,1,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:79)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1874)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1660)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1672)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:272)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:278)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:148)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:468)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8070) OutOfMemoryError in OptionalTasks
[
https://issues.apache.org/jira/browse/CASSANDRA-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-8070:
-
Description:
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [OptionalTasks:1] 2014-10-02 09:26:42,821 CassandraDaemon.java:166 -
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at com.yammer.metrics.stats.Snapshot.init(Snapshot.java:30)
~[metrics-core-2.2.0.jar:na]
at
com.yammer.metrics.stats.ExponentiallyDecayingSample.getSnapshot(ExponentiallyDecayingSample.java:131)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Histogram.getSnapshot(Histogram.java:180)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Timer.getSnapshot(Timer.java:183)
~[metrics-core-2.2.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:340)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
[na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source) [na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
was:
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception on start and
during operation.
{panel:title=system.log}
ERROR [OptionalTasks:1] 2014-10-02 09:26:42,821 CassandraDaemon.java:166 -
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at com.yammer.metrics.stats.Snapshot.init(Snapshot.java:30)
~[metrics-core-2.2.0.jar:na]
at
com.yammer.metrics.stats.ExponentiallyDecayingSample.getSnapshot(ExponentiallyDecayingSample.java:131)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Histogram.getSnapshot(Histogram.java:180)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Timer.getSnapshot(Timer.java:183)
~[metrics-core-2.2.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:340)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75)
~[apache-cassandra-2.1.0.jar:2.1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
[na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
Source) [na:1.7.0_67]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_67]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_67]
{panel}
OutOfMemoryError in OptionalTasks
-
Key: CASSANDRA-8070
URL: https://issues.apache.org/jira/browse/CASSANDRA-8070
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
Reporter: Andreas Schnitzerling
Attachments: system.log
Since update of 2 of 12 nodes from 2.0.10 to 2.1-release Exception during
operation.
{panel:title=system.log}
ERROR [OptionalTasks:1] 2014-10-02 09:26:42,821 CassandraDaemon.java:166 -
Exception in thread Thread[OptionalTasks:1,5,main]
java.lang.OutOfMemoryError: Java heap space
at com.yammer.metrics.stats.Snapshot.init(Snapshot.java:30)
~[metrics-core-2.2.0.jar:na]
at
com.yammer.metrics.stats.ExponentiallyDecayingSample.getSnapshot(ExponentiallyDecayingSample.java:131)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Histogram.getSnapshot(Histogram.java:180)
~[metrics-core-2.2.0.jar:na]
at com.yammer.metrics.core.Timer.getSnapshot(Timer.java:183)
~[metrics-core-2.2.0.jar:na]
at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:340)
~[apache-cassandra-2.1.0.jar:2.1.0]
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13961829#comment-13961829
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
That means in short words, deletion during normal operation should work? And
nodetool compact w/o -par should work as well? Tell me, if it's ready to test
in my environment.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.7
Attachments: 6283_StreamWriter_patch.txt, leakdetect.patch,
neighbor-log.zip, root-log.zip, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13920811#comment-13920811
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Hello,
since I don't know all code areas of C*, I describe, what I tested to
reproduce: I cleaned system.log and used again C* 2.0.5-rel with LEAK detection
and finalizer-patch in RAR.java. After starting again C* w/o doing anything I
got a lot LEAK messages. I waited until C* finished his own work (mainly
compacting I think). Now I started repair -par. Result are a lot of LEAK
messages. Here the first one:
{panel:title=nodetool repair -par events}
ERROR [Finalizer] 2014-03-05 13:45:25,932 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbyproject\events-eventsbyproject-jb-2002-Index.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:103)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
at
org.apache.cassandra.io.util.BufferedPoolingSegmentedFile.createReader(BufferedPoolingSegmentedFile.java:45)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:162)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:143)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:936)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
at
org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:783)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1186)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1174)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:252)
at
org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.init(CompactionManager.java:888)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:787)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:397)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
If I can make more tests, let me know. After Thursday I will be on holiday for
3 weeks and in office again at Mon, 03/31/2014.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: 6283_StreamWriter_patch.txt, leakdetect.patch,
screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13920811#comment-13920811
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 3/5/14 12:59 PM:
---
Hello,
since I don't know all code areas of C*, I describe, what I tested to
reproduce: I cleaned system.log and used again C* 2.0.5-rel with LEAK detection
and finalizer-patch in RAR.java. After starting again C* w/o doing anything I
got a lot LEAK messages. I waited until C* finished his own work (mainly
compacting I think). Now I started repair -par. Result are a lot of LEAK
messages. Here the first one:
{panel:title=nodetool repair -par events}
ERROR [Finalizer] 2014-03-05 13:45:25,932 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbyproject\events-eventsbyproject-jb-2002-Index.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:103)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
at
org.apache.cassandra.io.util.BufferedPoolingSegmentedFile.createReader(BufferedPoolingSegmentedFile.java:45)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:162)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:143)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:936)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
at
org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:783)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1186)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1174)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:252)
at
org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.init(CompactionManager.java:888)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:787)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:397)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
{panel:title=neighbor node}
ERROR [Finalizer] 2014-03-05 13:50:54,061 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\evrangesdevice\events-evrangesdevice-jb-905-Index.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:103)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
at
org.apache.cassandra.io.util.BufferedPoolingSegmentedFile.createReader(BufferedPoolingSegmentedFile.java:45)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:162)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:143)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:936)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
at
org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:788)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1186)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1174)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:252)
at
org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.init(CompactionManager.java:888)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:787)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13920811#comment-13920811
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 3/5/14 1:00 PM:
--
Hello,
since I don't know all code areas of C*, I describe, what I tested to
reproduce: I cleaned system.log and used again C* 2.0.5-rel with LEAK detection
and finalizer-patch in RAR.java. After starting again C* w/o doing anything I
got a lot LEAK messages. I waited until C* finished his own work (mainly
compacting I think). Now I started repair -par. Result are a lot of LEAK
messages. Here the first one:
{panel:title=nodetool repair -par events}
ERROR [Finalizer] 2014-03-05 13:45:25,932 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbyproject\events-eventsbyproject-jb-2002-Index.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:103)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
at
org.apache.cassandra.io.util.BufferedPoolingSegmentedFile.createReader(BufferedPoolingSegmentedFile.java:45)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:162)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:143)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:936)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
at
org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:783)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1186)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1174)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:252)
at
org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.init(CompactionManager.java:888)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:787)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:397)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
{panel:title=neighbor node}
ERROR [Finalizer] 2014-03-05 13:50:54,061 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\evrangesdevice\events-evrangesdevice-jb-905-Index.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:103)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
at
org.apache.cassandra.io.util.BufferedPoolingSegmentedFile.createReader(BufferedPoolingSegmentedFile.java:45)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:162)
at
org.apache.cassandra.io.util.SegmentedFile$SegmentIterator.next(SegmentedFile.java:143)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:936)
at
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:871)
at
org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:788)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1186)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1174)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:252)
at
org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.init(CompactionManager.java:888)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:787)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
[jira] [Updated] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-6283:
-
Attachment: neighbor-log.zip
root-log.zip
Logs during nodetool repair -par events. C* 2.0.5-rel with LEAK-log and
finalizer-patch under Win-7.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: 6283_StreamWriter_patch.txt, leakdetect.patch,
neighbor-log.zip, root-log.zip, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13921180#comment-13921180
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 3/5/14 6:40 PM:
--
I attached logs (root-log.zip and neighbor-log.zip) during nodetool repair
-par events. C* 2.0.5-rel with LEAK-log and finalizer-patch under Win-7.
was (Author: andie78):
Logs during nodetool repair -par events. C* 2.0.5-rel with LEAK-log and
finalizer-patch under Win-7.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: 6283_StreamWriter_patch.txt, leakdetect.patch,
neighbor-log.zip, root-log.zip, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13908144#comment-13908144
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Sounds interesting, I didn't work with such links yet. Can somebody check that
behavior on Windows XP (32) as well? Before migrating to Win7, we didn't have
that problems. Our nodes run on Win7 (32 Bit).
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13908241#comment-13908241
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
(Yep, but I thought, the difference can be the key to the solution :-) )
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906818#comment-13906818
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
So the repair issue (with snapshots) is another problem compared to the
compaction issue, where my finalizer patch works perfectly in normal operation
and shows, that handles are not closed from C* ? Relying to compaction issue,
if C* generates a tmp folder, there will be thousands of files (I had for
instance more than 64K files in one KS w/o finalizer patch). Is that a good
idea for the operating system, when C* is running for weeks?
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905320#comment-13905320
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
I think, not stopping java in general allows deleting, but stopping the
cassandra process which keeps file handles open. When I first studied the C*
code, I found it difficult to see, how filehandles are spread around in
different classes (?). Not easy, to keep the overview for me, but maybe its
because of not enough java experience compared to C++ where I had to care for
handles as well (file, memory). Later in C++, there were autopointers
(boost/TR1) with a finalizer-like approach, where memory was deleted
automatically (analog to closing files), when handle went out of scope or the
programmer forgot to free/close. I don't know about the linux' process
explorer, where filehandles are expected. But if I probably close every
filehandle after use (even only reading), does it matter, what's in this
process explorer, as long C* is able to delete? BTW if only a read process
don't close the file handle, it makes already sense to forbid to delete, as
long the os knows, the file is still used any way. For me, it's a failure of
the os as manager, to allow delete a file, as long there are any filehandles on
it. This strict permission allows me as well, to detect failures in my program
logic.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905320#comment-13905320
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 2/19/14 11:54 AM:
I think, not stopping java in general allows deleting, but stopping the
cassandra process which keeps file handles open. When I first studied the C*
code, I found it difficult to see, how filehandles are spread around in
different classes (?). Not easy, to keep the overview for me, but maybe its
because of not enough java experience compared to C++ where I had to care for
handles as well (file, memory). Later in C++, there were autopointers
(boost/TR1) with a finalizer-like approach, where memory was deleted
automatically (analog to closing files), when handle went out of scope or the
programmer forgot to free/close. I don't know about the linux' process
explorer, where filehandles are expected. But if I probably close every
filehandle after use (even only reading), does it matter, what's in this
process explorer, as long C* is able to delete? BTW if only a read process
don't close the file handle, it makes already sense for me to forbid to delete,
as long the os knows, the file is still used any way, except the user has to
kill or restart the process, where the handles automatically disappear, mostly
caused by problems. For me, it's a failure of the os as manager, to allow
delete a file, as long there are any filehandles on it. This strict permission
allows me as well, to detect failures in my program logic.
was (Author: andie78):
I think, not stopping java in general allows deleting, but stopping the
cassandra process which keeps file handles open. When I first studied the C*
code, I found it difficult to see, how filehandles are spread around in
different classes (?). Not easy, to keep the overview for me, but maybe its
because of not enough java experience compared to C++ where I had to care for
handles as well (file, memory). Later in C++, there were autopointers
(boost/TR1) with a finalizer-like approach, where memory was deleted
automatically (analog to closing files), when handle went out of scope or the
programmer forgot to free/close. I don't know about the linux' process
explorer, where filehandles are expected. But if I probably close every
filehandle after use (even only reading), does it matter, what's in this
process explorer, as long C* is able to delete? BTW if only a read process
don't close the file handle, it makes already sense to forbid to delete, as
long the os knows, the file is still used any way. For me, it's a failure of
the os as manager, to allow delete a file, as long there are any filehandles on
it. This strict permission allows me as well, to detect failures in my program
logic.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13903996#comment-13903996
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
I updated 2 of 8 nodes to 2.0.5-rel, with finalizer patch and LEAK-logging.
Result: After Start a lot of LEAK-finalizer Errors like:
{panel:title=start-up node}
ERROR [Finalizer] 2014-02-18 12:53:42,388 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
ERROR [Finalizer] 2014-02-18 12:53:42,388 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\schema_keyspaces\system-schema_keyspaces-jb-433-Data.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:55)
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1362)
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1147)
at
org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:69)
at
org.apache.cassandra.db.ColumnFamilyStore.getSequentialIterator(ColumnFamilyStore.java:1599)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1718)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1656)
at
org.apache.cassandra.db.SystemKeyspace.serializedSchema(SystemKeyspace.java:767)
at
org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:121)
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:525)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:242)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:462)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:552)
ERROR [Finalizer] 2014-02-18 12:53:42,388 RandomAccessReader.java (line 394)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\local\system-local-jb-168-Data.db allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:43)
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:48)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1195)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:57)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.cql3.statements.SelectStatement.readLocally(SelectStatement.java:232)
at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:250)
at
org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:58)
at
org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:255)
at
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:526)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:233)
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904251#comment-13904251
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
You're welcome! :-)
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13903474#comment-13903474
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Actually Im still running 2.0.5 Snapshot with my finalizer patch. During normal
operation no fault, but I didn't check the logs in the last 2 weeks yet. The
repair jobs I start with -par option. I will update to 2.0.5-rel with my patch
and test repair w/o -par option and compare the logs to yours. Correct my plan,
if necessary.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870952#comment-13870952
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 1/15/14 10:47 AM:
Yesterday I updated one node with 2.0.4-rel incl. finalizer-patch (see results
above). Nodetool repair -par caused node to repair endless and collecting
about 65K Files in datafolder. I updated now to pre-2.0.5 from today (commit
f6f50ddffe0821617fe29482f9ec918608560381). After starting, a lot of LEAK
messages and File-Not-Found messages appeared in system.log. But files reduce.
{panel:title=system.log (pre-2.0.5)}
ERROR [SSTableBatchOpen:1] 2014-01-14 18:18:42,753 CassandraDaemon.java:139 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:109)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.buildSummary(SSTableReader.java:595)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:575)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:527)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:328)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:230)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:364)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
~[na:1.7.0_25]
at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_25]
Caused by: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_25]
at java.io.RandomAccessFile.init(Unknown Source) ~[na:1.7.0_25]
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
... 13 common frames omitted
...
ERROR [Finalizer] 2014-01-14 18:27:45,076 RandomAccessReader.java:401 - LEAK
finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-5012-Statistics.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:65)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:88)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:98)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:167)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:125)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870606#comment-13870606
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Hello,
this problem is occurred, when node was not shutdown probably. As I know, that
issue is known as CASSANDRA-6531. Here is the stack trace:
{panel:title=system.log}
ERROR [ReadStage:2385] 2014-01-14 10:57:11,875 CassandraDaemon.java (line 187)
Exception in thread Thread[ReadStage:2385,5,main]
java.lang.RuntimeException: java.lang.IllegalArgumentException: bufferSize must
be positive
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:49)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException: bufferSize must be positive
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:75)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:43)
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:48)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1195)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:57)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1516)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1335)
at
org.apache.cassandra.db.index.composites.CompositesSearcher$1.computeNext(CompositesSearcher.java:245)
at
org.apache.cassandra.db.index.composites.CompositesSearcher$1.computeNext(CompositesSearcher.java:105)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at
org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1710)
at
org.apache.cassandra.db.index.composites.CompositesSearcher.search(CompositesSearcher.java:53)
at
org.apache.cassandra.db.index.SecondaryIndexManager.search(SecondaryIndexManager.java:537)
at
org.apache.cassandra.db.ColumnFamilyStore.search(ColumnFamilyStore.java:1698)
at
org.apache.cassandra.db.RangeSliceCommand.executeLocally(RangeSliceCommand.java:135)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
... 4 more
ERROR [Finalizer] 2014-01-14 10:57:12,005 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\nieste\niesteinverters\nieste-niesteinverters-jb-2669-Data.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:66)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:43)
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile.createReader(CompressedPoolingSegmentedFile.java:48)
at
org.apache.cassandra.io.util.PoolingSegmentedFile.getSegment(PoolingSegmentedFile.java:39)
at
org.apache.cassandra.io.sstable.SSTableReader.getFileDataInput(SSTableReader.java:1195)
at
org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:57)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870952#comment-13870952
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Yesterday I Updated one node with 2.0.4-rel incl. Finalizer-Patch (see results
above). nodetool repair -par caused node to repair endless and collecting
about 65K Files in datafolder. I updated now to pre-2.0.5 from today (commit
f6f50ddffe0821617fe29482f9ec918608560381). After starting, a lot of LEAK
messages and File-Not-Found messages. But Files reduce.
{panel:title=system.log (pre-2.0.5)}
ERROR [SSTableBatchOpen:1] 2014-01-14 18:18:42,753 CassandraDaemon.java:139 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:109)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.buildSummary(SSTableReader.java:595)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:575)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:527)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:328)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:230)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:364)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
~[na:1.7.0_25]
at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_25]
Caused by: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_25]
at java.io.RandomAccessFile.init(Unknown Source) ~[na:1.7.0_25]
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
... 13 common frames omitted
...
ERROR [Finalizer] 2014-01-14 18:27:45,076 RandomAccessReader.java:401 - LEAK
finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-5012-Statistics.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:65)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:88)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:98)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:167)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:125)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870952#comment-13870952
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 1/14/14 5:52 PM:
---
Yesterday I updated one node with 2.0.4-rel incl. finalizer-patch (see results
above). Nodetool repair -par caused node to repair endless and collecting
about 65K Files in datafolder. I updated now to pre-2.0.5 from today (commit
f6f50ddffe0821617fe29482f9ec918608560381). After starting, a lot of LEAK
messages and File-Not-Found messages appeared in system.log. But files reduce.
{panel:title=system.log (pre-2.0.5)}
ERROR [SSTableBatchOpen:1] 2014-01-14 18:18:42,753 CassandraDaemon.java:139 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:109)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.buildSummary(SSTableReader.java:595)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:575)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:527)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:328)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:230)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:364)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
~[na:1.7.0_25]
at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_25]
Caused by: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_25]
at java.io.RandomAccessFile.init(Unknown Source) ~[na:1.7.0_25]
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
... 13 common frames omitted
...
ERROR [Finalizer] 2014-01-14 18:27:45,076 RandomAccessReader.java:401 - LEAK
finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-5012-Statistics.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:65)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:88)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:98)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:167)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:125)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13870952#comment-13870952
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 1/14/14 5:57 PM:
---
Yesterday I updated one node with 2.0.4-rel incl. finalizer-patch (see results
above). Nodetool repair -par caused node to repair endless and collecting
about 65K Files in datafolder. I updated now to pre-2.0.5 from today (commit
f6f50ddffe0821617fe29482f9ec918608560381). After starting, a lot of LEAK
messages and File-Not-Found messages appeared in system.log. But files reduce.
{panel:title=system.log (pre-2.0.5)}
ERROR [SSTableBatchOpen:1] 2014-01-14 18:18:42,753 CassandraDaemon.java:139 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:109)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.buildSummary(SSTableReader.java:595)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:575)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:527)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:328)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:230)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:364)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
~[na:1.7.0_25]
at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_25]
Caused by: java.io.FileNotFoundException:
D:\Programme\cassandra\data\KSlogdata\CFlogdata\KSlogdata-CFlogdata-jb-27051-Index.db
(Das System kann die angegebene Datei nicht finden)
at java.io.RandomAccessFile.open(Native Method) ~[na:1.7.0_25]
at java.io.RandomAccessFile.init(Unknown Source) ~[na:1.7.0_25]
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:63)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
... 13 common frames omitted
...
ERROR [Finalizer] 2014-01-14 18:27:45,076 RandomAccessReader.java:401 - LEAK
finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-5012-Statistics.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:65)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:105)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:97)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:88)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:98)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:167)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:125)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1-SNAPSHOT.jar:2.1-SNAPSHOT]
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869458#comment-13869458
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Hello,
with 2.0.4 and leak detect patch I got expected errors
{panel title=system.log}
ERROR [Finalizer] 2014-01-13 13:13:15,033 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-3418-Data.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:66)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1355)
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1161)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1173)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:244)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:250)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:126)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
ERROR [Finalizer] 2014-01-13 13:13:15,053 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4984-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,073 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4984-Index.db
allocated
...
ERROR [CompactionExecutor:2] 2014-01-13 13:13:15,073 CassandraDaemon.java (line
187) Exception in thread Thread[CompactionExecutor:2,1,main]
java.lang.IllegalArgumentException: bufferSize must be positive
...
ERROR [Finalizer] 2014-01-13 13:13:15,083 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-5067-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,123 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-5067-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,133 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4302-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,153 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4302-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,163 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4981-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,173 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4981-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,183 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
[jira] [Comment Edited] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13869458#comment-13869458
]
Andreas Schnitzerling edited comment on CASSANDRA-6283 at 1/13/14 12:50 PM:
Hello,
with 2.0.4 and leak detect patch I got expected errors
{panel:title=system.log}
ERROR [Finalizer] 2014-01-13 13:13:15,033 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-3418-Data.db
allocated
at
org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:66)
at
org.apache.cassandra.io.compress.CompressedRandomAccessReader.init(CompressedRandomAccessReader.java:76)
at
org.apache.cassandra.io.compress.CompressedThrottledReader.init(CompressedThrottledReader.java:34)
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:48)
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1355)
at
org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1161)
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1173)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:244)
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:250)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:126)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
ERROR [Finalizer] 2014-01-13 13:13:15,053 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4984-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,073 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4984-Index.db
allocated
...
ERROR [CompactionExecutor:2] 2014-01-13 13:13:15,073 CassandraDaemon.java (line
187) Exception in thread Thread[CompactionExecutor:2,1,main]
java.lang.IllegalArgumentException: bufferSize must be positive
...
ERROR [Finalizer] 2014-01-13 13:13:15,083 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-5067-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,123 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-5067-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,133 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4302-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,153 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4302-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,163 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4981-Data.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,173 RandomAccessReader.java (line 398)
LEAK finalizer had to clean up
java.lang.Exception: RAR for
D:\Programme\cassandra\data\events\eventsbydevice\events-eventsbydevice-jb-4981-Index.db
allocated
...
ERROR [Finalizer] 2014-01-13 13:13:15,183 RandomAccessReader.java (line 398)
LEAK finalizer had to
[jira] [Commented] (CASSANDRA-6463) cleanup causes permission problems
[ https://issues.apache.org/jira/browse/CASSANDRA-6463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863039#comment-13863039 ] Andreas Schnitzerling commented on CASSANDRA-6463: -- Which node(s) need repair? The one(s) cleaned up or all nodes of the cluster? cleanup causes permission problems -- Key: CASSANDRA-6463 URL: https://issues.apache.org/jira/browse/CASSANDRA-6463 Project: Cassandra Issue Type: Bug Components: Core, Tools Environment: Windows 7 / Java 1.7.0.25 Reporter: Andreas Schnitzerling Labels: authentication, cleanup, consistency, cql3, newbie, nodetool, permissions Fix For: 2.0.5 After a cleanup I loose permissions (f.e. SELECT, MODIFY). When I listed the system_auth/permissions CF after cleanup I recognized, around the half of all permissions lost - BUT: If I list the permissions table with consistency ALL, all entries appear again and my programm continues working. Only if I manual trigger that read-repair! That tells me indirect, that system_auth is reading using consistency ONE, what causes problems after cleanup (?). A good approach (?) could be to re-read system_auth after permission fail with consistency ONE to trigger read-repair once and keep speed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6462) cleanup ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13846267#comment-13846267
]
Andreas Schnitzerling commented on CASSANDRA-6462:
--
What can I do now? repair doesn't help and major compact doesn't help. Cleanup
still stops with SSTableReader / ClassCastException.
Reset node (delete all and start new)? Wait for 2.0.4? Tx.
cleanup ClassCastException
--
Key: CASSANDRA-6462
URL: https://issues.apache.org/jira/browse/CASSANDRA-6462
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: Windows 7 / Java 1.7.0.25
Reporter: Andreas Schnitzerling
Assignee: Jonathan Ellis
Labels: cleanup, compaction
Fix For: 2.0.4
I enlarged the cluseter from 4 to 8 nodes. During cleaning up the old nodes
with nodetool cleanup it breaks up with exception. I started cleanup from a
different computer to manage them sequentially.
{panel:title=cmd.exe}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.ClassCastException:
org.apach
e.cassandra.io.sstable.SSTableReader$EmptyCompactionScanner cannot be cast to
or
g.apache.cassandra.io.sstable.SSTableScanner
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTabl
eOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(C
ompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilySt
ore.java:1054)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(Stor
ageService.java:2038)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.reflect.misc.Trampoline.invoke(Unknown Source)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown
So
urce)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown
So
urce)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown
Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown
Sou
rce)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown
Sour
ce)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run
(Unknown Source)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(U
nknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown
Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown
Sou
rce)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown
Sour
ce)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassCastException:
org.apache.cassandra.io.sstable.SSTable
Reader$EmptyCompactionScanner cannot be cast to
org.apache.cassandra.io.sstable.
SSTableScanner
at
org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompact
[jira] [Created] (CASSANDRA-6463) cleanup causes permission problems
Andreas Schnitzerling created CASSANDRA-6463: Summary: cleanup causes permission problems Key: CASSANDRA-6463 URL: https://issues.apache.org/jira/browse/CASSANDRA-6463 Project: Cassandra Issue Type: Bug Components: Core, Tools Environment: Windows 7 / Java 1.7.0.25 Reporter: Andreas Schnitzerling Fix For: 2.0.4 After a cleanup I loose permissions (f.e. SELECT, MODIFY). When I listed the system_auth/permissions CF after cleanup I recognized, around the half of all permissions lost - BUT: If I list the permissions table with consistency ALL, all entries appear again and my programm continues working. Only if I manual trigger that read-repair! That tells me indirect, that system_auth is reading using consistency ONE, what causes problems after cleanup (?). A good approach (?) could be to re-read system_auth after permission fail with consistency ONE to trigger read-repair once and keep speed. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (CASSANDRA-6462) cleanup ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843336#comment-13843336
]
Andreas Schnitzerling commented on CASSANDRA-6462:
--
Cleanup on that node not working anymore. There are already problems after
restart of the cassandra service. Any ideas how to fix? If u don't have idea as
well I will try to repair, but maybe cannot reproduce anymore.
{panel:title=system.log}
ERROR [CompactionExecutor:25] 2013-12-09 17:50:14,932 CassandraDaemon.java
(line 187) Exception in thread Thread[CompactionExecutor:25,1,RMI Runtime]
java.util.concurrent.RejectedExecutionException: Task
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1dbf287
rejected from
org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@10f8a54[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 780]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(Unknown
Source)
at java.util.concurrent.ScheduledThreadPoolExecutor.submit(Unknown
Source)
at
org.apache.cassandra.io.sstable.SSTableDeletingTask.schedule(SSTableDeletingTask.java:66)
at
org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1105)
at
org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:388)
at org.apache.cassandra.db.DataTracker.postReplace(DataTracker.java:353)
at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:347)
at
org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:252)
at
org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1078)
at
org.apache.cassandra.db.compaction.CompactionTask.replaceCompactedSSTables(CompactionTask.java:296)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:242)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
ERROR [ValidationExecutor:1] 2013-12-09 17:53:16,411 Validator.java (line 242)
Failed creating a merkle tree for [repair #65efe6e0-60f2-11e3-ac47-eb1c24a59bb8
on nieste/nfiles, (3837863087520363054,3847126934337600104]], /10.9.9.240 (see
log for details)
ERROR [ValidationExecutor:1] 2013-12-09 17:53:16,421 CassandraDaemon.java (line
187) Exception in thread Thread[ValidationExecutor:1,1,main]
FSWriteError in
D:\Programme\cassandra\data\nieste\nfiles\snapshots\65efe6e0-60f2-11e3-ac47-eb1c24a59bb8\nieste-nfiles-jb-19878-Index.db
at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
at
org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:382)
at
org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:378)
at
org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:416)
at
org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:1801)
at
org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:810)
at
org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
at
org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:397)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.nio.file.FileSystemException:
D:\Programme\cassandra\data\nieste\nfiles\snapshots\65efe6e0-60f2-11e3-ac47-eb1c24a59bb8\nieste-nfiles-jb-19878-Index.db:
Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen
Prozess verwendet wird.
at
[jira] [Updated] (CASSANDRA-6462) cleanup ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andreas Schnitzerling updated CASSANDRA-6462:
-
Component/s: Core
cleanup ClassCastException
--
Key: CASSANDRA-6462
URL: https://issues.apache.org/jira/browse/CASSANDRA-6462
Project: Cassandra
Issue Type: Bug
Components: Core, Tools
Environment: Windows 7 / Java 1.7.0.25
Reporter: Andreas Schnitzerling
Labels: cast, cleanup, compaction, concurrency, newbie,
nodetool, remote
Fix For: 2.0.4
I enlarged the cluseter from 4 to 8 nodes. During cleaning up the old nodes
with nodetool cleanup it breaks up with exception. I started cleanup from a
different computer to manage them sequentially.
{panel:title=cmd.exe}
Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.ClassCastException:
org.apach
e.cassandra.io.sstable.SSTableReader$EmptyCompactionScanner cannot be cast to
or
g.apache.cassandra.io.sstable.SSTableScanner
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTabl
eOperation(CompactionManager.java:227)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(C
ompactionManager.java:265)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilySt
ore.java:1054)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(Stor
ageService.java:2038)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.reflect.misc.Trampoline.invoke(Unknown Source)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown
So
urce)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown
So
urce)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown
Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown
Sou
rce)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown
Sour
ce)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run
(Unknown Source)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(U
nknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown
Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown
Sou
rce)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown
Sour
ce)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassCastException:
org.apache.cassandra.io.sstable.SSTable
Reader$EmptyCompactionScanner cannot be cast to
org.apache.cassandra.io.sstable.
SSTableScanner
at
org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompact
ion(CompactionManager.java:563)
at
org.apache.cassandra.db.compaction.CompactionManager.access$400(Compa
ctionManager.java:62)
at
org.apache.cassandra.db.compaction.CompactionManager$5.perform(Compac
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836458#comment-13836458
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
Over weekend I got the following:
{panel:title=system.log}
ERROR [STREAM-IN-/10.6.8.78] 2013-11-29 17:57:20,266 StreamSession.java (line
410) [Stream #ea9bd2c0-5916-11e3-a5b5-b13a5fe180aa] Streaming error occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at
org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
at
org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
at
org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
at java.lang.Thread.run(Unknown Source)
WARN [STREAM-IN-/10.6.8.78] 2013-11-29 17:57:20,266 StreamResultFuture.java
(line 210) [Stream #ea9bd2c0-5916-11e3-a5b5-b13a5fe180aa] Stream failed
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:54,266 SSTableDeletingTask.java
(line 81) Unable to delete
D:\Programme\cassandra\data\system\schema_columns\system-schema_columns-jb-468-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:54,266 SSTableDeletingTask.java
(line 81) Unable to delete
D:\Programme\cassandra\data\system\schema_columns\system-schema_columns-jb-469-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:58,446 SSTableDeletingTask.java
(line 81) Unable to delete
D:\Programme\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-jb-525-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:58,446 SSTableDeletingTask.java
(line 81) Unable to delete
D:\Programme\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-jb-527-Data.db
(it will be removed on server restart; we'll also retry after GC)
{panel}
The deleting problem occurs only once. So my GC patch seems to work. But
there's no log of the leakdetect-patch.
{panel:title=system.log 10.6.8.78}
ERROR [CompactionExecutor:40] 2013-11-29 17:56:16,368 CassandraDaemon.java
(line 187) Exception in thread Thread[CompactionExecutor:40,1,main]
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
down
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:145)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at
org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:752)
at
org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:817)
at
org.apache.cassandra.db.SystemKeyspace.forceBlockingFlush(SystemKeyspace.java:420)
at
org.apache.cassandra.db.SystemKeyspace.finishCompaction(SystemKeyspace.java:197)
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:225)
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Priority: Critical
Labels:
