[
https://issues.apache.org/jira/browse/CASSANDRA-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14585644#comment-14585644
]
ZhaoYang edited comment on CASSANDRA-7145 at 6/15/15 8:44 AM:
--------------------------------------------------------------
encounter the same problem: nodetool compact failed.
log: ERROR 08:38:42 Exception in thread Thread[CompactionExecutor:1,1,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
/home/zhaoyang/program/cassandra-2.1.1/./bin/../data/data/ycsb/usertable-4fb3447310e211e5a5c2f18cd916802f/ycsb-usertable-ka-6440-Data.db
(Too many open files)
at
org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:52)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1888)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:67)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1674)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1686)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:275)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:324)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:151)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:465)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_20]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_20]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_20]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_20]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]
I was doing YCSB testing.
Setup: C* 2.1.1, Ubuntu 14.04 LTS, 24GB Ram, 256 ssd.
was (Author: jasonstack):
encounter the same problem: nodetool compact failed.
log: ERROR 08:38:42 Exception in thread Thread[CompactionExecutor:1,1,main]
java.lang.RuntimeException: java.io.FileNotFoundException:
/home/zhaoyang/program/cassandra-2.1.1/./bin/../data/data/ycsb/usertable-4fb3447310e211e5a5c2f18cd916802f/ycsb-usertable-ka-6440-Data.db
(Too many open files)
I was doing YCSB testing.
Setup: C* 2.1.1, Ubuntu 14.04 LTS, 24GB Ram, 256 ssd.
> FileNotFoundException during compaction
> ---------------------------------------
>
> Key: CASSANDRA-7145
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7145
> Project: Cassandra
> Issue Type: Bug
> Environment: CentOS 6.3, Datastax Enterprise 4.0.1 (Cassandra 2.0.5),
> Java 1.7.0_55
> Reporter: PJ
> Assignee: Marcus Eriksson
> Fix For: 1.2.19, 2.0.11, 2.1.0
>
> Attachments:
> 0001-avoid-marking-compacted-sstables-as-compacting.patch, compaction -
> FileNotFoundException.txt, repair - RuntimeException.txt, startup -
> AssertionError.txt
>
>
> I can't finish any compaction because my nodes always throw a
> "FileNotFoundException". I've already tried the following but nothing helped:
> 1. nodetool flush
> 2. nodetool repair (ends with RuntimeException; see attachment)
> 3. node restart (via dse cassandra-stop)
> Whenever I restart the nodes, another type of exception is logged (see
> attachment) somewhere near the end of startup process. This particular
> exception doesn't seem to be critical because the nodes still manage to
> finish the startup and become online.
> I don't have specific steps to reproduce the problem that I'm experiencing
> with compaction and repair. I'm in the middle of migrating 4.8 billion rows
> from MySQL via SSTableLoader.
> Some things that may or may not be relevant:
> 1. I didn't drop and recreate the keyspace (so probably not related to
> CASSANDRA-4857)
> 2. I do the bulk-loading in batches of 1 to 20 millions rows. When a batch
> reaches 100% total progress (i.e. starts to build secondary index), I kill
> the sstableloader process and cancel the index build
> 3. I restart the nodes occasionally. It's possible that there is an on-going
> compaction during one of those restarts.
> Related StackOverflow question (mine):
> http://stackoverflow.com/questions/23435847/filenotfoundexception-during-compaction
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
