Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-16 Thread Yatong Zhang
Yes the global limits are OK. I added cassandra to '/etc/rc.local' to make it auto-startup, but seems the modification of limits didn't take effect. I observed this as Bryan suggested, so I added ulimit -SHn 99 to '/etc/rc.local' and before cassandra start command, and it worked. On Thu,

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-15 Thread Nikolay Mihaylov
sorry, probably somebody mentioned it, but did you checked global limit? cat /proc/sys/fs/file-max cat /proc/sys/fs/file-nr On Mon, May 5, 2014 at 10:31 PM, Bryan Talbot bryan.tal...@playnext.comwrote: Running # cat /proc/$(cat /var/run/cassandra.pid)/limits as root or your cassandra

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-05 Thread Bryan Talbot
Running # cat /proc/$(cat /var/run/cassandra.pid)/limits as root or your cassandra user will tell you what limits it's actually running with. On Sun, May 4, 2014 at 10:12 PM, Yatong Zhang bluefl...@gmail.com wrote: I am running 'repair' when the error occurred. And just a few days before

Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread Yatong Zhang
Hi there, After I changed compaction strategy to 'leveled', one of my nodes keeps reporting 'too many open files. But I have done some configuration following: http://www.datastax.com/docs/1.1/install/recommended_settingsand http://www.datastax.com/docs/1.1/troubleshooting/index#toomany I am

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread nash
The lsof command or /proc can tell you how many open files it has. How many is it? --nash

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread Yatong Zhang
[root@storage5 ~]# lsof -n | grep java | wc -l 5103 [root@storage5 ~]# lsof | wc -l 6567 It's mentioned in previous mail:) On Mon, May 5, 2014 at 9:03 AM, nash nas...@gmail.com wrote: The lsof command or /proc can tell you how many open files it has. How many is it? --nash

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread Philip Persad
Have you tried running ulimit -a as the Cassandra user instead of as root? It is possible that your configured a high file limit for root but not for the user running the Cassandra process. On Sun, May 4, 2014 at 6:07 PM, Yatong Zhang bluefl...@gmail.com wrote: [root@storage5 ~]# lsof -n |

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread Yatong Zhang
Cassandra is running as root [root@storage5 ~]# ps aux | grep java root 1893 42.0 24.0 7630664 3904000 ? Sl 10:43 60:01 java -ea -javaagent:/mydb/cassandra/bin/../lib/jamm-0.2.5.jar -XX:+CMSClassUnloadingEnabled -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms3959M

Re: Cassandra 2.0.7 always failes due to 'too may open files' error

2014-05-04 Thread Yatong Zhang
I am running 'repair' when the error occurred. And just a few days before I changed the compaction strategy to 'leveled'. don know if this helps On Mon, May 5, 2014 at 1:10 PM, Yatong Zhang bluefl...@gmail.com wrote: Cassandra is running as root [root@storage5 ~]# ps aux | grep java root