Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-21 Thread Bhaskar Singhal
I have not seen the issue after changing the commit log segment size to 1024MB. tpstats output: Pool Name    Active   Pending  Completed   Blocked  All time blocked ReadStage 0 0  0 0      0

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-21 Thread Robert Coli
On Mon, Jul 21, 2014 at 8:07 AM, Bhaskar Singhal bhaskarsing...@yahoo.com wrote: I have not seen the issue after changing the commit log segment size to 1024MB. Yes... your insanely over-huge commitlog will be contained in fewer files if you increase the size of segments that will not

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-18 Thread Robert Coli
On Mon, Jul 7, 2014 at 9:30 PM, Bhaskar Singhal bhaskarsing...@yahoo.com wrote: I am using Cassandra 2.0.7 (with default settings and 16GB heap on quad core ubuntu server with 32gb ram) 16GB of heap will lead to significant GC pauses, and probably will not improve total performance versus 8gb

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-17 Thread Benedict Elliott Smith
Are you still seeing the same exceptions about too many open files? On Thu, Jul 17, 2014 at 6:28 AM, Bhaskar Singhal bhaskarsing...@yahoo.com wrote: Even after changing ulimits and moving to the recommended production settings, we are still seeing the same issue. root@lnx148-76:~# cat

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-17 Thread Bhaskar Singhal
Yes, I am. lsof lists around 9000 open file handles.. and there were around 3000 commitlog segments. On Thursday, 17 July 2014 1:24 PM, Benedict Elliott Smith belliottsm...@datastax.com wrote: Are you still seeing the same exceptions about too many open files? On Thu, Jul 17, 2014 at

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-16 Thread Bhaskar Singhal
Even after changing ulimits and moving to the recommended production settings, we are still seeing the same issue. root@lnx148-76:~# cat /proc/17663/limits Limit Soft Limit   Hard Limit   Units Max cpu time  unlimited    unlimited   

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-09 Thread Ruchir Jha
We have these precise settings but are still seeing the broken pipe exception in our gc logs. Any clues? Sent from my iPhone On Jul 8, 2014, at 1:17 PM, Bhaskar Singhal bhaskarsing...@yahoo.com wrote: Thanks Mark. Yes the 1024 is the limit. I haven't changed it as per the recommended

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-09 Thread Robert Coli
On Tue, Jul 8, 2014 at 10:17 AM, Bhaskar Singhal bhaskarsing...@yahoo.com wrote: But I am wondering why does Cassandra need to keep 3000+ commit log segment files open? Because you are writing faster than you can flush to disk. =Rob

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-08 Thread Mark Reddy
Hi Bhaskar, Can you check your limits using 'ulimit -a'? The default is 1024, which needs to be increased if you have not done so already. Here you will find a list of recommended production settings:

Re: TTransportException (java.net.SocketException: Broken pipe)

2014-07-08 Thread Bhaskar Singhal
Thanks Mark. Yes the 1024 is the limit. I haven't changed it as per the recommended production settings. But I am wondering why does Cassandra need to keep 3000+ commit log segment files open? Regards, Bhaskar On Tuesday, 8 July 2014 1:50 PM, Mark Reddy mark.re...@boxever.com wrote: Hi