Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

Mutations get dropped because a node can't keep up with writes. If you understand the Cassandra write path, writes are ACKed when the mutation is appended to the commitlog which is why it's very fast. Knowing that, dropped mutations mean that the disk is not able to keep up with the IO. Another

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

m: Akhil Mehra [mailto:akhilme...@gmail.com] > Sent: Friday, July 21, 2017 4:24 PM > To: user@cassandra.apache.org > Subject: Re: MUTATION messages were dropped in last 5000 ms for cross node > timeout > > Hi Asad, > > The 5000 ms is not configurable > (https://

RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

!) again. Thank again/Asad From: Akhil Mehra [mailto:akhilme...@gmail.com] Sent: Friday, July 21, 2017 4:24 PM To: user@cassandra.apache.org Subject: Re: MUTATION messages were dropped in last 5000 ms for cross node timeout Hi Asad, The 5000 ms is not configurable (https://github.com/apache

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

cpu queue length never goes beyond > medium depth. I wonder if there is some internal limit that I’m still not > aware of. > > Thanks/Asad > > > From: Akhil Mehra [mailto:akhilme...@gmail.com] > Sent: Thursday, July 20, 2017 3:47 PM > To: user@cassandra.apache.or

RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

t some number that may not be optimal.     Thank you again.       From: Anuj Wadehra [mailto:anujw_2...@yahoo.co.in] Sent: Thursday, July 20, 2017 12:17 PM To: ZAIDI, ASAD A <az1...@att.com>; user@cassandra.apache.org Subject: Re: MUTATION messages were dropped in last 5000 ms fo

RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

, July 20, 2017 3:47 PM To: user@cassandra.apache.org Subject: Re: MUTATION messages were dropped in last 5000 ms for cross node timeout Hi Asad, http://cassandra.apache.org/doc/latest/faq/index.html#why-message-dropped<https://urldefense.proofpoint.com/v2/url?u=h

RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

Thanks for your reply Subroto – I’ll try you suggestions to see if this help. I’ll revert with results. From: Subroto Barua [mailto:sbarua...@yahoo.com.INVALID] Sent: Thursday, July 20, 2017 12:22 PM To: user@cassandra.apache.org Subject: Re: MUTATION messages were dropped in last 5000 ms

RE: MUTATION messages were dropped in last 5000 ms for cross node timeout

not be optimal. Thank you again. From: Anuj Wadehra [mailto:anujw_2...@yahoo.co.in] Sent: Thursday, July 20, 2017 12:17 PM To: ZAIDI, ASAD A <az1...@att.com>; user@cassandra.apache.org Subject: Re: MUTATION messages were dropped in last 5000 ms for cross node timeout Hi Asad, You can do fol

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

Hi Asad, http://cassandra.apache.org/doc/latest/faq/index.html#why-message-dropped As mentioned in the link above this is a load shedding mechanism used by Cassandra. Is you cluster under heavy load? Regards, Akhil

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

In a cloud environment, cross_node_timeout = true can cause issues; we had this issue in our environment and it is set to false now. Dropped messages is an another issue Subroto > On Jul 20, 2017, at 8:27 AM, ZAIDI, ASAD A wrote: > > Hello Folks – > > I’m using

Re: MUTATION messages were dropped in last 5000 ms for cross node timeout

Hi Asad,  You can do following things: 1.Increase memtable_flush_writers especially if you have a write heavy load.  2.Make sure there are no big gc pauses on your nodes. If yes,  go for heap tuning.  Please let us know whether above measures fixed your problem or not.  ThanksAnuj Sent from