Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
vice.java:888 - 228 MUTATION messages dropped in last 5000ms PS: As of now , there is no significant load on our cluster. The only load is of these hints been replayed. Can you Please help. Regards, Varun Saluja On 16 May 2017 at 18:28, varun saluja <saluj...@gmail.com> wrote: > Hi

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
rt? > Any agent or other process hogging system? > > Sent from my iPhone > >> On May 16, 2017, at 7:58 AM, varun saluja <saluj...@gmail.com> wrote: >> >> Hi Nitan, >> >> Thanks for response. >> >> Yes, I could see mutation drops and incre

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
Thanks a lot Jeff. You have explaned very well here. We have consitency as local quorum. Will follow truncate hints and repair therafter. I hope this brings cluster in stable state Thanks again. Regards, Varun Saluja Sent from my iPhone > On 16-May-2017, at 8:42 PM, Jeff Jirsa &

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
it. I would > say wait until cluster comes to normal state. May be some other expert can > suggest an alternate. > > Sent from my iPhone > >> On May 16, 2017, at 8:50 AM, varun saluja <saluj...@gmail.com> wrote: >> >> Thanks for update. >> I c

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
851658989612 bytes 1.31% Active compaction remaining time : 0h26m43s Will truncatehints takes time for completion? Could not see anything related truncatehints in system logs. Please let me know if anything else can be checked here. Regards, Varun Saluja On 16 May 2017 at 20:58, varun saluja

Re: Long running compaction on huge hint table.

2017-05-17 Thread varun saluja
gt; > > > > -- > Jeff Jirsa > > > On May 16, 2017, at 6:53 PM, varun saluja <saluj...@gmail.com> wrote: > > Hi, > > > Truncatehints on nodes is running for more than 7 hours now. Nothing > mentioned for same in sysemt logs even. > >

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
nhealthy nodes, focus on finding why you have mutation > drops, is it node, io or network etc. ideally you shouldn't see increasing > hints all the time. > > Sent from my iPhone > >> On May 16, 2017, at 7:58 AM, varun saluja <saluj...@gmail.com> wrote: >> >&g

Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
finished but hint table size is growing continuously. Can someone Please help me. Thanks & Regards, Varun Saluja

Re: memtable_allocation_type on Cassandra 2.1.x

2017-05-23 Thread varun saluja
parameters. memtable_flush_writers: memtable_cleanup_threshold: PS : We have high write intensive workload . 5Node cluster (12 Core , 62GB RAM and flash disk per node) Regards, Varun Saluja On 23 May 2017 at 03:26, Akhil Mehra <akhilme...@gmail.com> wrote: > I believe off-hea

memtable_allocation_type on Cassandra 2.1.x

2017-05-22 Thread varun saluja
should be use for below paramteres in 2.1.x memtable_allocation_type : Regards, Varun Saluja - To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Long running compaction on huge hint table.

2017-05-21 Thread varun saluja
Hi All, Can someone Please suggest any recommendations for write intensive jobs Regards, Varun Saluja Sent from my iPhone > On 17-May-2017, at 3:52 PM, varun saluja <saluj...@gmail.com> wrote: > > Thanks Jeff. > > I have taken backup and did manual removal of hints

Re: Long running compaction on huge hint table.

2017-05-21 Thread varun saluja
Thanks a lot Ben. Really appreciate your suggestions here. Regards, Varun Saluja Sent from my iPhone > On 21-May-2017, at 5:40 PM, Ben Slater <ben.sla...@instaclustr.com> wrote: > > My main suggestion would be to monitor the compaction backlog (pending > compaction

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
Hi Nitan, Thanks for response. Yes, I could see mutation drops and increase count in system.hints. Is there any way , i can proceed to truncate hints like using nodetool truncatehints. Regards, Varun Saluja On 16 May 2017 at 17:52, Nitan Kainth <ni...@bamlabs.com> wrote: > D

Re: Long running compaction on huge hint table.

2017-05-16 Thread varun saluja
Compaction system hints 12152557998 869257869352 bytes 1.40% Active compaction remaining time : 0h27m14s Can anything else be checked here? Will manually deleting system.hint files and restart node fix this. Regards, Varun Saluja On 16 May 2017 at 23:29, varun saluja <sa