RE: about memory problem in write heavy system..

2022-01-11 Thread Durity, Sean R
. Granted, it is not fun to deal with tight space on a Cassandra cluster. Sean R. Durity From: Bowen Song Sent: Tuesday, January 11, 2022 6:50 AM To: user@cassandra.apache.org Subject: [EXTERNAL] Re: about memory problem in write heavy system.. You don't really need 50% of free disk space

Re: about memory problem in write heavy system..

2022-01-11 Thread Bowen Song
You don't really need 50% of free disk space available if you don't keep backups and snapshots on the same server. The DataStax guide recommends 50% free space because it recommands you to take a snapshot (which is implemented as filesystem hardlink) before upgrading. If you don't have 50%

Re: about memory problem in write heavy system..

2022-01-10 Thread Eunsu Kim
Thank you Bowen. As can be seen from the chart, the memory of existing nodes has increased since new nodes were added. And I stopped writing a specific table. Write throughput decreased by about 15%. And memory usage began to decrease. I'm not sure if this was done by natural resolution or by

Re: about memory problem in write heavy system..

2022-01-07 Thread daemeon reiydelle
Maybe SSD's? Take a look at the IO read/write wait times. FYI, your config changes simply push more activity into memory. Trading IO for mem footprint ;{) *Daemeon Reiydelle* *email: daeme...@gmail.com * *San Francisco 1.415.501.0198/Skype daemeon.c.m.reiydelle* Cognitive Bias: (written in

Re: about memory problem in write heavy system..

2022-01-07 Thread Jeff Jirsa
3.11.4 is a very old release, with lots of known bugs. It's possible the memory is related to that. If you bounce one of the old nodes, where does the memory end up? On Thu, Jan 6, 2022 at 3:44 PM Eunsu Kim wrote: > > Looking at the memory usage chart, it seems that the physical memory usage

Re: about memory problem in write heavy system..

2022-01-06 Thread Eunsu Kim
Looking at the memory usage chart, it seems that the physical memory usage of the existing node has increased since the new node was added with auto_bootstrap=false. > > On Fri, Jan 7, 2022 at 1:11 AM Eunsu Kim > wrote: > Hi, > > I have a Cassandra

about memory problem in write heavy system..

2022-01-06 Thread Eunsu Kim
Hi, I have a Cassandra cluster(3.11.4) that does heavy writing work. (14k~16k write throughput per second per node) Nodes are physical machine in data center. Number of nodes are 30. Each node has three data disks mounted. A few days ago, a QueryTimeout problem occurred due to Full GC. So,