[ceph-users] Re: OSDs do not respect my memory tune limit

2022-12-02 Thread Daniel Brunner
Thanks for the hint, I tried turning that off: $ sudo ceph osd pool get cephfs_data pg_autoscale_mode pg_autoscale_mode: on $ sudo ceph osd pool set cephfs_data pg_autoscale_mode off set pool 9 pg_autoscale_mode to off $ sudo ceph osd pool get cephfs_data pg_autoscale_mode pg_autoscale_mode: off

[ceph-users] Re: OSDs do not respect my memory tune limit

2022-12-02 Thread Daniel Brunner
Can I get rid of PGs after trying to decrease the number on the pool again? Doing a backup and nuking the cluster seems a little too much work for me :) $ sudo ceph osd pool get cephfs_data pg_num pg_num: 128 $ sudo ceph osd pool set cephfs_data pg_num 16 $ sudo ceph osd pool get cephfs_data

[ceph-users] Re: OSDs do not respect my memory tune limit

2022-12-02 Thread Janne Johansson
> my OSDs are running odroid-hc4's and they only have about 4GB of memory, > and every 10 minutes a random OSD crashes due to out of memory. Sadly the > whole machine gets unresponsive when the memory gets completely full, so no > ssh access or prometheus output in the meantime. > I've set the

[ceph-users] Re: OSDs do not respect my memory tune limit

2022-12-02 Thread Daniel Brunner
for example on of my latest osd crashes looks like this in dmesg: [Dec 2 08:26] bstore_mempool invoked oom-killer: gfp_mask=0x24200ca(GFP_HIGHUSER_MOVABLE), nodemask=0, order=0, oom_score_adj=0 [ +0.06] bstore_mempool cpuset=ed46e6fa52c1e40f13389b349c54e62dcc8c65d76c4c7860e2ff7c39444d14cc