[ceph-users] PG count deviation alert on OSDs of high weight

2022-01-26 Thread Nicola Mori
I set up a test cluster (Pacific 16.2.7 deployed with cephadm) with several hdds of different sizes, 1.8 Tb and 3.6 TB; they have weight 1.8 and 3.6, respectively, with 2 pools (metadata+data for CephFS). I'm currently having a PG count varying from 177 to 182 for OSDs with small disks and

[ceph-users] Re: Monitoring ceph cluster

2022-01-26 Thread Anthony D'Atri
What David said! A couple of additional thoughts: o Nagios (and derivatives like Icinga and check_mk) have been popular for years. Note that they’re monitoring solutions vs metrics solutions — it’s good to have both. One issue I’ve seen multiple times with Nagios-family monitoring is that

[ceph-users] CephFS Snapshot Scheduling stops creating Snapshots after a restart of the Manager

2022-01-26 Thread Sebastian Mazza
I have a problem with the snap_schedule MGR module. It seams to forget at least parts of the configuration after the active MGR is restarted. The following cli commands (lines starting with ‘$’) and their std out (lines starting with >) demonstrates the problem. $ ceph fs snap-schedule add

[ceph-users] Re: Monitoring ceph cluster

2022-01-26 Thread David Orman
What version of Ceph are you using? Newer versions deploy a dashboard and prometheus module, which has some of this built in. It's a great start to seeing what can be done using Prometheus and the built in exporter. Once you learn this, if you decide you want something more robust, you can do an

[ceph-users] Do not use VMware Storage I/O Control with Ceph iSCSI GWs!

2022-01-26 Thread Frédéric Nass
Hi, For anyone using VMware ESXi (6.7) with Ceph iSCSI GWs (Nautilus), I thought you might benefit from our experience: I have finally identified what was causing a permanent ~500 MB/s and ~4k iops load on our cluster, specifically on one of our RBD image used as a VMware Datastore and it

[ceph-users] Re: Moving all s3 objects from an ec pool to a replicated pool using storage classes.

2022-01-26 Thread Irek Fasikhov
Hi. Basic logic: 1.bucket policy transition 2.radosgw-admin gc process --include-all 3.1.rados ls -p pool | grep >bucket_objects.txt 3.2.rados listxattr -p pool objname | xargs -L1 echo rados getattr -p pool objname >> objname.txt 3.3.rados create -p pool objname 3.4.cat objname.txt | xargs -L1

[ceph-users] Different OSD file structure

2022-01-26 Thread Zoth
I've got a cluster with different OSD structures, some are updated to 15.2.12 and the others are 15.2.9 (bluestore). No problem so far with the cluster, but I think it's better to normalize the situation. *15.2.9* drwxr-xr-x 23 ceph ceph 4096 Nov 30 15:50 ../ lrwxrwxrwx  1 ceph ceph   24 Nov

[ceph-users] Re: Is it possible to stripe rados object?

2022-01-26 Thread Sebastian Wagner
libradosstriper ? Am 26.01.22 um 10:16 schrieb lin yunfan: > Hi, > I know with rbd and cephfs there is a stripe setting to stripe data > into multiple rodos object. > Is it possible to use librados api to stripe a large object into many > small ones? > > linyunfan >

[ceph-users] Re: switch restart facilitating cluster/client network.

2022-01-26 Thread Marc
Thanks for the tips!!! > > I would still set noout on relevant parts of the cluster in case something > goes south and it does take longer than 2 minutes. Otherwise OSDs will > start outing themselves after 10 minutes or so by default and then you > have a lot of churn going on. > > The

[ceph-users] Re: How to remove stuck daemon?

2022-01-26 Thread Fyodor Ustinov
Hi! I restarted mgr - it didn't help. Or do you mean something else? > Hi, > > have you tried to failover the mgr service? I noticed similar > behaviour in Octopus. > > > Zitat von Fyodor Ustinov : > >> Hi! >> >> No one knows how to fix it? >> >> >> - Original Message - >>> From:

[ceph-users] Is it possible to stripe rados object?

2022-01-26 Thread lin yunfan
Hi, I know with rbd and cephfs there is a stripe setting to stripe data into multiple rodos object. Is it possible to use librados api to stripe a large object into many small ones? linyunfan ___ ceph-users mailing list -- ceph-users@ceph.io To

[ceph-users] Limitations of ceph fs snapshot mirror for read-only folders?

2022-01-26 Thread Manuel Holtgrewe
Dear all, I want to mirror a snapshot in Ceph v16.2.6 deployed with cephadm using the stock quay.io images. My source file system has a folder "/src/folder/x" where "/src/folder" has mode "ug=r,o=", in other words no write permissions for the owner (root). The sync of a snapshot "initial" now

[ceph-users] Re: How to remove stuck daemon?

2022-01-26 Thread Eugen Block
Hi, have you tried to failover the mgr service? I noticed similar behaviour in Octopus. Zitat von Fyodor Ustinov : Hi! No one knows how to fix it? - Original Message - From: "Fyodor Ustinov" To: "ceph-users" Sent: Tuesday, 25 January, 2022 11:29:53 Subject: [ceph-users] How

[ceph-users] Re: How to remove stuck daemon?

2022-01-26 Thread Fyodor Ustinov
Hi! No one knows how to fix it? - Original Message - > From: "Fyodor Ustinov" > To: "ceph-users" > Sent: Tuesday, 25 January, 2022 11:29:53 > Subject: [ceph-users] How to remove stuck daemon? > Hi! > > I have Ceph cluster version 16.2.7 with this error: > > root@s-26-9-19-mon-m1:~#