Re: [ceph-users] pg count question

The % should be based on how much of the storage you expect that pool to take up out of total available. 256PGs with Replication 3 will distribute themselves as 256 * 3 / 14 which will be about 54 per OSD. For the smaller pool 16 seems too low. You can go with 32 and 256 if you want lower number

[ceph-users] Luminous upgrade instructions include bad commands

[root@monitor07]# ceph version ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable) [root@monitor07]# ceph mon feature ls no valid command found; 10 closest matches: mon compact mon scrub mon metadata {} mon sync force {--yes-i-really-mean-it} {--i-know-what-i-am-doing}

Re: [ceph-users] [Ceph-deploy] Cluster Name

Just FYI. I asked about cluster names a month or two back and was told that support for them is being phased out. I've had all sorts of problems using clusters with cluster names, and stopped using it myself. On Fri, Aug 10, 2018 at 2:06 AM, Glen Baars wrote: > I have now gotten this

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

Looks like you got some duplicate inodes due to corrupted metadata, you likely tried to a disaster recovery and didn't follow through it completely or you hit some bug in Ceph. The solution here is probably to do a full recovery of the metadata/full backwards scan after resetting the inodes. I've

Re: [ceph-users] RBD image "lightweight snapshots"

On Fri, Aug 10, 2018 at 4:53 AM, Paweł Sadowsk wrote: > On 08/09/2018 04:39 PM, Alex Elder wrote: >> On 08/09/2018 08:15 AM, Sage Weil wrote: >>> On Thu, 9 Aug 2018, Piotr Dałek wrote: Hello, At OVH we're heavily utilizing snapshots for our backup system. We think there's an

Re: [ceph-users] Running 12.2.5 without problems, should I upgrade to 12.2.7 or wait for 12.2.8?

Am Fr., 10. Aug. 2018 um 19:29 Uhr schrieb : > > > Am 30. Juli 2018 09:51:23 MESZ schrieb Micha Krause : > >Hi, > > Hi Micha, > > > > >I'm Running 12.2.5 and I have no Problems at the moment. > > > >However my servers reporting daily that they want to upgrade to 12.2.7, > >is this save or should

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

Thanks for the info , paul. Our cluster is 130gb in size, at present. We are starting out in ceph adoption in our company. At present, I am looking for guidance from the community. It ll help us, as well in learning more about the product and available support. Thanks, On Fri, 10 Aug 2018

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

As Paul said, the MDS is loading "duplicate inodes" and that's very bad. If you've already gone through some of the disaster recovery steps, that's likely the cause. But you'll need to provide a *lot* more information about what you've already done to the cluster for people to be sure. The

Re: [ceph-users] Applicability and migration path

On Fri, Aug 10, 2018 at 8:25 AM Burkhard Linke < burkhard.li...@computational.bio.uni-giessen.de> wrote: > Hi, > > > On 08/10/2018 03:10 PM, Matthew Pounsett wrote: > > *snipsnap* > >> advisable to put these databases on SSDs. You can share one SSD for > several > >> OSDs (e.g. by creating

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

Thanks alot, Paul. we did (hopefully) follow through with the disaster recovery. however, please guide me in how to get the cluster back up ! Thanks, On Fri, Aug 10, 2018 at 9:32 PM Paul Emmerich wrote: > Looks like you got some duplicate inodes due to corrupted metadata, you > likely tried

Re: [ceph-users] bad crc/signature errors

I've built a work-around here: https://github.com/ceph/ceph/pull/23273 Paul 2018-08-10 12:51 GMT+02:00 Nikola Ciprich : > Hi, > > did this ever come to some conclusion? I've recently started seeing > those messages on one luminous cluster and am not sure whethere > those are dangerous or not..

Re: [ceph-users] Applicability and migration path

Hi, On 08/10/2018 03:10 PM, Matthew Pounsett wrote: *snipsnap* advisable to put these databases on SSDs. You can share one SSD for several OSDs (e.g. by creating partitions), but keep in mind that the failure of one of these SSDs also renders the OSD content useless. Do not use consumer grade

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

Sorry, a step-by-step guide through something like that is beyond the scope of what we can do on a mailing list. But what I would do here is carefully asses the situation/ the damage. My wild guess would be to reset and rebuild the inode table but that might be incorrect and unsafe without

Re: [ceph-users] Running 12.2.5 without problems, should I upgrade to 12.2.7 or wait for 12.2.8?

Am 30. Juli 2018 09:51:23 MESZ schrieb Micha Krause : >Hi, Hi Micha, > >I'm Running 12.2.5 and I have no Problems at the moment. > >However my servers reporting daily that they want to upgrade to 12.2.7, >is this save or should I wait for 12.2.8? > I guess you should Upgrade to 12.2.7 as soon

Re: [ceph-users] pg count question

Folks, I used your link to calculate PGs and i did following. Total OSD: 14 Replica: 3 Total Pools: 2 ( Images & vms) In %Data i gave 5% to images & 95% to vms (openstack) https://ceph.com/pgcalc/ It gave me following result vms - 512 PG images - 16 PG For safe side i set vms 256 PG is

Re: [ceph-users] removing auids and auid-based cephx capabilities

On Wed, Aug 8, 2018 at 1:33 PM, Sage Weil wrote: > There is an undocumented part of the cephx authentication framework called > the 'auid' (auth uid) that assigns an integer identifier to cephx users > and to rados pools and allows you to craft cephx capabilities that apply > to those pools.

Re: [ceph-users] Luminous upgrade instructions include bad commands

Your client is too old and doesn't know about these commands, check with "ceph -v" Paul 2018-08-11 1:11 GMT+02:00 Robert Stanford : > > [root@monitor07]# ceph version > ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous > (stable) > [root@monitor07]# ceph mon feature ls >

Re: [ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

Thanks for the response, gregory. We need to support a couple of production services we have migrated to ceph. So we are in a bit of soup. cluster is as follows: ``` ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 11.06848 root default -75.45799

Re: [ceph-users] RBD image "lightweight snapshots"

On 08/10/2018 06:24 PM, Gregory Farnum wrote: On Fri, Aug 10, 2018 at 4:53 AM, Paweł Sadowsk wrote: On 08/09/2018 04:39 PM, Alex Elder wrote: On 08/09/2018 08:15 AM, Sage Weil wrote: On Thu, 9 Aug 2018, Piotr Dałek wrote: Hello, At OVH we're heavily utilizing snapshots for our backup

Re: [ceph-users] pg count question

Thanks! Subhachandra On Fri, Aug 10, 2018 at 6:01 PM, Subhachandra Chandra wrote: > The % should be based on how much of the storage you expect that pool to > take up out of total available. 256PGs with Replication 3 will distribute > themselves as 256 * 3 / 14 which will be about 54 per OSD.

Re: [ceph-users] osd.X down, but it is still running on Luminous

Thanks for the reply. I don’t see “Segmentation fault” in logs ( On 10 Aug 2018, at 09:35, Eugen Block mailto:ebl...@nde.ag>> wrote: Hi, could you be hitting the bug from [1]? Watch out for segfaults in dmesg. Since a couple of days we see random OSDs with a segfault from safe_timer. We

Re: [ceph-users] understanding pool capacity and usage

Den fre 27 juli 2018 kl 12:24 skrev Anton Aleksandrov : > Hello, > > Might sounds strange, but I could not find answer in google or docs, might > be called somehow else. > > I dont understand pool capacity policy and how to set/define it. I have > created simple cluster for CephFS on 4 servers,

[ceph-users] RBD journal feature

Hello Ceph Users, I am trying to implement image journals for our RBD images ( required for mirroring ) rbd feature enable SLOWPOOL/RBDImage journaling --journal-pool SSDPOOL When we run the above command we still find the journal on the SLOWPOOL and not on the SSDPOOL. We are running 12.2.7

Re: [ceph-users] osd.X down, but it is still running on Luminous

Hi, could you be hitting the bug from [1]? Watch out for segfaults in dmesg. Since a couple of days we see random OSDs with a segfault from safe_timer. We didn't update any packages for months. Regards [1] https://tracker.ceph.com/issues/23352 Zitat von Rudenko Aleksandr : Hi, guys.

Re: [ceph-users] [Ceph-deploy] Cluster Name

I have now gotten this working. Thanks everyone for the help. The RBD-Mirror service is co-located on a MON server. Key points are: Start the services on the boxes with the following syntax ( depending on your config file names ) On primary systemctl start ceph-rbd-mirror@primary On

Re: [ceph-users] Applicability and migration path

Wow.. thanks for such a detailed reply! On 10 August 2018 at 07:08, Burkhard Linke < burkhard.li...@computational.bio.uni-giessen.de> wrote: > The default ceph setup uses 3 replicates on three different hosts, so you > need at least three hosts for a ceph cluster. Other configurations with a >

Re: [ceph-users] Applicability and migration path

On 10 August 2018 at 07:09, Janne Johansson wrote: > > As opposed to the previous setup, this will add some inter-host traffic > aswell, each write to the primary PG will then in turn cause that host to > replicate it > again over the network to X other hosts to form the required amount of >

Re: [ceph-users] pg count question

Re-sending it, because i found my i lost membership so wanted to make sure, my email went through On Fri, Aug 10, 2018 at 7:07 AM, Satish Patel wrote: > Thanks, > > Can you explain about %Data field in that calculation, is this total data > usage for specific pool or total ? > > For example > >

Re: [ceph-users] RBD image "lightweight snapshots"

On Fri, 10 Aug 2018, Paweł Sadowski wrote: > On 08/09/2018 04:39 PM, Alex Elder wrote: > > On 08/09/2018 08:15 AM, Sage Weil wrote: > >> On Thu, 9 Aug 2018, Piotr Dałek wrote: > >>> Hello, > >>> > >>> At OVH we're heavily utilizing snapshots for our backup system. We think > >>> there's an

Re: [ceph-users] Applicability and migration path

Den fre 10 aug. 2018 kl 04:33 skrev Matthew Pounsett : > > First, in my tests and reading I haven't encountered anything that > suggests I should expect problems from using a small number of large file > servers in a cluster. But I recognize that this isn't the preferred > configuration, and I'm

Re: [ceph-users] questions about rbd used percentage

You can halve the time by running "rbd du" once, keep the output and run the grep over the output instead. Den tors 2 aug. 2018 kl 12:53 skrev : > Hi! > > I want to monitor rbd image size to enable enlager size when > use percentage above 80%. > > I find a way with `rbd du`: > > total=$(rbd du

Re: [ceph-users] Applicability and migration path

Hi, just some thoughts and comments: Hardware: The default ceph setup uses 3 replicates on three different hosts, so you need at least three hosts for a ceph cluster. Other configurations with a smaller number of hosts are possible, but not recommended. Depending on the workload and

[ceph-users] ceph mds crashing constantly : ceph_assert fail … prepare_new_inode

We are facing constant crash from ceph mds. We have installed mimic (v13.2.1). mds: cephfs-1/1/1 up {0=node2=up:active(laggy or crashed)} *mds logs: https://pastebin.com/AWGMLRm0 * we have followed the DR steps listed at

Re: [ceph-users] Secure way to wipe a Ceph cluster

Den fre 27 juli 2018 kl 21:20 skrev Patrick Donnelly : > > > as part of deprovisioning customers, we regularly have the task of > > wiping their Ceph clusters. Is there a certifiable, GDPR compliant way > > to do so without physically shredding the disks? > > This should work and should be as

[ceph-users] Make a ceph options persist

Hi, The manual of dashboard suggests: ceph config-key set mgr/dashboard/server_addr ${MGR_IP} But that command is required after reboot. I have tried all kinds of versions, but was not able to get it working... How do I put this into a permanent version in /etc/ceph/ceph.conf --WjW

Re: [ceph-users] Re : Re : Re : bad crc/signature errors

Hi, did this ever come to some conclusion? I've recently started seeing those messages on one luminous cluster and am not sure whethere those are dangerous or not.. BR nik On Fri, Oct 06, 2017 at 05:37:00PM +0200, Olivier Bonvalet wrote: > Le jeudi 05 octobre 2017 à 21:52 +0200, Ilya Dryomov

Re: [ceph-users] pg count question

Thanks, Can you explain about %Data field in that calculation, is this total data usage for specific pool or total ? For example Pool-1 is small so should I use 20% Pool-2 is bigger so I should use 80% I'm confused there so can you give me just example how to calculate that field? Sent

[ceph-users] RBD image "lightweight snapshots"

On 08/09/2018 04:39 PM, Alex Elder wrote: > On 08/09/2018 08:15 AM, Sage Weil wrote: >> On Thu, 9 Aug 2018, Piotr Dałek wrote: >>> Hello, >>> >>> At OVH we're heavily utilizing snapshots for our backup system. We think >>> there's an interesting optimization opportunity regarding snapshots I'd

Re: [ceph-users] cephmetrics without ansible

Hi, I figure it out In case someone else is interested, here are some concise instructions on grafana server enter correct server names and DNS domain in dasboard.yml and update it "python dashUpdater.py" on collectd servers ( OSD, MON) mkdir -p /usr/lib64/collectd/python-plugins/collectors