Re: [ceph-users] Slow OPS

2019-03-21 Thread Brad Hubbard
A repop is a sub-operation between primaries and replicas mostly. That op only shows a duration of 1.3 seconds and the delay you mentioned previously was under a second. Do you see larger delays? Are they always between "sub_op_committed" and "commit_sent"? What is your workload and how heavily

Re: [ceph-users] Access cephfs from second public network

2019-03-21 Thread Paul Emmerich
clients also need to access the OSDs and MDS servers Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Thu, Mar 21, 2019 at 1:02 PM Andres Rojas Guerrero wrote: > >

Re: [ceph-users] 答复: CEPH ISCSI LIO multipath change delay

2019-03-21 Thread Mike Christie
On 03/21/2019 11:27 AM, Maged Mokhtar wrote: > > Though i do not recommend changing it, if there is a need to lower > fast_io_fail_tmo, then osd_heartbeat_interval + osd_heartbeat_grace sum > need to be lowered as well, their default sum is 25 sec, which i would > assume why fast_io_fail_tmo is

Re: [ceph-users] objects degraded higher than 100%

2019-03-21 Thread Simon Ironside
This behaviour is still an issue in mimic 13.2.5 and nautilus 14.2.0. I've logged https://tracker.ceph.com/issues/38841 for this. Apologies if this has already been done. Simon On 06/03/2019 20:17, Simon Ironside wrote: Yes, as I said that bug is marked resolved. It's also marked as only

Re: [ceph-users] 答复: CEPH ISCSI LIO multipath change delay

2019-03-21 Thread Maged Mokhtar
Though i do not recommend changing it, if there is a need to lower fast_io_fail_tmo, then osd_heartbeat_interval + osd_heartbeat_grace sum need to be lowered as well, their default sum is 25 sec, which i would assume why fast_io_fail_tmo is set to this.  you would want to have your higher

Re: [ceph-users] ceph-volume lvm batch OSD replacement

2019-03-21 Thread Dan van der Ster
On Thu, Mar 21, 2019 at 12:14 PM Eugen Block wrote: > > Hi Dan, > > I don't know about keeping the osd-id but I just partially recreated > your scenario. I wiped one OSD and recreated it. You are trying to > re-use the existing block.db-LV with the device path (--block.db > /dev/vg-name/lv-name)

Re: [ceph-users] cephfs manila snapshots best practices

2019-03-21 Thread Tom Barron
On 21/03/19 16:15 +0100, Dan van der Ster wrote: On Thu, Mar 21, 2019 at 1:50 PM Tom Barron wrote: On 20/03/19 16:33 +0100, Dan van der Ster wrote: >Hi all, > >We're currently upgrading our cephfs (managed by OpenStack Manila) >clusters to Mimic, and want to start enabling snapshots of the

Re: [ceph-users] cephfs manila snapshots best practices

2019-03-21 Thread Dan van der Ster
On Thu, Mar 21, 2019 at 1:50 PM Tom Barron wrote: > > On 20/03/19 16:33 +0100, Dan van der Ster wrote: > >Hi all, > > > >We're currently upgrading our cephfs (managed by OpenStack Manila) > >clusters to Mimic, and want to start enabling snapshots of the file > >shares. > >There are different ways

Re: [ceph-users] 答复: CEPH ISCSI LIO multipath change delay

2019-03-21 Thread Jason Dillaman
It's just the design of the iSCSI protocol. Sure, you can lower the timeouts (see "fast_io_fail_tmo" [1]) but you will just end up w/ more false-positive failovers. [1] http://docs.ceph.com/docs/master/rbd/iscsi-initiator-linux/ On Thu, Mar 21, 2019 at 10:46 AM li jerry wrote: > > Hi Maged > >

[ceph-users] 答复: CEPH ISCSI LIO multipath change delay

2019-03-21 Thread li jerry
Hi Maged thank you for your reply. To exclude the osd_heartbeat_interval and osd_heartbeat_grace factors, I cleared the current lio configuration, redeployed two CENTOS7 (not in any ceph role), and deployed rbd-target-api, rbd-target-gw, trum-runner on it. ; And do the following test 1.

Re: [ceph-users] OMAP size on disk

2019-03-21 Thread Brent Kennedy
They released Luminous 12.2.11 and now my large objects are starting to count down ( after running the rm command suggested in the release notes ). Seems dynamic sharding will clean up after itself now to! So case closed! -Brent -Original Message- From: ceph-users On Behalf Of Brent

Re: [ceph-users] cephfs manila snapshots best practices

2019-03-21 Thread Tom Barron
On 20/03/19 16:33 +0100, Dan van der Ster wrote: Hi all, We're currently upgrading our cephfs (managed by OpenStack Manila) clusters to Mimic, and want to start enabling snapshots of the file shares. There are different ways to approach this, and I hope someone can share their experiences with:

[ceph-users] Access cephfs from second public network

2019-03-21 Thread Andres Rojas Guerrero
Hi all, we have deployed a Ceph cluster configured with two public networks: [global] cluster network = 10.100.188.0/23 fsid = 88f62260-b8de-499f-b6fe-5eb66a967083 mon host = 10.100.190.9,10.100.190.10,10.100.190.11 mon initial members = mon1,mon2,mon3 osd_pool_default_pg_num = 4096 public

Re: [ceph-users] ceph-volume lvm batch OSD replacement

2019-03-21 Thread Eugen Block
Hi Dan, I don't know about keeping the osd-id but I just partially recreated your scenario. I wiped one OSD and recreated it. You are trying to re-use the existing block.db-LV with the device path (--block.db /dev/vg-name/lv-name) instead the lv notation (--block.db vg-name/lv-name):

Re: [ceph-users] v14.2.0 Nautilus released

2019-03-21 Thread John Hearns
Martin, my thanks to Croit for making this repository available. I have been building Ceph from source on Ubuntu Cosmic for the last few days. It is much more convenient to use a repo. On Thu, 21 Mar 2019 at 09:32, Martin Verges wrote: > Hello, > > we strongly believe it would be good for Ceph

Re: [ceph-users] v14.2.0 Nautilus released

2019-03-21 Thread Martin Verges
Hello, we strongly believe it would be good for Ceph to have the packaged directly on the official Debian mirrors, but for everyone out there having trouble with Ceph on Debian we are glad to help. If Ceph is not available on Debian, it might affect a lot of other Software, for example Proxmox.

[ceph-users] When to use a separate RocksDB SSD

2019-03-21 Thread Glen Baars
Hello Ceph, What is the best way to find out how the RocksDB is currently performing? I need to build a business case for NVME devices for RocksDB. Kind regards, Glen Baars This e-mail is intended solely for the benefit of the addressee(s) and any other named recipient. It is confidential and

Re: [ceph-users] CephFS: effects of using hard links

2019-03-21 Thread Dan van der Ster
On Thu, Mar 21, 2019 at 8:51 AM Gregory Farnum wrote: > > On Wed, Mar 20, 2019 at 6:06 PM Dan van der Ster wrote: >> >> On Tue, Mar 19, 2019 at 9:43 AM Erwin Bogaard >> wrote: >> > >> > Hi, >> > >> > >> > >> > For a number of application we use, there is a lot of file duplication. >> > This

Re: [ceph-users] SSD Recovery Settings

2019-03-21 Thread Marc Roos
Beware of start using this influx. I have 80GB db, and I regret using it. I have to move now to storing data in graphite. Collectd has also a plugin for that. Influx cannot downsample properly when having tags I think (still wait for a response to this [0]) What I have understood is that

Re: [ceph-users] CephFS: effects of using hard links

2019-03-21 Thread Gregory Farnum
On Wed, Mar 20, 2019 at 6:06 PM Dan van der Ster wrote: > On Tue, Mar 19, 2019 at 9:43 AM Erwin Bogaard > wrote: > > > > Hi, > > > > > > > > For a number of application we use, there is a lot of file duplication. > This wastes precious storage space, which I would like to avoid. > > > > When

Re: [ceph-users] Slow OPS

2019-03-21 Thread Glen Baars
Hello Brad, It doesn't seem to be a set of OSDs, the cluster has 160ish OSDs over 9 hosts. I seem to get a lot of these ops also that don't show a client. "description": "osd_repop(client.14349712.0:4866968 15.36 e30675/22264 15:6dd17247:::rbd_data.2359ef6b8b4567.0042766