Re: [ceph-users] mds lost very frequently

2018-12-13 Thread Yan, Zheng
On Fri, Dec 14, 2018 at 12:05 PM Sang, Oliver wrote: > > Thanks a lot, Yan Zheng! > > I enabled only 2 MDS - node1(active) and node2. Then I modified ceph.conf of > node2 to have - > debug_mds = 10/10 > > At 08:35:28, I observed degradation, the node1 was not a MDS any longer and > node2

Re: [ceph-users] EC Pool Disk Performance Toshiba vs Segate

2018-12-13 Thread Ashley Merrick
Hello, Yes 1) and 2) is correct, server provider does their only internal checking before they allow a particular disk model to be used. The two disk models are : TOSHIBA MG06ACA10TEY ST1NM0156-2AA111 They are just using the onboard motherboard SATA3 port's, again only difference between

Re: [ceph-users] disk controller failure

2018-12-13 Thread Christian Balzer
On Thu, 13 Dec 2018 19:44:30 +0100 Ronny Aasen wrote: > On 13.12.2018 18:19, Alex Gorbachev wrote: > > On Thu, Dec 13, 2018 at 10:48 AM Dietmar Rieder > > wrote: > >> Hi Cephers, > >> > >> one of our OSD nodes is experiencing a Disk controller problem/failure > >> (frequent resetting), so the

Re: [ceph-users] cephfs file block size: must it be so big?

2018-12-13 Thread Gregory Farnum
On Thu, Dec 13, 2018 at 3:31 PM Bryan Henderson wrote: > I've searched the ceph-users archives and found no discussion to speak of > of > Cephfs block sizes, and I wonder how much people have thought about it. > > The POSIX 'stat' system call reports for each file a block size, which is >

[ceph-users] cephfs file block size: must it be so big?

2018-12-13 Thread Bryan Henderson
I've searched the ceph-users archives and found no discussion to speak of of Cephfs block sizes, and I wonder how much people have thought about it. The POSIX 'stat' system call reports for each file a block size, which is usually defined vaguely as the smallest read or write size that is

Re: [ceph-users] disk controller failure

2018-12-13 Thread Ronny Aasen
On 13.12.2018 18:19, Alex Gorbachev wrote: On Thu, Dec 13, 2018 at 10:48 AM Dietmar Rieder wrote: Hi Cephers, one of our OSD nodes is experiencing a Disk controller problem/failure (frequent resetting), so the OSDs on this controller are flapping (up/down in/out). I will hopefully get the

Re: [ceph-users] disk controller failure

2018-12-13 Thread Alex Gorbachev
On Thu, Dec 13, 2018 at 10:48 AM Dietmar Rieder wrote: > > Hi Cephers, > > one of our OSD nodes is experiencing a Disk controller problem/failure > (frequent resetting), so the OSDs on this controller are flapping > (up/down in/out). > > I will hopefully get the replacement part soon. > > I have

Re: [ceph-users] disk controller failure

2018-12-13 Thread Matthew Vernon
Hi, On 13/12/2018 16:44, Dietmar Rieder wrote: So you say, that there will be no problem when after the rebalancing I restart the stopped OSDs? I mean the have still the data on them. (Sorry, I just don't like to mess somthing up) It should be fine[0]; when the OSDs come back in ceph will

Re: [ceph-users] disk controller failure

2018-12-13 Thread Dietmar Rieder
Hi Matthew, thanks for your reply and advise, I really appreciate it. So you say, that there will be no problem when after the rebalancing I restart the stopped OSDs? I mean the have still the data on them. (Sorry, I just don't like to mess somthing up) Best Dietmar On 12/13/18 5:11 PM,

Re: [ceph-users] disk controller failure

2018-12-13 Thread Matthew Vernon
Hi, On 13/12/2018 15:48, Dietmar Rieder wrote: one of our OSD nodes is experiencing a Disk controller problem/failure (frequent resetting), so the OSDs on this controller are flapping (up/down in/out). Ah, hardware... I have some simple questions, what are the best steps to take now before

[ceph-users] disk controller failure

2018-12-13 Thread Dietmar Rieder
Hi Cephers, one of our OSD nodes is experiencing a Disk controller problem/failure (frequent resetting), so the OSDs on this controller are flapping (up/down in/out). I will hopefully get the replacement part soon. I have some simple questions, what are the best steps to take now before an

Re: [ceph-users] Slow rbd reads (fast writes) with luminous + bluestore

2018-12-13 Thread Florian Haas
On 13/12/2018 15:10, Mark Nelson wrote: > Hi Florian, > > On 12/13/18 7:52 AM, Florian Haas wrote: >> On 02/12/2018 19:48, Florian Haas wrote: >>> Hi Mark, >>> >>> just taking the liberty to follow up on this one, as I'd really like to >>> get to the bottom of this. >>> >>> On 28/11/2018 16:53,

Re: [ceph-users] Should ceph build against libcurl4 for Ubuntu 18.04 and later?

2018-12-13 Thread Reed Dier
Figured I would chime in as also having this issue. Moving from 16.04 to 18.04 on some OSD nodes. I have been using the ceph apt repo > deb https://download.ceph.com/debian-luminous/ xenial main During the release-upgrade, it can’t find a candidate package, and actually removes the ceph-osd

Re: [ceph-users] Should ceph build against libcurl4 for Ubuntu 18.04 and later?

2018-12-13 Thread Matthew Vernon
Hi, Sorry for the slow reply. On 26/11/2018 17:11, Ken Dreyer wrote: On Thu, Nov 22, 2018 at 11:47 AM Matthew Vernon wrote: On 22/11/2018 13:40, Paul Emmerich wrote: We've encountered the same problem on Debian Buster It looks to me like this could be fixed simply by building the Bionic

Re: [ceph-users] Slow rbd reads (fast writes) with luminous + bluestore

2018-12-13 Thread Mark Nelson
Hi Florian, On 12/13/18 7:52 AM, Florian Haas wrote: On 02/12/2018 19:48, Florian Haas wrote: Hi Mark, just taking the liberty to follow up on this one, as I'd really like to get to the bottom of this. On 28/11/2018 16:53, Florian Haas wrote: On 28/11/2018 15:52, Mark Nelson wrote:

[ceph-users] problem w libvirt version 4.5 and 12.2.7

2018-12-13 Thread Tomasz Płaza
Hi Cephers, After update to CentOS 7.6, libvirt was updated from 3.9 to 4.5. Executing: "virsh vol-list ceph --details" makes libvirtd using 300% CPU for 2 minutes to show volumes on rbd. Quick pick at tcpdump shows accessing rbd_data.* which previous version of libvirtd did not need. Ceph

Re: [ceph-users] Slow rbd reads (fast writes) with luminous + bluestore

2018-12-13 Thread Florian Haas
On 02/12/2018 19:48, Florian Haas wrote: > Hi Mark, > > just taking the liberty to follow up on this one, as I'd really like to > get to the bottom of this. > > On 28/11/2018 16:53, Florian Haas wrote: >> On 28/11/2018 15:52, Mark Nelson wrote: >>> Option("bluestore_default_buffered_read",

Re: [ceph-users] mds lost very frequently

2018-12-13 Thread Yan, Zheng
On Thu, Dec 13, 2018 at 9:25 PM Sang, Oliver wrote: > > Thanks a lot, Yan Zheng! > > Regarding the " set debug_mds =10 for standby mds (change debug_mds to 0 > after mds becomes active)." > Could you please explain the purpose? Just want to collect debug log, or it > really has the side effect

Re: [ceph-users] How to troubleshoot rsync to cephfs via nfs-ganesha stalling

2018-12-13 Thread Marc Roos
I have been having this for some time, it pops up out of the blue. Next time this occurs I will enable the logging. Thanks, Marc -Original Message- From: Daniel Gryniewicz [mailto:d...@redhat.com] Sent: 12 December 2018 16:49 To: Marc Roos; ceph-users Subject: Re: [ceph-users]

Re: [ceph-users] mds lost very frequently

2018-12-13 Thread Sang, Oliver
Thanks a lot, Yan Zheng! Regarding the " set debug_mds =10 for standby mds (change debug_mds to 0 after mds becomes active)." Could you please explain the purpose? Just want to collect debug log, or it really has the side effect to prevent mds lost? Regarding the patch itself. Sorry we didn't

Re: [ceph-users] EC Pool Disk Performance Toshiba vs Segate

2018-12-13 Thread Jake Grimmett
Hi Ashley, Always interesting to see hardware benchmarks :) Do I understand the following correctly? 1) your host (server provider) rates the Toshiba drives as faster 2) Ceph osd perf rates the Seagate drives as faster Could you share the benchmark output and drive model numbers? Presumably

[ceph-users] ceph remote disaster recovery plan

2018-12-13 Thread Zhenshi Zhou
Hi all, I'm running a luminous cluster with tens of OSDs and the cluster runs well. As the data grows, ceph becomes more and more important. What worries me is that many services will down if the cluster is out, for instance, the engine room is out of electric or all ceph node are down at the

Re: [ceph-users] EC Pool Disk Performance Toshiba vs Segate

2018-12-13 Thread Maged Mokhtar
On 13/12/2018 09:53, Ashley Merrick wrote: I have a Mimic Bluestore EC RBD Pool running on 8+2, this is currently running across 4 node's. 3 Node's are running Toshiba disk's while one node is running Segate disks (same size, spinning speed, enterprise disks e.t.c), I have noticed huge

Re: [ceph-users] EC Pool Disk Performance Toshiba vs Segate

2018-12-13 Thread Ashley Merrick
Hello, Thanks for your reply!, while the performance within CEPH is different the two disk's are exactly the same in rated performance / type e.t.c just from two different manufacturers. Obviously id expected such a big difference between 5200 and 7200RPM or SMR and CMR for example but these are