Re: [ceph-users] Intel S3710 400GB and Samsung PM863 480GB fio results

On 12/21/2015 05:30 PM, Lionel Bouton wrote: > Hi, > > Sébastien Han just added the test results I reported for these SSDs on > the following page : > > http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/ > > The table in the original post

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

I'd be curious to compare benchmarks. What size objects are you putting? 10gig end to end from client to RGW server to OSDs? I wouldn't be surprised if mine is pretty slow though in comparison, since we still don't have SSD journals. So I have not paid much attention to upload speed. Our omap

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Mon, Dec 21, 2015 at 4:15 PM, Haomai Wang wrote: > > > On Mon, Dec 21, 2015 at 10:55 PM, Florian Haas wrote: >> >> On Mon, Dec 21, 2015 at 3:35 PM, Haomai Wang wrote: >> > >> > >> > On Fri, Dec 18, 2015 at 1:16 AM, Florian Haas

[ceph-users] incomplete pg, and some mess

hi every one, we some how got our cluster really messed up. we had one node down du to system disk failing. while we were working to bring it back we had few osds crashing, they kept crashing, so we stopped them. that would be a story for another thread though. now

Re: [ceph-users] OSDs stuck in booting state on CentOS 7.2.1511 and ceph infernalis 9.2.0

Bryan, Once the rest of the cluster was updated to v0.94.5 it now appears the one host running infernalis v9.2.0 OSDs are booting. Bob On Fri, Dec 18, 2015 at 3:44 PM, Bob R wrote: > Bryan, > > I rebooted another host which wasn't updated to CentOS 7.2 and those OSDs >

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Mon, Dec 21, 2015 at 9:15 PM, Ben Hines wrote: > I'd be curious to compare benchmarks. What size objects are you putting? As stated earlier, I ran rest-bench with 70KB objects which is a good approximation of the average object size in the underperforming system. > 10gig

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Thu, Dec 17, 2015 at 6:16 PM, Florian Haas wrote: > Hey everyone, > > I recently got my hands on a cluster that has been underperforming in > terms of radosgw throughput, averaging about 60 PUTs/s with 70K > objects where a freshly-installed cluster with near-identical >

Re: [ceph-users] Infernalis MDS crash (debug log included)

On Mon, Dec 21, 2015 at 10:15 AM, Florent B wrote: > Hi all, > > It seems I had an MDS crash being in standby-replay. > > Version is Infernalis, running on Debian Jessie (packaged version). > > Log is here (2.5MB) : http://paste.ubuntu.com/14126366/ > > Has someone

[ceph-users] 回复：Re: SSD only pool without journal

If it is true,why we still have no journal mode in the code? Thanks发自网易邮箱大师 在2015年12月18日 00:43，Loris Cuoghi 写道:Le 17/12/2015 16:47, Misa a écrit : > Hello everyone, > > does it make sense to create SSD only pool from OSDs without journal? > >  From my point of

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

resend On Mon, Dec 21, 2015 at 10:35 PM, Haomai Wang wrote: > > > On Fri, Dec 18, 2015 at 1:16 AM, Florian Haas wrote: > >> Hey everyone, >> >> I recently got my hands on a cluster that has been underperforming in >> terms of radosgw throughput, averaging

Re: [ceph-users] radosgw bucket index sharding tips?

On Mon, Dec 21, 2015 at 10:20 AM, Wido den Hollander wrote: >>> > Oh, and to answer this part. I didn't do that much experimentation >>> > unfortunately. I actually am using about 24 index shards per bucket >>> > currently and we delete each bucket once it hits about a million

Re: [ceph-users] rbd image mount on multiple clients

Hello Dan, please read here (i've just read it myself a few days ago): http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-May/021351.html Long story short: yes, you can mount it, but to be able to do reads/writes one has to use cluster aware filesystem on top of a block device. That

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Tue, Dec 22, 2015 at 3:33 AM, Florian Haas wrote: > On Mon, Dec 21, 2015 at 4:15 PM, Haomai Wang wrote: > > > > > > On Mon, Dec 21, 2015 at 10:55 PM, Florian Haas > wrote: > >> > >> On Mon, Dec 21, 2015 at 3:35 PM, Haomai Wang

Re: [ceph-users] cephfs, low performances

On Mon, Dec 21, 2015 at 11:46 PM, Don Waterloo wrote: > On 20 December 2015 at 22:47, Yan, Zheng wrote: >> >> >> --- >> >> >> >> >> fio tests AIO performance in this case. cephfs does not

Re: [ceph-users] [SOLVED] Monitor rename / recreate issue -- probing state

Hello Joao, Thanks for your help.I increased logging on the failed monitor and noticed a lot of cephx authentication errors. After verifying ntp sync, I noticed that the monitor keyring deployed on working monitors differed from what was stored in the management server’s

[ceph-users] Ceph armhf package updates

Hello all, i seem to have a problem with the ceph version available at ports.ubuntu.com in the armhf branch. The latest available version is now infernalis 9.2, however, whenever i try to update my system, i still get the hammer version (0.94.5). I've been checking everyday, and it seems the

Re: [ceph-users] radosgw bucket index sharding tips?

On 12/17/2015 05:27 PM, Florian Haas wrote: > Hey Wido, > > On Dec 17, 2015 09:52, "Wido den Hollander" > wrote: >> >> On 12/17/2015 06:29 AM, Ben Hines wrote: >> > >> > >> > On Wed, Dec 16, 2015 at 11:05 AM, Florian Haas

Re: [ceph-users] Infernalis MDS crash (debug log included)

On Mon, Dec 21, 2015 at 5:15 PM, Florent B wrote: > Hi all, > > It seems I had an MDS crash being in standby-replay. > > Version is Infernalis, running on Debian Jessie (packaged version). > > Log is here (2.5MB) : http://paste.ubuntu.com/14126366/ > > Has someone information

[ceph-users] rbd image mount on multiple clients

Hi, Is it possible/safe to mount a rbd image and use it on multiple clients will this break anything ? Thanks, Dan ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] cephfs 'lag' / hang

On Sat, Dec 19, 2015 at 4:34 AM, Don Waterloo wrote: > I have 3 systems w/ a cephfs mounted on them. > And i am seeing material 'lag'. By 'lag' i mean it hangs for little bits of > time (1s, sometimes 5s). > But very non repeatable. > > If i run > time find . -type f

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Tue, Dec 22, 2015 at 3:27 PM, Florian Haas wrote: > On Tue, Dec 22, 2015 at 3:10 AM, Haomai Wang wrote: > >> >> >> Hey everyone, > >> >> >> > >> >> >> I recently got my hands on a cluster that has been underperforming > >> >> >> in > >> >> >> terms

[ceph-users] RBD versus KVM io=native (safe?)

Hello, I came across this article today: http://www.sebastien-han.fr/blog/2013/08/12/openstack-unexplained-high-cpu-load-on-compute-nodes/ And am wondering the following things: 1. Is RBD (being sparse and certainly hosted mostly on XFS or EXT4 storage backends) being afflicted by this or is

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Tue, Dec 22, 2015 at 3:10 AM, Haomai Wang wrote: >> >> >> Hey everyone, >> >> >> >> >> >> I recently got my hands on a cluster that has been underperforming >> >> >> in >> >> >> terms of radosgw throughput, averaging about 60 PUTs/s with 70K >> >> >> objects where a

Re: [ceph-users] radosgw bucket index sharding tips?

On Mon, Dec 21, 2015 at 12:36 PM, Wido den Hollander wrote: > > > On 21-12-15 10:34, Florian Haas wrote: >> On Mon, Dec 21, 2015 at 10:20 AM, Wido den Hollander wrote: >> Oh, and to answer this part. I didn't do that much experimentation >> unfortunately.

Re: [ceph-users] radosgw bucket index sharding tips?

On 21-12-15 10:34, Florian Haas wrote: > On Mon, Dec 21, 2015 at 10:20 AM, Wido den Hollander wrote: > Oh, and to answer this part. I didn't do that much experimentation > unfortunately. I actually am using about 24 index shards per bucket > currently and we delete

Re: [ceph-users] Setting up a proper mirror system for Ceph

Hi, Some time later I have a script for mirroring Ceph ready: https://github.com/wido/ceph/tree/mirroring/mirroring We also have a ceph-mirrors mailinglist and I would ask for people who want to join to ping me. On this list I want to 'formalize' the mirroring stuff so we get this set up

Re: [ceph-users] rbd image mount on multiple clients

Hi Dan , When we say "mount" we are usually referring to a file system. Mounting a non shared filesystem on multiple hosts concurrently will certainly break things since each non shared filesystem host thinks it has exclusive access. Of course this is not true if a shared / clustered filesystem

Re: [ceph-users] Problem adding a new node

Hi Stephane, Your pg number seems really high compared the number of OSDs and the size of the cluster. Is there a reason ? Are you activating all the new OSD at once ? If so, I guess that adding 33% more OSDs at the same time make the cluster go sideways as it will move around a great portion of

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Mon, Dec 21, 2015 at 3:35 PM, Haomai Wang wrote: > > > On Fri, Dec 18, 2015 at 1:16 AM, Florian Haas wrote: >> >> Hey everyone, >> >> I recently got my hands on a cluster that has been underperforming in >> terms of radosgw throughput, averaging about 60

Re: [ceph-users] Cephfs: large files hang

On Fri, Dec 18, 2015 at 7:27 AM, Bryan Wright wrote: > Gregory Farnum writes: > >> >> Nonetheless, it's probably your down or incomplete PGs causing the >> issue. You can check that by seeing if seed 0.5d427a9a (out of that >> blocked request you mentioned)

Re: [ceph-users] cephfs 'lag' / hang

On 21 December 2015 at 03:23, Yan, Zheng wrote: > On Sat, Dec 19, 2015 at 4:34 AM, Don Waterloo > wrote: > > I have 3 systems w/ a cephfs mounted on them. > > And i am seeing material 'lag'. By 'lag' i mean it hangs for little bits > of > > time (1s,

Re: [ceph-users] Dealing with radosgw and large OSD LevelDBs: compact, start over, something else?

On Mon, Dec 21, 2015 at 10:55 PM, Florian Haas wrote: > On Mon, Dec 21, 2015 at 3:35 PM, Haomai Wang wrote: > > > > > > On Fri, Dec 18, 2015 at 1:16 AM, Florian Haas > wrote: > >> > >> Hey everyone, > >> > >> I recently got my hands on

[ceph-users] cluster_network goes slow during erasure code pool's stress testing

hi,all We meet a problem related to erasure pool with k:m=3:1 and stripe_unit=64k*3. We have a cluster with 96 OSDs on 4 Hosts(hosts are: srv1, srv2, srv3, srv4), each host have 24 OSDs, each host have 12 core processors (Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz) and 48GB memory. cluster

Re: [ceph-users] cephfs, low performances

On 20 December 2015 at 22:47, Yan, Zheng wrote: > >> --- > >> > > > fio tests AIO performance in this case. cephfs does not handle AIO > properly, AIO is actually SYNC IO. that's why cephfs is so slow in > this case.