Hi All,
Is there a way by which we can measure the performance of Ceph block devices ?
(Example : I/O stats, data to identify bottlenecks etc).
Also what are the available ways in which we can compare Ceph storage
performance with other storage solutions ?
Thanks
Kumar
On Thu, May 8, 2014 at 9:37 AM, Gregory Farnum wrote:
>
> Hmm, with 128 IOs at a time (I believe I'm reading that correctly?)
> that's about 40ms of latency per op (for userspace RBD), which seems
> awfully long.
Maybe this is off the topic, AFAIK "--iodepth=128" doesn't submits 128
IOs at a time
On Wed, 14 May 2014 19:28:17 -0500 Mark Nelson wrote:
> On 05/14/2014 06:36 PM, Tyler Wilson wrote:
> > Hey All,
>
> Hi!
>
> >
> > I am setting up a new storage cluster that absolutely must have the
> > best read/write sequential speed @ 128k and the highest IOps at 4k
> > read/write as possible
Anything in dmesg?
Just
[188924.137100] init: ceph-osd (ceph/6) main process (8262) killed by
ABRT signal
[188924.137138] init: ceph-osd (ceph/6) main process ended, respawning
When you say restart, do you mean a physical restart, or just
restarting the daemon? If it takes a physical re
On 28 April 2014 16:11, Sebastien Han wrote:
> Yes yes, just restart cinder-api and cinder-volume.
> It worked for me.
In my case the image is still downloaded:(
{"status": "active", "name": "instance.image-v0.98-1-cc2.img", "tags": [],
"container_format": "bare", "created_at": "2014-05-14T10
On 15/05/14 11:36, Tyler Wilson wrote:
Hey All,
I am setting up a new storage cluster that absolutely must have the best
read/write sequential speed @ 128k and the highest IOps at 4k read/write
as possible.
My current specs for each storage node are currently;
CPU: 2x E5-2670V2
Motherboard: SM
Anything in dmesg? When you say restart, do you mean a physical
restart, or just restarting the daemon? If it takes a physical restart
and you're using intel NICs, it might be worth upgrading network
drivers. Old versions have some bugs that cause them to just drop traffic.
On 5/14/2014 9:0
I have 4 OSDs that won't stay in the cluster. I restart them, they join
for a bit, then get kicked out because they stop responding to pings
from the other OSDs.
I don't know what the issue is. The disks look fine. SMART reports no
errors or reallocated sectors. iostat says the disks are n
On 5/14/14 06:36 , Dinu Vlad wrote:
Hi Dinu,
On Wed, 14 May 2014, Dinu Vlad wrote:
I'm running a ceph cluster with 3 mon and 4 osd nodes (32 disks total) and I've been
looking at the possibility to "migrate" the data to 2 new nodes. The operation
should happen by relocating the disks - I
On 05/14/2014 06:36 PM, Tyler Wilson wrote:
Hey All,
Hi!
I am setting up a new storage cluster that absolutely must have the best
read/write sequential speed @ 128k and the highest IOps at 4k read/write
as possible.
I assume random?
My current specs for each storage node are currently;
Hi,
I notice after create ceph cluster, the ruleset for the default pools (data,
metadata, rbd) are 0,1,2 respectively. After creating the cluster, are there
any impact if I change the default ruleset to other ruleset?
Wei Cao (Buddy)
___
ceph-users
I would suggest you start some smaller scale tests to get a feeling
for the performance before committing to a large purchase of this
hardware type.
Indeed, without some solid pointers, this is the only way left.
Even with solid pointers, that's the best way. :-)
--
*Craig Lewis*
Senior
Hey All,
I am setting up a new storage cluster that absolutely must have the best
read/write sequential speed @ 128k and the highest IOps at 4k read/write as
possible.
My current specs for each storage node are currently;
CPU: 2x E5-2670V2
Motherboard: SM X9DRD-EF
OSD Disks: 20-30 Samsung 840 1TB
Hi,
So, apparently tmpfs does not support non-root xattr due to a possible
DoS-vector. There's configuration set for enabling it as far as I can see.
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y
Anyone know a way around it? Saw that there's a patch for enabling it,
but recomp
Hi Greg,
wow, that was a fast answer! Thanks a lot!
Okay, I suspected that :(
Good night from Munic
h
Erik
Von: Gregory Farnum [g...@inktank.com]
Gesendet: Mittwoch, 14. Mai 2014 23:55
An: Lukac, Erik
Cc: ceph-us...@ceph.com
Betreff: Re: [ceph-users] s
On Wed, May 14, 2014 at 2:42 PM, Lukac, Erik wrote:
> Hi there,
>
> does anybody have an idea, how I can access my files created via librados
> through the s3 gateway on my ceph-cluster?
>
> Uploading via librados and then accessing via s3 seems to be impossible
> because I only see a bunch of ent
Hi there,
me again
is there anybody who uses librados in java? It seems like my company would be
the first one who thinks about using it and if I (as a part of OPS-Team) cant
convince our DEV-Team to use librados and improve performance they'll use
radosgw :(
I'd like to know best p
Hi there,
does anybody have an idea, how I can access my files created via librados
through the s3 gateway on my ceph-cluster?
Uploading via librados and then accessing via s3 seems to be impossible because
I only see a bunch of entries but not the files I uploaded.
The perfect solution would
Why are the defaults for 'cephx require signatures' and similar still
false? Is it still necessary to maintain backwards compatibility with
very old clients by default? It seems like from a security POV, you'd
want everything to be more secure out of the box, and require the user
to explicitl
On Wed, May 14, 2014 at 12:12 PM, Shesha Sreenivasamurthy
wrote:
> Hi,
>I was experimenting with Ceph and found an interesting behavior (at
> least to me) : Number of objects doubled when a new placement group was
> added.
>
> Experiment Set Up:
>
> 3 Nodes with one OSD per node
> Replication
Hi,
I was experimenting with Ceph and found an interesting behavior (at
least to me) : Number of objects doubled when a new placement group was
added.
Experiment Set Up:
- 3 Nodes with one OSD per node
- Replication = 1
- ceph osd pool create $poolName 1;
- ceph osd pool set $p
On Wed, May 14, 2014 at 10:52 AM, Pavel V. Kaygorodov wrote:
> Hi!
>
>> CRUSH can do this. You'd have two choose ...emit sequences;
>> the first of which would descend down to a host and then choose n-1
>> devices within the host; the second would descend once. I think
>> something like this shoul
On 05/14/2014 02:13 PM, Erwin Lubbers wrote:
> Hi,
>
> I'm trying to copy a sparse provisioned rbd image from pool A to pool B (both
> are replicated three times). The image has a disksize of 8 GB and contains
> around 1.4 GB of data. I do use:
>
> rbd cp PoolA/Image PoolB/Image
>
> After copy
Hi!
> CRUSH can do this. You'd have two choose ...emit sequences;
> the first of which would descend down to a host and then choose n-1
> devices within the host; the second would descend once. I think
> something like this should work:
>
> step take default
> step choose firstn 1 datacenter
> st
On Wed, May 14, 2014 at 9:56 AM, Fabrizio G. Ventola
wrote:
> Hi everybody,
>
> Is it possible with CRUSH map to make a rule that puts R-1 replicas on
> a node and the remaining one on a different node of the same failure
> domain (for example datacenter) putting the replicas considering a
> deepe
Hi everybody,
Is it possible with CRUSH map to make a rule that puts R-1 replicas on
a node and the remaining one on a different node of the same failure
domain (for example datacenter) putting the replicas considering a
deeper failure domain (e.g. room)? Statement "step emit" may help in
this?
Id
By the way, just to report my experience: I've upgraded another
testing cluster. Both clusters (this one and the one in my previous
mail) are ok now and aren't facing the cyclical "scrubbing" -
"active+clean" state issue. They have automatically reached a steady
"active+clean" status.
Best regards
On 05/14/2014 05:24 PM, Georg Höllrigl wrote:
Hello List,
I see a pool without a name:
ceph> osd lspools
0 data,1 metadata,2 rbd,3 .rgw.root,4 .rgw.control,5 .rgw,6 .rgw.gc,7
.users.uid,8 openstack-images,9 openstack-volumes,10
openstack-backups,11 .users,12 .users.swift,13 .users.email,14 .log
Hello List,
I see a pool without a name:
ceph> osd lspools
0 data,1 metadata,2 rbd,3 .rgw.root,4 .rgw.control,5 .rgw,6 .rgw.gc,7
.users.uid,8 openstack-images,9 openstack-volumes,10
openstack-backups,11 .users,12 .users.swift,13 .users.email,14 .log,15
.rgw.buckets,16 .rgw.buckets.index,17 .u
It won't pay any attention to the racks after you change the rule. So
some PGs may have all their OSDs in one rack, and others may be spread
across racks.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
On Tue, May 13, 2014 at 10:54 PM, Cao, Buddy wrote:
> BTW, I'd like to kno
Hi,
Yeah, running with MTU 9000 here, but the test was with sequential.
Just ran rbd -p shared-1 bench-write test --io-size $((32*1024*1024))
--io-pattern rand
The cluster itself showed 700MB/s write (3x replicas), but the test just
45MB/s. But I think rbd is a little bit broken ;)
Cheers,
Jose
Hello Sage,
Yes, original deployment was done via ceph-deploy - and I am very happy to read
this :)
Thank you!
Dinu
On May 14, 2014, at 4:17 PM, Sage Weil wrote:
> Hi Dinu,
>
> On Wed, 14 May 2014, Dinu Vlad wrote:
>>
>> I'm running a ceph cluster with 3 mon and 4 osd nodes (32 disks total
On 05/14/2014 07:24 AM, Christian Eichelmann wrote:
Hi Ceph User!
I had a look at the "official" collectd fork for ceph, which is quite
outdated and not compatible with the upstream version.
Since this was not an option for us, I've worte a Python Plugin for
Collectd, that gets all the precious
Hi Dan,
Le 13/05/2014 13:42, Dan van der Ster a écrit :
> Hi,
> I think you're not getting many replies simply because those are
> rather large servers and not many have such hardware in prod.
Good point.
> We run with 24x3TB drives, 64GB ram, one 10Gbit NIC. Memory-wise there
> are no problems. T
Hi Josef,
Thanks a lot for the quick answer.
yes 32M and rand writes
and also, do you get those values i guess with a MTU of 9000 or with
the traditional and beloved MTU 1500?
German Anders
Field Storage Support Engineer
Despegar.com - IT Team
--- Original message ---
Asunto: Re:
Hi Dinu,
On Wed, 14 May 2014, Dinu Vlad wrote:
>
> I'm running a ceph cluster with 3 mon and 4 osd nodes (32 disks total) and
> I've been looking at the possibility to "migrate" the data to 2 new nodes.
> The operation should happen by relocating the disks - I'm not getting any new
> hard-driv
I'm running a ceph cluster with 3 mon and 4 osd nodes (32 disks total) and I've
been looking at the possibility to "migrate" the data to 2 new nodes. The
operation should happen by relocating the disks - I'm not getting any new
hard-drives. The cluster is used as a backend for an openstack clou
Hi,
On 14/05/14 14:45, German Anders wrote:
> I forgot to mention, of course on a 10GbE network
>
>
>
> *German Anders*
> /Field Storage Support Engineer/**
>
> Despegar.com - IT Team
>
>
>
>
>
>
>
>
>
>> --- Original message ---
>> *Asunto:* Re: [ceph-users] Slow IOPS on RBD compared to jou
I forgot to mention, of course on a 10GbE network
German Anders
Field Storage Support Engineer
Despegar.com - IT Team
--- Original message ---
Asunto: Re: [ceph-users] Slow IOPS on RBD compared to journal
andbackingdevices
De: German Anders
Para: Christian Balzer
Cc:
Fecha: Wedn
Someone could get a performance throughput on RBD of 600MB/s or more
on (rw) with a block size of 32768k?
German Anders
Field Storage Support Engineer
Despegar.com - IT Team
--- Original message ---
Asunto: Re: [ceph-users] Slow IOPS on RBD compared to journal and
backingdevices
D
Hello!
On Wed, 14 May 2014 11:29:47 +0200 Josef Johansson wrote:
> Hi Christian,
>
> I missed this thread, haven't been reading the list that well the last
> weeks.
>
> You already know my setup, since we discussed it in an earlier thread. I
> don't have a fast backing store, but I see the slo
Hi Ceph User!
I had a look at the "official" collectd fork for ceph, which is quite
outdated and not compatible with the upstream version.
Since this was not an option for us, I've worte a Python Plugin for
Collectd, that gets all the precious informations out of the admin
sockets "perf dump" com
Hello Everyone,
The important thing here is, to include the rgw_dns_name in ceph.conf
and to restart radosgw. Also you need the DNS configured to point to
your radosgw + a wildcard subdomain.
For example s3cmd handles the access this way, and you'll see the
"Method Not Allowed" message if you
Thank you, that should do the trick.
2014-05-14 6:41 GMT+02:00 Kai Zhang :
> Hi Adrian,
>
> You may be interested in "rados -p poo_name df --format json", although
> it's pool oriented, you could probably add the values together :)
>
> Regards,
> Kai
>
> 在 2014-05-13 08:33:11，"Adrian Banasiak"
Why don't you want to update to one of the elrepo kernels? If you
already went to the openstack kernel, you are using an unsupported
kernel.
I don't think anybody from redhat bothered to backport the ceph client
code to a 2.6.32 kernel.
Cristian Falcas
On Wed, May 14, 2014 at 12:56 PM, maoqi1982
Hi,
I'm trying to copy a sparse provisioned rbd image from pool A to pool B (both
are replicated three times). The image has a disksize of 8 GB and contains
around 1.4 GB of data. I do use:
rbd cp PoolA/Image PoolB/Image
After copying "ceph -s" tells me that 24 GB diskspace extra is in use. Th
Try 3.x from elrepo repo...works for me, cloudstack/ceph...
Sent from Google Nexus 4
On May 14, 2014 11:56 AM, "maoqi1982" wrote:
> Hi list
> our ceph(0.72) cluster use ubuntu12.04 is ok . client server run
> openstack install "CentOS6.4 final", the kernel is up to
> kernel-2.6.32-358.123.2.ope
Hi list
our ceph(0.72) cluster use ubuntu12.04 is ok . client server run openstack
install "CentOS6.4 final", the kernel is up to
kernel-2.6.32-358.123.2.openstack.el6.x86_64.
the question is the kernel does not support the rbd.ko ceph.ko. can anyone
help me to add the rbd.ko ceph.ko in
kern
Am 14.05.2014 11:29, schrieb Josef Johansson:
> Hi Christian,
>
> I missed this thread, haven't been reading the list that well the last
> weeks.
>
> You already know my setup, since we discussed it in an earlier thread. I
> don't have a fast backing store, but I see the slow IOPS when doing
> ra
Hi Christian,
I missed this thread, haven't been reading the list that well the last
weeks.
You already know my setup, since we discussed it in an earlier thread. I
don't have a fast backing store, but I see the slow IOPS when doing
randwrite inside the VM, with rbd cache. Still running dumpling
50 matches
Mail list logo
