Re: How are you using Ceph?

2012-09-18 Thread Yehuda Sadeh
We had a similar idea on our mind for a while now. The thought was to add a key value support that leverages omaps and get it exposed through the RESTful rados gateway. Having a real world use for it will certainly help in understanding the requirements. Yehuda On Mon, Sep 17, 2012 at 10:44 PM,

Re: How are you using Ceph?

2012-09-18 Thread John Axel Eriksson
Well I've used Btrfs on and off for two years now I think - in less critical situations though (at home, on testing equipment at work and on easily rebuildable systems). I've been bitten several times before so I know there've been serious problems with it. With Kernel 3.5 I had a pretty good

Re: How are you using Ceph?

2012-09-18 Thread Plaetinck, Dieter
On Tue, 18 Sep 2012 01:26:03 +0200 John Axel Eriksson j...@insane.se wrote: another distributed storage solution that had failed us more than once and we lost data. Since the old system had an http interface (not S3 compatible though) can you say a bit more about this? failure stories are

Re: How are you using Ceph?

2012-09-18 Thread Smart Weblications GmbH - Florian Wiessner
Am 18.09.2012 04:32, schrieb Sage Weil: On Mon, 17 Sep 2012, Tren Blackburn wrote: On Mon, Sep 17, 2012 at 5:05 PM, Smart Weblications GmbH - Florian Wiessner f.wiess...@smart-weblications.de wrote: Hi, i use ceph to provide storage via rbd for our virtualization cluster delivering KVM

Re: enabling cephx by default

2012-09-18 Thread Guido Winkelmann
Am Dienstag, 11. September 2012, 17:25:49 schrieben Sie: The next stable release will have cephx authentication enabled by default. Hm, that could be a problem for me. I have tried multiple times to get cephx working in the past, without lasting success. (I cannot recall at the moment what the

Re: enabling cephx by default

2012-09-18 Thread Andrey Korolyov
On Tue, Sep 18, 2012 at 4:37 PM, Guido Winkelmann guido-c...@thisisnotatest.de wrote: Am Dienstag, 11. September 2012, 17:25:49 schrieben Sie: The next stable release will have cephx authentication enabled by default. Hm, that could be a problem for me. I have tried multiple times to get cephx

Re: enabling cephx by default

2012-09-18 Thread Andrey Korolyov
On Tue, Sep 18, 2012 at 5:34 PM, Andrey Korolyov and...@xdel.ru wrote: On Tue, Sep 18, 2012 at 4:37 PM, Guido Winkelmann guido-c...@thisisnotatest.de wrote: Am Dienstag, 11. September 2012, 17:25:49 schrieben Sie: The next stable release will have cephx authentication enabled by default. Hm,

Re: How are you using Ceph?

2012-09-18 Thread John Axel Eriksson
I actually opted to not specifically mention the product we had problems with since there have been lots of changes and fixes to it, which we unfortunately were unable to make use of(you'll know why later). But I guess it's interesting enough to go into a little more detail so... before moving to

Re: How are you using Ceph?

2012-09-18 Thread Plaetinck, Dieter
thanks a lot for the detailed writeup, I found it quite useful. the list of contestants is similar to the list I made when researching (and I also had luwak); while I also think ceph is very promising and probably deserves to dominate in the future, I'm focusing on openstack swift for now. FWIW

Re: How are you using Ceph?

2012-09-18 Thread Mark Nelson
Agreed, this was a really interesting writeup! Thanks John! Dieter, do you mind if I ask what is compelling for you in choosing swift vs the other options you've looked at including Ceph? Thanks, Mark On 09/18/2012 09:51 AM, Plaetinck, Dieter wrote: thanks a lot for the detailed writeup, I

Re: enabling cephx by default

2012-09-18 Thread Wido den Hollander
On 18-09-12 15:54, Andrey Korolyov wrote: On Tue, Sep 18, 2012 at 5:34 PM, Andrey Korolyov and...@xdel.ru wrote: On Tue, Sep 18, 2012 at 4:37 PM, Guido Winkelmann guido-c...@thisisnotatest.de wrote: Am Dienstag, 11. September 2012, 17:25:49 schrieben Sie: The next stable release will have

Re: How are you using Ceph?

2012-09-18 Thread Plaetinck, Dieter
I don't mind. Ultimately it came down to ceph vs swift for us. Nothing is cast in stone yet, but we choose swift for our new not-yet-production cluster, because swift has has been around longer and has more production deployments, and hence a bigger/more experienced community, better

Re: How are you using Ceph?

2012-09-18 Thread Mark Nelson
Hi Dieter, It sounds like some of those things will come with time (more experienced community, docs, deployments, papers, etc). Are there other things we could be doing that would make Ceph feel less risky for people doing similar comparisons? Thanks, Mark On 09/18/2012 10:19 AM,

Re: How are you using Ceph?

2012-09-18 Thread Plaetinck, Dieter
Right, it just takes time to grow these things. Maybe the process could be accelerated by being more out there, but what do I know about marketing.. not much :) Dieter On Tue, 18 Sep 2012 10:27:52 -0500 Mark Nelson mark.nel...@inktank.com wrote: Hi Dieter, It sounds like some of those

Re: How are you using Ceph?

2012-09-18 Thread Travis Rhoden
I am using Ceph mainly for it's KVM and OpenStack integration, and also RBD. I also needed to provide shared storage to clusters of nodes, and thus far I haven't needed the highest-possible performance. Thus, I create RBDs, format them with ext4, and re-export them with NFS. Clients do both NFS

Re: How are you using Ceph?

2012-09-18 Thread Sage Weil
Hi Matt, On Mon, 17 Sep 2012, Matt W. Benjamin wrote: Hi Just FYI, on the NFS integration front. A pnfs files (RFC5661)-capable NFSv4 re-exporter for Ceph has been committed to the Ganesha NFSv4 server development branch. We're continuing to enhance and elaborate this. We have had on

Re: How are you using Ceph?

2012-09-18 Thread Sage Weil
On Tue, 18 Sep 2012, Smart Weblications GmbH - Florian Wiessner wrote: Am 18.09.2012 04:32, schrieb Sage Weil: On Mon, 17 Sep 2012, Tren Blackburn wrote: On Mon, Sep 17, 2012 at 5:05 PM, Smart Weblications GmbH - Florian Wiessner f.wiess...@smart-weblications.de wrote: Hi, i use

Re: How are you using Ceph?

2012-09-18 Thread Matt W. Benjamin
Hi Sage, - Sage Weil s...@inktank.com wrote: Hi Matt, On Mon, 17 Sep 2012, Matt W. Benjamin wrote: Hi Just FYI, on the NFS integration front. A pnfs files (RFC5661)-capable NFSv4 re-exporter for Ceph has been committed to the Ganesha NFSv4 server development branch. We're

Re: How are you using Ceph?

2012-09-18 Thread Xiaopong Tran
Excellent write-up. We are exactly in the same mess with Riak Luwak, a decision that was made before I took over the project. I thought we were the only one :) We are still paying the price for it, as after over a month of migrating the data from Riak to Ceph, we barely moved 30% of the data.

Re: How are you using Ceph?

2012-09-18 Thread Tren Blackburn
On Mon, Sep 17, 2012 at 7:32 PM, Sage Weil s...@inktank.com wrote: On Mon, 17 Sep 2012, Tren Blackburn wrote: On Mon, Sep 17, 2012 at 5:05 PM, Smart Weblications GmbH - Florian Wiessner f.wiess...@smart-weblications.de wrote: Hi, i use ceph to provide storage via rbd for our

Re: How are you using Ceph?

2012-09-18 Thread Sage Weil
On Tue, 18 Sep 2012, Tren Blackburn wrote: On Mon, Sep 17, 2012 at 7:32 PM, Sage Weil s...@inktank.com wrote: On Mon, 17 Sep 2012, Tren Blackburn wrote: On Mon, Sep 17, 2012 at 5:05 PM, Smart Weblications GmbH - Florian Wiessner f.wiess...@smart-weblications.de wrote: Hi, i use

Re: How are you using Ceph?

2012-09-18 Thread John Axel Eriksson
Hey Xiaopong (is that your first or last name by the way? - sorry for my ignorance), I feel your pain believe me :-). We've had many sleepless nights salvaging data. We've actually completely migrated off Riak/Luwak by now and are pretty happy about it. As you say - we've watched the cluster go

Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Tren Blackburn
Hi List; I've been working with ceph 0.51 lately, and have noticed this for a while now, but it hasn't been a big enough issue for me to report. However today I'm turning up a 192 OSD cluster, and 30 seconds per OSD adds up pretty quick. For some reason it's taking 30 seconds between checking the

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Sage Weil
On Tue, 18 Sep 2012, Tren Blackburn wrote: Hi List; I've been working with ceph 0.51 lately, and have noticed this for a while now, but it hasn't been a big enough issue for me to report. However today I'm turning up a 192 OSD cluster, and 30 seconds per OSD adds up pretty quick. For some

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Tren Blackburn
On Tue, Sep 18, 2012 at 1:58 PM, Sage Weil s...@inktank.com wrote: On Tue, 18 Sep 2012, Tren Blackburn wrote: Hi List; I've been working with ceph 0.51 lately, and have noticed this for a while now, but it hasn't been a big enough issue for me to report. However today I'm turning up a 192

Re: Inject configuration change into cluster

2012-09-18 Thread Sage Weil
On Fri, 14 Sep 2012, Sage Weil wrote: Hi, On Wed, 5 Sep 2012, Skowron S?awomir wrote: Dnia 5 wrz 2012 o godz. 17:53 Sage Weil s...@inktank.com napisa?(a): On Wed, 5 Sep 2012, S?awomir Skowron wrote: Ok, but in global case, when i use, a chef/puppet or any other, i wish to change

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Tren Blackburn
On Tue, Sep 18, 2012 at 2:11 PM, Tren Blackburn t...@eotnetworks.com wrote: On Tue, Sep 18, 2012 at 1:58 PM, Sage Weil s...@inktank.com wrote: On Tue, 18 Sep 2012, Tren Blackburn wrote: Hi List; I've been working with ceph 0.51 lately, and have noticed this for a while now, but it hasn't

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Josh Durgin
On 09/18/2012 02:23 PM, Tren Blackburn wrote: On Tue, Sep 18, 2012 at 2:11 PM, Tren Blackburn t...@eotnetworks.com wrote: On Tue, Sep 18, 2012 at 1:58 PM, Sage Weil s...@inktank.com wrote: On Tue, 18 Sep 2012, Tren Blackburn wrote: Hi List; I've been working with ceph 0.51 lately, and have

Re: Why does mkcephfs take approximately 30 seconds per osd on ceph 0.51?

2012-09-18 Thread Tren Blackburn
On Tue, Sep 18, 2012 at 4:32 PM, Josh Durgin josh.dur...@inktank.com wrote: On 09/18/2012 02:23 PM, Tren Blackburn wrote: On Tue, Sep 18, 2012 at 2:11 PM, Tren Blackburn t...@eotnetworks.com wrote: On Tue, Sep 18, 2012 at 1:58 PM, Sage Weil s...@inktank.com wrote: On Tue, 18 Sep 2012, Tren

Re: on disk encryption

2012-09-18 Thread Dustin Kirkland
On Sat, Sep 15, 2012 at 6:54 AM, Sage Weil s...@inktank.com wrote: Hey, A common requirement that's come up in conversation a few times now is on-disk, at-rest encryption. Usually, this is really just about making sure the bits on an individual disk are useless in isolation, so that drives

Re: Inject configuration change into cluster

2012-09-18 Thread Skowron Sławomir
Ok, thanks, i try this on INT, but on production, i will stay with stable release, and wait for merge in next stable. Regards Slawomir Skowron Dnia 18 wrz 2012 o godz. 23:18 Sage Weil s...@inktank.com napisał(a): On Fri, 14 Sep 2012, Sage Weil wrote: Hi, On Wed, 5 Sep 2012, Skowron