Re: erasure code reliability model

2014-05-26 Thread Loic Dachary
On Monday, May 26, 2014 10:05 AM, Guang Yang yguan...@outlook.com wrote: Hello Loic and Koleos, Do we have any wiki page documenting the progress and report of this effort? We are very interested in such as well. Thanks, Guang On May 8, 2014, at 1:19 AM, Loic Dachary l...@dachary.org

Raring upstart reload bug workaround

2014-05-23 Thread Loic Dachary
Hi, I'm looking for the simplest workaround for a bug in the reload implementation of upstart as shipped with Ubuntu raring. It manifests itself by loosing track of the exec process. If it does not ring a bell, the steps to reproduce from a fresh Ubuntu raring installation are here :

Re: Signed-off-by and aliases

2014-05-21 Thread Loic Dachary
could disclose her/his identity to RedHat or any other trusted third party. Do you see any reason why RedHat would object to this ? Cheers On 19/05/2014 17:13, Loic Dachary wrote: Hi Richard, It was nice seeing you at the OpenStack summit. Do not hesitate to redirect if this question

Signed-off-by and aliases

2014-05-19 Thread Loic Dachary
Hi Richard, It was nice seeing you at the OpenStack summit. Do not hesitate to redirect if this question is best answered by someone else at RedHat. As most of us I'm still unsure who at RedHat has time to devote to Ceph ;-) Koleos Fuskus koleosfus...@yahoo.com would like to contribute code

Re: Signed-off-by and aliases

2014-05-19 Thread Loic Dachary
Anyway, I would like to receive advice on this topic. It may be convenient for me to do contributions using my real name. I am not aware of the copyright problems that I may have for using alias. Best, koleos On Monday, May 19, 2014 5:19 PM, Loic Dachary l...@dachary.org wrote: Hi

Re: Signed-off-by and aliases

2014-05-19 Thread Loic Dachary
Best, koleos On Monday, May 19, 2014 5:19 PM, Loic Dachary l...@dachary.org wrote: Hi Richard, It was nice seeing you at the OpenStack summit. Do not hesitate to redirect if this question is best answered by someone else at RedHat. As most of us I'm still unsure who at RedHat has time

bloom filters

2014-05-16 Thread Loic Dachary
Hi Sahid, Here are the files implementing the bloom filter we discussed tonight https://github.com/ceph/ceph/blob/master/src/common/bloom_filter.hpp https://github.com/ceph/ceph/blob/master/src/common/bloom_filter.cc and the associated unit tests

Ceph low hanging fruit

2014-05-16 Thread Loic Dachary
Hi Florent, Here is a low hanging fruit for you to walk through the Ceph contribution process: http://tracker.ceph.com/issues/7725 I'm announcing it on the development mailing list so that it is not taken from you ;-) Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc

Re: Implied parity and erasure code

2014-05-12 Thread Loic Dachary
with your prototype implementation? It was possible to distribute in this way, indeed. But there is no support for the implied parity idea. I wonder how that would fit. Cheers Cheers Andreas. From: Loic Dachary [l...@dachary.org] Sent: 09 May

Implied parity and erasure code

2014-05-09 Thread Loic Dachary
Hi Andreas, The implied parity bloc mentionned page 4 of http://anrg.usc.edu/~maheswaran/Xorbas.pdf is something we've not discussed in the context of the implementation of the pyramid erasure code plugin. I'm not sure if it would be useful to have. Do you have an opinion ? Cheers -- Loïc

erasure code reliability model

2014-05-07 Thread Loic Dachary
Hi Veronica, I was really happy to hear you're going to work on the erasure code reliability model. Unless I'm mistaken your action plan was not published. Would you mind adding it to the wiki so I can comment on it ? Someone else might be interested to contribute too. I've had a discussion

v0.80 contributors credits

2014-05-07 Thread Loic Dachary
josh.dur...@inktank.com 7 11 Loic Dachary l...@dachary.org 87 Yehuda Sadeh yeh...@inktank.com 95 John Wilkins john.wilk...@inktank.com 105 João Eduardo Luís joao.l...@inktank.com 115 Babu Shanmugam a...@enovance.com 124 Warren

Firefly contributors credits

2014-05-07 Thread Loic Dachary
, the details are at the end of this mail. Commits by authors 1 1144 Sage Weil s...@inktank.com 2 449 Loic Dachary l...@dachary.org 3 379 Samuel Just sam.j...@inktank.com 4 200 Yan, Zheng zheng.z@intel.com 5 161 Josh Durgin josh.dur...@inktank.com

Re: erasure code reliability model

2014-05-07 Thread Loic Dachary
this community bonding phase (I don't remember how google calls this for GSoC participants ;-) Cheers Cheers, Verónica On Wednesday, May 7, 2014 8:12 AM, Loic Dachary l...@dachary.org wrote: Hi Veronica, I was really happy to hear you're going to work on the erasure code reliability

Re: What should the Ceph Foundation be about ?

2014-05-06 Thread Loic Dachary
Hi, There are two rendez-vous points already, although not specifically for the Ceph foundation topic: Meetup sunday 11th, evening http://www.meetup.com/Ceph-in-Atlanta/events/174131332/ Ceph developers session tuesday 13th, 11:15-14:40 http://sched.co/1kczSZy It would be great if we could

Ceph User Committee monthly meeting #2 : executive summary

2014-05-03 Thread Loic Dachary
Hi Ceph, This month Ceph User Committee meeting was about: Elections RedHat and Inktank CephFS Meetings You will find an executive summary at: https://wiki.ceph.com/Community/Meetings/Ceph_User_Committee_meeting_2014-05-02 The full log of the IRC conversation is also

brag.ceph.com rebooted

2014-05-02 Thread Loic Dachary
Hi Babu, The hypervisor running http://brag.ceph.com/ has been rebooted this morning and it looks like it's back and in good shape. Please let me know if you see something wrong. Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature

What should the Ceph Foundation be about ?

2014-05-01 Thread Loic Dachary
Hi Ceph co-developers, I wrote down a few ideas about what I'd like the Ceph Foundation to provide, from a developer perspective : https://wiki.ceph.com/Development/Foundation This is no more than a wishlist, feel free to add whatever you have in mind ;-) Cheers -- Loïc Dachary, Artisan

Ceph User Committee monthly meeting #2 : May 2nd, 2014

2014-05-01 Thread Loic Dachary
Hi Ceph, This month Ceph User Committee meeting proposed agenda is at https://wiki.ceph.com/Community/Meetings#Proposed_topics: Feel free to add what you would like to discuss. Date: May 2nd, 2014 Time: 18:00-19:00 UTC 14:00-15:00 US-Eastern 12:00-13:00 US-Mountain

Re: New Defects reported by Coverity Scan for ceph (fwd)

2014-04-12 Thread Loic Dachary
I'll deal with the problems found in gf-complete: http://tracker.ceph.com/issues/8083 On 12/04/2014 06:06, Sage Weil wrote: Several new defects. This wasn't running for the last couple of weeks because the submodules didn't update on the test checkout. -- Loïc Dachary, Artisan Logiciel

v0.79 contributors credits

2014-04-08 Thread Loic Dachary
of this mail. Commits by authors 1 182 Sage Weil s...@inktank.com 2 94 Loic Dachary l...@dachary.org 3 70 Yan, Zheng zheng.z@intel.com 4 44 Danny Al-Gaaf danny.al-g...@bisect.de 5 24 David Zafman david.zaf...@inktank.com 6 21 John

gf-complete thread safe initialization

2014-04-08 Thread Loic Dachary
Hi Kevin, The following patch is initializing the array in a lock protected / not thread sensitive part of code. https://github.com/ceph/ceph/pull/1621/files Unless I'm mistaken, it ensures that it never happens automagically. The first few tests that were run indicate it's not more broken

Re: Deterministic thrashing

2014-04-07 Thread Loic Dachary
? There is no easy way to use the logs / events to significantly reduce the randomness of the workload ? I honestly have no clue ;-) Cheers -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Sun, Apr 6, 2014 at 3:29 AM, Loic Dachary l...@dachary.org wrote: Hi Ceph, It would

Re: jerasure/gf-complete segmentation violation

2014-04-07 Thread Loic Dachary
-kevin On Sun, Apr 6, 2014 at 12:06 PM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: On 06/04/2014 18:28, Kevin Greenan wrote: Hey Loic, Did this stuff start happening after a specific commit (or commits)? I see this bug was opened 6 days ago

Re: jerasure/gf-complete segmentation violation

2014-04-07 Thread Loic Dachary
thread safety. Cheers On 07/04/2014 20:29, Loic Dachary wrote: [re-adding the list for the record] On 07/04/2014 19:53, Kevin Greenan wrote: Hey Loic, BTW, you can get an illegal instruction fault if you are calling an intrinsic that is not supported on a particular platform. Is the code

Re: jerasure/gf-complete segmentation violation

2014-04-06 Thread Loic Dachary
...@inktank.com mailto:s...@inktank.com wrote: On Wed, 2 Apr 2014, Loic Dachary wrote: On 02/04/2014 19:44, Kevin Greenan wrote: Hey Loic, Are you ensuring that Jerasure (actually gf-complete) is getting memory buffers aligned on 16-byte boundaries? Without

Deterministic thrashing

2014-04-06 Thread Loic Dachary
Hi Ceph, It would be nice to have a way to replay the random events injected by stanzas such as - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 When a teuthology workload (such as tracker.ceph.com/issues/7914#note-34) crashes once a week and the error is not obvious, it would

Ceph User Committee monthly meeting #1 : executive summary

2014-04-04 Thread Loic Dachary
Hi Ceph, This month Ceph User Committee meeting was about: Tiering, erasure code Using http://tracker.ceph.com/ CephFS Miscellaneous You will find an executive summary at: https://wiki.ceph.com/Community/Meetings/Ceph_User_Committee_meeting_2014-04-03 The full log of the

jerasure/gf-complete segmentation violation

2014-04-02 Thread Loic Dachary
Hi Kevin, In the context of http://tracker.ceph.com/issues/7914 we're trying to figure out why jerasure dumps core. We don't know how to reproduce it yet (ran dozens of identical tests suites with no such crash in the past few days, which is to be expected for rare bugs because the test suite

Re: jerasure/gf-complete segmentation violation

2014-04-02 Thread Loic Dachary
are the best ;-) On 02/04/2014 19:35, Loic Dachary wrote: Hi Kevin, In the context of http://tracker.ceph.com/issues/7914 we're trying to figure out why jerasure dumps core. We don't know how to reproduce it yet (ran dozens of identical tests suites with no such crash in the past few days

Re: jerasure/gf-complete segmentation violation

2014-04-02 Thread Loic Dachary
/ErasureCodeJerasure.cc#L108 I'll re-read this logic tomorrow just to be sure. Cheers I can have a deeper look later today or tomorrow. -kevin On Wed, Apr 2, 2014 at 10:35 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Hi Kevin, In the context of http

Re: jerasure crash

2014-04-01 Thread Loic Dachary
Hi, I'll work on it right now. Cheers On 01/04/2014 02:38, Sage Weil wrote: Hey Loic, A new failure came up in QA over the weekend that looks like a problem inside jerasure: http://tracker.ceph.com/issues/7914 I think this is the first time this particular failure has come up.

Re: jerasure / gf-complete 32 bits

2014-03-30 Thread Loic Dachary
Nice catch ! https://github.com/ceph/ceph/pull/1569 On 30/03/2014 16:58, Justin Erenkrantz wrote: On Sat, Mar 29, 2014 at 5:38 PM, Loic Dachary l...@dachary.org wrote: I'm not sure it's worth fixing : https://github.com/ceph/ceph/pull/1563 Line 25 in https://github.com/ceph/ceph/pull/1563

Re: jerasure / gf-complete 32 bits

2014-03-29 Thread Loic Dachary
The gf-complete + jerasure tests run fine on Ubuntu saucy 32 bits. The problem is when trying to compile on a 32 bits machine as shown here http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-tarball-precise-i386-basic/log.cgi?log=cb624fc9ed1e891e51cc0d20622d43f6fcd08cd5 On 28/03/2014 18:15, Loic

Re: jerasure / gf-complete 32 bits

2014-03-29 Thread Loic Dachary
On 29/03/2014 17:16, Kevin Greenan wrote: Hey Loic, I'll take a look on Monday. I'll create a VM to try and reproduce (and hopefully fix) this. I'm not sure it's worth fixing : https://github.com/ceph/ceph/pull/1563 Thanks! -kevin On Mar 29, 2014, at 2:08 AM, Loic Dachary wrote

jerasure / gf-complete 32 bits

2014-03-28 Thread Loic Dachary
Hi Kevin, Someone mentionned a problem (no more information at the moment ;-) about running jerasure / gf-complete on a 32 bits machine. Does that ring a bell ? I'll try it out with the test method you suggested: 'gf_methods w -A -U' to run all of the tests for w=[4,8,16,32,64,128]. Then, I

Re: GCC -msse2 portability question

2014-03-26 Thread Loic Dachary
indeed. Created http://tracker.ceph.com/issues/7869 Cheers On Wed, Mar 26, 2014 at 3:14 AM, Loic Dachary l...@dachary.org wrote: Hi Kevin Milosz, So it would be if(sse4 sse3) = use a plugin compiled with sse + sse3 + sse4 activated else if(sse3) = use a plugin with sse2 + sse3 activated

Re: GCC -msse2 portability question

2014-03-26 Thread Loic Dachary
On 26/03/2014 19:44, Milosz Tanski wrote: On Wed, Mar 26, 2014 at 3:14 AM, Loic Dachary l...@dachary.org wrote: Hi Kevin Milosz, So it would be if(sse4 sse3) = use a plugin compiled with sse + sse3 + sse4 activated else if(sse3) = use a plugin with sse2 + sse3 activated but not sse4

AVX optimized gf-complete

2014-03-26 Thread Loic Dachary
Hi Kevin, Although https://bitbucket.org/jimplank/gf-complete does not have explicit #if related to AVX http://en.wikipedia.org/wiki/Advanced_Vector_Extensions , it is used when the compiler supports it (

Re: GCC -msse2 portability question

2014-03-25 Thread Loic Dachary
...@guerby.net wrote: On Tue, 2014-03-25 at 10:56 +0100, Loic Dachary wrote: Hi Laurent, Hi Loic, It occurs to me that all we're after is to enable SSE functions such as _mm_set_epi32. We're not trying to have the binary optimized in any implicit way, it is all explicit. The problem seems

Re: GCC -msse2 portability question

2014-03-25 Thread Loic Dachary
Andreas Peters suggested another approach, which makes sense to me : have one plugin with SSE optimizations enabled, another without them and chose at runtime between the two. What do you think ? On 23/03/2014 20:50, Loic Dachary wrote: Hi Laurent, In the context of optimizing erasure

Re: GCC -msse2 portability question

2014-03-25 Thread Loic Dachary
On Tue, Mar 25, 2014 at 12:08 PM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Andreas Peters suggested another approach, which makes sense to me : have one plugin with SSE optimizations enabled, another without them and chose at runtime between the two. What do

Re: GCC -msse2 portability question

2014-03-24 Thread Loic Dachary
On 23/03/2014 23:34, Laurent GUERBY wrote: On Sun, 2014-03-23 at 20:50 +0100, Loic Dachary wrote: Hi Laurent, In the context of optimizing erasure code functions implemented by Kevin Greenan (cc'ed) and James Plank at https://bitbucket.org/jimplank/gf-complete/ we ran accross a question

v0.78 contributors credits

2014-03-23 Thread Loic Dachary
of this mail. Commits by authors 1 255 Sage Weil s...@inktank.com 2 187 Samuel Just sam.j...@inktank.com 3 104 Loic Dachary l...@dachary.org 4 69 Josh Durgin josh.dur...@inktank.com 5 57 Danny Al-Gaaf danny.al-g...@bisect.de 6 47 Yan

GCC -msse2 portability question

2014-03-23 Thread Loic Dachary
Hi Laurent, In the context of optimizing erasure code functions implemented by Kevin Greenan (cc'ed) and James Plank at https://bitbucket.org/jimplank/gf-complete/ we ran accross a question you may have the answer to: can gcc -msse2 (or -msse* for that matter ) have a negative impact on the

Re: mon_command

2014-03-20 Thread Loic Dachary
Hi David, On 20/03/2014 03:40, David Zafman wrote: My understanding is that are going to be using rados_mon_command() to create pools according to Tracker #7586: deprecate rados_pool_create. What I found when building a test case for EC is that after using the mon_command to create the

Re: Ceph developer jobs

2014-03-19 Thread Loic Dachary
Hi Thorsten, Thanks for the update. And congratulations to the lucky developers who get to work on Ceph full time at Suse ! The positions have been removed from http://ceph.com/community/careers/ Cheers On 19/03/2014 14:59, Thorsten Behrens wrote: Loic Dachary wrote: While at FOSDEM I met

gf-complete contributions

2014-03-18 Thread Loic Dachary
Hi Jim, As of today jerasure 2 and gf-complete are used in the development version of ceph :-) https://github.com/ceph/ceph/pull/1384 There are a few pull requests to gf-complete here: https://bitbucket.org/jimplank/gf-complete/pull-requests Should they be changed so they can be

SSE extensions detection problem on the current master

2014-03-18 Thread Loic Dachary
Hi Ceph, Builds happening after https://github.com/ceph/ceph/commit/7bc9ade26b5af068318e50cb4d5f7824375dd28f which was merged yesterday and before the following is merged https://github.com/ceph/ceph/pull/1489 will produce binaries using SSE extensions on intel processors without

Re: Erasure code properties in OSDMap

2014-03-12 Thread Loic Dachary
:13 PM, Loic Dachary loic.dach...@cloudwatt.com wrote: Hi Sage Sam, I quickly sketched the replacement of the pg_pool_t::properties map with a OSDMap::properties list of maps at https://github.com/dachary/ceph/commit/fe3819a62eb139fc3f0fa4282b4d22aecd8cd398 and explained how I see

Re: Erasure code properties in OSDMap

2014-03-12 Thread Loic Dachary
or Sam during todays' standup to have their opinion about http://tracker.ceph.com/issues/7662#note-2 . If that approach sound sensible to them I'll provide an implementation for review in the next few days. Cheers Cheers, John On Wed, Mar 12, 2014 at 1:10 PM, Loic Dachary l

Re: erasured PG always peering

2014-03-12 Thread Loic Dachary
Hi On 12/03/2014 18:40, Lluís Pàmies i Juárez wrote: Hi ceph-devel, I have been playing with the new erasure code functionality and I have noticed that the erasure coded PG remains in peering state forever. Is that normal? I have a scenario with four servers each with two OSDs (total of

Re: erasured PG always peering

2014-03-12 Thread Loic Dachary
, Lluis On Wed, Mar 12, 2014 at 1:05 PM, Loic Dachary l...@dachary.org wrote: Hi On 12/03/2014 18:40, Lluís Pàmies i Juárez wrote: Hi ceph-devel, I have been playing with the new erasure code functionality and I have noticed that the erasure coded PG remains in peering state forever

Re: Erasure code properties in OSDMap

2014-03-11 Thread Loic Dachary
, manually adding the ruleset is also possible. I feel confortable explaining this but I'm probably much too familiar with the subject to be a good judge of what makes sense to someone new or not ;-) Cheers John On Sun, Mar 9, 2014 at 12:13 PM, Loic Dachary loic.dach...@cloudwatt.com wrote: Hi

Re: Ceph Puppet modules (again)

2014-03-10 Thread Loic Dachary
Hi Florian, New efforts should be directed to https://github.com/stackforge/puppet-ceph (mirrored at https://github.com/ceph/puppet-ceph) and evolving at https://review.openstack.org/#/q/status:open+project:stackforge/puppet-ceph,n,z I'm happily developing it with Andrew Woodward and David

Erasure code properties in OSDMap

2014-03-09 Thread Loic Dachary
Hi Sage Sam, I quickly sketched the replacement of the pg_pool_t::properties map with a OSDMap::properties list of maps at https://github.com/dachary/ceph/commit/fe3819a62eb139fc3f0fa4282b4d22aecd8cd398 and explained how I see it at http://tracker.ceph.com/issues/7662#note-2 It indeed makes

Re: CDS this afternoon (GMT+1)

2014-03-04 Thread Loic Dachary
You'll be missed :-) Patrick, if that's ok with you, we (Guilhem and myself) would like to discuss what is needed from Ceph from the point of view of configuration management tools, primarily Chef and Puppet. Title : Configuration management and Ceph ( Chef, Puppet, ... ) Cheers On

Re: CDS this afternoon (GMT+1)

2014-03-04 Thread Loic Dachary
14:11, Loic Dachary wrote: You'll be missed :-) Patrick, if that's ok with you, we (Guilhem and myself) would like to discuss what is needed from Ceph from the point of view of configuration management tools, primarily Chef and Puppet. Title : Configuration management and Ceph ( Chef

Re: Erasure Code user guide

2014-03-03 Thread Loic Dachary
On 03/03/2014 15:47, Wido den Hollander wrote: On 03/03/2014 02:20 PM, Loic Dachary wrote: Hi John, I'd like to draft a user guide for erasure coded pools, oriented toward system administrators. Where do you advise me to insert this ? Should I get inspiration from an example

Re: Erasure Code user guide

2014-03-03 Thread Loic Dachary
, Loic Dachary wrote: On 03/03/2014 15:47, Wido den Hollander wrote: On 03/03/2014 02:20 PM, Loic Dachary wrote: Hi John, I'd like to draft a user guide for erasure coded pools, oriented toward system administrators. Where do you

Re: Erasure Code user guide

2014-03-03 Thread Loic Dachary
) with the rst docs for man page style information. Neil On Mon, Mar 3, 2014 at 7:35 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: On 03/03/2014 15:47, Wido den Hollander wrote: On 03/03/2014 02:20 PM, Loic Dachary wrote: Hi John

Erasure Code Alpha Testing

2014-03-01 Thread Loic Dachary
Hi Ceph, The code implementing Erasure Code in Ceph has been merged in Firefly[1] and is being tested. The unit tests are run at compile time with make check. The integration tests are run by teuthology [2], the QA tool dedicated to Ceph. A few workloads have been added for erasure code [3]

Re: ceph-standalone

2014-02-25 Thread Loic Dachary
On 25/02/2014 04:34, Ricardo Rocha wrote: I did it mostly to learn about docker (and because i wanted a 'cluster in the office' like Loic had :-)). Not incredibly pretty though. :-) Robert and Stephan created this nice office cluster and it would be nice to know how it evolved. My

Re: ceph-standalone

2014-02-24 Thread Loic Dachary
On 24/02/2014 21:59, Jean-Charles LOPEZ wrote: Hi Sage, I think this is a great idea. However, I would recommend that two options to be available: 1) First options is a standalone package as you describe it 2) Second option is a prepackaged VM in OVA format people can just download

v0.77 contributors credits

2014-02-21 Thread Loic Dachary
of this mail. Commits by authors 1 75 Yehuda Sadeh yeh...@inktank.com 2 70 Sage Weil s...@inktank.com 3 66 Samuel Just sam.j...@inktank.com 4 42 Greg Farnum g...@inktank.com 5 36 Loic Dachary l...@dachary.org 6 24 John Wilkins john.wilk

Re: ceph-brag ready

2014-02-15 Thread Loic Dachary
Ping ? On 13/02/2014 01:22, Loic Dachary wrote: Cool :-) brag.ceph.com stands ready to host the server part. Could you send me your ssh public key to be granted access ? On 12/02/2014 21:58, Sebastien Han wrote: Hi guys, First implementation of the ceph-brag is ready. We have a public

Re: Google Summer of Code 2014

2014-02-12 Thread Loic Dachary
Hi, I've added the INCREASE CODE COVERAGE project description and volunteer to be a mentor. ALLOW A TUNABLE “PRIMARY AFFINITY” IN THE OSDMAP is already implemented but not merged yet at https://github.com/ceph/ceph/pull/1176 I volunteer to be a co-mentor for ADD ERASURE CODING TO THE

v0.76 contributors credits (beta)

2014-02-04 Thread Loic Dachary
of this mail. Commits by authors 1 99 Sage Weil s...@inktank.com 2 82 Loic Dachary l...@dachary.org 3 12 David Zafman david.zaf...@inktank.com 4 10 Noah Watkins noah.watk...@inktank.com 57 Josh Durgin josh.dur...@inktank.com 67 John

Re: controlling erasure code chunk size

2014-02-04 Thread Loic Dachary
Hi Andreas, For w=(multiple of 8) we could probably skip the (*sizeof(int)) and get the chunksize factor 4 down ... Loic we should check if this is ok with the Jerasure implementation I wonder if we should have 'packetsize' as a plugin parameter or we should just adjust the packetsize

Re: controlling erasure code chunk size

2014-02-03 Thread Loic Dachary
' as a plugin parameter or we should just adjust the packetsize based on the desired chunk_size to get it close. Cheers Andreas. From: Samuel Just [sam.j...@inktank.com] Sent: 02 February 2014 23:45 To: Andreas Joachim Peters Cc: Loic Dachary; Ceph

Re: controlling erasure code chunk size

2014-02-03 Thread Loic Dachary
check once more the Jerasure alignment requirements and our current implementation. To get rid of the platform dependency we could put a generic alignment requirement that chunksize has to be also 64-byte aligned. Cheers Andreas. From: Loic

Ceph developer jobs

2014-02-03 Thread Loic Dachary
Hi Ceph, While at FOSDEM I met with Thorsten who mentioned two job openings dedicated to Ceph at Suse. https://attachmatehr.silkroad.com/epostings/index.cfm?fuseaction=app.jobinfojobid=306062company_id=15495version=6source=ONLINEjobOwner=1013810aid=1

Re: firefly timing

2014-02-03 Thread Loic Dachary
Hi Sage, I'm both happy and stressed out to have more time to work on erasure code :-) If I'm not mistaken, the Firefly freeze is now set to Friday 14th, 2014. During this time I plan to focus on implementing whatever is required for Erasure Code to work with the default jerasure plugin ( I'm

caching get_chunk_size()

2014-02-02 Thread Loic Dachary
[cc' ceph-devel] Hi Sam, Assuming https://github.com/ceph/ceph/pull/1144 gets merged, MONs can query the plugin for a given pool or crush ruleset conveniently: https://github.com/dachary/ceph/blob/15c90fbf61445063967fc80ee1c916d70c593a54/src/mon/OSDMonitor.cc#L2803

controlling erasure code chunk size

2014-02-02 Thread Loic Dachary
[cc' ceph-devel] Hi Sam, Here is how chunks are expected to be aligned: https://github.com/ceph/ceph/blob/4c4e1d0d470beba7690d1c0e39bfd1146a25f465/src/osd/ErasureCodePluginJerasure/ErasureCodeJerasure.cc#L365 unsigned alignment = k*w*packetsize*sizeof(int); if (

State of the erasure code pyramid plugin

2014-01-30 Thread Loic Dachary
Hi, Your last comment made me realize I took an approach that is more complex than needed (the decode_chunks / encode_chunks methods are not necessary), which results in something that is also sub-optimal (it decodes too much and has no way to limit decoding to what is strictly necessary

Re: creation of the puppet-ceph repository

2014-01-25 Thread Loic Dachary
to be simpler to use than ceph-deploy and address more use cases, but it is not ready yet. Cheers On 24/01/2014 18:38, Gandalf Corvotempesta wrote: 2014/1/24 Loic Dachary l...@dachary.org: Hi, At the moment the module can only be used to deploy mons. It's making slow progress but a number of patches

Proxmox and ceph integration

2014-01-24 Thread Loic Dachary
Hi, I'm told today that the Ceph proxmox integration has recently been improved. I'm curious to take a look, would you be so kind as to point me to the URL where I could read about it ? Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature

Re: Proxmox and ceph integration

2014-01-24 Thread Loic Dachary
I found https://git.proxmox.com/?p=pve-manager.git;a=blob;f=PVE/CephTools.pm;h=f7f11ce2dc515cfd4fb423dbcf6dbbe4877b8afa;hb=3f5368bb67146d89f7ed098dc0c036c2217ed193 but not I'm not familiar with proxmox :-) On 24/01/2014 09:33, Loic Dachary wrote: Hi, I'm told today that the Ceph proxmox

Re: creation of the puppet-ceph repository

2014-01-24 Thread Loic Dachary
Hi, At the moment the module can only be used to deploy mons. It's making slow progress but a number of patches went in Ceph and should significantly simplify writing the puppet module, starting in Firefly. Cheers On 24/01/2014 14:57, Gandalf Corvotempesta wrote: 2013/12/14 Loic Dachary l

Pyramid Erasure Code plugin (draft)

2014-01-17 Thread Loic Dachary
Hi Andreas, I spent some time this week trying to figure out something that would be reasonably generic, readable from the sysadmin point of view and simple to implement. The input of the plugin is here: http://tracker.ceph.com/issues/7146#note-1 The json structure describes the pyramid and

Re: Pyramid Erasure Code plugin (draft)

2014-01-17 Thread Loic Dachary
17, 2014 at 10:48 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Hi Andreas, I spent some time this week trying to figure out something that would be reasonably generic, readable from the sysadmin point of view and simple to implement. The input of the plugin

Re: Pyramid Erasure Code plugin (draft)

2014-01-17 Thread Loic Dachary
, k: 4, m: 1, item: datacenter, mapping: --^--^--^, }, From: Loic Dachary [l...@dachary.org] Sent: 17 January 2014 12:00 To: Andreas Joachim Peters Cc: Ceph Development Subject: Re: Pyramid Erasure Code

Re: Pyramid Erasure Code plugin (draft)

2014-01-17 Thread Loic Dachary
that look ? Cheers Andreas. From: Loic Dachary [l...@dachary.org] Sent: 17 January 2014 14:56 To: Andreas Joachim Peters Cc: Ceph Development Subject: Re: Pyramid Erasure Code plugin (draft) On 17/01/2014 12:18, Andreas Joachim Peters wrote: Is k

Re: v0.75 contributors credits ( experimental) (take 2)

2014-01-16 Thread Loic Dachary
:12, Loic Dachary wrote: [Amended after comments from Noah Watkins, Patrick McGarry, Andrew Woodward, Ian Colle, Robin H. Johnson] Hi Ceph, Here is a sorted list of authors and organizations who contributed to v0.75, by number of commits or reviews back to v0.74. The affiliation

Re: [ceph-users] Come Talk at Ceph Day Frankfurt!

2014-01-16 Thread Loic Dachary
Hi Patrick, I would be happy to present the Ceph User Committee first the first time :-) Cheers On 16/01/2014 18:30, Patrick McGarry wrote: Greetings denizens of the Ceph universe! As you may have noticed, Inktank has announced the next Ceph Day which will be held in Frankfurt, Germany on

v0.75 contributors credits ( experimental)

2014-01-15 Thread Loic Dachary
about how it is done, the details are at the end of this mail. Commits by authors 1 104 Sage Weil s...@inktank.com 2 86 Loic Dachary l...@dachary.org 3 18 Noah Watkins noah.watk...@inktank.com 4 12 David Zafman david.zaf...@inktank.com 58

Re: [Ceph-community] v0.75 contributors credits ( experimental)

2014-01-15 Thread Loic Dachary
reviewing. Cheers On Wed, Jan 15, 2014 at 12:21 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Hi Ceph, Here is a sorted list of authors and organizations who contributed to v0.75, by number of commits or reviews back to v0.74. The affiliation of authors

Re: v0.75 contributors credits ( experimental)

2014-01-15 Thread Loic Dachary
Thanks for pointing this problem : using --after --before makes things complicated and wrong. I'll send a version two momentarily. On 15/01/2014 21:19, Robin H. Johnson wrote: On Wed, Jan 15, 2014 at 09:21:34AM +0100, Loic Dachary wrote: Hi Ceph, Here is a sorted list of authors

v0.75 contributors credits ( experimental) (take 2)

2014-01-15 Thread Loic Dachary
been reviewed. If you are curious about how it is done, the details are at the end of this mail. Commits by authors 1 259 Sage Weil s...@inktank.com 2 80 Loic Dachary l...@dachary.org 3 39 Greg Farnum g...@inktank.com 4 38 Yan, Zheng zheng.z@intel.com

Re: [ceph-users] crush choose firstn vs. indep

2014-01-14 Thread Loic Dachary
preventing the use of indep for replicated pools. Cheers Sincerely, Yuan On Mon, Jan 13, 2014 at 7:37 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: On 12/01/2014 15:55, Dietmar Maurer wrote: From the docs: step [choose|chooseleaf

Re: EC API to expose locality

2014-01-14 Thread Loic Dachary
On 14/01/2014 16:39, Sage Weil wrote: Hi Andreas, On Tue, 14 Jan 2014, Andreas Joachim Peters wrote: After some exchange with Loic and the recent list discussion, the API of the EC plugin might need some clarification/extension in the ::encode method: Currently ::encode returns a map

Re: Reliability models

2014-01-14 Thread Loic Dachary
Crystal clear, thanks ! On 13/01/2014 18:25, Kyle Bader wrote: Hi Loic, IIRC you figured out how to use https://github.com/ceph/ceph-tools/tree/master/models . Do you happen to have some kind of HOWTO or even the list of commands you've used to get me started ? git clone

Re: Pyramid erasure codes and replica hinted recovery

2014-01-13 Thread Loic Dachary
On 13/01/2014 03:35, Kyle Bader wrote: How is it different from what is described above? There must be something I fail to understand. No misunderstanding on your part, on second look that does achieve the desired placement. Could you please help walk me through the following scenarios:

Reliability models

2014-01-13 Thread Loic Dachary
Hi Kyle, IIRC you figured out how to use https://github.com/ceph/ceph-tools/tree/master/models . Do you happen to have some kind of HOWTO or even the list of commands you've used to get me started ? Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital

Re: Pyramid erasure codes and replica hinted recovery

2014-01-12 Thread Loic Dachary
On 12/01/2014 15:31, Kyle Bader wrote: If we had RS(6:3:3) 6 data chunks, 3 coding chunks, 3 local chunks, the following rule could be used to spread it over 3 datacenters: rule erasure_ruleset { ruleset 1 type erasure min_size 3 max_size 20 step

Re: Pyramid erasure codes and replica hinted recovery

2014-01-11 Thread Loic Dachary
On 11/01/2014 00:40, Kyle Bader wrote: I've been researching what features might be necessary in Ceph to build multi-site RADOS clusters, whether for purposes of scale or to meet SLA requirements more stringent than is achievable with a single datacenter. According to [1], typical

gitbuilder update request

2013-12-17 Thread Loic Dachary
Hi, The wip-use-pytest branch fails with: http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-precise-amd64-basic/log.cgi?log=687ec237ea06e36fb44bf23e3850a5b6bdca721b dpkg-checkbuilddeps: Unmet build dependencies: python-pytest warning: dpkg-buildpackage: Build dependencies/conflicts

gitbuilder make check errors

2013-12-16 Thread Loic Dachary
Hi, http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-tarball-saucy-amd64-basic/log.cgi?log=58d68995c4b09c0ec10cd80cd2e16bdb4fdfffc8 Shows a failure. However the output of make check is hidden in the log files. Would it be possible to display the content of the log files if an error happens ?

Intel Erasure Code library

2013-12-16 Thread Loic Dachary
Hi, A new Erasure Code library has been released as Free Software https://01.org/storage-acceleration-library/downloads/2013/isa-l-open-source-release It is quite tempting to write a Ceph erasure code plugin for it and see how it performs :-) Cheers -- Loïc Dachary, Artisan Logiciel Libre

<    4   5   6   7   8   9   10   11   12   >