PG Backend Proposal

2013-08-01 Thread Loic Dachary
Hi Sam, When the acting set changes order two chunks for the same object may co-exist in the same placement group. The key should therefore also contain the chunk number. That's probably the most sensible comment I have so far. This document is immensely useful (even in its current state)

Re: PG Backend Proposal

2013-08-01 Thread Loic Dachary
On 01/08/2013 18:42, Loic Dachary wrote: Hi Sam, When the acting set changes order two chunks for the same object may co-exist in the same placement group. The key should therefore also contain the chunk number. That's probably the most sensible comment I have so far. This document

Re: PG Backend Proposal

2013-08-01 Thread Loic Dachary
are available ). Cheers On 01/08/2013 19:14, Loic Dachary wrote: On 01/08/2013 18:42, Loic Dachary wrote: Hi Sam, When the acting set changes order two chunks for the same object may co-exist in the same placement group. The key should therefore also contain the chunk number. That's

Re: PG Backend Proposal

2013-08-02 Thread Loic Dachary
the position in the acting set and a history of osdmaps is kept, would this be used when loading the pg from disk to make sure it matches the expected chunk_id_t ? Cheers On 02/08/2013 09:39, Loic Dachary wrote: Hi Sam, I think I understand and paraphrasing you to make sure I do. We may save

Re: PG Backend Proposal

2013-08-05 Thread Loic Dachary
Hi Sam, * snapshots/clones: I assume we don't want to support snapshots/clones for erasure coded PG. If automatic tiering is implemented, snaphoting an object would be possible when a replicated PG is tiered with an erasure coded PG. The erasure coded PG would be used only for demoted

Re: PG Backend Proposal

2013-08-05 Thread Loic Dachary
at 8:10 AM, Loic Dachary l...@dachary.org wrote: Hi Sam, - coll_t needs to include a chunk_id_t. https://github.com/athanatos/ceph/blob/2234bdf7fc30738363160d598ae8b4d6f75e1dd1/doc/dev/osd_internals/erasure_coding.rst#distinguished-acting-set-positions That would be for sanity check ? Since

Re: PG Backend Proposal

2013-08-05 Thread Loic Dachary
Hi Sam, In case you don't get notified, I proposed a few changes to your erasure_coding document. Basically links to the tickets you created last friday. https://github.com/ceph/ceph/pull/482 I also reworked the coding tasks of the blueprint

Transcript: Erasure coded storage backend (step 2)

2013-08-07 Thread Loic Dachary
Hi Patrick, I wrote a transcript of the Erasure coded storage backend (step 2) session at : http://wiki.ceph.com/01Planning/CDS/Emperor/Transcript_:_Erasure_coded_storage_backend_%28step_2%29 It may be helpfull to add it as a [transcript] link next to the [video] link. Listening and reading

Call for participants : teuthology weekly meeting

2013-08-07 Thread Loic Dachary
Hi Ceph, During the Emperor summit this week, running and installing teuthology with a minimum effort was discussed ( see http://youtu.be/hrQuYzyXvVg?t=1h21m49s and http://pad.ceph.com/p/cds-teuthology ) I've installed it once, back in January 2013 and that was not too difficult but required a

github pull requests, comments and rebase

2013-08-08 Thread Loic Dachary
Hi Sage, During the discussions about continuous integration at the CDS this week ( http://youtu.be/cGosx5zD4FM?t=1h16m05s ) you mentionned that github was able to keep track of the successive versions of a pull request commit, even in the case of a rebase. I just tried the following: a)

teuthology : ulimit: error

2013-08-08 Thread Loic Dachary
Hi, Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure they have 2GB of RAM because ceph-test-dbg will not even install with 1GB of RAM ;-) and installing the key with wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key

Re: teuthology : ulimit: error

2013-08-10 Thread Loic Dachary
/fedora/redhat systems. Hi Dan, This is Ubuntu and it does work with sudo. It probably is something about how it's called rather than a system limit. Cheers On 08/08/2013 02:52 PM, Loic Dachary wrote: Hi, Trying to use Ubuntu precise virtual machines as teuthology targets ( making sure

Re: teuthology : ulimit: error

2013-08-13 Thread Loic Dachary
is the nofile setting. On 08/10/2013 01:34 AM, Loic Dachary wrote: On 10/08/2013 07:35, Dan Mick wrote: IIRC we had to adjust settings in /etc/security to allow ulimit adjustment of at least core: sed -i 's/^#\*.*soft.*core.*0/\*softcore unlimited/g' /etc/security

Re: Call for participants : teuthology weekly meeting

2013-08-13 Thread Loic Dachary
/teuthology/pull/27 Cheers On 07/08/2013 19:25, Loic Dachary wrote: Hi Ceph, During the Emperor summit this week, running and installing teuthology with a minimum effort was discussed ( see http://youtu.be/hrQuYzyXvVg?t=1h21m49s and http://pad.ceph.com/p/cds-teuthology ) I've installed

teuthology and code coverage

2013-08-13 Thread Loic Dachary
Hi, When running teuthology from a laptop with the configuration below and ./virtualenv/bin/teuthology --archive=/tmp/teuthology try.yaml it then fails on ./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html -o /tmp/lcov --cov-tools-dir $(pwd)/coverage /tmp/teuthology

Re: teuthology and code coverage

2013-08-14 Thread Loic Dachary
On 14/08/2013 09:38, Josh Durgin wrote: On 08/13/2013 10:01 AM, Loic Dachary wrote: Hi, When running teuthology from a laptop with the configuration below and ./virtualenv/bin/teuthology --archive=/tmp/teuthology try.yaml it then fails on ./virtualenv/bin/teuthology-coverage -v --html

review request : ReplicatedPG::AccessMode::wake removal

2013-08-14 Thread Loic Dachary
Hi Sam, IIRC, a few weeks ago you told me that https://github.com/ceph/ceph/blob/962b64a83037ff79855c5261325de0cd1541f582/src/osd/ReplicatedPG.cc#L4626 if (mode.wake) { requeue_ops(mode.waiting); for (listCond*::iterator p = mode.waiting_cond.begin(); p != mode.waiting_cond.end();

How to collect coverage with teuthology ?

2013-08-14 Thread Loic Dachary
Hi, Although http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-precise-amd64-gcov/log.cgi?log=26707923b23cb55583a7d6b46ef6deb9591fd165 claims to build packages with coverage ( -gcov in the name ), I can't figure out where it is nor how it is supposed to be used. A hint would be much

Re: review request : ReplicatedPG::AccessMode::wake removal

2013-08-14 Thread Loic Dachary
I removed the code in a minimal way instead of removing the data member itself, as you suggested. https://github.com/dachary/ceph/commit/2f73eea0b8d7d4633d5887ba725d258ce81f8ac0 On 14/08/2013 12:17, Loic Dachary wrote: Hi Sam, IIRC, a few weeks ago you told me that https://github.com

Deferred deletion of ObjectContext

2013-08-15 Thread Loic Dachary
Hi Sam, I've amended https://github.com/ceph/ceph/pull/414 with https://github.com/dachary/ceph/commit/66e42d0ec29ffdd2edc225f4090a132a50d9665b#L0R4668 bool locked = is_locked(); if (!locked) lock(); if (obc-ssc) put_snapset_context(obc-ssc); if (object_contexts.empty())

Show outdated diffs on github

2013-08-16 Thread Loic Dachary
Hi Josh, From https://github.com/ceph/ceph/pull/504 I see that your comments persisted despite a rebase and show as jdurgin discussed an outdated diff 8 hours ago I would like to do the same but when I comment inline, it shows differently and gets trashed by a rebase. I'm curious to learn

Re: Show outdated diffs on github

2013-08-16 Thread Loic Dachary
Thanks for the tip ! On 16/08/2013 09:01, Josh Durgin wrote: On 08/15/2013 11:45 PM, Loic Dachary wrote: Hi Josh, From https://github.com/ceph/ceph/pull/504 I see that your comments persisted despite a rebase and show as jdurgin discussed an outdated diff 8 hours ago I would like

Google recruitment spam

2013-08-16 Thread Loic Dachary
Hi Ceph, It looks like Google has a github bot harvesting mails from repositories and fishing for job applicants. I guess every committer got the exact same message addressed to the mail(s) that show in the commits. There is nothing to be done about it but it is worth mentioning, in public,

Review request : Erasure Code plugin loader implementation

2013-08-18 Thread Loic Dachary
Hi Ceph, I've implemented a draft of the Erasure Code plugin loader in the context of http://tracker.ceph.com/issues/5878. It has a trivial unit test and an example plugin. It would be great if someone could do a quick review. The general idea is that the erasure code pool calls something

Re: Review request : Erasure Code plugin loader implementation

2013-08-18 Thread Loic Dachary
' } If however, the chunk B is to be read but is missing it will be: decode([2], { 1 = 'A', 3 = 'Z' }) = { 2 = 'B' } Cheers On 18/08/2013 19:34, Sage Weil wrote: On Sun, 18 Aug 2013, Loic Dachary wrote: Hi Ceph, I've implemented a draft of the Erasure Code plugin loader in the context

Re: CEPH Erasure Encoding + OSD Scalability

2013-08-19 Thread Loic Dachary
in a virtual machine you need this emulation ... Cheers Andreas. From: Loic Dachary [l...@dachary.org] Sent: 06 July 2013 22:47 To: Andreas Joachim Peters Cc: ceph-devel@vger.kernel.org Subject: Re: CEPH Erasure Encoding + OSD Scalability Hi Andreas

Re: Review request : Erasure Code plugin loader implementation

2013-08-19 Thread Loic Dachary
Hi Sage, This makes a lot more sense indeed. I updated the http://tracker.ceph.com/issues/5878 description accordingly. ceph osd pool create poolname erasure-code-dir=/var/lib/ceph/erasure-code erasure-code-plugin=jerasure erasure-code-m=10 erasure-code-k=3 erasure-code-algorithm=Reed-Solomon

Re: Review request : Erasure Code plugin loader implementation

2013-08-19 Thread Loic Dachary
On 19/08/2013 02:01, Sage Weil wrote: On Sun, 18 Aug 2013, Loic Dachary wrote: Hi Sage, Unless I misunderstood something ( which is still possible at this stage ;-) decode() is used both for recovery of missing chunks and retrieval of the original buffer. Decoding the M data chunks

Re: Review request : Erasure Code plugin loader implementation

2013-08-20 Thread Loic Dachary
Hi Sage, I created erasure code : convenience functions to code / decode http://tracker.ceph.com/issues/6064 to implement the suggested functions. Please let me know if this should be merged with another task. Cheers On 19/08/2013 17:06, Loic Dachary wrote: On 19/08/2013 02:01, Sage Weil

Erasure Code plugin system with an example : review request

2013-08-20 Thread Loic Dachary
Hi Ceph, Yesterday I implemented a simple erasure code plugin that can sustain the loss of a single chunk. https://github.com/dachary/ceph/blob/wip-5878/src/osd/ErasureCodeExample.h and it works as shown in the unit test

Jerasure Ceph

2013-08-21 Thread Loic Dachary
Hi James, I'm working on adding jerasure as a plugin to Ceph. If I remember correctly the next version of jerasure was scheduled to be released this month. Although using jerasure-1.2 ( as found at http://web.eecs.utk.edu/~plank/plank/papers/CS-08-627.html ) would be perfectly fine, it may be

Re: Jerasure Ceph

2013-08-22 Thread Loic Dachary
on jerasure-1.2A at https://github.com/dachary/ceph/tree/wip-5879 https://github.com/dachary/ceph/commit/83845a66ae1cba63c122c0ef7658b97b474c2bd2 and I should be done by the end of next week. Cheers On 22/08/2013 18:27, Loic Dachary wrote: James Plank reply: My new GF library called GF-complete

jerasure-1.2A valgrind error

2013-08-23 Thread Loic Dachary
Hi James, I ran: valgrind --tool=memcheck ./encoder F 2 2 reed_sol_van 8 0 310 compiled from jerasure-1.2A as found at http://web.eecs.utk.edu/~plank/plank/papers/Jerasure-1.2A.tar and compiled without modifications except for -g in the makefile to get the error lines. ==22001== Use of

Re: jerasure-1.2A valgrind error

2013-08-23 Thread Loic Dachary
[cc'ing the mailing list for the record] On 23/08/2013 18:53, James Plank wrote: Hi Loic -- I see why valgrind is having issues with that, but it's safe -- I have double-checked it. Thanks, I'm reassured :-) Tell you what, though -- you should change the +10 in line 425 of encoder.c

Re: CEPH Erasure Encoding + OSD Scalability

2013-08-24 Thread Loic Dachary
xattr or leveldb ) to store meta information instead of creating a file format specifically designed for erasure code. Cheers Cheers Andreas. On Fri, Aug 23, 2013 at 1:03 AM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: On 22/08/2013 23:42, Andreas-Joachim

Re: CEPH Erasure Encoding + OSD Scalability

2013-08-25 Thread Loic Dachary
On 24/08/2013 21:41, Loic Dachary wrote: On 24/08/2013 15:30, Andreas-Joachim Peters wrote: Hi Loic, I will start to review Cool :-) ...maybe you can briefly explain few things beforehand: 1) the buffer management who allocates the output buffers for the encoding

Re: better temp objects

2013-08-29 Thread Loic Dachary
Hi Sage, What is the temp collection used for ? Is there a thread / document I could read to learn about it or a piece of code that is mainly about it ? Cheers On 29/08/2013 22:40, Sage Weil wrote: Here's what I'm thinking: Above the ObjectStore, we clear everything from temp on restart

Jerasure 1.2A plugin for Ceph

2013-08-30 Thread Loic Dachary
Hi James, The first version of the jerasure 1.2A plugin for Ceph is complete at https://github.com/ceph/ceph/pull/538#commits-pushed-763275e This commit introduces the main part: ErasureCodeJerasure: base class for jerasure ErasureCodeInterface

teuthology and jenkins

2013-09-06 Thread Loic Dachary
Hi, I've explored the idea to use jenkins as a remplacement for the queue + archive mechanism of teuthology. It turns out to be jenkins weak point. As Robert Colins ( who recently wrote a HOWTO on how to replicated the CI infrastrcture of OpenStack and is confident someone could do the same

Teuthology queuing archiving (was: teuthology and jenkins)

2013-09-10 Thread Loic Dachary
-infra where all these tools are discussed and will try to understand more about gear. Cheers On 06/09/2013 15:07, Loic Dachary wrote: Hi, I've explored the idea to use jenkins as a remplacement for the queue + archive mechanism of teuthology. It turns out to be jenkins weak point. As Robert

Re: Jerasure 1.2A plugin for Ceph

2013-09-11 Thread Loic Dachary
life as a developer easier? Thanks! Jim -- On Aug 30, 2013, at 4:20 AM, Loic Dachary wrote: Hi James, The first version of the jerasure 1.2A plugin for Ceph is complete at https://github.com/ceph/ceph/pull/538#commits-pushed-763275e This commit introduces the main part

Re: Call for participants : teuthology weekly meeting

2013-09-13 Thread Loic Dachary
: I want my own teuthology! ;-) It will be timeboxed to one hour max and the communication channel ( mumble, phone conference ... ) will be announced on #ceph-devel half an hour before ( my nick is loicd ). Cheers On 13/08/2013 14:56, Loic Dachary wrote: Hi Ceph, Teuthology was successfully

Paxos vs Raft

2013-09-14 Thread Loic Dachary
Hi, Ceph ( http://ceph.com/ ) relies on a custom implementation of Paxos to provide exabyte scale distributed storage. Like most people recently exposed to Paxos, I struggle to understand it ... but will keep studying until I get it :-) When a friend mentionned Raft (

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-14 Thread Loic Dachary
less than a ms ? I suggest you start a separate thread for this, chances are your question will not be noticed otherwise. Cheers Cheers, Andreas. From: Loic Dachary [l...@dachary.org] Sent: 25 August 2013 13:49 To: Andreas Joachim Peters

Re: Paxos vs Raft

2013-09-14 Thread Loic Dachary
While reading the Raft paper today and remembering the Paxos implementation in Ceph, I was amazed that it looked so similar. Thanks to your explanation I now understand why ;-) On 14/09/2013 18:48, Gregory Farnum wrote: On Fri, Sep 13, 2013 at 11:39 PM, Loic Dachary l...@dachary.org wrote: Hi

Re: [ceph-users] About ceph testing

2013-09-18 Thread Loic Dachary
Hi David, You're welcome to join the next teuthology meeting. It's going to happen thursday 19th september (i.e. tomorrow from where I stand ) at 6pm paris time ( CEST ). The location ( mumble, irc ... ) will be announced at 5:30pm paris time ( CEST ) on irc.oftc.net#ceph-devel . Cheers On

Re: Call for participants : teuthology weekly meeting

2013-09-19 Thread Loic Dachary
Hi, The meeting will use irc.oftc.net#ceph-devel and the mumble server at mumble.upstream-university.org . You don't need any authorization to connect. If you pick the same nickname as IRC you will minimize the risk of a name clash :-) Cheers On 13/09/2013 13:44, Loic Dachary wrote: Hi

Weekly teuthology meeting #1

2013-09-19 Thread Loic Dachary
Hi, Today 6pm paris time / CEST was the first teuthology meeting. The general idea of these meetings is to get together developers using teuthology ( or willing to use teuthology ) outside of Inktank and people using it inside inktank. The goal is to help transition from an internal tool to

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-20 Thread Loic Dachary
Hi Andreas, Great work on these benchmarks ! It's definitely an incentive to improve as much as possible. Could you push / send the scripts and sequence of operations you've used ? I'll reproduce this locally while getting rid of the extra copy. It would be useful to capture that into a script

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-20 Thread Loic Dachary
of hint would be welcome :-) Cheers On 20/09/2013 17:36, Sage Weil wrote: On Fri, 20 Sep 2013, Loic Dachary wrote: Hi Andreas, Great work on these benchmarks ! It's definitely an incentive to improve as much as possible. Could you push / send the scripts and sequence of operations you've used

Re: erasure coding / jerasure: topic of interest?

2013-09-21 Thread Loic Dachary
Hi Dan, It will be exciting to see the next jerasure release integrate the optimizations from gf-complete. It should happen before the end of the year :-) Cheers On 21/09/2013 07:22, Dan Mick wrote: At SDC this week, Ethan Miller presented about some work he has done along with others

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-21 Thread Loic Dachary
Hi Andreas, It's probably too soon to be smart about reducing the number of copies, but you're right : this copy is not necessary. The following pull request gets rid of it: https://github.com/ceph/ceph/pull/615 Cheers On 20/09/2013 18:49, Loic Dachary wrote: Hi, This is a first attempt

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-22 Thread Loic Dachary
encoding all blocks, then CRC32C on all blocks. Cheers Andreas. From: Loic Dachary [l...@dachary.org] Sent: 21 September 2013 17:11 To: Andreas Joachim Peters Cc: ceph-devel@vger.kernel.org Subject: Re: CEPH Erasure Encoding + OSD Scalability

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-23 Thread Loic Dachary
Hi Andreas, Very inefficient implementation indeed :-) I've integrated your change in https://github.com/ceph/ceph/pull/619 Cheers On 23/09/2013 01:00, Andreas Joachim Peters wrote: alignment = k*w*packetsize*sizeof(int); in_length += (alignment - (in_length%alignment); -- Loïc

crc32 for erasure code

2013-09-23 Thread Loic Dachary
Hi, Unless I'm mistaken, ceph_crc32() is currently used in master via the crc32c() method of bufferlist to: * encode_with_checksum/decode_with_checksum a PGLog entry * Message::decode_message/Message::encode_message a message via calc_*_crc *

Ceph users meetup

2013-09-25 Thread Loic Dachary
Hi Eric Patrick, Yesterday morning Eric suggested that organizing a ceph user meetup would be great and proposed his help to make it happen. Although I'd be very happy to attend a france based meetup, it may make sense to also organize a Europe wide meetup. For instance it would be great to

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-25 Thread Loic Dachary
Hi Andreas, It looks like this code would be useful as a standalone program instead of being integrated within unit tests. There are a few support program of that kind. The unit tests are probably not the best place for benchmarks. What do you think ? Cheers Note: got the updated file ;-)

Re: Ceph users meetup

2013-09-25 Thread Loic Dachary
does not see the benefit ( yet ;-) On Wed, Sep 25, 2013 at 10:53 AM, Loic Dachary l...@dachary.org wrote: Hi Eric Patrick, Yesterday morning Eric suggested that organizing a ceph user meetup would be great and proposed his help to make it happen. Although I'd be very happy to attend

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-25 Thread Loic Dachary
On 25/09/2013 20:33, Andreas Joachim Peters wrote: Yes, sure. I actually thought the same in the meanwhile ... I have some questions: Q: Can/should it stay in the framework of google test's or you would prefer just a plain executable ? A plain executable would make sense. An simple

Weekly teuthology meeting #2

2013-09-26 Thread Loic Dachary
Hi, Today 8pm paris time / CEST was the second teuthology meeting. The general idea of these meetings is to get together developers using teuthology ( or willing to use teuthology ) outside of Inktank and people using it inside inktank. The goal is to help transition from an internal tool to

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-26 Thread Loic Dachary
as to tell me what is confusing in the description ? Cheers Andreas. On Wed, Sep 25, 2013 at 8:48 PM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: On 25/09/2013 20:33, Andreas Joachim Peters wrote: Yes, sure. I actually thought the same in the meanwhile

Re: CEPH Erasure Encoding + OSD Scalability

2013-09-27 Thread Loic Dachary
will change an may require a different strategy. Cheers Thanks Andreas. From: Loic Dachary [l...@dachary.org] Sent: 26 September 2013 21:18 To: Andreas Joachim Peters Cc: Ceph Development Subject: Re: CEPH Erasure Encoding + OSD Scalability

Re: Ceph users meetup

2013-09-27 Thread Loic Dachary
- De: Loic Dachary l...@dachary.org À: Dan van der Ster d...@vanderster.com Cc: Patrick McGarry patr...@inktank.com, ERIC MOURGAYA VIRAPATRIN eric.mourgaya-virapat...@arkea.com, Ceph Development ceph-devel@vger.kernel.org Envoyé: Mercredi 25 Septembre 2013 17:27:37 Objet: Re: Ceph users

ghobjects: assert in FileStore::lfn_open

2013-09-27 Thread Loic Dachary
Hi David, In int FileStore::lfn_open(coll_t cid, const ghobject_t oid, bool create, FDRef *outfd, IndexedPath *path, Index *index) { assert(get_allow_sharded_objects() ||

Re: CEPH Erasure Encoding + OSD Scalability

2013-10-02 Thread Loic Dachary
. From: Loic Dachary [l...@dachary.org] Sent: 27 September 2013 11:40 To: Andreas Joachim Peters Cc: Ceph Development Subject: Re: CEPH Erasure Encoding + OSD Scalability On 26/09/2013 23:49, Andreas Joachim Peters wrote: Sure, this text is clear

A call for teuthology users

2013-10-03 Thread Loic Dachary
Hi Ross Patrick, During tonight talk about teuthology I suggested that, if you think it's appropriate, you could say a word during your talk in London next week. Although teuthology is still rough around the edges, it's definitely useable and useful, even outside inktank. If you could invite

Weekly teuthology meeting #3

2013-10-03 Thread Loic Dachary
Hi, Today 8pm paris time / CEST was the thrid teuthology meeting. The general idea of these meetings is to get together developers using teuthology ( or willing to use teuthology ) outside of Inktank and people using it inside inktank. The goal is to help transition from an internal tool to

OpenStack and ceph integration with puppet

2013-10-08 Thread Loic Dachary
Hi Ceph, Binding ceph to cinder and glance using puppet requires three steps: * Deploy ceph ( with ceph-deploy, puppet, chef ... ) * Follow the ceph documentation instructions ( valid for both cinder and glance ) http://ceph.com/docs/next/rbd/rbd-openstack/ * Part of the above instructions

Re: OpenStack and ceph integration with puppet

2013-10-08 Thread Loic Dachary
- 75009 Paris Web : www.enovance.com - Twitter : @enovance On October 8, 2013 at 4:18:00 PM, Loic Dachary (l...@dachary.org) wrote: Hi Ceph, Binding ceph to cinder and glance using puppet requires three steps: * Deploy ceph ( with ceph-deploy, puppet, chef ... ) * Follow the ceph

Re: Weekly teuthology meeting #3

2013-10-09 Thread Loic Dachary
://framadate.org/studs.php?sondage=iyawpvvq76omrh5p Greetings from the London Ceph days : it has been fantastic :-) http://cephdaylondon.eventbrite.com/ Cheers On 03/10/2013 20:00, Loic Dachary wrote: Hi, Today 8pm paris time / CEST was the thrid teuthology meeting. The general idea

Re: Weekly teuthology meeting #3

2013-10-09 Thread Loic Dachary
and recorded that I can attend any of the proposed times. I'm so glad it happened despite our ( Christophe and myself ) failure to attend :-D I'll wait 48h before announcing the next meeting. Cheers Thanks, Mike Dawson Co-Founder, Cloudapt LLC On 10/9/2013 2:09 PM, Loic Dachary wrote: Hi

Re: OpenStack and ceph integration with puppet

2013-10-09 Thread Loic Dachary
On Tue, Oct 8, 2013 at 11:03 AM, Loic Dachary l...@dachary.org wrote: On 08/10/2013 15:47, Sébastien Han wrote: Hi Loïc, Actually they are some steps that might be automated such as: * set the virsh secret * create both glance and cinder pools Where do you think it might already

Re: OpenStack and ceph integration with puppet

2013-10-09 Thread Loic Dachary
On 08/10/2013 16:20, Don Talton (dotalton) wrote: Hi Loic, We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after configuration changes are made. Since many of our

Re: OpenStack and ceph integration with puppet

2013-10-10 Thread Loic Dachary
On 09/10/2013 22:46, Loic Dachary wrote: On 08/10/2013 16:20, Don Talton (dotalton) wrote: Hi Loic, We utilize stackforge's puppet modules to do our heavy lifting, including p-openstack, p-cinder, p-glance. There are dependency chains so that services will be restarted after

Re: Weekly teuthology meeting #3

2013-10-10 Thread Loic Dachary
:59, Loic Dachary wrote: On 09/10/2013 20:25, Mike Dawson wrote: Loic, Zack, Tamil, and I were on the call. I started experimenting with Teuthology this week, so most of the meeting covered topics around attempting to gather and analyze performance metrics associated with Teuthology test

Ceph and puppet and OpenStack : open conference call

2013-10-11 Thread Loic Dachary
Hi Ceph, What about we discuss the state of Ceph deployment in the context of puppet and / or OpenStack ? If nothing else, it will be a good opportunity for all people actively working in this area to touch base. The proposed agenda is to list and possibly clarify the existing methods and

Erasure Coding and Homomorphism

2013-10-11 Thread Loic Dachary
Hi James, Please meet François Helt who is best know for his work on image processing in the movie industry. It has been suggested that there may be interesting intersections between Homomorphism and Erasure Coding. Do you have an opinion on this topic ? It is entirely possible that my

Re: Ceph and puppet and OpenStack : open conference call

2013-10-11 Thread Loic Dachary
US/Canada: +1.866.432.9903 United Kingdom: +44.20.8824.0117 India: +91.80.4350. Germany: +49.619.6773.9002 Japan: +81.3.5763.9394 China: +86.10.8515.5666 Room code: 208324756 Access code: 208324756 On 11/10/2013 16:47, Loic Dachary wrote: Hi Ceph, What about we discuss the state of Ceph

Re: Ceph and puppet and OpenStack : open conference call

2013-10-11 Thread Loic Dachary
and probably Salt, Ansible etc. I've not done any research in this area though. Don't you think it deserves a separate discussion ? Cheers On 11 October 2013 16:18, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Someone kindly pointed out that I did not mention the day

script for light weight testing of ceph

2013-10-13 Thread Loic Dachary
Hi, Today I wrote a script ( http://dachary.org/wp-uploads/2013/10/micro-osd.txt ) to quickly configure and spawn a ceph cluster for test purposes ( under 5 seconds ). The details are explained at http://dachary.org/?p=2374 . It could be useful for the ceph related work done with

rbd support in openstack-installer

2013-10-13 Thread Loic Dachary
Hi Dan, I'm looking for the path of least resistance to add rbd support to https://github.com/CiscoSystems/openstack-installer/ Being unfamiliar with the data oriented approach it would be great to get your advice on the following. * assume ceph has already been installed without cephx which

Re: rbd support in openstack-installer

2013-10-14 Thread Loic Dachary
On 14/10/2013 08:31, Dan Bode wrote: On Sun, Oct 13, 2013 at 4:34 PM, Loic Dachary l...@dachary.org mailto:l...@dachary.org wrote: Hi Dan, I'm looking for the path of least resistance to add rbd support to https://github.com/CiscoSystems/openstack-installer/ Being

Re: rbd support in openstack-installer

2013-10-14 Thread Loic Dachary
On 14/10/2013 19:34, Dan Bode wrote: On 14/10/2013 08:31, Dan Bode wrote: On Sun, Oct 13, 2013 at 4:34 PM, Loic Dachary l...@dachary.org mailto:l...@dachary.org mailto:l...@dachary.org mailto:l...@dachary.org wrote: Hi Dan, I'm looking

Re: Ceph and puppet and OpenStack : open conference call

2013-10-15 Thread Loic Dachary
code: 208324756 Cheers On 11/10/2013 17:18, Loic Dachary wrote: Someone kindly pointed out that I did not mention the day :-) Here are the complete informations: Tuesday 15th October 2013 16:30-17:00 UTC / 12:30-13:00 US-Eastern / 10:30-11:00 US-Mountain 09:30-10:00 US-Pacific / 18:30-19

Ceph puppet module

2013-10-16 Thread Loic Dachary
Hi Ceph, How about creating https://github.com/ceph/puppet-ceph to federate the development efforts for a puppet module exclusively focused on deploying / maintaining a Ceph cluster ? It looks like we are at a point in time where the fragmentation of the work created a desire for a central

Re: Ceph and puppet and OpenStack : open conference call

2013-10-16 Thread Loic Dachary
-and-simplify If you have any questions about the code, ask me since I didn't update the documentation to reflect the new code yet. Regards, Danny Am 15.10.2013 17:39, schrieb Loic Dachary: Hi, The call was rescheduled one hour later. It is now: 17:30-18:00 UTC / 13:30-14:00 US-Eastern

Re: PGBackend: Multiple PG implementations.

2013-10-18 Thread Loic Dachary
Hi Lluís, David Zafman and Samuel Just are working to re-architecture the core of Ceph so that and ErasureCodedBackend can be implemented. You can browse their commits to see what they are up to. The transcript of the last CDS summit will also give you a general view of the direction in which

Weekly teuthology meeting #5

2013-10-19 Thread Loic Dachary
Hi, Wednesday 16th, Ocotber 2013 8pm paris time / CEST was the fifth teuthology meeting. The general idea of these meetings is to get together developers using teuthology ( or willing to use teuthology ) outside of Inktank and people using it inside inktank. The goal is to help transition from

Ceph deployment : ceph-deploy, puppet, chef, salt, ansible...

2013-10-21 Thread Loic Dachary
Hi, It would be great if we could discuss Ceph deployment and share our experience accross configuration tools. The work in progress for the new puppet module ( https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint ) shows that it's worth discussing to find simple solutions. For

Removing disks / OSDs

2013-10-21 Thread Loic Dachary
Hi Ceph, In the context of the ceph puppet module ( https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint ) I tried to think about what should be provided to deal with disks / OSDs when they are removed or moved around. Here is a possible scenario: * Machine A dies and contains OSD

Re: Removing disks / OSDs

2013-10-21 Thread Loic Dachary
it will continue working properly even if moved to a different box. Cool. That makes it real simple from the point of view of tools like puppet :-) Cheers -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Oct 21, 2013 at 9:15 AM, Loic Dachary l...@dachary.org wrote

Re: Removing disks / OSDs

2013-10-22 Thread Loic Dachary
-Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Oct 21, 2013 at 9:15 AM, Loic Dachary l...@dachary.org wrote: Hi Ceph, In the context of the ceph puppet module ( https://wiki.openstack.org/wiki/Puppet-openstack/ceph-blueprint ) I tried to think about what

Re: Removing disks / OSDs

2013-10-23 Thread Loic Dachary
On 22/10/2013 19:26, Gregory Farnum wrote: On Mon, Oct 21, 2013 at 11:13 PM, Loic Dachary l...@dachary.org wrote: On 21/10/2013 18:49, Gregory Farnum wrote: I'm not quite sure what questions you're actually asking here... In general, the OSD is not removed from the system without explicit

Re: Removing disks / OSDs

2013-10-23 Thread Loic Dachary
: On Mon, Oct 21, 2013 at 11:13 PM, Loic Dachary l...@dachary.org wrote: On 21/10/2013 18:49, Gregory Farnum wrote: I'm not quite sure what questions you're actually asking here... In general, the OSD is not removed from the system without explicit admin intervention. When it is removed, all

Re: Weekly teuthology meeting #6

2013-10-24 Thread Loic Dachary
had in the past for those of us in North America. Does that still work for the rest of you? Cheers, Mike On 10/24/2013 6:30 AM, Loic Dachary wrote: Hi, Wednesday 23th, Ocotber 2013 8pm paris time / CEST was the sixth teuthology meeting. The general idea of these meetings is to get

Fwd: Re: Erasure Coding and Homomorphism

2013-10-25 Thread Loic Dachary
-solutions.com To: James Plank pl...@cs.utk.edu CC: Loic Dachary l...@dachary.org Hello a quick reaction. The papers are interesting indeed. Let me do a rough summary of what I understand and the application requirements I foresee A Client C is entrusting precious and large data to a Server S

Re: Removing disks / OSDs

2013-10-25 Thread Loic Dachary
://github.com/ceph/ceph/blob/v0.61.9/src/upstart/ceph-osd.conf#L29 ) ? Cheers On 23/10/2013 15:45, Loic Dachary wrote: Hi, As a conclusion to this thread and also because I realized that most people are unaware of the level of automation provided by ceph / udev, I just posted a quick summary

Re: Inktank Ceph Enterprise Launch

2013-10-30 Thread Loic Dachary
Hi Patrick, I wish Inktank was able to base its strategy and income on Free Software. Like RedHat does, for instance. In addition, as long as Inktank employs the majority of Ceph developers, publishing Calamari as a proprietary software is a conflict of interest. Should someone from the

Re: [PATCH] osd/erasurecode: Fix memory leak in jerasure_matrix_to_bitmatrix()

2013-11-02 Thread Loic Dachary
On 02/11/2013 08:03, Sage Weil wrote: Hi! On Sat, 2 Nov 2013, Xing Lin wrote: Free allocated memory before return because of NULL input Signed-off-by: Xing Lin xing...@cs.utah.edu --- src/osd/ErasureCodePluginJerasure/jerasure.c | 5 - 1 file changed, 4 insertions(+), 1

Weekly teuthology meeting #7

2013-11-03 Thread Loic Dachary
Hi, Wednesday 30th, Ocotber 2013 8pm paris time / CEST was the seventh teuthology meeting (with some DST confusion on my part). The general idea of these meetings is to get together developers using teuthology ( or willing to use teuthology ) outside of Inktank and people using it inside

  1   2   3   4   5   6   7   8   9   10   >