Hi Loic, Thank you for your advice which suggested providing SHEC as an extra-package. We believe it's generally a good idea, but we are hoping SHEC would be merged into Hammer because that would have an apparent advantage from a viewpoint of maintenance support.
As for the thread safety issue, we totally agree with you and we are trying to solve it. We will complete it in a few days and we'd like to ask you to review it again so that it might be in time for Hammer's feature freeze (v0.93). Best regards, Takeshi Miyamae -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Loic Dachary Sent: Wednesday, January 14, 2015 3:03 AM To: Miyamae, Takeshi/宮前 剛 Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; Paul Von-Stamwitz ([email protected]) Subject: Re: Deadline of Github pull request for Hammer release (question) Hi, On 13/01/2015 18:05, Miyamae, Takeshi wrote: > Hi Loic, > > Thank you for your quick review. > >> Could you reference the jerasure files (they are in the jerasure >> plugin already) instead of including them in your patch ? > > We had used the reference of jerasure v2.0 files before, but changed > to including v1.2 files after the patent issue. If you are refering to http://erasure-code-patents.xyz/, it can safely be ignored. > However, we can restore the reference right away if we are needed. > >> In ::prepare you reuse the matrix, if possible. If your intention is >> to improve performances, you should consider using the same approach as the >> isa plugin. > > We have found a performance issue caused by redundant initializations > of the module, and solved it by caching the initialized variables in memory. > If that is the same approach as isa-plugin you mentioned, we also do > not have the same issue any more. The ISA plugin approach is thread safe, that's the important part. >> I'll have more comments once you have unit / functional tests > > We do have the those unit/functional tests, but the server is unreachable at > this moment. > Please let us upload them to the repository tomorrow. Great. >> Regarding your initial question: I think it is too late for the Hammer >> release. > > We are so disappointed to hear that, but we must apologize for the late > request. > If there would be any chance that SHEC be pulled into hammer, we are > very happy to conduct all the necessary tests by ourselves. > Please let us know the detailed schedule if this is a feasible option. Note that since this is a plugin it will be easy for someone to install it on a Hammer cluster, simply by copying the plugin files to each OSD / MON and restarting them. If there is some kind of urgency for you to be able to install this new plugin, I would advise making a package that contains just this file, restart the OSD once installed and recommend that the installation is done on all machines of the cluster (because every OSD / MON need to know about the plugin). Cheers > Best regards, > Takeshi Miyamae > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Loic Dachary > Sent: Tuesday, January 13, 2015 9:46 PM > To: Miyamae, Takeshi/宮前 剛 > Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; > Paul Von-Stamwitz ([email protected]) > Subject: Re: Deadline of Github pull request for Hammer release > (question) > > Hi, > > It's a great start :-) A few comments: > > Could you reference the jerasure files (they are in the jerasure plugin > already) instead of including them in your patch ? > > In ::prepare you reuse the matrix, if possible. If your intention is > to improve performances, you should consider using the same approach > as the isa plugin. See > http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa > /ErasureCodePluginIsa.cc#L36 which is and independent type > http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa > /ErasureCodeIsaTableCache.h with only one instance and is populated as > needed and protected by a lock to make it thread safe : > http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa > /ErasureCodeIsaTableCache.h#L101 > > I'll have more comments once you have unit / functional tests ( similar to > http://workbench.dachary.org/ceph/ceph/blob/giant/src/test/erasure-code/TestErasureCodeJerasure.cc > ). > > Regarding your initial question: I think it is too late for the Hammer > release. But from what I read it looks like we'll be able to merge in the > early stages of the next release and that will give us time to properly test > it. > > Cheers > > On 13/01/2015 11:34, Miyamae, Takeshi wrote: >> Hi Loic, >> >> I'm so sorry. The following is the correct repository. >> >> https://github.com/t-miyamae/ceph >> >> Best regards, >> Takeshi Miyamae >> >> -----Original Message----- >> From: Loic Dachary [mailto:[email protected]] >> Sent: Tuesday, January 13, 2015 7:26 PM >> To: Miyamae, Takeshi/宮前 剛 >> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; >> Paul Von-Stamwitz ([email protected]) >> Subject: Re: Deadline of Github pull request for Hammer release >> (question) >> >> Hi, >> >> On 13/01/2015 11:24, Miyamae, Takeshi wrote: >>> Hi Loic, >>> >>>> Although we're late in the Hammer roadmap, it's a good time for an early >>>> preview. It will help show what needs to be changed to accomodate the SHEC >>>> plugin. >>> >>> Thank you for your advices. >>> We have uploaded our latest codes to the following folk repository for an >>> early review. >>> >>> https://github.com/miyamae-takeshi/multiple-shec >> >> It's 404 ? Is it a private repository maybe ? >> >>> >>> SHEC is located in src/erasure-code/shec directory. >>> >>> We are verifying SHEC's advantages on our Ceph cluster. It takes a little >>> bit more. >>> Would you please start the review before that? >>> >>> Best regards, >>> Takeshi Miyamae >>> >>> -----Original Message----- >>> From: [email protected] >>> [mailto:[email protected]] On Behalf Of Loic Dachary >>> Sent: Wednesday, January 7, 2015 12:52 AM >>> To: Miyamae, Takeshi/宮前 剛 >>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔 >>> Subject: Re: Deadline of Github pull request for Hammer release >>> (question) >>> >>> Hi, >>> >>> On 06/01/2015 12:49, Miyamae, Takeshi wrote: >>>> Dear Loic, >>>> >>>> I'm Takeshi Miyamae, one of the authors of SHEC's blueprint. >>>> >>>> Shingled Erasure Code (SHEC) >>>> https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_C >>>> o >>>> d >>>> e >>>> _(SHEC) >>> >>> The work you have done is quite impressive :-) >>> >>>> We have revised our blueprint shown in the last CDS to extend our >>>> erasure code layouts and describe the guideline for choosing SHEC among >>>> various EC plugins. >>>> We believe the blueprint now answers all the comments given at the CDS. >>> >>> Great. >>> >>>> In addition, we would like to ask for your advice on the schedule >>>> of our github pull request. More specifically, we would like to >>>> know its deadline for Hammer release. >>>> (As we have not really completed our verification of SHEC, we are >>>> wondering if we should make it open for early preview.) >>> >>> Although we're late in the Hammer roadmap, it's a good time for an early >>> preview. It will help show what needs to be changed to accomodate the SHEC >>> plugin. >>> >>> Cheers >>> >>>> Thank you in advance, >>>> Takeshi Miyamae >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>> in the body of a message to [email protected] More >>>> majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>> >>> -- >>> Loïc Dachary, Artisan Logiciel Libre >>> >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> > > -- > Loïc Dachary, Artisan Logiciel Libre > -- Loïc Dachary, Artisan Logiciel Libre N�����r��y����b�X��ǧv�^�){.n�+���z�]z���{ay� ʇڙ�,j��f���h���z� �w��� ���j:+v���w�j�m��������zZ+�����ݢj"��!�i
