Hi, I see them at https://github.com/t-miyamae/ceph/blob/master/src/test/erasure-code/TestErasureCodeShec.cc etc. thanks ! Would you be so kind as to update the https://github.com/t-miyamae/ceph/blob/master/src/test/erasure-code/Makefile.am to add the compilation instructions for these files ?
I see the tests includes a few threads (thread1 to thread5). It looks like they
are used for measuring performance, am I right ?
pthread_t tid;
pthread_create(&tid,NULL,thread1,shec);
sleep(1);
This is likely to fail on a slow machine. Instead of waiting you should use a
lock. You will find examples in other tests, using various techniques depending
on the context. If you'd like I could point to one. I'd have to better
understand the intent of this test before I do.
From init_5 to init_22 there are various combinations of parameters which
suggest checking for all kinds of errors (m being negative, or a string instead
of a number etc.). They however all have the same assert (
EXPECT_TRUE(shec->matrix == NULL); ). It would be better to have an expectation
that verifies the error has been caught as expected. Is it part of what your
completing at the moment ?
minimum_to_decode_2 expectation for when it succeeds should be more precise that
EXPECT_TRUE(minimum_chunks.size());
since minimum_chunks is expecting to have a size that does not vary. And if it
does that would be a problem.
In minimum_to_decode_3 after
EXPECT_NE(want_to_decode,minimum_chunks);
you could check the expected content of minimum_chunks also. Unless there is a
random aspect to it ?
Cheers
On 20/01/2015 10:07, Miyamae, Takeshi wrote:
> Hi Loic,
>
> I'd appreciated your help very much.
> I have just uploaded 2 test files into the fork repository.
>
> https://github.com/t-miyamae/ceph
> files:
> src/test/erasure-code/TestErasureCodeShec.cc
> src/test/erasure-code/TestErasureCodeShec364.cc
>
> I'm sorry that tests for the thread safety codes has not been included yet.
>
> Thank you in advance,
> Takeshi Miyamae
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Loic Dachary
> Sent: Tuesday, January 20, 2015 4:23 PM
> To: Miyamae, Takeshi/宮前 剛
> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔; Paul
> Von-Stamwitz ([email protected])
> Subject: Re: Deadline of Github pull request for Hammer release (question)
>
> Hi,
>
> Thanks for the update. To speed up things, I could review what's already
> published. Did you add the tests already ?
>
> Cheers
>
> On 20/01/2015 01:48, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>> Thank you for your advice which suggested providing SHEC as an extra-package.
>> We believe it's generally a good idea, but we are hoping SHEC would be
>> merged into Hammer because that would have an apparent advantage from
>> a viewpoint of maintenance support.
>>
>> As for the thread safety issue, we totally agree with you and we are trying
>> to solve it.
>> We will complete it in a few days and we'd like to ask you to review
>> it again so that it might be in time for Hammer's feature freeze (v0.93).
>>
>> Best regards,
>> Takeshi Miyamae
>>
>> -----Original Message-----
>> From: [email protected]
>> [mailto:[email protected]] On Behalf Of Loic Dachary
>> Sent: Wednesday, January 14, 2015 3:03 AM
>> To: Miyamae, Takeshi/宮前 剛
>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔;
>> Paul Von-Stamwitz ([email protected])
>> Subject: Re: Deadline of Github pull request for Hammer release
>> (question)
>>
>> Hi,
>>
>> On 13/01/2015 18:05, Miyamae, Takeshi wrote:
>>> Hi Loic,
>>>
>>> Thank you for your quick review.
>>>
>>>> Could you reference the jerasure files (they are in the jerasure
>>>> plugin already) instead of including them in your patch ?
>>>
>>> We had used the reference of jerasure v2.0 files before, but changed
>>> to including v1.2 files after the patent issue.
>>
>> If you are refering to http://erasure-code-patents.xyz/, it can safely be
>> ignored.
>>
>>> However, we can restore the reference right away if we are needed.
>>>
>>>> In ::prepare you reuse the matrix, if possible. If your intention is
>>>> to improve performances, you should consider using the same approach as
>>>> the isa plugin.
>>>
>>> We have found a performance issue caused by redundant initializations
>>> of the module, and solved it by caching the initialized variables in memory.
>>> If that is the same approach as isa-plugin you mentioned, we also do
>>> not have the same issue any more.
>>
>> The ISA plugin approach is thread safe, that's the important part.
>>
>>>> I'll have more comments once you have unit / functional tests
>>>
>>> We do have the those unit/functional tests, but the server is unreachable
>>> at this moment.
>>> Please let us upload them to the repository tomorrow.
>>
>> Great.
>>
>>>> Regarding your initial question: I think it is too late for the Hammer
>>>> release.
>>>
>>> We are so disappointed to hear that, but we must apologize for the late
>>> request.
>>> If there would be any chance that SHEC be pulled into hammer, we are
>>> very happy to conduct all the necessary tests by ourselves.
>>> Please let us know the detailed schedule if this is a feasible option.
>>
>> Note that since this is a plugin it will be easy for someone to install it
>> on a Hammer cluster, simply by copying the plugin files to each OSD / MON
>> and restarting them. If there is some kind of urgency for you to be able to
>> install this new plugin, I would advise making a package that contains just
>> this file, restart the OSD once installed and recommend that the
>> installation is done on all machines of the cluster (because every OSD / MON
>> need to know about the plugin).
>>
>> Cheers
>>
>>> Best regards,
>>> Takeshi Miyamae
>>>
>>> -----Original Message-----
>>> From: [email protected]
>>> [mailto:[email protected]] On Behalf Of Loic Dachary
>>> Sent: Tuesday, January 13, 2015 9:46 PM
>>> To: Miyamae, Takeshi/宮前 剛
>>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔;
>>> Paul Von-Stamwitz ([email protected])
>>> Subject: Re: Deadline of Github pull request for Hammer release
>>> (question)
>>>
>>> Hi,
>>>
>>> It's a great start :-) A few comments:
>>>
>>> Could you reference the jerasure files (they are in the jerasure plugin
>>> already) instead of including them in your patch ?
>>>
>>> In ::prepare you reuse the matrix, if possible. If your intention is
>>> to improve performances, you should consider using the same approach
>>> as the isa plugin. See
>>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/is
>>> a
>>> /ErasureCodePluginIsa.cc#L36 which is and independent type
>>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/is
>>> a /ErasureCodeIsaTableCache.h with only one instance and is populated
>>> as needed and protected by a lock to make it thread safe :
>>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/is
>>> a
>>> /ErasureCodeIsaTableCache.h#L101
>>>
>>> I'll have more comments once you have unit / functional tests ( similar to
>>> http://workbench.dachary.org/ceph/ceph/blob/giant/src/test/erasure-code/TestErasureCodeJerasure.cc
>>> ).
>>>
>>> Regarding your initial question: I think it is too late for the Hammer
>>> release. But from what I read it looks like we'll be able to merge in the
>>> early stages of the next release and that will give us time to properly
>>> test it.
>>>
>>> Cheers
>>>
>>> On 13/01/2015 11:34, Miyamae, Takeshi wrote:
>>>> Hi Loic,
>>>>
>>>> I'm so sorry. The following is the correct repository.
>>>>
>>>> https://github.com/t-miyamae/ceph
>>>>
>>>> Best regards,
>>>> Takeshi Miyamae
>>>>
>>>> -----Original Message-----
>>>> From: Loic Dachary [mailto:[email protected]]
>>>> Sent: Tuesday, January 13, 2015 7:26 PM
>>>> To: Miyamae, Takeshi/宮前 剛
>>>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾
>>>> 鷹詔;
>>>> Paul Von-Stamwitz ([email protected])
>>>> Subject: Re: Deadline of Github pull request for Hammer release
>>>> (question)
>>>>
>>>> Hi,
>>>>
>>>> On 13/01/2015 11:24, Miyamae, Takeshi wrote:
>>>>> Hi Loic,
>>>>>
>>>>>> Although we're late in the Hammer roadmap, it's a good time for an early
>>>>>> preview. It will help show what needs to be changed to accomodate the
>>>>>> SHEC plugin.
>>>>>
>>>>> Thank you for your advices.
>>>>> We have uploaded our latest codes to the following folk repository for an
>>>>> early review.
>>>>>
>>>>> https://github.com/miyamae-takeshi/multiple-shec
>>>>
>>>> It's 404 ? Is it a private repository maybe ?
>>>>
>>>>>
>>>>> SHEC is located in src/erasure-code/shec directory.
>>>>>
>>>>> We are verifying SHEC's advantages on our Ceph cluster. It takes a little
>>>>> bit more.
>>>>> Would you please start the review before that?
>>>>>
>>>>> Best regards,
>>>>> Takeshi Miyamae
>>>>>
>>>>> -----Original Message-----
>>>>> From: [email protected]
>>>>> [mailto:[email protected]] On Behalf Of Loic Dachary
>>>>> Sent: Wednesday, January 7, 2015 12:52 AM
>>>>> To: Miyamae, Takeshi/宮前 剛
>>>>> Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾
>>>>> 鷹詔
>>>>> Subject: Re: Deadline of Github pull request for Hammer release
>>>>> (question)
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 06/01/2015 12:49, Miyamae, Takeshi wrote:
>>>>>> Dear Loic,
>>>>>>
>>>>>> I'm Takeshi Miyamae, one of the authors of SHEC's blueprint.
>>>>>>
>>>>>> Shingled Erasure Code (SHEC)
>>>>>> https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_
>>>>>> C
>>>>>> o
>>>>>> d
>>>>>> e
>>>>>> _(SHEC)
>>>>>
>>>>> The work you have done is quite impressive :-)
>>>>>
>>>>>> We have revised our blueprint shown in the last CDS to extend our
>>>>>> erasure code layouts and describe the guideline for choosing SHEC among
>>>>>> various EC plugins.
>>>>>> We believe the blueprint now answers all the comments given at the CDS.
>>>>>
>>>>> Great.
>>>>>
>>>>>> In addition, we would like to ask for your advice on the schedule
>>>>>> of our github pull request. More specifically, we would like to
>>>>>> know its deadline for Hammer release.
>>>>>> (As we have not really completed our verification of SHEC, we are
>>>>>> wondering if we should make it open for early preview.)
>>>>>
>>>>> Although we're late in the Hammer roadmap, it's a good time for an early
>>>>> preview. It will help show what needs to be changed to accomodate the
>>>>> SHEC plugin.
>>>>>
>>>>> Cheers
>>>>>
>>>>>> Thank you in advance,
>>>>>> Takeshi Miyamae
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>>>>> in the body of a message to [email protected] More
>>>>>> majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>
>>>>> --
>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>>
>>>>
>>>> --
>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
>
> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay� ʇڙ�,j��f���h���z� �w���
> ���j:+v���w�j�m��������zZ+�����ݢj"��!tml=
>
--
Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP digital signature
