Re: [Gluster-Maintainers] Compound patches waiting for merging in release-3.8

[Pranith Kumar Karampuri]

On Thu, Sep 15, 2016 at 2:56 PM, Nigel Babu  wrote:

> On Wed, Sep 07, 2016 at 10:21:38PM +0530, Pranith Kumar Karampuri wrote:
> > >
> > > These massive changes +1500 lines of code, have not one test.
> Eventhough
> > > a new volume option is introduced. I really do hope there is some test
> > > in the master branch that runs soemthing useful with the new option
> > > enabled. Could you please backport that test-case too?
> > >
> >
> > These patches are stabilization of experimental feature compound-fops for
> > 3.8.x that is why you see massive changes because it is still
> stabilizing.
> > As per the guidelines it is okay to do this. As this is performance
> > enhancement, the nature of tests we did were manual and comparing the
> time
> > elapsed with and without the fixes. We see 10% performance improvement
> > overall in small file create workload where the write workload is 40%
> where
> > the enhancement comes into picture. That is the reason you don't see any
> > automated test as I don't think we can reliably test the performance
> > improvement at the moment in automation. May be we can integrate it with
> > the one Nigel & Shyam are driving once it is available.
>
> The performance testing that Shyam and I are driving will not help test
> these
> sort of scenarios. We do not have enough machines to run performance tests
> that
> are this fine-grained.
>

Goal is to flip the default option to use this option in some time. So it
will be exercised whether we want it or not if you have replica volume.


>
> --
> nigelb
>



-- 
Pranith
___
maintainers mailing list
maintainers@gluster.org
http://www.gluster.org/mailman/listinfo/maintainers

Re: [Gluster-Maintainers] Compound patches waiting for merging in release-3.8

[Nigel Babu]

On Wed, Sep 07, 2016 at 10:21:38PM +0530, Pranith Kumar Karampuri wrote:
> >
> > These massive changes +1500 lines of code, have not one test. Eventhough
> > a new volume option is introduced. I really do hope there is some test
> > in the master branch that runs soemthing useful with the new option
> > enabled. Could you please backport that test-case too?
> >
>
> These patches are stabilization of experimental feature compound-fops for
> 3.8.x that is why you see massive changes because it is still stabilizing.
> As per the guidelines it is okay to do this. As this is performance
> enhancement, the nature of tests we did were manual and comparing the time
> elapsed with and without the fixes. We see 10% performance improvement
> overall in small file create workload where the write workload is 40% where
> the enhancement comes into picture. That is the reason you don't see any
> automated test as I don't think we can reliably test the performance
> improvement at the moment in automation. May be we can integrate it with
> the one Nigel & Shyam are driving once it is available.

The performance testing that Shyam and I are driving will not help test these
sort of scenarios. We do not have enough machines to run performance tests that
are this fine-grained.

--
nigelb
___
maintainers mailing list
maintainers@gluster.org
http://www.gluster.org/mailman/listinfo/maintainers

Re: [Gluster-Maintainers] Compound patches waiting for merging in release-3.8

[Pranith Kumar Karampuri]

On Wed, Sep 7, 2016 at 10:21 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On Wed, Sep 7, 2016 at 8:00 PM, Niels de Vos  wrote:
>
>> Hi Krutika and Pranith,
>>
>> The patches that introduce the compound operations for release-3.8 are
>> almost ready for merging. Is there a particular order for the merging to
>> be done? I'd hate to break git-bisect if I do it in the wrong order.
>>
>>   http://review.gluster.org/#/q/status:open+project:glusterfs+
>> branch:release-3.8+topic:bug-1372693
>>
>> These massive changes +1500 lines of code, have not one test. Eventhough
>> a new volume option is introduced. I really do hope there is some test
>> in the master branch that runs soemthing useful with the new option
>> enabled. Could you please backport that test-case too?
>>
>
> These patches are stabilization of experimental feature compound-fops for
> 3.8.x that is why you see massive changes because it is still stabilizing.
> As per the guidelines it is okay to do this. As this is performance
> enhancement, the nature of tests we did were manual and comparing the time
> elapsed with and without the fixes. We see 10% performance improvement
> overall in small file create workload where the write workload is 40% where
> the enhancement comes into picture. That is the reason you don't see any
> automated test as I don't think we can reliably test the performance
> improvement at the moment in automation. May be we can integrate it with
> the one Nigel & Shyam are driving once it is available.
>
>
>>
>> Because this adds a new functionality based on the (experimental)
>> compount FOPs, we need to mention it in the release notes. Feel free to
>> start a new doc/release-notes/3.8.x.md with a little text about the
>> goal of feature, how users can enable it and verify results.
>>
>
> Hmm... it is not a feature, it is stabilization of feature and using it.
> We can probably add a short description of how the option needs to be used
> to take advantage of the perf enhancement in the bz. It can be taken as
> part of the normal process of aggregating release notes.
>
>
>>
>> I really want us to prevent introducing such a lot of new code, that
>> does not go through regular testing. Should this be included in 3.8.4,
>> or maybe the release after it?
>>
>
> For it to go through regular testing, by default the option
> use-compound-fops needs to be enabled, but since all of this is still
> experimental, I would like it to be disabled. What are the tentative
> release dates for 3.8.4/3.8.5?
>

I see that you plan to release it sometime next week. I think it is safer
to merge these just after the tagging so that it will go to 3.8.5. It will
also lessen the probability of surprises.


>
>
>>
>> Thanks,
>> Niels
>>
>
>
>
> --
> Pranith
>



-- 
Pranith
___
maintainers mailing list
maintainers@gluster.org
http://www.gluster.org/mailman/listinfo/maintainers

Re: [Gluster-Maintainers] Compound patches waiting for merging in release-3.8

[Pranith Kumar Karampuri]

On Wed, Sep 7, 2016 at 8:00 PM, Niels de Vos  wrote:

> Hi Krutika and Pranith,
>
> The patches that introduce the compound operations for release-3.8 are
> almost ready for merging. Is there a particular order for the merging to
> be done? I'd hate to break git-bisect if I do it in the wrong order.
>
>   http://review.gluster.org/#/q/status:open+project:glusterfs+
> branch:release-3.8+topic:bug-1372693
>
> These massive changes +1500 lines of code, have not one test. Eventhough
> a new volume option is introduced. I really do hope there is some test
> in the master branch that runs soemthing useful with the new option
> enabled. Could you please backport that test-case too?
>

These patches are stabilization of experimental feature compound-fops for
3.8.x that is why you see massive changes because it is still stabilizing.
As per the guidelines it is okay to do this. As this is performance
enhancement, the nature of tests we did were manual and comparing the time
elapsed with and without the fixes. We see 10% performance improvement
overall in small file create workload where the write workload is 40% where
the enhancement comes into picture. That is the reason you don't see any
automated test as I don't think we can reliably test the performance
improvement at the moment in automation. May be we can integrate it with
the one Nigel & Shyam are driving once it is available.


>
> Because this adds a new functionality based on the (experimental)
> compount FOPs, we need to mention it in the release notes. Feel free to
> start a new doc/release-notes/3.8.x.md with a little text about the
> goal of feature, how users can enable it and verify results.
>

Hmm... it is not a feature, it is stabilization of feature and using it. We
can probably add a short description of how the option needs to be used to
take advantage of the perf enhancement in the bz. It can be taken as part
of the normal process of aggregating release notes.


>
> I really want us to prevent introducing such a lot of new code, that
> does not go through regular testing. Should this be included in 3.8.4,
> or maybe the release after it?
>

For it to go through regular testing, by default the option
use-compound-fops needs to be enabled, but since all of this is still
experimental, I would like it to be disabled. What are the tentative
release dates for 3.8.4/3.8.5?


>
> Thanks,
> Niels
>



-- 
Pranith
___
maintainers mailing list
maintainers@gluster.org
http://www.gluster.org/mailman/listinfo/maintainers