Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-20 Thread Daniel Gryniewicz 
Yeah, except for drastic changes, we cannot take much away from
variations in Jenkins runs, I think.  Even if we know what hardware
it's running on, we can't know how loaded it is.

Daniel

On Tue, Mar 20, 2018 at 4:59 AM, Niels de Vos  wrote:
> On Tue, Mar 20, 2018 at 08:59:21AM +0100, Dominique Martinet wrote:
>> Hi,
>>
>> Girjesh Rajoria wrote on Mon, Mar 19, 2018 at 10:47:09PM +0530:
>> >> + tail -21 ../dbenchTestLog.txt
>> >>
>> >>  OperationCountAvgLatMaxLat
>> >>  --
>> >>  Deltree102 9.79927.590
>> >>  Flush   284316 1.637   203.259
>> >>  Close  2979801 0.007 0.330
>> >>  LockX13208 0.007 0.079
>> >>  Mkdir   51 0.011 0.059
>> >>  Rename  171774 0.073 0.463
>> >>  ReadX  6358865 0.01038.319
>> >>  WriteX 2022375 0.04840.888
>> >>  Unlink  819204 0.09038.363
>> >>  UnlockX  13208 0.006 0.063
>> >>  FIND_FIRST 1421549 0.04438.320
>> >>  SET_FILE_INFORMATION330438 0.024 0.310
>> >>  QUERY_FILE_INFORMATION  644319 0.004 0.242
>> >>  QUERY_PATH_INFORMATION 3676827 0.01540.851
>> >>  QUERY_FS_INFORMATION674193 0.01037.783
>> >>  NTCreateX  4056560 0.049   122.097
>> >>
>> >>
>> >> Where are the iozone results from ../ioZoneLog.txt?
>> >
>> > iozone suite doesn't give outputs result as dbench. So iozone test checks
>> > for successful completion and print message of success in the log. In cases
>> > where the test fails, it'll print error due to which test failed from
>> > ../ioZoneLog.txt.
>>
>> I think it's great to have this kind of dbench stats, and would be
>> awesome if we can have some raw figures from iozone as well (I think it
>> can output the results in csv format at least?)
>>
>>
>> jenkins can also take performance metrics from jobs and we could have
>> graphs of the performance over time if it keeps these metrics a bit
>> longer than the actual jobs (for example with the performance plugin[1],
>> but there might be other ways)
>>
>> On an individual basis as the tests are on VMs with various loads the
>> results will probably flicker a bit, but on a whole we should be able to
>> identify what week(s) introduced slowdowns/speedups after the fact quite
>> nicely if we can achieve that! :)
>
> Note that the tests in the CentOS CI run on different physical hosts.
> When a test is started, one or more machines are requested, and the
> scheduler (called Duffy) just returns a random system. This means that
> the performance results might differ quite a bit between runs, even for
> the same change-set.
>
> See https://wiki.centos.org/QaWiki/PubHardware for details about the
> hardware.
>
> So except for the performance results, it may be useful to gather some
> details about the hardware that was used.
>
> Niels
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-20 Thread Niels de Vos 
On Tue, Mar 20, 2018 at 08:59:21AM +0100, Dominique Martinet wrote:
> Hi,
> 
> Girjesh Rajoria wrote on Mon, Mar 19, 2018 at 10:47:09PM +0530:
> >> + tail -21 ../dbenchTestLog.txt
> >>
> >>  OperationCountAvgLatMaxLat
> >>  --
> >>  Deltree102 9.79927.590
> >>  Flush   284316 1.637   203.259
> >>  Close  2979801 0.007 0.330
> >>  LockX13208 0.007 0.079
> >>  Mkdir   51 0.011 0.059
> >>  Rename  171774 0.073 0.463
> >>  ReadX  6358865 0.01038.319
> >>  WriteX 2022375 0.04840.888
> >>  Unlink  819204 0.09038.363
> >>  UnlockX  13208 0.006 0.063
> >>  FIND_FIRST 1421549 0.04438.320
> >>  SET_FILE_INFORMATION330438 0.024 0.310
> >>  QUERY_FILE_INFORMATION  644319 0.004 0.242
> >>  QUERY_PATH_INFORMATION 3676827 0.01540.851
> >>  QUERY_FS_INFORMATION674193 0.01037.783
> >>  NTCreateX  4056560 0.049   122.097
> >>
> >>
> >> Where are the iozone results from ../ioZoneLog.txt?
> >
> > iozone suite doesn't give outputs result as dbench. So iozone test checks
> > for successful completion and print message of success in the log. In cases
> > where the test fails, it'll print error due to which test failed from
> > ../ioZoneLog.txt.
> 
> I think it's great to have this kind of dbench stats, and would be
> awesome if we can have some raw figures from iozone as well (I think it
> can output the results in csv format at least?)
> 
> 
> jenkins can also take performance metrics from jobs and we could have
> graphs of the performance over time if it keeps these metrics a bit
> longer than the actual jobs (for example with the performance plugin[1],
> but there might be other ways)
> 
> On an individual basis as the tests are on VMs with various loads the
> results will probably flicker a bit, but on a whole we should be able to
> identify what week(s) introduced slowdowns/speedups after the fact quite
> nicely if we can achieve that! :)

Note that the tests in the CentOS CI run on different physical hosts.
When a test is started, one or more machines are requested, and the
scheduler (called Duffy) just returns a random system. This means that
the performance results might differ quite a bit between runs, even for
the same change-set.

See https://wiki.centos.org/QaWiki/PubHardware for details about the
hardware.

So except for the performance results, it may be useful to gather some
details about the hardware that was used.

Niels

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-20 Thread Dominique Martinet 
Hi,

Girjesh Rajoria wrote on Mon, Mar 19, 2018 at 10:47:09PM +0530:
>> + tail -21 ../dbenchTestLog.txt
>>
>>  OperationCountAvgLatMaxLat
>>  --
>>  Deltree102 9.79927.590
>>  Flush   284316 1.637   203.259
>>  Close  2979801 0.007 0.330
>>  LockX13208 0.007 0.079
>>  Mkdir   51 0.011 0.059
>>  Rename  171774 0.073 0.463
>>  ReadX  6358865 0.01038.319
>>  WriteX 2022375 0.04840.888
>>  Unlink  819204 0.09038.363
>>  UnlockX  13208 0.006 0.063
>>  FIND_FIRST 1421549 0.04438.320
>>  SET_FILE_INFORMATION330438 0.024 0.310
>>  QUERY_FILE_INFORMATION  644319 0.004 0.242
>>  QUERY_PATH_INFORMATION 3676827 0.01540.851
>>  QUERY_FS_INFORMATION674193 0.01037.783
>>  NTCreateX  4056560 0.049   122.097
>>
>>
>> Where are the iozone results from ../ioZoneLog.txt?
>
> iozone suite doesn't give outputs result as dbench. So iozone test checks
> for successful completion and print message of success in the log. In cases
> where the test fails, it'll print error due to which test failed from
> ../ioZoneLog.txt.

I think it's great to have this kind of dbench stats, and would be
awesome if we can have some raw figures from iozone as well (I think it
can output the results in csv format at least?)


jenkins can also take performance metrics from jobs and we could have
graphs of the performance over time if it keeps these metrics a bit
longer than the actual jobs (for example with the performance plugin[1],
but there might be other ways)

On an individual basis as the tests are on VMs with various loads the
results will probably flicker a bit, but on a whole we should be able to
identify what week(s) introduced slowdowns/speedups after the fact quite
nicely if we can achieve that! :)


[1] https://plugins.jenkins.io/performance
-- 
Dominique

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-19 Thread Girjesh Rajoria 
On Sat, Mar 17, 2018 at 12:17 AM, William Allen Simpson <
william.allen.simp...@gmail.com> wrote:

> On 3/16/18 10:07 AM, GerritHub wrote:
>
>> william.allen.simp...@gmail.com has uploaded this change for *review*.
>>
>> View Change 
>>
>> I see that our ci.centos.org now provides dbench and iozone.

Yes, I have enabled them on last Thursday evening (don't know how they got
disabled).


>
>
The dbench results are in its log:


> + tail -21 ../dbenchTestLog.txt
>
>  OperationCountAvgLatMaxLat
>  --
>  Deltree102 9.79927.590
>  Flush   284316 1.637   203.259
>  Close  2979801 0.007 0.330
>  LockX13208 0.007 0.079
>  Mkdir   51 0.011 0.059
>  Rename  171774 0.073 0.463
>  ReadX  6358865 0.01038.319
>  WriteX 2022375 0.04840.888
>  Unlink  819204 0.09038.363
>  UnlockX  13208 0.006 0.063
>  FIND_FIRST 1421549 0.04438.320
>  SET_FILE_INFORMATION330438 0.024 0.310
>  QUERY_FILE_INFORMATION  644319 0.004 0.242
>  QUERY_PATH_INFORMATION 3676827 0.01540.851
>  QUERY_FS_INFORMATION674193 0.01037.783
>  NTCreateX  4056560 0.049   122.097
>
>
> Where are the iozone results from ../ioZoneLog.txt?
>
iozone suite doesn't give outputs result as dbench. So iozone test checks
for successful completion and print message of success in the log. In cases
where the test fails, it'll print error due to which test failed from
../ioZoneLog.txt.


> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>



-- 
Regards
Girjesh
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

Re: [Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-16 Thread William Allen Simpson 

On 3/16/18 10:07 AM, GerritHub wrote:

william.allen.simp...@gmail.com has uploaded this change for *review*.

View Change 


I see that our ci.centos.org now provides dbench and iozone.

The dbench results are in its log:

+ tail -21 ../dbenchTestLog.txt

 OperationCountAvgLatMaxLat
 --
 Deltree102 9.79927.590
 Flush   284316 1.637   203.259
 Close  2979801 0.007 0.330
 LockX13208 0.007 0.079
 Mkdir   51 0.011 0.059
 Rename  171774 0.073 0.463
 ReadX  6358865 0.01038.319
 WriteX 2022375 0.04840.888
 Unlink  819204 0.09038.363
 UnlockX  13208 0.006 0.063
 FIND_FIRST 1421549 0.04438.320
 SET_FILE_INFORMATION330438 0.024 0.310
 QUERY_FILE_INFORMATION  644319 0.004 0.242
 QUERY_PATH_INFORMATION 3676827 0.01540.851
 QUERY_FS_INFORMATION674193 0.01037.783
 NTCreateX  4056560 0.049   122.097


Where are the iozone results from ../ioZoneLog.txt?

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel

[Nfs-ganesha-devel] Change in ffilz/nfs-ganesha[next]: Pullup NTIRPC through #124

2018-03-16 Thread GerritHub 
>From :

william.allen.simp...@gmail.com has uploaded this change for review. ( 
https://review.gerrithub.io/404148


Change subject: Pullup NTIRPC through #124
..

Pullup NTIRPC through #124

 ! (work_pool) increment n_threads before spawning new thread
 ! (work_pool) shutdown bug fixes
 * Enable google pref tools profiling on rpcping
 ! Configure minimum worker threads
 * (rpcping) finer grained timer
 * (rpcping) bugfixes
 * (rpcping) ncreatef

! to backport

Special appreciation to Dominique Martinet 
for debugging the work_pool shutdown.

Change-Id: I4ba0b5e9aef15c024f429dbe0723a8e80e390ba5
Signed-off-by: William Allen Simpson 
---
M src/libntirpc
1 file changed, 1 insertion(+), 1 deletion(-)



  git pull ssh://review.gerrithub.io:29418/ffilz/nfs-ganesha 
refs/changes/48/404148/1
--
To view, visit https://review.gerrithub.io/404148
To unsubscribe, visit https://review.gerrithub.io/settings

Gerrit-Project: ffilz/nfs-ganesha
Gerrit-Branch: next
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4ba0b5e9aef15c024f429dbe0723a8e80e390ba5
Gerrit-Change-Number: 404148
Gerrit-PatchSet: 1
Gerrit-Owner: william.allen.simp...@gmail.com
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel