Hi Felix,
Thanks for your mail. I will test it more to make sure that it doesn't
break anything. Also I have added a configuration key for easier
switching to the older code in case if there is any problem. If you can
help me in any manner in testing or performance numbers, please let me know.
Regards
Rafi KC
On 27/05/20 12:37 pm, Felix Kölzow wrote:
Dear Rafi,
thanks for your effort. I think this is of great interest of many
gluster users. Thus, I would really encourage you to
test and to further improve this feature. Maybe it is beneficial to
create a certain guideline which things should be tested
to make this feature really ready for productive use.
Thanks in advance.
Felix
On 27/05/2020 07:56, RAFI KC wrote:
Hi All,
I have been working on POC to improve readdirp performance
improvement. At the end of the experiment, The results are showing
promising result in performance, overall there is a 104% improvement
for full filesystem crawl compared to the existing solution. Here is
the short test numbers. The tests were carried out in 16*3 setup with
1.5 Million dentries (Both files and dir). The system also contains
some empty directories. *In the result the proposed solution is 287%
faster than the plane volume and 104% faster than the
parallel-readdir based solution.*
*
*
Configuration
Plain volume
Parallel-readdir
Proposed Solution
FS Crawl Time in Seconds
16497.523
8717.872
4261.401
In short, the basic idea behind the proposal is the efficient
managing of readdir buffer in gluster along with prefetching the
dentries for intelligent switch-over to the next buffer. The detailed
problem description, deign description and results are available in
the
doc.https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit
If anybody can help with the testing on a different kind of
workloads, I would be very happy to assist. If wanted to test the
patch and run a performance test on your setup, I could help with
back-porting the patch to the version of your choice.
https://review.gluster.org/24469
https://review.gluster.org/24470
Regards
Rafi KC
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge:https://bluejeans.com/441850968
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users