Are your performance tests on NFS or on native Lustre clients? Native Lustre clients will likely be faster, and with many clients they can create files in parallel, even in the same directory. With a single NFS server they will be limited by the VFS locking for a single directory.
Are you using IB or TCP networking? IB will be faster for low-latency requests. Are you using the Data-on-MDT feature? This can reduce overhead for very small files. Are you using NVMe storage or e.g. SATA SSDs? Based on the OST size it looks like flash of some kind, unless you are using single-HDD OSTs? Cheers, Andreas On Mar 18, 2023, at 01:44, 王烁斌 via lustre-discuss <[email protected]<mailto:[email protected]>> wrote: Hi all, This is my Lustre FS. UUID 1K-blocks Used Available Use% Mounted on ltfs-MDT0000_UUID 307826072 36904 281574768 1% /mnt/lfs[MDT:0] ltfs-MDT0001_UUID 307826072 36452 281575220 1% /mnt/lfs[MDT:1] ltfs-MDT0002_UUID 307826072 36600 281575072 1% /mnt/lfs[MDT:2] ltfs-MDT0003_UUID 307826072 36300 281575372 1% /mnt/lfs[MDT:3] ltfs-OST0000_UUID 15962575136 1027740 15156068868 1% /mnt/lfs[OST:0] ltfs-OST0001_UUID 15962575136 1027780 15156067516 1% /mnt/lfs[OST:1] ltfs-OST0002_UUID 15962575136 1027772 15156074212 1% /mnt/lfs[OST:2] ltfs-OST0003_UUID 15962575136 1027756 15156067860 1% /mnt/lfs[OST:3] ltfs-OST0004_UUID 15962575136 1027728 15156058224 1% /mnt/lfs[OST:4] ltfs-OST0005_UUID 15962575136 1027772 15156057668 1% /mnt/lfs[OST:5] ltfs-OST0006_UUID 15962575136 1027768 15156058568 1% /mnt/lfs[OST:6] ltfs-OST0007_UUID 15962575136 1027792 15156056752 1% /mnt/lfs[OST:7] filesystem_summary: 127700601088 8222108 121248509668 1% /mnt/lfs Structure ias flow: <image.png> After testing, under the current structure, the write performance of 500,000 "8k" small files is: NFSclient1——IOPS：28,000; bandwidth——230MB NFSclient1——IOPS：27,500; bandwidth——220MB Now I want to improve the performance of small files to a better level，May I ask if there is a better way。 I have noticed a feature called "MIP-IO" that can improve small file performance, but I don't know how to deploy this feature. Is there any way to improve small file performance？ _______________________________________________ lustre-discuss mailing list [email protected]<mailto:[email protected]> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Whamcloud
_______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
