Aurelien Degremont writes: > The behavior is: some of our tools do many fseek-fread(few bytes) on > large files in huge quantity. Those tools face very poor performances > regarding to what could have been expected with Lustre traditional > performance on the same configuration. > > > f = fopen() > pos = somewhere > do > fseek(f, pos) > fread(10 bytes, f) > pos = somewhere_else > loop > > > Analyzing the behavior, it appears that, for each small fread(), a > read of 2MB was done instead, by the glibc, this leading to poor > performance, seen from the program (even if the global Lustre throughput > was good, for each 2MB read, only few bytes were useful). > So, why the glibc read so much data while only few were requested by the > binary?
Not sure it is relevant, but there was also some discussion about an fseek glibc optimisation last year : http://lkml.org/lkml/2006/2/28/187 "Drastic Slowdown of 'fseek()' Calls From 2.4 to 2.6 -- VMM Change?" -- solofo _______________________________________________ Lustre-devel mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-devel
