On 06/14/2011 11:56 AM, Ed W wrote: > On 14/06/2011 09:03, Timo Teräs wrote: >> And not long after that, module-init-tools implemented binary file >> formats that can be used to do quicker lookups for modprobe names. I >> never bothered to implement that. >> >> The implementation is "sub-optimal" because it adheres to the old file >> formats of module-init-tools. It involves reading the large alias files >> for each program invocation. >> >> Adding support for the binary file formats used by the current >> module-init-tools would fix the performance issues. > > I'm probably still chasing the wrong problem, so please beat me with a > clue stick, but I don't see that the performance problems are definitely > parsing issues here? > > My limited reading of the strace results are that it takes 0.05s to read > a 137KB text file via busybox and 0.004s to read a 113KB binary file via > modules-init-tools. Now I'm sure it's not like with like, but given > that both results should be generated with the module.alias files in > buffer cache, it feels like it's file reading time which is slow, rather > than whether they are binary/text? > > I do see that module-init-tools appears to do some seeks and so perhaps > doesn't read in the entire file, but my instinct is still to wonder > whether it's the single character read() calls which might be the > majority of the problem?
Did you have a strace on these? Busybox modprobe uses the busybox config_read, which eventually uses fgetc(). FILE * is buffered, and it will do underlying read() syscalls in blocks of some kilobytes. But busybox implementation *absolutely has* to read all of the file and parse all of it. module-init-tools file format is such, that it only reads few bytes, seeks once, and reads some more bytes which is small fraction of the data it would have to process if it were handling the text files. That's the whole idea of the large binary file: the program knows only to read/parse the part it really needs which is usually under a kilobyte. That's also why the binary file is usually larger then the text file. It would not make sense to design a file format which is larger than the original unless there is performance benefits. - Timo _______________________________________________ busybox mailing list [email protected] http://lists.busybox.net/mailman/listinfo/busybox
