Quoting r. Ronald G. Minnich <[email protected]>: > Subject: Re: performance counters in /sys > > > > On Mon, 23 May 2005, Michael S. Tsirkin wrote: > > > > I guess the thing that has me mystified about all this is I can > > > certainly appreciate the potential 'goodness' of having 1 var/file for > > > user oriented access but perhaps one of the better examples of why this > > > is just a bad idea for programmatic access is the individual process > > > stats. Is the implication of this that some day those too would be > > > moved to /sys as one stat per file?!? Can you imaging trying to run top > > > or ps if that were to happen? > > > > Does someone care enough to post a benchmark? > > I have one I will try to post
Good, it shall then be possible to profile it and look for a solution. > But ... I did create a test program to see > if I could run supermon in user mode with no kernel module. Supermon > kernel module presents one file in /proc/sys that provides an s-expression > containing info on avenrun, enet stats, disk, etc. There is an additional > file presnted with an s-expression containing descriptors of what the data > means, whether it is bounded, and if bounded, what the min/max is. The > module works correctly in the presence of hotplug (s-expressions are your > friend). > > On Linux, we can hand out S-expressions at 12 Khz; on Plan 9, at 30 Khz -- > i.e., 12,000 and 30.000 samples/second. Obviously, you don't run that > fast, the reason to know the max is to give you an idea of the impact of > reading the file at, e.g., 100hz; in the case of supermon, the impact is > in essence zero. > > My simple test program opened the files that the supermon kernel module > provides, and on a request for data, did a read() and lseek(0), Did you try pread(2)? > concatenating the 'single value per file' into an s-expression. > > Performance was pretty bad, I figure that you really can't sample at more > than 1/50 hz (once every 50 seconds) without impacting the cpu performance > in a measurable way. Plus, keeeping the files open means you can't unload > modules. > > So, I went to open/read/close for each file, to allow module unloading to > work, and of course performance was even worse. By how much? > One value per file is a fine rule if you don't need high performance data > collection. In supermon, we can sample at 100hz with no impact on the > apps, and have actually observed mpi collectives in progress in that way. > It's pretty neat. > > One stat per file is not a good idea if you need low-impact, high > performance data collection. > > ron > Its not immediately obvious from the description above that its the "one value per file" that is the performance bottleneck, is it? For example, I'm not familiar with the s-expression format. Is it possible the extra reformatting/string copying from the read buffuer to s-expression eats the CPU? Well, I guess when we see the benchmark it shall be possible to judge where does the time go. If all of sysfs has a speed problem, the right way seems to solve it and not to add more /proc files as a work-around, as supermon module seems to do. thanks, -- MST - Michael S. Tsirkin _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
