Kirill Korotaev wrote: >>>>- disk I/O bandwidth: >>>>we started to use CFQv2, but it is quite poor in this regard. First, it >>>>doesn't prioritizes writes and async disk operations :( And even for >>>>sync reads we found some problems we work on now... >>>> >>>> > > > >>CKRM (on e-series) had an implementation based on a modified CFQ >>scheduler. Shailabh is currently working on porting that controller to >>f-series. >> >> >can you explain what was changed by CKRM there? Did you made it to >control ASYNC read/writes? I don't think so... > > In e-series, CFQ was modified to - maintain request queues per ckrm-class (now resource group) rather than per-tgid - explicitly maintain I/O bandwidth of each request queue (in terms of I/O issued by the I/O scheduler) - select the "next request queue to service" based on its I/O bandwidth...if a queue exceeds its allocation (as calculated from the CKRM guarantee values), the queue gets skipped.
So this did not use the CFQ priority scheme as such and only implemented the "limit" part. The current plan is to exploit the CFQ prio levels and rely on CFQ doing a good enough job in maintaining an adequate bandwidth differential between those prio levels. Again, each queue would maintain a count of its consumed bandwidth as well as target bandwidth. While picking the next request from the queue, if its observed that the queue is above its "guarantee", its priority will get reduced (it'll still supply a request) while a queue that is below its share will get bumped up....Control will be much more gradual but the basic idea is to leverage CFQ's priority handling than supplant it (since we get anticipation in the form of time-slicing for free). One concern is whether the time-slicing of CFQ plays well with queues that aren't organized by tgid...I'm still looking into that. >Do you have any plots on what is concurrent bandwidth is depending on >weights? Because, our measurements show that CFQ is not ideal and >behaves poorly when prio 0,5,6,7 are used :/ Only 1,2,3,4 are really >linear-scalable... > > Interesting. Whats the time-scale over which you expect I/O bandwidth rates to get enforced ? Perhaps the iosched discussion should use a different thread.... --Shailabh _______________________________________________ ckrm-tech mailing list https://lists.sourceforge.net/lists/listinfo/ckrm-tech
