On Wed 29-06-11 12:32:13, Kevin Constantine wrote: > On 06/29/2011 07:20 AM, Michal Hocko wrote: > >Hi, > > > >On Thu 23-06-11 19:10:39, Kevin Constantine wrote: > >>On 06/23/2011 05:51 PM, Balbir Singh wrote: > >>>If you don't mind, can I ask how you are using libcgroup, what is the use > >>>case? > >>> > >> > >>We're using cgroups in our render environment to pin batch jobs and > >>their child processes to individual cpu's to avoid processes hopping > >>around to different cores and flushing their caches. Basically each > >>batch job gets its own cgroup with the cpuset controller and we allocate > >>a single cpu to that cgroup. > >> > >>We're also using them to run batch jobs on our interactive workstations > >>without impacting the end user. We're carving off a few cores and a few > >>GB's of RAM from each workstation to run batch jobs on throughout the day. > >> > >>Basically the batch scheduler is dynamically creating and destroying > >>cgroups pretty regularly. I wrote code to do all of that and then > >>realized that libcgroup exists, so now I'm going back through and > >>refactoring to use the api as much as possible. > > > >I am wondering why you need to write any code for that. Isn't cgred > >(man cgrules.conf) what you need? > > > > I think cgred makes sense for managing machines where I know what > kind of workload to expect. My understanding of it is that cgroups > are created ahead of time with cgconfig.conf and then the rules map > processes into those cgroups.
I may be wrong because I have never played with %p or %u for group name specification but they sound dynamic enough to require on demand cgroups creation. > The problem is that I have no idea what kind of workload to expect > on most of these machines. The processes could be named just about > anything, and they could come from just about any user. How do you do that in your code then? Do you have any active monitoring running which shuffles jobs around? > > I could set up a config so that anything that gets run from the > scheduler's client daemon process lands in a cgroup. But then we > run into the situation where I want each batch process to live in > its own cgroup and have one or more cpu's pinned to that cgroup > (depending on how many cpu's the job asks for). I'm not sure how to > configure something dynamic like that with these rules. So if I understand that correctly, you are missing some way how to setup dynamically created groups, right? This sounds like an interesting idea because if you create a group without initialization then you get only inherited values from the parent group which is very controller specific. I am not sure whether there is a way to communicate an application's requirements to the daemon, though. > It seems easier to add cgroup functionality to the scheduler than it > does to come up with some configuration management scheme for > creating and maintaining cgrules.conf files all over the place. Probably yes. > cgconfig/cgred feels like something that you'd set up and kind of > forget about on a server where I've got an ftp service and an email > service and want to keep them in resource boxes. In our > environment, rules might change depending on what time of day it is. > For example, at night the memory and number of cpus allocated to > batch jobs on a machine might get increased since there's very > little interactive work happening. Yes, that is the reason I was asking for your workload. Maybe some of the ideas you are already using can be pushed to the daemon as well. For example I would really love if I could be able to specify groups parameters for those rules that use %[uUgGpP] because they can hardly be created in advance. > > -kevin Thanks -- Michal Hocko SUSE Labs SUSE LINUX s.r.o. Lihovarska 1060/12 190 00 Praha 9 Czech Republic ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 _______________________________________________ Libcg-devel mailing list Libcg-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libcg-devel
