On Tue, 2008-09-09 at 16:18 +0100, Andy Whitcroft wrote: > On Mon, Sep 08, 2008 at 03:10:40PM -0500, Adam Litke wrote: > > With the arrival of recent kernel features (namely the dynamic pool and > > multiple huge page sizes), comes increased configuration complexity. To > > ease configuration for users, libhugetlbfs should provide a set of > > library interfaces and a configuration program (hugecfg) so that the > > system's hugetlb configuration can be easily displayed and modified. > > > > Please reply with your feedback on my design below... > > > > I have considered the following use cases for hugecfg: > > > > * List all page sizes supported by the running kernel on this system and > > indicate the default page size. Provide the pool counters for each size > > (including nr_overcommit_hugepages) > > * Display information about mounted hugetlbfs filesystems including: > > path, page size and quota > > * For a given page size, set the total number of pages (nr_hugepages) > > * For a given page size, set the max number of pages > > (nr_overcommit_huge_pages) > > Is this last one 'enable dynamic pool, and set its upper bound'. These > are all very much the library calls to me? I am not getting what > 'operations' an administrator would want to do. (And here I am assuming > that this is a tool for the admin to control and setup his system). I > am expecting to see the initial list more like: > > 1) add a new mount -- literally adding a new mount perhaps in a > sensible place, > 2) remove a mount -- get rid of it again,
I thought of these two as well, but wasn't sure if that was our library's business or not. Let's consider how we would implement these though since managing mount points could take additional burden away from administrators. > 3) size/resize this mount point -- set the limit for a specific mount, > 4) size/resize the memmory pool for this page size -- set the pool > limits for a specific page size > 5) create all the default mounts for all available sizes -- something > like this could be run on boot > > This list is not exhaustive or well thought out, just the sorts of > things I expected to see as the operations an admin would want. > > > The above use cases in turn demand the following library API extensions: > > > > /* > > * Fill the array page_sizes[] with at most max_nr values corresponding > > * to the supported huge page sizes for this system and kernel. > > * Return the number of page sizes found or < 0 on error. > > */ > > int gethugepagesizes(long page_sizes[], int max_nr); > > Does this correspond to anything else out there? Is there not a > getpagesizes() call or something, can we follow thats interface? http://docs.sun.com/app/docs/doc/816-5168/getpagesizes-3c?a=view I suppose I could make it use size_t, but then it would differ from gethugepagesize(). > > struct hugetlb_stat_pagesize { > > unsigned long nr_hugepages; > > unsigned long nr_overcommit_hugepages; > > unsigned long free_hugepages; > > unsigned long resv_hugepages; > > unsigned long surplus_hugepages; > > } > > This has recently had surplus added. How would we have coped with that > in this tool at this point. We are likely to have to do so in the > future. We can add fields to the struct without breaking the compatibility of old fields. I guess the problem of using too old of a library is still there, but couldn't GNU symbol versioning help here? > Also are not all of these actually node specific underneath. Are you > going to expose those? Also is there not a use case for setting things > up bound to specific nodes. Will I be able to say: > > hugectl --mount /huge/myapp --size NNN --nodes 1,2 Good point. We might want to add calls for node-specific settings. Fortunately, hugetlb_stat_pagesize can be used for node-specific counters as well. > > > /* > > * Populate the structure pointed to by stat with hugetlb pool counter > > * values for the specified page size. > > * Return 0 on success, other values to indicate an error. > > */ > > int hugetlb_stat_pagesize(long pagesize, struct hugetlb_stat_pagesize > > *stat); > > > > /* > > * Set, respectively, nr_hugepages and nr_overcommit_hugepages for the > > * specified page size. > > * Returns 0 if the setting was successful, other values to indicate an > > error. > > */ > > int hugetlb_set_nr_hugepages(long pagesize, unsigned long nr_pages); > > int hugetlb_set_nr_overcommit_hugepages(long pagesize, unsigned long > > nr_pages); > > > > > > Implementing the hugecfg utility would then become an orchestration of > > libhugetlbfs API calls (except for mount point scanning which would > > probably be implemented standalone in hugecfg). I envision the > > following hugecfg command line usage: > > > > hugecfg [options] > > -S,--show-page-sizes - Show available page sizes and stats > > -M,--show-mounts - Show mounted hugetlb filesystems > > -p,--page-size - Operate on a specific page size > > -n,--set-nr-hugepages - Set nr_hugepages for a specific > > size > > -o,--set-nr-overcommit-hugepages - Set nr_overcommit_hugepages for a > > specific size > > Is this interface extensible to allow nodes to be managed specially. Not yet and you are right that we'll need to. I'll make some updates. Thanks for your initial review. -- Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
