On (03/10/08 18:36), Andy Whitcroft didst pronounce: > Currently the admin has to understand how to configure pools directly. > They must know how to find the current pool allocations, and how to > manipulate them; which change from kernel level to kernel level and > may even differ from one pool size to another. They must know how to > mount filesystems to expose those pools and where there are more than > one filesystem of a pariticular size they must know how to ensure the > underlying pools are sizes appropriatly. > > It is into this void we hope to place hpoolcfg. The underlying plan is > that there should be one application which would simplify control of the > managment of huge pages. That the admin would be able to say things > like "I need 10 more 16MB pages", or even "make me a 16MB page backed > filesystem with 10 pages for oracle to use, and another for the batch > sytem with between 10 and 20 pages" and the pools would be resized as > required and appropraite filesystems would be created. > > Obviously at this point we are only at the foundation building phase, > this first step concentrates on page sizes and pool configuration. > For page sizes it is possible to query both the currently available and > all supported page sizes. It exposes the pool configuration in a simple > to understand form and allows the admin to manipulate the pool sizes, both > in terms of static and overcommit pages. This is both useful directly > for a more knowledgable admin and provides the interfaces needed for later > functionality. > > Page sizes are expose as a simple list of page sizes in bytes: > > [EMAIL PROTECTED] ./obj/hpoolcfg --page-sizes > 2097152 > 65536 > [EMAIL PROTECTED] ./obj/hpoolcfg --page-sizes-all > 2097152 > 65536 > 16777216 >
Is the distinction here between pagesizes supported by the processor and those that are available via mounts at the moment? Also, Solaris provides a pagesize utility for printing pagesizes. By default, it prints the base page size and with -a, prints all the available pagesizes (see https://www.cebitec.uni-bielefeld.de/cgi-bin/man.cgi?section=1&topic=pagesize). Rather than making this part of hpoolcfg, would it make sense to add a pagesize utility with matching behaviour? That would keep hpoolcfg confined to actual configuration printing and changes. > The pool is exposed as a simple table. This represents the static > allocation (Minimum), the current actual alloction which includes any > overcommit pages in use (Current), and the total pages which may be > allocated including any overcommit (Maximum): > > [EMAIL PROTECTED] ./obj/hpoolcfg --pool-list > Size Minimum Current Maximum > 2097152 4 4 4 > 65536 15 15 30 > 16777216 0 0 0 > How is this ordered? To me, it would seem natural to order them in pagesize. > Pool manipulation is exposed as both abolute and relative updates to the > virtual Minimum and Maximum exposed above: > > [EMAIL PROTECTED] ./obj/hpoolcfg --pool-minimum-adjust 64kb=5 \ > --pool-maximum-adjust 64kb=10 > [EMAIL PROTECTED] ./obj/hpoolcfg --pool-list > Size Minimum Current Maximum > 2097152 4 4 4 > 65536 5 5 10 > 16777216 0 0 0 > > They may be adjusted with relative deltas: > [EMAIL PROTECTED] ./obj/hpoolcfg --pool-maximum-adjust 64kb=+5 64kb+=5 ? to match what you would write in C for example. > [EMAIL PROTECTED] ./obj/hpoolcfg --pool-list > Size Minimum Current Maximum > 2097152 4 4 4 > 65536 5 5 15 > 16777216 0 0 0 > > This code is not intended for merging as it is, but more as a work in > progress allowing discussion of the overall strategy and direct discussion > of the interfaces and the semantics they provide. Any comments on the > direction greatly appreciated. > > Following this email are 15 patches. The first 4 patches are a rebase > of Adam Litke's recent counter management stack, this was rebased onto > the current master and the update of the Makefile was consolidated. The > 5th patch fixes a bug in the accessors: > > Do not allow zero as an explicit page size > utils: Make pool counter manipulation code part of the library API > shm: Shared memory always uses the meminfo page size > lib: Add gethugepagesizes() API call > hugeutils: pool accessors should compare default page size in KB > > The remaining patches represent the new hpoolcfg itself: > > hugeutils: size_to_smaller_unit should allow 2GB pages > hugeutils: pull out common filenames > hugeutils: add a TEST_ROOT prefix for testing sysfs/proc functionality > hpoolcfg: allow utilities to consist of more than one file > debug: allow utilities to share the common debug reporter > hpoolcfg: initial basic framework > hpoolcfg: expose the pool configuration > hpoolcfg: expose the available and possible page sizes > hugeutils: export the pool counter update interface > hpoolcfg: allow pools to be resized > > Comments? > > -apw > > Adam Litke (4): > Do not allow zero as an explicit page size > utils: Make pool counter manipulation code part of the library API > shm: Shared memory always uses the meminfo page size > lib: Add gethugepagesizes() API call > > Andy Whitcroft (11): > hugeutils: pool accessors should compare default page size in KB > hugeutils: size_to_smaller_unit should allow 2GB pages > hugeutils: pull out common filenames > hugeutils: add a TEST_ROOT prefix for testing sysfs/proc > functionality > hpoolcfg: allow utilities to consist of more than one file > debug: allow utilities to share the common debug reporter > hpoolcfg: initial basic framework > hpoolcfg: expose the pool configuration > hpoolcfg: expose the available and possible page sizes > hugeutils: export the pool counter update interface > hpoolcfg: allow pools to be resized > > Makefile | 22 ++- > hpoolcfg.c | 249 +++++++++++++++++++++++++ > hugetlbfs.h | 17 ++ > hugeutils.c | 345 > +++++++++++++++++++++++++++++++---- > libhugetlbfs_internal.h | 14 ++- > shm.c | 7 +- > tests/Makefile | 7 +- > tests/alloc-instantiate-race.c | 5 +- > tests/chunk-overcommit.c | 3 +- > tests/counters.c | 38 ++-- > tests/get_huge_pages.c | 14 +- > tests/gethugepagesizes.c | 226 +++++++++++++++++++++++ > tests/heap-overflow.c | 2 +- > tests/hugetests.h | 20 +-- > tests/libtestutils.c | 137 ++++++++++++++ > tests/misaligned_offset.c | 20 +- > tests/quota.c | 6 +- > tests/run_tests.sh | 1 + > tests/shmoverride_unlinked.c | 82 +++++++- > tests/testutils.c | 252 +------------------------- > tests/truncate_reserve_wraparound.c | 21 +-- > tests/truncate_sigbus_versus_oom.c | 3 +- > 22 files changed, 1109 insertions(+), 382 deletions(-) > create mode 100644 hpoolcfg.c > create mode 100644 tests/gethugepagesizes.c > create mode 100644 tests/libtestutils.c > -- Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
