On (09/10/08 20:20), Andy Whitcroft didst pronounce:
> From: Adam Litke <[EMAIL PROTECTED]>
>
> Now that libhugetlbfs supports multiple huge page sizes, it would be nice to
> have a way to discover the huge page sizes that are supported by the system.
> This new call should adhere to the semantics established by precedent
> implementations (see
> http://docs.sun.com/app/docs/doc/816-5168/getpagesizes-3c?a=view). The
> following patch implements a gethugepagesizes() call with these prevailing
> semantics as described below:
>
> NAME
> gethugepagesizes() - Get the system supported huge page sizes
>
> SYNOPSIS
> #include <hugetlbfs.h>
>
> int gethugepagesizes(long pagesizes[], int n_elem)
>
> DESCRIPTION
> The gethugepagesizes() function returns either the number of system
> supported huge page sizes or the sizes themselves. If pagesizes is
> NULL and n_elem is 0, then the number of huge pages the system supports
> is returned. Otherwise, pagesizes is filled with at most n_elem page
> sizes.
>
> RETURN VALUE
> On success, either the number of huge page sizes supported by the sys-
> tem or the number of huge page sizes stored in pagesizes is returned.
> On failure, -1 is returned and errno is set appropriately.
>
> ERRORS
> EINVAL n_elem is less than zero or n_elem is greater than zero and
> pagesizes is NULL.
>
> Also see opendir(3) for other possible values for errno.
>
> Open discussion points:
>
> This call will return all huge page sizes as reported by the kernel. Not all
> of these sizes may be usable by the programmer since mount points may not be
> available for all sizes. To test whether a size will be usable by
> libhugetlbfs, hugetlbfs_find_path_for_size() can be called on a specific size
> to see if a mount point is configured. I think this requirement is sensible.
> The only problem I see is that hugetlbfs_find_path_for_size() is not an
> obvious
> function name for this usage.
>
> Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugetlbfs.h | 1 +
> hugeutils.c | 65 +++++++++++++
> tests/Makefile | 3 +-
> tests/gethugepagesizes.c | 226
> ++++++++++++++++++++++++++++++++++++++++++++++
> tests/run_tests.sh | 1 +
> 5 files changed, 295 insertions(+), 1 deletions(-)
> create mode 100644 tests/gethugepagesizes.c
>
> diff --git a/hugetlbfs.h b/hugetlbfs.h
> index fbff43b..985e138 100644
> --- a/hugetlbfs.h
> +++ b/hugetlbfs.h
> @@ -22,6 +22,7 @@
> #define HUGETLBFS_MAGIC 0x958458f6
>
> long gethugepagesize(void);
> +int gethugepagesizes(long pagesizes[], int n_elem);
> int hugetlbfs_test_path(const char *mount);
> long hugetlbfs_test_pagesize(const char *mount);
> const char *hugetlbfs_find_path(void);
> diff --git a/hugeutils.c b/hugeutils.c
> index 21f788d..a48fa36 100644
> --- a/hugeutils.c
> +++ b/hugeutils.c
> @@ -552,6 +552,71 @@ long gethugepagesize(void)
> return hpage_size;
> }
>
> +int gethugepagesizes(long pagesizes[], int n_elem)
> +{
> + long default_size;
> + DIR *sysfs;
> + struct dirent *ent;
> + int nr_sizes = 0;
> +
> + if (n_elem < 0) {
> + errno = EINVAL;
> + return -1;
> + }
> +
> + if (n_elem > 0 && pagesizes == NULL) {
> + errno = EINVAL;
> + return -1;
> + }
> +
> + errno = 0;
> +
> + /* Get the system default size from /proc/meminfo */
> + default_size = read_meminfo("Hugepagesize:") * 1024;
> + if (default_size < 0)
> + return 0;
> +
> + if (n_elem && pagesizes)
> + pagesizes[nr_sizes] = default_size;
> + nr_sizes++;
> + if (n_elem && (nr_sizes == n_elem))
> + return nr_sizes;
> +
> + /*
> + * Scan sysfs to look for other sizes.
> + * Non-existing dir is not an error, we got one size from /proc/meminfo.
> + */
> + sysfs = opendir(SYSFS_HUGEPAGES_DIR);
> + if (!sysfs) {
> + if (errno == ENOENT) {
> + errno = 0;
> + return nr_sizes;
> + } else
> + return -1;
If the sysfs entry exists but we fail to open it, errno will be set to
something specified by opendir and we fail. Failing I agree with but
maybe errno should be set to EACCESS to indicate that other pagesizes
existed but we couldn't read them?
> + }
> + while ((ent = readdir(sysfs)) &&
> + ((n_elem == 0) || (nr_sizes < n_elem))) {
> + long size;
> +
> + if (strncmp(ent->d_name, "hugepages-", 10))
> + continue;
> +
> + size = strtol(ent->d_name + 10, NULL, 10);
> + if (size == LONG_MIN || size == LONG_MAX)
> + continue;
> + size *= 1024; /* Convert from KB to Bytes */
> +
> + if (size == default_size)
> + continue;
> + if (n_elem && pagesizes)
> + pagesizes[nr_sizes] = size;
> + nr_sizes++;
> + }
> + closedir(sysfs);
> +
> + return nr_sizes;
> +}
> +
> int hugetlbfs_test_path(const char *mount)
> {
> struct statfs64 sb;
> diff --git a/tests/Makefile b/tests/Makefile
> index beee18e..fdbe864 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -7,7 +7,8 @@ LIB_TESTS = gethugepagesize test_root find_path unlinked_fd
> misalign \
> truncate_reserve_wraparound truncate_sigbus_versus_oom \
> map_high_truncate_2 truncate_above_4GB direct \
> misaligned_offset brk_near_huge task-size-overrun stack_grow_into_huge \
> - counters quota heap-overflow get_huge_pages shmoverride_linked
> + counters quota heap-overflow get_huge_pages shmoverride_linked \
> + gethugepagesizes
> LIB_TESTS_64 = straddle_4GB huge_at_4GB_normal_below \
> huge_below_4GB_normal_above
> NOLIB_TESTS = malloc malloc_manysmall dummy heapshrink shmoverride_unlinked
> diff --git a/tests/gethugepagesizes.c b/tests/gethugepagesizes.c
> new file mode 100644
> index 0000000..34e9957
> --- /dev/null
> +++ b/tests/gethugepagesizes.c
> @@ -0,0 +1,226 @@
> +/*
> + * libhugetlbfs - Easy use of Linux hugepages
> + * Copyright (C) 2008 Adam Litke, IBM Corporation.
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public License
> + * as published by the Free Software Foundation; either version 2.1 of
> + * the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#define _GNU_SOURCE
> +
> +#include <stdlib.h>
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <dirent.h>
> +#include <dlfcn.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <hugetlbfs.h>
> +
> +#include "hugetests.h"
> +
> +#define REAL_SYSFS_DIR "/sys/kernel/mm/hugepages/"
> +char fake_sysfs[] = "/tmp/sysfs-XXXXXX";
> +DIR *(*real_opendir)(const char *name);
> +int cleanup_dir = 0;
> +
> +long (*real_read_meminfo)(const char *tag);
> +enum {
> + OVERRIDE_OFF, /* Pass-through to real function */
> + OVERRIDE_ON, /* Ovewrride with local function */
> + OVERRIDE_MISSING, /* Emulate missing support */
> +};
> +int meminfo_state = OVERRIDE_OFF;
> +int sysfs_state = OVERRIDE_OFF;
> +
> +/*
> + * Override opendir so we'll open the fake sysfs dir if intended
> + */
> +DIR *opendir(const char *name)
> +{
> + if (!real_opendir)
> + real_opendir = dlsym(RTLD_NEXT, "opendir");
> +
> + /* Only override calls to the sysfs dir */
> + if (strcmp(name, REAL_SYSFS_DIR))
> + return real_opendir(name);
> +
> + switch (sysfs_state) {
> + case OVERRIDE_OFF:
> + return real_opendir(name);
> + case OVERRIDE_ON:
> + /* Only safe to override of fake_sysfs was set up */
> + if (cleanup_dir)
> + return real_opendir(fake_sysfs);
> + else
> + FAIL("Trying to override opendir before initializing "
> + "fake_sysfs directory\n");
> + default:
> + errno = ENOENT;
> + return NULL;
> + }
> +}
> +
> +/*
> + * Override read_meminfo to simulate various conditions
> + */
> +long read_meminfo(const char *tag)
> +{
> + if (!real_read_meminfo)
> + real_read_meminfo = dlsym(RTLD_NEXT, "read_meminfo");
> +
> + /* Only override calls that check the page size */
> + if (strcmp(tag, "Hugepagesize:"))
> + return real_read_meminfo(tag);
> +
> + switch (meminfo_state) {
> + case OVERRIDE_OFF: return real_read_meminfo(tag);
> + case OVERRIDE_ON: return 16 * 1024;
> + default: return -1;
> + }
> +}
> +
> +void cleanup_fake_sysfs(void)
> +{
> + DIR *dir;
> + struct dirent *ent;
> + char fname[PATH_MAX+1];
> +
> + cleanup_dir = 0;
> + dir = real_opendir(fake_sysfs);
> + if (!dir)
> + FAIL("opendir %s: %s", fake_sysfs, strerror(errno));
> +
> + while ((ent = readdir(dir))) {
> + if (strncmp(ent->d_name, "hugepages-", 10))
> + continue;
> + snprintf(fname, PATH_MAX, "%s/%s", fake_sysfs,
> + ent->d_name);
> + if (rmdir(fname))
> + FAIL("rmdir %s: %s", fake_sysfs, strerror(errno));
> + }
> + closedir(dir);
> + if (rmdir(fake_sysfs))
> + FAIL("rmdir %s: %s", fake_sysfs, strerror(errno));
> +}
> +
> +void setup_fake_sysfs(long sizes[], int n_elem)
> +{
> + int i;
> + char fname[PATH_MAX+1];
> +
> + if (cleanup_dir)
> + cleanup_fake_sysfs();
> +
> + if (!mkdtemp(fake_sysfs))
> + FAIL("mkdtemp: %s", strerror(errno));
> + cleanup_dir = 1;
> +
> + for (i = 0; i < n_elem; i++) {
> + snprintf(fname, PATH_MAX, "%s/hugepages-%lukB", fake_sysfs,
> + sizes[i] / 1024);
> + if (mkdir(fname, 0700))
> + FAIL("mkdir %s: %s", fname, strerror(errno));
> + }
> +}
> +
> +void cleanup(void)
> +{
> + if (cleanup_dir)
> + cleanup_fake_sysfs();
> +}
> +
> +#define INIT_SIZES(a, v1, v2, v3) {a[0] = v1; a[1] = v2; a[2] = v3; a[3] =
> -1;}
> +
> +void expect_sizes(int line, int expected, int actual,
> + long expected_sizes[], long actual_sizes[])
> +{
> + int i, j;
> + if (expected != actual)
> + FAIL("Line %i: Wrong number of sizes returned -- expected %i "
> + "got %i", line, expected, actual);
> +
> + for (i = 0; i < expected; i++) {
> + for (j = 0; j < actual; j++)
> + if (expected_sizes[i] == actual_sizes[j])
> + break;
> + if (j >= actual)
> + FAIL("Line %i: Expected size %li not found in actual "
> + "results", line, expected_sizes[i]);
> + }
> +}
> +
> +int main(int argc, char *argv[])
> +{
> + long expected_sizes[4], actual_sizes[4], meminfo_size;
> + int nr_sizes;
> +
> + test_init(argc, argv);
> +
> + /*
> + * ===
> + * Argment error checking tests
> + * ===
> + */
> + meminfo_state = OVERRIDE_OFF;
> + sysfs_state = OVERRIDE_OFF;
> + if (gethugepagesizes(actual_sizes, -1) != -1 || errno != EINVAL)
> + FAIL("Mishandled params (n_elem < 0)");
> + if (gethugepagesizes(NULL, 1) != -1 || errno != EINVAL)
> + FAIL("Mishandled params (pagesizes == NULL, n_elem > 0)");
> +
> + /*
> + * ===
> + * Test some corner cases using a fake system configuration
> + * ===
> + */
> +
> + /*
> + * Check handling when /proc/meminfo indicates no huge page support
> + */
> + meminfo_state = OVERRIDE_MISSING;
> + if (gethugepagesizes(actual_sizes, 1) != 0 || errno != 0)
> + FAIL("Incorrect handling when huge page support is missing");
> +
> + /*
> + * When the sysfs heirarchy is not present ...
> + */
> + sysfs_state = OVERRIDE_MISSING;
> +
> + /* ... only the meminfo size is returned. */
> + meminfo_state = OVERRIDE_ON;
> + meminfo_size = read_meminfo("Hugepagesize:") * 1024;
> + INIT_SIZES(expected_sizes, meminfo_size, -1, -1);
> +
> + /* Use 2 to give the function the chance to return too many sizes */
> + nr_sizes = gethugepagesizes(actual_sizes, 2);
> + expect_sizes(__LINE__, 1, nr_sizes, expected_sizes, actual_sizes);
> +
> + /*
> + * When sysfs defines additional sizes ...
> + */
> + INIT_SIZES(expected_sizes, meminfo_size, 1024, 2048);
> + setup_fake_sysfs(expected_sizes, 3);
> + sysfs_state = OVERRIDE_ON;
> +
> + /* ... make sure all sizes are returned without duplicates */
> + nr_sizes = gethugepagesizes(actual_sizes, 4);
> + expect_sizes(__LINE__, 3, nr_sizes, expected_sizes, actual_sizes);
> +
> + /* ... we can check how many sizes are supported. */
> + if (gethugepagesizes(NULL, 0) != 3)
> + FAIL("Unable to check the number of supported sizes");
> +
> + PASS();
> +}
> diff --git a/tests/run_tests.sh b/tests/run_tests.sh
> index d616731..4fe6eed 100755
> --- a/tests/run_tests.sh
> +++ b/tests/run_tests.sh
> @@ -249,6 +249,7 @@ functional_tests () {
>
> # Library tests requiring kernel hugepage support
> run_test gethugepagesize
> + run_test gethugepagesizes
> run_test HUGETLB_VERBOSE=1 empty_mounts
> run_test HUGETLB_VERBOSE=1 large_mounts
>
Other than errno possibly being something stupid due to opendir()
failing
Acked-by: Mel Gorman <[EMAIL PROTECTED]>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
