On Wed, 2008-07-30 at 11:07 +0100, Mel Gorman wrote:
> There are applications that are not hugepage-aware but use shared memory such
> as the postgres database. It is possible that the SHM_HUGETLB flag could
> be added to such an application via libhugetlbfs and overriding shmget()
> to add the SHM_HUGETLB to the flags. This patch enables libhugetlbfs to add
> the SHM_HUGETLB when a HUGETLB_SHM environment variable is set to "yes".
> It will work whether the application is linked to libhugetlbfs or loaded
> via LD_PRELOAD.
>
> Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
> ---
>
> HOWTO | 32 ++++++++++++++++++
> Makefile | 4 +-
> shm.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++
> tests/Makefile | 7 ++--
> tests/run_tests.sh | 4 ++
> 5 files changed, 125 insertions(+), 4 deletions(-)
This patch seems to have missed the test case.
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> libhugetlbfs-gitlatest-0010-private-export/HOWTO
> libhugetlbfs-gitlatest-0020-shmget-override/HOWTO
> --- libhugetlbfs-gitlatest-0010-private-export/HOWTO 2008-07-29
> 19:52:52.000000000 +0100
> +++ libhugetlbfs-gitlatest-0020-shmget-override/HOWTO 2008-07-30
> 11:01:03.000000000 +0100
> @@ -275,6 +275,38 @@ By default, the hugepage heap does not s
> shrinking, set HUGETLB_MORECORE_SHRINK=yes. NB: We have been seeing some
> unexpected behavior from glibc's malloc when this is enabled.
>
> +Using hugepage shared memory
> +----------------------------
> +
> +Hugepages are used for shared memory segments if the SHM_HUGETLB flag is
> +set when calling shmget() and the pool is large enough. For hugepage-unaware
> +applications, libhugetlbfs overrides shmget and adds the SHM_HUGETLB if the
> +environment variable HUGETLB_SHM is set to "yes". The steps to use hugepages
> +with applications not linked to libhugetlbfs are similar to morecore except
> +for step 3.
> +
> +1. Set LD_PRELOAD=libhugetlbfs.so
> + This tells the dynamic linker to load the libhugetlbfs shared
> + library, even though the program wasn't originally linked against it.
> +
> + Note: If the program is linked against libhugetlbfs, preloading the
> + library may lead to application crashes. You should skip this
> + step in that case.
> +
> +2. Set LD_LIBRARY_PATH to the directory containing libhugetlbfs.so
> + This is only necessary if you haven't installed libhugetlbfs.so to a
> + system default path. If you set LD_LIBRARY_PATH, make sure the
> + directory referenced contains the right version of the library
> + (32-bit or 64-bit) as appropriate to the binary you want to run.
> +
> +3. Set HUGETLB_SHM=yes
> + The shmget() call is overridden whether the application is linked or the
> + libhugetlbfs library is preloaded. When this environment variable is set,
> + the SHM_HUGETLB flag is added to the call and the size parameter is
> aligned
> + to back the shared memory segment with huge pages. In the event hugepages
> + cannot be used, small pages will be used instead and a warning will be
> + printed to explain the failure.
> +
> Using hugepage text, data, or BSS
> ---------------------------------
>
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> libhugetlbfs-gitlatest-0010-private-export/Makefile
> libhugetlbfs-gitlatest-0020-shmget-override/Makefile
> --- libhugetlbfs-gitlatest-0010-private-export/Makefile 2008-07-29
> 19:52:52.000000000 +0100
> +++ libhugetlbfs-gitlatest-0020-shmget-override/Makefile 2008-07-30
> 09:12:08.000000000 +0100
> @@ -1,7 +1,7 @@
> PREFIX = /usr/local
> EXEDIR = /bin
>
> -LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
> +LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o shm.o
> INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
> BIN_OBJ_DIR=obj
> INSTALL_BIN = hugectl hugeedit
> @@ -16,7 +16,7 @@ NODEPTARGETS=<version.h> <clean>
>
> INSTALL = install
>
> -LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds
> +LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds -ldl
> CFLAGS ?= -O2 -g
> CFLAGS += -Wall -fPIC
> CPPFLAGS += -D__LIBHUGETLBFS__
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> libhugetlbfs-gitlatest-0010-private-export/shm.c
> libhugetlbfs-gitlatest-0020-shmget-override/shm.c
> --- libhugetlbfs-gitlatest-0010-private-export/shm.c 2008-07-30
> 09:48:17.000000000 +0100
> +++ libhugetlbfs-gitlatest-0020-shmget-override/shm.c 2008-07-30
> 11:04:01.000000000 +0100
> @@ -0,0 +1,82 @@
> +/*
> + * libhugetlbfs - Easy use of Linux hugepages
> + * Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation.
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public License
> + * as published by the Free Software Foundation; either version 2.1 of
> + * the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +#define _GNU_SOURCE
> +#include <dlfcn.h>
> +#include <errno.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <sys/ipc.h>
> +#include <sys/shm.h>
> +#include <sys/types.h>
> +#include "libhugetlbfs_internal.h"
> +#include "hugetlbfs.h"
> +
> +int shmget(key_t key, size_t size, int shmflg)
> +{
> + static int (*real_shmget)(key_t key, size_t size, int shmflg) = NULL;
> + char *error;
> + int retval;
> + char *hugetlbshm_env;
> + size_t aligned_size = size;
> + int hugetlbshm_enabled = 0;
> +
> + DEBUG("hugetlb_shmem: entering overridden shmget() call\n");
> +
> + /* Get a handle to the "real" shmget system call */
> + if (!real_shmget) {
> + real_shmget = dlsym(RTLD_NEXT, "shmget");
> + if ((error = dlerror()) != NULL) {
> + ERROR("%s", error);
> + return -1;
> + }
> + }
> +
> + /* Determine if shmget() calls should be overridden */
> + hugetlbshm_env = getenv("HUGETLB_SHM");
> + if (hugetlbshm_env && !strcmp(hugetlbshm_env, "yes"))
> + hugetlbshm_enabled = 1;
> +
> + /* Align the size and set SHM_HUGETLB on request */
> + if (hugetlbshm_enabled) {
> + aligned_size = ALIGN(size, gethugepagesize());
> + if (size != aligned_size) {
> + DEBUG("hugetlb_shmem: size growth align %zd -> %zd\n",
> + size, aligned_size);
> + }
> +
> + DEBUG("hugetlb_shmem: Adding SHM_HUGETLB flag\n");
> + shmflg |= SHM_HUGETLB;
> + } else {
> + DEBUG("hugetlb_shmem: shmget override not requested\n");
> + }
> +
> + /* Call the "real" shmget. If hugepages fail, use small pages */
> + retval = real_shmget(key, aligned_size, shmflg);
> + if (retval == -1 && hugetlbshm_enabled) {
> + WARNING("While overriding shmget(%zd) to add SHM_HUGETLB: %s\n",
> + aligned_size, strerror(errno));
> + shmflg &= ~SHM_HUGETLB;
> + retval = real_shmget(key, size, shmflg);
> + DEBUG("Using small pages for shmget despite HUGETLB_SHM\n");
> + }
> +
> + return retval;
> +}
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> libhugetlbfs-gitlatest-0010-private-export/tests/Makefile
> libhugetlbfs-gitlatest-0020-shmget-override/tests/Makefile
> --- libhugetlbfs-gitlatest-0010-private-export/tests/Makefile 2008-07-29
> 19:52:52.000000000 +0100
> +++ libhugetlbfs-gitlatest-0020-shmget-override/tests/Makefile
> 2008-07-30 09:11:38.000000000 +0100
> @@ -7,10 +7,10 @@ LIB_TESTS = gethugepagesize test_root fi
> truncate_reserve_wraparound truncate_sigbus_versus_oom \
> map_high_truncate_2 truncate_above_4GB direct \
> misaligned_offset brk_near_huge task-size-overrun stack_grow_into_huge \
> - counters quota heap-overflow get_huge_pages
> + counters quota heap-overflow get_huge_pages shmoverride_linked
> LIB_TESTS_64 = straddle_4GB huge_at_4GB_normal_below \
> huge_below_4GB_normal_above
> -NOLIB_TESTS = malloc malloc_manysmall dummy heapshrink
> +NOLIB_TESTS = malloc malloc_manysmall dummy heapshrink shmoverride_unlinked
> LDSCRIPT_TESTS = zero_filesize_segment
> HUGELINK_TESTS = linkhuge linkhuge_nofd linkshare
> HUGELINK_RW_TESTS = linkhuge_rw
> @@ -67,6 +67,9 @@ endif
>
> all: $(ALLTESTS) $(ALLHELPERS) $(ALLHELPERLIBS)
>
> +shmoverride_linked.c: shmoverride_unlinked.c
> + ln -s shmoverride_unlinked.c shmoverride_linked.c
> +
> obj32/%.o: %.c
> @$(VECHO) CC32 $@
> @mkdir -p obj32
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> libhugetlbfs-gitlatest-0010-private-export/tests/run_tests.sh
> libhugetlbfs-gitlatest-0020-shmget-override/tests/run_tests.sh
> --- libhugetlbfs-gitlatest-0010-private-export/tests/run_tests.sh
> 2008-07-29 19:52:52.000000000 +0100
> +++ libhugetlbfs-gitlatest-0020-shmget-override/tests/run_tests.sh
> 2008-07-30 09:11:38.000000000 +0100
> @@ -275,6 +275,10 @@ check_linkhuge_tests
> # Test direct allocation API
> run_test get_huge_pages
>
> +# Test overriding of shmget()
> + run_test shmoverride_linked
> + run_test LD_PRELOAD=libhugetlbfs.so shmoverride_unlinked
> +
> # Test hugetlbfs filesystem quota accounting
> run_test quota
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Libhugetlbfs-devel mailing list
> Libhugetlbfs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
>
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
