Latest version of the daemon killing patch. I've cleaned up the
previously rather contorted code paths for sharing in elflink.c. Next
step per-user hugetlbfs directories. For sure, this time ;-).
Index: libhugetlbfs/HOWTO
===================================================================
--- libhugetlbfs.orig/HOWTO 2006-10-26 13:49:40.000000000 +1000
+++ libhugetlbfs/HOWTO 2006-10-26 13:49:45.000000000 +1000
@@ -326,42 +326,26 @@ libhugetlbfs:
Sharing remapped segments:
--------------------------
-Sharing of program segments among processes is a two-fold process.
-
-First, a system-wide daemon must be started, hugetlbd. The executable
-(which will be installed into $PREFIX/sbin by make install) should be
-run as root and will automatically spawn a child process to wait for
-segment sharing into the background. The process will log any output
-requested to /tmp/hugetlbd.log. The complete usage for hugetlbd is:
-
-hugetlbd [-v] [-V verbosity] [-p timeout1] [-s timeout2]
- -v: display this help and exit
- -V: specify log level (1-3) [default=1]
- 1: only log errors
- 2: log errors and warnings
- 3: log all output
- -p: specify how long to wait before unlinking files in the hugetlbfs
- mountpoint (seconds) [default=600]
- -s: specify when a file last needs to have been shared for it not to be
- unlinked (seconds) [default=600]
-
-Second, in the applications that wish to share segments, the
-HUGETLB_SHARE environment variable must be set appropriately. The
-variable takes on three values: 0, the default, indicates no segments
-should be shared; 1 indicates that only read-only segments (text) should
-be shared; and 2 indicates that all segments should be shared. Note, as
-this is a per-process variable, it is important to keep its usage
-consistent with HUGETLB_MINIMAL_COPY. If MINIMAL_COPY is used by the
-first process to share a particular executable's segments, it should be
-used by all subsequent ones, to guarantee no failures while copying
-segments. Similarly, if MINIMAL_COPY is not used by the first process
-sharing, it should not be used by subsequent ones.
-
-Having taken these two steps, the same application run twice with the
-same environment variable settings will use the same file both times
-(reducing the overall hugetlbfs file usage). If, for any reason, segment
-sharing is requested but unavailable, the library will fallback to
-unlinked files.
+Sharing of program segments among processes is a simple process. For
+the applications that wish to share segments, the HUGETLB_SHARE
+environment variable must be set appropriately. The variable takes on
+three values: 0, the default, indicates no segments should be shared;
+1 indicates that only read-only segments (text) should be shared; and
+2 indicates that all segments should be shared. Note, as this is a
+per-process variable, it is important to keep its usage consistent
+with HUGETLB_MINIMAL_COPY. If MINIMAL_COPY is used by the first
+process to share a particular executable's segments, it should be used
+by all subsequent ones, to guarantee no failures while copying
+segments. Similarly, if MINIMAL_COPY is not used by the first process
+sharing, it should not be used by subsequent ones. Additionally, as a
+security precaution, sharing is only possible between users in the
+same group.
+
+Now, the same application run twice with the same environment variable
+settings will use the same file both times (reducing the overall
+hugetlbfs file usage). If, for any reason, segment sharing is
+requested but unavailable, the library will fallback to unlinked
+files.
Examples
========
Index: libhugetlbfs/Makefile
===================================================================
--- libhugetlbfs.orig/Makefile 2006-10-26 13:49:40.000000000 +1000
+++ libhugetlbfs/Makefile 2006-10-26 13:49:45.000000000 +1000
@@ -2,7 +2,6 @@ PREFIX = /usr/local
BASEOBJS = hugeutils.o version.o
LIBOBJS = $(BASEOBJS) elflink.o morecore.o debug.o
-SBINOBJS = hugetlbd
INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
LDSCRIPT_TYPES = B BDT
LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
@@ -84,14 +83,13 @@ endif
DEPFILES = $(LIBOBJS:%.o=%.d)
-all: libs sbin tests
+all: libs tests
.PHONY: tests libs
libs: $(foreach file,$(INSTALL_OBJ_LIBS),$(OBJDIRS:%=%/$(file)))
-sbin: $(foreach file,$(SBINOBJS),$(OBJDIRS:%=%/$(file)))
-tests: libs sbin # Force make to build the library first
+tests: libs # Force make to build the library first
tests: tests/all
tests/%:
@@ -168,14 +166,6 @@ obj64/%.s: %.c
@$(VECHO) CC64 -S $@
$(CC64) $(CPPFLAGS) $(CFLAGS) -o $@ -S $<
-obj32/hugetlbd: hugetlbd.c $(BASEOBJS:%=obj32/%)
- @$(VECHO) CC32 $@
- $(CC32) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS32) -o $@ $^
-
-obj64/hugetlbd: hugetlbd.c $(BASEOBJS:%=obj64/%)
- @$(VECHO) CC64 $@
- $(CC64) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS64) -o $@ $^
-
clean:
@$(VECHO) CLEAN
rm -f *~ *.o *.so *.a *.d *.i core a.out $(VERSION)
Index: libhugetlbfs/elflink.c
===================================================================
--- libhugetlbfs.orig/elflink.c 2006-10-26 13:49:40.000000000 +1000
+++ libhugetlbfs/elflink.c 2006-10-26 16:48:49.000000000 +1000
@@ -1,6 +1,6 @@
/*
* libhugetlbfs - Easy use of Linux hugepages
- * Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation.
+ * Copyright (C) 2005-2006 David Gibson & dam Litke, IBM Corporation.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public License
@@ -24,17 +24,20 @@
#include <stdlib.h>
#include <malloc.h>
#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
#include <signal.h>
#include <sys/syscall.h>
+#include <sys/file.h>
#include <linux/unistd.h>
#include <sys/mman.h>
#include <errno.h>
+#include <limits.h>
#include <elf.h>
#include <dlfcn.h>
#include "hugetlbfs.h"
#include "libhugetlbfs_internal.h"
-
#ifdef __LP64__
#define Elf_Ehdr Elf64_Ehdr
#define Elf_Phdr Elf64_Phdr
@@ -179,9 +182,18 @@ static void unmapped_abort(const char *f
#define MAX_HTLB_SEGS 2
+struct seg_info {
+ void *vaddr;
+ unsigned long filesz, memsz;
+ int prot;
+ int fd;
+ int phdr;
+};
+
static struct seg_info htlb_seg_table[MAX_HTLB_SEGS];
static int htlb_num_segs;
static int minimal_copy = 1;
+static int sharing; /* =0 */
int __debug = 0;
static Elf_Ehdr *ehdr;
@@ -231,6 +243,7 @@ static void parse_phdrs(Elf_Ehdr *ehdr)
htlb_seg_table[htlb_num_segs].filesz = filesz;
htlb_seg_table[htlb_num_segs].memsz = memsz;
htlb_seg_table[htlb_num_segs].prot = prot;
+ htlb_seg_table[htlb_num_segs].phdr = i;
htlb_num_segs++;
}
}
@@ -249,13 +262,63 @@ static void check_bss(unsigned long *sta
}
}
+/**
+ * get_shared_file_name - create a shared file name from program name, gid and
phdr
+ * @htlb_seg_info: pointer to program's segment data
+ * @file_path: pointer to a PATH_MAX array to store filename in
+ *
+ * The file name created is *not* intended to be unique, except when the name,
+ * gid or phdr number differ. The goal here is to have a standar means of
+ * accessing particular segments of particular executables.
+ *
+ * returns:
+ * -1, on failure
+ * 0, on success
+ */
+static int get_shared_file_name(struct seg_info *htlb_seg_info, char
*file_path)
+{
+ int ret;
+ char binary[PATH_MAX];
+ char *binary2;
+ const char *hugetlbfs_path;
+
+ ret = readlink("/proc/self/exe", binary, PATH_MAX);
+ if (ret < 0) {
+ ERROR("shared_file: readlink() on /proc/self/exe "
+ "failed: %s\n", strerror(errno));
+ return -1;
+ }
+
+ binary2 = basename(binary);
+ if (!binary2) {
+ ERROR("shared_file: basename() on %s failed: %s\n",
+ binary, strerror(errno));
+ return -1;
+ }
+
+ hugetlbfs_path = hugetlbfs_find_path();
+ if (!hugetlbfs_path) {
+ ERROR("shared_file: hugetlbfs_find_path() failed\n");
+ return -1;
+ }
+
+ ret = snprintf(file_path, PATH_MAX, "%s/%s_%d_%zd_%d",
+ hugetlbfs_path, binary2, getgid(), sizeof(unsigned long) * 8,
+ htlb_seg_info->phdr);
+ if (ret < 0) {
+ ERROR("shared_file: snprintf() failed\n");
+ return -1;
+ }
+
+ return 0;
+}
+
/*
* Subtle: Since libhugetlbfs depends on glibc, we allow it
* it to be loaded before us. As part of its init functions, it
* initializes stdin, stdout, and stderr in the bss. We need to
* include these initialized variables in our copy.
*/
-
static void get_extracopy(struct seg_info *seg, void **extra_start,
void **extra_end)
{
@@ -387,7 +450,6 @@ bail3:
* smallest amount of data possible, unless the user disables this
* optimization via the HUGETLB_ELFMAP environment variable.
*/
-
static int prepare_segment(struct seg_info *seg)
{
int hpage_size = gethugepagesize();
@@ -437,6 +499,139 @@ static int prepare_segment(struct seg_in
return 0;
}
+/**
+ * find_or_prepare_shared_file - get one shareable file
+ * @htlb_seg_info: pointer to program's segment data
+ *
+ * returns:
+ * -1, on failure
+ * 0, on success
+ */
+static int find_or_prepare_shared_file(struct seg_info *htlb_seg_info)
+{
+ int fdx, fds;
+ int errnox, errnos;
+ int ret;
+ char final_path[PATH_MAX+1];
+ char tmp_path[PATH_MAX+5];
+
+ ret = get_shared_file_name(htlb_seg_info, final_path);
+ if (ret < 0)
+ return -1;
+ strcpy(tmp_path, final_path);
+ strcat(tmp_path, ".tmp");
+
+ do {
+ /* NB: mode is modified by umask */
+ fdx = open(tmp_path, O_CREAT | O_EXCL | O_RDWR, 0666);
+ errnox = errno;
+ fds = open(final_path, O_RDONLY);
+ errnos = errno;
+
+ if (fds > 0) {
+ /* Got an already-prepared file -> use it */
+ if (fdx > 0) {
+ /* Also got an exclusive file -> clean up */
+ ret = unlink(tmp_path);
+ if (ret != 0)
+ ERROR("shared_file: unable to clean up"
+ " unneeded file %s: %s\n",
+ tmp_path, strerror(errno));
+ close(fdx);
+ } else if (errnox != EEXIST) {
+ WARNING("shared_file: Unexpected failure on
exclusive"
+ " open of %s: %s\n", tmp_path,
+ strerror(errnox));
+ }
+ htlb_seg_info->fd = fds;
+ return 0;
+ }
+
+ if (fdx > 0) {
+ /* It's our job to prepare */
+ if (errnos != ENOENT)
+ WARNING("shared_file: Unexpected failure on"
+ " shared open of %s: %s\n", final_path,
+ strerror(errnos));
+
+ htlb_seg_info->fd = fdx;
+
+ DEBUG("Got unpopulated shared fd -- Preparing\n");
+ ret = prepare_segment(htlb_seg_info);
+ if (ret < 0)
+ goto fail;
+
+ DEBUG("Prepare succeeded\n");
+ /* move to permanent location */
+ ret = rename(tmp_path, final_path);
+ if (ret != 0) {
+ ERROR("shared_file: unable to rename %s"
+ " to %s: %s\n", tmp_path, final_path,
+ strerror(errno));
+ goto fail;
+ }
+
+ return 0;
+ }
+
+ /* Both opens failed, somebody else is still preparing */
+ /* Wait and try again */
+ sleep(1);
+ /* FIXME: should have a timeout */
+ } while (1);
+
+ fail:
+ if (fdx > 0) {
+ ret = unlink(tmp_path);
+ if (ret != 0)
+ ERROR("shared_file: Unable to clean up temp file %s on"
+ " failure: %s\n", tmp_path, strerror(errno));
+ close(fdx);
+ }
+ if (fds > 0)
+ close(fds);
+
+ return -1;
+}
+
+/**
+ * obtain_prepared_file - multiplex callers depending on if
+ * sharing or not
+ * @htlb_seg_info: pointer to program's segment data
+ *
+ * returns:
+ * -1, on error
+ * 0, on success
+ */
+static int obtain_prepared_file(struct seg_info *htlb_seg_info)
+{
+ int fd = -1;
+ int ret;
+
+ /* Either share all segments or share only read-only segments */
+ if ((sharing == 2) || ((sharing == 1) &&
+ !(htlb_seg_info->prot & PROT_WRITE))) {
+ /* first, try to share */
+ ret = find_or_prepare_shared_file(htlb_seg_info);
+ if (ret == 0)
+ return 0;
+ /* but, fall through to unlinked files, if sharing fails */
+ DEBUG("Falling back to unlinked files\n");
+ }
+ fd = hugetlbfs_unlinked_fd();
+ if (fd < 0)
+ return -1;
+ htlb_seg_info->fd = fd;
+
+ ret = prepare_segment(htlb_seg_info);
+ if (ret < 0) {
+ DEBUG("Failed to prepare segment\n");
+ return -1;
+ }
+ DEBUG("Prepare succeeded\n");
+ return 0;
+}
+
static void remap_segments(struct seg_info *seg, int num)
{
int hpage_size = gethugepagesize();
@@ -482,49 +677,6 @@ static void remap_segments(struct seg_in
*/
}
-static int maybe_prepare(int fd_state, struct seg_info *seg)
-{
- int ret, reply = 0;
-
- switch (fd_state) {
- case 0:
- DEBUG("Got populated shared fd -- Not preparing\n");
- return 0;
-
- case 1:
- DEBUG("Got unpopulated shared fd -- Preparing\n");
- reply = 1;
- break;
- case 2:
- DEBUG("Got unshared fd, wanted shared -- Preparing\n");
- break;
- case 3:
- DEBUG("Got unshared fd as expected -- Preparing\n");
- break;
-
- default:
- ERROR("Unexpected fd state: %d in maybe_prepare\n",
- fd_state);
- return -1;
- }
-
- ret = prepare_segment(seg);
- if (ret < 0) {
- /* notify daemon of failed prepare */
- DEBUG("Failed to prepare segment\n");
- finished_prepare(seg, -1);
- return -1;
- }
- DEBUG("Prepare succeeded\n");
- if (reply) {
- ret = finished_prepare(seg, 0);
- if (ret < 0)
- DEBUG("Failed to communicate successful "
- "prepare to hugetlbd\n");
- }
- return ret;
-}
-
static int check_env(void)
{
char *env;
@@ -550,6 +702,20 @@ static int check_env(void)
minimal_copy = 0;
}
+ env = getenv("HUGETLB_SHARE");
+ if (env)
+ sharing = atoi(env);
+ DEBUG("HUGETLB_SHARE=%d, sharing ", sharing);
+ if (sharing == 2) {
+ DEBUG_CONT("enabled for all segments\n");
+ } else {
+ if (sharing == 1) {
+ DEBUG_CONT("enabled for only read-only segments\n");
+ } else {
+ DEBUG_CONT("disabled\n");
+ }
+ }
+
env = getenv("HUGETLB_DEBUG");
if (env) {
DEBUG("HUGETLB_DEBUG=%s, enabling extra checking\n", env);
@@ -581,22 +747,13 @@ static void __attribute__ ((constructor)
return;
}
- /* Step 1. Get access to the files we're going to mmap for the
- * segments */
+ /* Step 1. Obtain hugepage files with our program data */
for (i = 0; i < htlb_num_segs; i++) {
- ret = hugetlbfs_set_fd(&htlb_seg_table[i]);
+ ret = obtain_prepared_file(&htlb_seg_table[i]);
if (ret < 0) {
DEBUG("Failed to setup hugetlbfs file\n");
return;
}
-
- /* Step 2. Map the hugetlbfs files anywhere to copy data, if we
- * need to prepare at all */
- ret = maybe_prepare(ret, &htlb_seg_table[i]);
- if (ret < 0) {
- DEBUG("Failed to prepare hugetlbfs file\n");
- return;
- }
}
/* Step 3. Unmap the old segments, map in the new ones */
Index: libhugetlbfs/hugetlbd.c
===================================================================
--- libhugetlbfs.orig/hugetlbd.c 2006-10-26 13:49:40.000000000 +1000
+++ /dev/null 1970-01-01 00:00:00.000000000 +0000
@@ -1,872 +0,0 @@
-/*
- * libhugetlbfs - Easy use of Linux hugepages
- * Copyright (C) 2006 Nishanth Aravamudan, IBM Corporation
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1 of
- * the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with this library; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#define _GNU_SOURCE
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <stdint.h>
-#include <errno.h>
-#include <getopt.h>
-#include <signal.h>
-#include <time.h>
-#include <unistd.h>
-#include <fcntl.h>
-
-#include <sys/mman.h>
-#include <sys/poll.h>
-#include <sys/socket.h>
-#include <sys/stat.h>
-#include <sys/types.h>
-#include <sys/un.h>
-
-#include "libhugetlbfs_internal.h"
-#include "hugetlbfs.h"
-#include "hugetlbd.h"
-
-#define QUEUE_LENGTH SOMAXCONN
-#define HUGETLBD_PORT 60046 /* some random number */
-/* how often to reap due to idleness */
-#define DEFAULT_POLL_TIMEOUT 600000 /* milliseconds */
-/* how long ago does a share need to have occurred to reap? */
-#define DEFAULT_SHARE_TIMEOUT 600 /* seconds */
-
-struct shared_mapping {
- /* unique identifier */
- exe_ident_t id;
- /* filename */
- char path_to_file[PATH_MAX+1];
- /* virtual address of segments */
- vaddr_t vaddr;
- /* timestamp for file cleanup */
- time_t timestamp;
- /* linked-list pointer */
- struct shared_mapping *next;
-};
-
-static struct shared_mapping *head;
-
-/* visible for multiple manipulation points */
-static int sock;
-/* visible for SIGALRM handling */
-static int timeout;
-/* visible for error handling */
-static int shared_fd;
-static FILE *logfd;
-static unsigned int share_timeout;
-static unsigned int poll_timeout;
-
-int __hugetlbfs_verbose = 0;
-
-static void usage()
-{
- printf("Usage: hugetlbd [-v] [-V verbosity] [-p timeout1] [-s
timeout2]\n"
- "Start libhugetlbfs daemon for sharing program segments\n"
- " -v: display this help and exit\n"
- " -V: specify log level (1-3) [default=1]\n"
- " 1: only log errors\n"
- " 2: log errors and warnings\n"
- " 3: log all output\n"
- " -p: specify how long to wait before unlinking files in the
hugetlbfs\n"
- " mountpoint (seconds) [default=600]\n"
- " -s: specify when a file last needs to have been shared for
it not to\n"
- " be unlinked (seconds) [default=600]\n"
- "Report bugs to [EMAIL PROTECTED]");
-}
-
-/**
- * reap_files - unlink all shared files
- * @quitting: 1 if exiting daemon
- *
- * no return value
- */
-static void reap_files(int quitting)
-{
- struct shared_mapping *smptr, *nextsmptr;
- time_t now = time(NULL);
-
- DEBUG("Reaping files\n");
- smptr = head;
- /* list is sorted by timestamp, youngest is at head */
- while (smptr != NULL) {
- /*
- * Iterate through list until we get to an old enough
- * sharing
- */
- if (!quitting && (now - smptr->timestamp < share_timeout)) {
- smptr = smptr->next;
- continue;
- }
- /* keep pointer to the next one */
- nextsmptr = smptr->next;
- unlink(smptr->path_to_file);
- /* if we're deleting head, head has to move */
- if (smptr == head) {
- head = nextsmptr;
- }
- free(smptr);
- smptr = nextsmptr;
- }
-}
-
-/**
- * kill_daemon - exit the daemon process
- * @ret: if 0, expected quit
- * 1, unexpected quit
- * also functions as return value to exit with
- *
- * no return value
- */
-static void kill_daemon(int ret)
-{
- DEBUG("Killing daemon\n");
- reap_files(1);
- /*
- * Close the socket here, because we might dying due to an
- * asynchronous signal.
- */
- close(sock);
- exit(ret);
-}
-
-/**
- * signal_handler - respond to catchable signals
- * @signal: the signal to handle
- *
- * no return value
- */
-static void signal_handler(int signal)
-{
- switch (signal) {
- case SIGHUP:
- /* reap files */
- DEBUG("Caught SIGHUP\n");
- reap_files(1);
- break;
- case SIGINT:
- /* expectedly quit */
- kill_daemon(0);
- break;
- case SIGALRM:
- timeout = 1;
- break;
- }
-}
-
-/**
- * idcpy - copy id from src to dst
- * @src: source id
- * @dst: destination id
- *
- * Would need to be changed if exe_ident_t is changed to not be a basic
- * type.
- *
- * no return value
- */
-static void idcpy(exe_ident_t *dst, exe_ident_t *src)
-{
- *dst = *src;
-}
-
-/**
- * idcmp - compare id1 to id2
- * @id1: first id
- * @dst: second id
- *
- * Would need to be changed if exe_ident_t is changed to not be a basic
- * type.
- *
- * returns: 0, if ids match
- * 1, if ids don't match
- */
-static int idcmp(exe_ident_t *i1, exe_ident_t *i2)
-{
- if (*i1 == *i2)
- return 0;
- else
- return 1;
-}
-
-/**
- * set_path_to_file - create new unique file in hugetlbfs
- * @tgt: target character array to store filename in
- * @size: size of @tgt
- *
- * Creates a new uniquely named file in the hugetlbfs mount point. File
- * is opened in mkstemp, and the file descriptor is stored in a globally
- * visible variable, because the caller's return value is already
- * overloaded.
- *
- * Effectively identical to get_unlinked_fd().
- *
- * no return value
- */
-static void set_path_to_file(char *tgt, int size)
-{
- const char *path;
- shared_fd = -1;
-
- tgt[size - 1] = '\0';
-
- path = hugetlbfs_find_path();
- if (!path) {
- shared_fd = -1;
- return;
- }
-
- strcpy(tgt, path);
- strncat(tgt, "/libhugetlbfs.tmp.XXXXXX", size - 1);
-
- shared_fd = mkstemp(tgt);
- if (shared_fd > 0) {
- /* prevent access to shared file */
- int ret = chmod(tgt, 0);
- if (ret < 0) {
- DEBUG("chmod failed\n");
- shared_fd = -1;
- }
- }
-}
-
-/**
- * do_shared_file - find shareable mapping, or create new one
- * @req: contains executable id and virtual addresses of segments
- * @resp: contains filename and whether prepare() needs to be run
- *
- * The core routine for sharing mappings. The request and response
- * structures allow for accepting many parameters and returning many
- * response values independent of the actual return value, which acts
- * kind of like metadata about @resp.
- *
- * returns: -1, if failed to create mapping
- * NULL, if sharing mapping already at head of list
- * pointer to previous entry in linked list, else
- */
-static struct shared_mapping *do_shared_file(struct client_request *creq,
- struct daemon_response *dresp)
-{
- struct shared_mapping *smptr, *prevsmptr;
-
- if (head == NULL) {
- /* allocating first member of linked list */
- goto create_mapping;
- }
-
- prevsmptr = NULL;
- smptr = head;
-
- while (smptr != NULL) {
- if (!idcmp(&(smptr->id), &(creq->id))
- && (smptr->vaddr == creq->vaddr))
- /* found match */
- goto share_mapping;
-
- /*
- * Keep track of the previous entry in the list, because
- * we're going to need to move the one we're currently
- * at. This prevents an extra loop later.
- */
- prevsmptr = smptr;
- smptr = smptr->next;
- }
-
-create_mapping:
- /* first attempt to map this segment, create a new mapping */
- smptr = (struct shared_mapping *)malloc(sizeof(struct shared_mapping));
- if (smptr == NULL) {
- ERROR("Failed to allocate new shared mapping: %s\n",
- strerror(errno));
- return (struct shared_mapping *)-1;
- }
-
- set_path_to_file(smptr->path_to_file, sizeof(smptr->path_to_file));
- if (shared_fd < 0) {
- ERROR("Failed to create new shared file\n");
- free(smptr);
- return (struct shared_mapping *)-1;
- }
-
- DEBUG("created shared mapping to %s\n", smptr->path_to_file);
-
- /* grab data out of request */
- idcpy(&(smptr->id), &(creq->id));
- smptr->vaddr = creq->vaddr;
- /* timestamp will be set upon insert */
-
- /* client will need to prepare file */
- dresp->need_to_prepare = 1;
- strcpy(dresp->path_to_file, smptr->path_to_file);
-
- /*
- * Cannot be NULL. Caller is responsible for freeing if an error
- * occurs
- */
- return smptr;
-
-share_mapping:
- /* store data in response */
- dresp->need_to_prepare = 0;
- strcpy(dresp->path_to_file, smptr->path_to_file);
-
- DEBUG("sharing mapping to %s\n", smptr->path_to_file);
-
- /*
- * Can be NULL, if sharing at the head of the list. Caller does
- * not need to deal with freeing, but must deal with both NULL
- * and non-NULL cases.
- * We are moving smptr to the front of the list if the sharing
- * succeeds, but send prevsmptr back so that we don't need to
- * iterate to do the move.
- */
- return prevsmptr;
-}
-
-static int parse_args(int argc, char *argv[]) {
- int opt;
- int ret = 0;
- while (1) {
- opt = getopt(argc, argv, ":p:s:dV:v");
- if (opt == -1)
- break;
-
- switch (opt) {
- case 'p':
- /* needs to be in milliseconds */
- poll_timeout = 1000*atoi(optarg);
- break;
- case 's':
- share_timeout = atoi(optarg);
- break;
- case 'd':
- /* do not daemonize */
- ret = 1;
- break;
- case 'V':
- __hugetlbfs_verbose = atoi(optarg);
- break;
- case 'v':
- usage();
- return -1;
- case ':':
- printf("getopt: missing parameter for option
%c\n", optopt);
- usage();
- return -1;
- case '?':
- printf("getopt: unknown option: %c\n", optopt);
- usage();
- return -1;
- default:
- printf("getopt: returned character code
0%o\n", opt);
- usage();
- return -1;
- }
- }
-
- if (poll_timeout == 0) {
- poll_timeout = DEFAULT_POLL_TIMEOUT;
- }
- if (share_timeout == 0) {
- share_timeout = DEFAULT_SHARE_TIMEOUT;
- }
-
- return ret;
-}
-
-static void daemonize()
-{
- pid_t pid, sid;
-
- pid = fork();
- if (pid < 0) {
- ERROR("daemonize: fork() failed\n");
- exit(1);
- }
- if (pid > 0)
- exit(0);
-
- umask(0);
-
- sid = setsid();
- if (sid < 0) {
- ERROR("daemonize: Failed to set session ID\n");
- exit(1);
- }
-
- if ((chdir("/")) < 0) {
- ERROR("daemonize: Failed to change directory\n");
- exit(1);
- }
-
- logfd = fopen(LOGFILE, "a+");
- if (!logfd) {
- ERROR("daemonize: Failed to open log file %s\n", LOGFILE);
- exit(1);
- }
-
- close(STDIN_FILENO);
- close(STDOUT_FILENO);
- close(STDERR_FILENO);
- stderr = logfd;
-}
-
-/**
- * read_client_request - receive request from client
- * @local_sock: socket to read from
- * @cresp: pointer to store request in
- *
- * Wait for client's request for sharing
- *
- * returns 0, on success
- * negative values, on failure
- */
-static int read_client_request(int local_sock, struct client_request
- *creq)
-{
- int ret;
- alarm(60);
- ret = read(local_sock, creq, sizeof(*creq));
- /* cancel any alarms */
- alarm(0);
- if (ret < 0) {
- /* timeout == 1 means that the SIGALRM handler ran */
- if (timeout) {
- DEBUG("read timed out\n");
- return -1;
- }
- ERROR("read failed: %s\n", strerror(errno));
- return -2;
- }
- if (ret != sizeof(*creq)) {
- /*
- * This should never happen, but is useful for
- * debugging size issues with mixed environments
- */
- DEBUG("short first read (got %d, should have been "
- "%zu\n", ret, sizeof(*creq));
- return -1;
- }
- return 0;
-}
-
-/**
- * write_daemon_response - respond to client
- * @local_sock: socket to write to
- * @cresp: pointer to response
- *
- * Acknowledge a client's request with a filename and whether the caller
- * needs to prepare
- *
- * returns 0, on success
- * negative values, on failure
- */
-static int write_daemon_response(int local_sock, struct daemon_response
- *dresp)
-{
- int ret;
- alarm(60);
- ret = write(local_sock, dresp, sizeof(*dresp));
- alarm(0);
- /*
- * If dresp.need_to_prepare is set, then we know we
- * dynamically allocated an element (which is pointed to
- * by smptr). We need to backout any changes it made to
- * the system state (open file, memory consumption) on
- * errors.
- */
- if (ret < 0) {
- if (timeout) {
- DEBUG("write timed out\n");
- if (dresp->need_to_prepare)
- return -1;
- return -2;
- }
- ERROR("write failed: %s\n", strerror(errno));
- if (dresp->need_to_prepare)
- return -3;
- return -4;
- }
- if (ret != sizeof(*dresp)) {
- DEBUG("short write (got %d, should have been %zu\n",
- ret, sizeof(*dresp));
- if (dresp->need_to_prepare)
- return -1;
- return -2;
- } /* else successfully wrote */
- return 0;
-}
-
-/**
- * read_client_response - wait for a client's ACK
- * @local_sock: socket to read from
- * @dresp: pointer to daemon's previous reponse
- * @cresp: pointer to store response in
- *
- * Wait for notification that file is prepared and segments have been
- * remapped. We need @dresp so that we can backout on errors
- * appropriately.
- *
- * returns 0, on success
- * negative values, on failure
- */
-static int read_client_response(int local_sock, struct daemon_response
- *dresp, struct client_response *cresp)
-{
- int ret;
- DEBUG("waiting for client's response\n");
- /* this may be unnecessarily long */
- alarm(300);
- /*
- */
- ret = read(local_sock, cresp, sizeof(*cresp));
- alarm(0);
- if (ret < 0) {
- if (timeout) {
- DEBUG("read timed out\n");
- if (dresp->need_to_prepare)
- return -1;
- return -2;
- }
- ERROR("read failed: %s\n", strerror(errno));
- if (dresp->need_to_prepare)
- return -3;
- return -4;
- }
- if (ret != sizeof(*cresp)) {
- DEBUG("short second read (got %d, should have been "
- "%zu\n", ret, sizeof(*cresp));
- if (dresp->need_to_prepare)
- return -1;
- return -2;
- }
- return 0;
-}
-
-/**
- * sharing_control_loop - main event loop for sharing
- * @sock: socket to wait on for connections
- *
- * This is expected to be an infinite loop. If we we ever return, it is
- * a fail case.
- *
- * returns 1
- */
-static int sharing_control_loop(int sock)
-{
- int ret;
- struct sockaddr_un from;
- socklen_t len = (socklen_t)sizeof(from);
- /*
- * Needed for socket messages -- could switch to dynamic
- * allocation
- */
- struct client_request creq;
- struct daemon_response dresp;
- struct client_response cresp;
- int local_sock;
- /* handle on some state that may need to be freed */
- struct shared_mapping *smptr;
- mode_t mode;
- /* needed for poll */
- struct pollfd ufds;
- /* used to allow for handling SIGALRM and SIGHUP */
- struct sigaction sigact = {
- .sa_handler = signal_handler,
- .sa_flags = 0,
- };
- struct sigaction prevact;
-
- sigemptyset(&(sigact.sa_mask));
-
- /* We will poll on sock for any incoming connections */
- ufds.fd = sock;
- ufds.events = POLLIN;
-
- /* install our custom signal-handler */
- sigaction(SIGINT, &sigact, NULL);
-
- for (;;) {
- /* gets set by SIGALRM handler */
- timeout = 0;
- /* not sure if this is necessary, but ok */
- ufds.revents = 0;
- /* helps with error handling */
- dresp.need_to_prepare = 0;
- /* reset mode of shared file */
- mode = 0;
-
- sigaction(SIGHUP, &sigact, &prevact);
-
-do_poll:
- ret = poll(&ufds, 1, poll_timeout);
- if (ret < 0) {
- /*
- * catch SIGHUP as non-fail case
- * SIGINT will have made us exit already
- */
- if (errno == EINTR)
- goto do_poll;
- ERROR("poll() failed: %s\n", strerror(errno));
- goto die;
- } else if (ret == 0) {
- /* poll timeout, reap any old files */
- DEBUG("poll() timed out\n");
- reap_files(0);
- continue;
- }
-
- switch (ufds.revents) {
- case POLLERR:
- ERROR("poll returned a POLLERR event\n");
- goto die;
- break;
- case POLLNVAL:
- ERROR("poll returned a POLLNVAL event\n");
- goto die;
- break;
- case POLLHUP:
- ERROR("poll returned a POLLHUP event\n");
- goto die;
- break;
- case POLLIN:
- break;
- default:
- ERROR("poll() returned an event we did not
expect: %d\n", ufds.revents);
- goto die;
- }
-
- local_sock = accept(sock, (struct sockaddr *)(&from), &len);
- if (local_sock < 0) {
- if (errno == ECONNABORTED) {
- /*
- * I think this is ok as a non-failure
- * case, but not positive, since the
- * client is what failed, not us.
- */
- DEBUG("accept() returned connection aborted\n");
- continue;
- }
- /*
- * Don't fail on a SIGHUP
- */
- if (errno == EINTR)
- goto do_poll;
- ERROR("accept() failed: %s\n", strerror(errno));
- goto die;
- }
-
- /*
- * Disable the races due to removing files that may be
- * sent to clients for sharing by just not handling
- * SIGHUP for a bit
- */
- sigaction(SIGHUP, &prevact, NULL);
- sigaction(SIGALRM, &sigact, NULL);
-
- ret = read_client_request(local_sock, &creq);
- switch (ret) {
- case 0:
- break;
- case -1:
- goto next;
- case -2:
- goto close_and_die;
- }
-
- /* Use the request to figure out the mapping ... */
- smptr = do_shared_file(&creq, &dresp);
- if (smptr == (struct shared_mapping *)-1) {
- ERROR("do_shared_file failed\n");
- /*
- * Just because we didn't get a hugetlbfs file,
- * doesn't mean we should fail outright. For
- * instance, if mkstemp() failed for some reason
- * we should still continue sharing any
- * previously shared segments.
- */
- goto next;
- }
-
- mode = S_IRUSR | S_IRGRP | S_IROTH;
- /* only allow preparer to write to file */
- if (dresp.need_to_prepare)
- mode |= S_IWUSR | S_IWGRP | S_IWOTH;
- ret = chmod(dresp.path_to_file, mode);
- if (ret < 0) {
- DEBUG("chmod(%s) to read (maybe writable) failed:
%s\n", dresp.path_to_file, strerror(errno));
- if (dresp.need_to_prepare)
- goto failed_next;
- goto next;
- }
-
- ret = write_daemon_response(local_sock, &dresp);
- switch (ret) {
- case 0:
- break;
- case -1:
- goto failed_next;
- case -2:
- goto next;
- case -3:
- goto failed_close_and_die;
- case -4:
- goto failed_next;
- }
-
- ret = read_client_response(local_sock, &dresp, &cresp);
- switch (ret) {
- case 0:
- break;
- case -1:
- goto failed_next;
- case -2:
- goto next;
- case -3:
- goto failed_close_and_die;
- case -4:
- goto failed_next;
- }
-
- /*
- * Determine if the segments were prepared/remapped as
- * expected.
- */
- if (cresp.succeeded != 0) {
- DEBUG("file not prepared\n");
- if (dresp.need_to_prepare)
- goto failed_next;
- goto next;
- }
-
- DEBUG("File prepared / opened successfully\n");
- if (dresp.need_to_prepare) {
- mode = S_IRUSR | S_IRGRP | S_IROTH;
- ret = chmod(dresp.path_to_file, mode);
- if (ret < 0) {
- DEBUG("chmod(%s) to read-only failed: %s\n",
dresp.path_to_file, strerror(errno));
- goto failed_next;
- }
- /*
- * Insert new element at head of list
- */
- if (head != NULL)
- smptr->next = head;
- else
- smptr->next = NULL;
- head = smptr;
-
- close(shared_fd);
- } else {
- /*
- * Move element to head of list. We have pointer
- * to the element just before the element to
- * move. If it is NULL, then we are sharing at
- * the head of the list, and no moving is
- * necessary. Else, shift the pointers around.
- * We obviously also know that head != NULL.
- */
- if (smptr != NULL) {
- struct shared_mapping *temp;
- temp = smptr->next;
- smptr->next = temp->next;
- temp->next = head;
- head = temp;
- }
- }
- /*
- * Shared mapping is at the head of the list now, update
- * timestamp.
- */
- head->timestamp = time(NULL);
-
- goto next;
-
-failed_next:
- close(shared_fd);
- unlink(dresp.path_to_file);
- free(smptr);
-next:
- close(local_sock);
- }
-
-failed_close_and_die:
- close(shared_fd);
- unlink(dresp.path_to_file);
- free(smptr);
-close_and_die:
- close(local_sock);
-die:
- kill_daemon(1);
-
- /* indicate failure if we ever get this far */
- return 1;
-}
-
-/**
- * main - set up socket and enter central control loop
- * @argc: number of command line arguments
- * @argv: command line arguments
- *
- * returns: 0, on success
- * -1, on failure
- */
-int main(int argc, char *argv[])
-{
- int ret;
- /* local socket structures */
- struct sockaddr_un sun;
-
- ret = parse_args(argc, argv);
- if (ret < 0)
- exit(1);
- if (ret == 0)
- /* Perform all the steps to become a real daemon */
- daemonize();
-
- sock = socket(AF_UNIX, SOCK_STREAM, 0);
- if (sock < 0) {
- ERROR("socket() failed: %s\n", strerror(errno));
- return -1;
- }
-
- sun.sun_family = AF_UNIX;
- /* clear out any previous socket */
- unlink("/tmp/libhugetlbfs-sock");
- strcpy(sun.sun_path, "/tmp/libhugetlbfs-sock");
- ret = bind(sock, (struct sockaddr *)(&sun), sizeof(sun));
- if (ret < 0) {
- ERROR("bind() failed: %s\n", strerror(errno));
- goto die;
- }
-
- chmod("/tmp/libhugetlbfs-sock", 0666);
-
- ret = listen(sock, QUEUE_LENGTH);
- if (ret < 0) {
- ERROR("listen() failed: %s\n", strerror(errno));
- goto die;
- }
-
- return sharing_control_loop(sock);
-
-die:
- kill_daemon(1);
-
- /* indicate failure if we ever get this far */
- return 1;
-}
Index: libhugetlbfs/hugetlbd.h
===================================================================
--- libhugetlbfs.orig/hugetlbd.h 2006-10-26 13:49:40.000000000 +1000
+++ /dev/null 1970-01-01 00:00:00.000000000 +0000
@@ -1,60 +0,0 @@
-/*
- * libhugetlbfs - Easy use of Linux hugepages
- * Copyright (C) 2006 Nishanth Aravamudan, IBM Corporation.
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public License
- * as published by the Free Software Foundation; either version 2.1 of
- * the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with this library; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#ifndef _HUGETLBD_H
-#define _HUGETLBD_H
-
-#ifndef __LIBHUGETLBFS__
-#error This header should not be included by library users.
-#endif /* __LIBHUGETLBFS__ */
-
-#define ID_KEY 0x56
-#define LOGFILE "/tmp/hugetlbd.log"
-
-/*
- * Ideally, would like to deal with this better, so that a 32-bit daemon
- * only deals in 32-bit addresses. But, for now, this makes 32-bit and
- * 64-bit apps deal ok with the socket messages.
- */
-typedef uint64_t vaddr_t;
-typedef uint64_t exe_ident_t;
-
-struct client_request {
- /*
- * unique identifer
- * if id == 0, then the request is actually a response that the
- * file is prepared
- */
- exe_ident_t id;
- /*
- * to identify the segment
- */
- vaddr_t vaddr;
-};
-
-struct client_response {
- int succeeded;
-};
-
-struct daemon_response {
- int need_to_prepare;
- char path_to_file[PATH_MAX+1];
-};
-
-#endif /* _HUGETLBD_H */
Index: libhugetlbfs/hugeutils.c
===================================================================
--- libhugetlbfs.orig/hugeutils.c 2006-10-26 13:49:40.000000000 +1000
+++ libhugetlbfs/hugeutils.c 2006-10-26 13:49:45.000000000 +1000
@@ -36,18 +36,14 @@
#include <sys/vfs.h>
#include <sys/statfs.h>
#include <sys/types.h>
-#include <sys/ipc.h>
-#include <sys/socket.h>
-#include <sys/un.h>
#include <sys/mman.h>
+#include <sys/file.h>
#include "libhugetlbfs_internal.h"
#include "hugetlbfs.h"
-#include "hugetlbd.h"
static int hpage_size; /* = 0 */
static char htlb_mount[PATH_MAX+1]; /* = 0 */
-int sharing; /* =0 */
/********************************************************************/
/* Internal functions */
@@ -247,297 +243,6 @@ int hugetlbfs_unlinked_fd(void)
return fd;
}
-/**
- * create_id - create unique id for current executable
- *
- * Uses IPC ftok() to generate a roughly unique identifier for our own
- * executable. This is used to match shareable mappings.
- *
- * returns: -1, on error
- * a unique identifier, otherwise
- */
-static exe_ident_t create_id(void)
-{
- exe_ident_t id;
- int ret;
- char my_exe[PATH_MAX];
-
- ret = readlink("/proc/self/exe", my_exe, PATH_MAX);
- if (ret < 0) {
- ERROR("readlink failed: %s\n", strerror(errno));
- return -1;
- }
-
- id = (exe_ident_t)ftok(my_exe, ID_KEY);
- if (id < 0) {
- ERROR("ftok failed: %s\n", strerror(errno));
- return -1;
- }
-
- return id;
-}
-
-static int sock;
-static int timeout;
-/* global to allow multiple callsites to set members */
-
-static void signal_handler(int signal)
-{
- switch (signal) {
- case SIGALRM:
- timeout = 1;
- }
-}
-
-static int client_complete(struct seg_info *htlb_seg_info, int success);
-
-/**
- * hugetlbfs_shared_file - communicate with daemon to get one shareable file
- * @htlb_seg_info: pointer to program's segment data
- *
- * returns:
- * -1, on failure
- * 0, on success, and caller does not need to prepare the segments
- * 1, on success, and caller does need to prepare the segments
- */
-static int hugetlbfs_shared_file(struct seg_info *htlb_seg_info)
-{
- int ret, fd;
- struct client_request creq;
- struct daemon_response dresp;
- struct sockaddr_un sun;
- struct sigaction sigact = {
- .sa_handler = signal_handler,
- .sa_flags = 0,
- };
- struct sigaction prev;
- int open_flags;
-
- sock = socket(AF_UNIX, SOCK_STREAM, 0);
- if (sock < 0) {
- ERROR("socket() failed: %s\n", strerror(errno));
- return -1;
- }
-
- sun.sun_family = AF_UNIX;
- strcpy(sun.sun_path, "/tmp/libhugetlbfs-sock");
- ret = connect(sock, &sun, sizeof(sun));
- if (ret < 0) {
- ERROR("connect() failed: %s\n", strerror(errno));
- return -1;
- }
-
- sigemptyset(&(sigact.sa_mask));
-
- sigaction(SIGALRM, &sigact, &prev);
-
- /* add data to request */
- creq.id = create_id();
- /*
- * This results in (non-harmful) compilation warning for
- * 32-bit applications
- */
- creq.vaddr = (vaddr_t)htlb_seg_info->vaddr;
-
- alarm(60);
- ret = write(sock, &creq, sizeof(creq));
- /* cancel any alarms */
- alarm(0);
- if (ret < 0) {
- if (timeout) {
- DEBUG("write timed out\n");
- goto fail;
- } else {
- ERROR("client request write failed: %s\n",
- strerror(errno));
- goto fail;
- }
- } else if (ret != sizeof(creq)) {
- DEBUG("short write (got %d, should have been %zu)\n",
- ret, sizeof(creq));
- goto fail;
- } /* else successfully wrote */
-
- /* wait slightly longer for the response */
- alarm(120);
- ret = read(sock, &dresp, sizeof(dresp));
- alarm(0);
- if (ret < 0) {
- if (timeout) {
- DEBUG("read timed out\n");
- goto fail;
- } else {
- ERROR("read failed: %s\n", strerror(errno));
- goto fail;
- }
- } else if (ret != sizeof(dresp)) {
- DEBUG("short read (got %d, should have been %zu)\n",
- ret, sizeof(dresp));
- goto fail;
- } /* else successfully read */
-
- sigaction(SIGALRM, &prev, NULL);
-
- /* Convert daemon response to open mode bits */
- if (dresp.need_to_prepare) {
- open_flags = O_RDWR;
- } else {
- open_flags = O_RDONLY;
- }
-
- /* get local handle on file */
- fd = open(dresp.path_to_file, open_flags);
- if (fd < 0) {
- ERROR("open(%s) failed: %s\n", dresp.path_to_file,
- strerror(errno));
- goto fail;
- }
-
- if (dresp.need_to_prepare) {
- DEBUG("daemon said to prepare\n");
- } else {
- /* ACK right away (closes socket) */
- DEBUG("daemon said not to prepare\n");
- ret = client_complete(htlb_seg_info, 0);
- if (ret < 0)
- goto fail;
- }
-
- htlb_seg_info->fd = fd;
-
- /* leave socket open until finished_prepare is called */
- return dresp.need_to_prepare;
-
-fail:
- close(sock);
- return -1;
-}
-
-/**
- * client_complete - send ACK to daemon that we no longer need file
- * @htlb_seg_info: pointer to program's segment data
- * @success: if 0, successful prepare, else failed
- *
- * returns: 0, if successful transmission (none occurs if
- * not sharing)
- * -1, if failed transmission
- */
-static int client_complete(struct seg_info *htlb_seg_info, int success)
-{
- struct sigaction sigact = {
- .sa_handler = signal_handler,
- .sa_flags = 0,
- };
- struct sigaction prev;
- struct client_response cresp;
- int ret;
-
- sigemptyset(&(sigact.sa_mask));
-
- sigaction(SIGALRM, &sigact, &prev);
-
- cresp.succeeded = success;
-
- alarm(60);
- ret = write(sock, &cresp, sizeof(cresp));
- alarm(0);
- if (ret < 0) {
- if (timeout) {
- DEBUG("write timed out\n");
- goto fail;
- } else {
- ERROR("client response write failed: %s\n",
- strerror(errno));
- goto fail;
- }
- } else if (ret != sizeof(cresp)) {
- DEBUG("short write (got %d, should have been %zu)\n",
- ret, sizeof(cresp));
- goto fail;
- } /* else successfully wrote */
-
- sigaction(SIGALRM, &prev, NULL);
-
- if (success < 0)
- /* back out open'd fd since we failed */
- close(htlb_seg_info->fd);
-
- close(sock);
- return 0;
-
-fail:
- close(htlb_seg_info->fd);
- close(sock);
- return -1;
-}
-
-/**
- * finished_prepare - callback for ACK'ing prepare is done
- * @htlb_seg_info: pointer to program's segment data
- * @success: if 0, successful prepare, else failed
- *
- * returns: 0, if successful transmission (none occurs if
- * not sharing)
- * -1, if failed transmission
- */
-int finished_prepare(struct seg_info *htlb_seg_info, int success)
-{
- if (sharing)
- return client_complete(htlb_seg_info, success);
- return 0;
-}
-
-/**
- * hugetlbfs_set_fds - multiplex callers depending on if sharing or not
- * @htlb_seg_info: pointer to program's segment data
- *
- * returns:
- * -1, on error
- * 0, on success, caller expects and gets shared, does not need to prepare
- * 1, on success, caller expects and gets shared, does need to prepare
- * 2, on success, caller expects shared and gets unlinked, does need to
prepare
- * 3, on success, caller expects and gets unlinked, does need to prepare
- */
-int hugetlbfs_set_fd(struct seg_info *htlb_seg_info)
-{
- char *env;
- int fd;
- /* assume unlinked, all unlinked files need to be prepared */
- int ret = 3;
-
- env = getenv("HUGETLB_SHARE");
- if (env)
- sharing = atoi(env);
- DEBUG("HUGETLB_SHARE=%d, sharing ", sharing);
- if (sharing == 2) {
- DEBUG_CONT("enabled for all segments\n");
- } else {
- if (sharing == 1) {
- DEBUG_CONT("enabled for only read-only segments\n");
- } else {
- DEBUG_CONT("disabled\n");
- }
- }
-
- /* Either share all segments or share only read-only segments */
- if ((sharing == 2) || ((sharing == 1) &&
- !(htlb_seg_info->prot & PROT_WRITE))) {
- /* first, try to share */
- ret = hugetlbfs_shared_file(htlb_seg_info);
- if (ret >= 0)
- return ret;
- /* but, fall through to unlinked files, if sharing fails */
- ret = 2;
- DEBUG("Falling back to unlinked files\n");
- }
- fd = hugetlbfs_unlinked_fd();
- if (fd < 0)
- return -1;
- htlb_seg_info->fd = fd;
- return ret;
-}
-
-
/********************************************************************/
/* Library user visible DIAGNOSES/DEBUGGING ONLY functions */
/********************************************************************/
Index: libhugetlbfs/libhugetlbfs_internal.h
===================================================================
--- libhugetlbfs.orig/libhugetlbfs_internal.h 2006-10-26 13:49:40.000000000
+1000
+++ libhugetlbfs/libhugetlbfs_internal.h 2006-10-26 13:49:45.000000000
+1000
@@ -30,8 +30,6 @@
extern int __hugetlbfs_verbose;
-extern int sharing;
-
#define ERROR(...) \
if (__hugetlbfs_verbose >= 1) \
fprintf(stderr, "libhugetlbfs: ERROR: " __VA_ARGS__)
@@ -48,16 +46,6 @@ extern int sharing;
if (__hugetlbfs_verbose >= 3) \
fprintf(stderr, __VA_ARGS__)
-struct seg_info {
- void *vaddr;
- unsigned long filesz, memsz;
- int prot;
- int fd;
-};
-
-int finished_prepare(struct seg_info *htlb_seg_info, int success);
-int hugetlbfs_set_fd(struct seg_info *htlb_seg_info);
-
#if defined(__powerpc64__) && !defined(__LP64__)
/* Older binutils fail to provide this symbol */
#define __LP64__
Index: libhugetlbfs/tests/run_tests.sh
===================================================================
--- libhugetlbfs.orig/tests/run_tests.sh 2006-10-26 13:49:42.000000000
+1000
+++ libhugetlbfs/tests/run_tests.sh 2006-10-26 13:49:45.000000000 +1000
@@ -13,6 +13,14 @@ function free_hpages() {
echo "$H"
}
+function hugetlbfs_path() {
+ if [ -n "$HUGETLB_PATH" ]; then
+ echo "$HUGETLB_PATH"
+ else
+ grep hugetlbfs /proc/mounts | cut -f2 -d' '
+ fi
+}
+
TOTAL_HPAGES=$(grep 'HugePages_Total:' /proc/meminfo | cut -f2 -d:)
[ -z "$TOTAL_HPAGES" ] && TOTAL_HPAGES=0
HPAGE_SIZE=$(grep 'Hugepagesize:' /proc/meminfo | awk '{print $2}')
@@ -64,15 +72,15 @@ elfshare_test () {
baseprog="${args[$N]}"
unset args[$N]
set -- "[EMAIL PROTECTED]"
+ # Run each elfshare test invocation independently - clean up the
+ # sharefiles before and after:
+ rm -f $(hugetlbfs_path)/{xB,xBDT}.$baseprog*
NUM_THREADS=2
- killall -HUP hugetlbd
run_test HUGETLB_SHARE=2 "$@" "xB.$baseprog" $NUM_THREADS
- killall -HUP hugetlbd
run_test HUGETLB_SHARE=1 "$@" "xB.$baseprog" $NUM_THREADS
- killall -HUP hugetlbd
run_test HUGETLB_SHARE=2 "$@" "xBDT.$baseprog" $NUM_THREADS
- killall -HUP hugetlbd
run_test HUGETLB_SHARE=1 "$@" "xBDT.$baseprog" $NUM_THREADS
+ rm -f $(hugetlbfs_path)/{xB,xBDT}.$baseprog*
}
setup_shm_sysctl() {
@@ -131,21 +139,11 @@ functional_tests () {
elflink_test linkhuge
# Sharing tests
- # stop all running instances for clean testing
- killall -INT hugetlbd
- # start the daemon in the bg
- PATH=../obj32:../obj64:$PATH hugetlbd
- # XXX: Wait for daemon to start
- sleep 5
elfshare_test linkshare
- # stop our instance of the daemon
- killall -INT hugetlbd
# Accounting bug tests
# reset free hpages because sharing will have held some
# alternatively, use
-# killall -HUP hugetlbd
-# to make the sharing daemon give up the files
run_test chunk-overcommit `free_hpages`
run_test alloc-instantiate-race `free_hpages`
run_test truncate_reserve_wraparound
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
