The branch, master has been updated via 4f751e2 nwrap: Modify AUTHORS file. via 3bf0d8f TESTS: Add new test which checks multithread support. via 5dc7e5d nwrap: Add basic locking for support multithreaded applications via ffc262d TESTS: Add test for getaddrinfo. via f839fb8 TESTS: Add test_nwrap_gethostbyname_multiple test via c28adab nwrap: Add BSD libc support for gethost*_r functios. via 4c956bf nwrap: Use nwrap vectors as memory backend for getaddrinfo() and gethostbyname() via f067a6e nwrap: Add string manipulation functions. via 8710061 nwrap: Add nwrap_add_hname() and nwrap_add_hname_alias(). via 449f8d9 nwrap: Add function nwrap_add_ai via 0ff0c85 nwrap: Add a hash table to the nwrap structure via 6215857 nwrap: Simplify file loading. via 69efe68 nwrap: Add nwrap vector memory management functions from 827183d tests: Only run shadow test when shadow.h is available
https://git.samba.org/?p=nss_wrapper.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 4f751e256734082f3d4ae4b2dbf35d239840d694 Author: Robin Hack <hack.ro...@gmail.com> Date: Mon Mar 30 16:08:25 2015 +0200 nwrap: Modify AUTHORS file. Assign emails to authors. Also add me (Robin Hack) as a author. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 3bf0d8f63d340a844f6c1510b6eebe35581982e5 Author: Robin Hack <hack.ro...@gmail.com> Date: Tue Mar 24 16:45:19 2015 +0100 TESTS: Add new test which checks multithread support. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 5dc7e5dbf725718a34db3dce26cbb5aa67ee4efa Author: Robin Hack <hack.ro...@gmail.com> Date: Tue Mar 24 17:54:34 2015 +0100 nwrap: Add basic locking for support multithreaded applications Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit ffc262d08c16d98f67b0005145b1690ec0b9c4b5 Author: Robin Hack <hack.ro...@gmail.com> Date: Mon Mar 30 14:36:29 2015 +0200 TESTS: Add test for getaddrinfo. Test tries to get multiple and mixed (ipv4 and ipv6) records. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit f839fb8fbd69c0d2614c92b7f852a735912d75bc Author: Robin Hack <hack.ro...@gmail.com> Date: Tue Mar 24 12:03:14 2015 +0100 TESTS: Add test_nwrap_gethostbyname_multiple test Test cover multiple records support. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit c28adabf78fa8a5f62b7782c5c650db6b8ef361f Author: Robin Hack <hack.ro...@gmail.com> Date: Fri Mar 27 15:51:11 2015 +0100 nwrap: Add BSD libc support for gethost*_r functios. BSD libc stores data in thread local storage. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 4c956bf0c1e79862263ef848e15e0bd70bf72a9f Author: Robin Hack <hack.ro...@gmail.com> Date: Thu Jul 16 16:10:20 2015 +0200 nwrap: Use nwrap vectors as memory backend for getaddrinfo() and gethostbyname() Previous patches introduced nwrap vectors and hash functions. This patch will join all together. Main purpose of this patch is: - support more records per hostname. For example: hostname now can have associated IPv4 and IPv6 address. - better handling of big number of hostname records Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit f067a6e867db3752592f21bba567fb14bfbc174d Author: Robin Hack <hack.ro...@gmail.com> Date: Tue Mar 24 15:14:35 2015 +0100 nwrap: Add string manipulation functions. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 8710061f0ad45b5956ff98312dbe39ea1984304c Author: Robin Hack <hack.ro...@gmail.com> Date: Wed Jul 15 15:05:28 2015 +0200 nwrap: Add nwrap_add_hname() and nwrap_add_hname_alias(). Functions adds a hostname or an alias to the hash table. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 449f8d9a414e35a3520ff0305661119193e794c8 Author: Robin Hack <hack.ro...@gmail.com> Date: Wed Jul 15 15:04:32 2015 +0200 nwrap: Add function nwrap_add_ai This adds an ip to the hash table. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 0ff0c85cd5d4e30fd132d507213ad97c37713b38 Author: Robin Hack <hack.ro...@gmail.com> Date: Wed Jul 15 15:01:48 2015 +0200 nwrap: Add a hash table to the nwrap structure The hash table will hold hostnames and ips internally. The new environment variable which is introduced is NSS_WRAPPER_MAX_HOSTENTS. It sets the maximum count of hostent (hostnames and ips) structs which the hash table can hold. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 6215857195ab9cf4a2b321d984380f9530c0569e Author: Robin Hack <hack.ro...@gmail.com> Date: Mon Mar 23 14:39:28 2015 +0100 nwrap: Simplify file loading. Lines are stored into vectors now. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> commit 69efe68825dee4c3e48879dc098aa2dd706291c0 Author: Robin Hack <hack.ro...@gmail.com> Date: Mon Mar 23 14:10:18 2015 +0100 nwrap: Add nwrap vector memory management functions Nwrap vectors are just more inteligent arrays. They can look like very simple version of vectors from C++. TESTS: Add nwrap_vector tests. Signed-off-by: Robin Hack <hack.ro...@gmail.com> Reviewed-by: Andreas Schneider <a...@samba.org> Reviewed-by: Michael Adam <ob...@samba.org> ----------------------------------------------------------------------- Summary of changes: AUTHORS | 7 +- src/CMakeLists.txt | 6 +- src/nss_wrapper.c | 1021 +++++++++++++++++++++++++++++++------- tests/CMakeLists.txt | 8 + tests/hosts.in | 7 + tests/test_getaddrinfo.c | 73 +++ tests/test_gethostby_name_addr.c | 82 +++ tests/test_nwrap_vector.c | 118 +++++ 8 files changed, 1132 insertions(+), 190 deletions(-) create mode 100644 tests/test_nwrap_vector.c Changeset truncated at 500 lines: diff --git a/AUTHORS b/AUTHORS index 8423d3b..335ed0a 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,3 +1,4 @@ -Stefan Metzmacher -Guenther Deschner -Andreas Schneider +Stefan Metzmacher <me...@samba.org> +Guenther Deschner <g...@samba.org> +Andreas Schneider <a...@samba.org> +Robin Hack <hack.ro...@gmail.com> diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 36b05a8..7b89866 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -2,7 +2,11 @@ project(libnss_wrapper C) include_directories(${CMAKE_BINARY_DIR}) add_library(nss_wrapper SHARED nss_wrapper.c) -target_link_libraries(nss_wrapper ${NWRAP_REQUIRED_LIBRARIES}) +target_link_libraries(nss_wrapper ${NWRAP_REQUIRED_LIBRARIES} ${CMAKE_THREAD_LIBS_INIT}) + +if (BSD) + add_definitions(-DBSD) +endif (BSD) set_target_properties( nss_wrapper diff --git a/src/nss_wrapper.c b/src/nss_wrapper.c index e985bb3..bf51ec2 100644 --- a/src/nss_wrapper.c +++ b/src/nss_wrapper.c @@ -35,6 +35,8 @@ #include "config.h" +#include <pthread.h> + #include <sys/types.h> #include <sys/stat.h> #include <sys/socket.h> @@ -50,6 +52,9 @@ #include <unistd.h> #include <ctype.h> +#include <search.h> +#include <assert.h> + /* * Defining _POSIX_PTHREAD_SEMANTICS before including pwd.h and grp.h gives us * the posix getpwnam_r(), getpwuid_r(), getgrnam_r and getgrgid_r calls on @@ -153,6 +158,62 @@ typedef nss_status_t NSS_STATUS; #define NWRAP_INET_ADDRSTRLEN INET_ADDRSTRLEN #endif +#define NWRAP_LOCK(m) do { \ + pthread_mutex_lock(&( m ## _mutex)); \ +} while(0) + +#define NWRAP_UNLOCK(m) do { \ + pthread_mutex_unlock(&( m ## _mutex)); \ +} while(0) + + +static bool nwrap_initialized = false; +static pthread_mutex_t nwrap_initialized_mutex = PTHREAD_MUTEX_INITIALIZER; + +/* The mutex or accessing the id */ +static pthread_mutex_t nwrap_global_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t nwrap_gr_global_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t nwrap_he_global_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t nwrap_pw_global_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t nwrap_sp_global_mutex = PTHREAD_MUTEX_INITIALIZER; + +/* Add new global locks here please */ +/* Also don't forget to add locks to + * nwrap_init() function. + */ +# define NWRAP_LOCK_ALL do { \ + NWRAP_LOCK(nwrap_initialized); \ + NWRAP_LOCK(nwrap_global); \ + NWRAP_LOCK(nwrap_gr_global); \ + NWRAP_LOCK(nwrap_he_global); \ + NWRAP_LOCK(nwrap_pw_global); \ + NWRAP_LOCK(nwrap_sp_global); \ +} while (0); + +# define NWRAP_UNLOCK_ALL do {\ + NWRAP_UNLOCK(nwrap_sp_global); \ + NWRAP_UNLOCK(nwrap_pw_global); \ + NWRAP_UNLOCK(nwrap_he_global); \ + NWRAP_UNLOCK(nwrap_gr_global); \ + NWRAP_UNLOCK(nwrap_global); \ + NWRAP_UNLOCK(nwrap_initialized); \ +} while (0); + +static void nwrap_thread_prepare(void) +{ + NWRAP_LOCK_ALL; +} + +static void nwrap_thread_parent(void) +{ + NWRAP_UNLOCK_ALL; +} + +static void nwrap_thread_child(void) +{ + NWRAP_UNLOCK_ALL; +} + enum nwrap_dbglvl_e { NWRAP_LOG_ERROR = 0, NWRAP_LOG_WARN, @@ -474,15 +535,162 @@ struct nwrap_main { struct nwrap_libc *libc; }; -struct nwrap_main *nwrap_main_global; -struct nwrap_main __nwrap_main_global; +static struct nwrap_main *nwrap_main_global; +static struct nwrap_main __nwrap_main_global; + +/* + * PROTOTYPES + */ +static int nwrap_convert_he_ai(const struct hostent *he, + unsigned short port, + const struct addrinfo *hints, + struct addrinfo **pai, + bool skip_canonname); + +/* + * VECTORS + */ + +#define DEFAULT_VECTOR_CAPACITY 16 + +struct nwrap_vector { + void **items; + size_t count; + size_t capacity; +}; + +/* Macro returns pointer to first element of vector->items array. + * + * nwrap_vector is used as a memory backend which take care of + * memory allocations and other stuff like memory growing. + * nwrap_vectors should not be considered as some abstract structures. + * On this level, vectors are more handy than direct realloc/malloc + * calls. + * + * nwrap_vector->items is array inside nwrap_vector which can be + * directly pointed by libc structure assembled by cwrap itself. + * + * EXAMPLE: + * + * 1) struct hostent contains char **h_addr_list element. + * 2) nwrap_vector holds array of pointers to addresses. + * It's easier to use vector to store results of + * file parsing etc. + * + * Now, pretend that cwrap assembled struct hostent and + * we need to set h_addr_list to point to nwrap_vector. + * Idea behind is to shield users from internal nwrap_vector + * implementation. + * (Yes, not fully - array terminated by NULL is needed because + * it's result expected by libc function caller.) + * + * + * CODE EXAMPLE: + * + * struct hostent he; + * struct nwrap_vector *vector = malloc(sizeof(struct nwrap_vector)); + * ... don't care about failed allocation now ... + * + * ... fill nwrap vector ... + * + * struct hostent he; + * he.h_addr_list = nwrap_vector_head(vector); + * + */ +#define nwrap_vector_head(vect) ((void *)((vect)->items)) + +#define nwrap_vector_foreach(item, vect, iter) \ + for (iter = 0, (item) = (vect).items == NULL ? NULL : (vect).items[0]; \ + item != NULL; \ + (item) = (vect).items[++iter]) + +static inline bool nwrap_vector_init(struct nwrap_vector *const vector) +{ + if (vector == NULL) { + return false; + } + + /* count is initialized by ZERO_STRUCTP */ + ZERO_STRUCTP(vector); + vector->items = malloc(sizeof(void *) * (DEFAULT_VECTOR_CAPACITY + 1)); + if (vector->items == NULL) { + return false; + } + vector->capacity = DEFAULT_VECTOR_CAPACITY; + memset(vector->items, '\0', sizeof(void *) * (DEFAULT_VECTOR_CAPACITY + 1)); + + return true; +} + +static bool nwrap_vector_add_item(struct nwrap_vector *cont, void *const item) +{ + assert (cont != NULL); + + if (cont->items == NULL) { + nwrap_vector_init(cont); + } + + if (cont->count == cont->capacity) { + /* Items array _MUST_ be NULL terminated because it's passed + * as result to caller which expect NULL terminated array from libc. + */ + void **items = realloc(cont->items, sizeof(void *) * ((cont->capacity * 2) + 1)); + if (items == NULL) { + return false; + } + cont->items = items; + + /* Don't count ending NULL to capacity */ + cont->capacity *= 2; + } + + cont->items[cont->count] = item; + + cont->count += 1; + cont->items[cont->count] = NULL; + + return true; +} + +static bool nwrap_vector_merge(struct nwrap_vector *dst, + struct nwrap_vector *src) +{ + void **dst_items = NULL; + size_t count; + + if (src->count == 0) { + return true; + } + + count = dst->count + src->count; + + /* We don't need reallocation if we have enough capacity. */ + if (src->count > (dst->capacity - dst->count)) { + dst_items = (void **)realloc(dst->items, (count + 1) * sizeof(void *)); + if (dst_items == NULL) { + return false; + } + dst->items = dst_items; + dst->capacity = count; + } + + memcpy((void *)(((long *)dst->items) + dst->count), + src->items, + src->count * sizeof(void *)); + dst->count = count; + + return true; +} struct nwrap_cache { const char *path; int fd; + FILE *fp; struct stat st; - uint8_t *buf; void *private_data; + + struct nwrap_vector lines; + bool (*parse_line)(struct nwrap_cache *, char *line); void (*unload)(struct nwrap_cache *); }; @@ -537,24 +745,34 @@ static void nwrap_he_unload(struct nwrap_cache *nwrap); struct nwrap_addrdata { unsigned char host_addr[16]; /* IPv4 or IPv6 address */ - char *h_addr_ptrs[2]; /* host_addr pointer + NULL */ }; +static size_t max_hostents = 100; + struct nwrap_entdata { - struct nwrap_addrdata *addr; + struct nwrap_addrdata addr; struct hostent ht; + + struct nwrap_vector nwrap_addrdata; + + ssize_t aliases_count; + + struct nwrap_entdata *ed_next; + struct nwrap_entdata *ed_tail; }; struct nwrap_he { struct nwrap_cache *cache; struct nwrap_entdata *list; + struct nwrap_vector entdata; + int num; int idx; }; -struct nwrap_cache __nwrap_cache_he; -struct nwrap_he nwrap_he_global; +static struct nwrap_cache __nwrap_cache_he; +static struct nwrap_he nwrap_he_global; /********************************************************* @@ -706,6 +924,18 @@ static void *_nwrap_load_lib_function(enum nwrap_lib lib, const char *fn_name) _nwrap_load_lib_function(lib, #fn_name); \ } +/* INTERNAL HELPER FUNCTIONS */ +static void nwrap_lines_unload(struct nwrap_cache *const nwrap) +{ + size_t p; + for (p = 0; p < nwrap->lines.count; p++) { + /* Maybe some vectors were merged ... */ + SAFE_FREE(nwrap->lines.items[p]); + } + SAFE_FREE(nwrap->lines.items); + nwrap->lines.count = 0; +} + /* * IMPORTANT * @@ -776,6 +1006,37 @@ static int libc_getpwuid_r(uid_t uid, } #endif +static inline void str_tolower(char *dst, char *src) +{ + register char *src_tmp = src; + register char *dst_tmp = dst; + + while (*src_tmp != '\0') { + *dst_tmp = tolower(*src_tmp); + ++src_tmp; + ++dst_tmp; + } +} + +static bool str_tolower_copy(char **dst_name, const char *const src_name) +{ + char *h_name_lower; + + if ((dst_name == NULL) || (src_name == NULL)) { + return false; + } + + h_name_lower = strdup(src_name); + if (h_name_lower == NULL) { + NWRAP_LOG(NWRAP_LOG_DEBUG, "Out of memory while strdup"); + return false; + } + + str_tolower(h_name_lower, h_name_lower); + *dst_name = h_name_lower; + return true; +} + static void libc_setpwent(void) { nwrap_load_lib_function(NWRAP_LIBC, setpwent); @@ -1262,10 +1523,55 @@ static void nwrap_backend_init(struct nwrap_main *r) static void nwrap_init(void) { - static bool initialized; + const char *env; + char *endptr; + size_t max_hostents_tmp; - if (initialized) return; - initialized = true; + NWRAP_LOCK(nwrap_initialized); + if (nwrap_initialized) { + NWRAP_UNLOCK(nwrap_initialized); + return; + } + + /* + * Still holding nwrap_initialized lock here. + * We don't use NWRAP_(UN)LOCK_ALL macros here because we + * want to avoid overhead when other threads do their job. + */ + NWRAP_LOCK(nwrap_global); + NWRAP_LOCK(nwrap_gr_global); + NWRAP_LOCK(nwrap_he_global); + NWRAP_LOCK(nwrap_pw_global); + NWRAP_LOCK(nwrap_sp_global); + + nwrap_initialized = true; + + /* Initialize pthread_atfork handlers */ + pthread_atfork(&nwrap_thread_prepare, &nwrap_thread_parent, + &nwrap_thread_child); + + env = getenv("NSS_WRAPPER_MAX_HOSTENTS"); + if (env != NULL) { + max_hostents_tmp = (size_t)strtol(env, &endptr, 10); + if (((env != '\0') && (endptr == '\0')) || + (max_hostents_tmp == 0)) { + NWRAP_LOG(NWRAP_LOG_DEBUG, + "Error parsing NSS_WRAPPER_MAX_HOSTENTS " + "value or value is too small. " + "Using default value: %lu.", + max_hostents); + } else { + max_hostents = max_hostents_tmp; + } + } + /* Initialize hash table */ + NWRAP_LOG(NWRAP_LOG_DEBUG, + "Initializing hash table of size %lu items.", max_hostents); + if (hcreate(max_hostents) == 0) { + NWRAP_LOG(NWRAP_LOG_ERROR, + "Failed to initialize hash table"); + goto done; + } nwrap_main_global = &__nwrap_main_global; @@ -1277,6 +1583,7 @@ static void nwrap_init(void) nwrap_pw_global.cache = &__nwrap_cache_pw; nwrap_pw_global.cache->path = getenv("NSS_WRAPPER_PASSWD"); + nwrap_pw_global.cache->fp = NULL; nwrap_pw_global.cache->fd = -1; nwrap_pw_global.cache->private_data = &nwrap_pw_global; nwrap_pw_global.cache->parse_line = nwrap_pw_parse_line; @@ -1287,6 +1594,7 @@ static void nwrap_init(void) nwrap_sp_global.cache = &__nwrap_cache_sp; nwrap_sp_global.cache->path = getenv("NSS_WRAPPER_SHADOW"); + nwrap_sp_global.cache->fp = NULL; nwrap_sp_global.cache->fd = -1; nwrap_sp_global.cache->private_data = &nwrap_sp_global; nwrap_sp_global.cache->parse_line = nwrap_sp_parse_line; @@ -1297,6 +1605,7 @@ static void nwrap_init(void) nwrap_gr_global.cache = &__nwrap_cache_gr; nwrap_gr_global.cache->path = getenv("NSS_WRAPPER_GROUP"); + nwrap_gr_global.cache->fp = NULL; nwrap_gr_global.cache->fd = -1; nwrap_gr_global.cache->private_data = &nwrap_gr_global; nwrap_gr_global.cache->parse_line = nwrap_gr_parse_line; @@ -1306,10 +1615,15 @@ static void nwrap_init(void) nwrap_he_global.cache = &__nwrap_cache_he; nwrap_he_global.cache->path = getenv("NSS_WRAPPER_HOSTS"); + nwrap_he_global.cache->fp = NULL; nwrap_he_global.cache->fd = -1; nwrap_he_global.cache->private_data = &nwrap_he_global; nwrap_he_global.cache->parse_line = nwrap_he_parse_line; nwrap_he_global.cache->unload = nwrap_he_unload; + +done: + /* We hold all locks here so we can use NWRAP_UNLOCK_ALL. */ + NWRAP_UNLOCK_ALL; } bool nss_wrapper_enabled(void) @@ -1367,88 +1681,78 @@ static bool nwrap_hostname_enabled(void) static bool nwrap_parse_file(struct nwrap_cache *nwrap) { - int ret; - uint8_t *buf = NULL; - char *nline; + char *line = NULL; + ssize_t n; + /* Unused but getline needs it */ + size_t len; + bool ok; if (nwrap->st.st_size == 0) { NWRAP_LOG(NWRAP_LOG_DEBUG, "size == 0"); - goto done; + return true; } + /* Support for 32-bit system I guess */ if (nwrap->st.st_size > INT32_MAX) { NWRAP_LOG(NWRAP_LOG_ERROR, "Size[%u] larger than INT32_MAX", (unsigned)nwrap->st.st_size); -- NSS Wrapper Repository