are you sure that O_CLOEXEC and ACCESSPERMS are portable ?
Vincent On Wed, 13 Apr 2011, Enlightenment SVN wrote: > Log: > eina: add Eina_File API. > > NOTE: the purpose of this API is to replace mmap user in the > EFL, share cache and more code across them. The potential user > are eet, evas, efreet, eio and enlil. More patch are needed for > them to use this infra. Help welcome :-) > > NOTE2: this API also need more test and is waiting for some > more pthread infra before being thread safe. But at the end > it will be thread safe if eina thread safety is requested. > > > Author: cedric > Date: 2011-04-13 09:15:30 -0700 (Wed, 13 Apr 2011) > New Revision: 58637 > Trac: http://trac.enlightenment.org/e/changeset/58637 > > Modified: > trunk/eina/ChangeLog trunk/eina/src/include/eina_file.h > trunk/eina/src/lib/eina_file.c trunk/eina/src/lib/eina_main.c > > Modified: trunk/eina/ChangeLog > =================================================================== > --- trunk/eina/ChangeLog 2011-04-13 13:29:54 UTC (rev 58636) > +++ trunk/eina/ChangeLog 2011-04-13 16:15:30 UTC (rev 58637) > @@ -47,3 +47,8 @@ > * Add eina_inlist_sort. > * Add eina_mempool_repack. > * Add Eina_Object API. > + > +2011-05-13 Cedric Bail & Vincent Torri > + > + * Add Eina_File API. > + > > Modified: trunk/eina/src/include/eina_file.h > =================================================================== > --- trunk/eina/src/include/eina_file.h 2011-04-13 13:29:54 UTC (rev > 58636) > +++ trunk/eina/src/include/eina_file.h 2011-04-13 16:15:30 UTC (rev > 58637) > @@ -81,6 +81,15 @@ > EINA_FILE_WHT /**< Whiteout file type (unused on Windows). */ > } Eina_File_Type; > > +typedef struct _Eina_File Eina_File; > + > +typedef enum { > + EINA_FILE_RANDOM, /**< Advise random memory access to the mapped > memory. */ > + EINA_FILE_SEQUENTIAL, /**< Advise sequential memory access to the mapped > memory. */ > + EINA_FILE_WILLNEED, /**< Advise need for all the mapped memory. */ > + EINA_FILE_POPULATE /**< Request all the mapped memory. */ > +} Eina_File_Populate; > + > /* Why do this? Well PATH_MAX may vary from when eina itself is compiled > * to when the app using eina is compiled. exposing the path buffer below > * cant safely and portably vary based on how/when you compile. it should > @@ -239,6 +248,82 @@ > EAPI Eina_Iterator *eina_file_direct_ls(const char *dir) > EINA_WARN_UNUSED_RESULT EINA_ARG_NONNULL(1) EINA_MALLOC; > > /** > + * @brief Get a read-only handler to a file. > + * > + * @param name Filename to open > + * @param shared Requested a shm > + * > + * The file are only open in read only mode. Name should be an absolute path > to > + * prevent cache mistake. A handler could be shared among multiple instance > and > + * will be correctly refcounted. File are automatically closed on exec. > + */ > +EAPI Eina_File *eina_file_open(const char *name, Eina_Bool shared) > EINA_WARN_UNUSED_RESULT EINA_ARG_NONNULL(1) EINA_MALLOC; > + > +/** > + * @brief Unref file handler. > + * > + * @param file File handler to unref. > + * > + * This doesn't close the file, it will remain open until it leave the cache. > + */ > +EAPI void eina_file_close(Eina_File *file); > + > +/** > + * @brief Get file size at open time. > + * > + * @param file The file handler to request the size from. > + * @return The length of the file. > + */ > +EAPI unsigned long int eina_file_size_get(Eina_File *file); > + > +/** > + * @brief Get the last modification time of an open file. > + * > + * @param file The file handler to request the modification time from. > + * @return The last modification time. > + */ > +EAPI time_t eina_file_mtime_get(Eina_File *file); > + > +/** > + * @brief Get the filename of an open file. > + * > + * @param file The file handler to request the name from. > + * @return Stringshared filename of the file. > + */ > +EAPI const char *eina_file_filename_get(Eina_File *file); > + > +/** > + * @brief Map all the file to a buffer. > + * > + * @param file The file handler to map in memory > + * @param rule The rule to apply to the mapped memory > + * @return A pointer to a buffer that map all the file content. @c NULL if > it fail. > + */ > +EAPI void *eina_file_map_all(Eina_File *file, Eina_File_Populate rule); > + > +/** > + * @brief Map a part of the file. > + * > + * @param file The file handler to map in memory > + * @param rule The rule to apply to the mapped memory > + * @param offset The offset inside the file > + * @param length The length of the memory to map > + * @return A valid pointer to the system memory with @p length valid byte in > it. And @c NULL if not inside the file or anything else goes wrong. > + * > + * This does handle refcounting so it will share map that target the same > memory area. > + */ > +EAPI void *eina_file_map_new(Eina_File *file, Eina_File_Populate rule, > + unsigned long int offset, unsigned long int > length); > + > +/** > + * @brief Unref and unmap memory if possible. > + * > + * @param file The file handler to unmap memory from. > + * @param map Memory map to unref and unmap. > + */ > +EAPI void eina_file_map_free(Eina_File *file, void *map); > + > +/** > * @} > */ > > > Modified: trunk/eina/src/lib/eina_file.c > =================================================================== > --- trunk/eina/src/lib/eina_file.c 2011-04-13 13:29:54 UTC (rev 58636) > +++ trunk/eina/src/lib/eina_file.c 2011-04-13 16:15:30 UTC (rev 58637) > @@ -44,6 +44,8 @@ > #include <sys/types.h> > #include <sys/stat.h> > #include <unistd.h> > +#include <sys/mman.h> > +#include <fcntl.h> > > #define PATH_DELIM '/' > > @@ -60,15 +62,38 @@ > #include "eina_safety_checks.h" > #include "eina_file.h" > #include "eina_stringshare.h" > +#include "eina_hash.h" > +#include "eina_list.h" > > /*============================================================================* > * Local > * > > *============================================================================*/ > > +#ifndef EINA_LOG_COLOR_DEFAULT > +#define EINA_LOG_COLOR_DEFAULT EINA_COLOR_CYAN > +#endif > + > +#ifdef ERR > +#undef ERR > +#endif > +#define ERR(...) EINA_LOG_DOM_ERR(_eina_file_log_dom, __VA_ARGS__) > + > +#ifdef WRN > +#undef WRN > +#endif > +#define WRN(...) EINA_LOG_DOM_WARN(_eina_file_log_dom, __VA_ARGS__) > + > +#ifdef DBG > +#undef DBG > +#endif > +#define DBG(...) EINA_LOG_DOM_DBG(_eina_file_log_dom, __VA_ARGS__) > + > /** > * @cond LOCAL > */ > typedef struct _Eina_File_Iterator Eina_File_Iterator; > +typedef struct _Eina_File_Map Eina_File_Map; > + > struct _Eina_File_Iterator > { > Eina_Iterator iterator; > @@ -79,6 +104,42 @@ > char dir[1]; > }; > > +struct _Eina_File > +{ > + const char *filename; > + > + Eina_Hash *map; > + Eina_Hash *rmap; > + void *global_map; > + > + unsigned long length; > + time_t mtime; > + > + int refcount; > + int global_refcount; > + > + int fd; > + > + Eina_Bool shared : 1; > + Eina_Bool delete_me : 1; > +}; > + > +struct _Eina_File_Map > +{ > + void *map; > + > + unsigned long int offset; > + unsigned long int length; > + > + int refcount; > +}; > + > +static Eina_Hash *_eina_file_cache = NULL; > +static Eina_List *_eina_file_cache_lru = NULL; > +static Eina_List *_eina_file_cache_delete = NULL; > + > +static int _eina_file_log_dom = -1; > + > /* > * This complex piece of code is needed due to possible race condition. > * The code and description of the issue can be found at : > @@ -310,6 +371,121 @@ > return EINA_TRUE; > } > > +static void > +_eina_file_real_close(Eina_File *file) > +{ > + eina_hash_free(file->rmap); > + eina_hash_free(file->map); > + > + if (file->global_map != MAP_FAILED) > + munmap(file->global_map, file->length); > + > + close(file->fd); > + > + eina_stringshare_del(file->filename); > + > + free(file); > +} > + > +static void > +_eina_file_map_close(Eina_File_Map *map) > +{ > + munmap(map->map, map->length); > + free(map); > +} > + > +static unsigned int > +_eina_file_map_key_length(__UNUSED__ const void *key) > +{ > + return sizeof (unsigned long int) * 2; > +} > + > +static int > +_eina_file_map_key_cmp(const unsigned long int *key1, __UNUSED__ int > key1_length, > + const unsigned long int *key2, __UNUSED__ int > key2_length) > +{ > + if (key1[0] - key2[0] == 0) return key1[1] - key2[1]; > + return key1[0] - key2[0]; > +} > + > +static int > +_eina_file_map_key_hash(const unsigned long int *key, __UNUSED__ int > key_length) > +{ > + return eina_hash_int64(&key[0], sizeof (unsigned long int)) > + ^ eina_hash_int64(&key[1], sizeof (unsigned long int)); > +} > + > +static void > +_eina_file_map_rule_apply(Eina_File_Populate rule, void *addr, unsigned long > int size) > +{ > + int flag; > + > + switch (rule) > + { > + case EINA_FILE_RANDOM: flag = MADV_RANDOM; break; > + case EINA_FILE_SEQUENTIAL: flag = MADV_SEQUENTIAL; break; > + case EINA_FILE_WILLNEED: > + case EINA_FILE_POPULATE: > + flag = MADV_WILLNEED; > + break; > + } > + > + madvise(addr, size, flag); > +} > + > +Eina_Bool > +eina_file_init(void) > +{ > + _eina_file_log_dom = eina_log_domain_register("eina_file", > + EINA_LOG_COLOR_DEFAULT); > + if (_eina_file_log_dom < 0) > + { > + EINA_LOG_ERR("Could not register log domain: eina_file"); > + return EINA_FALSE; > + } > + > + _eina_file_cache = > eina_hash_string_djb2_new(EINA_FREE_CB(_eina_file_real_close)); > + if (!_eina_file_cache) > + { > + ERR("Could not create cache."); > + eina_log_domain_unregister(_eina_file_log_dom); > + _eina_file_log_dom = -1; > + return EINA_FALSE; > + } > + > + return EINA_TRUE; > +} > + > +Eina_Bool > +eina_file_shutdown(void) > +{ > + Eina_File *f; > + Eina_List *l; > + > + EINA_LIST_FREE(_eina_file_cache_delete, f) > + _eina_file_real_close(f); > + > + EINA_LIST_FOREACH(_eina_file_cache_lru, l, f) > + eina_hash_del(_eina_file_cache, f->filename, f); > + > + if (eina_hash_population(_eina_file_cache) > 0) > + { > + Eina_Iterator *it; > + const char *key; > + > + it = eina_hash_iterator_key_new(_eina_file_cache); > + EINA_ITERATOR_FOREACH(it, key) > + ERR("File [%s] still open !", key); > + eina_iterator_free(it); > + } > + > + eina_hash_free(_eina_file_cache); > + > + eina_log_domain_unregister(_eina_file_log_dom); > + _eina_file_log_dom = -1; > + return EINA_FALSE; > +} > + > /** > * @endcond > */ > @@ -527,3 +703,228 @@ > > return &it->iterator; > } > + > +EAPI Eina_File * > +eina_file_open(const char *filename, Eina_Bool shared) > +{ > + Eina_File *file; > + Eina_File *n; > + struct stat file_stat; > + int fd; > + Eina_Bool create = EINA_FALSE; > + > + /* FIXME: always open absolute path (need to fix filename according to > current > + directory) */ > + > + if (shared) > + fd = shm_open(filename, O_RDONLY | O_CLOEXEC, ACCESSPERMS); > + else > + fd = open(filename, O_RDONLY | O_CLOEXEC, ACCESSPERMS); > + > + if (fd < 0) return NULL; > + > + if (fstat(fd, &file_stat)) > + { > + close(fd); > + return NULL; > + } > + > + file = eina_hash_find(_eina_file_cache, filename); > + if (file && (file->mtime != file_stat.st_mtime > + || file->length != file_stat.st_size)) > + { > + create = EINA_TRUE; > + > + if (file->refcount == 0) > + { > + _eina_file_cache_lru = eina_list_prepend(_eina_file_cache_lru, > file); > + eina_hash_del(_eina_file_cache, file->filename, file); > + > + file = NULL; > + } > + else if (!file->delete_me) > + { > + file->delete_me = EINA_TRUE; > + _eina_file_cache_delete = > eina_list_prepend(_eina_file_cache_delete, file); > + } > + } > + > + if (!file || create) > + { > + n = malloc(sizeof (Eina_File)); > + if (!n) goto on_error; > + > + n->filename = eina_stringshare_add(filename); > + n->map = eina_hash_new(EINA_KEY_LENGTH(_eina_file_map_key_length), > + EINA_KEY_CMP(_eina_file_map_key_cmp), > + EINA_KEY_HASH(_eina_file_map_key_hash), > + EINA_FREE_CB(_eina_file_map_close), > + 3); > + n->rmap = eina_hash_pointer_new(NULL); > + n->global_map = MAP_FAILED; > + n->length = file_stat.st_size; > + n->mtime = file_stat.st_mtime; > + n->refcount = 0; > + n->fd = fd; > + n->shared = shared; > + n->delete_me = EINA_FALSE; > + > + eina_hash_set(_eina_file_cache, filename, n); > + } > + else > + { > + close(fd); > + > + n = file; > + > + if (n->refcount == 0) > + _eina_file_cache_lru = eina_list_remove(_eina_file_cache_lru, n); > + } > + > + n->refcount++; > + > + return n; > + > + on_error: > + close(fd); > + return NULL; > +} > + > +EAPI void > +eina_file_close(Eina_File *file) > +{ > + file->refcount--; > + > + if (file->refcount != 0) return ; > + > + if (file->delete_me) > + { > + _eina_file_cache_delete = eina_list_remove(_eina_file_cache_delete, > file); > + _eina_file_real_close(file); > + } > + else > + { > + _eina_file_cache_lru = eina_list_prepend(_eina_file_cache_lru, file); > + } > +} > + > +EAPI unsigned long int > +eina_file_size_get(Eina_File *file) > +{ > + return file->length; > +} > + > +EAPI time_t > +eina_file_mtime_get(Eina_File *file) > +{ > + return file->mtime; > +} > + > +EAPI const char * > +eina_file_filename_get(Eina_File *file) > +{ > + return file->filename; > +} > + > +EAPI void * > +eina_file_map_all(Eina_File *file, Eina_File_Populate rule) > +{ > + int flags = MAP_SHARED; > + > + if (rule == EINA_FILE_POPULATE) flags |= MAP_POPULATE; > + > + if (file->global_map == MAP_FAILED) > + file->global_map = mmap(NULL, file->length, PROT_READ, flags, file->fd, > 0); > + > + if (file->global_map != MAP_FAILED) > + { > + _eina_file_map_rule_apply(rule, file->global_map, file->length); > + file->global_refcount++; > + return file->global_map; > + } > + return NULL; > +} > + > +EAPI void * > +eina_file_map_new(Eina_File *file, Eina_File_Populate rule, > + unsigned long int offset, unsigned long int length) > +{ > + Eina_File_Map *map; > + unsigned long int key[2]; > + > + if (offset > file->length) > + return NULL; > + if (offset + length > file->length) > + return NULL; > + > + if (offset == 0 && length == file->length) > + return eina_file_map_all(file, rule); > + > + key[0] = offset; > + key[1] = length; > + > + map = eina_hash_find(file->map, &key); > + if (!map) > + { > + int flags = MAP_SHARED; > + > + if (rule == EINA_FILE_POPULATE) flags |= MAP_POPULATE; > + > + map = malloc(sizeof (Eina_File_Map)); > + if (!map) return NULL; > + > + map->map = mmap(NULL, length, PROT_READ, flags, file->fd, offset); > + map->offset = offset; > + map->length = length; > + map->refcount = 0; > + > + if (map->map == MAP_FAILED) > + { > + free(map); > + return NULL; > + } > + > + eina_hash_add(file->map, &key, map); > + eina_hash_direct_add(file->rmap, map->map, map); > + } > + > + map->refcount++; > + > + _eina_file_map_rule_apply(rule, map->map, length); > + > + return map->map; > +} > + > +EAPI void > +eina_file_map_free(Eina_File *file, void *map) > +{ > + if (file->global_map == map) > + { > + file->global_refcount--; > + > + if (file->global_refcount > 0) return ; > + > + munmap(file->global_map, file->length); > + file->global_map = MAP_FAILED; > + } > + else > + { > + Eina_File_Map *em; > + unsigned long int key[2]; > + > + em = eina_hash_find(file->rmap, &map); > + if (!em) return ; > + > + em->refcount--; > + > + if (em->refcount > 0) return ; > + > + key[0] = em->offset; > + key[1] = em->length; > + > + eina_hash_del(file->rmap, &map, em); > + eina_hash_del(file->map, &key, em); > + } > +} > + > + > > Modified: trunk/eina/src/lib/eina_main.c > =================================================================== > --- trunk/eina/src/lib/eina_main.c 2011-04-13 13:29:54 UTC (rev 58636) > +++ trunk/eina/src/lib/eina_main.c 2011-04-13 16:15:30 UTC (rev 58637) > @@ -124,6 +124,7 @@ > S(ustrbuf); > S(quadtree); > S(simple_xml); > + S(file); > #undef S > > struct eina_desc_setup > @@ -156,7 +157,8 @@ > S(strbuf), > S(ustrbuf), > S(quadtree), > - S(simple_xml) > + S(simple_xml), > + S(file) > #undef S > }; > static const size_t _eina_desc_setup_len = sizeof(_eina_desc_setup) / > > > ------------------------------------------------------------------------------ > Forrester Wave Report - Recovery time is now measured in hours and minutes > not days. Key insights are discussed in the 2010 Forrester Wave Report as > part of an in-depth evaluation of disaster recovery service providers. > Forrester found the best-in-class provider in terms of services and vision. > Read this report now! http://p.sf.net/sfu/ibm-webcastpromo > _______________________________________________ > enlightenment-svn mailing list > enlightenment-...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/enlightenment-svn > > ------------------------------------------------------------------------------ Forrester Wave Report - Recovery time is now measured in hours and minutes not days. Key insights are discussed in the 2010 Forrester Wave Report as part of an in-depth evaluation of disaster recovery service providers. Forrester found the best-in-class provider in terms of services and vision. Read this report now! http://p.sf.net/sfu/ibm-webcastpromo _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
