We cannot build HEAD of the engine branch.
The reworks on remove() method makes caller not to need hold items to be
removed.
gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I./include -fvisibility=hidden -pthread
-g -O2 -Wall -Werror -pedantic -Wmissing-prototypes -Wmissing-declarations
-Wredundant-decls -fno-strict-aliasing -MT memcached-memcached.o -MD -MP -MF
.deps/memcached-memcached.Tpo -c -o memcached-memcached.o `test -f
'memcached.c' || echo './'`memcached.c
cc1: warnings being treated as errors
memcached.c: In function ‘process_bin_delete’:
memcached.c:2476: error: unused variable ‘it’
memcached.c: In function ‘process_delete_command’:
memcached.c:3314: error: unused variable ‘it’
And, even if we fixed these matters,
$ ./configure --enable-default-engine
:
$ make
:
libtool: link: gcc -std=gnu99 -fvisibility=hidden -pthread -g -O2 -Wall -Werror
-pedantic -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls
-fno-strict-aliasing -o memcached memcached-memcached.o memcached-hash.o
memcached-thread.o
memcached-daemon.o memcached-stats.o memcached-topkeys.o memcached-cache.o
memcached-assoc.o memcached-default_engine.o memcached-items.o
memcached-slabs.o memcached-config_parser.o memcached-util.o
memcached-genhash.o memcached-config_parser.o
memcached-util.o memcached-genhash.o -lm -ldl -levent -pthread -Wl,-rpath
-Wl,/usr/local/lib
memcached-config_parser.o: In function `parse_config':
/home/kaigai/repo/memcached-selinux/config_parser.c:69: multiple definition of
`parse_config'
memcached-config_parser.o:/home/kaigai/repo/memcached-selinux/config_parser.c:69:
first defined here
memcached-util.o: In function `ntohll':
/home/kaigai/repo/memcached-selinux/util.c:142: multiple definition of `ntohll'
memcached-util.o:/home/kaigai/repo/memcached-selinux/util.c:142: first defined
here
memcached-util.o: In function `htonll':
/home/kaigai/repo/memcached-selinux/util.c:146: multiple definition of `htonll'
memcached-util.o:/home/kaigai/repo/memcached-selinux/util.c:146: first defined
here
memcached-util.o: In function `vperror':
/home/kaigai/repo/memcached-selinux/util.c:108: multiple definition of `vperror'
memcached-util.o:/home/kaigai/repo/memcached-selinux/util.c:108: first defined
here
memcached-util.o: In function `safe_strtof':
/home/kaigai/repo/memcached-selinux/util.c:93: multiple definition of
`safe_strtof'
memcached-util.o:/home/kaigai/repo/memcached-selinux/util.c:93: first defined
here
memcached-util.o: In function `safe_strtol':
/home/kaigai/repo/memcached-selinux/util.c:78: multiple definition of
`safe_strtol'
memcached-util.o:/home/kaigai/repo/memcached-selinux/util.c:78: first defined
here
memcached-util.o: In function `safe_strtoul':
:
In the Makefile,
default_engine_la_SOURCES= \
assoc.c assoc.h \
default_engine.c default_engine.h \
items.c items.h \
slabs.c slabs.h
default_engine_la_SOURCES += $(libmcd_util_la_SOURCES)
Does it try to link libmcd_util.la twice when built-in default engine?
Thanks,
(2010/04/01 8:59), KaiGai Kohei wrote:
> Trond,
>
>> (5). The remove() method
>>
>> When its definition is modified to deliver the key, not item itself?
>> The memcached calls ->get() method prior to ->remove() invocations, although
>> the given request never wants to read contents of the item.
>
> This commit fixes the matter, and it enables us to see get() method as an
> operation
> to read contents of the item. It is a good progress for me.
>
> | commit 61a362fd12c8ed1a6e30f0d018266920006f45ab
> | Author: Trond Norbye<[email protected]>
> | Date: Tue Mar 30 15:59:23 2010 +0200
> |
> | Refactor: remove() in the engine interface should not require an item
> |
> | Previously remove of an item required 3 calls through the engine API in
> order
> | to remove an item (get, remove, release), and the core never had the
> item
> | when it wanted to remove an item so it made perfect sense to push all
> the
> | logic down to the engine.
>
> Thanks,
>
> (2010/03/05 16:02), KaiGai Kohei wrote:
>> Now I'm under working on access control feature with engine framework.
>> The git repository is at:
>> http://github.com/kaigai/memcached-selinux
>>
>> Under the development, I could find some matters to implement such kind
>> of features on the engine framework.
>>
>> (1). We have no server API to obtain socket file descriptor.
>>
>> SELinux provides an API to get security context of the peer process which
>> connects to the server process. This API takes an argument to inform socket
>> file descriptor.
>> However, memcached encapsulates conn->sfd, and we have no official interface
>> to translate the given cookie pointer to the socket file descriptor.
>> As a workaround, I cast the cookie into conn, and refer the socket.
>>
>> I need a server API to obtain a socket file for the given cookie.
>>
>>
>> (2). Order of the invocation of callback functions.
>>
>> The SELinux module wants to use engine specific data region in the conn
>> structure, it defines the following data structure which also provide
>> a capability to sore engine specific data of the secondary module.
>>
>> typedef struct selinux_connect
>> {
>> security_id_t sid; /* security context of the client process */
>> void *data; /* private data for the secondary engine */
>> } selinux_connect_t;
>>
>> The selinux_connect_t object is allocated and initialized at ON_CONNECT
>> callback, and it allows secondary module to register its engine specific
>> data. Then, it is released at ON_DISCONNECT callback.
>>
>> It means ON_CONNECT callback of SELinux must be invoked earlier than any
>> other modules, and ON_DISCONNECT callback of SELinux must be invoked last.
>> But we have no way to ensure the order of callback invocations.
>>
>> Now, I did a hack depending on internals of memcached.
>> The memcached calls a callback function earlier registered later, so
>> I registered ON_DISCONNECT callback prior to initialization of the secondary
>> engine, then I registered ON_CONNECT callback after the initialization.
>>
>>
>> http://github.com/kaigai/memcached-selinux/blob/selinux/selinux_engine.c#L310
>>
>> As long as the secondary engine registers its callbacks in the initialize()
>> handler, it will work well.
>>
>>
>> (3). ENGINE_HANDLE is not delivered to a few engine handlers.
>>
>> The engine framework does not deliver ENGINE_HANDLE pointer to the following
>> handlers.
>> - item_get_cas()
>> - item_set_cas()
>> - item_get_key()
>> - item_get_data()
>> - item_get_clsid()
>>
>> If we use the first dozens bytes of the item data field to store security
>> context of the item, we have to specify how many first bytes will be used
>> to store the security attribute.
>> The ENGINE_HANDLE can contain some of meaningful information, and here is
>> no reason why we cannot deliver it.
>>
>>
>> (4). The way to store per item security attribute.
>>
>> It is the most headache matter for me.
>>
>> A straightforward idea is to store security context of the item as a part
>> of data field.
>> For example, when we receive a pair of key="aaaa" and value="xyzxyz",
>> selinux engine can modify the given data field, then it delivers the modified
>> key/value pair. If the default security context is "classified", it may be
>> modified to "classified\0xyzxyz", for instance.
>> The secondary module will store the given key / modified value pair as is,
>> even if it has backing storage system.
>>
>> User -----+
>> | key = "aaaa", value="xyzxyz"
>> V
>> Memcached
>> |<-- nbytes -->
>> | | aaaa | xyzxyz |
>> | |________|______________|
>> | key value
>> |
>> V
>> selinux engine
>> |<--- nbytes + 11 ------>
>> | | aaaa | classified\0 | xyzxyz |
>> | |________|________________________|
>> | key value
>> V
>> secondary engine (such as default_engine)
>>
>> However, here is a headache.
>> The secondary engine assumes the data field of the item is numeric on
>> CAS operations, although the primary engine modified the data field to
>> store its security attribute.
>>
>> One idea is to add "offset" as an argument of the engine handlers.
>> In above example, the value has 17 bytes in total. 11 bytes of them are used
>> to selinux engine, and rest of them are used to the secondary engine.
>>
>> If item_set_cas() handler has an offset argument to inform secondary engine
>> how many bytes are already consumed by the primary engine, we can store the
>> modified value correctly, even if selinux engine modified it.
>>
>> In addition, the core memcached should not refer it->nbytes directly,
>> because it->nbytes is the total length of the item with value and security
>> context. The item_get_data() should return both of data pointer and nbytes
>> that engine modules want to show the core memcached. In this case, it should
>> return 6, not 17.
>>
>>
>> (5). The remove() method
>>
>> When its definition is modified to deliver the key, not item itself?
>> The memcached calls ->get() method prior to ->remove() invocations, although
>> the given request never wants to read contents of the item.
>>
>> Thanks,
>
>
--
KaiGai Kohei <[email protected]>
