Re: [Webware-discuss] Anybody made a memcached SessionStore?

[Ben Parker]

I've been thinking more on this issue and I've decided to implement a database-backed SessionStore, specifically using MySQL memory (or heap) tables. So the obvious first question is somewhat a repeat of David's - Has anyone built a SessionDatabaseStore before? I did some more reading up on memcached and was scared off using it as a session store. No data is replicated, only hashed and stored to a particular server, which means that portion of the cache is lost when that server dies. You can wrap your own replication around memcached, to store the data twice with two different keys guaranteed to hash to different servers, but at that point I started looking at the built-in replication of MySQL instead. I need active sessions to remain available when web/app servers or db servers die. I like memcached for it's intended purpose though. Currently I'm using a home-rolled in-process caching scheme for frequently accessed db objects (i.e. singleton classes with dictionaries keyed by primary key of the db table). My intention when moving to the clustered environment was to use some kind of RPC between the appservers to delete an object from the cache when it is saved to the db. Then each appserver machine lazily updates as needed. This means each appserver potentially has the full cache, as opposed to memcached which spreads the cache over multiple servers. If the total object cache is small enough to fit in the usable memory of each appserver machine, then the per-machine cache will obviously always be faster (assuming the app has more cache hits than db writes!). But once the object cache grows beyond that single server limit, memcached looks pretty sweet, and the python API looks quite straightforward. Regarding the "hard to find bugs" when using SessionFileStore, they are probably related to multiple threads accessing and saving the session, with one clobbering the other. This can happen if thread A grabs session 1, then thread B grabs session 1, then thread A saves session 1, then thread B saves session 1 (overwriting thread A's save). This can never happen with SessionMemoryStore because it doesn't pickle the session objects - they are kept in memory in a dictionary - meaning that thread B "sees" thread A's changes as soon as they occur, because both threads point to the same object. A SessionStore backed by a database or memcached would exhibit the same functionality as the SessionFileStore. It's another concern I have to investigate with when migrating to a cluster... If others have input/experience with clustered appservers, or alternative session store algorithms, please speak out! Peace - Ben Hancock, David (DHANCOCK) wrote: We aren't clustered (currently), but soon I want to run the ThreadedAppServer on multiple machines again. We backed away from this a while ago for other reasons, but at the time we used the session FileStore with an NFS-mounted directory. We tried sticky sessions in our load-balancer (Cisco LocalDirector) but we never got them to work reliably.   Our applications (WebKit included) are already quite database-intensive, so I'd rather try something like memcached before doing an additional update on each page switch. The memcached daemons in a few places on the network seems ideal, and my (simple-minded) benchmarks comparing Oracle retrievals vs memcached retrievals shows them to be 10-12 times faster.   Before implementing our earlier load-balanced configuration, we benchmarked the difference between FileStore and MemoryStore. It was only about a 15% penalty for the FileStore. That testing was with a local filesystem, not NFS. FileStore can introduce some hard-to-find bugs when two windows of the same session attempt to update the session file at the same time.   I haven't got any numbers comparing MemoryStore (or DynamicStore) with memcached storage, but I would expect MemoryStore to perform a few percent better (data is already in the process, no need for possible network traffic).   I'm still hoping to hear of an existing implementation, though... Cheers! -- David Hancock | [EMAIL PROTECTED] | 410-266-4384 -----Original Message----- From: Ben Parker [mailto:[EMAIL PROTECTED]] Sent: Tuesday, October 25, 2005 6:22 PM To: Hancock, David (DHANCOCK) Cc: webware-discuss@lists.sourceforge.net Subject: Re: [Webware-discuss] Anybody made a memcached SessionStore? Hancock, David (DHANCOCK) wrote: We're thinking about creating a memcached SessionStore backend, but (being quite lazy and not as smart as most other people) I'd much rather copy or adapt somebody else's code. Has anyone done this already? Any advice before I start in on this project? Interesting - are you looking to use this in a clustered environment?  I wonder if memcached would give any performance benefit over SessionMemoryStore for a single server. Fairly soon, I will have to implement clustered app server machines for load-balancing and failover, and I'm looking at using a database session store to achieve session management across multiple machines without requiring sticky sessions. although I'm still just beginning to get the test environment together. peace! Ben ------------------------------------------------------- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42" plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php _______________________________________________ Webware-discuss mailing list Webware-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/webware-discuss