Re: Do beaker session store supports using memcached ?
On Dec 31 2008, 5:01 pm, Tycon adie...@gmail.com wrote: if it's not a system service then why does it have it's own /etc/ init.d file (the way it's packaged for most major distributions) ? oh I always just build it from source :) .that theres an /etc/ init.d entry indicates its a decision of those distros to have a single memcached daemon running, its not a decision of memcached. I've read through the memcached website very thoroughly regarding this question and I don't see any guidelines on this issue in either direction. I wouldn't think memcached itself has much of an opinion on how its run. I would almost guarantee that high volume websites which use memcached do not multipurpose a single memcached instance across multiple unrelated applications, though...apart from the difficulty of multiple performance profiles affecting the single memcached process in different ways, why bother taking the risk that two different applications might use the same keynames for different purposes. While it is possible to have a memcached server dedicated to one application, you can't assume this is the default. with that said, of course I agree on this. I made it pretty clear in the CHANGELOG that the remove() method with the memcached backend now does flush_all(), which is not a method that's called by the library except by the erroneous usage of it in Session, and I've sent Ben a patch which changes Session to store a single dict on one key in the namespace regardless of backend so namespace.remove() would not be used. I've also advised that namespace.remove(), just like namespace.keys(), raise a NotImplementedError() when using the memcached backend. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Do beaker session store supports using memcached ?
On Dec 31, 3:19 pm, Tycon adie...@gmail.com wrote: whatever you end up doing NEVER EVER DO a flush_all on memcached. Memcached is a global system service, it is not your private scratch pad. yessir ! though I've never considered a single memcached process as a global service, like say apache. since it's not bound to a public port or anything like that. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Do beaker session store supports using memcached ?
if it's not a system service then why does it have it's own /etc/ init.d file (the way it's packaged for most major distributions) ? While it is possible to have a memcached server dedicated to one application, you can't assume this is the default. On Dec 31, 12:48 pm, Michael Bayer zzz...@gmail.com wrote: On Dec 31, 3:19 pm, Tycon adie...@gmail.com wrote: whatever you end up doing NEVER EVER DO a flush_all on memcached. Memcached is a global system service, it is not your private scratch pad. yessir ! though I've never considered a single memcached process as a global service, like say apache. since it's not bound to a public port or anything like that. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Do beaker session store supports using memcached ?
Cause if I use this setting in development.ini : beaker.session.type = ext:memcached beaker.session.url = 127.0.0.1:11211 then if I try to access the session global, I get this error: NotImplementedError: Memcache caching does not support iteration of all cache keys This is raised from Module beaker.ext.memcached:76 in keys(). Am I doing something wrong, or is using memcahced store for sessions messed up ?? Everything works if I don't set the beaker.session.type in development.ini, cause then it's using the filesystem to store sessions. Funny that when using the filesystem it's pickling the session object and storing it in a file, but when trying to use memcached, it's actually trying to store each session attribute as a separate record in memcached. And the code in beaker.ext.memcached seems to actually do a flush_all of memcached when the session is to be removed ! hahahahaha - that means that it will flush everything else from memcached, unless it expects memcahced to be dedicated to only one session's data (at a time) Not to mention the expiration memchanism also doesnt seem to work, as any setting I tried it always called memcached with no expiration specified. Not that it matters cause of the above error nothing works anyway. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Do beaker session store supports using memcached ?
On Tue, Dec 30, 2008 at 1:10 PM, Tycon adie...@gmail.com wrote: So I guess no one is actually using memcached for storing sessions ? Or of you do, do you define your own memcached interface instead of the Beaker memcached interface for session storage ? I haven't use memcached, and I doubt many Pylons applications do. Some of the problems you're describing look like they may be Beaker bugs, but I think Ben is the only one who knows enough about Beaker's internals to evaluate that. -- Mike Orr sluggos...@gmail.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Do beaker session store supports using memcached ?
Let me clarify that using memcached in beaker doesn't work ONLY FOR STORING SESSIONS. It does work for regular caching in beaker (e.g. using @beaker_cache decorator to cache controller actions etc). This is not a bug, it's basically an incorrect implementation, which uses wrong design and is completely untested. For storing sessions in mecached you need to do the same thing as when storing session in a file - pickle the session and store it as a single key in memcached. So you can update or remove the session object in a single mecached call, and all the sessions attributes will have the same expiration. Storing each session attribute as a separate record is incorrect because memcached doesn't allow you to update/iterate/delete groups of records together. On Dec 30, 2:51 pm, Mike Orr sluggos...@gmail.com wrote: On Tue, Dec 30, 2008 at 1:10 PM, Tycon adie...@gmail.com wrote: So I guess no one is actually using memcached for storing sessions ? Or of you do, do you define your own memcached interface instead of the Beaker memcached interface for session storage ? I haven't use memcached, and I doubt many Pylons applications do. Some of the problems you're describing look like they may be Beaker bugs, but I think Ben is the only one who knows enough about Beaker's internals to evaluate that. -- Mike Orr sluggos...@gmail.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to pylons-discuss+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Do beaker session store supports using memcached ?
On Dec 30, 2008, at 3:09 PM, Tycon wrote: It does work for regular caching in beaker (e.g. using @beaker_cache decorator to cache controller actions etc). This is not a bug, it's basically an incorrect implementation, which uses wrong design and is completely untested. For storing sessions in mecached you need to do the same thing as when storing session in a file - pickle the session and store it as a single key in memcached. So you can update or remove the session object in a single mecached call, and all the sessions attributes will have the same expiration. Storing each session attribute as a separate record is incorrect because memcached doesn't allow you to update/iterate/delete groups of records together. Yup, that pretty much sums it up. A recent refactor of the memcached container (the storage system Beaker uses) resulted in the session breakage. I'm not sure how easy it will be to fix this as right now the exact same container object is used whether its for caching, or for sessions. As you clearly note, in this case they should definitely have different implementations. I'll try and get this fixed before 0.9.7 final comes out (which I was hoping to do in the next few days). Cheers, Ben smime.p7s Description: S/MIME cryptographic signature
Re: Session store
On 4/27/07, Cliff Wells [EMAIL PROTECTED] wrote: On Fri, 2007-04-27 at 17:24 -0700, Shannon -jj Behrens wrote: This topic is covered very nicely in Building Scalable Web Sites and Scalable Internet Architectures. Okay, one more plug for these books and there's going to be a demand for full disclosure ;-) hahahaha Nope, I don't have any connection to them. It takes me so long to read books that if I invest 3 months of train-reading-time, I want to get as much mileage out of the knowledge as possible ;) Happy Hacking! -jj -- http://jjinux.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Session store
On 4/24/07, Damjan [EMAIL PROTECTED] wrote: You can store sessions (or the needed info) in just cookies, a database (MySQL, Postgresql, etc) or memcached. It's not neccesseary to store the session in files. Agreed. memcached is very popular for scalable session servers, not just in the Pylons world. Cookies are a good option if you can keep the sessions really small. You may want to sign and/or encrypt them to prevent tampering. At my company, we're trying our hardest to simply not use sessions in order to avoid the problem. An ugly solution is to have your load balancer implement session stickiness so that you can use Pylons normal session mechanism. However, that's ugly because you can't just pull servers out of production anytime you want. This topic is covered very nicely in Building Scalable Web Sites and Scalable Internet Architectures. Happy Hacking! -jj -- http://jjinux.blogspot.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Session store
On Fri, 2007-04-27 at 17:24 -0700, Shannon -jj Behrens wrote: This topic is covered very nicely in Building Scalable Web Sites and Scalable Internet Architectures. Okay, one more plug for these books and there's going to be a demand for full disclosure ;-) Cliff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
Re: Session store
After some digging I came up with the following solution: In config/middleware.py at the top op make_app I've added the following lines: from sqla import SQLAlchemyNamespaceManager app_conf['session_namespace_class'] = SQLAlchemyNamespaceManager app_conf['session_dburi'] = 'mysql://dbuser:[EMAIL PROTECTED]/ sessions_db' I still need to move the sqlalchemy.create_engine call from the constructor of SQLAlchemyNamespaceManager so that requests can share the same connection to the session database. But other than that it seems to work. I am interested to hear what setup people are using for scalable and longer-lived sessions. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups pylons-discuss group. To post to this group, send email to pylons-discuss@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~--~~~~--~~--~--~---
