Morning Oleg, Everyone,
I found that to get memcache working, I had do two things;
*) get rid of the cache writelocks inside main.py (eg; remark out
any val._SO_writeLock.acquire()) mostly due to explosions whilst pickling
*) subclass the SQLObject into another class (which I called
CompassObject, since our reservation system is called Compass ;) and
overloaded _SO_setValue to do a delete inside memcache on change and
overloaded 'get' such that it looks up memcache. I have to del/create
_SO_writeLock, sqlmeta and _connection on each object when it's returned.
Note that if someone was calling a series of 'setValue' instead of
one large 'set', this would cause a lot of cache deletes and then misses
when it tries to set. Thankfully, we have already audited the codebase
for that (it was about a 3-4% "win" for us to change it over).
Also, in a totally random thought, you will probably hit the 1mb
memcache limit when storing objects. Since we are storing objects
without blobs, it works. The only real workaround if people start to
complain about memcache 'not storing their objects' is to either go
Tokyo Tyrant or Redis. If you go Redis, you also end up with more speed
and m->s replication for free ;).
I am more than willing to help and/or bounce ideas off if you want.
The solution I have may not be 'perfect' but it's definitely performant
enough for our needs :D
Regards
Stef
On 02/01/2010 12:05 PM, Oleg Broytman wrote:
> Hello. I have returned to investigate the issue. Are you still interested?
>
> On Fri, Jul 31, 2009 at 01:10:34PM -0400, Stef Telford wrote:
>
>> Okay.. so.. assuming I can get a writeLock again after del'ing it..
>> we should be good there. If you can let me know exactly -what- in the
>> sqlmeta blows up, that may also help matters. The connection surely
>> explodes (And rightly so I think).
>>
> The problem with sqlmeta is that it's a class generated on the fly. It
> doesn't even have a module - and pickle protocol prefers stable classes,
> with proper modules and names.
> But I think it's easy to deal with it anyway. We don't need to pickle
> sqlmeta *class* - the class is in the corresponding SQLObject class. We
> probably even don't need to pickle sqlmeta instance - we can just recreate
> it on unpickling 'cause its only instance data is a reference to SQLObject
> instance and we know the instance - at the time of unpickling it's 'self'
> so we can reconstruct a sqlmeta instance using the known class and known
> instance data. Just for the (rare) case a user would have put some data
> into the instance I can save sqlmeta.__dict__ and restore it on unpickling.
> I am going to add the corresponding __getstate__ and __setstate__ to
> SQLObject and test them a bit.
>
> Oleg.
>
------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com
_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
