I starred it, but I don't even care if it is guaranteed if it scales with my instances. If every instance got X amount that would be ok.
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Philip Sent: Thursday, November 24, 2011 10:38 PM To: Google App Engine Subject: [google-appengine] Re: Potential Feature Request Memcache Allocation and Delayed Writes Brandon: http://code.google.com/p/googleappengine/issues/detail?id=6078 On Nov 25, 3:20 am, "Brandon Wirtz" <[email protected]> wrote: > Nick can you confirm my suspicion that MemCache does not scale in size > with the number of instances? > > The bigger ask here was really the ability to specify a size of memcache. > The preferred Muli-tenant model breaks down if a shared resource > doesn't scale with the "load" . If I have 1 gig of data being accessed > daily memcache helps a lot. When I push 6 gigs a day my cache hit > ratio falls to nothing. So right now I'm far better with lots of apps > optimized for 500-ish megs of unique data than I am one at 60gigs. > That feels backwards of what I want in a cloud environment. > > If there my assumption is wrong I need to go looking at why this is > happening. > > From: [email protected] > [mailto:[email protected]] On Behalf Of Nick Johnson > Sent: Thursday, November 24, 2011 5:52 PM > To: [email protected] > Subject: Re: [google-appengine] Potential Feature Request Memcache > Allocation and Delayed Writes > > Hi Brandon, > > On Thu, Nov 24, 2011 at 8:05 PM, Brandon Wirtz <[email protected]> wrote: > > Realizing my app is different than from most everybody elses.. > > I got to thinking about the thread where we were talking about reading > keys from memcache. I know all the reasons this is a bad idea. But I > got to thinking what if it wasn't? > > My app is just a big optimized cache, but I rely on the 3 tiers of > storage to make it all work and do so quickly. The sum of all the data > for the day is about 2 gigs. In a virtual machine environment I would > typically allocate a bunch of ram and every so often dump that to > longer term storage, but since most my caching is measured in minutes, > some is in days, and the longest I ever care about data is a month. > the only reason I need long term storage is so that when the memory > gets reset, or a new "instance" comes online that I don't have a 100% miss rate. > > The system you describe won't evict data unless you explicitly choose > to do so - you are in full control of cache eviction. This is not the > case for memcache, where memcache may choose to evict items without > notice. I don't think your proposal really applies to memcache, since > by the time you do your 'dump', some indeterminate amount of your > original data will have already been evicted from the cache. > > A much better approach if you don't mind data loss is a write-behind > cache, where you schedule a task to write the data to the datastore > after each time you update it in memcache. > > -Nick Johnson > > Why can't I do that with Memcache? Allocate 2 gigs, populate it with > data only on a version change. Once a day take all the values and dump > them back to datastore so that if the world ends that I don't have to > start from nothing. (maybe only write all the values that have an > expiration so many hours away) > > Since Backends share Memcache this "long" operation could be a > scheduled task and execute in the background. > > In my case this would save a lot of cycles since my writes are Local > Memory, MemCache, Datastore. And I do so with every piece of data > because I can't count on getting a hit from Memory or MemCache because of their volatility. > > But if I had a set amount of Memcache I wouldn't need to worry, it > wouldn't be volatile, and Google Could charge me for the resource. > Doesn't even have to be perfectly non-volatile because even if I only > "back-up" 75% of the data that's fine it is just a cache. > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected] > <mailto:google-appengine%[email protected]> . > For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. > > -- > Nick Johnson, Developer Programs Engineer, App Engine > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
