Likely you aren't running large enough datasets to see huge advantages. If you are on a free Appengine, I'm going to go so far as to say that the trying to check if something is in memcache likely costs you more than just doing the read from the datastore because your miss ratio will be so high.
Mem cache is a scaling solution not "this will always be faster" best practice. From: [email protected] [mailto:[email protected]] On Behalf Of Daniel Sent: Sunday, May 08, 2011 2:57 PM To: [email protected] Subject: [google-appengine] First try at memcache doesn't seem to help much I have a game server running on AppEngine for my iOS game "SkyWords". The most cpu intensive and most commonly run method I have accounts for over 50% of my cpu usage. That method queries for all the games a user is attached to and returns them. If a user or any of their opponents haven't updated one of the games the result will be the same.. so I implemented memcache to see if I could cut down on cpu usage as I am creeping up to my quota limit. Memcache doesn't seem to help much for me, but this is the first time I'm trying to use it.. maybe I'm doing something stupid Here is the method in question with irrelevant details stripped: class GetActiveGameListHandler(SkyWordsHandler): def handle(self): sender = self.getUser(self.request.get('id'), self.request.get('password')) if not sender: return #See if data is in memcache.. if so return it. Each user has it's own memcache entry based on user email. #What's stored in memcache is just a return string of all the games data = memcache.get("gameList-"+sender.email) if data is not None: self.response.out.write(data) #logging.info("Found memcache: %s", str(sender.email)) return #Get the last 100 where I am the sender query = db.Query(SkyWordsGame) query.filter('sender = ', sender.email).filter('senderHideGame = ', False) senderGames = query.fetch(100) #Get the last 100 where I am the recipient query = db.Query(SkyWordsGame) query.filter('recipient = ', sender.email).filter('recipientHideGame = ', False) recipientGames = query.fetch(100) #All Games games = recipientGames + senderGames #Sort Games games.sort(key=lambda x: x.lastUpdate, reverse=True) e = ElementTree.Element("ok") userRecordDict = {} actualLength = 0 for game in games: #Collect info about each game and shove into the ElementTree memcache.add("gameList-"+sender.email, ElementTree.tostring(e)) self.response.out.write(ElementTree.tostring(e)) I have other methods where I update a game, resign a game, etc.. anything that would effect the validity of the cache. For any of those methods I invalidate the cache so it's regenerated the next time the query the game list. I have no time limit on my cache as I should know when it's going to get invalid and can strip it then memcache.delete("gameList-"+sender.email) I suppose I'm not seeing a big gain if the data is often invalid and keeps getting deleted.. I almost feel like I'm seeing just as heavy if not heavier cpu usage. Only explanation I have for that is if the cache is often invalidated and regenerated and the memcache.add is just adding to my cpu usage. So the overhead of the .add is outweighing how much I'm using the cache. Sort of hard to believe but maybe. Any thoughts ? The other angle I have is to make a new list attribute on each user's record and list every game they are a part of there.. so I don't have to do the big queries.. that's going to be a bit more work so I went for this angle first. Thanks Daniel -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
