I think there must be a way to optimize, but I agree with you that it's not worth redesigning and rewriting your site.
In my case, I spent dozens of hours to reduce about 50% datastore reads of my blog. I could fetch entities with just one line of code before, and it's automatically memcached. But now, I have to query keys first, then check memcache to find missing entities, batch get the rest, set them into memcache, merge all the entities in the proper order. Since I have several memcache keys for each entity (like id, name, title, url, etc.), the reality is even worse. It seems you can't reduce the cost as acceptable as before without dropping many features or implementing in a extremely strange way. With the rework time, it's almost enough to move to other platform and scale better. Well, GAE may be easy to scale, but money isn't. I seldom see other platforms raise price, but GAE keeps raising price and reducing the free quotas, at least 3 times I known. ---------- keakon My blog(Chinese): www.keakon.net Blog source code: https://bitbucket.org/keakon/doodle/ On Fri, Nov 4, 2011 at 4:39 AM, Paul <[email protected]> wrote: > 1000+ entities on one page? How did you manage that? It really sounds > like architectural flaw. I am writing an app with huge amount of > constantly changing data (that needs to be persisted) and I don't have > that problem. > > On Nov 3, 8:21 pm, Xin Liu <[email protected]> wrote: > > My site's typical page is like this: > > > > http://www.v2ex.com/t/7577 > > > > As you have seen, a topic with dozens replies, each reply carry 3 > entities: > > reply, member and avatar. And there are other supporting dynamic elements > > on the page (most of them can be cached). > > > > But when someone replied to the topic, then it will trigger a lot > memcached > > objects to be removed. For hot topics, memcached objects can just survive > > minutes or even seconds. > > > > And there could be 100+ or even 1000+ entities on one page. > > > > If I knew they're going to charge datastore reads/writes, I may think > > another way to write the code. But now, it's too painful to rewrite the > > whole site. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
