On Nov 17, 6:37 pm, Jon McAlister <[EMAIL PROTECTED]> wrote: > On Nov 16, 10:05 pm, Anders <[EMAIL PROTECTED]> wrote: > > > And another thing: In a distributed system, must there really be any > > functional difference between RAM and disk memory? Of course, disk > > access is magnitudes slower than accessing RAM, but as I see it, at > > least theoretically, in a distributed system RAM could be made to > > function as a permanent storage. > > > Think of a single huge distributed virtual memory where data is stored > > in a duplicated and fail-safe way. Then the disk space would only be > > used for swapping memory pages to and from faster memory such as RAM > > (and in the future maybe even other forms of fast memory). Only at the > > very lowest level (Distributed OS 'kernel' level) would developers > > need to know if some chunk of memory is physically stored on disk or > > on some other form of memory. > > The limitations and design concerns you're talking about here are not > unique to App Engine. The tradeoffs between disk and memory, and the > tradeoffs between different kinds of cache (e.g. disk buffer cache, L1/ > L2, memcache), are core computer science. What's the latency? The > throughput? The cache-hit rate? Does the kernel or does the app have a > better understanding of the data access patterns? If the operating > system was not able to solve these issues and remove them from the > mind-set of the developer, then it's unlikely that App Engine will :-P > > In the case of automatic caching of the datastore, it's the case that > the application does have a better opportunity to cache than we could > from behind the scenes. The app knows better what is the tradeoff > between data staleness and end-user latency. The app knows how to keep > the cached data consistent with the true data. If we tried to do these > things within the datastore, since it is an API that is perfectly > consistent and guarantees that new writes immediately appear in query > results, we wouldn't be able to achieve a good cache-hit rate as every > datastore write would force us to invalidate every single query-cache > for an app. The app can do much better than we can for datastore > caching. > > Jon
Hmm... I looked at the dashboard for my application. It shows that a datastore query in a frequently accessed page takes a lot of CPU time. It's a simple query but it must be the query that takes most of the CPU time because the rest of the code for that page is basically only a few print statements. Maybe it could be a good idea to use Memcache for that page. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
