Hi Robert, I got it! I have not considered that query offset also causes small operation.
thank you very much Robert for your input. On Wed, Feb 1, 2012 at 9:00 AM, Robert Kluin <[email protected]> wrote: > Hey Andrew, > So you need to remember that running a query will result in many reads > (the number of results, plus a scan iirc) -- not just one. That could well > be the source of your problem. > > As for why memcache isn't being effective, that's harder to say; that > could be a coding issue or a memcache issue. > > > Robert > > > > > > On Tue, Jan 31, 2012 at 07:18, Andrew Osipenko <[email protected]> wrote: > >> Hi Robert, >> >> Thanks for you being interested to help me. >> >> Let me tell my long sad story: >> I have Java appengine app. >> The app has only one page. This page performs search with various >> parameters, fetches 11 entities and displays search result. >> My app has 3400 page views per day and each day it exceeds it's datastore >> read quota. >> >> Lets count: (11 entities + 1 query) * 3400 = 41800 >> 41800 is nearly 50000 (I have not counted other pages) >> This looks ok. >> >> After this I added memcache with refresh interval = 1 hour. But I am >> still getting datastore read quota exceeding. >> >> Appstats reports the following data: >> user.CreateLoginURL 674 datastore_v3.RunQuery 311 memcache.Get 309 >> memcache.Set >> 150 datastore_v3.Put 34 datastore_v3.Get 20 blobstore.CreateUploadURL >> 14 datastore_v3.Delete 8 mail.Send 3 >> Appstats data is not clear to me. According to docs it describes recent >> 1000 requests or so. >> user.CreateLoginURL is called on each page, so appstats describe 674 page >> requests >> datastore_v3.RunQuery=311 >> But why memcache.Set = 150? It should be = 311= datastore_v3.RunQuery = >> cache miss count >> >> Perhaps appstats is not the app which can provide exact numbers. >> >> My question is: do you have an idea why my app still get's datastore read >> quota exceeding even after memcache introducing? >> >> -thanks, >> Andrew >> >> >> On Tue, Jan 31, 2012 at 7:51 AM, Robert Kluin <[email protected]>wrote: >> >>> Hey Andrew, >>> What symptoms are you seeing exactly? >>> >>> >>> Robert >>> >>> >>> >>> >>> >>> On Mon, Jan 30, 2012 at 05:48, Andrew Osipenko <[email protected]> >>> wrote: >>> > Hi Robert, >>> > >>> > no, >>> > session disabling is the first thing I did to reduce datastore access >>> count. >>> > See >>> > >>> http://code.google.com/p/rent-map/source/browse/trunk/trunk/src/main/webapp/WEB-INF/appengine-web.xml >>> > >>> > Second thing I did was memcache introducing. So I used all standard >>> > datastore friendly technics. >>> > >>> > >>> > On Mon, Jan 30, 2012 at 7:33 AM, Robert Kluin <[email protected]> >>> > wrote: >>> >> >>> >> Are you using sessions? >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> On Sun, Jan 29, 2012 at 05:37, Andrew Osipenko <[email protected]> >>> wrote: >>> >> > Hi Frank, >>> >> > >>> >> > Have you solved it? >>> >> > >>> >> > I have nearly the same problem. >>> >> > >>> >> > I think that appstats do not count some datastore read operations. I >>> >> > thought >>> >> > my application accessed datastore during intialization before >>> appstat >>> >> > filter >>> >> > is being invoked. But I can't find anything bad during my app >>> >> > initialization. Do you have any ideas? >>> >> > >>> >> > -- >>> >> > You received this message because you are subscribed to the Google >>> >> > Groups >>> >> > "Google App Engine" group. >>> >> > To view this discussion on the web visit >>> >> > https://groups.google.com/d/msg/google-appengine/-/Tr6-P0a_XWAJ. >>> >> > >>> >> > To post to this group, send email to >>> [email protected]. >>> >> > To unsubscribe from this group, send email to >>> >> > [email protected]. >>> >> > For more options, visit this group at >>> >> > http://groups.google.com/group/google-appengine?hl=en. >>> >> >>> >> -- >>> >> You received this message because you are subscribed to the Google >>> Groups >>> >> "Google App Engine" group. >>> >> To post to this group, send email to >>> [email protected]. >>> >> To unsubscribe from this group, send email to >>> >> [email protected]. >>> >> For more options, visit this group at >>> >> http://groups.google.com/group/google-appengine?hl=en. >>> >> >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "Google App Engine" group. >>> > To post to this group, send email to [email protected] >>> . >>> > To unsubscribe from this group, send email to >>> > [email protected]. >>> > For more options, visit this group at >>> > http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
