On Thursday, July 5, 2012 3:14:52 PM UTC+1, barryhunter wrote: > > >> > >> Your 5.8M was READ operations anyway. It wasnt't the writing that cost > as > >> such. > > > > > > That is also strange. Why should it need 5.8m OPs to do 1600 reads? > > Well you didnt do 1600 reads. You obvisoully did a lot more. > > > > > > It was the update that was taking all this time. Updates from remote > shell > > are very slow. So it took 3 hrs for 1000 or so entities that it updated > and > > consumed 5.8m OPs in the process! > > Ah, well there you have your answer. The remote shell is not efficent > for certain things. > > Looking back at the code you posted in the thread, looks like reading > https://developers.google.com/appengine/articles/remote_api > > Would be worthwhile, seems like did two mistakes as noted in the > Limitations section. > > An artefact of how the remote_api works, caused all the reads :) >
To elucidate for anyone else reading, because this wasn't obvious to me (and I just hit this problem): It isn't _just_ that the remote_api has limitations / is inefficient, if you try and query past 1000 or so results, *it will loop until your quota is exhausted* retrying the query. That's why you have 5.8 million reads ... Rewriting the query to fetch batches with key based offset/filtering as suggested in the docs solves this, of course. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/IRmtKVIIxPAJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
