Hi Venkatesh, On Oct 1, 11:09 am, "Venkatesh Rangarajan" <[EMAIL PROTECTED]> wrote: > Hello there, > > Question 1 : I have an entity with 700K records. I am running search queries > on those and keep getting timeout error consistently. > Please advice on how I could avoid getting into these errors ? I have added > and __searchable_text_index and its is "building" for more than 24 hours. > > Question 2 : Did anybody figure out how to retrieve more than 1000 records ?
For a single query, you are able to fetch the first 1000 results, but fetching all 1000 at once will likely take a fair amount of time. This could be one source of the timeout errors you are seeing. A best practice for dealing with a large number of entities in a kind is to include a property which can be used for slicing your data set. Take a look at Brett Slatkin's talk on Building Scalable Web Applications with Google App Engine for advice on paging through a large data set http://sites.google.com/site/io/building-scalable-web-applications-with-google-app-engine . > > Question 3: Is google working on providing "wrappers" for datastore ? Like a > wrapper to get count(*) or sum(column) or retrieve more than 1000 records or > resume bulk upload etc. I know each one of us is writing our own code to > achieve this, but it would be desirable that google provides optimized > wrappers to make developers life easier. Please add another level of > abstraction to datastore. > > I am starting to get frustrated with Google Appengine. Run into > performance/scalability issues for simple operations ...simple things take > way too long and they never scale. > > The marketing statements below are false at this point... things are way too > convoluted at this point. > *No assembly required.* > Google App Engine exposes a fully-integrated development > environment<http://code.google.com/appengine/docs/whatisgoogleappengine.html>. > *It's easy to scale.* > Google App Engine makes it easy to design scalable applications that grow > from one to millions of users without infrastructure headaches. > > My error code > > Traceback (most recent call last): > File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", > line 499, in __call__ > handler.get(*groups) > File "/base/data/home/apps/payrate/9.10/Main.py", line 176, in get > for result in get_visas(keyword, int(page),s): > File "/base/data/home/apps/payrate/9.10/Main.py", line 138, in get_visas > visas = db_visas(keyword,offset) > File "/base/data/home/apps/payrate/9.10/Main.py", line 148, in db_visas > for result in query.Get(101, offset): > File "/base/python_lib/versions/1/google/appengine/api/datastore.py", > line 938, in Get > return self._Run(limit, offset)._Next(limit) > File "/base/python_lib/versions/1/google/appengine/api/datastore.py", > line 1227, in _Next > raise _ToDatastoreError(err) > File "/base/python_lib/versions/1/google/appengine/api/datastore.py", > line 1627, in _ToDatastoreError > raise errors[err.application_error](err.error_detail) > Timeout: datastore timeout: operation took too long. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
