We need to write an app/site whose general usage pattern matches quite well AppEngine, except for one thing: the site will be collecting data / logs, and every now and then, we need to download bulk data from the database tables collecting such data / logs, for offline analysis. If we went with a "non-appengine" solution, we could obviously do this by generating e.g. csv files from DB queries or dumps, and then processing the files. Will it be possible to have the same kind of simple access to bulk data on AppEngine? We can of course build a (web) API that issues DB queries and produces csv files for download, but my concerns are:
1. I remember that there is a limit to how many records can be extracted from a DB using a query (1000?), so that we would have to implement the query with continuation parameters etc -- feasible, but complicating the design. 2. I worry about the execution time of the query (is it still true that processes taking over 1s are killed?). I guess access to log files / bulk data must be a pretty common requirement for many apps, so I am hoping someone has good words of advice... Many thanks!! Luca -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FD50slblUYEJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
