many thanks alberto
On 11 nov, 17:45, James Cooper <[email protected]> wrote: > Hi there, > > Good question. I think you'll probably need to make the export an > asynchronous process that iterates through your dataset and writes out > a CSV blob to a temp Datastore entity, or to memcache in <1MB > chunks. > > I've done some toy examples of this, but nothing in production. My > recipe involves 3 basic parts: > - AJAX progress-bar type control on the client UI (reusable) > - Main request handler that is response for queueing the export > request, polling for status, and fetching the completed results > (reusable) > - Async task queue handler that does the work of building the export > data (custom per-export type) > > Pseudo-code might work like this: > > - User clicks 'export' button, which fires off an AJAX request to your > GAE request handler URL to kick off an export > - Request handler is invoked in your app > - Creates an "ExportJob" entity in the datastore. ExportJob might > have these properties: > - Key key > - boolean isDone > - int entitiesProcessed > - List<String> memcacheKeys > - Queues up a TaskQueue invocation with params: offset=0, > exportJobKey=[key] (key of ExportJob entity in datastore) > - Returns the exportJobKey back to the client > - AJAX call completes - client now has a key to the job > - Client polls the server using that key, which returns the > "entitiesProcessed" and "isDone" fields, so you can tell the user > "Processed xxx entities" > > - ExportJobTask handler gets invoked in background > - In loop, loads a window of entities from the DataStore using the > offset passed in > - Appends CSV data to a StringBuilder > - Loop exit when a timeout upper limit is reached (say, 15 > seconds) OR when stringbuilder.size() is close to 1MB OR when all > entities are done > - Loads ExportJob with this key from DataStore > - Puts the CSV stringbuilder into a new memcache entry, using the > key + a numeric offset > - Appends the memcache key to ExportJob.memcacheKeys > - Sets ExportJob.entitiesProcessed += CSV line count > - If no more entities: > - Sets ExportJob.isDone = true > - Saves ExportJob back to DataStore > - Else > - Stores ExportJob back to DataStore > - Queues another taskqueue job back to itself with the same > key, and with offset = entitiesProcessed-1 > > - When job done, AJAX handler invokes initial request handler with key > and another flag "getData=1" or something > - Loads ExportJob from datastore > - Loops through ExportJob.memcacheKeys > - Loads CSV blob from memcache > - streams to servletresponse > - deletes memache entry > - Deletes ExportJob > - Done.. > > Maybe there's a simpler way? > > -- James > > On Nov 11, 2:12 am,alf<[email protected]> wrote: > > > > > hi all, > > > I need export a large dataset may be 25k entitys with 15string > > properties. I know limit of 1000 and 30 seconds. Any idea to can > > export data to csv, I can split in little chuck ok but how download?. > > I know appcfg but my feature is for end user and I would like do a > > simple page with a button. > > > Can I create a datataset in memory and build and put in a datastore to > > dowload. What happend if download process spent more than 30 sec. ? > > > many thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
