If you are using bulkloader you might want to tune some flags from: http://code.google.com/appengine/docs/python/tools/uploadingdata.html#Command_Line_Arguments
The particular ones that might interest you are: --num_threads --batch_size --bandwidth_limit --rps_limit --http_limit Running with the defaults will probably lead to really slow uploads especially with very small entities that can be batched 1000 at a time. That would give you a 100x speedup right off the bat. 2009/4/13 Josh Steiner <[email protected]> > > So lets see, I need to import 12 million records... > > 83.33 days to import. > > On the latest GAE blog posting it says: > > "Database import: move GBs of data easily into your App Engine app. > Matching export capabilities are coming soon, hopefully within a > month." > > Clearly this is going to have to be something other than bulkloader.py > , anyone have any more info on this new capability? > > -josh > > On Sun, Apr 12, 2009 at 5:26 AM, 风笑雪 <[email protected]> wrote: > > You should do it this way: > > http://code.google.com/appengine/docs/python/tools/uploadingdata.html > > It will take about 1 second to insert 100 records, so I think 4000+ > record > > will be uploaded in 1 minute. > > > > 2009/4/12 sevenstar <[email protected]> > >> > >> hi all, > >> I have some 4000+ records to be uploaded on the appengine. > >> Does GAE allow to upload so much data, if yes then i have some > >> questions? > >> 1) How to upload that data? > >> 2) How much time will it take to complete this thing? > >> 3) What is the fastest method to upload it. > >> > >> Please reply ASAP. I have been reading about it but couldn't find a > >> solution. > >> > > > > > > > > > > > > > -- Alkis --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
