I've noticed that the the instance of python that is running the GAE local server seems to just keep growing in size with use. With just manual web browsing (i.e. not using the bulk loader) I've observed it grow to consume over 100MB or RAM and noted that performance dropped off quite a bit as well, so I've got in the habit of killing it and starting again after a while. Anyway, its not just you experiencing perf problems with the GAE local server.
In my own performance investigations on the production server I've found that list attributes are quite expensive and that its best to pickle any attributes you don't need to query on into a blob (an entity with one blob attribute puts in fewer mcycles then one with a single int attribute on it). On Nov 19, 4:35 pm, "Jyoti Shete-Javadekar" <[EMAIL PROTECTED]> wrote: > Hi, > I am trying to load my development datastore using the bulk loader script. > However the import is very slow. I had to kill the import process since it > was not completed even after 12 hours. I have about 13K rows in the CSV > file. One data model entity is about 300 bytes. 10 entities are imported at > a time. The model has two unicode attributes, two unicode list attributes , > one url and one long attribute. I use unicode.split to populate list > attributes. I am running the bulk loader in a virtual machine having 512MB > memory. During the import about 91% memory is utilized. I have not specified > any custom index in index.yaml. > > Could someone please tell me why the import is so slow? What optimization > should I do to improve the performance? I have not yet loaded the data at > appspot. Above observations are for my local development server. Are there > any performance numbers available for the bulk loader script? > > Thanks in advance, > Jyoti --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
