Hi djerdo, You can specify key names by overriding the generate_key method of your loader. See here for more details: http://blog.notdot.net/2009/9/Advanced-Bulk-Loading-part-2-Customization
-Nick Johnson On Sat, Sep 26, 2009 at 1:54 PM, djerdo <[email protected]> wrote: > > > > On 17 Sep, 23:25, djerdo <[email protected]> wrote: > > Using the bulkloader with the dev appserver, the script adds rows > > (Entities) at a progressively slower rate, to the point where it > > becomes unusable when the csv file is large (20,000 rows). Why? Is > > this a known issue? Are there any workarounds? > > > > Thanks > > Ok, I wasn't doing it right. FWIW - to resolve reference property > fields, I was using code such as: > > select __key__ from MyModel where id = :1 > > id being the unique field in the incoming csv file. This seemed to > work but when i figured out how to specify a key_name when adding an > entity, I could replace this with > > MyModel.get_by_key_name(id).key() > > which sped things up a lot. > > How to specify a key name when bulkloading entities? I couldn't find > any examples of this, so (again FWIW), here's a 'Baseloader' class > that i used: > > from google.appengine.api import datastore > from google.appengine.ext import bulkload > > class BaseLoader(bulkload.Loader): > #abstract class > > def __init__(self): > bulkload.Loader.__init__(self, self.model, self.mapping) > > def HandleEntity(self, entity): > if not hasattr(self, 'unique_field'): > return entity > #entity is a dict subclass > keyname = entity.pop(self.unique_field) > named_entity = datastore.Entity(self.model, name=keyname) > named_entity.update(entity) > return named_entity > > Example: > > class ContactLoader(Loader): > model = 'Contact' > mapping = [ > ('id', str), > ('name', str), > ('phone', str), > ] > unique_field = 'id' > > > > > > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
