I am encountering multiple issues when trying to use PolyModels and
bulkload
data. At a minimum, it appears the documentation is incomplete on how
to
bulkload data for PolyModels. For instance, this
issue
http://code.google.com/p/googleappengine/issues/detail?id=1133
has been fixed, but I cannot find the documentation for why one must
use 'model:' in the bulkloader config instead of
'kind:'.
Here is an example set of models, loaders and the issues I currently
encounter:
class
Record(db.Model):
...
class
Parent(polymodel.PolyModel):
ival =
db.IntegerValue()
class
Child(Parent):
sval =
db.StringValue(required=True)
record = db.ReferenceProperty(Record,
required=True)
bulkloader.yaml:
- model:
models.Child
name:
Child
connector:
simplexml
connector_options:
xpath_to_nodes: /Children/
Child
style:
element_centric
property_map:
- property:
__key__
external_name:
key
export_transform:
transform.key_id_or_name_as_string
import_transform: transform.create_foreign_key('Parent',
True)
- property:
ival
external_name:
ival
import_transform:
transform.none_if_empty(long)
- property:
sval
external_name:
sval
- property:
record
external_name:
record
import_transform: transform.create_foreign_key('Record',
True)
export_transform:
transform.key_id_or_name_as_string
Here is my XML data file which I am trying to import into the
datastore:
<Children>
<Child>
<key>11</
key>
<ival>10</
ival>
<sval>someval</
sval>
<record>1</
record>
</
Child>
<Child>
<key>2</
key>
<ival>20</
ival>
<sval>20val</
sval>
<record>2</
record>
</
Child>
<Child>
<key>3</
key>
<ival>30</
ival>
<sval>30val</
sval>
<record>1</
record>
</
Child>
</
Children>
There are two issues that I
hit.
a) If the bulkloader.yaml specifies a property transform for __key__
then, no
instances are uploaded (and bulkloader says N records successfully
transferred, N being correct).
b) If I implement workarounds for a) and proceed, then I hit the
second
issue. The second issue is that if a property is marked with
required=True, then
a BadValueError is thrown since the created model does not have any
properties
set.
Both issues center around DictConverter.__create_instance
in
bulkloader_config.py.
def __create_instance(self, input_dict,
bulkload_state):
key =
None
if
self._create_key:
key = self.__dict_to_prop(self._create_key, input_dict,
bulkload_state)
if isinstance(key, (int,
long)):
key = datastore.Key.from_path(self._transformer_spec.kind,
key)
if isinstance(key,
datastore.Key):
parent =
key.parent()
if key.name() ==
None:
return
datastore.Entity(self._transformer_spec.kind,
parent=parent,
id=key.id())
else:
return
datastore.Entity(self._transformer_spec.kind,
parent=parent,
name=key.name())
if
self._transformer_spec.model:
return
self._transformer_spec.model(key=key)
return datastore.Entity(self._transformer_spec.kind,
name=key)
For issue a), the code ends up creating an entity
with
self._transformer_spec.kind (and the specified id) which causes
problems later
in __run_import_transforms() which is unable to set attributes on the
instance with setattr(). So the instance ends up being
empty.
The workaround I tried was to make the code
so:
if
self._create_key:
key = self.__dict_to_prop(self._create_key, input_dict,
bulkload_state)
if
self._transformer_spec.model:
return
self._transformer_spec.model(key=key)
This works great until I hit issue b). It turns out that creating
models through
their constructions causes BadValueErrors during validation of
properties with
required=True.
At this point, I am stuck. I really want to continue keeping the
required=True
validators and also would like to use PolyModels for their
conceptual
elegance. Can anyone suggest some ways of
proceeding?
thanks,
Vasanth
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
