Okay, so I've uploaded the code for the serializer so you can take a
look: https://gist.github.com/9167eff88d787b3b5279
Disclaimer: Again, I freely admit that this code is extremely messy
and may well contain a few bugs (I don't have any test cases for it).
It isn't something I use on a production server; we just use it to
load test data on our dev machines. But, the idea is there and it does
serialize/deserialize (for what we use it for) well. It's pretty slow
on import due to all the queries it's making, and I'm sure you can
improve on that =)
On 30 Mar, 00:33, Russ Amos <taal...@gmail.com> wrote:
> Oliver, examining unique and unique_together attributes is exactly what I
> have in mind for attempting to follow relationships when deserializing. I
> realize it needs to be dealt with carefully in the inner workings of Django,
> to provide consistency, but anything you're willing to provide, I'm willing
> to read through and maybe refactor.
>
> So, yeah, I'd like to take a look at what you've written. Pragmatism may not
> make for the cleanest code, but I think it's a great way to find "gotchas"
> that might otherwise be difficult to unearth.
>
> Russ
>
>
>
> On Sun, Mar 29, 2009 at 6:23 PM, Oliver Beattie <oli...@obeattie.com> wrote:
>
> > I'll admit that I haven't read your whole post (sorry), but one part
> > caught my eye… the bit about storing relationships not just as primary
> > keys. If I am right in thinking that you are wanting to do some sort
> > of "relationship following" I think I can probably help by providing
> > some initial (but a bit messy) code. I've recently run into the issue
> > of primary-key-dependant fixtures breaking in interesting ways on a
> > real database, so had to implement the relationship following (I have
> > a serializer I've named jsonfk). Basically, it will look at unique and
> > unique_together fields in models and figure out what it can use,
> > falling back to the primary key if either no unique fields exist or
> > the lookup fails on deserialization.
>
> > Sorry if I'm rambing a bit… it's late, but let me know if this is
> > something that may be of interest to see. It's not something I want to
> > release publicly since it's a bit of a mess and doesn't handle M2M
> > relations (on the project I've used it on M2M relationships aren't
> > really used).
>
> > On 29 Mar, 22:57, Russ <taal...@gmail.com> wrote:
> > > My apologies for the length!
>
> > > Concisely, I intend to provide the Django user with some granular
> > > control of the data to be serialized without sacrificing backwards
> > > compatibility for old code, or for users who need the straightforward,
> > > current functionality.
>
> > > Any serialized Model contains only the data in the source row of the
> > > database table. Regarding inheritance, this is only an issue when the
> > > model in question derives from one or more concrete models. In that
> > > case, however, any data contained on the far end of the one-to-one
> > > relation is not serialized at all. The overarching issue is rooted in
> > > the wayDjango serializers treat relationships. This is clearly a
> > > complex issue, because both shallow and deep serialization are useful
> > > in a large range of situations.
>
> > > For the purpose of argument and demonstration, please examine the
> > > following use case. A business wishes to begin providing their
> > > products online, but they wish to continue using the inventory
> > > management software with which they are familiar
> > > (henceforth,ezInventory). The models.py looks like:
>
> > > from django.db import models
>
> > > class Product(models.Model):
> > > name = models.CharField(max_length=200)
> > > description = models.TextField()
> > > # We're assuming all prices are in USD...
> > > price = models.DecimalField(max_digits=6, decimal_places=2)
>
> > > class Order(models.Model):
> > > products = models.ManyToManyField(Product)
> > > order_placed = models.DateTimeField()
>
> > > def total_price(self):
> > > return self.products.aggregate(models.Sum('price'))
>
> > > Django's serialization facilities make importing and exporting
> > > products and orders between their website and ezInventory a breeze.
> > > Six months later, however, they have a feature request.
>
> > > They would like to begin providing support for some of their products
> > > via subscription. This is an easy addition:
>
> > > # imports...
> > > # original models...
>
> > > class Subscription(Product):
> > > recurrence = models.PositiveIntegerField(choices=((12, 'annual'),
> > > (1, 'monthly')))
>
> > > Immediately, a problem is evident: the business relies on easy
> > > communication of data between ezInventory and the website, but Django
> > > serializes the class Subscription as follows:
>
> > > [{
> > > "pk": 13,
> > > "model": "app.subscription",
> > > "fields": {
> > > "recurrence": 12
> > > }
>
> > > }]
>
> > > Now, this is wonderful, as long as we know to pair this information
> > > with the product with id 13. However, ezInventory, though supportive
> > > of subscriptions in general, merely overwrites the product with id 13
> > > (assuming the products table was imported first) and complains about
> > > the missing fields.
>
> > > Skipping merrily back to the real world, here is the issue, plain and
> > > simple. Sometimes, you need to include more information than is just
> > > in the one model. Sometimes, that's going to appear in the form of
> > > inherited models, sometimes via one-to-many or many-to-many
> > > relationships, and sometimes just little relevant bits of information
> > > that may not actually be in the model at all. There are numerous open
> > > tickets about this issue [1][2][3][4]. Clearly, this needs some
> > > thought.
>
> > > I like the idea [5] of providing a Serializer class, defined similarly
> > > to the ModelAdmin class, to allow custom ways of serializing data,
> > > something like:
>
> > > # in serializers.py (or something)
> > > class ProductListing(serializers.Serializer):
> > > fields = {
> > > 'Product': ['price', 'name', 'description'],
> > > 'Subscription': ['price', 'name', 'description', 'recurrence']
> > > }
>
> > > serializers.register(ProductListing)
>
> > > $ python manage.py shell>>> from project.app import models
> > > >>> from django.core import serializers
> > > >>> print serializers.serialize('json',
> > list(models.Product.objects.all()) +
>
> > > ... list(models.Subscription.objects.all()),
> > > ... serializer='ProductListing', indent=4)
> > > [{
> > > "pk": 1,
> > > "model": "app.product",
> > > "fields": {
> > > "price": 9.98,
> > > "name": "Product #1",
> > > "description": "Description for Product #1"
> > > }},
>
> > > # other products...
> > > {
> > > "pk": 13,
> > > "model": "app.subscription",
> > > "fields": {
> > > "price": 14.98,
> > > "name": "Subscription #1, Product #13",
> > > "description: "My demonstrations aren't particularly
> > > creative.",
> > > "recurrence": 12
> > > }
>
> > > }]
>
> > > Not only is this backwards-compatible (no new serializers == no new
> > > behavior), but it also continues to allow deserialization: The
> > > deserialization process would see that app.subscription is a child of
> > > product, and fill in the data appropriately. This circumvents one of
> > > the largest drawbacks, IMO, ofDjangoFullSerializers [6]; being able to
> > > deserialize this data is often just as important as serializing it
> > > (lookin ' at you, fixtures). The observant reader will note that, as
> > > thus demonstrated, it only solves this issue for models using
> > > inheritance, not those using one-to-many or many-to-many fields. So,
> > > let's serialize some orders.
>
> > > # in serializers.py (or something)
> > > class OrderSerializer(serializers.Serializer):
> > > fields = {
> > > # include values from the products, and the return value of
> > > total_price
> > > 'order': [{'products': ['price', 'name']}, 'total_price']
> > > }
>
> > > serializers.register(OrderSerializer)
>
> > > $ python manage.py shell>>> from project.app import models
> > > >>> from django.core import serializers
> > > >>> print serializers.serialize('json', models.Order.objects.get(pk=1),
>
> > > ... serializer='OrderSerializer', indent=4)
> > > [{
> > > "pk": 1,
> > > "model": "app.order",
> > > "fields": {
> > > "products": [{
> > > "pk": 1,
> > > "model": "app.product",
> > > "fields": {
> > > "price": 9.98,
> > > "name": "Product #1"
> > > }
> > > },
> > > {
> > > "pk": 13,
> > > "model": "app.subscription",
> > > "fields": {
> > > "price": 14.98,
> > > "name": "Subscription #1, Product #13"
> > > }
> > > }],
> > > "total_price": 14.96
> > > }
>
> > > }]
>
> > > As long as enough data is presented, the deserializer stands no less
> > > chance of accurately connecting models together than in its current
> > > form. Arguments could be provided to search for matching rows and
> > > correct the primary keys from serialized data, or to update the fields
> > > in the row of given primary key.
>
> > > A couple of notes. I think that providing 'excludes' behavior could
> > > be done by prefixing a field name with a minus sign, i.e.:
>
> > > fields = {'order': ['-order_placed']}
>
> > > If the only fields listed are prefixed with a minus sign, we can
> > > assume that all remaining fields are to be included. Mixing prefixed
> > > and normal fields assumes implicit negations for any fields not
> > > mentioned:
>
> > > fields = {
> > > 'order': ['-order_placed', 'total_price']
> > > }
>
> > > serializes only the total_price pseudo-field, ignoring both
> > > order_placed, explicitly, and products, implicitly. This is just
> > > personal preference -- the decision needs to be made, and an 'exclude'
> > > member to provide explicit field exclusions is just as sensible.
>
> > > Providing a member to allow shortcut fields definition is a good
> > > idea. For example:
>
> > > class OrderSerializer(serializers.Serializer):
> > > # similar to QuerySet.select_related('product')...
> > > select_related = ['product']
> > > # or, to mirror QuerySet.select_related(depth=1)...
> > > select_related = 1
>
> > > This would use the default behavior, except the 'products' key in the
> > > output
>
> ...
>
> read more »
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
