I appreciate you taking the time to identify my now-glaring misconceptions,
Russ (... I have to laugh. I've never met another Russ). I would like to
finish airing out my naivete before attempting to salvage my proposal before
the due date.
Would writing an appropriate template, while certainly not ideal, provide
most of the functionality for the common use case being discussed? A
template would not simplify the process, certainly, but it would provide
everything that templates do... There's more than enough logic to fit data
to a format, and the host of filters would be undeniably useful. Again,
this is certainly not ideal -- the '{' escaping for JSON alone would drive a
person insane. Are there use cases this is excluding?
I ask not because I think that's the best solution, but obviously I need a
more accurate mental image of the goal, as seen by the core developers. So
long as deserialization is no issue, as is typical in AJAX applications (or
anything where an external system is looking at an app's state), providing a
'shortcut' interface to provide structure and some form of pre-processing
hooks seems like a good way to go. For example, using the models in my
original proposal,
from project.app.models import Order
from django.core.serializers import Serializer
class OrderSerializer(Serializer):
structure = {
"order_id": "pk",
''products": [
{
"name": "products__name",
"price": "products__price",
"description": "products__description"
}
],
"total": "total_price"
}
>>> from project.app.models import Order
>>> from project.app.serializers import OrderSerializer
>>> from django.core import serializers
>>> print serializers.serialize("json", Order.objects.all()[:1],
... indent=4, serializer=OrderSerializer)
[{
"order_id": 1,
"products": [
{
"name": "Product #1",
"price": 9.98,
"description": "Description for Product #1"
},
{
"name": "Subscription #1, Product #13",
"price": 14.98,
"description": "My demonstrations aren't particularly creative."
}
],
"total": "total_price"
}]
Within the structure attribute, dictionary values are evaluated by looking
for attributes of the object, methods of the object, and methods of the
serializer.
Some "helpers" I think might be useful would be hooks for the various types
of fields, including but not limited to relations, to allow things like
special text processing or dependency traversal, and providing the current
default "structure" in case the user simply wants to do some pre-processing
of some form.
It's obviously rough, and hashing out the specifics of the exposed API will
take some research and discussion, but does this synergize better with the
general use case? Let me know, also, if I haven't quite established my take
on your take, so to speak.
Thanks for your time,
Russ
On Mon, Mar 30, 2009 at 8:09 AM, Russell Keith-Magee <freakboy3...@gmail.com
> wrote:
>
> On Mon, Mar 30, 2009 at 5:57 AM, Russ <taal...@gmail.com> wrote:
> >
> > My apologies for the length!
> >
> >
> > Concisely, I intend to provide the Django user with some granular
> > control of the data to be serialized without sacrificing backwards
> > compatibility for old code, or for users who need the straightforward,
> > current functionality.
>
> Hi Russ,
>
> First off - I would warn you (if you aren't already aware) that there
> are several other people floating around with proposals based on the
> serialization system. You would be well advised to read up on those
> other proposals, and my comments on those proposals. You will probably
> find that those comments help clarify my comments on your specific
> proposal.
>
> > Skipping merrily back to the real world, here is the issue, plain and
> > simple. Sometimes, you need to include more information than is just
> > in the one model. Sometimes, that's going to appear in the form of
> > inherited models, sometimes via one-to-many or many-to-many
> > relationships, and sometimes just little relevant bits of information
> > that may not actually be in the model at all. There are numerous open
> > tickets about this issue [1][2][3][4]. Clearly, this needs some
> > thought.
>
> Those 4 tickets don't have anything to do with the problem you describe.
>
> #4565 is a dependency discovery problem, and only applies to
> serialization, not deserialization.
>
> #7052 is the exact opposite of the problem you describe -
> deserializing when you know that data already exists in the database,
> and you don't want to recreate it. This is a particular problem for
> the auth and contenttype autogenerated types.
>
> #9422 is an edge case for serializing foreign keys.
>
> #10295 is a feature request based around changing the fundamental
> serialization format. There are a number of ways to tackle this
> problem, ranging from the trivial (just include the extra attribute)
> to the complex (provide a fully flexible rendering mechanism so end
> users can determine what attributes are included during rendering.
>
> The problem you describe is a combination of #9549. Your observations
> about inherited models adds complexity to the problem, but
> fundamentally, you're talking about clobbering of existing data during
> deserialization.
>
> > I like the idea [5] of providing a Serializer class, defined similarly
> > to the ModelAdmin class, to allow custom ways of serializing data,
> > something like:
> >
> > # in serializers.py (or something)
> > class ProductListing(serializers.Serializer):
> > fields = {
> > 'Product': ['price', 'name', 'description'],
> > 'Subscription': ['price', 'name', 'description', 'recurrence']
> > }
> >
> > serializers.register(ProductListing)
>
> As I've noted elsewhere - why do we need to register anything?
> Registration like this is only required when there is a default mode
> of internal operation that you don't have control over. However,
> serialization is controlled by a call to serializers.serialize(). You
> have direct access to the arguments to pass in a set of serialization
> instructions.
>
> > $ python manage.py shell
> >>>> from project.app import models
> >>>> from django.core import serializers
> >>>> print serializers.serialize('json', list(models.Product.objects.all())
> +
> > ... list(models.Subscription.objects.all()),
> > ... serializer='ProductListing', indent=4)
> > [{
> > "pk": 1,
> > "model": "app.product",
> > "fields": {
> > "price": 9.98,
> > "name": "Product #1",
> > "description": "Description for Product #1"
> > }
> > },
> > # other products...
> > {
> > "pk": 13,
> > "model": "app.subscription",
> > "fields": {
> > "price": 14.98,
> > "name": "Subscription #1, Product #13",
> > "description: "My demonstrations aren't particularly
> > creative.",
> > "recurrence": 12
> > }
> > }]
> >
> > Not only is this backwards-compatible (no new serializers == no new
> > behavior), but it also continues to allow deserialization: The
> > deserialization process would see that app.subscription is a child of
> > product, and fill in the data appropriately.
>
> This isn't as simple as you make out. There is a very good reason that
> the current serializers omit parent fields from the serialized fields
> list. In order to create a Subscription with PK 13, you need to create
> a Product with pk 13. Product objects.all() contains a list of all
> products... including a product with primary key #13. Unless you plan
> on implementing a filter on Product.objects.all() that excludes all
> instances that have subclasses that are also being serialized, you're
> going to have headaches.
>
> > This circumvents one of
> > the largest drawbacks, IMO, ofDjangoFullSerializers [6]; being able to
> > deserialize this data is often just as important as serializing it
> > (lookin ' at you, fixtures).
>
> Erm... what exactly what this epithet about? Django has an extensive
> test suite that demonstrates the round-trip nature of fixtures. I'm
> not for a second going to claim that there are no bugs in the existing
> serializers, but I will claim a high degree of confidence that the
> most common use cases work exactly as advertised.
>
> As for the rest of this proposal: I'm all in favour of a class-based
> declarative structure, and there is plenty of precedent for this sort
> of approach (e..g., Feeds). However, this proposal doesn't really seem
> to be exploiting the capabilities of having a class - it's really just
> a bunch of keyword arguments that could just as easily be passed in to
> the existing serialize() function. Once you strip away the new syntax,
> all you really get for your effort is a set of fairly minor
> modifications that allow you to tweak the existing base serialization
> format (as well as introducing a bunch of new serialization problems
> and ambiguities that you won't discover until you're knee deep in
> fixtures).
>
> Again, as I've noted elsewhere: The real challenge for serialization
> is to allow for completely arbitrary serialization output formats -
> for example, a list of (author name, book title) tuples. The goal here
> isn't just to allow for minor tweaks around a base structure. The goal
> is to allow for easy reconfiguration of serialized output to suit any
> external data consumer - an AJAX library, for example. It's also worth
> noting that there's no guarantee that an arbitrary serialization
> format will be deserializable. While it's important that the default
> Django serializers can maintain round trip serialization, that doesn't
> necessarily have to be true for a customized serializer.
>
> Unless I'm missing something, the modifications contained in
> DjangoFullSerializers pretty much already implement the interesting
> tweaks you have described. However, the approach contained in
> DjangoFullSerializers can't allow for a completely arbitrary
> serialization output. This is the challenge that a strong GSoC
> application will tackle.
>
> Yours,
> Russ Magee %-)
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
