On Tue, Mar 31, 2009 at 11:43 AM, Russ Amos <taal...@gmail.com> wrote:
> I appreciate you taking the time to identify my now-glaring misconceptions,
> Russ (... I have to laugh. I've never met another Russ).
Soon we will take over the world. :-)
> Would writing an appropriate template, while certainly not ideal, provide
> most of the functionality for the common use case being discussed? A
> template would not simplify the process, certainly, but it would provide
> everything that templates do... There's more than enough logic to fit data
> to a format, and the host of filters would be undeniably useful. Again,
> this is certainly not ideal -- the '{' escaping for JSON alone would drive a
> person insane. Are there use cases this is excluding?
Depends on exactly what you mean by 'template'. I would expect that
the end serialization would still occur using the underlying
JSON/XML/YAML libraries, so you can't really use a template in the
sense of a Django HTML template. However, if you're talking about a
format in which you can express serialization instructions, then I
could be convinced (but I need to see details).
> I ask not because I think that's the best solution, but obviously I need a
> more accurate mental image of the goal, as seen by the core developers. So
> long as deserialization is no issue, as is typical in AJAX applications (or
> anything where an external system is looking at an app's state), providing a
> 'shortcut' interface to provide structure and some form of pre-processing
> hooks seems like a good way to go.
To clarify - it's not that deserialization *isn't* an issue, it's that
deserialization isn't always possible. Django's default serializers
have sufficient data in them to allow deserialization. That same data
_could_ be presented in a different format, and it would certainly be
nifty if, in those cases, deserialization could be preserved. However,
I accept that this is a non-trivial goal, so if it turns out to not be
possible (or only possible under specific circumstances - such as a
serializer explicitly marked as deserializable), then I won't lose
much sleep.
> For example, using the models in my
> original proposal,
>
> from project.app.models import Order
> from django.core.serializers import Serializer
>
> class OrderSerializer(Serializer):
> structure = {
> "order_id": "pk",
> ''products": [
> {
> "name": "products__name",
> "price": "products__price",
> "description": "products__description"
> }
> ],
> "total": "total_price"
> }
Ok - I can see the approach you're aiming at here. I can see how it
could work - there are some details to sort out, but on the whole, I
like the elegance.
Some immediate concerns/questions:
* How do you deal with objects of different type? At present, you can
pass a disparate list of objects to the serializer. The only
requirement is that every element in the list is a Django object - it
doesn't need to be a homogeneous list.
* How does this translate to non-JSON serializers? The transition to
YAML shouldn't be too hard, but what about XML? How does `structure`
get interpreted by the XML serializer? How do you differentiate
between the element name, element attributes, and child nodes that can
be used in XML serialization?
> Some "helpers" I think might be useful would be hooks for the various types
> of fields, including but not limited to relations, to allow things like
> special text processing or dependency traversal, and providing the current
> default "structure" in case the user simply wants to do some pre-processing
> of some form.
I appreciate that this is one of those details that we will need to
finesse with time, but it would be interesting to hear your
preliminary thoughts on this - in particular, on how you plan to link
the string in the 'template' to the helper.
> It's obviously rough, and hashing out the specifics of the exposed API will
> take some research and discussion, but does this synergize better with the
> general use case? Let me know, also, if I haven't quite established my take
> on your take, so to speak.
This attempt is certainly a lot closer than the last. As you note -
there are still details, but it's certainly going in the right
direction. I haven't got any particularly concrete ideas on what the
final solution should look like - I just know the sort of limitations
that the existing framework has, and the sort of use cases I'd like to
be able to hit.
However, here's my brain dump, such as it is:
My initial thoughts was that the serializers would end up being a lot
like the Feeds framework - a base class with lots of
methods/attributes that can be overridden to provide specific
rendering behaviour. If you tear down the serialization problem, you
end up with a set of relatively simple questions:
* What is the top level structure (e.g.,, the outer [] in JSON, the
XML header and root tag)?
* What is the wrapping structure for each element in the list of
objects (e.g., the {} in JSON, the <object> tag in XML)
* What descriptive attributes exist for each element in the list?
(pk, model name)
* How/where are these descriptive attributes rendered? ( dict
entries? root node attributes? child nodes?)
* Which fields (including extra fields, model properties, computed
fields, etc) should be included in the list of fields?
* How is that list of fields presented to the user? (fields:{} in
JSON, child elements in XML)
* How is each field rendered? (key-value string pairs? <value>
nodes?) If the field is itself a serializable object (e.g., another
Django object) how is it serialized?
* Is there any optional metadata for each data field, such as
datatype? How is that optional metadata interpreted?
I haven't checked that this list is comprehensive, but it should give
you an idea of the sorts of questions you need to answer. Each of
these questions essentially becomes a function/interface on the
serializer. The default serializer obviously already has answers for
these questions; a custom serializer needs an interface for providing
alternate answers.
The arguments and return values for these functions is a bit of an
open question - I haven't dug into enough details to be able to give
much immediate guidance here. The `template` approach you've proposed
here is certainly one way to approach this problem, and could even
allow you to combine several of these questions into one - but we need
to make sure we avoid coming up with a whole new language that needs
to be parsed and interpreted.
I was also thinking that you aren't necessarily going to be
subclassing the serializer itself. The answers to these questions are
really just rendering instructions that can be followed by any
serializer, once some common ground rules are established. The
existing serialization engin has a hard-coded set of answers; what we
need to do is refactor those answers out into a default definition
that can be subclassed, overridden, or rewritten to suit specific
needs.
Some of the serialization instructions will be ignored by some
renderers: for example, a 'child-name=value' attribute may be used to
describe the fact that <value> tags are required for XML, but be
ignored by the JSON and YAML serializer. Obviously, an important task
here is to define what attributes are required, which are optional,
and how they map onto each serializer.
Anyway, there's 10c worth of brain dump - make of it what you will. I
make no claim that these ideas are watertight - I'm willing to listen
to any reasonable counterideas or objections to what I have proposed.
Yours,
Russ Magee %-)
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
