[GSoC] Customizable Serialization check-in

[Piotr Grabowski]

Hi!

I'm Piotr Grabowski, student from University of Wroclaw, Poland
In this Google Summer of Code I will  deal with problem of customizable 
serialization in Django.


You can find my proposal here https://gist.github.com/2319638

It's obviously not a finished idea, it's need to be simplified for sure. 
My mentor Russel Keith Magee told me to look at Tom Christie's 
serialization API. I found it similar to my proposal, there is a lot in 
common - declarative fields, same approach to various aspect of 
serialization , but his API is simpler and it feels better.


Since Tom already post on group about his project I can refer to it:

W dniu 27.04.2012 06:44, Tom Christie pisze:

...

Given that Piotr's GSoC proposal has now been accepted, I'm wondering 
what the
right way forward is?  I'd like to continue to push forward with this, 
but I'm
also aware that it might be a bit of an issue if there's already an 
ongoing

GSoC project along the same lines?

Having taken a good look through the GSoC proposal, it looks good, and 
there

seems to be a fair bit of overlap, so hopefully he'll find what I've done
useful, and I'm sure I'll have plenty of comments on his project as it
progresses.

I'd consider suggesting a collaborative approach, but the rules of the 
GSoC

wouldn't allow that right?

--
Like I said above, your work will be very useful for me. I must read 
GSoC regulations carefully but for sure collaboration with code writing 
is impossible. I don't know that I could use your existing code base but 
I think it's also impossible. However sharing ideas and discuss how the 
API should look and work it will be very desirable.



My plan for next few weeks is to meet Django contribution requirements, 
solve ticket to prove I now the process off doing it, and what's most 
important  have discussion about serialization API. I hope community 
will be interested in this feature.


After weekend I will post my proposal with updates from Tom's API.

--
Piotr Grabowski


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC] Customizable Serialization Proposal

[Russell Keith-Magee]

On Wed, Mar 23, 2011 at 1:28 PM, DaNmarner  wrote:
> First of all: my native language is not English, so I apologize for
> any potential natural language error (or any error at all) below.
>
> After (briefly) reading through the current implementaion as well as
> the "Issues to consider" section in the GSoC2011 page on wiki, my
> initial observation is that the current code structure/abstraction is
> almost good enough to cater all the "Issues" on the wiki if The need
> for "Serializing non-database attributes/properties" is not considered
> for now.
...
> I feel like I'm still missing a lot to consider. I'm aware of the old
> proposals in the past and I'm planning to digest those as soon as
> possible. But I really want to hear thoughts about my idea from
> y'all. So please, tell me what's wrong in it.

It feels like that to me, too. My initial reaction here is that you
need to do a *lot* more research -- in particular looking into the
types of problems that we're trying to solve. In addition to reading
the many past discussions and tickets, there is another GSoC
serialization proposal kicking around at the moment -- that proposal
is much closer to the mark. If you want to pursue this project, I
suggest you take a look at it.

Alternatively, you've come up with a magnificent solution that nobody
has considered, but you haven't described it clearly enough that we
can understand what you're proposing.

Either way, this proposal needs a lot more work before we can provide
any serious feedback.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

[GSoC] Customizable Serialization Proposal

[DaNmarner]

First of all: my native language is not English, so I apologize for
any potential natural language error (or any error at all) below.

After (briefly) reading through the current implementaion as well as
the "Issues to consider" section in the GSoC2011 page on wiki, my
initial observation is that the current code structure/abstraction is
almost good enough to cater all the "Issues" on the wiki if The need
for "Serializing non-database attributes/properties" is not considered
for now.

It seems to me that an simple enhancement on the configuration options
of the current serializer would suffice a lot of the needs here. The
best approch I thought of so far is adding a configuration object to
the parameter. It contains infomation about what modification needs to
be done during the serialization. Does any field name needs to be
mapped to a different one? How many levels to go as far as foreign key
is concerned? What are the unnecessary fields (of course, the current
"fields" option would fit well in here, too)?  What are the relations
between two fields in the result?

This configuration object will be available to all the
subclass-serializers for different format all the time, so that
everytime a field is being serialized, the serializer will look for
infomation the configuration object recarding this field.

Further, when a fk is encountered, the configuration can contain
reference to another configuration object, which, if desired (level of
serialization not yet reached), will be used to serialize the object
fk refers to.

Naturally, the configuration object should be defined with classes (I
almost wanted to call it Meta, as in the Models).

I feel like I'm still missing a lot to consider. I'm aware of the old
proposals in the past and I'm planning to digest those as soon as
possible. But I really want to hear thoughts about my idea from
y'all. So please, tell me what's wrong in it.

PS: I have about 3 years Python experience, big Django fan, sort of a
user (wrote my own blog, of course), my ID has appeared in Django
commit msgs for about 3 times so far :).

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GSoC: Customizable Serialization

[Russell Keith-Magee]

On Mon, Mar 29, 2010 at 10:00 PM, gombiuda JHL  wrote:
> Hi all. My name is Gombiuda Jiang. I want to apply the `Customizable
> serialization` project. Here is my proposal. Thanks for your reading and
> commenting.
> As far as I can understand the project, its aim is to make serialization can
> be used out of fixture loading. For example, once we can customize our
> serialization for our models, we can use this built-in functions inside
> `django.core.serialization` instead of writing non-reusable `__unicode__`
> methods.
> With this aim, formats for serialization is necessary because of different
> requirements for different projects. I was shocked when thinking of this,
> because requirements are unlimited. But soon I got the key point: the core
> of this problem is the format customizable, fields selectable, and nested
> fields' serialize-recursion control. Everything got cleared.
> So APIs would maybe like this:
> # inside the model definition
> @serializable(
>     config = (
>                   ((Model_X, Model_Y), ('field_a', 'field_b',
> 'field_fk1_follow', 'field_fk2', 'field_m2m'), {'model_start':'{',
> 'model_end':'}', 'field_start':'name', 'field_end':'', 'separator':','}),
>                   ((Model_P, Model_Q), ('field_a', 'field_b',
> 'field_fk1_follow', 'field_fk2', 'field_m2m'), {'model_start':'[',
> 'model_end':']', 'field_start':'name', 'field_end':'', 'separator':','})
>                   # .
>                   )
> Class Model_A(models.Model):
>     # ..
> Class CustomizeSerializer(serializers.Serializer):
>     def start_serialize(self):
>         # .
>     def end_serialize(self):
>         # ..
>     def start_field(self):
>         # .
>     def end_field(self):
>         # ..
>     def serialize(self):
>         # ..
> The decorator seems ugly, but if possible, the config can be put into the
> meta options of the Model.

"Ugly" is one way to put it. Completely bass-ackwards wrong would be
another. :-)

I can see two major flaws with your suggesion of using a decorator
(or, for that matter, a meta declaration):

 - You're attempting to cram a complex serialization declaration into
the argument list for a decorator (or a single line in a Meta).
Serialization is a complex process. There are many things to declare
and define, and many things to potentially customize. This isn't
something you can declare with a simple argument list.

 - There's no reason serialization should be bound to the model class.
There isn't single clear and correct way to serialize a given model.
Django's fixture loader needs one format. If you use two AJAX
libraries, they each may need a different format. If you have some
back-office system you need to integrate with, it may have another
format. The serialization format needs to be independent of the model
definition.

> Feel free to figure out my faults. Thank you.

It's difficult to comment beyond the feedback I've already given. For
example - you've given an example of a serialization format
declaration, but you haven't given an example of how that declaration
would look when used (i.e., what serialized output would that
declaration provide?).

You also haven't given much detail on the Serializer class itself. The
"start_field/end_field" type interface you've defined as an extension
of Serializer looks like you're just overriding the existing base
class -- I'm afraid I don't see how this improves or simplifies the
process from what is already possible. I'm not saying you don't have a
good idea here - just that you haven't explained your idea in
sufficient detail for anyone to evaluate it in a meaningful way.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

GSoC: Customizable Serialization

[gombiuda JHL]

Hi all. My name is Gombiuda Jiang. And firstly I have to apologize for my
mistake of sending a wrong version of the proposal.
I want to apply the `Customizable serialization` project. Here is my
proposal. Thanks for your reading and commenting.

As far as I can understand the project, its aim is to make serialization can
be used out of fixture loading. For example, once we can customize our
serialization for our models, we can use this built-in functions inside
`django.core.serialization` instead of writing non-reusable `__unicode__`
methods.

With this aim, formats for serialization is necessary because of different
requirements for different projects. I was shocked when thinking of this,
because requirements are unlimited. But soon I got the key point: the core
of this problem is the format customizable, fields selectable, and nested
fields' serialize-recursion control. Everything got cleared.

So APIs would maybe like this:
# inside the model definition
Class Model_A(models.Model):
# ..
class Meta:
serializer_config = (
((Model_X, Model_Y), ('field_a', 'field_fk1_follow',
'field_fk2', 'field_m2m'), {'model_start':'{', 'model_end':'}',
'field_start':'name', 'field_end':'', 'separator':','}),
((Model_P, Model_Q), ('field_a', 'field_b', 'field_fk2',
'field_m2m'), {'model_start':'[', 'model_end':']', 'field_start':'name',
'field_end':'', 'separator':';'}),
# .
)

Class CustomizeSerializer(serializers.base.Serializer):
def start_serialize(self):
# .
def end_serialize(self):
# ..
def start_object(self, obj):
# .
def end_object(self, obj):
# ..
def handle_field(self, obj, field):
# ..
def handle_fk_field(self, obj, field):
# ..
def handle_m2m_field(self, obj, field):
# ..
def serialize(self, obj, parent_list):
# ..

End users can customize what they want in serialization on formatting,
fields selection, foreign key/many2many key following if they correctly
pass-in the configuration.

As an immature plan above, I need comments to improve it. Feel free to
figure out the faults. Thank you.
-- 
Sincerely,
江海龙
Gombiuda Jiang

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

GSoC: Customizable Serialization

[gombiuda JHL]

Hi all. My name is Gombiuda Jiang. I want to apply the `Customizable
serialization` project. Here is my proposal. Thanks for your reading and
commenting.

As far as I can understand the project, its aim is to make serialization can
be used out of fixture loading. For example, once we can customize our
serialization for our models, we can use this built-in functions inside
`django.core.serialization` instead of writing non-reusable `__unicode__`
methods.

With this aim, formats for serialization is necessary because of different
requirements for different projects. I was shocked when thinking of this,
because requirements are unlimited. But soon I got the key point: the core
of this problem is the format customizable, fields selectable, and nested
fields' serialize-recursion control. Everything got cleared.

So APIs would maybe like this:
# inside the model definition
@serializable(
config = (
  ((Model_X, Model_Y), ('field_a', 'field_b',
'field_fk1_follow', 'field_fk2', 'field_m2m'), {'model_start':'{',
'model_end':'}', 'field_start':'name', 'field_end':'', 'separator':','}),
  ((Model_P, Model_Q), ('field_a', 'field_b',
'field_fk1_follow', 'field_fk2', 'field_m2m'), {'model_start':'[',
'model_end':']', 'field_start':'name', 'field_end':'', 'separator':','})
  # .
  )
Class Model_A(models.Model):
# ..

Class CustomizeSerializer(serializers.Serializer):
def start_serialize(self):
# .
def end_serialize(self):
# ..
def start_field(self):
# .
def end_field(self):
# ..
def serialize(self):
# ..

The decorator seems ugly, but if possible, the config can be put into the
meta options of the Model.

Feel free to figure out my faults. Thank you.
-- 
Sincerely,
Gombiuda Jiang

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.