Hi I just stumbled across this discussion ... even though I have always been very interested about contributing to Django, I never took the step yet ! But this is actually a topic that interests me very much (even though for the moment it is Vivek's).
For several projects I am working on (a lot of APIs), I needed highly customized serialization (and sometimes on objects that are not Models at all). After a few attempts at writing directly some serialization functions (a lot of very repetitive code), I decided to take a wider view of the problem, and I wrote a generic serialization library for Python : https://bitbucket.org/sebpiq/spiteat/. There is still quite a lot of work to make the whole thing simpler, the docs are not well organized ... so I am sorry for that. And also, that might be a little overkill for django's built-in serialization framework (I even implemented some debug logging functionalities for my serializers), and it is probably too generic ... however it might give you interesting ideas, and some inspiration on this problem ! The base idea are the following (https://bytebucket.org/sebpiq/spiteat/ wiki/doc_pages/conception.html): 1. treat all serialization/deserialization operations as recursive : a serializer just divide its work, finds other serializers to delegate sub-operations to, and then combines the results. 2. serialize/deserialize to a pivot format (a python dictionary {<attr_name>: <attr_value>}), which allows to separate the heavy (de)serialization work from the conversion to/from one or another serial format. Then chain the result to another serializer for example : pivot = pivot_srz.serialize(queryset) serialized = emitter_srz.serialize(pivot) With SpitEat's system of settings which makes the serializers very easily configurable, it has proven quite powerful for me. For example say you have a model: class MyModel(models.Model): some_fk = models.ForeignKey(OtherModel) some_field = models.TextField() # ... Say you have a serializer class ModelSrz that is used by default for all model instances ... delegating the work would mean for this serializer to find another serializer for every field ... by default, for 'some_fk', ModelSrz would be used as well. But say you can configure your serializers like this (using SpitEat's syntax): #create new serializer that does really crazy stuff for my foreign key class SomeFkSrz(Srz): def serialize(self, inpt): return "A super duper transformation %s" def deserialize(self, inpt): return a_super_duper_way_of_getting_my_object(inpt) #then building my serializer for MyModel my_model_srz = ModelSrz(attr_srz_map={'some_fk': SomeFkSrz()}) So basically, treating that as a recursive operation, allows a huge flexibility while allowing to reuse a lot of code (if you provide good mechanisms to plug-in your custom serialization at any level), because you re-use ModelSrz, but you plug-in your own custom serializer for 'some_field'. By doing like that, you can address all the requirements in http://code.djangoproject.com/wiki/SummerOfCode2011 very easily, and provide a lot of flexibility to the end-user. Then, of course, this post doesn't address the problem of registering your custom serializers to Django, nor registering your custom emitters... but it probably ain't the complicated part. I would be happy to provide any help on that topic, though I stopped to be a student a few months ago :'-( Cheers, Sébastien PS : There is an attempt at writing a SpitEat serializer for django there : https://bitbucket.org/sebpiq/django-spiteat/src/312cb47ab200/serializers.py it is veeeeery ugly, quite complicated (on purpose, because I needed to handle a lot of things : MTI, GenericForeignKeys, ...), but it should work fine for manytomany, fks, etc etc ... On Feb 26, 6:54 pm, Russell Keith-Magee <[email protected]> wrote: > On Thu, Feb 24, 2011 at 10:24 PM, Vivek Narayanan <[email protected]> wrote: > > Hi, > > > I am Vivek Narayanan, an undergrad student at IIT, Varanasi in India > > and am interested in participating in this year's SoC > > Hi Vivek, and thanks for your interest in the GSoC! > > > > > Problem > > ------------ > > Django provides a serialization framework that is very useful for > > loading and saving fixtures, but not very flexible if one wants to > > provide an API for a Django application or use a serialization format > > different from what is defined by Django. Also the current handling of > > foreign keys and many to many relationships is not really useful > > outside the context of fixtures. > > > Solution > > ------------ > > I propose a class based Serializer, extending/refactoring the current > > base class which would be entirely configurable by the user. I would > > also like to provide some configurations for XML and JSON/YAML, that > > can serve as building blocks for writing other serializers. The class > > would have the following configurable options: > > > • A basic structure, markup or template representing each field an > > object and also representing the object. > > > • An external wrapping structure, a structure for an array, delimiters > > and assignment symbols. > > > • Arbitrary level of nesting depth, Which fields of related models are > > to be represented? etc. This nesting can be handled by recursion. The > > data about related models can be extracted in the start_object or > > end_object methods. > > > • Choosing which datatypes to dump as is, which ones to convert to > > something else. The user can provide a mapping between types in the > > form of a dict and conversion functions when needed. > > > • Adding metadata about each field, like data-type or content-length; > > this can be represented as additional attributes in a tag in XML or > > arrays in a key-value representation. This can be done by adding some > > class methods. > > > • The level of indentation. > > > The idea is to store them as a python list or list of dicts till the > > final 'dumping' stage. This way we can still use existing libraries > > like SimpleXML, SimpleJSON etc. This 'dumping' method would be > > overrideable. The user would have to choose between using a standard > > library and specifying a format. I would also like to add a couple of > > features: > > > • An exclude argument, which would be a list of fields to exclude from > > the model, this would also contain the fields to exclude in related > > models. I would like to extend the fields argument in the same way. > > > • An extras argument, which would allow properties and data returned > > by some methods to be serialized. > > > While this is by no means a complete proposal, I was looking for some > > feedback on the idea and would be happy to incorporate your > > suggestions. > > My feedback at this point is that you've been very verbose, but not > especially clear. You've covered a lot of ground here, which shows > you're aware of the broad problems that exist -- but you haven't > really provided enough detail for us to work out if you're on the > right track. > > Of course, this is your first post on the topic, so this may have been > intentional -- i.e., use this post as a 'taster' for the broad issues, > which you refine later. However, if you want to get accepted as a SoC > participant, we're going to need to have a very clear idea of what it > is that you're going to implement. History has shown us that students > that begin the SoC with a vague description don't end the SoC with a > completed project. > > The suggestion I have made in the past is this: As a proof of concept, > show how you would define Django's existing serialization format using > your definition language. This will be a requirement of the final > deliverable anyway, so you might as well show how it can be done. > > Once you've done that, provide a couple of other examples -- showing > both the definition, and the resulting output. The more examples you > provide of specific edge cases, the better we will be able to > understand your proposal. > > Yours, > Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
