Re: [Repoze-dev] Using colander with real world semistructured data

2010-10-23 Thread Chris McDonough
On Sat, 2010-10-23 at 14:56 -0200, Luciano Ramalho wrote:
> I've been very intrigued by Colander, and thinking of ways I could use
> it to migrate bibliographic data from legacy ISIS databases to
> CouchDB.
> 
> But here is a problem: Colander deals with missing data by generating
> nodes with a marker value, or some specified default. What we need
> when serializing bibliographic data is the option to skip the node
> altogether, and so far I have been unable to see how to specify that.
> 
> For example, if an article record does not have an "institutional
> author", we don't want this serialized value:
> 
> {"title": "Onthology is overrated", 'institutional_author' = colander.null}
> 
> and neither this:
> 
> {"title": "Onthology is overrated", 'institutional_author' = ''}
> 
> This is what we need:
> 
> {"title": "Onthology is overrated"}
> 
> >From the documentation, I haven't seen how to make colander behave as
> described. Is there a way?

The best way to do this would be to create a subclass of
colander.Mapping which:

- omits values from the appstruct created during deserialization if
  they don't exist in the input cstruct.

- omits values from the cstruct created during serialization if they
  don't exist in the input appstruct.

This is mostly a matter of subclassing colander.Mapping and overriding
its "_impl" method.  Let's say that subclass is called
"ForgivingMapping".  Once that's done, you can do:

class ForgivingSchema(colander.Schema):
schema_type = ForgivingMapping

Then when you create schemas, you can do:

class MySchema(ForgivingSchema):
attr1 = colander.SchemaNode(...)
attr2 = colander.SchemaNode(...)

(IOW, always use ForgivingSchema to represent mappings rather than
colander.Schema).

- C


___
Repoze-dev mailing list
Repoze-dev@lists.repoze.org
http://lists.repoze.org/listinfo/repoze-dev


[Repoze-dev] Using colander with real world semistructured data

2010-10-23 Thread Luciano Ramalho
I've been very intrigued by Colander, and thinking of ways I could use
it to migrate bibliographic data from legacy ISIS databases to
CouchDB.

But here is a problem: Colander deals with missing data by generating
nodes with a marker value, or some specified default. What we need
when serializing bibliographic data is the option to skip the node
altogether, and so far I have been unable to see how to specify that.

For example, if an article record does not have an "institutional
author", we don't want this serialized value:

{"title": "Onthology is overrated", 'institutional_author' = colander.null}

and neither this:

{"title": "Onthology is overrated", 'institutional_author' = ''}

This is what we need:

{"title": "Onthology is overrated"}

>From the documentation, I haven't seen how to make colander behave as
described. Is there a way?

Cheers,

Luciano

PS. Anyone interested in the problem may want to read a paper about
what I've been doing here:
http://reddes.bvsaude.org/projects/isisnbp/wiki/ISIS-DM

-- 
Luciano Ramalho
programador repentista || stand-up programmer
Twitter: @luciano
___
Repoze-dev mailing list
Repoze-dev@lists.repoze.org
http://lists.repoze.org/listinfo/repoze-dev