Re: Support POST of application/json content type

2014-05-05 Thread Dmitry Mugtasimov
It looks like there is a lot of work to do for full featured content-type 
support. Meanwhile if you need just to integrate Django + Django REST 
Framework and django-oauth-plus for PUT-parameters support, please, take a 
look at this thread 
https://groups.google.com/forum/#!topic/django-rest-framework/679fOg0QpzI 
and this post 
https://groups.google.com/d/msg/django-rest-framework/679fOg0QpzI/2qeQGdzq-FsJ 
in particular.

понедельник, 18 ноября 2013 г., 17:42:31 UTC+4 пользователь Tom Christie 
написал:
>
> I've created the ticket 21442to 
> track 'Configurable request parsing'.
>
> I've also made a first pass at refactoring the current request parsing 
> into a generic interface that can support arbitrary parsers,
> and pushed the current status to the ticket_21442 
> branchon my GitHub 
> account.
>
> It's likely that the API will need a few design decisions along the way, 
> so I'd suggest anyone who's interested in contributing to this work makes 
> sure to follow the ongoing work in that branch.
>
> Cheers all,
>
>   Tom
>
> On Thursday, 12 September 2013 11:20:07 UTC+1, Tom Christie wrote:
>>
>> > why keep data and files separated
>>
>> Mostly because that's the way it already works, so...
>>
>> * request.data would essentially provide a strict superset of the 
>> functionality that request.POST provides.  In general you'd be able to 
>> replace `request.POST` with `request.data` anywhere and seemlessly start 
>> supporting JSON or other data without any other changes required.
>> * Form expect the data and files to be provided separately which would be 
>> awkward otherwise.
>>
>> > In the absence of strong objection, I will start working on this base. 
>>
>> Sure thing.  As it happens, I was also considering taking a crack at this 
>> in the coming weeks, so please do follow up on this thread linking to your 
>> repo if you start working on it, so myself and others can track any 
>> progress.  (And perhaps collaborate.)
>>
>> Cheers :)
>>
>>   Tom
>>
>> On Wednesday, 11 September 2013 04:52:08 UTC+1, Stefan Berder wrote:
>>>
>>> On Tue, Sep 10, 2013 at 12:17 PM, S Berder  wrote: 
>>> > On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney 
>>> >  wrote: 
>>> >> 
>>> >> On 9 September 2013 19:50, S Berder  wrote: 
>>> >>> 
>>> >>> Gents, 
>>> >>> to sum it up, arguments made and details of how I see the 
>>> >>> implementation of a response/request encode/decode framework: 
>>> >>> 
>>> >>> * need a pluggable interface so current content-types are supported 
>>> >>> (`application/x-www-form-urlencoded`, `multipart/form-data`), new 
>>> >>> types (`application/json`), custom and future types 
>>> >>> (`application/vnd.foobar+json` anybody? See 
>>> >>> 
>>> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future 
>>> >>> for example, `application/msgpack`, `application/protobuf`, 
>>> >>> `application/capnproto`, etc). 
>>> >>> * decoder/encoder map (content-type, decoder) should be smart to 
>>> >>> handle patterns like `text/*` or `application/*xml*` and match 
>>> things 
>>> >>> like `Accept: application/json, text/plain, * / *` 
>>> >>> * choice of decoder would be made on the Content-Type header, maybe 
>>> >>> supporting a raw by default so data is just passed in case of 
>>> unknown 
>>> >>> content type. 
>>> >>> * decoder/encoder should be available through `request` and 
>>> `response` 
>>> >>> objects. 
>>> >>> * decoded data structure (python object) would be stored in 
>>> `request.data` 
>>> >>> * first step is to support requests, next step is to handle 
>>> responses 
>>> >>> with the same pluggable functionality and coherent API. 
>>> >>> * A sensible default for response Content-type would be `text/html; 
>>> >>> charset=UTF-8`. It should be made available through a setting entry 
>>> >>> anyway 
>>> >>> 
>>> >> 
>>> >> You should also have access to the decision made by the data parser 
>>> as to 
>>> >> which parser was used, instead of having to infer it yourself from 
>>> the 
>>> >> content type header. 
>>> > 
>>> > Indeed, that's the 4th point of my list, maybe it's not clear as it is 
>>> > but this would be supported. 
>>> > 
>>> >>> 
>>> >>> Some questions though: 
>>> >>> 
>>> >>> * why keep data and files separated, I see no good reason for this 
>>> >>> except mimicking PHP's structure. An uploaded file comes from a 
>>> named 
>>> >>> input, I hope to find it in request.data (why do a common structure 
>>> >>> otherwise). I might be missing something but nothing indicates a 
>>> real 
>>> >>> need for this in django/http/request.py 
>>> >> 
>>> >> 
>>> >> True, there's some added complexity [small as it is] in forms because 
>>> File 
>>> >> fields need to look elsewhere for their values. 
>>> >> 
>>> >>> 
>>> >>> * isn't more or less any data sent to your backend representable as 
>>> a 
>>> 

Re: Support POST of application/json content type

2013-11-18 Thread Tom Christie
I've created the ticket 21442 to 
track 'Configurable request parsing'.

I've also made a first pass at refactoring the current request parsing into 
a generic interface that can support arbitrary parsers,
and pushed the current status to the ticket_21442 
branchon my GitHub 
account.

It's likely that the API will need a few design decisions along the way, so 
I'd suggest anyone who's interested in contributing to this work makes sure 
to follow the ongoing work in that branch.

Cheers all,

  Tom

On Thursday, 12 September 2013 11:20:07 UTC+1, Tom Christie wrote:
>
> > why keep data and files separated
>
> Mostly because that's the way it already works, so...
>
> * request.data would essentially provide a strict superset of the 
> functionality that request.POST provides.  In general you'd be able to 
> replace `request.POST` with `request.data` anywhere and seemlessly start 
> supporting JSON or other data without any other changes required.
> * Form expect the data and files to be provided separately which would be 
> awkward otherwise.
>
> > In the absence of strong objection, I will start working on this base. 
>
> Sure thing.  As it happens, I was also considering taking a crack at this 
> in the coming weeks, so please do follow up on this thread linking to your 
> repo if you start working on it, so myself and others can track any 
> progress.  (And perhaps collaborate.)
>
> Cheers :)
>
>   Tom
>
> On Wednesday, 11 September 2013 04:52:08 UTC+1, Stefan Berder wrote:
>>
>> On Tue, Sep 10, 2013 at 12:17 PM, S Berder  wrote: 
>> > On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney 
>> >  wrote: 
>> >> 
>> >> On 9 September 2013 19:50, S Berder  wrote: 
>> >>> 
>> >>> Gents, 
>> >>> to sum it up, arguments made and details of how I see the 
>> >>> implementation of a response/request encode/decode framework: 
>> >>> 
>> >>> * need a pluggable interface so current content-types are supported 
>> >>> (`application/x-www-form-urlencoded`, `multipart/form-data`), new 
>> >>> types (`application/json`), custom and future types 
>> >>> (`application/vnd.foobar+json` anybody? See 
>> >>> 
>> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future 
>> >>> for example, `application/msgpack`, `application/protobuf`, 
>> >>> `application/capnproto`, etc). 
>> >>> * decoder/encoder map (content-type, decoder) should be smart to 
>> >>> handle patterns like `text/*` or `application/*xml*` and match things 
>> >>> like `Accept: application/json, text/plain, * / *` 
>> >>> * choice of decoder would be made on the Content-Type header, maybe 
>> >>> supporting a raw by default so data is just passed in case of unknown 
>> >>> content type. 
>> >>> * decoder/encoder should be available through `request` and 
>> `response` 
>> >>> objects. 
>> >>> * decoded data structure (python object) would be stored in 
>> `request.data` 
>> >>> * first step is to support requests, next step is to handle responses 
>> >>> with the same pluggable functionality and coherent API. 
>> >>> * A sensible default for response Content-type would be `text/html; 
>> >>> charset=UTF-8`. It should be made available through a setting entry 
>> >>> anyway 
>> >>> 
>> >> 
>> >> You should also have access to the decision made by the data parser as 
>> to 
>> >> which parser was used, instead of having to infer it yourself from the 
>> >> content type header. 
>> > 
>> > Indeed, that's the 4th point of my list, maybe it's not clear as it is 
>> > but this would be supported. 
>> > 
>> >>> 
>> >>> Some questions though: 
>> >>> 
>> >>> * why keep data and files separated, I see no good reason for this 
>> >>> except mimicking PHP's structure. An uploaded file comes from a named 
>> >>> input, I hope to find it in request.data (why do a common structure 
>> >>> otherwise). I might be missing something but nothing indicates a real 
>> >>> need for this in django/http/request.py 
>> >> 
>> >> 
>> >> True, there's some added complexity [small as it is] in forms because 
>> File 
>> >> fields need to look elsewhere for their values. 
>> >> 
>> >>> 
>> >>> * isn't more or less any data sent to your backend representable as a 
>> >>> dict or object with dict access modes? I try to think about 
>> >>> occurrences where some data would not have a 'name'. 
>> >>> 
>> >> 
>> >> I frequently send JSON lists of data to my APIs... 
>> > Ok, was a bit short sighted on this one, still thinking in terms of 
>> > form bound data, it was a long day here in Shanghai. I suppose that 
>> > the kind of python object you receive is not so important as you 
>> > should do data validation anyway. Your earlier concern about checking 
>> > for different content-types doesn't apply to the solution I have in 
>> > mind as to whatever data representation you have at the beginning, you 
>> > should get a 

Re: Support POST of application/json content type

2013-10-09 Thread abh2134
Hi all,

I would love to be involved with this feature. 

My suggestion is to do the following:

   - Check requst.is_ajax()
   - Check request.META.get('CONTENT_TYPE').count('application/json')
   - Parse request.body using django.utils.simplejson.loads
   - ... and set *request.JSON* to the result 

I have a small piece of middleware that implements this procedure and have 
posted it as a gist, https://gist.github.com/abhillman/6910689.

If there is interest in using my code, I would be very happy to write some 
unit tests.

abh

On Thursday, September 12, 2013 3:46:53 AM UTC-7, Stefan Berder wrote:
>
> Tom,
> I get the context and that form is tied to that behavior but outside of 
> "that's how it is now" is there any technical reason for this? I can't read 
> forms code at the moment but will do tonight and will look at how FILES is 
> used in there. I'm not usually afraid by impacts of it's the "right thing 
> to do". I think it would make for a stronger interface of you could simply 
> find your data in request.data.
>
> I will create a branch in my fork on github and will send it here for 
> progress.
>
> Stefan, from my mobile
> -- 
> http://www.bonz.org/
> /(bb|[^b]{2}/
> On 12 Sep 2013 18:20, "Tom Christie"  
> wrote:
>
>> > why keep data and files separated
>>
>> Mostly because that's the way it already works, so...
>>
>> * request.data would essentially provide a strict superset of the 
>> functionality that request.POST provides.  In general you'd be able to 
>> replace `request.POST` with `request.data` anywhere and seemlessly start 
>> supporting JSON or other data without any other changes required.
>> * Form expect the data and files to be provided separately which would be 
>> awkward otherwise.
>>
>> > In the absence of strong objection, I will start working on this base. 
>>
>> Sure thing.  As it happens, I was also considering taking a crack at this 
>> in the coming weeks, so please do follow up on this thread linking to your 
>> repo if you start working on it, so myself and others can track any 
>> progress.  (And perhaps collaborate.)
>>
>> Cheers :)
>>
>>   Tom
>>
>> On Wednesday, 11 September 2013 04:52:08 UTC+1, Stefan Berder wrote:
>>>
>>> On Tue, Sep 10, 2013 at 12:17 PM, S Berder  wrote: 
>>> > On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney 
>>> >  wrote: 
>>> >> 
>>> >> On 9 September 2013 19:50, S Berder  wrote: 
>>> >>> 
>>> >>> Gents, 
>>> >>> to sum it up, arguments made and details of how I see the 
>>> >>> implementation of a response/request encode/decode framework: 
>>> >>> 
>>> >>> * need a pluggable interface so current content-types are supported 
>>> >>> (`application/x-www-form-**urlencoded`, `multipart/form-data`), new 
>>> >>> types (`application/json`), custom and future types 
>>> >>> (`application/vnd.foobar+json` anybody? See 
>>> >>> http://developer.github.com/**v3/media/#api-v3-media-type-**
>>> and-the-future
>>>  
>>> >>> for example, `application/msgpack`, `application/protobuf`, 
>>> >>> `application/capnproto`, etc). 
>>> >>> * decoder/encoder map (content-type, decoder) should be smart to 
>>> >>> handle patterns like `text/*` or `application/*xml*` and match 
>>> things 
>>> >>> like `Accept: application/json, text/plain, * / *` 
>>> >>> * choice of decoder would be made on the Content-Type header, maybe 
>>> >>> supporting a raw by default so data is just passed in case of 
>>> unknown 
>>> >>> content type. 
>>> >>> * decoder/encoder should be available through `request` and 
>>> `response` 
>>> >>> objects. 
>>> >>> * decoded data structure (python object) would be stored in 
>>> `request.data` 
>>> >>> * first step is to support requests, next step is to handle 
>>> responses 
>>> >>> with the same pluggable functionality and coherent API. 
>>> >>> * A sensible default for response Content-type would be `text/html; 
>>> >>> charset=UTF-8`. It should be made available through a setting entry 
>>> >>> anyway 
>>> >>> 
>>> >> 
>>> >> You should also have access to the decision made by the data parser 
>>> as to 
>>> >> which parser was used, instead of having to infer it yourself from 
>>> the 
>>> >> content type header. 
>>> > 
>>> > Indeed, that's the 4th point of my list, maybe it's not clear as it is 
>>> > but this would be supported. 
>>> > 
>>> >>> 
>>> >>> Some questions though: 
>>> >>> 
>>> >>> * why keep data and files separated, I see no good reason for this 
>>> >>> except mimicking PHP's structure. An uploaded file comes from a 
>>> named 
>>> >>> input, I hope to find it in request.data (why do a common structure 
>>> >>> otherwise). I might be missing something but nothing indicates a 
>>> real 
>>> >>> need for this in django/http/request.py 
>>> >> 
>>> >> 
>>> >> True, there's some added complexity [small as it is] in forms because 
>>> File 
>>> >> fields need to look 

Re: Support POST of application/json content type

2013-09-12 Thread S Berder
Tom,
I get the context and that form is tied to that behavior but outside of
"that's how it is now" is there any technical reason for this? I can't read
forms code at the moment but will do tonight and will look at how FILES is
used in there. I'm not usually afraid by impacts of it's the "right thing
to do". I think it would make for a stronger interface of you could simply
find your data in request.data.

I will create a branch in my fork on github and will send it here for
progress.

Stefan, from my mobile
-- 
http://www.bonz.org/
/(bb|[^b]{2}/
On 12 Sep 2013 18:20, "Tom Christie"  wrote:

> > why keep data and files separated
>
> Mostly because that's the way it already works, so...
>
> * request.data would essentially provide a strict superset of the
> functionality that request.POST provides.  In general you'd be able to
> replace `request.POST` with `request.data` anywhere and seemlessly start
> supporting JSON or other data without any other changes required.
> * Form expect the data and files to be provided separately which would be
> awkward otherwise.
>
> > In the absence of strong objection, I will start working on this base.
>
> Sure thing.  As it happens, I was also considering taking a crack at this
> in the coming weeks, so please do follow up on this thread linking to your
> repo if you start working on it, so myself and others can track any
> progress.  (And perhaps collaborate.)
>
> Cheers :)
>
>   Tom
>
> On Wednesday, 11 September 2013 04:52:08 UTC+1, Stefan Berder wrote:
>>
>> On Tue, Sep 10, 2013 at 12:17 PM, S Berder  wrote:
>> > On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney
>> >  wrote:
>> >>
>> >> On 9 September 2013 19:50, S Berder  wrote:
>> >>>
>> >>> Gents,
>> >>> to sum it up, arguments made and details of how I see the
>> >>> implementation of a response/request encode/decode framework:
>> >>>
>> >>> * need a pluggable interface so current content-types are supported
>> >>> (`application/x-www-form-**urlencoded`, `multipart/form-data`), new
>> >>> types (`application/json`), custom and future types
>> >>> (`application/vnd.foobar+json` anybody? See
>> >>> http://developer.github.com/**v3/media/#api-v3-media-type-**
>> and-the-future
>> >>> for example, `application/msgpack`, `application/protobuf`,
>> >>> `application/capnproto`, etc).
>> >>> * decoder/encoder map (content-type, decoder) should be smart to
>> >>> handle patterns like `text/*` or `application/*xml*` and match things
>> >>> like `Accept: application/json, text/plain, * / *`
>> >>> * choice of decoder would be made on the Content-Type header, maybe
>> >>> supporting a raw by default so data is just passed in case of unknown
>> >>> content type.
>> >>> * decoder/encoder should be available through `request` and
>> `response`
>> >>> objects.
>> >>> * decoded data structure (python object) would be stored in
>> `request.data`
>> >>> * first step is to support requests, next step is to handle responses
>> >>> with the same pluggable functionality and coherent API.
>> >>> * A sensible default for response Content-type would be `text/html;
>> >>> charset=UTF-8`. It should be made available through a setting entry
>> >>> anyway
>> >>>
>> >>
>> >> You should also have access to the decision made by the data parser as
>> to
>> >> which parser was used, instead of having to infer it yourself from the
>> >> content type header.
>> >
>> > Indeed, that's the 4th point of my list, maybe it's not clear as it is
>> > but this would be supported.
>> >
>> >>>
>> >>> Some questions though:
>> >>>
>> >>> * why keep data and files separated, I see no good reason for this
>> >>> except mimicking PHP's structure. An uploaded file comes from a named
>> >>> input, I hope to find it in request.data (why do a common structure
>> >>> otherwise). I might be missing something but nothing indicates a real
>> >>> need for this in django/http/request.py
>> >>
>> >>
>> >> True, there's some added complexity [small as it is] in forms because
>> File
>> >> fields need to look elsewhere for their values.
>> >>
>> >>>
>> >>> * isn't more or less any data sent to your backend representable as a
>> >>> dict or object with dict access modes? I try to think about
>> >>> occurrences where some data would not have a 'name'.
>> >>>
>> >>
>> >> I frequently send JSON lists of data to my APIs...
>> > Ok, was a bit short sighted on this one, still thinking in terms of
>> > form bound data, it was a long day here in Shanghai. I suppose that
>> > the kind of python object you receive is not so important as you
>> > should do data validation anyway. Your earlier concern about checking
>> > for different content-types doesn't apply to the solution I have in
>> > mind as to whatever data representation you have at the beginning, you
>> > should get a very similar object after decoding. What I mean is if 

Re: Support POST of application/json content type

2013-09-12 Thread Tom Christie
> why keep data and files separated

Mostly because that's the way it already works, so...

* request.data would essentially provide a strict superset of the 
functionality that request.POST provides.  In general you'd be able to 
replace `request.POST` with `request.data` anywhere and seemlessly start 
supporting JSON or other data without any other changes required.
* Form expect the data and files to be provided separately which would be 
awkward otherwise.

> In the absence of strong objection, I will start working on this base. 

Sure thing.  As it happens, I was also considering taking a crack at this 
in the coming weeks, so please do follow up on this thread linking to your 
repo if you start working on it, so myself and others can track any 
progress.  (And perhaps collaborate.)

Cheers :)

  Tom

On Wednesday, 11 September 2013 04:52:08 UTC+1, Stefan Berder wrote:
>
> On Tue, Sep 10, 2013 at 12:17 PM, S Berder  
> wrote: 
> > On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney 
> >  wrote: 
> >> 
> >> On 9 September 2013 19:50, S Berder  
> wrote: 
> >>> 
> >>> Gents, 
> >>> to sum it up, arguments made and details of how I see the 
> >>> implementation of a response/request encode/decode framework: 
> >>> 
> >>> * need a pluggable interface so current content-types are supported 
> >>> (`application/x-www-form-urlencoded`, `multipart/form-data`), new 
> >>> types (`application/json`), custom and future types 
> >>> (`application/vnd.foobar+json` anybody? See 
> >>> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future 
> >>> for example, `application/msgpack`, `application/protobuf`, 
> >>> `application/capnproto`, etc). 
> >>> * decoder/encoder map (content-type, decoder) should be smart to 
> >>> handle patterns like `text/*` or `application/*xml*` and match things 
> >>> like `Accept: application/json, text/plain, * / *` 
> >>> * choice of decoder would be made on the Content-Type header, maybe 
> >>> supporting a raw by default so data is just passed in case of unknown 
> >>> content type. 
> >>> * decoder/encoder should be available through `request` and `response` 
> >>> objects. 
> >>> * decoded data structure (python object) would be stored in 
> `request.data` 
> >>> * first step is to support requests, next step is to handle responses 
> >>> with the same pluggable functionality and coherent API. 
> >>> * A sensible default for response Content-type would be `text/html; 
> >>> charset=UTF-8`. It should be made available through a setting entry 
> >>> anyway 
> >>> 
> >> 
> >> You should also have access to the decision made by the data parser as 
> to 
> >> which parser was used, instead of having to infer it yourself from the 
> >> content type header. 
> > 
> > Indeed, that's the 4th point of my list, maybe it's not clear as it is 
> > but this would be supported. 
> > 
> >>> 
> >>> Some questions though: 
> >>> 
> >>> * why keep data and files separated, I see no good reason for this 
> >>> except mimicking PHP's structure. An uploaded file comes from a named 
> >>> input, I hope to find it in request.data (why do a common structure 
> >>> otherwise). I might be missing something but nothing indicates a real 
> >>> need for this in django/http/request.py 
> >> 
> >> 
> >> True, there's some added complexity [small as it is] in forms because 
> File 
> >> fields need to look elsewhere for their values. 
> >> 
> >>> 
> >>> * isn't more or less any data sent to your backend representable as a 
> >>> dict or object with dict access modes? I try to think about 
> >>> occurrences where some data would not have a 'name'. 
> >>> 
> >> 
> >> I frequently send JSON lists of data to my APIs... 
> > Ok, was a bit short sighted on this one, still thinking in terms of 
> > form bound data, it was a long day here in Shanghai. I suppose that 
> > the kind of python object you receive is not so important as you 
> > should do data validation anyway. Your earlier concern about checking 
> > for different content-types doesn't apply to the solution I have in 
> > mind as to whatever data representation you have at the beginning, you 
> > should get a very similar object after decoding. What I mean is if you 
> > send the *same* data through Yaml or JSON, the object in request.data 
> > should be the same or extremely close. I say extremely close because 
> > I'm thinking about xml that is always way more verbose than the others 
> > and *might* add more data to the resulting object. (hint: I don't like 
> > XML, don't need it in what I do and last used it ~8/9 years ago in a 
> > disastrous explosion of SOAP and unix/microsoft interfaces) 
> > 
> > Stefan 
> > -- 
> > http://www.bonz.org/ 
> >  /(bb|[^b]{2})/ 
>
> In the absence of strong objection, I will start working on this base. 
>
> Stefan 
> -- 
> http://www.bonz.org/ 
>  /(bb|[^b]{2})/ 
>

-- 
You received this message because you are subscribed to the Google 

Re: Support POST of application/json content type

2013-09-10 Thread S Berder
On Tue, Sep 10, 2013 at 12:17 PM, S Berder  wrote:
> On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney
>  wrote:
>>
>> On 9 September 2013 19:50, S Berder  wrote:
>>>
>>> Gents,
>>> to sum it up, arguments made and details of how I see the
>>> implementation of a response/request encode/decode framework:
>>>
>>> * need a pluggable interface so current content-types are supported
>>> (`application/x-www-form-urlencoded`, `multipart/form-data`), new
>>> types (`application/json`), custom and future types
>>> (`application/vnd.foobar+json` anybody? See
>>> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future
>>> for example, `application/msgpack`, `application/protobuf`,
>>> `application/capnproto`, etc).
>>> * decoder/encoder map (content-type, decoder) should be smart to
>>> handle patterns like `text/*` or `application/*xml*` and match things
>>> like `Accept: application/json, text/plain, * / *`
>>> * choice of decoder would be made on the Content-Type header, maybe
>>> supporting a raw by default so data is just passed in case of unknown
>>> content type.
>>> * decoder/encoder should be available through `request` and `response`
>>> objects.
>>> * decoded data structure (python object) would be stored in `request.data`
>>> * first step is to support requests, next step is to handle responses
>>> with the same pluggable functionality and coherent API.
>>> * A sensible default for response Content-type would be `text/html;
>>> charset=UTF-8`. It should be made available through a setting entry
>>> anyway
>>>
>>
>> You should also have access to the decision made by the data parser as to
>> which parser was used, instead of having to infer it yourself from the
>> content type header.
>
> Indeed, that's the 4th point of my list, maybe it's not clear as it is
> but this would be supported.
>
>>>
>>> Some questions though:
>>>
>>> * why keep data and files separated, I see no good reason for this
>>> except mimicking PHP's structure. An uploaded file comes from a named
>>> input, I hope to find it in request.data (why do a common structure
>>> otherwise). I might be missing something but nothing indicates a real
>>> need for this in django/http/request.py
>>
>>
>> True, there's some added complexity [small as it is] in forms because File
>> fields need to look elsewhere for their values.
>>
>>>
>>> * isn't more or less any data sent to your backend representable as a
>>> dict or object with dict access modes? I try to think about
>>> occurrences where some data would not have a 'name'.
>>>
>>
>> I frequently send JSON lists of data to my APIs...
> Ok, was a bit short sighted on this one, still thinking in terms of
> form bound data, it was a long day here in Shanghai. I suppose that
> the kind of python object you receive is not so important as you
> should do data validation anyway. Your earlier concern about checking
> for different content-types doesn't apply to the solution I have in
> mind as to whatever data representation you have at the beginning, you
> should get a very similar object after decoding. What I mean is if you
> send the *same* data through Yaml or JSON, the object in request.data
> should be the same or extremely close. I say extremely close because
> I'm thinking about xml that is always way more verbose than the others
> and *might* add more data to the resulting object. (hint: I don't like
> XML, don't need it in what I do and last used it ~8/9 years ago in a
> disastrous explosion of SOAP and unix/microsoft interfaces)
>
> Stefan
> --
> http://www.bonz.org/
>  /(bb|[^b]{2})/

In the absence of strong objection, I will start working on this base.

Stefan
-- 
http://www.bonz.org/
 /(bb|[^b]{2})/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-09 Thread S Berder
On Tue, Sep 10, 2013 at 9:05 AM, Curtis Maloney
 wrote:
>
> On 9 September 2013 19:50, S Berder  wrote:
>>
>> Gents,
>> to sum it up, arguments made and details of how I see the
>> implementation of a response/request encode/decode framework:
>>
>> * need a pluggable interface so current content-types are supported
>> (`application/x-www-form-urlencoded`, `multipart/form-data`), new
>> types (`application/json`), custom and future types
>> (`application/vnd.foobar+json` anybody? See
>> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future
>> for example, `application/msgpack`, `application/protobuf`,
>> `application/capnproto`, etc).
>> * decoder/encoder map (content-type, decoder) should be smart to
>> handle patterns like `text/*` or `application/*xml*` and match things
>> like `Accept: application/json, text/plain, * / *`
>> * choice of decoder would be made on the Content-Type header, maybe
>> supporting a raw by default so data is just passed in case of unknown
>> content type.
>> * decoder/encoder should be available through `request` and `response`
>> objects.
>> * decoded data structure (python object) would be stored in `request.data`
>> * first step is to support requests, next step is to handle responses
>> with the same pluggable functionality and coherent API.
>> * A sensible default for response Content-type would be `text/html;
>> charset=UTF-8`. It should be made available through a setting entry
>> anyway
>>
>
> You should also have access to the decision made by the data parser as to
> which parser was used, instead of having to infer it yourself from the
> content type header.

Indeed, that's the 4th point of my list, maybe it's not clear as it is
but this would be supported.

>>
>> Some questions though:
>>
>> * why keep data and files separated, I see no good reason for this
>> except mimicking PHP's structure. An uploaded file comes from a named
>> input, I hope to find it in request.data (why do a common structure
>> otherwise). I might be missing something but nothing indicates a real
>> need for this in django/http/request.py
>
>
> True, there's some added complexity [small as it is] in forms because File
> fields need to look elsewhere for their values.
>
>>
>> * isn't more or less any data sent to your backend representable as a
>> dict or object with dict access modes? I try to think about
>> occurrences where some data would not have a 'name'.
>>
>
> I frequently send JSON lists of data to my APIs...
Ok, was a bit short sighted on this one, still thinking in terms of
form bound data, it was a long day here in Shanghai. I suppose that
the kind of python object you receive is not so important as you
should do data validation anyway. Your earlier concern about checking
for different content-types doesn't apply to the solution I have in
mind as to whatever data representation you have at the beginning, you
should get a very similar object after decoding. What I mean is if you
send the *same* data through Yaml or JSON, the object in request.data
should be the same or extremely close. I say extremely close because
I'm thinking about xml that is always way more verbose than the others
and *might* add more data to the resulting object. (hint: I don't like
XML, don't need it in what I do and last used it ~8/9 years ago in a
disastrous explosion of SOAP and unix/microsoft interfaces)

Stefan
-- 
http://www.bonz.org/
 /(bb|[^b]{2})/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-09 Thread Curtis Maloney
On 9 September 2013 19:50, S Berder  wrote:

> Gents,
> to sum it up, arguments made and details of how I see the
> implementation of a response/request encode/decode framework:
>
> * need a pluggable interface so current content-types are supported
> (`application/x-www-form-urlencoded`, `multipart/form-data`), new
> types (`application/json`), custom and future types
> (`application/vnd.foobar+json` anybody? See
> http://developer.github.com/v3/media/#api-v3-media-type-and-the-future
> for example, `application/msgpack`, `application/protobuf`,
> `application/capnproto`, etc).
> * decoder/encoder map (content-type, decoder) should be smart to
> handle patterns like `text/*` or `application/*xml*` and match things
> like `Accept: application/json, text/plain, * / *`
> * choice of decoder would be made on the Content-Type header, maybe
> supporting a raw by default so data is just passed in case of unknown
> content type.
> * decoder/encoder should be available through `request` and `response`
> objects.
> * decoded data structure (python object) would be stored in `request.data`
> * first step is to support requests, next step is to handle responses
> with the same pluggable functionality and coherent API.
> * A sensible default for response Content-type would be `text/html;
> charset=UTF-8`. It should be made available through a setting entry
> anyway
>
>
You should also have access to the decision made by the data parser as to
which parser was used, instead of having to infer it yourself from the
content type header.



> Some questions though:
>
> * why keep data and files separated, I see no good reason for this
> except mimicking PHP's structure. An uploaded file comes from a named
> input, I hope to find it in request.data (why do a common structure
> otherwise). I might be missing something but nothing indicates a real
> need for this in django/http/request.py
>

True, there's some added complexity [small as it is] in forms because File
fields need to look elsewhere for their values.


> * isn't more or less any data sent to your backend representable as a
> dict or object with dict access modes? I try to think about
> occurrences where some data would not have a 'name'.
>
>
I frequently send JSON lists of data to my APIs...

--
Curtis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-09 Thread S Berder
Gents,
to sum it up, arguments made and details of how I see the
implementation of a response/request encode/decode framework:

* need a pluggable interface so current content-types are supported
(`application/x-www-form-urlencoded`, `multipart/form-data`), new
types (`application/json`), custom and future types
(`application/vnd.foobar+json` anybody? See
http://developer.github.com/v3/media/#api-v3-media-type-and-the-future
for example, `application/msgpack`, `application/protobuf`,
`application/capnproto`, etc).
* decoder/encoder map (content-type, decoder) should be smart to
handle patterns like `text/*` or `application/*xml*` and match things
like `Accept: application/json, text/plain, * / *`
* choice of decoder would be made on the Content-Type header, maybe
supporting a raw by default so data is just passed in case of unknown
content type.
* decoder/encoder should be available through `request` and `response` objects.
* decoded data structure (python object) would be stored in `request.data`
* first step is to support requests, next step is to handle responses
with the same pluggable functionality and coherent API.
* A sensible default for response Content-type would be `text/html;
charset=UTF-8`. It should be made available through a setting entry
anyway

Some questions though:

* why keep data and files separated, I see no good reason for this
except mimicking PHP's structure. An uploaded file comes from a named
input, I hope to find it in request.data (why do a common structure
otherwise). I might be missing something but nothing indicates a real
need for this in django/http/request.py
* isn't more or less any data sent to your backend representable as a
dict or object with dict access modes? I try to think about
occurrences where some data would not have a 'name'.

I'm sure I left some obvious things out like more settings and others
but this will become obvious while coding.

If no strong objection, I'm ready to start in that direction and
provide code sometime next week.

Stefan

On Thu, Sep 5, 2013 at 7:31 AM, Curtis Maloney
 wrote:
> To weight in from an author of a different API tool
>
> In django-nap there's a simple "get_request_data" method which, essentially,
> determines how to parse the request according to the content type.  However,
> currently each API only supports a single serialiser format [and HTTP Form
> encoding] so there's little guessing involved.
>
> However, I wouldn't want to see the same on Request, unless there was also a
> direct way to imply the type you want.  For instance, if I get a request
> that's in XML, and I ask for request.JSON, I'd like it to either yield empty
> or raise an error.  Whereas when I didn't case, accessing request.DATA would
> make a best guess.
>
> So I see two paths... either you use a "Decode it for me according to
> content-type" interface, or a "Treat is as this type, and fail predictably
> if it's not" one.
>
> The current GET/POST is, of course, the latter.  And there's no reason we
> can't have both :)
>
> --
> Curtis
>
>
>
> On 5 September 2013 04:06, Jonathan Slenders 
> wrote:
>>
>> Would that mean that the object returned by request.DATA/POST/whatever
>> could be a different type, depending on what the user posted?
>>
>> I don't want to see code like:
>>
>> if isinstance(request.DATA, YamlObject): ...
>> elif isinstance(request.DATA, dict): ...
>>
>> although, I'm not sure how any view could handle any random
>> content-type...
>>
>>
>>
>> Le mercredi 4 septembre 2013 13:57:29 UTC+2, Marc Tamlyn a écrit :
>>>
>>> The thing with request.POST is that it's kinda unintuitive when extended
>>> to other HTTP methods (e.g. PUT). This is why the old request.raw_post_data
>>> was renamed request.body.
>>>
>>> request.POST would behave in the expected traditional web way of picking
>>> up form encoded POST data, which would also be available in request.DATA as
>>> well, but request.DATA is the "new" way of doing it. Personally, I'd
>>> lowercase it though, to remove confusion with the PHP $POST $GET $REQUEST
>>> which we mimic on the request object. The generally have different use cases
>>> anyway - one for complex web things and the other for standard web browsers.
>>>
>>> (the above is what tom said...)
>>>
>>> Tom - what else do you have in DRF's Request that you would need?
>>>
>>>
>>> On 4 September 2013 12:56, Tom Christie  wrote:

 > Creating a request.DATA attribute would break compatibility with old
 > code and seem to me less intuitive.

 The implementation would ensure `request.POST` would still be populated
 for form data.

 There wouldn't have to be any backwards incompatible changes, and usage
 of `request.DATA` (or whatever better name there might be, perhaps simply
 `request.data`) would be entirely optional for using with generic request
 parsing, instead of form data only.


 > Where 

Re: Support POST of application/json content type

2013-09-06 Thread Tom Christie
In REST framework, we expose the accepted renderer as an 
`request.accepted_renderer` property.
This allows you to switch based on the media type when needed by either 
doing something like:

if request.accepted_renderer.format == 'yaml':
# do something specific

or like this:

if request.accepted_renderer.media_type == 'application/json':
# do something specific

That allows you to do media type specific stuff if needed, but generally 
you'll just access `request.DATA`, and not need to be concerned about 
exactly what media type was decoded for you.  (Of course the API specifics 
could be open to bike shedding.)  In response to Jonathan's particular 
query, REST framework supports a YAML parser, and there's never any need to 
do specific branching to detect it, because it just returns native python 
datatypes: lists, dicts, etc... exactly the same as the JSON parser does.

And yes, different media types can and do have different capabilities (eg. 
form data decodes into QueryDict objects, rather than regular dicts) but 
you're able to switch on that if specifically needed.

I'm not at all keen on the idea of injecting differing attribute names on 
the request based on the media type that is decoded, partly because it just 
feels like an odd API to me, but more concretely because it unnecessarily 
ties your view logic to your parser classes, and doesn't allow you to 
easily support views that accept multiple content types.  (The combination 
of either form data or json being the most common.) 

> what else do you have in DRF's Request that you would need

The bits of API we'd need to support something like this:

* `request.DATA` / `request.data` for accessing the parsed data.
* `request.parsers` or similar for modifying the parsers prior to parsing 
on a per-view basis.  (Note the parallel to request.upload_handlers)
* `request.accepted_parser` or similar for branching on media type if 
needed.
* A setting that controls the default set of installed parsers.

There are a couple of other bits of behaviour/API, but they're not related 
to request parsing.  I wouldn't mind discussing those as well at some 
point, but I don't want to go off on a tangent just yet.

The other core thing would be the question of if dealing with request 
parsing also means we should necessarily also tackle response rendering 
(and ideally, content negotiation) at the same time.  I don't really mind 
whether that'd be seen as a requirement or not, but it is important that we 
have a consistent design between both parsing and rendering.

Cheers all,

  Tom

On Thursday, 5 September 2013 00:31:43 UTC+1, Curtis Maloney wrote:

To weight in from an author of a different API tool
>
> In django-nap there's a simple "get_request_data" method which, 
> essentially, determines how to parse the request according to the content 
> type.  However, currently each API only supports a single serialiser format 
> [and HTTP Form encoding] so there's little guessing involved.
>
> However, I wouldn't want to see the same on Request, unless there was also 
> a direct way to imply the type you want.  For instance, if I get a request 
> that's in XML, and I ask for request.JSON, I'd like it to either yield 
> empty or raise an error.  Whereas when I didn't case, accessing 
> request.DATA would make a best guess.
>
> So I see two paths... either you use a "Decode it for me according to 
> content-type" interface, or a "Treat is as this type, and fail predictably 
> if it's not" one.
>
> The current GET/POST is, of course, the latter.  And there's no reason we 
> can't have both :)
>
> --
> Curtis
>
>
>
> On 5 September 2013 04:06, Jonathan Slenders 
>  > wrote:
>
>> Would that mean that the object returned by request.DATA/POST/whatever 
>> could be a different type, depending on what the user posted?
>>
>> I don't want to see code like:
>>
>> if isinstance(request.DATA, YamlObject): ...
>> elif isinstance(request.DATA, dict): ...
>>
>> although, I'm not sure how any view could handle any random 
>> content-type...
>>
>>
>>
>> Le mercredi 4 septembre 2013 13:57:29 UTC+2, Marc Tamlyn a écrit :
>>>
>>> The thing with request.POST is that it's kinda unintuitive when extended 
>>> to other HTTP methods (e.g. PUT). This is why the old request.raw_post_data 
>>> was renamed request.body.
>>>
>>> request.POST would behave in the expected traditional web way of picking 
>>> up form encoded POST data, which would also be available in request.DATA as 
>>> well, but request.DATA is the "new" way of doing it. Personally, I'd 
>>> lowercase it though, to remove confusion with the PHP $POST $GET $REQUEST 
>>> which we mimic on the request object. The generally have different use 
>>> cases anyway - one for complex web things and the other for standard web 
>>> browsers.
>>>
>>> (the above is what tom said...)
>>>
>>> Tom - what else do you have in DRF's Request that you would need?
>>>
>>>
>>> On 4 September 2013 

Re: Support POST of application/json content type

2013-09-04 Thread Curtis Maloney
To weight in from an author of a different API tool

In django-nap there's a simple "get_request_data" method which,
essentially, determines how to parse the request according to the content
type.  However, currently each API only supports a single serialiser format
[and HTTP Form encoding] so there's little guessing involved.

However, I wouldn't want to see the same on Request, unless there was also
a direct way to imply the type you want.  For instance, if I get a request
that's in XML, and I ask for request.JSON, I'd like it to either yield
empty or raise an error.  Whereas when I didn't case, accessing
request.DATA would make a best guess.

So I see two paths... either you use a "Decode it for me according to
content-type" interface, or a "Treat is as this type, and fail predictably
if it's not" one.

The current GET/POST is, of course, the latter.  And there's no reason we
can't have both :)

--
Curtis



On 5 September 2013 04:06, Jonathan Slenders wrote:

> Would that mean that the object returned by request.DATA/POST/whatever
> could be a different type, depending on what the user posted?
>
> I don't want to see code like:
>
> if isinstance(request.DATA, YamlObject): ...
> elif isinstance(request.DATA, dict): ...
>
> although, I'm not sure how any view could handle any random content-type...
>
>
>
> Le mercredi 4 septembre 2013 13:57:29 UTC+2, Marc Tamlyn a écrit :
>>
>> The thing with request.POST is that it's kinda unintuitive when extended
>> to other HTTP methods (e.g. PUT). This is why the old request.raw_post_data
>> was renamed request.body.
>>
>> request.POST would behave in the expected traditional web way of picking
>> up form encoded POST data, which would also be available in request.DATA as
>> well, but request.DATA is the "new" way of doing it. Personally, I'd
>> lowercase it though, to remove confusion with the PHP $POST $GET $REQUEST
>> which we mimic on the request object. The generally have different use
>> cases anyway - one for complex web things and the other for standard web
>> browsers.
>>
>> (the above is what tom said...)
>>
>> Tom - what else do you have in DRF's Request that you would need?
>>
>>
>> On 4 September 2013 12:56, Tom Christie  wrote:
>>
>>> > Creating a request.DATA attribute would break compatibility with old
>>> code and seem to me less intuitive.
>>>
>>> The implementation would ensure `request.POST` would still be populated
>>> for form data.
>>>
>>> There wouldn't have to be any backwards incompatible changes, and usage
>>> of `request.DATA` (or whatever better name there might be, perhaps simply
>>> `request.data`) would be entirely optional for using with generic request
>>> parsing, instead of form data only.
>>>
>>>
>>> > Where should request.FILE go in that case
>>>
>>> request.FILES would be populated by form data as normal, and would be
>>> empty on JSON or other submissions.  In order to support multipart parsing
>>> the request parsing API would need to provide for parsers that support file
>>> upload, in a similar way that REST framework currently does.
>>>
>>> > Would request.POST just be a call to request.DATA?
>>>
>>> That's an open question, but I'd probably expect it to only return data
>>> if the request contains multipart or url-encoded form data.  (Ie. the
>>> behaviour wouldn't change.)
>>>
>>> Cheers,
>>>
>>>   Tom
>>>
>>> On Wednesday, 4 September 2013 12:33:00 UTC+1, Stefan Berder wrote:

 Tom,
 I agree that the middleware solution is not the best and in my mind
 creates fragmentation of the same type of request handling.

 A generic content-type framework would make the request parsing more
 flexible, stronger to change in the future and open the door to any type.

 I'm curious about the choice of request.DATA though, when doing a POST
 request, I expect the data to be in the POST dictionary and nowhere else.
 Creating a request.DATA attribute would break compatibility with old code
 and seem to me less intuitive. Where should request.FILE go in that case?
 Would request.POST just be a call to request.DATA?

 Stefan

 On Wednesday, 4 September 2013 18:13:12 UTC+8, Tom Christie wrote:
>
> Hi Stefan,
>
> Sure, I'd be interested in seeing us improve how we deal with JSON
> requests and responses.
>
> My preference would be to introduce a request parsing and response
> rendering API that allows us to support not just JSON, but any media type
> that has a parser installed for it.  (I've commented on some of this
> before, 
> here,
> although I think I'm warming towards the idea that it's probably about 
> time
> we started addressing at least some of this in core.)
>
> Unsurprisingly I'd suggest the same general approach that 

Re: Support POST of application/json content type

2013-09-04 Thread Jonathan Slenders
Would that mean that the object returned by request.DATA/POST/whatever 
could be a different type, depending on what the user posted?

I don't want to see code like:

if isinstance(request.DATA, YamlObject): ...
elif isinstance(request.DATA, dict): ...

although, I'm not sure how any view could handle any random content-type...



Le mercredi 4 septembre 2013 13:57:29 UTC+2, Marc Tamlyn a écrit :
>
> The thing with request.POST is that it's kinda unintuitive when extended 
> to other HTTP methods (e.g. PUT). This is why the old request.raw_post_data 
> was renamed request.body.
>
> request.POST would behave in the expected traditional web way of picking 
> up form encoded POST data, which would also be available in request.DATA as 
> well, but request.DATA is the "new" way of doing it. Personally, I'd 
> lowercase it though, to remove confusion with the PHP $POST $GET $REQUEST 
> which we mimic on the request object. The generally have different use 
> cases anyway - one for complex web things and the other for standard web 
> browsers.
>
> (the above is what tom said...)
>
> Tom - what else do you have in DRF's Request that you would need?
>
>
> On 4 September 2013 12:56, Tom Christie  > wrote:
>
>> > Creating a request.DATA attribute would break compatibility with old 
>> code and seem to me less intuitive.
>>
>> The implementation would ensure `request.POST` would still be populated 
>> for form data.
>>
>> There wouldn't have to be any backwards incompatible changes, and usage 
>> of `request.DATA` (or whatever better name there might be, perhaps simply 
>> `request.data`) would be entirely optional for using with generic request 
>> parsing, instead of form data only.
>>
>>
>> > Where should request.FILE go in that case
>>
>> request.FILES would be populated by form data as normal, and would be 
>> empty on JSON or other submissions.  In order to support multipart parsing 
>> the request parsing API would need to provide for parsers that support file 
>> upload, in a similar way that REST framework currently does.
>>
>> > Would request.POST just be a call to request.DATA?
>>
>> That's an open question, but I'd probably expect it to only return data 
>> if the request contains multipart or url-encoded form data.  (Ie. the 
>> behaviour wouldn't change.)
>>
>> Cheers,
>>
>>   Tom
>>
>> On Wednesday, 4 September 2013 12:33:00 UTC+1, Stefan Berder wrote:
>>>
>>> Tom,
>>> I agree that the middleware solution is not the best and in my mind 
>>> creates fragmentation of the same type of request handling.
>>>
>>> A generic content-type framework would make the request parsing more 
>>> flexible, stronger to change in the future and open the door to any type.
>>>
>>> I'm curious about the choice of request.DATA though, when doing a POST 
>>> request, I expect the data to be in the POST dictionary and nowhere else. 
>>> Creating a request.DATA attribute would break compatibility with old code 
>>> and seem to me less intuitive. Where should request.FILE go in that case? 
>>> Would request.POST just be a call to request.DATA?
>>>
>>> Stefan
>>>
>>> On Wednesday, 4 September 2013 18:13:12 UTC+8, Tom Christie wrote:

 Hi Stefan,

 Sure, I'd be interested in seeing us improve how we deal with JSON 
 requests and responses.

 My preference would be to introduce a request parsing and response 
 rendering API that allows us to support not just JSON, but any media type 
 that has a parser installed for it.  (I've commented on some of this 
 before, 
 here,
  
 although I think I'm warming towards the idea that it's probably about 
 time 
 we started addressing at least some of this in core.)

 Unsurprisingly I'd suggest the same general approach that is used in 
 REST framework - A lazy `request.DATA` attribute (or similar) that when 
 accessed, inspects the media type on the request, and parses the request 
 stream with an appropriate parser if possible.  The installed parsers can 
 be configured globally, or on a per-request basis.  The existing multipart 
 and form-encoded parsing behaviour would no longer be a special case baked 
 directly into the request object, but instead be the default installed 
 parsers.

 Taking this approach makes it trivial to write views that can handle 
 both JSON and form data, and providing a proper parser API makes it easy 
 for developers to package up and share their own parser implementation, 
 such as YAML, XML and MsgPack.  (And, looking forwards, JSON-based media 
 types such as hypermedia types.)

 In REST framework this behaviour is (by necessity) implemented in a 
 Request object that wraps the underlying HttpRequest, but the same basic 
 implementation can be applied to implementing it 

Re: Support POST of application/json content type

2013-09-04 Thread Marc Tamlyn
The thing with request.POST is that it's kinda unintuitive when extended to
other HTTP methods (e.g. PUT). This is why the old request.raw_post_data
was renamed request.body.

request.POST would behave in the expected traditional web way of picking up
form encoded POST data, which would also be available in request.DATA as
well, but request.DATA is the "new" way of doing it. Personally, I'd
lowercase it though, to remove confusion with the PHP $POST $GET $REQUEST
which we mimic on the request object. The generally have different use
cases anyway - one for complex web things and the other for standard web
browsers.

(the above is what tom said...)

Tom - what else do you have in DRF's Request that you would need?


On 4 September 2013 12:56, Tom Christie  wrote:

> > Creating a request.DATA attribute would break compatibility with old
> code and seem to me less intuitive.
>
> The implementation would ensure `request.POST` would still be populated
> for form data.
>
> There wouldn't have to be any backwards incompatible changes, and usage of
> `request.DATA` (or whatever better name there might be, perhaps simply
> `request.data`) would be entirely optional for using with generic request
> parsing, instead of form data only.
>
>
> > Where should request.FILE go in that case
>
> request.FILES would be populated by form data as normal, and would be
> empty on JSON or other submissions.  In order to support multipart parsing
> the request parsing API would need to provide for parsers that support file
> upload, in a similar way that REST framework currently does.
>
> > Would request.POST just be a call to request.DATA?
>
> That's an open question, but I'd probably expect it to only return data if
> the request contains multipart or url-encoded form data.  (Ie. the
> behaviour wouldn't change.)
>
> Cheers,
>
>   Tom
>
> On Wednesday, 4 September 2013 12:33:00 UTC+1, Stefan Berder wrote:
>>
>> Tom,
>> I agree that the middleware solution is not the best and in my mind
>> creates fragmentation of the same type of request handling.
>>
>> A generic content-type framework would make the request parsing more
>> flexible, stronger to change in the future and open the door to any type.
>>
>> I'm curious about the choice of request.DATA though, when doing a POST
>> request, I expect the data to be in the POST dictionary and nowhere else.
>> Creating a request.DATA attribute would break compatibility with old code
>> and seem to me less intuitive. Where should request.FILE go in that case?
>> Would request.POST just be a call to request.DATA?
>>
>> Stefan
>>
>> On Wednesday, 4 September 2013 18:13:12 UTC+8, Tom Christie wrote:
>>>
>>> Hi Stefan,
>>>
>>> Sure, I'd be interested in seeing us improve how we deal with JSON
>>> requests and responses.
>>>
>>> My preference would be to introduce a request parsing and response
>>> rendering API that allows us to support not just JSON, but any media type
>>> that has a parser installed for it.  (I've commented on some of this
>>> before, 
>>> here,
>>> although I think I'm warming towards the idea that it's probably about time
>>> we started addressing at least some of this in core.)
>>>
>>> Unsurprisingly I'd suggest the same general approach that is used in
>>> REST framework - A lazy `request.DATA` attribute (or similar) that when
>>> accessed, inspects the media type on the request, and parses the request
>>> stream with an appropriate parser if possible.  The installed parsers can
>>> be configured globally, or on a per-request basis.  The existing multipart
>>> and form-encoded parsing behaviour would no longer be a special case baked
>>> directly into the request object, but instead be the default installed
>>> parsers.
>>>
>>> Taking this approach makes it trivial to write views that can handle
>>> both JSON and form data, and providing a proper parser API makes it easy
>>> for developers to package up and share their own parser implementation,
>>> such as YAML, XML and MsgPack.  (And, looking forwards, JSON-based media
>>> types such as hypermedia types.)
>>>
>>> In REST framework this behaviour is (by necessity) implemented in a
>>> Request object that wraps the underlying HttpRequest, but the same basic
>>> implementation can be applied to implementing it directly in the Request
>>> object, and would be somewhat easier.
>>>
>>> I'm interested to see Marc suggesting middleware specifically for
>>> handling JSON requests.  That'd work, and would be a simple approach.  My
>>> reservations with that would be:
>>>
>>> * We'd not be making it any easier for users to deal with request
>>> parsing generally.
>>> * There's no way to write views that deal with request data in an
>>> agonistic way, and dealing with differing media types would require
>>> switching based on the media type in the view itself.  For example, the
>>> 

Re: Support POST of application/json content type

2013-09-04 Thread Tom Christie
> Creating a request.DATA attribute would break compatibility with old code 
and seem to me less intuitive.

The implementation would ensure `request.POST` would still be populated for 
form data.

There wouldn't have to be any backwards incompatible changes, and usage of 
`request.DATA` (or whatever better name there might be, perhaps simply 
`request.data`) would be entirely optional for using with generic request 
parsing, instead of form data only.

> Where should request.FILE go in that case

request.FILES would be populated by form data as normal, and would be empty 
on JSON or other submissions.  In order to support multipart parsing the 
request parsing API would need to provide for parsers that support file 
upload, in a similar way that REST framework currently does.

> Would request.POST just be a call to request.DATA?

That's an open question, but I'd probably expect it to only return data if 
the request contains multipart or url-encoded form data.  (Ie. the 
behaviour wouldn't change.)

Cheers,

  Tom

On Wednesday, 4 September 2013 12:33:00 UTC+1, Stefan Berder wrote:
>
> Tom,
> I agree that the middleware solution is not the best and in my mind 
> creates fragmentation of the same type of request handling.
>
> A generic content-type framework would make the request parsing more 
> flexible, stronger to change in the future and open the door to any type.
>
> I'm curious about the choice of request.DATA though, when doing a POST 
> request, I expect the data to be in the POST dictionary and nowhere else. 
> Creating a request.DATA attribute would break compatibility with old code 
> and seem to me less intuitive. Where should request.FILE go in that case? 
> Would request.POST just be a call to request.DATA?
>
> Stefan
>
> On Wednesday, 4 September 2013 18:13:12 UTC+8, Tom Christie wrote:
>>
>> Hi Stefan,
>>
>> Sure, I'd be interested in seeing us improve how we deal with JSON 
>> requests and responses.
>>
>> My preference would be to introduce a request parsing and response 
>> rendering API that allows us to support not just JSON, but any media type 
>> that has a parser installed for it.  (I've commented on some of this 
>> before, 
>> here,
>>  
>> although I think I'm warming towards the idea that it's probably about time 
>> we started addressing at least some of this in core.)
>>
>> Unsurprisingly I'd suggest the same general approach that is used in REST 
>> framework - A lazy `request.DATA` attribute (or similar) that when 
>> accessed, inspects the media type on the request, and parses the request 
>> stream with an appropriate parser if possible.  The installed parsers can 
>> be configured globally, or on a per-request basis.  The existing multipart 
>> and form-encoded parsing behaviour would no longer be a special case baked 
>> directly into the request object, but instead be the default installed 
>> parsers.
>>
>> Taking this approach makes it trivial to write views that can handle both 
>> JSON and form data, and providing a proper parser API makes it easy for 
>> developers to package up and share their own parser implementation, such as 
>> YAML, XML and MsgPack.  (And, looking forwards, JSON-based media types such 
>> as hypermedia types.)
>>
>> In REST framework this behaviour is (by necessity) implemented in a 
>> Request object that wraps the underlying HttpRequest, but the same basic 
>> implementation can be applied to implementing it directly in the Request 
>> object, and would be somewhat easier.
>>
>> I'm interested to see Marc suggesting middleware specifically for 
>> handling JSON requests.  That'd work, and would be a simple approach.  My 
>> reservations with that would be:
>>
>> * We'd not be making it any easier for users to deal with request parsing 
>> generally.
>> * There's no way to write views that deal with request data in an 
>> agonistic way, and dealing with differing media types would require 
>> switching based on the media type in the view itself.  For example, the 
>> generic views would still only support form data.  As another example, if 
>> you wanted to add, say, MsgPack support to your application, you'd need to 
>> re-write all your views.
>>
>> From my point of view this is already a solved problem, and I'd really 
>> like to see a generic approach to handling request data, and a 
>> corresponding approach to rendering responses into an appropriate media 
>> type.
>>
>>  All the best,
>>
>>   Tom
>>
>> On Tuesday, 3 September 2013 06:30:04 UTC+1, Stefan Berder wrote:
>>>
>>> Hi,
>>> I looked around the list and couldn't find any mention of this subject.
>>>
>>> In `django.http.request.HttpRequest._load_post_and_files()` there is 
>>> explicit mention of two content type ('multipart/form-data' and 
>>> 'application/x-www-form-urlencoded'), any other content type will get empty 
>>> values for self._post.

Re: Support POST of application/json content type

2013-09-04 Thread Stefan Berder
Tom,
I agree that the middleware solution is not the best and in my mind creates 
fragmentation of the same type of request handling.

A generic content-type framework would make the request parsing more 
flexible, stronger to change in the future and open the door to any type.

I'm curious about the choice of request.DATA though, when doing a POST 
request, I expect the data to be in the POST dictionary and nowhere else. 
Creating a request.DATA attribute would break compatibility with old code 
and seem to me less intuitive. Where should request.FILE go in that case? 
Would request.POST just be a call to request.DATA?

Stefan

On Wednesday, 4 September 2013 18:13:12 UTC+8, Tom Christie wrote:
>
> Hi Stefan,
>
> Sure, I'd be interested in seeing us improve how we deal with JSON 
> requests and responses.
>
> My preference would be to introduce a request parsing and response 
> rendering API that allows us to support not just JSON, but any media type 
> that has a parser installed for it.  (I've commented on some of this 
> before, 
> here,
>  
> although I think I'm warming towards the idea that it's probably about time 
> we started addressing at least some of this in core.)
>
> Unsurprisingly I'd suggest the same general approach that is used in REST 
> framework - A lazy `request.DATA` attribute (or similar) that when 
> accessed, inspects the media type on the request, and parses the request 
> stream with an appropriate parser if possible.  The installed parsers can 
> be configured globally, or on a per-request basis.  The existing multipart 
> and form-encoded parsing behaviour would no longer be a special case baked 
> directly into the request object, but instead be the default installed 
> parsers.
>
> Taking this approach makes it trivial to write views that can handle both 
> JSON and form data, and providing a proper parser API makes it easy for 
> developers to package up and share their own parser implementation, such as 
> YAML, XML and MsgPack.  (And, looking forwards, JSON-based media types such 
> as hypermedia types.)
>
> In REST framework this behaviour is (by necessity) implemented in a 
> Request object that wraps the underlying HttpRequest, but the same basic 
> implementation can be applied to implementing it directly in the Request 
> object, and would be somewhat easier.
>
> I'm interested to see Marc suggesting middleware specifically for handling 
> JSON requests.  That'd work, and would be a simple approach.  My 
> reservations with that would be:
>
> * We'd not be making it any easier for users to deal with request parsing 
> generally.
> * There's no way to write views that deal with request data in an 
> agonistic way, and dealing with differing media types would require 
> switching based on the media type in the view itself.  For example, the 
> generic views would still only support form data.  As another example, if 
> you wanted to add, say, MsgPack support to your application, you'd need to 
> re-write all your views.
>
> From my point of view this is already a solved problem, and I'd really 
> like to see a generic approach to handling request data, and a 
> corresponding approach to rendering responses into an appropriate media 
> type.
>
>  All the best,
>
>   Tom
>
> On Tuesday, 3 September 2013 06:30:04 UTC+1, Stefan Berder wrote:
>>
>> Hi,
>> I looked around the list and couldn't find any mention of this subject.
>>
>> In `django.http.request.HttpRequest._load_post_and_files()` there is 
>> explicit mention of two content type ('multipart/form-data' and 
>> 'application/x-www-form-urlencoded'), any other content type will get empty 
>> values for self._post.
>>
>> Given that a lot of user form interaction is now happening through 
>> 'XMLHttpRequest', I think that the 'application/json' content type should 
>> be supported. A lot of javascript libraries will use json as the default 
>> format:
>> * angularjs: http://docs.angularjs.org/api/ng.$http, see "Setting HTTP 
>> Headers"
>> * emberjs: 
>> https://github.com/emberjs/data/blob/master/packages/ember-data/lib/adapters/rest_adapter.js#L974
>> * backbone: http://backbonejs.org/#Sync
>> * jquery: http://api.jquery.com/jQuery.ajax/ (the only one using 
>> 'application/x-www-form-urlencoded' by default)
>>
>> I'm trying primarily to create a discussion on the subject and am ready 
>> to provide the code for it as I had to write it. This would help avoid 
>> hacks to handle the request object in my view. 
>>
>> I know there are some apps to handle API construction (django-tastypie, 
>> django-rest, django-piston and certainly others) they use either request 
>> wrappers or request handling in their views.
>>
>> Stefan
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails 

Re: Support POST of application/json content type

2013-09-04 Thread Tom Christie
Hi Stefan,

Sure, I'd be interested in seeing us improve how we deal with JSON requests 
and responses.

My preference would be to introduce a request parsing and response 
rendering API that allows us to support not just JSON, but any media type 
that has a parser installed for it.  (I've commented on some of this 
before, 
here,
 
although I think I'm warming towards the idea that it's probably about time 
we started addressing at least some of this in core.)

Unsurprisingly I'd suggest the same general approach that is used in REST 
framework - A lazy `request.DATA` attribute (or similar) that when 
accessed, inspects the media type on the request, and parses the request 
stream with an appropriate parser if possible.  The installed parsers can 
be configured globally, or on a per-request basis.  The existing multipart 
and form-encoded parsing behaviour would no longer be a special case baked 
directly into the request object, but instead be the default installed 
parsers.

Taking this approach makes it trivial to write views that can handle both 
JSON and form data, and providing a proper parser API makes it easy for 
developers to package up and share their own parser implementation, such as 
YAML, XML and MsgPack.  (And, looking forwards, JSON-based media types such 
as hypermedia types.)

In REST framework this behaviour is (by necessity) implemented in a Request 
object that wraps the underlying HttpRequest, but the same basic 
implementation can be applied to implementing it directly in the Request 
object, and would be somewhat easier.

I'm interested to see Marc suggesting middleware specifically for handling 
JSON requests.  That'd work, and would be a simple approach.  My 
reservations with that would be:

* We'd not be making it any easier for users to deal with request parsing 
generally.
* There's no way to write views that deal with request data in an agonistic 
way, and dealing with differing media types would require switching based 
on the media type in the view itself.  For example, the generic views would 
still only support form data.  As another example, if you wanted to add, 
say, MsgPack support to your application, you'd need to re-write all your 
views.

>From my point of view this is already a solved problem, and I'd really like 
to see a generic approach to handling request data, and a corresponding 
approach to rendering responses into an appropriate media type.

 All the best,

  Tom

On Tuesday, 3 September 2013 06:30:04 UTC+1, Stefan Berder wrote:
>
> Hi,
> I looked around the list and couldn't find any mention of this subject.
>
> In `django.http.request.HttpRequest._load_post_and_files()` there is 
> explicit mention of two content type ('multipart/form-data' and 
> 'application/x-www-form-urlencoded'), any other content type will get empty 
> values for self._post.
>
> Given that a lot of user form interaction is now happening through 
> 'XMLHttpRequest', I think that the 'application/json' content type should 
> be supported. A lot of javascript libraries will use json as the default 
> format:
> * angularjs: http://docs.angularjs.org/api/ng.$http, see "Setting HTTP 
> Headers"
> * emberjs: 
> https://github.com/emberjs/data/blob/master/packages/ember-data/lib/adapters/rest_adapter.js#L974
> * backbone: http://backbonejs.org/#Sync
> * jquery: http://api.jquery.com/jQuery.ajax/ (the only one using 
> 'application/x-www-form-urlencoded' by default)
>
> I'm trying primarily to create a discussion on the subject and am ready to 
> provide the code for it as I had to write it. This would help avoid hacks 
> to handle the request object in my view. 
>
> I know there are some apps to handle API construction (django-tastypie, 
> django-rest, django-piston and certainly others) they use either request 
> wrappers or request handling in their views.
>
> Stefan
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-03 Thread Curtis Maloney
Someone remind me never to post to list before I've had my second coffee? :)

Of course the above would trivially be handled using middleware.

--
C


On 4 September 2013 10:05, Curtis Maloney wrote:

> What do people think of an extensible registration approach to handling
> request content types?
>
> Something along the lines of ('POST', [list of content types], class to
> handle it)
>
> This would mean adding a request.JSON would be simple for the projects
> that want it, and even overriding how the existing content types are
> handled would be possible. [I haven't seen a use case for such yet, but
> it's a wild world out there :)]
>
> --
> Curtis
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-03 Thread Curtis Maloney
What do people think of an extensible registration approach to handling
request content types?

Something along the lines of ('POST', [list of content types], class to
handle it)

This would mean adding a request.JSON would be simple for the projects that
want it, and even overriding how the existing content types are handled
would be possible. [I haven't seen a use case for such yet, but it's a wild
world out there :)]

--
Curtis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Support POST of application/json content type

2013-09-03 Thread Marc Tamlyn
+1 to providing a better way of accessing JSON encoded post data. -1 to it
being munged into request.POST - I feel this is unintuitive.

A nice possibility could be a simple piece of middleware which detects
`application/json` and attaches request.json as an attribute, if it can
decode it. This attribute could actually be made lazy so as not to
introduce any real overhead on other pages, or on views already decoding
the data directly from request.data.

There is a (somewhat stalled) ticket for a JsonResponse class at #17942 (
https://code.djangoproject.com/ticket/17942).


On 3 September 2013 06:30, Stefan Berder  wrote:

> Hi,
> I looked around the list and couldn't find any mention of this subject.
>
> In `django.http.request.HttpRequest._load_post_and_files()` there is
> explicit mention of two content type ('multipart/form-data' and
> 'application/x-www-form-urlencoded'), any other content type will get empty
> values for self._post.
>
> Given that a lot of user form interaction is now happening through
> 'XMLHttpRequest', I think that the 'application/json' content type should
> be supported. A lot of javascript libraries will use json as the default
> format:
> * angularjs: http://docs.angularjs.org/api/ng.$http, see "Setting HTTP
> Headers"
> * emberjs:
> https://github.com/emberjs/data/blob/master/packages/ember-data/lib/adapters/rest_adapter.js#L974
> * backbone: http://backbonejs.org/#Sync
> * jquery: http://api.jquery.com/jQuery.ajax/ (the only one using
> 'application/x-www-form-urlencoded' by default)
>
> I'm trying primarily to create a discussion on the subject and am ready to
> provide the code for it as I had to write it. This would help avoid hacks
> to handle the request object in my view.
>
> I know there are some apps to handle API construction (django-tastypie,
> django-rest, django-piston and certainly others) they use either request
> wrappers or request handling in their views.
>
> Stefan
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.