Great points and thanks for the code example (I wish more stuff like this was 
floating around in Catalyst oriented blogs, etc.)

Just stepping back a bit, the purpose of the research spike was to see what, if 
any, features could be incorporated into Catalyst core to improve and 
streamline how we work with some of the contemporary use cases around AJAX and 
REST.  Its very possible that the answer to that is no, there's nothing that 
belongs in Catalyst core around this.  I think the reason for the research is 
that a lot of modern web frameworks do provide such support out of the box, and 
the fact that Catalyst doesn't have it at first might seem like a deficiency in 
the framework. Other reasons for baking this into core might come to mind (and 
have various levels of validity)

-- having a common approach could result in the benefits of DRY (better, more 
secure code, less stuff for newbies to figure out)
-- For the common cases it would help people new to a project (instead of 
having to figure out how this project handles AJAX, or the five different ways 
its handled, for example).
-- a common, core approach might help focus the community a bit by removing a 
common use case that has no clearly agreed upon approach.

I'm personally not in love with the 'gigantic core' approach, but since 
Catalyst already incorporates support for classic form parameters it does I 
think make sense (for at least the 5.9xxxx series) for us to support common 
AJAX use cases.  And I do think some of the things people would like to do with 
REST, such as match actions based on content negotiation are valid.  

The main issue that I see is that we have too many ways to do exactly the same 
thing (return JSON for AJAX endpoints) and no clear reason why any of them are 
better for a given purpose.  Additionally, some of them are a bit verbose, and 
Catalyst already has reputation for being the long way to do simple things.

However it does seem to me after we've all talked about it quite a bit that at 
this point there really doesn't seem to be a really exciting approach that 
would be useful (talking about a common way to handle the serialize / format 
stuff, like allowing res->body to take a ref and convert it to JSON or XML, 
etc.)   At least in terms of something that belongs in Catalyst core.  Why not 
lets pull that part out of the spec and make it a separate research project, 
and for now continue to let the community play with various approaches.  I 
think there appears to be less controversy on the request side, in terms of 
building in alternative content parsing, and possibly a first go at subroutine 
attribute content type negotiation.  

Any thoughts on the request side of the proposal?

Thanks everyone for joining in!


> On Tuesday, August 13, 2013 4:16 AM, Craig Chant 
> <> wrote:
> > Hi John / Alex
> Am I missing something here?
> In my model I create and return the JSON...
>     use JSON qw(encode_json);
>    my %json_hash ;
> ... add some hash stuff to be made into JSON ...
>     # return JSON
>     return encode_json \%json_hash
> ----------------------------------
> In my controller I output it to the view / browser
>     $c->response->body( 
> $c->model('My::Model')->my_method_JSON );
> ---------------------------------
> With an AJAX call I retrieve the JSON
> //////////////////////////////////
> /////////////////////////////////
> var myJSON;
> function myAJAXFunction()
> {
>     // call AJAX
>     makeRequest('/my_controller/my_method_JSON ','',set_JSON);
> }
> function setJSON(json)
> {
>     // store JSON returned from AJAX call
>     myJSON = $.parseJSON(json);
> }
> --------------------------------------------
> This is working fine, what shift in paradigm are you trying to create and how 
> would it affect / alter what I currently do, or indeed make it any easier / 
> simpler, I can't see it  getting much simpler than the few lines of code I 
> already have, can it?
> Or am I doing something wrong?
> Cheers,
> Craig Chant.
> -----Original Message-----
> From: John Napiorkowski []
> Sent: 12 August 2013 21:27
> To: The elegant MVC web framework
> Subject: Re: [Catalyst] More detailed proposal for changes related to content 
> negotiation and REST
>>  On Monday, August 12, 2013 2:33 PM, Alexander Hartmaier 
> <> wrote:
>>  > On 2013-08-12 16:58, John Napiorkowski wrote:
>>>   Hey Bill (and the rest of you lurkers!)
>>>   I just updated the spec over at
>>  otiation/blob/master/README.pod
>>>   I decided to remove regular expression matching from the body parser
>>>  stuff,
>>  which I think resolves the merging questions (since now there will not
>>  be a possibility that more that one can match the request).  I think
>>  we can add this back eventually using some standard request content
>>  negotiation, using mime type patterns and quality settings, so that we
>>  can have some rules that dictate what is the best match, rather than try to 
> invent our own.  For example:
>>>  tion
>>>   The idea would be to not reinvent.  I think we could instead of
>>>  doing an
>>  equality match here we just use something like this to figure out what
>>  the best match is works pretty well.  Thoughts?
>>>   jnap
>>  Hi John,
>>  I thought about it for the last few days and wonder why the, lets call
>>  it rendering, of the data isn't left to the view as defined by the MVC
>>  pattern?
>>  I'd expect that a different view is used depending on the negotiated
>>  content-type.
>>  How do other MVC frameworks choose the view to use?
>>  Should a single action be able to limit the output format or is
>>  controller level granular enough?
>>  Best regards, Alex
> Alex,
> I think you put your finger on one of the major uneasiness I (and others) 
> have 
> around the idea of having in the global application model all these 
> registered 
> formatters.  Yes, in theory it feels like we are cheating on the View side of 
> MVC.  I have personally always thought that Catalyst doesn't exactly get it 
> right the way it is (I think C and V are actually a little too detached for 
> one 
> thing) and that leads to odd code sometimes.  The commonly used 
> Catalyst::Action::Renderview is a bit too detached for my taste.  And what we 
> call a View tends to mostly just be a View handler (or factory I guess).  On 
> the 
> other hand the basic idea of separation of concerns is sound.
> I think the main thing is that this is not intended to replace view, but for 
> the 
> simple cases where people just want to quickly serialize data (like for all 
> those ajax endpoints you need to do nowadays, not full on RESTful APIs but 
> quick 
> and dirty communication between the back and front end.  Maybe that's not a 
> great thing for Catalyst (and honestly I put this out there in the hopes of 
> provocation some strong reactions.
> Personally I prefer to use templates even for creating JSON output, I think 
> you 
> get cleaner separation that is easier to maintain over time (I really don't 
> like when I see something like ->body (json_encode 
> $sql->run->get_all_rows).  That feels fragile to me.  On the other hand I 
> see the attraction of some of the more lightweight web frameworks where they 
> make it easy to just spew out JSON.
> This is partly why I sketched out an action/controller level alternative, 
> with 
> the proposed response->body_format thing and the proposed new action 
> subroutine attributes  (just to recap)
> sub myaction :Local {
>   My ($self, $c) = @_;
>   # ...
>   # ...
>   $c->response->format(
>      'application/json' => sub {  json_encode $stuff },
>      # ...
>      # ...
>   );
> }
> I think this approach feels similar to how some other frameworks operate.  
> Some 
> offer more sugary syntax for the common stuff, perhaps
> $c->response
>   ->json( sub { ... } )
>   ->html ( sub { ... } ).
>   -> ...
>   -> ... ;
> and I guess we could say there's a shortcut to forward to a View instead
> $c->response
>   ->json("JSON")
>   ->html ("TTHTML").
>   -> ...
>   -> ... ;
> But that can all be worked out after the basic thought is in place.
> and again, some other frameworks (some java system) they use annotations 
> similar 
> to our action level subroutine attributes.  I think we also try to hit that 
> with 
> the proposed "Provides/Consumes" attributes.  The main thing is I 
> can't see a way to properly do content negotiatin with ssubroutine 
> attributes given the exiting catalyst dispatcher (basically the system is 
> mostly 
> a first match win)
> Perhaps that is all we need, and we can skip idea of needing default global 
> body 
> formatters?  Or maybe we'd prefer to think about leveraging more of 
> Web::Dispatch, and mst has this great notion of setting response filters, 
> which 
> we could get for free if we use web-dispatch.  Instead of setting a global 
> point 
> for the encoding, we could control is more granularly that way.
> I guess ultimately it comes down to a question over do we need a full on view 
> for handling REST and straight up data encoding.  Personally I do think there 
> is 
> a use case here that Catalyst isn't hitting right, and I am pretty sure some 
> of the ideas in the stand alone Catalyst-Action-REST do apply but I'd like 
> to see that more native and probably scoped more tightly (I don't think we 
> need or should have the full CAR in core, but I do think we should ask 
> ourselves.
> I guess its a bit tough to look at other frameworks since one thing about 
> Catalyst is that our idea of a Controller isn't so central as in other 
> frameworks, since with action chaining all the fun happens in the action 
> really.  ALthough chaining is powerful it does lead to some confusions in 
> terms 
> of how to lay out the applications and so forth.
> I have some thoughts about that, but its really aimed at the future.
> I guess we could just drop the global format stuff, given the questions and 
> controversies.  I'd love to find a way that doesn't suck for people to 
> be able to do JSON response in Catalyst without a lot of boilerplate, but 
> maybe 
> Catalyst isn't aimed to cater to that... The Catalyst::View::JSON is not too 
> bad, just has some docs that need updating I think.
> More thoughts and comments?
>>>>   On Friday, August 9, 2013 5:38 PM, John Napiorkowski
>>  <> wrote:
>>>>   On Friday, August 9, 2013 4:52 PM, Bill Moseley
>>  <> wrote:
>>>>>   On Fri, Aug 9, 2013 at 12:11 PM, John Napiorkowski
>>  <>
>>>>   wrote:
>>>>   What's the use case you have in mind?  Something like first 
> check
>>  for
>>>>   something like 'application/vnd.mycompany.user+json' and 
> then
>>  fall back
>>>>   to 'application/(?:vmd.*+)?json' if you don't find 
> it?  Is
>>  that an
>>>>   actual case you've come across?
>>>>>   Ya, that's kind of what I was thinking.   Or also having a
>>  final
>>>>   fallback parser that tries to figure out the type by other means
>>>>  than
>>  just
>>>>   looking at the Content type provided in the request.  Or even a
>>  '.'
>>>>   final match-anything that does some special logging.
>>>>>   It would be easy enough to find out if application/json was in 
> the
>>  array
>>>>   more than once by mistake.
>>>>   Seems like a reasonable use case then, although I would encourage
>>  future
>>>>   development to aim at putting more specificity in the controller,
>>  rather than
>>>>   rely on the global application.  The primary reason to have
>>>>  anything
>>  here at all
>>>>   is to have a base people can build on.  I do fear the globalness 
> of
>>>>  it,
>>  but it
>>>>   seems not an unreasonable compromise based on how Catalyst 
> actually
>>  works today.
>>>>>>   We've spoken before about the parsing larger incoming 
> and
>>  chunked
>>>>   data thing before.  I would love to address this, but right now it
>>  seems like
>>>>   something we need better agreement on in the psgi level.  For
>>>>  example,
>>  since
>>>>   star man already buffers incoming input, it feels silly to me to
>>>>  have
>>  catalyst
>>>>   then try to re-stream that.  You've already paid the full 
> price of
>>  buffering
>>>>   in terms of memory, and performance right?  Or am I not 
> understanding?
>>>>>   I added a Plack middleware to handle chunked encoded requests 
> -- I
>>  needed it
>>>>   for the Catalyst dev server and for Apache/mod_perl.   Yes, 
> Starman
>>  already
>>>>   de-chunks and buffers and works perfectly.
>>>>>   Apache actually de-chunks the request, but doesn't update 
> the
>>>>   Content-Length header and leaves on the Transfer-Encoding: chunked
>>  header.  So,
>>>>   sadly, I do flush this to a temporary file only to get the
>>  content-length to
>>>>   make Catalyst happy.
>>>>   Right, so I think in the end we all agreed it was psgi that should
>>>>  be  responsible for dealing with chunks or whatever (based on the
>>>>  http
>>  level support
>>>>   of the server).  The only think would be could there be some sane
>>  approach that
>>>>   exposed the input stream as a non blockable file handle that has
>>>>  not
>>  already
>>>>   been read into a buffer (memory or otherwise).  I do see the
>>>>  possible
>>  advantage
>>>>   there for processing efficiently large POST or PUT.  However again
>>>>  this
>>  needs to
>>>>   be done at the PSGI level, something like or similar.
>>  That would
>>>>   smooth over chucked versus non chunked and expose a readable 
> stream
>>>>  of
>>  the input
>>>>   that has not yet been buffered.
>>>>>   I'd really like to have something at the Catalyst level 
> that
>>  sanely
>>>>   acheives this end, but I think part of the price we paid when 
> going
>>>>  to
>>  PSGi at
>>>>   the core, is that most of the popular plack handlers are pre
>>>>  loading
>>  and
>>>>   buffering input, even large request input.  This seems to be an
>>>>  area
>>  where it
>>>>   might behoove us to work with the psgi group to find something
>>>>  stable.
>>  Even the
>>>>   optional isn't always going to work out, since some 
> people
>>>>  don't want to support that in the handler (its a somewhat vague
>>  definition I
>>>>   guess and makes people uncomfortable).
>>>>>>   Until them, or until someone helps me understand that my
>>  thinking is
>>>>   totally wrong on this score, it seems the best thing to do is to
>>>>  put
>>  this out of
>>>>   scope for now.  That way we can move on supporting a goodly number
>>>>  of
>>  real use
>>>>   cases.
>>>>>   Agreed.
>>>>>>   I intended to say that $_ equals a string that is the 
> buffered
>>  request
>>>>   body.  This way we can reserve other args for handling the future
>>  streaming
>>>>   case.  I was actually pondering something were the sub ref returns
>>>>  a
>>  sub ref
>>>>   that gets called over and over to do the parse.
>>>>>   I just don't want file uploads in memory.   (Oh, I have 
> another
>>  post
>>>>   coming on that -- thanks for the reminder.)  Well, Catalyst 
> doesn't
>>>>  but I think Starman might depending on the
>>  size of
>>>>   the incoming.  However I think you can override that with a monkey
>>  patch.
>>>>>    >
>>>>>>   I not quite sure about $c->res->body( \%data );  
> I
>>  think body
>>>>   should be the raw body.   What does $c->res->body return?  
> The
>>  serialized
>>>>   json?  The original hashref?
>>>>>>   I'm not sure I like it either.  I would say body 
> returns
>>  whatever
>>>>   you set it to, until the point were encoding happens.  It does 
> feel
>>>>  a
>>  bit flaky,
>>>>   but I can't actually put my finger on a real code smell here.
>>>>>>   Any other suggestions?  This is certainly a part of the
>>  proposal that is
>>>>   going to raise doubt, but I can't think of something better, 
> or
>>  assemble
>>>>   problematic use cases in my head over it either.
>>>>>   I don't really mind adding to 
> $c->stash->{rest}.
>>  It's
>>>>   kind of a staging area to put data until it's ready to be 
> encoded
>>  into the
>>>>   body.   I might get it partially loaded with data and then never
>>>>  use it
>>  and
>>>>   return some other body.   Noting precludes that, of course.   Ya,
>>>>  tough
>>  one.
>>>>   Well, I definitely don't want to stick this in the stash, you 
> all
>>  will have
>>>>   to tie me down to get that past!  Given that body already allows a
>>>>  file
>>  handle,
>>>>   I thought adding into that would be the most simple thing, but 
> lets
>>  give it more
>>>>   thought and maybe some other minds will come up with better ideas.
>>  I'll
>>>>   bounce it off t0m and mst as well.
>>>>>>>   If a parser dies what kind of exception is thrown?  
> You
>>  say they
>>>>   would not set any response status, but wouldn't we want to 
> catch
>>  the error
>>>>   and then set a 400?  (I use exception objects that carry http
>>>>  status, a
>>  message
>>>>   to return in the body and a message used for logging at a given
>>>>  level.)
>>>>>>   How people do exceptions in Perl tends to be nearly 
> religious,
>>  and I
>>>>   didn't want to hold this up based on figuring that stuff out 
> :)  I
>>  was
>>>>   thinking to just raise an exception and let the existing Catalyst
>>>>  stuff
>>  do its
>>>>   thing.  I'm just thinking not to add anything special for this 
> type
>>  of
>>>>   error, but just do the existing behavior, for better or worse.
>>>>>   Agreed.  If I were to write everything from scratch again 
> I'd
>>  be doing
>>>>   $c->throw_not_found or $c->throw_forbidden with exception 
> objects
>>  as the
>>>>   code ends up much cleaner and sane.   But, everyone has their own
>>  approaches.
>>>>   One thing is to have the response->from_psgi thing which would 
> make
>>  ti easy
>>>>   to graft in something like
>>>>   sub myaction :Local {
>>>>   my ($self, $c) = @_;
>>>>   $c->res->from_psgi( http_throw({
>>>>       status_code => 500,
>>>>       reason      => 'Internal Server Error',
>>>>       message     => 'Something has gone very wrong!'
>>>>   }))
>>>>   }
>>>>   somthing along those lines I think.
>>>>>   since request->body_data is intended to be lazy, we 
> won't
>>  run that
>>>>   parse code until you ask for the data.  We don't need to parse 
> the
>>  data to
>>>>   do the basic match here, this is just based on the HTTP meta data,
>>>>  no
>>  the actual
>>>>   content.  I think for common cases this is fine (I realize that 
> yet
>>  again this
>>>>   might not be the best approach for multipart uploads...)
>>>>>   Another tough one.    Just seems like PUT /user should accept 
> the
>>  same data
>>>>   regardless of how it is serialized.   And GET /user would get the
>>>>  user
>>  data and
>>>>   then serialize that to JSON or whatever but it's the same 
> data.
>>>>>   But, maybe you have a point.    I would worry that someone 
> assumes
>>  JSON and
>>>>   adds that content type match and then wonder why later it's 
> not
>>  working for
>>>>   other request serializations.
>>>>   well strikely speaking restful content negotiation should tell the
>>  client what
>>>>   can and can't be accepted, for other purposes we have docs.  I
>>  think its
>>>>   safe for the first go to just support json since for one of the
>>>>  main
>>  use cases,
>>>>   making it easy for people building websites with some ajax forms,
>>>>  that
>>  is all
>>>>   you need.  For more hard core REST I could easily see returning
>>>>  data
>>  very
>>>>   differently based on what is asked.  Like an endoint could serve 
> an
>>  image if you
>>>>   ask for png, but metadata on the image if you ask for json.
>>>>>   --
>>>>>   Bill Moseley
>>>>   _______________________________________________
>>>>   List:
>>>>   Listinfo:
>>>>   Searchable archive:
>>>>   Dev site:
>>>   _______________________________________________
>>>   List:
>>>   Listinfo:
>>>   Searchable archive:
>>>   Dev site:
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"
>>  *"*"*"* T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
>>  Handelsgericht Wien, FN 79340b
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"
>>  *"*"*"*
>>  Notice: This e-mail contains information that is confidential and may
>>  be privileged.
>>  If you are not the intended recipient, please notify the sender and
>>  then delete this e-mail immediately.
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"
>>  *"*"*"*
>>  _______________________________________________
>>  List:
>>  Listinfo:
>>  Searchable archive:
>>  Dev site:
> _______________________________________________
> List:
> Listinfo:
> Searchable archive:
> Dev site:
> This Email and any attachments contain confidential information and is 
> intended 
> solely for the individual to whom it is addressed. If this Email has been 
> misdirected, please notify the author as soon as possible. If you are not the 
> intended recipient you must not disclose, distribute, copy, print or rely on 
> any 
> of the information contained, and all copies must be deleted immediately. 
> Whilst 
> we take reasonable steps to try to identify any software viruses, any 
> attachments to this e-mail may nevertheless contain viruses, which our 
> anti-virus software has failed to identify. You should therefore carry out 
> your 
> own anti-virus checks before opening any documents. HomeLoan Partnership will 
> not accept any liability for damage caused by computer viruses emanating from 
> any attachment or other document supplied with this e-mail. HomeLoan 
> Partnership 
> reserves the right to monitor and archive all e-mail communications through 
> its 
> network. No representative or employee of HomeLoan Partnership has the 
> authority 
> to enter into any contract on behalf of HomeLoan Partnership by email. 
> HomeLoan 
> Partnership is a trading name of H L Partnership Limited, registered in 
> England 
> and Wales with Registration Number 5011722. Registered office: Pharos House, 
> 67 
> High Street, Worthing, West Sussex, BN11 1DN. H L Partnership Limited is 
> authorised and regulated by the Financial Conduct Authority.

Searchable archive:
Dev site:

Reply via email to