Re: [Catalyst] Problems with Encoding support in Catalyst.
On Sat, Jan 7, 2017 at 9:55 PM, Bill Moseley <mose...@hank.org> wrote:
>
> # Let it be set to undef
> if (my $wanted = shift) {
> $encoding = Encode::find_encoding($wanted)
> or Carp::croak( qq/Unknown encoding '$wanted'/ );
> *binmode(STDERR, ':encoding(' . $encoding->name . ')');*
>
>
> The problem with that is it seems to turn off autoflush on STDERR. In my
> case that is causing intermixing of lines in the log files.
>
> If I turn on STDERR->autoflush(1) then the logging works again and log
> lines are not mangled.
>
> But, I'm not sure we should apply a layer to STDERR like that. For
> example, I log to stderr with JSON and that is already encoded.
>
Just in case it isn't clear that is a problem:
use strict;
use warnings;
use Encode;
use JSON;
binmode( STDERR, ':utf8' );
my $json = encode_json( { chars => decode_utf8( '雨下降' ) } );
print STDERR "$json\n";
Results in:
{"chars":"é›¨ä¸‹é™ "}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Problems with Encoding support in Catalyst.
I just finally removed my custom encoding code to use the built-in code
Catalyst now provides to decode input data and encode output.
But, I came across two issues that I wanted to get some feedback on.
First one is simple. This code will die if $value is already decoded. Any
reason not to test it?
sub _handle_param_unicode_decoding {
my ( $self, $value, $check ) = @_;
return unless defined $value; # not in love with just ignoring undefs
- jnap
return $value if blessed($value); #don't decode when the value is an
object.
+return $value if Encode::is_utf8($value); # already decoded?
This next one is what I have questions about. This is also in Catalyst.pm
in the encoding() method:
# Let it be set to undef
if (my $wanted = shift) {
$encoding = Encode::find_encoding($wanted)
or Carp::croak( qq/Unknown encoding '$wanted'/ );
*binmode(STDERR, ':encoding(' . $encoding->name . ')');*
The problem with that is it seems to turn off autoflush on STDERR. In my
case that is causing intermixing of lines in the log files.
If I turn on STDERR->autoflush(1) then the logging works again and log
lines are not mangled.
But, I'm not sure we should apply a layer to STDERR like that. For
example, I log to stderr with JSON and that is already encoded.
Is there a compelling reason to binmode STDERR there?
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] From Development to Production.
On Wed, Mar 9, 2016 at 7:35 AM, Dermot <paik...@gmail.com> wrote: I want to caution you, in the nicest possible way. Writing software > requires a number of skills and a lot of research and learning. You can't > avoid the latter. What may seem like a lot of unnecessary aggravation > (testing and version control) have come about because it's no fun fixing > problems after the event. > Very true. Andrew, there's an overwhelming amount of reading on this topic. If you think there's *any* chance that your app will grow in usage and developers over time then it's well worth planning ahead. Automate *everything* from the start. It's critical to make deployments fast, repeatable, and dead simple. Probably should also include "scalable" there as a good buzzword. Here's a few random links to get you thinking: http://12factor.net/ https://zachholman.com/posts/deploying-software https://guides.github.com/introduction/flow/ (we do a modified version of this) The bottom line is you want to be spending your energy on the application functionality and not on deploying and running the application -- which seems to consume time exponentially if not careful. To be honest we have not really found a perfect way of managing Perl dependencies. I think the standard Perl module versioning system falls apart when there's lots of developers, many "apps" involved, and a version control system. Our single app turned into many Catalyst apps that (with the goal of) have "thin" controllers and many models. Those models were originally separate Dist::Zilla Perl distributions in our local DarkPAN. As those were updated with new version numbers the apps were also updated to depended on those. This breaks down when using Git and heavy use of branches because versions are no longer linear -- instead small features and fixes are merged individually. We have swung quite far the other way now and moving toward separate repos for each "app" and even copying the same module into multiple repos to promote decoupling (and the cost of code duplication). We are also using Carton to lock down versions and bundle dependencies. We have also put Perlbrew into Git with mixed results.I'm not a big fan of placing build artifacts into Git, but having the entire app in a single repo does have benefits. We would like to get to a CI pipeline that generates a Docker container and use that for all non-dev deployments. You never know when you will need to deploy a million containers <https://www.hashicorp.com/c1m.html>... -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Plugin attributes and Moose
I wanted to note something (before the week gets started and time vanishes)
about how Catalyst is behaving, and see if it's expected.
I asked about this on the Moose list, so sorry if you already saw this.
A Catalyst app extends Catalyst::Component, which has a BUILDARGS sub to
merge in class configuration when creating instances of components.
This is how configuration can set initial values on attributes in Models,
Views, Controllers, and the application class itself.
If a (non-role) Moose-based plugin is loaded (which is common) it's added
to the app's inheritance like this:
$meta->superclasses($plugin, $meta->superclasses);
Which can be thought of like this:
@App::ISA = ( qw/ Catalyst::Plugin::Foo Catalyst / );
The result then is that BUILDARGS in Catalyst::Component is no longer
called, and then attributes in the App class are no longer populated from
the config.
So, the behavior changes depending on what plugins are brought in.
Another odd issue I came against is that MooseX::Emulate::Class::Accessor::Fast
causes odd behavior of Moose attributes. This role is widely used in
Catalyst.
In the code below note how the "foo" attribute has init_arg => undef to
prevent it from being initialized. With the the role not only is it
initialized, but with a value that isn't an "Int".
package Foo;
use Moose;
with 'MooseX::Emulate::Class::Accessor::Fast';
has foo => (
is => 'ro',
isa => 'Int', # for error
init_arg => undef,
);
package main;
use strict;
use warnings;
my $foo = Foo->new( { foo => 'bar' } );
use Data::Dumper;
print Dumper $foo->foo;
Generates 'bar', which is not an Int.
$VAR1 = 'bar';
Comment out "with 'MooseX::Emulate::Class::Accessor::Fast';" and it behaves
as expected. Also comment out "init_arg" and Moose will complain about
the string not begin an Int.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Url Encoded UTF8 parameters
BTW -- I wonder about the Catalyst behavior here.
On Sat, Aug 1, 2015 at 10:36 PM, Bill Moseley mose...@hank.org wrote:
On Sat, Aug 1, 2015 at 6:31 AM, Stefan maill...@s.profanter.me wrote:
Hi,
if a URL parameter contains a Unicode character (e.g.
www.example.com/?param=%D6lso%DF which stands for param=Ölsoße), the
parameter is not correctly parsed as Unicode.
One note here -- data over the wire must be encoded into octets. So, all
Unicode characters must be encoded and then decoded when received. (You
can't send Unicode characters.) UTF-8 is used now (for obvious
reasons). http://tools.ietf.org/html/rfc3986.
You are specifying %D6 -- although the Unicode characters is U+00D6, the
UTF-8 octet sequence is 0xC3 0x96. See:
http://www.fileformat.info/info/unicode/char/00D6/index.htm
Unless otherwise instructed, Catalyst uses UTF-8
https://github.com/perl-catalyst/catalyst-runtime/blob/master/lib/Catalyst/Engine.pm#L579
as the encoding for decoding query parameters -- query parameters are
decoded from UTF-8 octets to Perl characters.
As your example showed, if you use invalid UTF-8 sequences then
Encode::decode() as used by Catalyst will replace those with the U+FFFD
substitution character
http://www.fileformat.info/info/unicode/char/fffd/index.htm �.
This may or may not be what you want. Personally, I think it's not
correct to silently modify user input. You intended to pass Ölsoße but
ended up with �lso�e -- is that really the data you would want to
process/store for the request? Seems unlikely.
If param is suppose to be passed as textual, UTF-8-encoded octets, and it
isn't, then maybe returning a 400 is a better way of handling that. That
probably would have helped you see what is wrong in this case.
i.e. use eval { decode( $default_query_encoding, $str, FB_CROAK |
LEAVE_SRC ); } to catch invalid data and return to the client the $str
that failed and why.
Of course, it is also possible that you have some query parameters that you
want decoded as UTF-8 and some that might represent something else (a raw
sequence of bytes), and want more manual control. In that case
$c-config-{do_not_decode_query} could be used to bypass the decoding.
But then, you must manually decode() yourself.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Configure psgi.input as optional?
I created a pull for checking the return for the psgi.input file:
https://github.com/perl-catalyst/catalyst-runtime/pull/100
I created a ticket for HTTP::Body:
https://rt.cpan.org/Ticket/Display.html?id=105021
I'll create a separate pull for a config setting to make creating the
psgi.input file optional when I have a bit of time.
On Fri, Jun 5, 2015 at 4:38 PM, Lasse Makholm la...@unity3d.com wrote:
On Fri, Jun 5, 2015 at 8:26 PM, Bill Moseley mose...@hank.org wrote:
Hi,
Our app handles a lot of uploads, often quite large uploads.As we
know, uploads (and any non-parsed body) gets written to temp files via
HTTP::Body.
Now, Catalyst::Request also writes every body to a temp file (via
Stream::Buffered). So, depending on the body size, can take up to twice
the space in temp files as the content length.
If we are not going to use psgi.input and just use Catalyst's
(HTTP::Body's) temp files could we make the creation of psgi.input optional?
I have a few other concerns about the current code.
prepare_body in Catalyst::Request does not check the return code when it
writes to the psgi.input file. HTTP::Body also fails to check return
codes when it writes. This means if the partition where temp files are
created fills then Catalyst request will ignore this and continue as if
there's no problem. I trust the risk here is recognized.
I think the fix in Catalyst::Request is pretty simple by checking return
values:
# Check for definedness as you could read '0'
while ( defined ( my $chunk = $self-read() ) ) {
$self-prepare_body_chunk($chunk);
next unless $stream_buffer;
$stream_buffer-print($chunk)
|| die sprintf Failed to write %d bytes to psgi.input file:
$!, length( $chunk );
}
HTTP::Body needs similar changes.
On a related note, things like NFS mounted file systems tend to not
guarantee that data is safely written to disk until all write() calls AND
the final close() have completed successfully.
Code that checks that write() succeeds but then fails to do the same for
close() is still broken.
Thanks for looking into this. Our app handles lots of large uploads too
and I would love to see this fixed in HTTP::Body.
/L
See any problem with making the psgi.input file optional? And any
reason not to throw an exception when writing to the temp file fails?
BTW -- Plack::Handler::Apache2 sets psgi.input to the Apache request
object $r -- it would be handy to preserve this object for later use.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Configure psgi.input as optional?
Hi,
Our app handles a lot of uploads, often quite large uploads.As we know,
uploads (and any non-parsed body) gets written to temp files via HTTP::Body.
Now, Catalyst::Request also writes every body to a temp file (via
Stream::Buffered). So, depending on the body size, can take up to twice
the space in temp files as the content length.
If we are not going to use psgi.input and just use Catalyst's
(HTTP::Body's) temp files could we make the creation of psgi.input optional?
I have a few other concerns about the current code.
prepare_body in Catalyst::Request does not check the return code when it
writes to the psgi.input file. HTTP::Body also fails to check return
codes when it writes. This means if the partition where temp files are
created fills then Catalyst request will ignore this and continue as if
there's no problem. I trust the risk here is recognized.
I think the fix in Catalyst::Request is pretty simple by checking return
values:
# Check for definedness as you could read '0'
while ( defined ( my $chunk = $self-read() ) ) {
$self-prepare_body_chunk($chunk);
next unless $stream_buffer;
$stream_buffer-print($chunk)
|| die sprintf Failed to write %d bytes to psgi.input file:
$!, length( $chunk );
}
HTTP::Body needs similar changes.
See any problem with making the psgi.input file optional? And any reason
not to throw an exception when writing to the temp file fails?
BTW -- Plack::Handler::Apache2 sets psgi.input to the Apache request object
$r -- it would be handy to preserve this object for later use.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Catalyst::Request::Upload-filename is not decoded.
All my upload forms have accept-charset=utf-8.We expect that uploaded
filenames could have wide-characters.
The problem I hit was -basename does this:
$ perl -le 'use Catalyst::Request::Upload; my $upload =
Catalyst::Request::Upload-new( { filename = q[документ обучения.pdf] } );
print $upload-basename;'
_.pdf
That's pretty mangled.
The problem is that $upload-filename is not decoded so the substitution is
working on octets not characters.
sub _build_basename {
my $self = shift;
my $basename = $self-filename;
$basename =~ s|\\|/|g;
$basename = ( File::Spec::Unix-splitpath($basename) )[2];
$basename =~ s|[^\w\.-]+|_|g;
return $basename;
}
Obviously, we want \w to work on characters, not encoded octets. Decoding
the filename should be done -- it's character data.
Does it make sense to do it in Engine's prepare_uploads?
For example:
my $u = Catalyst::Request::Upload-new(
size = $upload-{size},
type = scalar $headers-content_type,
headers = $headers,
tempname = $upload-{tempname},
filename =
*$c-_handle_unicode_decoding($upload-{filename})*,
);
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst Unicode
On Fri, Jan 31, 2014 at 3:58 AM, Will Crawford billcrawford1...@gmail.comwrote: If the string has been decoded *from* UTF-8 to Perl's internal representation, it's *not* going to be marked as UTF8 internally; it *shouldn't* be. It's no longer a UTF8 string but a Unicode string, complete with wide characters. If anything, the internal UTF8 flag means this string needs decoding rather than has been decoded. $ perl -le 'use Encode; my $chars = decode_utf8( bytes ); print Encode::is_utf8( $chars ) ? Is flagged utf8\n : not flagged\n; use Devel::Peek; Dump($chars)' Is flagged utf8 SV = PV(0x7fb8c10023f0) at 0x7fb8c102b6a8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK,UTF8) PV = 0x7fb8c0e01170 bytes\0 [UTF8 bytes] CUR = 5 LEN = 16 Everything is encoded. The flag tells Perl that its internal representation is encoded as utf8 so knows to work with it as utf8 characters (e.g. length() is length of chars, matching works on chars, etc.) $ perl -le 'use Encode; my $chars = decode( 'latin1', bytes ); print Encode::is_utf8( $chars ) ? Is flagged utf8\n : not flagged\n; use Devel::Peek; Dump($chars)' Is flagged utf8 -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Implementing Webhooks.
I'm running Catalyst under mod_perl2 (currently, subject to change). Some requests trigger a callback to a user-provided URL -- a webhook. Obviously, it's best to do those asynchronously and not in-process. Don't want a web request (or really an Apache process) hanging while waiting on an external web server to respond. A queue is probably the best approach, but there's also some advantage of having the Catalyst app make the webhook request -- specifically because the Catalyst app has the context of the request and has application logging available. How would you implement this? How would that change if you wanted more than just fire-and-forget? For example, if you wanted to provide some kind of retry interval for failed callbacks. The external server might be temporarily down for maintenance. Thanks, -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Changing format of date field
On Tue, Jan 7, 2014 at 1:41 AM, Adam Witney awit...@sgul.ac.uk wrote:
[% user_time( foo.event_start, '_DT_TIMESTAMP_WITH_ZONE' ) | html %]
Hi Bill,
Thanks for your email. I agree that the template seems like a good place
for this, but to give a little extra detail.
I have a form that is used for both object create and editing. When I edit
an object I have this in my controller:
$c-stash(formdata = $object);
and in the template:
input name=datefield type=text value=[% formdata.datefield.dmy('/')
%] /
But when creating a new object, if the form validation fails I pass the
form data back to the form like so:
$c-stash( formdata = $c-request-params );
Have you looked at HTML::FormHandler? I would assume it addresses this.
I use something similar and the concept is that you have a field object
that knows how to render either from the existing object or from user
provided input.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Changing format of date field
On Mon, Jan 6, 2014 at 7:18 AM, Adam Witney awit...@sgul.ac.uk wrote:
Hi,
I have a date field in a DBIx::Class Result class using
InflateColumn::DateTime. When I pass this through a Catalyst/TT
application, the date is presented in -MM-DD format.
I can modify this to DD/MM/ in my TT template with object.dmy('/'),
but I would prefer to change the default DateTime stringification, however
I am not quite sure where or how to do this?
Any ideas greatly appreciated.
Seem like formatting in the template is the right place -- may want to have
different formats in different places in your app. Then you might also
think about how best to localize.
One idea is to localize a set of names for a set of formats:
[% dt_fmt = c.localize( '_DT_DATE_ONLY' ); foo.some_dt_object.strftime(
dt_fmt ) | html %]
I have also used a function that does the above, but also clones and sets
the time zone and locale based on the user's preferences:
[% user_time( foo.event_start, '_DT_TIMESTAMP_WITH_ZONE' ) | html %]
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Anyone have experience with locales?
I'm using the Catalyst plugin to support utf8 encoded input and output, I have my database driver set to read in utf8, and I have Template Toolkit configured for utf8 templates. I want to use available locale to render numbers. I'm using Number::Format in a number of places and it works with POSIX's setlocale() function. What I'm not clear on is if setlocale() plays nicely with everything above. Anyone here have experience with this? http://perldoc.perl.org/perllocale.html#Unicode-and-UTF-8 says: It is strongly recommended that when combining Unicode and locale (starting in v5.16), you use 1. use http://perldoc.perl.org/functions/use.html locale ':not_characters'; When this form of the pragma is used, only the non-character portions of locales are used by Perl, for exampleLC_NUMERIC . Perl assumes that you have translated all the characters it is to operate on into Unicode (actually the platform's native character set (ASCII or EBCDIC) plus Unicode). For data in files, this can conveniently be done by also specifying... For one thing I'm not using 5.16 yet - and I'm not clear what means for pre 5.16. Should I only use setlocale( LC_NUMERIC, $locale ), for example? That is, using can setlocale( LC_ALL, $locale ) cause problems with my existing character handling? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Bump Catalyst::Model::Adaptor
I'm a big fan of Catalyst::Model::Adaptor.It helps maintain good separation of concerns and code resue. But, there's an annoying issue where it sends its entire config to the model's constructor instead of just what is in args. Some code will complain when it finds unknown constructor arguments. For example I'm seeing lots of these: Unrecognised options: args catalyst_component_name class at /var/lib/jenkins/perl5/perlbrew/perls/jenkins/lib/site_perl/5.14.4/Catalyst/Model/Adaptor/Base.pm line 27. In this case it is Cache::Memcached::libmemcached complaining. I think there's a suggested patch in one of these. Is there someone that can review and maybe push out a new version? https://rt.cpan.org/Public/Bug/Display.html?id=67078 https://rt.cpan.org/Public/Bug/Display.html?id=78663 -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Bump Catalyst::Model::Adaptor
On Sun, Dec 15, 2013 at 4:49 PM, neil.lunn n...@mylunn.id.au wrote:
Isn't this just a case of adding to your model class:
Yes, that's good. Thanks.
sub mangle_arguments {
my ( $self, $args ) = @_;
return $args-{args};
}
Thus overriding the default with what you want to do for your constructor.
This clears the error for me.
In this case it is Cache::Memcached::libmemcached complaining.
I think there's a suggested patch in one of these. Is there someone
that can review and maybe push out a new version?
https://rt.cpan.org/Public/Bug/Display.html?id=67078
https://rt.cpan.org/Public/Bug/Display.html?id=78663
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
http://www.avast.com/
This email is free from viruses and malware because avast!
Antivirushttp://www.avast.com/protection is active.
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Bump Catalyst::Model::Adaptor
On Sun, Dec 15, 2013 at 6:09 PM, neil.lunn n...@mylunn.id.au wrote:
But I prefer the mangle_arguments particulary for classes that do not use
a Moose like contructor.
The change from the old code seems to be deliberate in passing in extra
arguments to the model constructor, so it seems best to override these
hooks where needed rather than patch the base.
It's a bit confusing, though. The docs for args say this:
This is a hashref of arguments to pass to the constructor of class. It is
optional, of course. If you omit it, nothing is passed to the constructor
(as opposed to {}, an empty hashref).
But maybe there was a use case where having the extra config (in addition
to config in args) passed to the constructor was necessary.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Content Disposition filename
On Tue, Nov 19, 2013 at 10:32 AM, Bill Moseley mose...@hank.org wrote:
Anyone aware of a good, portable way in Perl to encode the filename in a
Content-Disposition header? I would like to support UTF8 filenames, but
support in browsers is unclear (if not changing).
Is this complexity something that the Catalyst framework should handle?
It's one of those areas where it's easy to get wrong (I can see many
different approaches in our own code).
http://greenbytes.de/tech/tc2231/
http://stackoverflow.com/questions/93551/how-to-encode-the-filename-parameter-of-content-disposition-header-in-http
I have no idea what the client can accept or what its OS uses as a
path-separator, and I don't want to go down the client-sniffing path,
anyway.
I have a user-supplied character string that I want to use as the filename,
which I have to assume can contain any unicode character since it's
user-supplied data.
From my limited tests it seems most modern browsers are supporting the
filename* extension. Each browser does some special handling (like
replacing the path-separator, or adding a file extension based on
content-type if no file extension is in the filename).
All I want to do is make valid HTTP headers and let the client decide how
to handle it, but also provide a usable filename (not just underscores, for
example).
So, all I'm after is to make this valid markup:
$c-res-header( content_disposition =
qq[attachment; filename=$ascii_file; filename*=UTF-8''$utf8_file]
);
The filename* is easy, I'm finding:
my $utf8_file = uri_escape( Encode::encode( 'UTF-8' = $filename ) );
But the $ascii_file is a bit more work. Percent-encoding doesn't work.
So, have to do a bit of filtering.
See any easier/cleaner/more-correct approach? When I see this much code I
tend to think it's the wrong approach.
# Convert to ASCII using underscore as replacement
my $ascii_file = Encode::encode( ascii = $filename, sub { '_' } );
# Remove quotes as we want to use quoted form of filename and preserve
whitespace.
$ascii_file =~ s//_/g;
# Replace non-printable characters with underscore, and collapse dups
$ascii_file =~ s/[^[:print:]]/_/g;
$ascii_file =~ s/_{2,}/_/g;
# Split off the extension so can check length of filename w/o extension.
# Of course, $ext could end up as dot + underscore.
my ( $base, $ext ) = split /(\.\w+)$/, $ascii_file;
# Use default filename if we don't have more than three meaningful
characters.
# very subjective.
$base = 'your_file' unless ( () = $base =~ /[A-Za-z0-9]/g ) 3;
# Stuff the extension back on.
$ascii_file = $base;
$ascii_file .= $ext if defined $ext;
Again, filename* support is good, and I'm not trying to prevent buggy
clients from doing something stupid (e.g. filename=/etc/passwd), but want
to provide a reasonable fallback to filename.
Perhaps the simple solution is to always use filename=your_file and hope
most clients use the filename* extension.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Setting file handle as the response body generates warnings.
On Wed, Nov 20, 2013 at 11:32 PM, neil.lunn n...@mylunn.id.au wrote: This approach may work for you is the compressed data is actually in a scalar and not too large. And not too small. YMMV. my $z = read_file product.json.gz; my $io = IO::Scalar-new( \$z ); $io-seek( -4, 2 ); $io-read( my $buf, 4); my $uncompressed_size = unpack( 'V', $buf ); This indeed does work in my tests. Thanks for all the help, Neil. I really appreciate the time you spent on this. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Setting file handle as the response body generates warnings.
On Wed, Nov 20, 2013 at 4:08 AM, neil.lunn n...@mylunn.id.au wrote: my $length = $body-getHeaderInfo Well, that's helpful. Thanks. Completely missed that in the docs. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Setting file handle as the response body generates warnings.
On Wed, Nov 20, 2013 at 7:37 AM, Bill Moseley mose...@hank.org wrote:
On Wed, Nov 20, 2013 at 4:08 AM, neil.lunn n...@mylunn.id.au wrote:
my $length = $body-getHeaderInfo
Well, that's helpful. Thanks. Completely missed that in the docs.
Well, except for the actual files I have no ISIZE:
$VAR1 = {
'Time' = 0,
'Flags' = 0,
'TextFlag' = 0,
'MethodID' = 8,
'ExtraField' = [],
'CommentFlag' = 0,
'Type' = 'rfc1952',
'NameFlag' = 0,
'ExtraFlags' = 0,
'HeaderCRC' = undef,
'isMinimalHeader' = 0,
'MethodName' = 'Deflated',
'ExtraFlag' = 0,
'HeaderLength' = 10,
'ExtraFieldRaw' = undef,
'Comment' = undef,
'OsName' = 'Unix',
'FingerprintLength' = 2,
'HeaderCRCFlag' = 0,
'OsID' = 3,
'TrailerLength' = 8,
'Name' = undef,
'Header' = ''
};
And even if I gzip a file (e.g. $ gzip foo.txt) then the resulting
foo.txt.gz has 'ISIZE' = 0,
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Setting file handle as the response body generates warnings.
On Wed, Nov 20, 2013 at 3:45 PM, neil.lunn n...@mylunn.id.au wrote:
2.3.1.2. Compliance
A compliant compressor must produce files with correct ID1,
ID2, CM, CRC32, and ISIZE, but may set all the other fields in
the fixed-length part of the header to default values (255 for
OS, 0 for all others). The compressor must set all reserved
bits to zero.
/snip
Seems noncompliance may be rampant.
Anyway, sounds like Catalyst isn't quite supporting this kind of file
handle as expected. John, is there anything you would want me to try?
$ perl -MIO::Uncompress::Gunzip -le 'use Data::Dumper; print Dumper
+IO::Uncompress::Gunzip-new( Catalyst-Runtime-5.90051.tar.gz
)-getHeaderInfo'
$VAR1 = {
'Time' = 1383843952,
'Flags' = 8,
'TextFlag' = 0,
'MethodID' = 8,
'ExtraField' = [],
'CommentFlag' = 0,
'Type' = 'rfc1952',
'NameFlag' = 1,
'ExtraFlags' = 2,
'HeaderCRC' = undef,
'isMinimalHeader' = 0,
'MethodName' = 'Deflated',
'ExtraFlag' = 0,
'HeaderLength' = 39,
'ExtraFieldRaw' = undef,
'Comment' = undef,
'OsName' = 'Unix',
'FingerprintLength' = 2,
'HeaderCRCFlag' = 0,
'OsID' = 3,
'TrailerLength' = 8,
'Name' = 'Catalyst-Runtime-5.90051.tar',
'Header' = p�{RCatalyst-Runtime-5.90051.tar'
};
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Content Disposition filename
Anyone aware of a good, portable way in Perl to encode the filename in a Content-Disposition header? I would like to support UTF8 filenames, but support in browsers is unclear (if not changing). Is this complexity something that the Catalyst framework should handle? It's one of those areas where it's easy to get wrong (I can see many different approaches in our own code). http://greenbytes.de/tech/tc2231/ http://stackoverflow.com/questions/93551/how-to-encode-the-filename-parameter-of-content-disposition-header-in-http Thanks, -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Setting file handle as the response body generates warnings.
Speaking of downloads, I have a gzipped file. Well, it's a file in memory. If I see Accept-Encoding that includes gzip it's easy. I simply return the content and set Content-Encoding: gzip and set the content length. But, if the client does not accept gzip I uncompress it using IO::Uncompress::Gunzip. IO::Uncompress::Gunzip looks like a IO::File handle. So, I can do: $c-res-body( IO::Uncompress::Gunzip-new( \$gzipped_data ) ); Apache will add the header Transfer-Encoding: chunked because there's no Content-Length header. This works, but Catalyst in finalize_headers issues two warnings: -s on unopened filehandle GEN10 at /home/bill/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst.pm line 1893. [warn] Serving filehandle without a content-length Are those warning a problem with how Catalyst is handling this, or something wrong with how IO::Uncompress::Gunzip is working? The uncompressed file could be quite large, which is why I'd prefer to not uncompress it in memory. I suppose I could uncompress to /tmp and then serve the file from there. Of course, not using Catalyst to serve large files is perhaps another solution. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Setting file handle as the response body generates warnings.
On Tue, Nov 19, 2013 at 6:34 PM, John Napiorkowski jjn1...@yahoo.comwrote:
This looks familiar, but I thought we fixed this... what version of
Catalyst are you using here?
I just updated:
$ perl -MCatalyst -le 'print Catalyst-VERSION'
5.90051
I can debug more in the morning -- anything specific I should look at?
It's pretty easy to reproduce:
package Zip::Controller::Root;
...
use IO::Uncompress::Gunzip;
use IO::Compress::Gzip qw/ gzip $GzipError /;
sub uncompress : Local {
my ( $self, $c ) = @_;
my $uncompressed = This is some text that can be compressed.\n x 5;
my $compressed;
gzip( \$uncompressed, \$compressed ) || die $GzipError;
$c-res-body( IO::Uncompress::Gunzip-new( \$compressed ) );
return;
}
$ script/zip_test.pl /uncompress
-s on unopened filehandle GEN2 at
/home/bill/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst.pm
line 1948, DATA line 1000.
[warn] Serving filehandle without a content-length
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.
This is some text that can be compressed.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Have exceeded the maximum number of attempts (1000) to open temp file/dir
On Mon, Nov 4, 2013 at 4:20 PM, neil.lunn n...@mylunn.id.au wrote:
I'd be thinking along the lines of mod_perl is evil. From a quick google
of mod_perl srand there seem to be some similar cases. And a where to
call srand in this post:
http://blogs.perl.org/users/brian_phillips/2010/06/when-rand-isnt-random.html
Give it a try.
Thanks. That indeed was part of the problem. Other part was a leak.
First, our code to manage database replication was causing a leak of $c.
That's why I could not replicate in dev where I don't normally run with a
replicated database.
The result was that a temp file would be created but not destroyed at the
end of the request. It would be destroyed at the start of the *next*
request.
So, the temp files would be deleted, but just not at the right time. The
one exception was the last request -- when Apache killed off a child
process then that final temp file would be left behind.
Here's where mod_perl comes in.
If I understand Perl correctly, the first time rand() is called a new seed
is generated. So, if you fork a bunch of children and each calls rand()
each will get a new seed. But, if you explicitly call srand() once in
the parent before forking then all the children get the same seed, and all
the children will generate the same random sequence.
When using Starman or even the forking test server this is not a problem as
I believe they both call srand() for each child. With Apache the children
end up with the same seed which likely means the parent process called
rand() or srand().
Over time, /tmp would fill, and fill with a pre-defined set of filenames
due to the common seed eventually the first 1000 names would get used up.
My fix is simply this in the app base class:
my $srand;
before handle_request = sub { srand() unless $srand++ };
The question if this is something Catalyst should handle.
As for the leak, we use the same replication code in different apps -- so
need a way for the specific app to work with this.
The broken code was in an ACCEPT_CONTEXT that included:
$replication_object-callback( sub { $self-callback_method( $c ) } );
I'm not thrilled by $c getting passed here, but the callback need quite a
bit from $c.
The fix that seems to work is simply this:
weaken( $c );
$replication_object-callback( sub { $self-callback_method( $c ) } );
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Have exceeded the maximum number of attempts (1000) to open temp file/dir
On Fri, Oct 25, 2013 at 6:51 AM, Bill Moseley mose...@hank.org wrote:
[ERROR] Caught exception in engine Error in tempfile() using
/tmp/XX: Have exceeded the maximum number of attempts (1000) to
open temp file/dir
I don't really see how this can be a Catalyst issue, but I can't reproduce
it outside of Catalyst -- and outside of our production environment.
Can anyone think of anything else that might be going on here?
The template has 10 X that are replaced by I think 63 random ascii
characters. 63^10 is a huge number of random strings. File::Temp only
loops when sysopen returns EEXISTS -- that is, when sysopen fails AND the
error is that the file already exists.
Sure, there's 50 web processes but the odds of them all being in exact
lock-step with calling rand() is unlikely. And even if they started out
that way if two processes opened the exact same name at the same time one
process would just try the next random name and be done.
I have something like 26K files in /tmp, so nothing compared to 63^10.
And each web server is only seeing about 10 request/sec.
It's just not making sense.
Again, I'm unable to replicate the problem with a simple test script that
is designed to clash.
I fork 50 (or more) child processes to replicate the web server processes
and then in each one I do this:
# Wait until top of the second so each child procss starts about
the same time.
my $t = time(); # Time::HiRes
sleep( int( $t ) + 1 - $t );
for ( 1 .. 500 ) {
my $fh = File::Temp-new(
TEMPLATE = 'bill_X',
DIR = '/tmp',
);
}
And never see any contention.
The File::Temp docs say:
If you are forking many processes in parallel that are all creating
temporary files, you may need to reset the random number seed using
srand(EXPR) in each child else all the children will attempt to walk
through the same set of random file names and may well cause
themselves to give up if they exceed the number of retry attempts.
We are running under mod_perl. Could it be as simple as the procs all
were in sync? I'm just surprised this has not happened before. Is there
another explanation?
Where would you suggest to call srand()?
Another problem, and one I've
commentedhttps://rt.cpan.org/Public/Bug/Display.html?id=84004on before, is
that HTTP::Body doesn't use File::Temp's unlink feature and
depends on Catalyst cleaning up. This results in orphaned files left on
temp disk.
--
Bill Moseley
mose...@hank.org
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Response traits.
On Fri, Nov 1, 2013 at 8:31 AM, John Napiorkowski jjn1...@yahoo.com wrote: I think a patch that made sure strings being set to location conformed to the expected standard would be very welcomed! on the other hand I thought this was also caught by https://metacpan.org/pod/Plack::Middleware::Lint Yes, you might be right about that. That's probably the right (final) place to catch this. From the framework point of view I'm not quite clear what $res-redirect should accept. That is, if it contains \n should it get precent-encoded? Maybe it should (have) only accept a URI object? And only a fully-qualified URI.And what if redirect is passed wide characters? Maybe it's legit to pass wide characters encoded in some format. as well? In any case Catalyst response could reject any attempts to set -location with incorrect values. The only thing is that people could still probably get around it by setting headers directly. I think for now we'll say if you do that we have to assume you know what you are doing! One would hope. Experience often shows that manually doing it often is an indicator of not knowing what one is doing. The framework's benefit is so you don't screw things up. John On Thursday, October 31, 2013 7:03 PM, Bill Moseley mose...@hank.org wrote: On Thu, Oct 31, 2013 at 2:34 PM, John Napiorkowski jjn1...@yahoo.comwrote: I'm currently recommending people take advantage of native PSGI support in the newer Catalyst and use Middleware for when you need to munge and or alter the response (if its being done globally). The interface is more straightforward. Do you think that Catalyst::Response should validate the location provided to redirect()? The issue that came up was a newline was ending up in the location provided which resulted in a split responsehttp://en.wikipedia.org/wiki/HTTP_response_splitting. I was thinking of doing something like: $self-location( URI-new( $location )-as_string ); But with perhaps a bit more error handling. johnn On Thursday, October 31, 2013 11:33 AM, Bill Moseley mose...@hank.org wrote: On Thu, Oct 31, 2013 at 12:51 AM, Aristotle Pagaltzis pagalt...@gmx.dewrote: CatalystX::RoleApplicator Thanks. That was what I was looking for. Just missed it when looking. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Response traits.
On Thu, Oct 31, 2013 at 12:51 AM, Aristotle Pagaltzis pagalt...@gmx.dewrote: CatalystX::RoleApplicator Thanks. That was what I was looking for. Just missed it when looking. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Have exceeded the maximum number of attempts (1000) to open temp file/dir
On Thu, Oct 31, 2013 at 2:44 PM, John Napiorkowski jjn1...@yahoo.comwrote:
am calling -cleanup(1) when we create the HTTP::Body. is that not enough
to cleanup tmp files ?
I haven't look at this in a while, but I think it's described here:
https://rt.cpan.org/Public/Bug/Display.html?id=84004
HTTP::Body assumes $self-{upload} exists before deleting, and that might
not be created yet.
I have my own version for handling 'multipart/form-data' that sets UNLINK
= 1.
Now, the application/octet-stream handling is another issue. There
HTTP::Body uses the default File::Temp (e.g. UNLINK = 1), but I'm still
finding a large number of those files left around.
In my dev environment I have not been able to make it leave files on /tmp.
On production I can run watch 'ls /tmp | wc -l' and see the counts
increase and decrease so I know files are being deleted, but every once in
a while a file gets left behind. I don't see segfaults in the logs, and
I've tested with Apache's MaxRequestPerChild low (so recycling child
processes often) and not seeing that leave files behind.
I'm going to update our copy of HTTP::Body and put the process ID in the
temp file template to essentially namespace and use cron to keep /tmp
cleaner. But, I still have yet to figure out why those are left behind.
With UNLINK = 1 they should not be left there. File::Temp doesn't appear
to check the return value from unlink.
They come and go but some stick around:
$ for i in $(seq 10); do ls /tmp | wc -l; sleep 2; done
23861
23865
23863
23864
23862
23862
23865
23865
23864
23866
$ ls -lt /tmp | head -2
total 95492
-rw--- 1 tii-rest tii-rest 14 Oct 31 16:40 Nudjp9WDNy
$ ls -lt /tmp | tail -2
-rw--- 1 tii-rest tii-rest 16 Oct 28 13:36 NWwxOhwhRW
-rw--- 1 tii-rest tii-rest 16 Oct 28 13:35 Ll1Ze0TNPL
regarding the tmp file thing, wow I have no idea, but I hope you find out
and report it to us!
Johnn
On Friday, October 25, 2013 8:53 AM, Bill Moseley mose...@hank.org
wrote:
I have an API where requests can include JSON. HTTP::Body saves those
off to temp files.
Yesterday got a very large number of errors:
[ERROR] Caught exception in engine Error in tempfile() using
/tmp/XX: Have exceeded the maximum number of attempts (1000) to
open temp file/dir
The File::Temp docs say:
If you are forking many processes in parallel that are all creating
temporary files, you may need to reset the random number seed using
srand(EXPR) in each child else all the children will attempt to walk
through the same set of random file names and may well cause
themselves to give up if they exceed the number of retry attempts.
We are running under mod_perl. Could it be as simple as the procs all
were in sync? I'm just surprised this has not happened before. Is there
another explanation?
Where would you suggest to call srand()?
Another problem, and one I've
commentedhttps://rt.cpan.org/Public/Bug/Display.html?id=84004on before, is
that HTTP::Body doesn't use File::Temp's unlink feature and
depends on Catalyst cleaning up. This results in orphaned files left on
temp disk.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Response traits.
On Thu, Oct 31, 2013 at 2:34 PM, John Napiorkowski jjn1...@yahoo.comwrote: I'm currently recommending people take advantage of native PSGI support in the newer Catalyst and use Middleware for when you need to munge and or alter the response (if its being done globally). The interface is more straightforward. Do you think that Catalyst::Response should validate the location provided to redirect()? The issue that came up was a newline was ending up in the location provided which resulted in a split responsehttp://en.wikipedia.org/wiki/HTTP_response_splitting. I was thinking of doing something like: $self-location( URI-new( $location )-as_string ); But with perhaps a bit more error handling. johnn On Thursday, October 31, 2013 11:33 AM, Bill Moseley mose...@hank.org wrote: On Thu, Oct 31, 2013 at 12:51 AM, Aristotle Pagaltzis pagalt...@gmx.dewrote: CatalystX::RoleApplicator Thanks. That was what I was looking for. Just missed it when looking. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Response traits.
What is the recommended way to apply a Response trait? Include it in a
response subclass similar to how Catalyst::Action::REST injects
Catalyst::Request::REST?
http://cpansearch.perl.org/src/JJNAPIORK/Catalyst-Action-REST-1.12/lib/Catalyst/Action/SerializeBase.pm
What I was interested in is wrapping redirect and filtering out any
white-space to prevent response splitting.
sub redirect {
my $self = shift;
if (@_) {
my $location = shift;
my $status = shift || 302;
$self-location($location);
$self-status($status);
}
return $self-location;
}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Have exceeded the maximum number of attempts (1000) to open temp file/dir
I have an API where requests can include JSON. HTTP::Body saves those off to temp files. Yesterday got a very large number of errors: [ERROR] Caught exception in engine Error in tempfile() using /tmp/XX: Have exceeded the maximum number of attempts (1000) to open temp file/dir The File::Temp docs say: If you are forking many processes in parallel that are all creating temporary files, you may need to reset the random number seed using srand(EXPR) in each child else all the children will attempt to walk through the same set of random file names and may well cause themselves to give up if they exceed the number of retry attempts. We are running under mod_perl. Could it be as simple as the procs all were in sync? I'm just surprised this has not happened before. Is there another explanation? Where would you suggest to call srand()? Another problem, and one I've commentedhttps://rt.cpan.org/Public/Bug/Display.html?id=84004on before, is that HTTP::Body doesn't use File::Temp's unlink feature and depends on Catalyst cleaning up. This results in orphaned files left on temp disk. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst::Request path method as a setter
On Mon, Oct 21, 2013 at 9:13 AM, John Napiorkowski jjn1...@yahoo.comwrote:
My guess here is that path should be RO and that if you need to write code
that messed with the path, that should happen in plack middleware.
Although I do want to let the app know what was removed from the path.
Essentially, (for better or worse) I have existing chained action
/account/123/foo/455 and want to add *optional* path prefixes. So, that
means accepting /api/v4/account/123/foo/455. So, the idea is simply pull
off the /api/v4/ from the path and stick it on $req-base base leaving the
original dispatching in place.
The version v4 is to version specific end points. Something we don't
want to do very often. The version would be available in the stash so
that the endpoint can adjust its behavior.
I'm not thrilled by any of that, but that's the solution I'm looking at
right now.
I'll ask around IRC to try and find out why this was allowed in the first
place. My guess it that we wanted to allow people to change the path for
doing complex path rewriting.
Thanks. Just seemed a bit odd that setting $req-path to its current
value would change $req-uri dramatically.
On Monday, October 21, 2013 10:00 AM, Bill Moseley mose...@hank.org
wrote:
=head2 $req-path
Returns the path, i.e. the part of the URI after $req-base, for the
current request.
Pasted below is Catalyst::Request's path method. Note from the final
else block that $req-path returns the request uri's path ($req-uri-path)
with the $req-base-path *removed* as the documentation says.
So, if the request URI and base URI are these:
http://localhost/myapp/path/to/action # $req-uri
http://localhost/myapp/ # $req-base
then $req-path is:
path/to/action
Using the example above, and looking at what $req-path( ) does as a
setter:
$req-path( $req-path );
would result in a new request URI of:
http://localhost/path/to/action.
The path method doesn't document what it does as a setter, but this
behavior looks broken because it alters the request URI's path.
What do you think?
sub path {
my ( $self, @params ) = @_;
if (@params) {
$self-uri-path(@params);
$self-_clear_path;
}
elsif ( $self-_has_path ) {
return $self-_path;
}
else {
my $path = $self-uri-path;
my $location = $self-base-path;
$path =~ s/^(\Q$location\E)?//;
$path =~ s/^\///;
$self-_path($path);
return $path;
}
}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Optional path prefix
On Mon, Oct 21, 2013 at 9:08 AM, John Napiorkowski jjn1...@yahoo.comwrote:
I'd probably myself want some plack middleware that would convert
/api/v1/account/id/widget/id
to
/account/id/widget/idwith accept type application/mycompany.v1+json
I guess that would separate that out of the app.
I'm currently using this approach in an app role:
my @path_seg = split '/', $c-req-path, -1;
my $base_uri = $c-req-base;
return unless @path_seg $path_seg[0] =~ /$path_prefix_version_regex/;
my $match = $1;
die path_prefix_version_regex ($path_prefix_version_regex) matched but
failed to capture any value
unless defined $match;
$c-stash-{path_prefix_version} = $match;
$base_uri-path( $base_uri-path . shift( @path_seg ) . '/' );
# Force $req-path to reload _path next time $req-path is called.
$c-req-_clear_path;
But you could probably support changing the URL path pretty easily with
either setting the controller namespace to have v1 in it, or adding a root
tot he change that specifies the new extra path part.
But would that support it being an optional prefix? Need both to work at
the same time.
I understand the development word seems to prefer making version part of
the path. depending on your logic and the type of changes introduced it
may or may not be easier to take one approach or the other.
It does seem like that. Deciding to go with the flow vs. doing it the
right way is the decision to be made. I like your suggestion to map it
to an Accept header -- best of both worlds.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Catalyst::Request path method as a setter
=head2 $req-path
Returns the path, i.e. the part of the URI after $req-base, for the
current request.
Pasted below is Catalyst::Request's path method. Note from the final else
block that $req-path returns the request uri's path ($req-uri-path) with
the $req-base-path *removed* as the documentation says.
So, if the request URI and base URI are these:
http://localhost/myapp/path/to/action # $req-uri
http://localhost/myapp/ # $req-base
then $req-path is:
path/to/action
Using the example above, and looking at what $req-path( ) does as a setter:
$req-path( $req-path );
would result in a new request URI of:
http://localhost/path/to/action.
The path method doesn't document what it does as a setter, but this
behavior looks broken because it alters the request URI's path.
What do you think?
sub path {
my ( $self, @params ) = @_;
if (@params) {
$self-uri-path(@params);
$self-_clear_path;
}
elsif ( $self-_has_path ) {
return $self-_path;
}
else {
my $path = $self-uri-path;
my $location = $self-base-path;
$path =~ s/^(\Q$location\E)?//;
$path =~ s/^\///;
$self-_path($path);
return $path;
}
}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Optional path prefix
Over in this threadhttp://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg14226.html was a discussion on API versioning and implementing via Accept: headers vs. adding a version in the URL. Looks like using a version in the URL is winning. We have existing chained actions that might look like this: /account/id/widget/id If want to migrate to a new version scheme in the URL like this: /api/v1/account/id/widget/id This would be the same action chain as the first path -- and both would work at the same time. Is there any way to support both actions via Chained dispatching? Or will I need a role that looks for that pattern and strips it of the request during prepare_action? I've done something similar in the past where I added a language tag at the start of every path: /en_us/some/path/1234 I strip that off and then update $c-req-path for dispatching. Again, I'm in the Accept: header camp for versioning, but I'm finding more and more discussion on using URLs.There's an e-book http://pages.apigee.com/web-api-design-ebook.htmlthat seems to be cited often. I'd be interested in other's view on that book -- it seems written from a practical Rails programmer point of view instead of a REST purist view. There's a lot in that e-book I don't really agree with (plural nouns?), but the practical usage seems to be winning out. Hope it's not a mistake in the long run. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] REST and versioning
I've once again used up an hour or so reading Stack Overflow and blog posts
on REST API versioning. (And the arguments for and against.)
Perhaps extending the discussion on how Catalyst supports REST:
https://github.com/perl-catalyst/CatalystX-Proposals-REStandContentNegotiation
I'm wondering if Catalyst might help in supporting API versions. Somewhat
similar to how Catalyst::Action::REST will call methods based on the HTTP
method, perhaps call actions based on some version (provided by some means
-- like a version in the path or in an Accept header).
Catalyst::Action::REST helps keep the actions tidy by calling methods
specific to each method (foo_GET, foo_PUT). Obviously, we could simply
check if ( $req-method eq 'GET' ) but would end up with pretty ugly
actions and no automatic Allow header.
With versions I'm concerned about that my foo_GET method will end up with a
bunch of if ( $version 1.1 ) {} elsif ($version 1.0 ) ...
So, running with the C::Action::REST approach, something like:
sub foo_GET { ... } # Default
sub foo_GET : Version( 1.1 ) { ... } # Use if client requests version is
1.1
Frankly, seems like maintenance nightmare and Action explosion. Where
that version comes from (url, Accpet header) is often debated (see links).
Any better ideas how to support versioning in Catalyst actions?
The subject of versioning is a bit overwhelming. Here's some starting
points, if curious:
-
http://stackoverflow.com/questions/389169/best-practices-for-api-versioning
-
http://www.lexicalscope.com/blog/2012/03/12/how-are-rest-apis-versioned/
- http://www.subbu.org/blog/2008/05/avoid-versioning-please
-
http://stackoverflow.com/questions/2024600/rest-api-versioning-only-version-the-representation-not-the-resource-itself?lq=1
- http://www.informit.com/articles/article.aspx?p=1566460
- http://stackoverflow.com/questions/972226/how-to-version-rest-uris
- and plenty more...
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] REST and versioning
On Tue, Sep 17, 2013 at 10:12 AM, John Napiorkowski jjn1...@yahoo.comwrote:
People seem to get religious over how to version their API. I doubt I'd
want to take sides on this but here's how I think we could do both side
(URL version and content type versioning
Ya, I'm swayed by the Accept: header approach because in my mind
/api/v1/account/123 and /api/v2/account/123 seems like different resources.
(Well, I guess they are.) So, the versions are lazy way to make a new
resource location.
And even more so, seems like a dark path to go down. Is just that
individual resource versioned or is the entire API versioned? And if it's
the entire API, and the app needs to support multiple versions at the same
time, then need a way for methods to fall-back to v1 when only a few
methods change.
Or maybe the client would have to know which methods are v2 vs. v1 and
pick-and-choose.
Realistically, the problem that would likely come up is more related to
client versioning where an old client cannot support some new feature of
the API.
For example, say a service has a method to fetch a widget and a method to
list them.
GET /widget/123 # get widget 124
GET /widget # list all widgets.
So, some client app is designed to list the widgets and then fetch them
(for display or whatever).
Later, the service is upgraded and adds a new *type* of widget -- and that
is a type that the existing old client app cannot support.
Does the service need to know what the client can support?
sub widget_GET : Args(0) ... {
...
push @include_types, $new_widget_type if $client_version = 1.1;
That's going to lead to some nasty spaghetti code over time.
Maybe not such a great example as one could argue here that the client
could GET /widget?type=1type=2type=3, but other changes might make that
not so easy.
In your chained example below how does that work with longer paths?
I'm trying to come up with a good example.
/v1/document/1234 # fetch a document
/v1/document/1234/share # list who the document is shared with.
Then what happens if it's just the share method that has a new version?
package Myapp::Web::Controller::API;
use base 'Catalyst::Controller';
sub start : ChainedParent
PathPrefix CaptureArgs(0)
{
my ($self, $ctx) = @_;
}
sub version_one : Chained('start') PathPart('1') Args(0) { ... }
sub version_two : Chained('start') PathPart('2') Args(0) { ... }
1;
package Myapp::Web::Controller::API::1;
use base 'Catalyst::Controller';
sub start : ChainedParent
PathPrefix CaptureArgs(0)
{
my ($self, $ctx) = @_;
}
1;
package Myapp::Web::Controller::API::2;
use base 'Catalyst::Controller';
sub start : ChainedParent
PathPrefix CaptureArgs(0)
{
my ($self, $ctx) = @_;
}
1;
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] More detailed proposal for changes related to content negotiation and REST
On Thu, Aug 8, 2013 at 9:27 PM, John Napiorkowski jjn1...@yahoo.com wrote:
https://github.com/perl-catalyst/CatalystX-Proposals-REStandContentNegotiation
I currently extend HTTP::Body in a similar way that you describe. But I
have them a separate classes so I just do:
use My::Multipart;
then in that I hack in my type for HTTP::Body:
package My::MultiPart;
use HTTP::Body;
$HTTP::Body::TYPES-{'multipart/form-data'} = 'My::MultiPart';
As you propose, mapping a mime type to a sub seems pretty flexible. I
assume the sub could return a filehandle. File uploads still need to
stream the uploads to disk while making the parameters available as
HTTP::Body does now.
I like the regex mimetype keys, but should they be an array instead of a
hash so can set the order checked?
I think we must consider large inputs/streams.You say $_ is the request
content. Is that the full request with headers? Is the request already
de-chunked, for example? Or am I thinking too low level?
In some apps I'm using Catalyst::Action::REST and others I have some custom
code where I use HTTP::Body to parse JSON. I'm mixed about having the
request data end up in $c-req-parameters vs $c-req-data.I don't
really see why form-data/urlencoded should be treated differently than
other encodings like JSON.
I not quite sure about $c-res-body( \%data ); I think body should be
the raw body. What does $c-res-body return? The serialized json? The
original hashref?
If a parser dies what kind of exception is thrown? You say they would not
set any response status, but wouldn't we want to catch the error and then
set a 400? (I use exception objects that carry http status, a message to
return in the body and a message used for logging at a given level.)
I'm not sure what to think of the Provides and Consumes controller
attributes. It's one thing to realize early that the client cannot handle
any response I'm willing to provide (json, yaml, whatever), but I'm worried
this would blur separation of concerns. That is, by the time the
controller runs we would have already decoded the request and want to just
work with the data provided. And would we want different data returned if
the client is asking for json vs yaml?
Maybe I'm missing the point of that.
BTW - I practice I find it pretty handy to be able to specify/override
response encoding via a query param.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] More detailed proposal for changes related to content negotiation and REST
regardless of how it is serialized. And GET /user would get the user data and then serialize that to JSON or whatever but it's the same data. But, maybe you have a point.I would worry that someone assumes JSON and adds that content type match and then wonder why later it's not working for other request serializations. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] CSV / UTF-8 / Unicode
On Tue, Jul 2, 2013 at 2:59 AM, Craig Chant
cr...@homeloanpartnership.comwrote:
# output header
$c-response-content_type('application/vnd.ms-excel');
$c-response-content_length(length($xls));
$c-response-header(Content_Disposition =
'attachment;filename=NBCS_Export.csv');
# create an IO::File for Catalyst
use IO::File;
my $iof = IO::File-new;
$iof-open(\$xls, r);
$iof-binmode(:encoding(UTF-8));
# output XLS data
$c-response-body($iof);
All the above seems overkill. I suspect what you want is closer to this:
(but see notes below).
$c-response-content_type('text/csv');
$c-response-body($xls);
$c-response-header(Content_Disposition =
'attachment;filename=NBCS_Export.csv');
Then with that content type the plugin would encode $xls as utf8 and add
;charset=utf8 (or whatever it is configured to encode as).
Notes:
First, you are not returning Excel, so the content type is not what you
first listed above, right?
Second, be aware that $c-response-content_length(length($xls)); could be
very wrong. If $xls is really CSV text AND it's decoded then length($xls)
is the length in characters, not octets. Don't set the content length.
Third, Catalyst::Plugin::Unicode::Encoding, IMO, has some issues.
The plugin limits to just these content types.
return $c-next::method(@_)
unless $c-response-content_type =~ /^text|xml$|javascript$/;
Then it does this:
$c-response-body( $c-encoding-encode( $body, $CHECK ) )
if ref(\$body) eq 'SCALAR';
Personally, I think the correct approach is to only encode *character* data
-- that is check to see if the utf8 flag is set before calling encode.
Maybe limit to the content types listed above, but throw an exception for
other content types where the body is a scalar AND has the utf8 flag on.
After all, we can only write out octets or else we get the Wide Character
error.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] CSV / UTF-8 / Unicode
Before we get a long utf8 discussion here: On Tue, Jul 2, 2013 at 8:58 AM, Bill Moseley mose...@hank.org wrote: Personally, I think the correct approach is to only encode *character* data -- that is check to see if the utf8 flag is set before calling encode. I say that with the caveat that ALL textual input data is correctly decoded -- as it should be. That is, everything that was decoded into characters has the utf8 flag set. $ perl -MEncode -le 'print yes, this is characters if Encode::is_utf8( Encode::decode( ASCII, This is ASCII))' yes, this is characters -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] unicode warnings (+ errors C::P::UploadProgress)
On Fri, Jun 14, 2013 at 11:29 AM, John Napiorkowski jjn1...@yahoo.comwrote: Yup this is a regression! We have a patch in HEAD and it should go out later today. Sorry for the trouble ( but at least I know who's upgrading) I upgraded yesterday and noticed the same thing in out tests: [error] Caught exception in engine Can't use string (Can't call method decode on an...) as a HASH ref while strict refs in use at /Users/bill/perl5/perlbrew/perls/perl-5.16.1/lib/site_perl/5.16.1/Catalyst/Plugin/Unicode/Encoding.pm line 101. I'll wanted to investigate I have my own code that handles decode/encode and need time to move to the Catalyst code. Until we can migrate what would you recommend so that we don't have two plugins trying to handle the decoding? I also noticed that UploadProgress got pulled in -- but I don't think we are using that so was a bit surprised by that requirement. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] /Depreci?ate/ message
Could this message be disabled by environment variable or config? Besides ending up in logs, we have tests that are failing because of it. Our tests trap all warnings and will fail on unaccounted for warnings. Thanks, On Thu, May 23, 2013 at 3:16 PM, John Napiorkowski jjn1...@yahoo.comwrote: speeling error noted! thanks! -john - Original Message - From: Toby Corkindale t...@dryft.net To: The elegant MVC web framework catalyst@lists.scsys.co.uk Cc: Sent: Wednesday, May 22, 2013 1:52 AM Subject: [Catalyst] /Depreci?ate/ message I noticed that the recent Catalyst release has added a Depreciation warning to the Regex class. Are you sure you meant depreciate? I think old features are normally deprecated. Cheers, Toby -- Turning and turning in the widening gyre The falcon cannot hear the falconer Things fall apart; the center cannot hold Mere anarchy is loosed upon the world ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Chained and exceptions
On Fri, May 10, 2013 at 1:29 AM, Tomas Doran bobtf...@bobtfish.net wrote: You're after this: https://metacpan.org/module/Catalyst::ActionRole::DetachOnDie which gives you the alternate behaviour (i.e. detaching from the chain on first exception). We have a number of applications, a few quite large, where some controllers inherit from different base classes. We could try and retro fit all existing code, but it would be a good-sized project. So the monkey patch we did (as well as Dami Laurent had done in 2008http://lists.scsys.co.uk/pipermail/catalyst/2008-March/017789.html) is better for us. I'm pretty sure this issue is not well known amongst our current (and future) developers and thus it's quite likely someone would forget this in a new Controller. We all understand that an uncaught exception should not bring down the server and instead generate a 500, but I think few would assume that when using Chained an exception would not stop the request dead in its tracks and instead is implicitly trapped and allowed to continue. I think the more likely situation now is code running when it is not expected -- which could be a serious security issue if the earlier action in a chain is used for access control. What would the developers think of deprecating this behavior (for the few that might actually be relying on this) and issue a warning if a config option is not set that fixes the issue? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Chained and exceptions
I have a feeling I asked this before, but cannot find the post.
[info] Exception powered by Catalyst 5.90030
What's the reasoning that chained actions continue to run after an earlier
exception?
sub start : Chained( '/' ) : CaptureArgs(0) {
warn in start\n;
}
sub middle : Chained( 'start' ) : CaptureArgs(0) {
warn in middle\n;
die died in middle\n; # or e.g. throw access violation
}
sub lastpart : Chained( 'middle' ) : Args(0) {
my ( $self, $c ) = @_;
$c-res-body( finished\n );
warn in lastpart\n;
}
$ script/exception_test.pl /start/middle/lastpart
in start
in middle
*in lastpart*
[error] Caught exception in Exception::Controller::Root-middle died in
middle
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Chained and exceptions
On Thu, May 9, 2013 at 7:25 AM, Lukas Thiemeier
spamcatc...@thiemeier.netwrote:
On 05/09/2013 03:25 PM, Bill Moseley wrote:
I have a feeling I asked this before, but cannot find the post.
[info] Exception powered by Catalyst 5.90030
What's the reasoning that chained actions continue to run after an
earlier exception?
sub start : Chained( '/' ) : CaptureArgs(0) {
warn in start\n;
}
sub middle : Chained( 'start' ) : CaptureArgs(0) {
warn in middle\n;
die died in middle\n; # or e.g. throw access violation
}
sub lastpart : Chained( 'middle' ) : Args(0) {
my ( $self, $c ) = @_;
$c-res-body( finished\n );
warn in lastpart\n;
}
$ script/exception_test.pl http://exception_test.pl
/start/middle/lastpart
in start
in middle
*in lastpart*
[error] Caught exception in Exception::Controller::Root-middle died in
middle
Hi Bill,
This is because you don't want Catalyst to die. Imagine you are running
a fastcgi server and you accidentally created an action which dies on
certain user input.
Hi Lukas,
Sorry, you missed the point. Yes, Catalyst traps exceptions. That is
expected and is done in handle_request when calling $c-dispatch.
I'm talking about breaking a chain of actions. Why would a later part
of the chain run if an earlier part threw an exception. For example, what
if an earlier part of a chain threw a access violation. In that case you
would not want the later chain to run.
If Catalyst would die, the fastcgi server would die and your whole
application would not be available anymore. Instead, you want to report
the incident and keep the fastcgi server (Catalyst) running.
Because of this, every action is wrapped in an eval{...}. Potential
errors are logged, but the server keeps running.
See execute in Catalyst.pm for implementation details.
To solve your problem, you can wrap your unsafe code in an eval or use
Try::Tiny (or whatever you prefer) and detach if the unsafe code dies.
Your Example would look like this:
use Try::Tiny;
sub start : Chained( '/' ) : CaptureArgs(0) {
warn in start\n;
}
sub middle : Chained( 'start' ) : CaptureArgs(0) {
my ($self, $c) = @_;
warn in middle\n;
try{
die died in middle\n; # or e.g. throw access violation
}
catch{ $c-detach };
}
Don't forget your semicolon. :)
sub lastpart : Chained( 'middle' ) : Args(0) {
my ( $self, $c ) = @_;
$c-res-body( finished\n );
warn in lastpart\n;
}
If you do it like this, actions chained to the unsafe action will not
get executed if the unsafe action dies. In your case, lastpart will
never be executed, because middle always dies.
Lukas
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive:
http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Chained and exceptions
On Thu, May 9, 2013 at 8:31 AM, Bill Moseley mose...@hank.org wrote: Hi Bill, This is because you don't want Catalyst to die. Imagine you are running a fastcgi server and you accidentally created an action which dies on certain user input. Hi Lukas, Sorry, you missed the point. Yes, Catalyst traps exceptions. That is expected and is done in handle_request when calling $c-dispatch. No, sorry, I responded too quickly. I'm not talking about the entire have exiting, of course. I'm just not sure it makes sense to not check $c-errors after calling each action in a chain. What's the use case for that? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Chained and exceptions
On Thu, May 9, 2013 at 9:34 AM, Aristotle Pagaltzis pagalt...@gmx.dewrote:
* Bill Moseley mose...@hank.org [2013-05-09 15:30]:
What's the reasoning that chained actions continue to run after an
earlier exception?
Seems like an accident of the design to me, borderline bug.
Agreed. Seems like something that could be easily overlooked.
If like me you don’t like it, Catalyst::ActionRole::DetachOnDie
Oh, that's nice. Tricks for applying it globally?
I went the monkey patch route. Shield your eyes:
use Catalyst::ActionChain;
sub Catalyst::ActionChain::dispatch {
my ( $self, $c ) = @_;
my @captures = @{$c-req-captures||[]};
my @chain = @{ $self-chain };
my $last = pop(@chain);
foreach my $action ( @chain ) {
my @args;
if (my $cap = $action-number_of_captures) {
@args = splice(@captures, 0, $cap);
}
local $c-request-{arguments} = \@args;
$action-dispatch( $c );
* return if @{ $c-error }; # Patch*
}
$last-dispatch( $c );
}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] How to get IP address of the interface which the request come through
On Sat, Mar 23, 2013 at 7:31 AM, N.A. n...@u01.gate01.com wrote: (2013年03月23日 23:03), Peter Flanigan wrote: On 23/03/13 12:12, N.A. wrote: I wan to get the IP address of the interface(network device) which the request come through. My bad. Use $c-req-uri-host to get the hostname of the server Sorry, $c-req-uri-host is 'hostname', not a IP. I want to get IPv4 address like '127.0.0.1' even if I access the page by 'http://localhost/XXX' Interesting. What's your use case here? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst, ExtJS and TT
On Mon, Mar 18, 2013 at 3:56 AM, Dave Howorth dhowo...@mrc-lmb.cam.ac.ukwrote:
I have a javascript file named EntrepriseWindow.js in root dir. It
contains some TT code. I must add a extension (I renamed the file
EntrepriseWindow.js) for it to work. Otherwise, it does not work :
sub EntrepriseWindow :Path('EntrepriseWindow.js') {
my ($self, $c) = @_;
$c-stash-{template} = 'EntrepriseWindow.js.tt';
$c-response-content_type('application/javascript');
}
I cannot write this ?
$c-stash-{template} = 'EntrepriseWindow.js';
Most frameworks that use TT have a mechanism to find template source
files based on their extension. In the case of Catalyst, see
http://search.cpan.org/~jjnapiork/Catalyst-View-TT-0.41/lib/Catalyst/View/TT.pm#TEMPLATE_EXTENSION
You should be able to specify the full template name via the stash. Here's
the code:
sub process {
my ( $self, $c ) = @_;
my $template = $c-stash-{template}
|| $c-action . $self-config-{TEMPLATE_EXTENSION};
If the template is explicit in the stash then TEMPLATE_EXTENSION should not
come into use.
If you think about it, the file you want TT to process is *not* a .js
file, it's a file with some TT syntax in it that needs processing to
*become* a .js file. So it would be unhelpful if Catalyst tried to
TT-process real .js files and it's reasonable, IMHO, to have something
like a .js.tt extension.
In this case Jean-Marc said that EntrepriseWindow.js has some TT code
added. I agree that it would be clearer if it had a different extension
so that later someone does not mistaken it for a js file.
But, a javascript file really should be served statically.
Jean-Marc, what per-request changes are you making to that file?
Something like setting strings based on the user's language preference for
that request?
I would think it would be better to serve it statically (with caching
headers) and then serve any pre-request changes in the actual html
response. Serving up different javascript files with the same name could
lead to problems.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Not cleaning up temporary files / HTTP::Body
Just to be clear, this opens up all Catalyst apps to a pretty easy DoS --
or in our case denial of sleep as people were woken up in the middle of the
night as servers ran low on disk space. Good thing we have monitoring set
up.
The risk to any individual Catalyst user is obviously limited, but
something to be aware of.
The design flaw here means Catalyst apps, by default, give write acces to
/tmp (which often isn't that big) to any user of your app and doesn't clean
up after itself. Running out of /tmp space is never fun.
HTTP::Body should be fixed because of this issue, and I've created a ticket
against it. A fix there could be done w/o any changes needed for Catalyst.
But, there's other things to consider. For example, there's nothing to
limit the size of a single request. Some of those tests should be higher
up in the stack.
Apache has a setting LimitRequestBody but IIRC that only looks at the
Content-Length header. But, HTTP::Body does limit the body to what is
reported in the Content-Length header.
Starman buffers requests to /tmp (in an unlinked file) w/o any limit that I
could see in my limited tests.
On Thu, Mar 14, 2013 at 9:53 PM, Bill Moseley mose...@hank.org wrote:
On Wed, Jan 13, 2010 at 6:53 AM, Bill Moseley mose...@hank.org wrote:
HTTP::Body::Multipart creates temporary files for uploads. The temp
files are created with File::Temp( UNLINK = 0 ).
Well, this is still broken.
Yes, since 2010 HTTP::Body has been updated to have a DESTROY where it
removes files, and Catalyst::Request calls the $body-cleanup(1) attribute
to enable this feature.
But, the problem is DESTROY (see below) looks for files to remove in
$self-{upload} hash. But, that is an empty hash until the upload is
completed. So, if the upload is aborted (which is when you need the clean
up to work) there's nothing in the upload hash -- so nothing to unlink.
Three years ago when I first posted about this I copied
HTTP::Body::MultiPart and set UNLINK = 1 on File::Temp-new and created
my own version that overrides.That still works.
But, if I simply modify the current H::B::MultiPart and add UNLINK=1 then
the file gets deleted and I get an error. So, the fix isn't that simple any
more. That needs some looking at.
Clearly, what we are after is deleting the temp files. So, might as well
let File::Temp do its job. Just have to make sure the handle doesn't go
out of scope too early.
Oh, BTW, you can't test this with the dev server. I had to use Apache.
With Apache chunks are sent to Catalyst. But, when running on the dev
server it seems like the entire upload was buffered before prepare_body
gets called -- so even with a 16MB upload I could not hit esc to abort
the upload while HTTP::Body was processing.
sub DESTROY {
my $self = shift;
if ( $self-{cleanup} ) {
my @temps = ();
*for my $upload ( values %{ $self-{upload} } ) {*
push @temps, map { $_-{tempname} || () }
( ref $upload eq 'ARRAY' ? @{$upload} : $upload );
}
unlink map { $_ } grep { -e $_ } @temps; ## this is empty when
aborted
}
}
Catalyst then deletes these temp files in $c-finalize. The problem is
that an exception can happen and then the temp files are not deleted.
Happens quite often, it turns out. I seem to always see this in the logs
at the time of the orphaned temp file:
Caught exception in engine Apache2::RequestIO::read: (104) Connection
reset by peer
Aborting an upload isn't that unusual.
I can set temp directory for these uploads and use cron to clean them
out, but I wonder if they could not be cleaned up better automatically.
For example, unlink in DESTROY when the request object goes out of
scope. Or perhaps better, don't have HTTP::Body set UNLINK = 1 by default
and when the upload object finally goes out of scope File::Temp will remove
the temp file. HTTP::Body explicitly removes the File::Temp object from the
upload part, but I'm not sure why it needs to do that. Why not leave the
File::Temp object in the upload part object then let it go out of scope at
the end of the request.
Where should this be addressed? In Catalyst or in HTTP::Body?
--
Bill Moseley
mose...@hank.org
--
Bill Moseley
mose...@hank.org
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Not cleaning up temporary files / HTTP::Body
On Wed, Jan 13, 2010 at 6:53 AM, Bill Moseley mose...@hank.org wrote:
HTTP::Body::Multipart creates temporary files for uploads. The temp files
are created with File::Temp( UNLINK = 0 ).
Well, this is still broken.
Yes, since 2010 HTTP::Body has been updated to have a DESTROY where it
removes files, and Catalyst::Request calls the $body-cleanup(1) attribute
to enable this feature.
But, the problem is DESTROY (see below) looks for files to remove in
$self-{upload} hash. But, that is an empty hash until the upload is
completed. So, if the upload is aborted (which is when you need the clean
up to work) there's nothing in the upload hash -- so nothing to unlink.
Three years ago when I first posted about this I copied
HTTP::Body::MultiPart and set UNLINK = 1 on File::Temp-new and created
my own version that overrides.That still works.
But, if I simply modify the current H::B::MultiPart and add UNLINK=1 then
the file gets deleted and I get an error. So, the fix isn't that simple any
more. That needs some looking at.
Clearly, what we are after is deleting the temp files. So, might as well
let File::Temp do its job. Just have to make sure the handle doesn't go
out of scope too early.
Oh, BTW, you can't test this with the dev server. I had to use Apache.
With Apache chunks are sent to Catalyst. But, when running on the dev
server it seems like the entire upload was buffered before prepare_body
gets called -- so even with a 16MB upload I could not hit esc to abort
the upload while HTTP::Body was processing.
sub DESTROY {
my $self = shift;
if ( $self-{cleanup} ) {
my @temps = ();
*for my $upload ( values %{ $self-{upload} } ) {*
push @temps, map { $_-{tempname} || () }
( ref $upload eq 'ARRAY' ? @{$upload} : $upload );
}
unlink map { $_ } grep { -e $_ } @temps; ## this is empty when
aborted
}
}
Catalyst then deletes these temp files in $c-finalize. The problem is
that an exception can happen and then the temp files are not deleted.
Happens quite often, it turns out. I seem to always see this in the logs
at the time of the orphaned temp file:
Caught exception in engine Apache2::RequestIO::read: (104) Connection
reset by peer
Aborting an upload isn't that unusual.
I can set temp directory for these uploads and use cron to clean them out,
but I wonder if they could not be cleaned up better automatically.
For example, unlink in DESTROY when the request object goes out of scope.
Or perhaps better, don't have HTTP::Body set UNLINK = 1 by default and
when the upload object finally goes out of scope File::Temp will remove the
temp file. HTTP::Body explicitly removes the File::Temp object from the
upload part, but I'm not sure why it needs to do that. Why not leave the
File::Temp object in the upload part object then let it go out of scope at
the end of the request.
Where should this be addressed? In Catalyst or in HTTP::Body?
--
Bill Moseley
mose...@hank.org
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Using the Catalyst Makefile to install
On Mon, Mar 11, 2013 at 1:29 PM, Alejandro Imass alejandro.im...@gmail.comwrote: This pre-dates Module::ShareDir. So the templates, images (i.e. the /root) are considered shared data? When I was converting our Catalyst apps to use Dist::Zilla I looked at changing where Catalyst looks for /root -- just so I could use Module::ShareDir (supported by Dist::Zilla) and not have to rewrite Module::Install::Catalyst. I think I had some discussion of these back in July: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg13186.html In the end I did implement the code to copy in the /root (and other files and dirs) into blib just like Module::Install::Catalyst does. And the configuration file should go for example in /etc in Debian and /usr/local/etc on FBSD?? We deploy with RPM. FWIW, this is something we discussed at work quite a bit. Operations wanted config in standard location, but we wanted config with the application. In general, of our config is more constants that config and so there's rarely changes to our app config. Plus, having config with the app makes it a bit easier to have multiple copies installed. (In the past used symlinks to point to live version.) In other words, the bulk of our config is more like static data, so placing in share would be ok. What we did in the end to make the site operations team happy was update our config system to look in /etc/$app_name/config.yml for an extra config file that would override (merge with) the app's config. Our config loads config.yml filrst, then it loads and merges any mode-specific (i.e. qa, staging, production) config, then finally looks for /etc/$app_name/config.yml. Some day we will look more closely at centralized configuration tools -- Puppet and Chef, for example. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Backlog for proposed changes in next Catalyst release
On Sun, Mar 10, 2013 at 2:06 PM, Peter Flanigan p...@roxsoft.co.uk wrote: # Only touch 'text-like' contents return $c-next::method(@_) unless $c-response-content_type =~ /^text|xml$|javascript$/; smells bad, it will apply to text/x-json but not application/json The code that I use only encodes if the utf8 flag is set. The JSON encoders (at least what I use) encode to utf8, so my code doesn't touch JSON as it's already been encoded. I decode all input -- database, templates, text files, web requests and API requests. All data that represents characters must be decoded on input. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Using the Catalyst Makefile to install
On Thu, Mar 7, 2013 at 8:57 PM, Francisco Obispo fobi...@isc.org wrote: Module::Install lets you do some limited configuration, but if you're looking to customize it, you're better off with a custom distribution package. A bit off the topic, but I've been using Dist::Zilla to help package Catalyst apps. My favorite features are auto versioning, managing the Changes file, and automatic dependency creation. And, of course, having a custom PluginBundle that centralizes how we build distributions. Perhaps not for everyone, but worth a look. Check out the Dist::Zilla::Plugins on CPAN. But the end result is still just a normal Perl module. All our modules (Catalyst and others) use Dist::Zilla now. I have a Dist::Zilla::Plugin::CatalystFiles that helps build the dist and builds a custom Makefile.PL. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Using the Catalyst Makefile to install
On Fri, Mar 8, 2013 at 1:43 PM, Alejandro Imass alejandro.im...@gmail.comwrote: Thanks to both of you for your valuable insight. We've also looked at the PAR route. Up until now we've always installed manually and only used the Makefile to keep track of dependencies and easier installs of them, but we've never used it to install the app as such. We are currently deploying a beautiful code and we wanted to do it end-to-end including the make install. We're going to experiment with this first and then analyze all other options. Just to add a bit: As you know, Catalyst is like any other Perl module. So, we deploy our Catalyst apps to an internal DarkPAN along with all our other internal modules. What that means is on a new machine (with a few environment variables set) you can type: cpanm Our::Catalyst::App and it an all its dependencies are installed. So, that Makefile actually comes in pretty handy. (And since we use Dist::Zilla, releasing to our internal CPAN is just dzil release. So, that's pretty slick.) We then build RPMs from these modules and that's what is used for deployment. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Backlog for proposed changes in next Catalyst release
On Tue, Mar 5, 2013 at 7:09 AM, John Napiorkowski jjn1...@yahoo.com wrote: Bill, I think there's general agreement that the plugin is stable enough and right enough that it should represent core behavior. Although one of Catalyst's strengths has been to be un-opinionated and leave stuff to external distributions when possible, I think this does fall under the 'all good apps should work this way'. I've tentatively pegged it for the March Catalyst release, since it seems pretty straightforward, at least for now. If you think you can fit it in (or talk your boss/clients/whatever into paying for your time) please give a shout out. It would be great to see more people get to know Catalyst internals a bit, otherwise I don't really see how we are going to be able to move the platform forward. The month of May is more likely for me, unfortunately. I was thinking of turning the plugin into a Role and then consuming if the plugin was not already listed or if a config option to disable was not set. That is, on by default but can be disabled. Is that in line with what you were thinking? Ha! I was looking at the code just now and saw a comment that included this URL: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg02350.html I guess I've thought about this before I wonder what percent of Catalyst apps make use of that plugin. Not sure, but I think the play-perl ticket has a good way to not break most people's code -- Bill Moseley mose...@hank.org -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Backlog for proposed changes in next Catalyst release
On Mon, Mar 4, 2013 at 9:24 AM, John Napiorkowski jjn1...@yahoo.com wrote: http://play-perl.org/quest/5134cdc18e0a96450f38 There was a discussion about this on #catalyst-dev and in general people seem to think its ok. Bill, are you up to giving the work a try? I'd be happy to mentor you. I don't think its killer work, but would really help I would like to give it a try, yes. I'm just not sure when. I guess there's no rush since there's a plugin. I was just thinking it would be wise to make it a standard part of the framework since it's something that every app probably should to, but easy to ignore or get wrong. I wonder what percent of Catalyst apps make use of that plugin. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Backlog for proposed changes in next Catalyst release
On Fri, Mar 1, 2013 at 9:38 AM, John Napiorkowski jjn1...@yahoo.com wrote: Hey All, http://jjnapiorkowski.typepad.com/modern-perl/2013/03/catalyst-backup-for-next-release-on-play-perl.html This is over on play Perl (http://play-perl.org/player/jnap) for your comments and votes. Hope to see you there! I think I've asked this before, but is there any talk of native encoding support -- meaning make Catalyst::Plugin::Unicode::Encoding part of the framework? Having it a plugin makes it appear as optional, but the correct approach is to decoded all text requests and encode all text responses. This is fresh in my mind because last week had problems with two separate encoding issues. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst::Controller::REST and Serializer/Deserializer
On Sun, Feb 24, 2013 at 11:13 PM, Jean-Marc Choulet jm130...@gmail.comwrote: Hello, I'm new to Catalyst and I've certainly a stupid question... I use a Catalyst::Controller::REST to implement a web service. My question is : How can I force a application/json content-type response when I ask my ws with FF or Chrome... ? Do you mean adding content-type=application/json to your GET request query parameters in the browser? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] May be asynchronous communication between Catalyst applications
On Sun, Feb 24, 2013 at 8:16 AM, James R. Leu j...@mindspring.com wrote: I do not have any suggestions about the App1 to App2 hand off but perhaps my technique for handling async/long running requests will help. I solved my async requirements by using Catalyst::Plugin::Cache and Catalyst::Plugin::RunAfterRequest. Long running requests have to run somewhere, of course, and often running in the Catalyst app is the easy solution and may work fine if you are not running a busy site or expecting many concurrent users -- ever. But, in general designing your web app in a way that you expect processes to block for a long time is very risky. You don't want to DoS your own site. It's all about the specifics, but long running processes are often resource intensive, and when they run long users start reloading which users more resources and just compounds the problem. Soon you have an unresponsive site. Maybe you can manage or limit the number of long running web processes in some way, but this is a problem that's already been solved. Francisco gave very good advice. Use a queue to decouple the web app from the long-running jobs. You then have control over your resources -- workers pull in work as they are free instead of stuffing work into web processes. Scaling is then trivial as it's just more workers. It may seem like more work up front to set up but will making things like this much easer -- and safer. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst::Model::DBIC::Schema and caching row objects
On Tue, Feb 12, 2013 at 10:05 PM, Octavian Rasnita orasn...@gmail.comwrote: ** *From:* Bill Moseley mose...@hank.org If you want to use DBIC in more apps, don't use Catalyst::Model::... but create a standalone module that can be used from more apps, even in CLI scripts. And access that standalone module using Catalyst::Model::Adaptor. To be clear, I do have a separate DBIC schema class that can be, and is, used by CLI scripts. I also think it's a good recommendation to use Catalyst::Model::Adaptor instead. Unfortunately, the two apps sharing the schema are years old and followed best practices at the time and used Catalyst::Model::DBIC::Schema. It would be a big project to change at this point. Can anyone explain the reasoning of this code in Catalyst::Model::DBIC::Schema? Why call compose_namespace($class) here? my $class = $self-_original_class_name; ... $self-composed_schema(*$schema_class-compose_namespace($class*)) unless $is_installed; My question is what would break if instead it was just this? $self-composed_schema( *$schema_class *) unless $is_installed; That would allow row object to use their default class. In case that's not clear, in the first code snippet $class is the name of the *Catalyst model* class. For example MyApp::Model::MyDB. Using that class in compose_namespace means an Account row object ends up as *MyApp::Model::MyDB::Account* instead of *Schema::Result::Account*. Again, that is the problem for me because that's the class name that ends up in Memcached when cached and if want to share those cached row object between Catalyst apps it breaks because OtherApp doesn't know how to load MyApp::Model::MyDB::Account because that class only exists in MyApp. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Catalyst::Model::DBIC::Schema and caching row objects
Not sure if the is a Catalyst or DBIC question. Catalyst::Model::DBIC::Schema returns objects blessed into classes related to the app. Then the problem is I cannot share (cached) objects across applications. I have a set of DBIC classes MySchema. I have an often used and expensive query where I cache the returned rows. So, a row object like this: $schema-resultset( 'Account' )-find( 123 ); ends up blessed into MySchema::Result::Account. When row objects are cached the class name is stored so it can be thawed back into that class, of course. Now, I have an app MyApp that uses Catalyst::Model::DBIC::Schema to access MySchema. In this case row objects are blessed into: MyApp::Model::MyDB::Account. And that's the name that is serialized when freezing. The problem is another app OtherApp also uses the DBIC MySchema class and when it fetches a cached entry it tries to bless them into MyApp::Model::MyDB::Account which it cannot do. (Needs to thaw into OtherApp::Model::MyDB::Account.) Error: Can't locate MyApp/Model/MyDB/Account.pm in @INC Any ideas what might be done here, short of not storing the DBIC row objects? I do want to end up with row objects after thawing, so they can be used just as if I had done the query directly. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Detecting when running under mod_perl
Pre-Plack I had used this to determine if running under Apache/mod_perl.
if ( $c-engine-can( 'apache' ) ) { ... }
Is this the best, future-proof approach now?
During setup check $ENV{MOD_PERL}
and per request check if $c-req-env-{'psgi.input'}
isa 'Apache2::RequestRec'?
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Use of uninitialized value in delete
On Mon, Oct 29, 2012 at 3:45 AM, Will Crawford
billcrawford1...@gmail.comwrote:
On 19 October 2012 18:37, Bill Moseley mose...@hank.org wrote:
delete $c-stash-{foo};
Is there a function / method called foo anywhere in scope?
I've ignored this for a while, but still seeing in the app's logs.
No, there's no function foo defined.
Another thing that is odd is I have a $SIG{__WARN__} handler and all other
warning are caught by that but not the Use of uninitialized value in
delete.
This machine is running 5.10.1 -- maybe it's related to that version of
Perl.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] IPv6 client addresses
I'm just starting to look into IPv6 and how it might effect our applications. Perhaps this is a Plack question, but I'm currently using $c-req-address to get at the client address. Our plan is to run dual-stack v4 and v6. Will there be a separate method for the IPv6 address, or will $c-req-address return either v4 or a v6 address? Anyone already in a dual-stack environment? Any other gotchas to consider? I use $c-req-address to limit access -- for example to limit some actions to our local LAN or for customers to limit access to our API via a customer-supplied CIDR. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Does uri_for() URL-escape arguments correctly ?
On Tue, Dec 4, 2012 at 5:22 AM, Marc SCHAEFER schae...@alphanet.ch wrote:
Hi,
for some time I write things like this in my templates:
a href=[% c.uri_for(c.controller.action_for('object'), [ file ])
%]img src=[% c.uri_for(c.controller.action_for('thumbnail'), [ file ])
%] alt=[% video | html %] //a
I've always used href=[% c.uri_for( ... ) | html %]
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] log4perl and logging request parameters
I use a log4perl config file, and I have two appenders -- one is for INFO
and above and is a screen appender (web server logs), and the other is for
ERROR and is directed to a separate file.
For *only *WARN and above messages I want to include a dump of request
parameters in the log message. That is, for INFO messages I don't want to
include the parameters.
That's pretty easy for normal exceptions. At the end of a request I look
for $c-error and when found I use Log::Log4perl::MDC-put( params =
Dumper $c-req-parameters ) to include the params in the log using
%X{params}. (Except, I sanitize the params first -- e.g. replace values
that match /password/).
I'd like to find a more generic way, and something that will work for
calling $c-log-warn and $c-log-error directly, too. (I also trap
__WARN__ and __DIE__ to use log4perl and dump a stack trace).
Again, I don't want the params in all messages, so I don't want to
set Log4perl::MDC-put( params = $dump ) at the start of each request. I
only want to do that for WARN and ERROR messages.
Should I put $c-req-parameters in a global at the start of each request
and then wrap the warn() and error() methods and set the MDC params in
those methods?
Monkey-patch (redefined) the warn() and error() log methods each request
with the current params?
Neither of those sound that great. Is there an approach I'm missing?
Thanks,
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Catalyst with Twiggy with Pocket.IO (Comet)
On Wed, Nov 28, 2012 at 4:21 AM, Jaro Zajonc jaro.zaj...@gmail.com wrote: But if I direct traffic from Apache directly to Twiggy server I'd bypass Catalyst Authentication/Authorization part for Comet session, right? I'd like to allow only authenticated users to subscribe to comet channel. I am sure I am missing some really simple piece of the puzzle :-\ Are you over SSL by chance? I've done this by constructing a token on the authenticated server and then have the secondary server that can't fully authenticate validate the token which might be a simple digets of secret + timestamp. That is, the server w/o the auth validates that the token is legitimate and the SSL tells me it came from the client I gave it to. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Access Catalyst context object from script
On Wed, Oct 31, 2012 at 6:10 AM, Anthony Gladdish
anthony.gladd...@newcastle.ac.uk wrote:
Great - this worked, thanks. I used:
use Catalyst::Test 'MyApp';
my($res, $c) = ctx_request('/');
How does uri_for() know what host:port and script path to use to build
correct URLs to your site?
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Global 'helper' methods
On Tue, Oct 30, 2012 at 7:03 AM, Craig Chant
cr...@homeloanpartnership.comwrote:
So it seems it’s OK to whack them in the main MyApp.pm , here is an
example of what is currently in a ‘MemberGlobs.pm’, which is ‘required’ in
90% of all the perl scripts.
Be careful with that approach. I'd stick with method that need to get
inserted into the request cycle -- not just a catch-all for something you
don't know where it really belongs.
##
# Yank Date #
##
sub YankDate {
#_[0] = UK Date
# split to parts
my @dte = split(/\//, $_[0]);
Where does $_ come from? Now might be a good time to start using
PerlCritic because it's a pain later to try and clean up code.
# rebuild to yank date
my $yank = $dte[2]-$dte[1]-$dte[0];
Where's this date coming from? The database? Look at inflating all dates
and times to DateTime objects.
Then the rendering of that object becomes a View issue.
How do you see the timezone? That's also a rendering/View issue.
It's not pretty, but what I tend to do in the View (say, in a Template) is:
Event Time: [% set_user_time_zone( event.start_time ).strftime( loc(
'_time_with_zone' ) ) %]
set_user_time_zone clones the DateTime object and sets the timezone and
locale on the DateTime objet. The _time_with_zone is a string that gets
translated -- although mostly to %c, %x, and %X. But, by using
localization it can be anything.
# return result
$yank;
Yes, check out PerlCritic.
}
I’m forever having to switch between UK dates and USA dates so have helper
methods.
See above.
Where do these go?
Also where do you put application global constants?
Not a clear line, but you have an app config for app-specific config. But,
I also have App::Constants that export constats which is useful for when
you need the constants outside of Catalyst.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Use of uninitialized value in delete
On Sat, Oct 27, 2012 at 9:34 PM, Larry Leszczynski lar...@emailplus.orgwrote:
Hi Bill -
On Fri, Oct 19, 2012, at 11:37 AM, Bill Moseley wrote:
Use of uninitialized value in delete
with a line number pointing to this line:
delete $c-stash-{foo};
I didn't think that delete() issued a warning,
Are both $c and $c-stash defined at that point?
Those would be different errors in Perl:
$ perl -wle 'my $c; $c-foo'
Can't call method foo on an undefined value at -e line 1.
$ perl -wle 'sub Foo::stash { return {} }; $s = bless {}, Foo; delete
$s-stash-{bar}'
$ perl -wle 'sub Foo::stash { undef }; $s = bless {}, Foo; delete
$s-stash-{bar}'
Can't use an undefined value as a HASH reference at -e line 1.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Thu, Oct 25, 2012 at 5:29 AM, Aaron Trevena aaron.trev...@gmail.comwrote: On 24 October 2012 17:59, Bill Moseley mose...@hank.org wrote: PerlBal (as in this old post: http://lists.danga.com/pipermail/perlbal/2005-November/000138.html ) can do this as well. I wonder about the topology. We used to run with Perlbal (and heartbeat and IP failover) in front of a pool of web servers. We now run with hardware load balancers in front of a pool of web servers. The load balancer does make it easy to adjust the pool -- as well as gracefully handle a web server dropping out of the pool. I don't want to add yet another set of servers for an extra proxy layer. So, I'm currently thinking of running Nginx on each web server. (Keep-alive between the load balancer and Nginx, and no keep-alive between Nginx and Catalyst with maybe Starman.) Um... how is adding nginx instead of perlbal not adding yet another set of servers? I'm talking hardware. I'd rather not add another set of servers between the load balancer and the web pool. It's just more to manage and monitor and one more place to fail. We used to run Perlbal like that when we had less capable load balancers. If I put Nginx/Perlbal on each web server it simplifies the architecture. No need to run redundant Perlbal/Nginx with heartbeat and failover on a separate set of servers in each data center. What features of nginx are you looking to use vs say perlbal - depends on how you'd use it and what for, and how easily either would acheive your goals easily - perlbal *could* have a short/shallower learning curve, or nginix may be drop-in job that just works without any customisation or special extensions Obviously, I'm looking for working progress bar, but also upload buffering so the web app processes are not tied up waiting on client uploads. Of course, scaling extra web processes is pretty easy which is why we haven't worried too much about buffering uploads. Haven't had the need to reproxy. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Wed, Oct 24, 2012 at 5:44 AM, Lorn l...@lornlab.org wrote: I'm not following the thread but, did you guys know about http://wiki.nginx.org/HttpUploadProgressModule. ? PerlBal (as in this old post: http://lists.danga.com/pipermail/perlbal/2005-November/000138.html ) can do this as well. I wonder about the topology. We used to run with Perlbal (and heartbeat and IP failover) in front of a pool of web servers. We now run with hardware load balancers in front of a pool of web servers. The load balancer does make it easy to adjust the pool -- as well as gracefully handle a web server dropping out of the pool. I don't want to add yet another set of servers for an extra proxy layer. So, I'm currently thinking of running Nginx on each web server. (Keep-alive between the load balancer and Nginx, and no keep-alive between Nginx and Catalyst with maybe Starman.) Anyone see why this might be a bad (or good) approach? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Sat, Oct 20, 2012 at 1:51 PM, Tomas Doran bobtf...@bobtfish.net wrote: And UploadProgress is shipped, should be available once it's reindexed (permissions cock up), which should be shortly :) So, when running under Starman the uploads are buffered before chunked to Catalyst, which means the progress bar data isn't updated until the upload has completed. This renders the upload progress bar pretty useless with Starman. The progress bar works fine running the app under mod_perl. I suppose using something like Nginx or Perlbal in front of the app would work (because those do cache uploads but also provide a hook for reading upload progress). But, we already have hardware load balancers in front of the app, so don't really need an extra proxy in front of every web server. Any other options? Using a upload/request caching proxy is probably THE correct answer since don't really want to tie up the app with slow uploads. I guess I should test, but I wonder if there's a limit on what Starman will buffer -- I assume it's buffering in memory. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Use of uninitialized value in delete
Sorry for the duplicate if you are on the SF Perl list.
In server logs I'm seeing this warnings:
Use of uninitialized value in delete
with a line number pointing to this line:
delete $c-stash-{foo};
I didn't think that delete() issued a warning, and I can't seem to make it
happen:
$ perl -Mwarnings -Mstrict -Wle 'use warnings; use strict; my $x = {};
delete $x-{a}'
Is there something special about $c-stash that might trigger this in Perl?
Anyone else spot these in the logs.
This is on Perl 5.14.2 / Catalyst 5.90016 / mod_perl 2.0.7
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Wed, Oct 17, 2012 at 12:34 AM, Tomas Doran bobtf...@bobtfish.net wrote: Can I get a 'yes this new code works' from someone trying it before we ship new versions? :) Yes it works like it used to, now. Thanks. From what I remember this can't work on Safari and Chrome (Webkit). Need to use the iframe trick on those browsers. https://bugs.webkit.org/show_bug.cgi?id=23933 How close is this version of Catalyst for a release? Unfortunately, I noticed this while preparing for an app release next week. Thanks, -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Wed, Oct 17, 2012 at 12:34 AM, Tomas Doran bobtf...@bobtfish.net wrote: On 17 Oct 2012, at 07:13, Dimitar Petrov wrote: Hey Bill, I think t0m fixed that a week ago in trunk and it should be working. You might want to check the trunk of the Catalyst and the Plugin and to see if it works for you and probably ask t0m to upload new version on CPAN. This needs a Catalyst release first - I was waiting till there were a few more changes for that, but if it's hurting people now, we could do a new version with just the tiny changes I made to support this. Can I get a 'yes this new code works' from someone trying it before we ship new versions? :) I'll try and test today. Thanks for addressing this before I even was aware it was a problem. ;) Is $c-prepare_body_chunk every called anymore? I'm wondering if that should be removed so that if someone's app is wrapping that they would see an error. That would identified this issue sooner, perhaps. I was looking at a separate issue where large uploads were stalling. That's when I noticed the progress bar not working. I saw in the logs was this: $ fgrep Read error_log Read error: Read error: Read error: Read error: followed immediately by a restart of the child process (testing with Starman). So, that's something else I need to try and track down. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Progress bar
On Wed, Oct 17, 2012 at 12:34 AM, Tomas Doran bobtf...@bobtfish.net wrote: This needs a Catalyst release first - I was waiting till there were a few more changes for that, but if it's hurting people now, we could do a new version with just the tiny changes I made to support this. By the way, I wonder if it wouldn't be cleaner to use _build_request instead of a builder for the args. In this case might have to completely override _build_request to inject in the cache -- or make the cache rw so that it can be set after _build_request. Seems like giving the request the app as an attribute would be a bad idea, but not clear what that is. :) I've been struggling a bit with hierarchy of classes where I need to pass child construction params down to child constructors. I've added parent attributes to the child classes, manually passed args (as you are doing here), and even had separate config objects that I pass around. Haven't found anything that I'm that fond of. Can I get a 'yes this new code works' from someone trying it before we ship new versions? :) Cheers t0m ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Progress bar
At one point Catalyst::Engine did this:
# Check for definedness as you could read '0'
while ( defined ( my $buffer = $self-read($c) ) ) {
$c-prepare_body_chunk($buffer);
}
Which meant that Catalyst::Plugin::UploadProgress could wrap/override
$c-prepare_body_chunk.
The code now does this ( in Catalyst::Request ):
while ( defined ( my $buffer = $self-read() ) ) {
$self-prepare_body_chunk($buffer);
}
Which breaks that functionality.
How can I get the upload byte count into $c-cache so that my AJAX request
can get updated upload stats?
Thanks,
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] thoughts on a catalyst roadmap
On Thu, Oct 11, 2012 at 8:04 AM, John Napiorkowski jjn1...@yahoo.comwrote: http://jjnapiorkowski.typepad.com/modern-perl/2012/10/thoughts-on-a-catalyst-roadmap-1.html I tend to describe Catalyst as glorified dispatcher and object instance manager. The goal there is to make people think about thin controllers and lots of models. Controller bloat tends to happen if not careful. Plugins are big, but looking at our apps we probably use more custom plugins than CPAN ones. Granted, mostly CPAN inspired but end up customizing for our own needs. Obviously, being able to easily hook into the request process is huge. I agree with your thoughts about REST and web sockets. Much (most?) of our development is REST based. I've written two REST replacement systems now (Controller base class and Action class). One is very similar to Catalyst::Action::Rest, but much more heavy weight with CRUD and a bit of magic built in, and another rather light weight based on chaining. Views are becoming less important/interesting since more of that is client-side and much is just returned as JSON. Pain points I have are around scalability. I'd like to have our app be made up of many small apps because dev on a very large app can get slow. Having separate apps isn't too hard, although it's a bit more of a pain for rendering templates and sharing a standard layout across separate apps. I've also been looking at ways to manage features -- or really a way to control features dynamically. In other words, all developers work adding features in trunk and always release from trunk. And have a way to dynamically turn features on or off -- or on to just a sub-set of accounts/IP addresses/etc. That's mostly looking for ways to make the build, test, release process less painful -- and less expensive. And releases more frequent and less anxious. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Trapping added errors with the correct caller
On Thu, Oct 11, 2012 at 5:38 AM, Robert Rothenberg rob...@gmail.com wrote:
I would like to trap every error added to $c-error() and log it, noting
the
caller (filename, line number) in the logs.
I've not gotten Catalyst::Plugin::ErrorCatcher to work, so I wrote my own
plugin that overrides $c-error with the following method:
use MRO::Compat;
use namespace::autoclean;
sub error {
my ($c, @args) = @_;
foreach my $arg (@args) {
if ($arg) {
$c-log-error($arg);
}
}
Isn't that what finalize already does?
I have a very old and out-dated plugin I use that wraps execute() and sets
__DIE__ and __WARN__ to catch those and uses Devel::StackTrace to add in a
stack trace. Then I use log4perl to format and direct the messages.
I also use Moose's Throwable which includes a stack trace.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Better Testing
Anyone using Test::Sweet or Test::Class(::Most) with your large Catalyst apps? Opinions about either -- or lessons learned? Anything else to look at? We have tended to write pretty standard procedural-style tests which are easy to understand across our team (for a few dozen developers) and often end up factoring out common code into utility classes. But, those utility classes are often one-offs and at the whim of the developer how to structure it. I do want something more standardized but easily extensible. I'm finding Test::Class works great for unit tests -- especially when testing related sub-classes -- but we tend to write more feature/integration tests. In other words, tests are longer and depend on previous tests. I guess another term might be workflow tests. I want a framework that enforces good structure, and also that makes it easy to run single tests (that is, run a single tests within a single .t file) to make debugging faster. (e.g. Run the first five tests of 20.) On the other side, and perhaps unrelated to above, but I do want to make sure our tests can be run in parallel so we can run full-test more often and take advantage of our larger multi-core machines. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Modify config in a test
On Wed, Sep 26, 2012 at 1:15 AM, Manni Heumann heum...@strato-rz.de wrote: Bill Moseley mose...@hank.org schrieb am 25.09.2012: The app has a myapp.yml config which includes configuration for creating an instance of a Model component -- and part of that Model's config is a database dsn attribute. When running my test I do want to have the app load this config file -- but I want to modify it on-the-fly. Why? Why not simply create a myapp_test.yml file and set CATALYST_CONFIG_LOCAL_SUFFIX to test when you run your test-suite? I guess this is probably the cleanest/easiest. Create my database and write out the config in a BEGIN block -- or somehow otherwise delay loading Catalyst::Test. My other option could be to create my own instance of the Model object and then swap it out in the MyApp-components hash. The fact that my model is dynamically generated at startup based on config adds a bit of complexity to it -- but that can be handled. BTW -- for tests where I need to build a database on-the-fly what I currently do is set my make test config use dsn = 'dbi:Pg' and then set the PG* environment variables. But, in this case what I'm doing is swapping my Postgresql config in my configuration with one to use a SQLite database in a temporary file just for a few tests. Thanks for the ideas. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Modify config in a test
I'm using Catalyst::Test on an app. The app has a myapp.yml config which includes configuration for creating an instance of a Model component -- and part of that Model's config is a database dsn attribute. When running my test I do want to have the app load this config file -- but I want to modify it on-the-fly. My test builds a SQLite database when it runs, and I'd like to be able to modify the config as (before) Catalyst creates the instance of the model component -- that is just filter the config to inject my temporary database. The model dsn attribute is readonly -- otherwise I could just do MyApp-model( 'MyModel' )-dns( $new_dsn ); What's a good, clean way of doing this? Or perhaps a better approach all together? -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Best-practices question: caching a search
On Sat, Sep 15, 2012 at 7:06 PM, will trillich will.trill...@serensoft.comwrote: Hi Francisco -- I'm not talking about paginating a resultset, I'm talking about returning to a previously-established resultset on some future HTTP request. Here's the scenario: You are asking just how to display multiple pages of search results? 12:01 Bob submits search form for Chicago between 1-Apr-2012 and 30-Apr-2012 Use a GET, not a POST. I'm Ignoring timezones, but you should not. http://example.com/event_list?city=Chicagoafter=2012-04-01before=2012-04-30rows=100page=1http://example.com/event_list?city=Chicagobetween=2012-04-01:2012-04-30rows=100page=1 12:02 Bob sees first page of results, records 1-100 12:03 Bob clicks to see second page of results, records 101-200 = how do we re-establish the recordset here, from the original search form at 12:01? http://example.com/event_list?city=Chicagoafter=2012-04-01before=2012-04-30rows=100page=http://example.com/event_list?city=Chicagobetween=2012-04-01:2012-04-30rows=100page=1 2 12:04 Bob clicks thru to see detail on record 135 12:05 Bob clicks breadcrumbs to return to search = how do we re-establish the recordset here, from the original search form at 12:01? http://example.com/event_list?city=Chicagoafter=2012-04-01before=2012-04-30rows=100page=1http://example.com/event_list?city=Chicagobetween=2012-04-01:2012-04-30rows=100page=1 All independent requests -- someone may bookmark page 2 and go back there directly. Then (later) think about caching. That depends on usage, your database load, your SLA, your traffic, load, etc. You could fetch the entire result list on the first request and cache, for example, or you could instead cache the entire page (with a separate page cache layer). Use a cache, not the session, for caching. There's nothing here related to the session unless you wan to display something like recent searches to show on all pages -- and even that might be better in the cache keyed by user's ID if users are required to log in. On Sat, Sep 15, 2012 at 2:55 PM, Francisco Obispo fobi...@isc.org wrote: Some databases provide means to return a specific set of records, and even an offset, In DBIx::Class, when you search, you can actually specify the page as an option [1], if you're not querying against a database, you might want to use something like Memcached or the like to store your resultset and paginate accordingly. [1] http://search.cpan.org/dist/DBIx-Class/lib/DBIx/Class/ResultSet.pm#ATTRIBUTES On Sep 15, 2012, at 11:41 AM, will trillich will.trill...@serensoft.com wrote: User enters some search parameters (location, date-range, etc). Gets 500 results which we paginate. Once the user pages to the item of interest, he/she can then click thru to edit or see more detail. It'd be nice to have 'breadcrumbs' that take the user back to that page of that search. What's the recommended way of doing that? A) stash the whole recordset into the session (can you even serialize/deserialize a recordset object?) B) stash the search params and page-no and page-size and recreate the recordset each time C) ...something else? -- Will Trillich :: 812.454.6431 “Waiting for perfect is never as smart as making progress.” -- Seth Godin ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ Francisco Obispo email: fobi...@isc.org Phone: +1 650 423 1374 || INOC-DBA *3557* NOC PGP KeyID = B38DB1BE ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Will Trillich :: 812.454.6431 “Waiting for perfect is never as smart as making progress.” -- Seth Godin ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Module::Install::Catalyst replacement
I'm using Dist::Zilla and creating a Plugin to replace what
Module::Install::Catalyst does. By default all files in home are copied
to blib (to allow installing) except the following patterns:
our @IGNORE =
qw/Build Build.PL Changes MANIFEST META.yml Makefile.PL Makefile README
_build blib lib script t inc .*\.svn \.git _darcs \.bzr \.hg
debian build-stamp install-stamp configure-stamp/;
I'm thinking of reversing that logic and having a Dist::Zilla plugin
that *explicitly
defines* what is copied. So, any file in home that match these patterns
would be included:
^root$
^Changes$
\.yml$
\.conf$
\.inc$
\.psgi$
Is there anything else that should be in that default list of patterns to
include?
And, of course, allow to add files/dirs to that list via dist.ini
configuration:
; replaces [MakeMaker]
[CatalystFiles]
include = conf
include = config.yml
I'm still not convinced this is the best approach, though. Copying the
files when Makefile.PL runs always seems a bit of a kludge -- compared to,
say, having make copy the files like a normal Makefile.
And one could argue that these files and directories should go into share
in the distribution and then get installed as File::ShareDir expects (i.e.
have Catalyst::Utils::home() use dist_dir( $c-config-{name} ) ). But,
that's kind of a big change.
Anyone have thoughts on the that?
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Applying roles that contain actions.
On Fri, Sep 7, 2012 at 11:50 AM, Tomas Doran bobtf...@bobtfish.net wrote:
On 5 Sep 2012, at 21:48, Bill Moseley wrote:
Currently I just manually apply the role directly in the existing
controller, so not a huge issue, but would be handy if I could just add a
role to the base class and pull in all the components
What's stopping you doing this?
Sorry, I wasn't very clear in that sentence. It's a bit more involved
than this, but I'm looking at applying a controller role (one with actions)
dynamically. So, the code ends up something like:
package MyApp;
after setup_components = sub {
my $self = shift;
my $controller $self-controller( 'Foo' );
use My::Role::With::Actions;
My::Role::With::Actions-meta-apply( $controller );
Results in:
Composing MooseX::MethodAttributes::Role::Meta::Role::Application onto
instances is unsupported
That makes sense to me -- for one thing Catalyst would not know about the
actions in the role.
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Applying roles that contain actions.
I've been injecting models into Catalyst apps (either using CatalystX::InjectComponent or just stuffing an instance into $app-components hash). I'd also like to be able to apply roles to existing controllers. If I try to -apply the role it fails if using MooseX::MethodAttributes::Role. Which I assume is because the actions would not be registered with Catalyst. Is there a way to inject actions in this way? Currently I just manually apply the role directly in the existing controller, so not a huge issue, but would be handy if I could just add a role to the base class and pull in all the components -- or dynamically add the actions. In general, what I'm after is a way to specify with App::Feature::Widget; in my app base class and bring in an entire feature. I'm also looking at a gatekeeper type of feature control that Facebook is known for: https://www.facebook.com/video/video.php?v=10100259101684977oid=9445547199comments -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] [JOB] Oakland, CA, USA
Catalyst web app developer job posting: http://jobs.perl.org/job/16350 -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] working around request timeouts
On Mon, Aug 20, 2012 at 12:10 PM, Sergey Dmitriev sergey.program...@gmail.com wrote: Looks like what you need is the job queue. Take a look at Queue::DBI (https://metacpan.org/module/Queue::DBI) - simpler TheSchwartz (https://metacpan.org/module/TheSchwartz) Yes, and what about long polling? Would that solve the ajax request timing out? That is, have your client code get sent a message when the backend is done instead of waiting. Sergey 2012/8/20 James R. Leu j...@mindspring.com Hello all, Problem description === I have a catalyst application server that responds to 'API' requests for web applications via XHTMLRequests. Sometimes the requests are timing out due to the backend database queries taking too long. I'm looking for ways to work around this and prevent the 'API' requests from timing out. I know some of the possible resolutions to this are - fix the queries - fix the database - frontend the RDBMS with NoSQL I'm working towards those fixes, but they are long term projects, I'm looking for an interim solution. That would notify the web application that it will need to come back later for the response (ie decouple request handling from the actual request/response). My attempt == In my handler I fork a child process. In the parent I send a response with a 'job id' so the web application knows to poll the 'API' for completion. In the child I close the IO socket so it cannot send a response and then let it finish processing the request, but it looks like I've lost my database connections so the request fails. My wish === What I would like to do is avoid the fork and instead have the handler send an early response to the web application and then finish processing the request and not try to send a response when done. Is there a common term for what I'm trying to do like continuation or something like that? Has anyone already done this? Thank you for your time. -- James R. Leu j...@mindspring.com ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Parsing of undecoded UTF-32 at /home/davidw/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst/Test.pm
http://cpansearch.perl.org/src/BOBTFISH/Catalyst-Runtime-5.90015/lib/Catalyst/Test.pm In the mangle_response sub the code does this: require HTML::HeadParser; my $parser = HTML::HeadParser-new(); $parser-xml_mode(1) if $resp-content_is_xhtml; $parser-utf8_mode(1) if $] = 5.008 $HTML::Parser::VERSION = 3.40; $parser-parse( $resp-content ); Shouldn't that code check that the content-type is HTML before attempting to parse? For example, this response was causing problems: Cache-Control: no-store, no-cache, must-revalidate Content-Length: 24043 Content-Type: audio/mp4a-latm Expires: Wed, 31 Dec 1969 23:59:59 GMT Content-Disposition: attachment; filename*=UTF-8''trumpet.m4a (Warnings are fatal): Parsing of undecoded UTF-32 at /home/moseley/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/Catalyst/Test.pm line 314. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Change in $c-engine-prepare_parameters
I'll work on a patch and docs, yes. On Sat, Aug 4, 2012 at 6:53 AM, Tomas Doran bobtf...@bobtfish.net wrote: On 3 Aug 2012, at 14:39, Bill Moseley wrote: What it doesn't do any more is set $c-req-parameters when called directly, only when called as a builder. I don't think prepare_parameters should be both a builder AND a method. What about this approach: +1, except I think the clearer should be _clear_parameters - we don't want to add a new public method for this if it can be avoided. Fancy making a 'proper' patch someone can apply which does these changes and updates the relevant doc? Cheers t0m ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Change in $c-engine-prepare_parameters
See attached. Let me know if you want something else. I'm mixed on of the clear should be private. I made it that way, but I wonder if there could be a use case to reset the parameters so next time $c-req-parameters is called they get built again. I guess in that case why wouldn't they just call $c-prepare_parameters again? On Sat, Aug 4, 2012 at 6:53 AM, Tomas Doran bobtf...@bobtfish.net wrote: On 3 Aug 2012, at 14:39, Bill Moseley wrote: What it doesn't do any more is set $c-req-parameters when called directly, only when called as a builder. I don't think prepare_parameters should be both a builder AND a method. What about this approach: +1, except I think the clearer should be _clear_parameters - we don't want to add a new public method for this if it can be avoided. Fancy making a 'proper' patch someone can apply which does these changes and updates the relevant doc? Cheers t0m ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Bill Moseley mose...@hank.org prepare_parameters.diff Description: Binary data live_engine_request_prepare_parameters.t Description: Troff document ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Change in $c-engine-prepare_parameters
I have some older code that calls $c-prepare_parameters after making some
modification to the request. If that's an acceptable use is questionable,
but we have a change in behavior regardless from 5.90007 to 5.90008-TRIAL.
Essentially, now if you do some thing a bit odd like this:
$c-req-parameters( {} );
$c-prepare_parameters;
Then $c-req-parameters stays an empty hash. Previous to 5.90008-TRIAL
this would prepare the parameters again.
The documentation in C::Engine says:
=head2 $self-prepare_parameters($c)
sets up parameters from query and post parameters.
which is only true now when $req-parameters has not already been called.
And in 5.90008-TRIAL the prepare_parameters was moved to Catalyst::Request,
with the documentation:
=head2 $self-prepare_parameters()
Ensures that the body has been parsed, then builds the parameters, which are
combined from those in the request and those in the body.
What it doesn't do any more is set $c-req-parameters when called
directly, only when called as a builder.
I don't think prepare_parameters should be both a builder AND a method.
What about this approach:
Change the Request parameters attribute to have a builder and a clearer:
has parameters = (
is = 'rw',
lazy = 1,
builder = '_build_parameters',
clearer = 'clear_parameters',
);
Then rename the existing prepare_parameters to _build_parameters,
because that's what it is, and then add a new method:
sub prepare_parameters {
my $self = shift;
$self-clear_parameters;
return $self-parameters;
}
And likewise in the Engine.pm:
sub prepare_parameters {
my ( $self, $c ) = @_;
$c-request-clear_parameters;
return $c-request-parameters;
}
--
Bill Moseley
mose...@hank.org
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Why does the installer include the 'xt' directory
On Wed, Jul 25, 2012 at 3:05 PM, Robert Rothenberg rob...@gmail.com wrote: Doc patch please? :) I'm not sure what it is. It doesn't work with regular expressions, but it doesn't behave like proper file globs either. next CATFILES if $name =~ /^$ignore$/; catalyst_ignore( 'xt' ); -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Why does the installer include the 'xt' directory
On Tue, Jul 24, 2012 at 4:28 AM, Robert Rothenberg rob...@gmail.com wrote: Everything works fine, however, Module::Install::Catalyst installs the 'xt' directory in lib/perl5/MyApp. Why does it do this, and how does one tell it not to? Try this: http://search.cpan.org/~bobtfish/Catalyst-Devel-1.37/lib/Module/Install/Catalyst.pm#catalyst_ignore(@ignore) -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Why does the installer include the 'xt' directory
On Tue, Jul 24, 2012 at 9:35 AM, Robert Rothenberg rob...@gmail.com wrote: Try this: http://search.cpan.org/~bobtfish/Catalyst-Devel-1.37/lib/Module/Install/Catalyst.pm#catalyst_ignore(@ignore) Maybe run make distclean first? You are calling catalyst_ignore before calling catalyst in your Makefile.PL? If those are done then this always works.. :) vim $(perldoc -l Module::Install::Catalyst) directory. -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Extending Catalyst::Script::Server to proxy
I need a little Plack help. I have a web app that is running as two apps -- one is Catalyst and the other, well, isn't. What I'd like is a script/myapp_proxy.pl script that works just like script/ myapp_server.pl but mounts /foo to the Catalyst app and all other requests get proxied via Plack::App::Proxy. For example: $ script/myapp_proxy.pl --port 5000 --proxy_to http://localhost:6000/ Then GET http://localhost:5000/foo/whatever is handled by the Catalyst app, but anything else (GET http://localhost:5000/other) is reverse-proxied to localhost:6000. I was hoping to use the existing framework for running scripts -- mostly so the script works where ever the Catalyst app is installed. Can anyone help with this? Thanks, -- Bill Moseley mose...@hank.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
