Re: [Catalyst] UTF-16 surrogate message when writing binary data(image)
I've found a case ...stderr: UTF-16 surrogate 0xdb71 at /usr/lib/perl/5.8/IO/Handle.pm line 199... FastCGI: ... [info] picture size is 55966 55966 = da9e 56177 = db71 So almost the same. Perhaps I have 100 bytes of headers. 2008/5/20 Martin Ellison [EMAIL PROTECTED]: The Content-Type is set to 'UTF-8' for most of my pages, but, in this case, it is image/jpeg and the content disposition is set to inline. 2008/5/19 Ash Berlin [EMAIL PROTECTED]: On 19 May 2008, at 12:34, Martin Ellison wrote: I've finally had the opportunity to test my code a bit more. The $pic_image is a string of some kind and not a file handle. Also I tried doing binmode STDOUT but this fails -- the binmode call returns false. There could be any binary data in my string, so any UTF-8-bad byte pairs are definitely possible. I would think the solution is to inform Catalyst that the data is not UTF-8 but binary. In the normal case of writing to a file, one can use binmode to tell perl that the data is binary but that does not work here. Is this a gap in the Catalyst API? Are you by any chance setting a header to something with UTF8 content? I had a problem similar to this that was caused by me setting a Content-Disposition header to something including unicode. -ash ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: new catalyst version breaks test in Catalyst-Model-Adaptor-0.02 and Catalyst-Plugin-Unicode-0.8
On Mon, May 19, 2008 at 5:35 PM, John Goulah [EMAIL PROTECTED] wrote: Hi folks, I've narrowed down that the new version (5.7013) breaks tests in both of these modules, but I'm not sure why (they work against the last version of Catalyst 5.7012, so there has been some change thats affected this) Heres the test output, as you can see its looking for the wrong output. Catalyst-Model-Adaptor-0.02 t/00-loadok t/live-test..1/47 [error] No default action defined # Failed test 'get main page' # at t/live-test.t line 17. # 500 # Internal Server Error # Failed test 'see if it has our text' # at t/live-test.t line 18. # got: !DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Trans... # length: 3747 # doesn't match '(?i-xsm:it works)' # Looks like you failed 2 tests of 47. t/live-test.. Dubious, test returned 2 (wstat 512, 0x200) Failed 2/47 subtests Test Summary Report --- t/live-test.t (Wstat: 512 Tests: 47 Failed: 2) Failed test number(s): 2-3 Non-zero exit status: 2 Files=2, Tests=51, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.52 cusr 0.08 csys = 0.62 CPU) Result: FAIL Failed 1/2 test programs. 2/51 subtests failed. make: *** [test_dynamic] Error 2 - Catalyst-Plugin-Unicode-0.8 t/01use..ok t/02pod..skipped: set TEST_POD to enable this test t/03podcoverage..skipped: Test::Pod::Coverage 1.04 required t/live-test..1/12 [error] No default action defined # Failed test 'get main page' # at t/live-test.t line 22. # 500 # Internal Server Error # Failed test 'see if it has our text' # at t/live-test.t line 23. # got: !DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Trans... # length: 3747 # doesn't match '(?i-xsm:it works)' # Looks like you failed 2 tests of 12. t/live-test.. Dubious, test returned 2 (wstat 512, 0x200) Failed 2/12 subtests Test Summary Report --- t/live-test.t(Wstat: 512 Tests: 12 Failed: 2) Failed test number(s): 2-3 Non-zero exit status: 2 Files=4, Tests=13, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.44 cusr 0.08 csys = 0.54 CPU) Result: FAIL Failed 1/4 test programs. 2/13 subtests failed. make: *** [test_dynamic] Error 2 mst thinks this is almost certainly an error screen, anyone care to comment? Thanks, John ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] UTF-16 surrogate message when writing binary data(image)
Another case... the surrogate is d862 = 55394 but the image size is 45296, so it is not looking consistent. So perhaps the 'surrogate' data is just random bytes from the JPEG. 2008/5/23 Martin Ellison [EMAIL PROTECTED]: I've found a case ...stderr: UTF-16 surrogate 0xdb71 at /usr/lib/perl/5.8/IO/Handle.pm line 199... FastCGI: ... [info] picture size is 55966 55966 = da9e 56177 = db71 So almost the same. Perhaps I have 100 bytes of headers. 2008/5/20 Martin Ellison [EMAIL PROTECTED]: The Content-Type is set to 'UTF-8' for most of my pages, but, in this case, it is image/jpeg and the content disposition is set to inline. 2008/5/19 Ash Berlin [EMAIL PROTECTED]: On 19 May 2008, at 12:34, Martin Ellison wrote: I've finally had the opportunity to test my code a bit more. The $pic_image is a string of some kind and not a file handle. Also I tried doing binmode STDOUT but this fails -- the binmode call returns false. There could be any binary data in my string, so any UTF-8-bad byte pairs are definitely possible. I would think the solution is to inform Catalyst that the data is not UTF-8 but binary. In the normal case of writing to a file, one can use binmode to tell perl that the data is binary but that does not work here. Is this a gap in the Catalyst API? Are you by any chance setting a header to something with UTF8 content? I had a problem similar to this that was caused by me setting a Content-Disposition header to something including unicode. -ash ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: new catalyst version breaks test in Catalyst-Model-Adaptor-0.02 and Catalyst-Plugin-Unicode-0.8
John Goulah wrote:
On Mon, May 19, 2008 at 5:35 PM, John Goulah [EMAIL PROTECTED] wrote:
Hi folks,
I've narrowed down that the new version (5.7013) breaks tests in both
of these modules, but I'm not sure why (they work against the last
version of Catalyst 5.7012, so there has been some change thats
affected this)
mst thinks this is almost certainly an error screen, anyone care to comment?
This was a regression I created when trying to solve the Path(0) bug.
Essentially Path by itself means Path('') which is defined but of 0
length.
Fixed in svn r7778 (+ test).
-Brian
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] UTF-8 Conversion on the fly
Hello Everyone, Got a question about utf8 conversions on the fly. I was wondering if anyone had experience and some pointers of good practices for taking a webpage thats stored as a blob in mysql then serving that page after its been converted to utf8. Right now the way this is implemented is conversion before storage but I never liked that idea. The engineer who did it thought utf8 conversion is always going to be a smooth thing. I also don't like the fact we don't store the orginal copy of the page in the db its the converted copy. So I wanted to see if anyone here had good suggestions on how the original can be stored and just converted on the fly. Thanks, -- Ali Mesdaq (CISSP, GIAC-GREM) Security Researcher II Websense Security Labs http://www.WebsenseSecurityLabs.com -- Protected by Websense Messaging Security -- www.websense.com ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] UTF-8 Conversion on the fly
* On Fri, May 23 2008, Mesdaq, Ali wrote:
Hello Everyone,
Got a question about utf8 conversions on the fly. I was wondering if
anyone had experience and some pointers of good practices for taking a
webpage thats stored as a blob in mysql then serving that page after its
been converted to utf8. Right now the way this is implemented is
conversion before storage but I never liked that idea. The engineer who
did it thought utf8 conversion is always going to be a smooth thing. I
also don't like the fact we don't store the orginal copy of the page in
the db its the converted copy. So I wanted to see if anyone here had
good suggestions on how the original can be stored and just converted on
the fly.
I'm not sure what this has to do with Catalyst, but just store the
original data and the encoding in the database, then
my $octets =
Encode::encode('utf-8', Encode::decode($stored_encoding, $stored_webpage));
Regards,
Jonathan Rockway
--
print just = another = perl = hacker = if $,=$
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Ajax, Jemplate, forms, rest data validation
On Fri, May 23, 2008 at 3:10 PM, Henry Drinkwater [EMAIL PROTECTED] wrote: Hi, I'm new to Catalyst so apologies if this is a dumb question. I'm working on a web site using Catalyst and want to use quite a bit of Ajax interactivity to make the site as user friendly and modern as possible. The majority of my knowledge of Catalyst comes from the Jonathon Rockway book, so I have so far been concentrating on using Jemplate for the AJAX, and using a REST interface to handle the data serialization. This is working very nicely for the retrieve/get part, but I am struggling a bit to figure out the best way of handling forms and the submission validation of the data. The formbuilder controller makes it incredibly simple to build and validate a form, on both the client and the server, and I want to have similar functionality but AJAX enabled - i.e. I envisage the form submission method being a PUT to a REST controller, and for the submission to cause a div to be updated in place, rather than a load of a new page. In the book the example uses a POST to a pre-existing formbuilder controller as the submission method of the ajax form, which is neat in that it handles the data validation part without writing any code, but not so good in that it is not really ajax-y, as it results in loading a new page. The book also mentions putting data validation into the Model and not in the controller, which I guess would work fine but will require quite a bit more code to be written. I'd rather not have to write the validation myself if there's a good module out there which would do it for me, based on the form definition. So I'm looking for recommendations as to how to handle this - would I be better taking the time to learn one of the javascript libraries (e.g. YUI or Dojo) and moving away from Jemplate, or is there a way to combine formbuilder and Jemplate in a more dynamic ajax-y way? If anyone has any suggestions for other tutorials or example code I could look at to get more ideas of how to do this, I'd be very grateful. regards, Henry ps I have been following the thread on RFC: Catalyst::Controller::REST::DBIC on the list, which seems to be talking about stuff along these lines, but I haven't really gleaned any ideas on how best to tackle this. I also found Jay Shirley's slides on No REST for the Wicked: Catalyst, REST and YUI which sounded very relevant, but again I didn't learn enough to really get me coding. pps I found the book incredibly helpful, once I found the errata web site and got past chapter 3. It got me very quickly over the learning curve on Catalyst and writing real code. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ Henry, At this point I'd advise you to look through the examples/ dir in the Catalyst SVN and you'll get more applicable knowledge. http://dev.catalystframework.org/repos/Catalyst/trunk/examples/ My talk on REST coincides with principles that are coded up in http://dev.catalystframework.org/repos/Catalyst/trunk/examples/RestYUI (although the application name is AdventREST, it is modified from the original Advent calendar entry to work with YUI). On the validation note, I use Data::FormValidator in conjunction with a DBIx::Class ResultSet - still in the model, in that it is outside of the Catalyst scope. Easier to unit test this stuff, too, since you don't have to muck with a controller. Lots of other folk use FormBuilder or FormFu, so it is really just picking what makes the most sense to you. Good luck, -J ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re[6]: [Catalyst] uri_for() doesn't encode to utf8 first argument
Hello, Dmitriy.
You write 23 мая 2008 г., 23:48:56:
DSS Hello, J..
DSS You write 23 мая 2008 г., 0:18:11:
JS On Thu, May 22, 2008 at 10:05 AM, Dmitriy S. Sinyavskiy [EMAIL
PROTECTED] wrote:
People? what about my patch? I heven't got received any response.
I'll be glad to now some news.
Thanks.
JS Your mailserver is rejecting mail. I suggest fixing your mail server
JS or viewing the list archives to make sure that you're not missing
JS things.
JS Thread:
JS http://www.mail-archive.com/catalyst@lists.scsys.co.uk/msg02771.html
DSS Test. Must be saved as UTF-8:
DSS =
DSS use strict;
DSS use warnings;
DSS use Test::More 'no_plan';
DSS use URI;
DSS use_ok('Catalyst');
DSS my $request = Catalyst::Request-new( {
base = URI-new('http://127.0.0.1/foo')
DSS } );
DSS my $context = Catalyst-new( {
request = $request,
namespace = 'yada',
DSS } );
DSS # test encode first argument with utf-8,
DSS {
$request-base( URI-new('http://127.0.0.1/') );
$context-namespace('');
DSS is( Catalyst::uri_for( $context, '/animal/ёж', 'чёт', { param1 = щуп
})-as_string,
DSS
'http://127.0.0.1/animal/%D1%91%D0%B6/%D1%87%D1%91%D1%82?param1=%C3%91%C2%89%C3%91%C2%83%C3%90%C2%BF',
DSS 'URI for undef action with first param as string in unicode'
DSS );
DSS }
The previous test was wrong, sorry. I've found a mistake and correct
it. Test case including check for special chars ?.
Tested today - it's working right with patch.
Test
=
use strict;
use warnings;
use Test::More 'no_plan';
use URI;
use_ok('Catalyst');
my $request = Catalyst::Request-new( {
base = URI-new('http://127.0.0.1/foo')
} );
my $context = Catalyst-new( {
request = $request,
namespace = 'yada',
} );
# test encode first argument with utf-8,
{
$request-base( URI-new('http://127.0.0.1/') );
$context-namespace('');
is( Catalyst::uri_for( $context, '/animal/ёж', 'чёт', { param1 = щуп
})-as_string,
'http://127.0.0.1/animal/%D1%91%D0%B6/%D1%87%D1%91%D1%82?param1=%D1%89%D1%83%D0%BF',
'URI for with first param as string in unicode'
);
is( Catalyst::uri_for( $context, '/??', '?', { param1 = ? })-as_string,
'http://127.0.0.1/%3F%3F/%3F?param1=%3F',
'URI for with special char ? in args and param'
);
}
--
dreelmailto:[EMAIL PROTECTED]
Dmitriy S. Sinyavskiy
Web-developerPerl, Catalyst, MSSQL
FGUE EZANTelecommunication, data transfer networks and
devices.
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
