Re: [Catalyst] utf8 / pg double encoding problem
I don't follow this. The way I understand it, C::P::Unicode should
have already gotten the input into Perl's internal format, which is
utf8. So, for us to see a sequence of chars encoded as utf8, with the
flag on, here, is what I expect.
What am I misunderstanding?
On Jan 6, 2008 12:28 AM, Andrew Rodland [EMAIL PROTECTED] wrote:
On Saturday 05 January 2008 04:54:59 pm Daniel McBrearty wrote:
well I'm damned, I thought I had this stuff working squeaky clean. But
I was wrong. I actually had two bugs cancelling each other out -
usually.
[snip]
--' [debug] abçöeü
[debug] $VAR1 = ab\x{c3}\x{a7}\x{c3}\x{b6}e\x{c3}\x{bc};
[debug] it's UTF8!
Looks like the problem is here... the utf8 flag is on, indicating that $edit
is a string of characters, rather than bytes -- but the dumper output seems
to show that these characters correspond to UTF-8 encoded bytes, instead of
the actual characters of the data -- meaning that the bytes actually stored
in the string are along the lines of ab\x{c3}\x{83}\x{c2}\x{a7}... not
good. Somewhere, your data got the utf8 flag set by assumption instead of
by decoding. $edit = decode(UTF-8, $edit) should clear it up, although
finding the original problem is probably a better idea. :)
Andrew
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
http://www.engoi.com
http://danmcb.vox.com
http://danmcb.blogger.com
find me on linkedin and facebook
BTW : 0873928131
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: [Dbix-class] Re: utf8 / pg double encoding problem
because it is utf8? shouldn't it be?
On Jan 6, 2008 1:29 AM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote:
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 00:00]:
[debug] abçöeü
[debug] $VAR1 = ab\x{c3}\x{a7}\x{c3}\x{b6}e\x{c3}\x{bc};
[debug] it's UTF8!
Err, why doesn't Dumper say ab\x{e7}\x{f6}e\x{fc}? Strange that
the first line looks correct, though.
Regards,
--
Aristotle Pagaltzis // http://plasmasturm.org/
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/[EMAIL PROTECTED]
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
http://www.engoi.com
http://danmcb.vox.com
http://danmcb.blogger.com
find me on linkedin and facebook
BTW : 0873928131
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: utf8 / pg double encoding problem
Never trust dumper or printing to your terminal to tell if something
is in UTF8 or not. Terminals try to be too smart.
This is what Devel::Peek is for. Use it. Trust that.
On Jan 6, 2008, at 1:08 PM, Aristotle Pagaltzis wrote:
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 13:30]:
On Jan 6, 2008 1:29 AM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote:
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 00:00]:
[debug] abçöeü
[debug] $VAR1 = ab\x{c3}\x{a7}\x{c3}\x{b6}e\x{c3}\x{bc};
[debug] it's UTF8!
Err, why doesn't Dumper say ab\x{e7}\x{f6}e\x{fc}? Strange
that the first line looks correct, though.
because it is utf8? shouldn't it be?
What Dumper outputs is the UTF-8 byte sequence; but the next line
says that the Unicode flag is set, so this is a character string,
not a byte string. So it’s already double-encoded.
I don’t understand why the first line looks correct though.
In any case the raw HTTP request that leads to all this would be
interesting.
Regards,
--
Aristotle Pagaltzis // http://plasmasturm.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Going through tutorial, part 2 and getting error
Does this give you any clues?
Can't locate MyAppDB/Customer.pm
Ok here is how it goes, I suspect you are using DBIx::Class
app from here on stands for the name your app, of course
You should have a Model class in your lib/app/Model/appDB.pm which
should be based on Catalyst::Model::DBIC::Schema
In this file, you will have config method where you specify the
connection parameters and the actual class that maps you DB objects.
Generally you can use the same name for the actual DB class, but with
another namespace (the actual schema mapper class will usually be
sitting directly in the lib/ directory).
In the schema mapper class, say lib/appDB.pm you will see that it is
based on DBIx::Class::Schema, here you should have a load_classes
method implemented with your tables as such:
# Cargar clases de DBIC
__PACKAGE__-load_classes({
vdcDB = [qw/
creadores
vida_creador
categorias
sub_categorias
[...]
Each entry will correspond to an individual ORM class in the appDB
subdirectory sitting right there in your main lib directory.
Each ORM class will derive from DBIx::Class directly and implement
your actual M class. Later in you C code you do stuff like:
$c-stash-{subcats_creador} = [$c-model('vdcDB::sub_categorias_creadores')
-search(
{
id_creador = $id,
},
{
join = {'sub_categorias' =
'categorias'},
prefetch = {'sub_categorias'
= 'categorias'},
},
)];
Which of course will put any Rails or PHP programmer to shame ;-] (in
fact don't know of any other ORM technology that can actually do this)
Here is a sample ORM class for your enjoyment. SPECIAL NOTE: I have
read about autoloading features of DBIx and generating the DB from the
ORM and vice versa. My recommendation is that DBIx::Class is so
powerful that defining you mappings by hand is by far the best method
(besides the fact that is very easy). There is nothing better like
good and sound database modeling (decent normalization, referential
integrity, default values, etc.) and then good and sound MANUAL ORM
mapping. Never try to push the ORM too far, even though DBIx::Class
can handle very complex things, it is usually best to resolve
complicated stuff in the DB with a View or Rule (if you're lucky or
smart enough to be working with PostgreSql ;-] ) If you actually
exploit a REAL ORDBMS (like Postgres, not the fake ones like Oracle)
things with a powerful ORM like DBIx::Class can get pretty
fascinating. Anyway, here's the example:
package vdcDB::sub_categorias;
use base qw/DBIx::Class/;
# Cosas requeridas de DBIC
__PACKAGE__-load_components(qw/PK::Auto Core/);
# Nombre de la tabla
__PACKAGE__-table('sub_categorias');
# Columnas de la tabla
__PACKAGE__-add_columns(qw/
id
id_categoria
nombre
descripcion
/);
# Clave primaria
__PACKAGE__-set_primary_key(qw/id/);
# relación clave foránea con categorías
__PACKAGE__-belongs_to(categorias = vdcDB::categorias, 'id_categoria');
# Relación 1..n con sub_categorias_creadores
__PACKAGE__-has_many(sub_categorias_creadores =
'vdcDB::sub_categorias_creadores', 'id_sub_categoria');
1;
Perl. There is no substitute.
Alejandro Imass
P.S. Sorry about the spanish comments.
On Jan 6, 2008 2:59 AM, Andrew [EMAIL PROTECTED] wrote:
A newbie getting back into perl and decided to start using it w/ Catalyst.
I'm going through the tutorial, part 2 and modifying it by changing the
Models, and slightly different presentation, but the structure of the data
is the same (e.g. instead of a Book object I'm using a Customer object, but
it's the same in that a Book can have many authors, a Customer can have many
credit cards). So I'm making the example more applicable to what I'd to use
it for and basically renaming stuff.
When I run script/myApp_server.pl I get the following error:
Couldn't instantiate component register::Model::registerModel, Cannot
load schema class 'registerDB': DBIx::Class::Schema::throw_exception():
Can't locate MyAppDB/Customer.pm in @INC (@INC contains:
/mnt/hgfs/mac/Desktop/share/catalyst/webpage/apps/register/script/../lib
/home/owner/local/lib/perl5 /home/owner/local/share/perl/5.8.8
/home/owner/local/lib/perl/5.8.8 /home/owner/local/lib /etc/perl
/usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5
/usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8
/usr/local/lib/site_perl .) at (eval 110) line 3.
Compilation failed in require at
[Catalyst] Re: [Dbix-class] Re: utf8 / pg double encoding problem
so do you mean that Dumper should be seeing and outputting this as a
char sequence? (what it actually shows is a mix of chars and hex
bytes, in fact ...)
On Jan 6, 2008 2:08 PM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote:
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 13:30]:
On Jan 6, 2008 1:29 AM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote:
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 00:00]:
[debug] abçöeü
[debug] $VAR1 = ab\x{c3}\x{a7}\x{c3}\x{b6}e\x{c3}\x{bc};
[debug] it's UTF8!
Err, why doesn't Dumper say ab\x{e7}\x{f6}e\x{fc}? Strange
that the first line looks correct, though.
because it is utf8? shouldn't it be?
What Dumper outputs is the UTF-8 byte sequence; but the next line
says that the Unicode flag is set, so this is a character string,
not a byte string. So it's already double-encoded.
I don't understand why the first line looks correct though.
In any case the raw HTTP request that leads to all this would be
interesting.
Regards,
--
Aristotle Pagaltzis // http://plasmasturm.org/
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/[EMAIL PROTECTED]
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
http://www.engoi.com
http://danmcb.vox.com
http://danmcb.blogger.com
find me on linkedin and facebook
BTW : 0873928131
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: [Dbix-class] Re: utf8 / pg double encoding problem
In any case the raw HTTP request that leads to all this would be interesting. I can tell you that the data in the raw request is just the 9 bytes of UTF8, exactly as shown by Dumper. I looked at it with wireshark to be sure. To give some background, this is getting pulled out of a form by javascript (which sees unicode), converted to UTF8, and then submitted as a POST to the cat controller. The $edit param is just another CGI parameter as far as Cat is concerned though. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: [Dbix-class] Re: utf8 / pg double encoding problem
On 06/01/2008, Daniel McBrearty [EMAIL PROTECTED] wrote: In any case the raw HTTP request that leads to all this would be interesting. I can tell you that the data in the raw request is just the 9 bytes of UTF8, exactly as shown by Dumper. I looked at it with wireshark to be sure. To give some background, this is getting pulled out of a form by javascript (which sees unicode), converted to UTF8, and then submitted as a POST to the cat controller. The $edit param is just another CGI parameter as far as Cat is concerned though. Just a note to be careful about your terminology. which see unicode is meaningless really, utf8 is unicode, utf16le is unicode, utf32 is unicode, etc they are just different encodings. I think you mean raw codepoints, but i doubt that JS operates on UTF32 (aka raw codepoints at this time), it most likely operates on utf16le or utf16be. (Whichever one Windows uses internally). Yves -- perl -Mre=debug -e /just|another|perl|hacker/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: utf8 / pg double encoding problem
* Daniel McBrearty [EMAIL PROTECTED] [2008-01-06 19:40]: I can tell you that the data in the raw request is just the 9 bytes of UTF8, exactly as shown by Dumper. I looked at it with wireshark to be sure. I expected that. What I’d like to know is what headers go with the request. Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
