On 18 joulu, 15:03, achipa <[email protected]> wrote:
> For mysql for *empty* tables this would be
>
> ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8;
>
> If you want to *keep* the data, it's a different story. The CONVERT TO
> operation converts column values between the character sets. This is
> not what you want if you have a column in one character set (like
> latin1) but the stored values actually use some other, incompatible
> character set (like utf8). In this case, you have to do the following
> for each such column:
>
> ALTER TABLE t1 CHANGE c1 c1 BLOB;
> ALTER TABLE t1 CHANGE c1 c1 TEXT CHARACTER SET utf8;
That worked like a charm! Now unserializing works as supposed. Thank
you for your advice, even not web2py specific problem. I hope other
questions and developments tips are considered / commented also from
this thread.
Thanks again,
-Marko
>
> The reason this works is that there is no conversion when you convert
> to or from BLOB columns.
>
> On Dec 18, 1:48 pm, mmstud <[email protected]> wrote:
>
> > Tables are not made by web2py... is it possible to change table
> > encoding afterwards? Now schema description says: ENGINE=MyISAM
> > AUTO_INCREMENT=387210 DEFAULT CHARSET=latin1
>
> > Yet another feature id like to see is from Pylons, which enables doing
> > console like excecution of python command on browser debug screen
> > after something fails.
> > Let me check the format of string from console...
>
> > -Marko
>
> > On 18 joulu, 14:26, achipa <[email protected]> wrote:
>
> > > PHP not having problems does not mean it's doing things right, it
> > > might be just repeating the same error on retrieval and thus making
> > > things look ok when in fact they aren't. As I pointed it out in the,
> > > other thread you might get away with this but only as long as you
> > > don't have many utf characters (truncation) or don't use string
> > > operations, orderby, etc.
>
> > > The string you pasted is obviously utf-8. As web2py creates utf-8
> > > tables, and reads from them accordingly, this most likely means that
>
> > > a) either the table was created by someone else as latin1 OR
>
> > > b) an utf string was inserted from a mysql connection that was latin1
> > > (i.e. the system locale is not utf-8 and you have not issued SET NAMES
> > > 'utf8' in your connection, so mysql thought the data you're sending
> > > was in latin1, when in fact it was utf, thus resulting in the string
> > > being utfencoded twice).
>
> > > Things to check:
> > > If you did not create the tables from web2py, check their encoding.
> > > Check the data in the db. Do a select from the console or a mysqldump
> > > and inspect the file.
>
> > > On Dec 18, 1:03 pm, mmstud <[email protected]> wrote:
>
> > > > Im using MySQL. Same data is retrieved correctly on php side and there
> > > > are no problems showing it on ui. Using php's own serialize and
> > > > unserialize method ofcource. Now on python side some of data seems be
> > > > problematic. Actually the part of the data, which seems to be a
> > > > problems is:
>
> > > > "Philosophy By Example « Cliche"
>
> > > > The library im using here
> > > > is:http://hurring.com/scott/code/python/serialize/
>
> > > > -Marko
>
> > > > On 18 joulu, 13:35, achipa <[email protected]> wrote:
>
> > > > > What database backend are you using ? We have another topic hammering
> > > > > at this at the moment, so there might be development it is handled.
> > > > > Generally, if all is well, you should not be doing any encode/decode-
> > > > > ing (unless your data is broken in the first place).
>
> > > > > On Dec 18, 12:21 pm, mmstud <[email protected]> wrote:
>
> > > > > > Hi,
>
> > > > > > a lot of wuestions ahs risen after using web2py, if i just got time
> > > > > > to
> > > > > > write them here. One of the latest is, that id like to know, how to
> > > > > > retrieve data with correct encoding from database. My situatio is,
> > > > > > that i have a php serialized data field, which i want to get to
> > > > > > web2py
> > > > > > application. I have a PHPUnserialize class for that, but
> > > > > > unserializing
> > > > > > fails because there seems to be special characters on database field
> > > > > > data. I have an example here:
>
> > > > > > """
> > > > > > a:2:{s:10:"allintitle";i:97;s:5:"pages";a:5:{i:0;a:13:{s:3:"url";s:
> > > > > > 35:"http://www.angelfire.com/ill/niche/";s:16:"google_pagetitle";s:
> > > > > > 52:"CHAPTER 16:Friedrich Nietzsche: The Anti-Philosopher";s:
> > > > > > 15:"google_pagerank";i:0;s:12:"google_links";s:1:"2";s:
> > > > > > 10:"alexa_rank";s:3:"786";s:11:"alexa_reach";s:3:"619";s:
> > > > > > 13:"alexa_linksin";s:6:"67,533";s:15:"yahoo_backlinks";i:0;s:
> > > > > > 9:"msn_links";i:0;s:15:"alltheweb_links";i:0;s:15:"altavista_links";i:
> > > > > > 0;s:9:"seo_stats";a:6:{s:4:"body";i:0;s:4:"meta";i:0;s:2:"h1";i:0;s:
> > > > > > 3:"alt";i:0;s:3:"url";i:0;s:5:"title";i:0;}s:15:"keyword_density";d:0;}
> > > > > > i:1;a:13:{s:3:"url";s:65:"http://corporatecartoons.blogspot.com/
> > > > > > 2007/03/nietzshe-niche.html";s:16:"google_pagetitle";s:35:"Scrambled
> > > > > > Toast: The Nietzshe Niche";s:15:"google_pagerank";i:1;s:
> > > > > > 12:"google_links";b:0;s:10:"alexa_rank";s:9:"5,490,647";s:
> > > > > > 11:"alexa_reach";s:9:"4,692,194";s:13:"alexa_linksin";s:2:"81";s:
> > > > > > 15:"yahoo_backlinks";s:1:"1";s:9:"msn_links";i:0;s:
> > > > > > 15:"alltheweb_links";s:2:"97";s:15:"altavista_links";s:2:"97";s:
> > > > > > 9:"seo_stats";a:6:{s:4:"body";i:0;s:4:"meta";i:0;s:2:"h1";i:0;s:
> > > > > > 3:"alt";i:0;s:3:"url";i:0;s:5:"title";i:0;}s:15:"keyword_density";d:0;}
> > > > > > i:2;a:13:{s:3:"url";s:61:"http://www.scribd.com/doc/9330/Nietzsche-
> > > > > > Philosophy-and-Truth";s:16:"google_pagetitle";s:32:"Nietzsche -
> > > > > > Philosophy and
> > > > > > Truth";s:15:"google_pagerank";i:1;s:12:"google_links";b:
> > > > > > 0;s:10:"alexa_rank";s:3:"312";s:11:"alexa_reach";s:3:"262";s:
> > > > > > 13:"alexa_linksin";s:6:"10,649";s:15:"yahoo_backlinks";s:1:"1";s:
> > > > > > 9:"msn_links";i:0;s:15:"alltheweb_links";s:2:"32";s:
> > > > > > 15:"altavista_links";s:3:"594";s:9:"seo_stats";a:6:{s:4:"body";i:27;s:
> > > > > > 4:"meta";i:0;s:2:"h1";i:0;s:3:"alt";i:0;s:3:"url";i:0;s:5:"title";i:0;}
> > > > > > s:15:"keyword_density";d:0;}i:3;a:13:{s:3:"url";s:66:"http://
> > > > > > clicheniche.wordpress.com/2007/09/15/philosophy-by-example/";s:
> > > > > > 16:"google_pagetitle";s:37:"Philosophy By Example « Cliche Niche";s:
> > > > > > 15:"google_pagerank";i:2;s:12:"google_links";b:0;s:10:"alexa_rank";s:
> > > > > > 10:"27,056,874";s:11:"alexa_reach";s:10:"26,656,375";s:
> > > > > > 13:"alexa_linksin";s:1:"3";s:15:"yahoo_backlinks";i:0;s:
> > > > > > 9:"msn_links";i:0;s:15:"alltheweb_links";s:1:"6";s:
> > > > > > 15:"altavista_links";s:1:"6";s:9:"seo_stats";a:6:{s:4:"body";i:0;s:
> > > > > > 4:"meta";i:0;s:2:"h1";i:0;s:3:"alt";i:0;s:3:"url";i:0;s:5:"title";i:0;}
> > > > > > s:15:"keyword_density";d:0;}i:4;a:13:{s:3:"url";s:38:"http://www.nichephilosophersstone.com/";s:16:"google_pagetitle";s:57:"Niche
> > > > > > Philosopher's Stone - Turns Tiny Things to Gold";s:
> > > > > > 15:"google_pagerank";i:0;s:12:"google_links";s:2:"19";s:
> > > > > > 10:"alexa_rank";s:9:"1,855,647";s:11:"alexa_reach";s:9:"2,730,154";s:
> > > > > > 13:"alexa_linksin";i:0;s:15:"yahoo_backlinks";s:2:"17";s:
> > > > > > 9:"msn_links";s:1:"2";s:15:"alltheweb_links";s:2:"13";s:
> > > > > > 15:"altavista_links";s:2:"14";s:9:"seo_stats";a:6:{s:4:"body";i:1;s:
> > > > > > 4:"meta";i:0;s:2:"h1";i:0;s:3:"alt";i:0;s:3:"url";i:1;s:5:"title";i:1;}
> > > > > > s:15:"keyword_density";d:0.02;}}}
> > > > > > """
>
> > > > > > "Philosophy By Example « Cliche" part shows errorneous characters
> > > > > > when
> > > > > > i use DAL to seek data. How should i decode data and what should i
> > > > > > do
> > > > > > in this case?
>
> > > > > > Another thing is, that on webeditor (appadmin) toggle code should
> > > > > > keep
> > > > > > latest setting, not switching back to on state after submitting
> > > > > > edited
> > > > > > text. In some cases plain text edit without editor features is so
> > > > > > much
> > > > > > faster, that id like to edit just in textarea mode.
>
> > > > > > -Marko
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py Web Framework" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
