On 10/24/06, Ian Bicking <[EMAIL PROTECTED]> wrote: > Shannon -jj Behrens wrote: > >> I haven't done much testing around this, so I don't know. This could > >> potentially be done with a wrapper around MultiDict too, that lazily > >> decodes the values. > > > > This can be a frustrating subject. Did you know if you set > > accept-encodings="US-ASCII" in the form, but the user tries to submit > > Japanese characters, Firefox will send them as HTML entities like > > Ӓ whereas IE will ignore you and send UTF-8 (assuming your page > > was originally in UTF-8)? Bleh! > > Oh the pain! Does it start HTML quoting everything once it starts > adding entities (but not before)? Too bad there isn't a unicode option > for url escaping. And if you do multipart/form-data, does the encoding > show up there? Does the cgi module keep that information or throw it away? > > Clearly it would be nice to do this right, but "nice" isn't "easy". > Probably heuristics are also called for, as well as looking at the user > agent. And what about IE 7?
I haven't started using Paste yet though I'm planning to try Pylons soon (yeah, I've been saying that for a year), but this same character input problem has hit me in Quixote applications. Namely, people paste text from a Word document or FileMaker database on a PC or Mac into a web form, and who the hell knows what character set it is or what the browser may have done to it? Windows curly quotes are in the gap between ASCII and Latin-1 though Firefox/Linux "knows" how to display them, the degree symbol and copyright symbol are in different places in Latin-1 vs MacRoman, the "masculine ordinal indicator" (a circle with a horizontal line underneath) looks like a degree symbol on Windows so people choose it, etc. Then Python blows up when it implicitly converts an 8-bit string to Unicode and it contains non-ASCII characters, and you can't predict what all characters the input might contain... it's quite a mess to code a web app around. If this can be solved in a generic way it will be very helpful. -- Mike Orr <[EMAIL PROTECTED]> _______________________________________________ Paste-users mailing list [email protected] http://webwareforpython.org/cgi-bin/mailman/listinfo/paste-users
