Re: Unicode charmap decoders slow

2005-10-03 Thread Tony Nelson
In article <[EMAIL PROTECTED]>, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Tony Nelson wrote: > > I had seen iconv. Even if my system supports it and it is faster than > > Python's charmap decoder, it might not be available on other systems. > > Requiring something unusual in order to do

Re: Unicode charmap decoders slow

2005-10-03 Thread Martin v. Löwis
Tony Nelson wrote: > I had seen iconv. Even if my system supports it and it is faster than > Python's charmap decoder, it might not be available on other systems. > Requiring something unusual in order to do a trivial LUT task isn't an > acceptable solution. If I write a charmap decoder as an

Re: Unicode charmap decoders slow

2005-10-03 Thread Tony Nelson
In article <[EMAIL PROTECTED]>, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Tony Nelson wrote: > > Is there a faster way to decode from charmaps to utf-8 than unicode()? > > You could try the iconv codec, if your system supports iconv: > > http://cvs.sourceforge.net/viewcvs.py/python-codecs/

Re: Unicode charmap decoders slow

2005-10-03 Thread Martin v. Löwis
Tony Nelson wrote: > Is there a faster way to decode from charmaps to utf-8 than unicode()? You could try the iconv codec, if your system supports iconv: http://cvs.sourceforge.net/viewcvs.py/python-codecs/practicecodecs/iconv/ Regards, Martin -- http://mail.python.org/mailman/listinfo/python-l

Unicode charmap decoders slow

2005-10-02 Thread Tony Nelson
Is there a faster way to decode from charmaps to utf-8 than unicode()? I'm writing a small card-file program. As a test, I use a 53 MB MBox file, in mac-roman encoding. My program reads and parses the file into messages in about 3..5 seconds, but takes about 13.5 seconds to iterate over the c