I had a similar problem and I used two approaches. First I used Django's
smart_str:
from django.utils.encoding import smart_str
text = smart_str(text)
Then I had the same problem with fields using the csv module and I used
these functions that I found on the web:
def unicode_csv_reader(unicode_csv_data, dialect=csv.excel, **kwargs):
# csv.py doesn't do Unicode; encode temporarily as UTF-8:
csv_reader = csv.reader(utf_8_encoder(unicode_csv_data),
dialect=dialect, **kwargs)
for row in csv_reader:
# decode UTF-8 back to Unicode, cell by cell:
yield [unicode(cell, 'utf-8') for cell in row]
def utf_8_encoder(unicode_csv_data):
for line in unicode_csv_data:
yield line.encode('utf-8')
I'm not sure I fully understand your problem, so maybe they won't suit you,
but possibly doing something with the encode function or smart_str will help
you.
Good luck,
Paulo
On Fri, Mar 19, 2010 at 2:44 PM, Steven L Smith <ssmit...@naz.edu> wrote:
> We have a freelance designer that, for various political reasons, we need
> to
> give access to our templates directory. The problem is, he frequently
> pastes
> from Microsoft Word, and other sources, and then we're seeing HTTP 500
> errors
> because the output contains unescaped UTF-8 / unicode characters.
>
> For example, "UnicodeDecodeError: 'utf8' codec can't decode byte 0x94 in
> position 30076: unexpected code byte".
>
> Is there a way to prevent these errors, other than revisiting the level of
> access we give folks like this? Some kind of middleware that would filter
> these characters out? Some kind of way to replace weird characters with a
> bright pink blinking exclamation point? Something else?
>
> Thanks!
>
> Steven L Smith
> Web Developer
> Nazareth College of Rochester
> http://www.naz.edu/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com<django-users%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
