Ok. These are the changes I made to get this to work. I think utf-8
encoding was not happening on prod for some reason which is why the
write method in scgi was asserting as it has an explicit check there.
All of this annoying and weird and a big hack just to get utf-8/
unicode working:
I added the following line to my environment.py
tmpl_options['mako.default_filters'] = ['decode.utf8']
I deleted the following line in my environment.py
request_settings = dict(charset='utf-8', errors='replace')
My Routes version in site-packages is 1.6.3 now but I don't think this
really matters. 1.7 has an extra assert check for unicode in the
url_for code. I don't think the package version matters.
My create_engine sqlalchemy call has the following two default args
set: convert_unicode=True and encoding="utf8". For some reason, on
my dev environment running the standard paste http server, I had to
also add the following parameter to the mysql:// dburi :
use_unicode=1. I don't require this for prod running SCGI + lighttpd
for some reason.
Thanks.
On Nov 1, 5:24 pm, SamDonaldson <[EMAIL PROTECTED]> wrote:
> Ah, so I noticed that if a string is a unicode object, after encoding
> it in utf-8, we get back a str. So I guess the expectation is that
> the final string should be of type str but encoded using utf-8. Could
> the problem be that these templates are not getting s.encode('utf8')
> and are being left as unicode objects?
>
> Sam
>
> On Oct 31, 11:55 pm, SamDonaldson <[EMAIL PROTECTED]> wrote:
>
> > FYI, I noticed ths post that's very similar:
>
> >http://groups.google.com/group/pylons-discuss/browse_thread/thread/42...
>
> > I noticed that the url_for implementation for Routes 1.7 is different
> > from Routes 1.6.8 under util.py. In 1.7, which my production server
> > is running, there's an exception that gets thrown and an explicit
> > check on making sure it's a string and not unicode.
>
> > As far as FLUP is concerned, it seems like the write() method is
> > getting the template as a Unicode string to render and it's crapping
> > out on that. Does FLUP SCGI actually understand Unicode. The above
> > thread is very similar to what I'm going through.
>
> > Sam
>
> > On Oct 31, 9:09 pm, "Mike Orr" <[EMAIL PROTECTED]> wrote:
>
> > > use_unicode and convert_unicode should not be used together.
> > > use_unicode makes MySQLdb convert the database values to Unicode.
> > > convert_unicode does the same thing in SQLAlchemy.. encoding twice
> > > will lead to exceptions or wrong values if there are any non-ASCII
> > > characters. If users paste text into forms from Word, they'll contain
> > > curly quotes, bullet characters, 1/2 ligatures, degree symbols,
> > > accented letters, etc.
>
> > > But you are getting errors that don't make any sense, so who knows
> > > what's happening on your server. There should be no relationship
> > > between SQLAlchemy settings and url_for.
>
> > > On 10/31/07, SamDonaldson <[EMAIL PROTECTED]> wrote:
>
> > > > On my dev machine running pastehttpserver, I noticed that it only
> > > > works if I use the use_unicode=1 param on my db uri, even after
> > > > passing in the convert_unicode and encoding params to create_engine
> > > > (note that I'm calling my own create_engine).
>
> > > > Thanks,
>
> > > > Saureen
>
> > > > On Oct 30, 10:25 pm, "Mike Orr" <[EMAIL PROTECTED]> wrote:
> > > > > On 10/30/07, SamDonaldson <[EMAIL PROTECTED]> wrote:
>
> > > > > > Hello,
>
> > > > > > I ran into another problem on production, where, if I add the
> > > > > > use_unicode parameter on the mysql:// URI, I get back unicode but I
> > > > > > see that I get a crash in a particular case with the following
> > > > > > stack:
>
> > > > > > Module pylons.helpers:65 in redirect_to
> > > > > > << and cookies extracted from it and added into the redirect
> > > > > > issued. and cookies extracted from it and added into the redirect
> > > > > > issued."""
> > > > > > response = kargs.pop('_response', None)
> > > > > > found = httpexceptions.HTTPFound(url_for(*args, **kargs))
> > > > > > if response:
> > > > > > if str(response.status_code).startswith('3'):>> found =
> > > > > > httpexceptions.HTTPFound(url_for(*args, **kargs))
> > > > > > Module routes.util:200 in url_for
> > > > > > <<
> > > > > > if not isinstance(url, str) and url is not None:
> > > > > > raise Exception("url_for can only return a string or
> > > > > > None,
> > > > > > got "
> > > > > > " unicode instead: %s" % url)>> raise
> > > > > > Exception("url_for can only return a string or None, got "
> > > > > > Exception: url_for can only return a string or None, got unicode
> > > > > > instead:
>
> > > > > > There's an explicity check on 'str' to see if it's an instance even
> > > > > > thought it's a unicode object. How do I get this to work with
> > > > > > Unicode? Note, I don't get this problem on my dev environment. On
> > > > > > production I'm running SCGI + Lighthttpd.
>
> > > > > That's really strange, both this and your other error. Of course, any
> > > > > time str and unicode are used together in an expression, the str gets
> > > > > promoted to unicode. So one place to start would be to look back ward
> > > > > thorough the code and look at the intermediate variables: where was a
> > > > > unicode value first introduced? In your routes definitions? Do they
> > > > > show up in the route map result? What does url_for return in "paster
> > > > > shell"?
>
> > > > > scgi is of course a different environment than pastehttpserver.
> > > > > Perhaps pastehttpserver is more forgiving about converting unicode to
> > > > > str while scgi just raises an exception. 'use_unicode' is
> > > > > disrecommended now; the engine option 'convert_unicode' is recommended
> > > > > instead.
>
> > > > > --
> > > > > Mike Orr <[EMAIL PROTECTED]>
>
> > > --
> > > Mike Orr <[EMAIL PROTECTED]>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---
