I brought this up with Alberto Valverde and Ian Bicking because I was using toscawidgets to provoke the problems, but there are issues with formencode that affect pylons' form handling as well so I will repeat them here. Alberto has fixed them by overriding the relevant formencode behavior inside of toscawidgets; please consider whether to do the same for pythons is appropriate or not.
1) formencode's UnicodeString from_python method encodes the string to utf8, which would be used if you have a default value in the form field. This can fail to render because the template is expecting unicode strings. The fix is that formencode should not assume it is the last step in a render pipeline and simply not encode in from_python; on the other hand, it should definitely decode if necessary in to_python. 2) if validation fails, UnicodeString fields are left in the browser-encoded form for error display which can fail rendering, because unicode objects are being expected in the template. Alberto fixed this locally by raising the priority of UnicodeStrings so that they will be validated before other fields. It probably makes sense to do that in pylons as well, if not directly in formencode. The UnicodeString class is the only field type to change the encoding of its contents on to/from python and thus is the only one that can either pass or fail rendering based on encoding alone. In particular, genshi always expects unicode strings and thus fails consistently. Patches and testcases for the above are in the toscawidgets, please reference its SVN or my darcs repository if you want to see where in the code this is happening. -- David D. Smith
pgpzTNLpDbvqp.pgp
Description: PGP signature
