New issue 378: UnicodeDecodeError after upgrade https://bitbucket.org/conservancy/kallithea/issues/378/unicodedecodeerror-after-upgrade
Akos: I am upgradeing kallithea 0.3.2 to 0.6.1, and after upgrade when I try to access users and repos than contain unicode names, I got this error in logs: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1: character maps to <undefined> Full logfile when I try to access Users page \(I have LDAP users with unicode charactes\): Jun 15 16:23:56 hg gearbox\[17152\]: Traceback \(most recent call last\): Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/appwrappers/session.py", line 71, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: response = self.next\_handler\(controller, environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/appwrappers/i18n.py", line 71, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: return self.next\_handler\(controller, environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/wsgiapp.py", line 243, in \_dispatch Jun 15 16:23:56 hg gearbox\[17152\]: return controller\(environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/kallithea/lib/base.py", line 536, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: return super\(BaseController, self\).**call**\(environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/dispatcher.py", line 118, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: response = self.\_perform\_call\(context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/dispatcher.py", line 107, in \_perform\_call Jun 15 16:23:56 hg gearbox\[17152\]: r = self.\_call\(action, params, remainder=remainder, context=context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/decoratedcontroller.py", line 129, in \_call Jun 15 16:23:56 hg gearbox\[17152\]: output = controller\_caller\(context\_config, bound\_controller\_callable, remainder, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/decoration.py", line 21, in \_decorated\_controller\_caller Jun 15 16:23:56 hg gearbox\[17152\]: return application\_controller\_caller\(tg\_config, controller, remainder, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/configurator/components/dispatch.py", line 114, in \_call\_controller Jun 15 16:23:56 hg gearbox\[17152\]: return controller\(\*remainder, \*\*params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/kallithea/controllers/admin/users.py", line 70, in index Jun 15 16:23:56 hg gearbox\[17152\]: .order\_by\(func.lower\(User.username\)\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3319, in all Jun 15 16:23:56 hg gearbox\[17152\]: return list\(self\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3481, in **iter** Jun 15 16:23:56 hg gearbox\[17152\]: return self.\_execute\_and\_instances\(context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3506, in \_execute\_and\_instances Jun 15 16:23:56 hg gearbox\[17152\]: result = conn.execute\(querycontext.statement, self.\_params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1020, in execute Jun 15 16:23:56 hg gearbox\[17152\]: return meth\(self, multiparams, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in \_execute\_on\_connection Jun 15 16:23:56 hg gearbox\[17152\]: return connection.\_execute\_clauseelement\(self, multiparams, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1139, in \_execute\_clauseelement Jun 15 16:23:56 hg gearbox\[17152\]: distilled\_params, Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1324, in _execute\_context_ _Jun 15 16:23:56 hg gearbox\[17152\]: e, statement, parameters, cursor, context_ _Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1521, in handle\_dbapi\_exception_ _Jun 15 16:23:56 hg gearbox\[17152\]: util.raise_\(exc\_info\[1\], with\_traceback=exc\_info\[2\]\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise Jun 15 16:23:56 hg gearbox\[17152\]: raise exception Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in \_execute\_context Jun 15 16:23:56 hg gearbox\[17152\]: cursor, statement, parameters, context Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do\_execute Jun 15 16:23:56 hg gearbox\[17152\]: cursor.execute\(statement, parameters\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute Jun 15 16:23:56 hg gearbox\[17152\]: res = self.\_query\(query\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 317, in \_query Jun 15 16:23:56 hg gearbox\[17152\]: self.\_post\_get\_result\(\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 352, in \_post\_get\_result Jun 15 16:23:56 hg gearbox\[17152\]: self.\_rows = self.\_fetch\_row\(0\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 325, in \_fetch\_row Jun 15 16:23:56 hg gearbox\[17152\]: return self.\_result.fetch\_row\(size, self.\_fetch\_type\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/usr/lib/python3.6/encodings/cp1252.py", line 15, in decode Jun 15 16:23:56 hg gearbox\[17152\]: return codecs.charmap\_decode\(input,errors,decoding\_table\) Jun 15 16:23:56 hg gearbox\[17152\]: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1: character maps to <undefined> Jun 15 16:23:56 hg gearbox\[17152\]: Traceback \(most recent call last\): Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/appwrappers/session.py", line 71, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: response = self.next\_handler\(controller, environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/appwrappers/i18n.py", line 71, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: return self.next\_handler\(controller, environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/wsgiapp.py", line 243, in \_dispatch Jun 15 16:23:56 hg gearbox\[17152\]: return controller\(environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/kallithea/lib/base.py", line 536, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: return super\(BaseController, self\).**call**\(environ, context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/dispatcher.py", line 118, in **call** Jun 15 16:23:56 hg gearbox\[17152\]: response = self.\_perform\_call\(context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/dispatcher.py", line 107, in \_perform\_call Jun 15 16:23:56 hg gearbox\[17152\]: r = self.\_call\(action, params, remainder=remainder, context=context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/decoratedcontroller.py", line 129, in \_call Jun 15 16:23:56 hg gearbox\[17152\]: output = controller\_caller\(context\_config, bound\_controller\_callable, remainder, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/controllers/decoration.py", line 21, in \_decorated\_controller\_caller Jun 15 16:23:56 hg gearbox\[17152\]: return application\_controller\_caller\(tg\_config, controller, remainder, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/tg/configurator/components/dispatch.py", line 114, in \_call\_controller Jun 15 16:23:56 hg gearbox\[17152\]: return controller\(\*remainder, \*\*params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/kallithea/controllers/admin/users.py", line 70, in index Jun 15 16:23:56 hg gearbox\[17152\]: .order\_by\(func.lower\(User.username\)\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3319, in all Jun 15 16:23:56 hg gearbox\[17152\]: return list\(self\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3481, in **iter** Jun 15 16:23:56 hg gearbox\[17152\]: return self.\_execute\_and\_instances\(context\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3506, in \_execute\_and\_instances Jun 15 16:23:56 hg gearbox\[17152\]: result = conn.execute\(querycontext.statement, self.\_params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1020, in execute Jun 15 16:23:56 hg gearbox\[17152\]: return meth\(self, multiparams, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in \_execute\_on\_connection Jun 15 16:23:56 hg gearbox\[17152\]: return connection.\_execute\_clauseelement\(self, multiparams, params\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1139, in \_execute\_clauseelement Jun 15 16:23:56 hg gearbox\[17152\]: distilled\_params, Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1324, in _execute\_context_ _Jun 15 16:23:56 hg gearbox\[17152\]: e, statement, parameters, cursor, context_ _Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1521, in handle\_dbapi\_exception_ _Jun 15 16:23:56 hg gearbox\[17152\]: util.raise_\(exc\_info\[1\], with\_traceback=exc\_info\[2\]\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise Jun 15 16:23:56 hg gearbox\[17152\]: raise exception Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in \_execute\_context Jun 15 16:23:56 hg gearbox\[17152\]: cursor, statement, parameters, context Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do\_execute Jun 15 16:23:56 hg gearbox\[17152\]: cursor.execute\(statement, parameters\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute Jun 15 16:23:56 hg gearbox\[17152\]: res = self.\_query\(query\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 317, in \_query Jun 15 16:23:56 hg gearbox\[17152\]: self.\_post\_get\_result\(\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 352, in \_post\_get\_result Jun 15 16:23:56 hg gearbox\[17152\]: self.\_rows = self.\_fetch\_row\(0\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/srv/kallithea/venv/lib/python3.6/site-packages/MySQLdb/cursors.py", line 325, in \_fetch\_row Jun 15 16:23:56 hg gearbox\[17152\]: return self.\_result.fetch\_row\(size, self.\_fetch\_type\) Jun 15 16:23:56 hg gearbox\[17152\]: File "/usr/lib/python3.6/encodings/cp1252.py", line 15, in decode Jun 15 16:23:56 hg gearbox\[17152\]: return codecs.charmap\_decode\(input,errors,decoding\_table\) Jun 15 16:23:56 hg gearbox\[17152\]: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1: character maps to <undefined> _______________________________________________ kallithea-general mailing list [email protected] https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
