The string will be getting output correctly from Python. The problem is that the Apache logging system when passed any strings which contain characters outside of the ASCII character set, will convert them to encoded byte values. So that string is the escaped UTF-8 encoded value of the byte string.
If you need the original original strings, you should use the Python logging module and set it up to save log output to a separate file. > On 4 Sep 2019, at 3:14 pm, '[email protected]' via modwsgi > <[email protected]> wrote: > > Unable to write unicode log messages > > I have a Flask app running with Apache and modwsgi with uses the Python > logging module to write log messages to stderr. These messages sometimes > contain non-ascii text (Japanese specifically). When run with the Flask > development server (no Apache, no modwsgi) the log messages are fine, eg: > ERROR:root:test message: 日本語 > > When run under Apache/modwsgi, the message appearing in the Apache log > file is escaped bytes string: > ERROR:root:test message: \xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e > > I am using Ubuntu-18.04, apache2-2.4.29, libapache2-mod-wsgi-py3-4.5.17, > python3-3.6.7, flask-0.12.2, all installed from the usual Ubuntu > repositories. > > What I have done: > - Lots of googling. Similar problems seem to be old (eg python-2), > involve unicode encode or decode exceptions (not a problem in my > case) and the suggested solutions I tried don't fix my problem. > - Added following settings to the WSGIDaemonProcess line in the > mod_wsgi config file: > locale=en_US.UTF-8 lang=en_US.UTF-8 > - In /etc/apache2/envvars, uncommented line as directed: > ## Uncomment the following line to use the system default locale instead: > . /etc/default/locale > - In /etc/apache2/envvars, added below the two lines above the lines: > export LANG='en_US.UTF-8' > export LC_ALL='en_US.UTF-8' > export PYTHONIOENCODING=UTF-8 > - Printed out following values in the python app code just before > the problem logging line: > sys.filesystemencoding = utf-8 > sys.defaultencoding = utf-8 > type(sys.stderr) = <class '_io.TextIOWrapper'>, encoding = utf-8 > locale = ('en_US', 'UTF-8') > - In the app, wrote the log message line directly to sys.stderr > without going through the logging module. The results were > still escaped bytes rather then the correct encoded unicode text. > > None of the above made any difference. What now??? > > Thanks for any advice! > > Addendum: here is a minimal test program. > > ======== test.py ======== > import sys, logging > from flask import Flask > App = Flask(__name__) > L = logging.getLogger > logging.basicConfig() > > @App.route('/') > def hello_world(): > L().error("test message: 日本語") > return 'Hello, World!' > > if __name__ == '__main__': > App.run (debug=True) > > ======== test.wsgi ======== > import sys, os > appdir, _ = os.path.split (os.path.abspath(__file__)) > sys.path.insert (0, appdir) > from test import App as application > > ======== test.conf ======== > WSGIDaemonProcess jmtest processes=2 threads=4 \ > display-name=apache2-jmtest \ > locale=en_US.UTF-8 lang=en_US.UTF-8 > WSGIProcessGroup jmtest > > WSGIScriptAlias /jmtest /home/.../jmtest/test.wsgi > <Directory /home/.../jmtest> > Require all granted > </Directory> > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/modwsgi/538fb660-7145-49d0-89da-8eaf7e1f694a%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/538fb660-7145-49d0-89da-8eaf7e1f694a%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/744B0BF4-139A-4523-8191-3EADDC580383%40gmail.com.
