[
https://issues.apache.org/jira/browse/AIRFLOW-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16627176#comment-16627176
]
Minh Quan TRAN commented on AIRFLOW-3093:
-----------------------------------------
The issue is that the container does not have locale set to UTF-8 and Python
uses ANSI encoding by detection
> Airflow UnicodeEncodeError with logging (file_task_handler.py)
> --------------------------------------------------------------
>
> Key: AIRFLOW-3093
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3093
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
> Affects Versions: 1.10.0
> Reporter: Minh Quan TRAN
> Priority: Major
> Labels: encoding
>
> I ran into this exception stack when using BashOperator along with docker &
> composer install inside the container. I ran some more tests with some other
> set of dependencies, this error is not always present, just for some cases
> the exception is thrown.
>
> The exception is thrown at *file_task_handler.py*
>
> {code:java}
> Python 3.5{code}
> {code:java}
> [2018-09-19 20:42:18,708] {bash_operator.py:111} INFO - Package operations:
> 134 installs, 0 updates, 0 removals
> [2018-09-19 20:42:18,790] {bash_operator.py:111} INFO - - Installing
> ocramius/package-versions (1.3.0): Downloading (100%)
> [2018-09-19 20:42:18,850] {bash_operator.py:111} INFO - - Installing
> symfony/flex (v1.1.1): Downloading (100%)
> [2018-09-19 20:42:18,897] {bash_operator.py:111} INFO -
> [2018-09-19 16:12:51,554] {logging_mixin.py:95} WARNING - --- Logging error
> ---
> [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Traceback (most
> recent call last):
> [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 983, in emit
> stream.write(msg)
> [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - UnicodeEncodeError:
> 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordinal not
> in range(128)
> [2018-09-19 16:12:51,555] {logging_mixin.py:95} WARNING - Call stack:
> [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File
> "/usr/local/bin/airflow", line 32, in <module>
> args.func(args)
> [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/utils/cli.py", line 74, in
> wrapper
> return f(*args, **kwargs)
> [2018-09-19 16:12:51,557] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/bin/cli.py", line 498, in run
> _run(args, dag, ti)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/bin/cli.py", line 402, in _run
> pool=args.pool,
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/utils/db.py", line 74, in
> wrapper
> return func(*args, **kwargs)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/models.py", line 1633, in
> _run_raw_task
> result = task_copy.execute(context=context)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/operators/bash_operator.py",
> line 110, in execute
> self.log.info(line)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 1280, in info
> self._log(INFO, msg, args, **kwargs)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 1416, in _log
> self.handle(record)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 1426, in handle
> self.callHandlers(record)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 1488, in callHandlers
> hdlr.handle(record)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.5/logging/__init__.py", line 856, in handle
> self.emit(record)
> [2018-09-19 16:12:51,558] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.5/dist-packages/airflow/utils/log/file_task_handler.py",
> line 61, in emit
> self.handler.emit(record){code}
> {code:java}
> Python 3.6{code}
> {code:java}
> [2018-09-25 09:33:04,622] {logging_mixin.py:95} WARNING - --- Logging error
> ---
> [2018-09-25 09:33:04,627] {logging_mixin.py:95} WARNING - Traceback (most
> recent call last):
> [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 995, in emit
> stream.write(msg)
> [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - UnicodeEncodeError:
> 'ascii' codec can't encode character '\U0001f3b6' in position 81: ordinal not
> in range(128)
> [2018-09-25 09:33:04,628] {logging_mixin.py:95} WARNING - Call stack:
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/bin/airflow", line 32, in <module>
> args.func(args)
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/utils/cli.py", line 74, in
> wrapper
> return f(*args, **kwargs)
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/bin/cli.py", line 498, in run
> _run(args, dag, ti)
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/bin/cli.py", line 402, in _run
> pool=args.pool,
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 74, in
> wrapper
> return func(*args, **kwargs)
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/models.py", line 1633, in
> _run_raw_task
> result = task_copy.execute(context=context)
> [2018-09-25 09:33:04,653] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/operators/bash_operator.py",
> line 110, in execute
> self.log.info(line)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 1307, in info
> self._log(INFO, msg, args, **kwargs)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 1443, in _log
> self.handle(record)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 1453, in handle
> self.callHandlers(record)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 1515, in callHandlers
> hdlr.handle(record)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/lib/python3.6/logging/__init__.py", line 864, in handle
> self.emit(record)
> [2018-09-25 09:33:04,654] {logging_mixin.py:95} WARNING - File
> "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/file_task_handler.py",
> line 61, in emit
> self.handler.emit(record){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
