iwanbolzern opened a new issue #15287:
URL: https://github.com/apache/airflow/issues/15287
**Apache Airflow version**: 2.0.1
**Kubernetes version (if you are using kubernetes)** (use `kubectl
version`):
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.14",
GitCommit:"5de7fd1f9555368a86eb0f8f664dc58055c17269", GitTreeState:"clean",
BuildDate:"2021-01-18T09:31:01Z", GoVersion:"go1.13.15", Compiler:"gc",
Platform:"linux/amd64"}
**Environment**: TEST
- **Cloud provider or hardware configuration**: Azure Kubernetes
- **OS** (e.g. from /etc/os-release): airflow docker image with tag
apache/airflow:2.0.1-python3.8
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**:
```
[2021-04-08 15:23:24,215] {dagbag.py:448} INFO - Filling up the DagBag from
/opt/airflow/dags/dag_xxxx.py
Running <TaskInstance: xxx.xxxx 2021-02-03T13:00:00+00:00 [queued]> on host
xxxx.392653599def4692b6d1eecae Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line
40, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line
89, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 235, in task_run
_run_task_by_selected_method(args, dag, ti)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 64, in _run_task_by_selected_method
_run_task_by_local_task_job(args, ti)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 124, in _run_task_by_local_task_job
logging.shutdown()
File "/usr/local/lib/python3.8/logging/__init__.py", line 2123, in shutdown
h.close()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py",
line 97, in close
self.wasb_write(log, remote_loc, append=True)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py",
line 178, in wasb_write
log = '\n'.join([old_log, log]) if old_log else log
TypeError: sequence item 0: expected str instance, bytes found
[2021-04-08 15:23:51,271] {wasb.py:304} INFO - Attempting to create
container: airflow-logs
[2021-04-08 15:23:51,280] {wasb.py:309} INFO - Container airflow-logs
already exists
[2021-04-08 15:23:51,287] {wasb.py:304} INFO - Attempting to create
container: airflow-logs
[2021-04-08 15:23:51,293] {wasb.py:309} INFO - Container airflow-logs
already exists
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/logging/__init__.py", line 2123, in shutdown
h.close()
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py",
line 97, in close
self.wasb_write(log, remote_loc, append=True)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/microsoft/azure/log/wasb_task_handler.py",
line 178, in wasb_write
log = '\n'.join([old_log, log]) if old_log else log
TypeError: sequence item 0: expected str instance, bytes found
```
When I download the log file, it looks like:
```
xxxx.2deb6142e11d432797e534081bc
*** Reading remote log from
wasb-airflow-logs-test/xxxx/ftp_agg_ch_sensor/2021-02-01T13:00:00+00:00/1.log.
b"[2021-04-01 10:11:10,052] ...."
```
**What you expected to happen**:
To me it seems as if load_string does not proper encode the string value
before uploading into the wasb container. But this is just a feeling and not
validated.
```
def load_string(self, string_data: str, container_name: str, blob_name: str,
**kwargs) -> None:
"""
Upload a string to Azure Blob Storage.
:param string_data: String to load.
:type string_data: str
:param container_name: Name of the container.
:type container_name: str
:param blob_name: Name of the blob.
:type blob_name: str
:param kwargs: Optional keyword arguments that ``BlobClient.upload()``
takes.
:type kwargs: object
"""
# Reorder the argument order from
airflow.providers.amazon.aws.hooks.s3.load_string.
self.upload(container_name, blob_name, string_data, **kwargs)
```
**How to reproduce it**:
Simply use wasb_task_handler logging and then execute a task twice by
clearing the DAG.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
