turbaszek opened a new issue #12315:
URL: https://github.com/apache/airflow/issues/12315
**Apache Airflow version**:
2.0 / master
**Environment**:
breeze
**What happened**:
I want to use custom XCom backend that returns in `deserialize_value` a
pandas DataFrame object. This should work as expected because the type
annotation suggests it:
```
def deserialize_value(result) -> Any:
```
However, not every object implements `__nonzero__` method that returns
boolean value. For example:
```
File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line
1326, in __nonzero__
raise ValueError(
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty,
a.bool(), a.item(), a.any() or a.all().
```
This is problematic because if users click in Airflow webui Admin > XComs
then they will the following:
```
Python version: 3.8.6
Airflow version: 2.0.0b2
Node: 950a17127708
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2447, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1952, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1821, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39,
in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1950, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1936, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py",
line 109, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/views.py",
line 552, in list
return self.render_template(
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/baseviews.py", line
280, in render_template
return render_template(
File "/usr/local/lib/python3.8/site-packages/flask/templating.py", line
137, in render_template
return _render(
File "/usr/local/lib/python3.8/site-packages/flask/templating.py", line
120, in _render
rv = template.render(context)
File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line
1090, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line
832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28,
in reraise
raise value.with_traceback(tb)
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html",
line 2, in top-level template code
{% import 'appbuilder/general/lib.html' as lib %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/base.html",
line 1, in top-level template code
{% extends base_template %}
File "/opt/airflow/airflow/www/templates/airflow/master.html", line 20, in
top-level template code
{% extends 'appbuilder/baselayout.html' %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html",
line 2, in top-level template code
{% import 'appbuilder/baselib.html' as baselib %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/init.html",
line 46, in top-level template code
{% block body %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html",
line 19, in block "body"
{% block content %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html",
line 13, in block "content"
{% block list_list scoped %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html",
line 15, in block "list_list"
{{ widgets.get('list')()|safe }}
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/widgets.py",
line 37, in __call__
return template.render(args)
File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line
1090, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.8/site-packages/jinja2/environment.py", line
832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/local/lib/python3.8/site-packages/jinja2/_compat.py", line 28,
in reraise
raise value.with_traceback(tb)
File "/opt/airflow/airflow/www/templates/airflow/model_list.html", line
21, in top-level template code
{% extends 'appbuilder/general/widgets/base_list.html' %}
File
"/usr/local/lib/python3.8/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/base_list.html",
line 23, in top-level template code
{% block begin_loop_values %}
File "/opt/airflow/airflow/www/templates/airflow/model_list.html", line
80, in block "begin_loop_values"
{% elif item[value] != None %}
File "/usr/local/lib/python3.8/site-packages/pandas/core/generic.py", line
1326, in __nonzero__
raise ValueError(
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty,
a.bool(), a.item(), a.any() or a.all().
```
**What you expected to happen**:
It would be nice to allow returning **any** object in deserialise method
**How to reproduce it**:
Create `xcom_backend.py` in `files` directory (assuming breeze) with the
following content:
```py
import pandas as pd
from airflow.models.xcom import BaseXCom
class CustomXComBackend(BaseXCom):
@staticmethod
def serialize_value(value: Any):
return BaseXCom.serialize_value(value)
@staticmethod
def deserialize_value(result) -> Any:
return pd.DataFrame({"a": [1, 2]})
```
then set
```
export PYTHONPATH=$PYTHONPATH:/files
export AIRFLOW__CORE__XCOM_BACKEND=xcom_backend. CustomXComBackend
```
and start webserver
```
airflow webserver -w 1 -D
```
You may need to first run a DAG to populate the table.
**Anything else we need to know**:
N/A
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
