ahuynh3 opened a new issue #12515:
URL: https://github.com/apache/airflow/issues/12515
**Apache Airflow version**: 1.10.12
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**:
- **Cloud provider or hardware configuration**: AWS
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**: MySQL (RDS) metadata backend
**What happened**:
We recently turned on DAG serialization and noticed that when we tried to
click on large DAGs in the UI, we get an error:
<details>
<summary>Traceback</summary>
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2447, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1952, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1821, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 39,
in reraise
raise value
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1950, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1936, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File
"/usr/local/lib/python3.7/dist-packages/airflow/www_rbac/decorators.py", line
121, in wrapper
return f(self, *args, **kwargs)
File
"/usr/local/lib/python3.7/dist-packages/flask_appbuilder/security/decorators.py",
line 109, in wraps
return f(self, *args, **kwargs)
File
"/usr/local/lib/python3.7/dist-packages/airflow/www_rbac/decorators.py", line
92, in view_func
return f(*args, **kwargs)
File
"/usr/local/lib/python3.7/dist-packages/airflow/www_rbac/decorators.py", line
56, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/airflow/utils/db.py", line
74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/airflow/www_rbac/views.py",
line 1407, in tree
dag = dagbag.get_dag(dag_id)
File "/usr/local/lib/python3.7/dist-packages/airflow/models/dagbag.py",
line 136, in get_dag
self._add_dag_from_db(dag_id=dag_id)
File "/usr/local/lib/python3.7/dist-packages/airflow/models/dagbag.py",
line 191, in _add_dag_from_db
row = SerializedDagModel.get(dag_id)
File "/usr/local/lib/python3.7/dist-packages/airflow/utils/db.py", line
74, in wrapper
return func(*args, **kwargs)
File
"/usr/local/lib/python3.7/dist-packages/airflow/models/serialized_dag.py", line
217, in get
row = session.query(cls).filter(cls.dag_id == dag_id).one_or_none()
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py",
line 3459, in one_or_none
ret = list(self)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py",
line 100, in instances
cursor.close()
File
"/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/langhelpers.py", line
70, in __exit__
with_traceback=exc_tb,
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py",
line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py",
line 80, in instances
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py",
line 80, in <listcomp>
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py",
line 588, in _instance
populators,
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py",
line 725, in _populate_full
dict_[key] = getter(row)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/type_api.py",
line 1278, in process
return process_value(impl_processor(value), dialect)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/sqltypes.py",
line 2454, in process
return json_deserializer(value)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column
16275 (char 16274)
```
</details>
We’ve determined the issue is that in the `serialized_dag` table with MySQL,
the `data` column type is `TEXT`, which has a max of 64KB, but some of our DAG
code is larger than that. We were able to get around this by running the
following manually on the `serialized_dag` table then waiting for the table to
get re-updated:
```
CREATE TABLE serialized_dag_backup AS SELECT * FROM serialized_dag;
ALTER TABLE serialized_dag MODIFY data MEDIUMTEXT;
SELECT * FROM serialized_dag
WHERE LENGTH(data) = 65535;
DELETE FROM serialized_dag
WHERE LENGTH(data) = 65535;
```
**What you expected to happen**:
Should be able to click on the DAG in the UI without error
**How to reproduce it**:
With a MySQL metadata backend, create a DAG with code that is larger than
64KB and enable DAG serialization. Then attempt to click on that DAG in the UI.
**Anything else we need to know**: N/A
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
