Andreas Költringer created AIRFLOW-2329:
-------------------------------------------
Summary: airflow initdb fails: Specified key was too long; max key
length is 767 bytes
Key: AIRFLOW-2329
URL: https://issues.apache.org/jira/browse/AIRFLOW-2329
Project: Apache Airflow
Issue Type: Bug
Components: db
Affects Versions: 1.9.0
Reporter: Andreas Költringer
Turns out that the default charset in MariaDB is {{utf8mb4}}, and the default
max. keylength is 767 bytes ([MariaDB <
10.2.2|https://mariadb.com/kb/en/library/xtradbinnodb-server-system-variables/#innodb_large_prefix]
// [MySQL <
5.7.7|https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_large_prefix]
). The field {{dag_id}} is defined as {{VARCHAR(250)}}, {{250 x 4 = 1000 >
767}}, hence the problem.
Possible workarounds:
* Avoid database versions in question
* change the encoding to {{utf8}}, which is [not recommended
however|https://stackoverflow.com/a/766996/6699237]
* use a MariaDB/MySQL Docker container
Solution in Airflow could be to turn on [{{innodb_large_prefix}} and related
configs|https://wiki.archlinux.org/index.php/MySQL#Increase_character_limit].
However, this requires the option {{ROW_FORMAT=DYNAMIC}} (and maybe also
{{ENGINE=InnoDB}}) to be set on each CREATE statement.
[SqlAlchemy supports
this|http://docs.sqlalchemy.org/en/latest/dialects/mysql.html#create-table-arguments-including-storage-engines],
but the question is whether Airflow has some mechanics built in to pass this
option in via some config?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
