potiuk commented on code in PR #25938: URL: https://github.com/apache/airflow/pull/25938#discussion_r953983060
########## docs/apache-airflow/howto/set-up-database.rst: ########## @@ -305,12 +305,146 @@ without any cert options provided. However if you want to use other drivers visit the `MySQL Dialect <https://docs.sqlalchemy.org/en/13/dialects/mysql.html>`__ in SQLAlchemy documentation for more information regarding download and setup of the SqlAlchemy connection. -In addition, you also should pay particular attention to MySQL's encoding. Although the ``utf8mb4`` character set is more and more popular for MySQL (actually, ``utf8mb4`` becomes default character set in MySQL8.0), using the ``utf8mb4`` encoding requires additional setting in Airflow 2+ (See more details in `#7570 <https://github.com/apache/airflow/pull/7570>`__.). If you use ``utf8mb4`` as character set, you should also set ``sql_engine_collation_for_ids=utf8mb3_bin``. +In addition, you also should pay particular attention to MySQL's encoding. Although the ``utf8mb4`` character set is more and more popular for MySQL (actually, ``utf8mb4`` becomes default character set in MySQL8.0), using the ``utf8mb4`` encoding requires additional setting in Airflow 2+ (See more details in `#7570 <https://github.com/apache/airflow/pull/7570>`__.). If you use ``utf8mb4`` as character set, you should also set ``sql_engine_collation_for_ids=utf8mb4_bin``. .. note:: In strict mode, MySQL doesn't allow ``0000-00-00`` as a valid date. Then you might get errors like ``"Invalid default value for 'end_date'"`` in some cases (some Airflow tables use ``0000-00-00 00:00:00`` as timestamp field default value). To avoid this error, you could disable ``NO_ZERO_DATE`` mode on you MySQL server. Read https://stackoverflow.com/questions/9192027/invalid-default-value-for-create-date-timestamp-field for how to disable it. See `SQL Mode - NO_ZERO_DATE <https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sqlmode_no_zero_date>`__ for more information. +.. warning:: + + + If you are using old Airflow 1.10 as a database created initially either manually or with previous version of MySQL, + depending on the original character set of your database, you might have problems with migrating to a newer + version of Airflow and your migration might fail with strange errors ("key size too big", "missing indexes" etc). + The next chapter describes how to fix the problem manually. Review Comment: Fixed. While doing it I also recalled that there is one more thing we missed there. We often ask people "did you make a backup as we recommend?" when they have problem with migration. But - surprisingly - we never ACTUALLY recommended to make a backup before migration. I was pretty sure we did, but when you look closer at our docs, we do not :scream: This PR also fixes that problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
