> *What about* variable sql_engine_collation_for_ids=utf8mb3_bin mentioned next on same page > https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database > should I set that in /etc/mysql/my.cnf also ?
> In the airflow.cfg the default setting is sql_engine_encoding = utf-8 > Should I change from sql_engine_encoding = utf-8 in airflow,cfg to sql_engine_encoding = utf8mb4 in airflow.cfg if I am changing sql_engine_collation_for_ids=utf8mb3_bin in /etc/mysql/my.cnf * ?* No - those are "airflow" configuration settings, not MySQL driver settings. The former are described in https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html, the latter - in the driver documentation (when you pass the URL to sqlalchemy, you actually select the driver to use, and this driver parameters (`mysql+mysqldb`) has its own parameters - those are configured in a way that is specific for the driver (so my.cnf in your case). For example in case of mysql +mysqldb - you can start from this page - which describes mysql dialects of sqlalchemy https://docs.sqlalchemy.org/en/20/dialects/mysql.html > Perhaps you may wish to update the page. Absolutely Zahid. And I have a kind request after you got all the explanations - you are absolutely welcome to propose a PR. Can I ask you to help with it please? When you look at the page with documentation - you can see "Suggest a change on this page" at the bottom right. If you click it, a PR will be opened for you and you will be able to modify the sources of the page in the way you think is best for people like you to better understand. You are probably one of the best people now to update the docs as you struggled with it and got confused and then got helped. So giving back to the community (like 2700+ other contributors) by making the documentation clearer is a great idea. It would be fantastic if a person like you phrases it in a way that will be easier to digest for people like you. Can we count on this ? J. On Fri, Dec 22, 2023 at 5:36 PM Zahid Rahman <zahidr1...@gmail.com> wrote: > > Thanks, I have updated my.cnf > $ cat /etc/mysql/my.cnf > # > # The MySQL database server configuration file. > # > # You can copy this to one of: > # - "/etc/mysql/my.cnf" to set global options, > # - "~/.my.cnf" to set user-specific options. > # > # One can use all long options that the program supports. > # Run program with --help to get a list of available options and with > # --print-defaults to see which it would actually understand and use. > # > # For explanations see > # http://dev.mysql.com/doc/mysql/en/server-system-variables.html > > # > # * IMPORTANT: Additional settings that can override those from this file! > # The files must end with '.cnf', otherwise they'll be ignored. > # > > !includedir /etc/mysql/conf.d/ > !includedir /etc/mysql/mysql.conf.d/ > explicit_defaults_for_timestamp=1 > --------------------------EOF my.cnf -------------------- > > When I said it worked I meant the two warning messages disappeared when > you first start > airflow with commands > $ airflow standalone > by making these two changes in airflow.cfg > *from * > executor = SequentialExecutor > > *to*executor = LocalExecutor > *&* > *from* > sql_alchemy_conn = sqlite:////home/zahid/airflow/airflow.db > *to* > sql_alchemy_conn = mysql+mysqldb://airflow_user:airflow_pass@localhost > :3306/airflow_db > > > *What about* variable sql_engine_collation_for_ids=utf8mb3_bin mentioned > next on same page > > https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database > should I set that in /etc/mysql/my.cnf also ? > > In the airflow.cfg the default setting is sql_engine_encoding = utf-8 > Should I change from sql_engine_encoding = utf-8 in airflow,cfg to > sql_engine_encoding = utf8mb4 in airflow.cfg if I am changing > sql_engine_collation_for_ids=utf8mb3_bin in /etc/mysql/my.cnf * ?* > > > Regards > Zahid > > http://www.frontbutton.com/ > ¯\_(ツ)_/¯ > ♡۶♡ ۶♡۶ > > <http://www.backbutton.co.uk> > > > On Fri, 22 Dec 2023 at 19:33, Lars Winderling <lars.winderl...@posteo.de> > wrote: > >> Hi Zahid, >> >> the additional MySQL config for the timestamps should go into a config >> file for MySQL. By default, it is called my.cnf. But you can pass it as a >> cli option, when starting the mysql daemon (as cited in your message). As >> an option to the mysql daemon mysqld, it defines the behavior of only the >> mysql daemon, so you cannot set it on client side. The airflow.cfg is the >> wrong place. It might work, but maybe in the long run not as expected due >> to quirks with MySQL not following the SQL standard very closely. >> Best, Lars >> >> >> On 22 December 2023 15:07:07 CET, Zahid Rahman <zahidr1...@gmail.com> >> wrote: >> >>> Hi, >>> >>> It was unclear from this page >>> >>> https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database >>> which is a link from the airflow home when airflow first started. >>> that I need to update the *airflow.cfg* to configure the mysql >>> database like so. >>> >>> # sql_alchemy_conn = sqlite:////home/zahid/airflow/airflow.db >>> sql_alchemy_conn = mysql+mysqldb://airflow_user:airflow_pass@localhost >>> :3306/airflow_db >>> >>> Perhaps you may wish to update the page. >>> >>> ALSO >>> I did not make any change to file "my.cnf" nor do I have one that I >>> found in the ~/airflow as described below. >>> >>> "We rely on more strict ANSI SQL settings for MySQL in order to have >>> sane defaults. Make sure to have specified >>> explicit_defaults_for_timestamp=1 option under [mysqld] section in your >>> my.cnf file. You can also activate these options with the >>> --explicit-defaults-for-timestamp switch passed to mysqld executable" >>> >>> This instruction did confuse me a bit at first until I took the guess >>> to make change in the airflow.cfg and *it worked!* >>> >>> Regards >>> Zahid >>> >>> http://www.frontbutton.com/ >>> ¯\_(ツ)_/¯ >>> ♡۶♡ ۶♡۶ >>> >>> <http://www.backbutton.co.uk> >>> >>
