Sure - anything more you think would be useful is a great contribution to the docs.
On Fri, Dec 22, 2023 at 9:48 PM Zahid Rahman <zahidr1...@gmail.com> wrote: > >> Can we count on this ? > Sure I looked all over the internet to configure mysql to overcome the > warning message. > There is a lot more work required then I have mentioned so far. > > http://www.frontbutton.com/ > ¯\_(ツ)_/¯ > ♡۶♡ ۶♡۶ > > <http://www.backbutton.co.uk> > > > On Sat, 23 Dec 2023 at 00:28, Jarek Potiuk <ja...@potiuk.com> wrote: > >> > *What about* variable sql_engine_collation_for_ids=utf8mb3_bin >> mentioned next on same page >> > >> https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database >> > should I set that in /etc/mysql/my.cnf also ? >> >> > In the airflow.cfg the default setting is sql_engine_encoding = utf-8 >> > Should I change from sql_engine_encoding = utf-8 in airflow,cfg to >> sql_engine_encoding = utf8mb4 in airflow.cfg if I am changing >> sql_engine_collation_for_ids=utf8mb3_bin in /etc/mysql/my.cnf * ?* >> >> No - those are "airflow" configuration settings, not MySQL driver >> settings. The former are described in >> https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html, >> the latter - in the driver documentation (when you pass the URL to >> sqlalchemy, you actually select the driver to use, and this driver >> parameters (`mysql+mysqldb`) has its own parameters - those are configured >> in a way that is specific for the driver (so my.cnf in your case). For >> example in case of mysql +mysqldb - you can start from this page - which >> describes mysql dialects of sqlalchemy >> https://docs.sqlalchemy.org/en/20/dialects/mysql.html >> >> >> > Perhaps you may wish to update the page. >> >> Absolutely Zahid. And I have a kind request after you got all the >> explanations - you are absolutely welcome to propose a PR. Can I ask you to >> help with it please? >> >> When you look at the page with documentation - you can see "Suggest a >> change on this page" at the bottom right. If you click it, a PR will be >> opened for you and you will be able to modify the sources of the page in >> the way you think is best for people like you to better understand. You are >> probably one of the best people now to update the docs as you struggled >> with it and got confused and then got helped. So giving back to the >> community (like 2700+ other contributors) by making the documentation >> clearer is a great idea. >> >> It would be fantastic if a person like you phrases it in a way that will >> be easier to digest for people like you. >> >> Can we count on this ? >> >> J. >> >> On Fri, Dec 22, 2023 at 5:36 PM Zahid Rahman <zahidr1...@gmail.com> >> wrote: >> >>> >>> Thanks, I have updated my.cnf >>> $ cat /etc/mysql/my.cnf >>> # >>> # The MySQL database server configuration file. >>> # >>> # You can copy this to one of: >>> # - "/etc/mysql/my.cnf" to set global options, >>> # - "~/.my.cnf" to set user-specific options. >>> # >>> # One can use all long options that the program supports. >>> # Run program with --help to get a list of available options and with >>> # --print-defaults to see which it would actually understand and use. >>> # >>> # For explanations see >>> # http://dev.mysql.com/doc/mysql/en/server-system-variables.html >>> >>> # >>> # * IMPORTANT: Additional settings that can override those from this >>> file! >>> # The files must end with '.cnf', otherwise they'll be ignored. >>> # >>> >>> !includedir /etc/mysql/conf.d/ >>> !includedir /etc/mysql/mysql.conf.d/ >>> explicit_defaults_for_timestamp=1 >>> --------------------------EOF my.cnf -------------------- >>> >>> When I said it worked I meant the two warning messages disappeared when >>> you first start >>> airflow with commands >>> $ airflow standalone >>> by making these two changes in airflow.cfg >>> *from * >>> executor = SequentialExecutor >>> >>> *to*executor = LocalExecutor >>> *&* >>> *from* >>> sql_alchemy_conn = sqlite:////home/zahid/airflow/airflow.db >>> *to* >>> sql_alchemy_conn = mysql+mysqldb://airflow_user:airflow_pass@localhost >>> :3306/airflow_db >>> >>> >>> *What about* variable sql_engine_collation_for_ids=utf8mb3_bin >>> mentioned next on same page >>> >>> https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database >>> should I set that in /etc/mysql/my.cnf also ? >>> >>> In the airflow.cfg the default setting is sql_engine_encoding = utf-8 >>> Should I change from sql_engine_encoding = utf-8 in airflow,cfg to >>> sql_engine_encoding = utf8mb4 in airflow.cfg if I am changing >>> sql_engine_collation_for_ids=utf8mb3_bin in /etc/mysql/my.cnf * ?* >>> >>> >>> Regards >>> Zahid >>> >>> http://www.frontbutton.com/ >>> ¯\_(ツ)_/¯ >>> ♡۶♡ ۶♡۶ >>> >>> <http://www.backbutton.co.uk> >>> >>> >>> On Fri, 22 Dec 2023 at 19:33, Lars Winderling <lars.winderl...@posteo.de> >>> wrote: >>> >>>> Hi Zahid, >>>> >>>> the additional MySQL config for the timestamps should go into a config >>>> file for MySQL. By default, it is called my.cnf. But you can pass it as a >>>> cli option, when starting the mysql daemon (as cited in your message). As >>>> an option to the mysql daemon mysqld, it defines the behavior of only the >>>> mysql daemon, so you cannot set it on client side. The airflow.cfg is the >>>> wrong place. It might work, but maybe in the long run not as expected due >>>> to quirks with MySQL not following the SQL standard very closely. >>>> Best, Lars >>>> >>>> >>>> On 22 December 2023 15:07:07 CET, Zahid Rahman <zahidr1...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> It was unclear from this page >>>>> >>>>> https://airflow.apache.org/docs/apache-airflow/2.8.0/howto/set-up-database.html#setting-up-a-mysql-database >>>>> which is a link from the airflow home when airflow first started. >>>>> that I need to update the *airflow.cfg* to configure the mysql >>>>> database like so. >>>>> >>>>> # sql_alchemy_conn = sqlite:////home/zahid/airflow/airflow.db >>>>> sql_alchemy_conn = mysql+mysqldb://airflow_user:airflow_pass@localhost >>>>> :3306/airflow_db >>>>> >>>>> Perhaps you may wish to update the page. >>>>> >>>>> ALSO >>>>> I did not make any change to file "my.cnf" nor do I have one that I >>>>> found in the ~/airflow as described below. >>>>> >>>>> "We rely on more strict ANSI SQL settings for MySQL in order to have >>>>> sane defaults. Make sure to have specified >>>>> explicit_defaults_for_timestamp=1 option under [mysqld] section in >>>>> your my.cnf file. You can also activate these options with the >>>>> --explicit-defaults-for-timestamp switch passed to mysqld executable" >>>>> >>>>> This instruction did confuse me a bit at first until I took the guess >>>>> to make change in the airflow.cfg and *it worked!* >>>>> >>>>> Regards >>>>> Zahid >>>>> >>>>> http://www.frontbutton.com/ >>>>> ¯\_(ツ)_/¯ >>>>> ♡۶♡ ۶♡۶ >>>>> >>>>> <http://www.backbutton.co.uk> >>>>> >>>>