potiuk commented on issue #17546: URL: https://github.com/apache/airflow/issues/17546#issuecomment-898905167
> 1. Without any dependency with MySQL. airflow still depend on mysql. It does not. Airflow only uses mysql when mysql is configured via sqlalchemy URL https://docs.sqlalchemy.org/en/14/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqlconnector (This actually brings me to another option to try @JavierLopezT ) - maybe using different mysql driver might help as well - those can be chosen by a different connection string. > 2. If import airflow before pandas, the bug disappears. > I'm not sure is this mysql issue or airflow issure I think it's neither. It looks like it is some 3rd-party library (maybe even pandas) which badly uses the feature and causes subsequent initialization of memory via stdlibc++. There are multiple similar examples scattered around the internet (completely unrelated to Airflow) that show similar problems and they also show that exchanging import sequence might fix the problem: For example here: https://www.codegrepper.com/code-examples/shell/ImportError%3A+%2Fusr%2Flib%2Faarch64-linux-gnu%2Flibgomp.so.1%3A+cannot+allocate+memory+in+static+TLS+block > I had a similar issue where I was unable to import cv2. I can import opencv in console > I was importing tensorflow before cv2. > For some reason switching the order (importing cv2 first) worked. > I am not sure how it worked but it might help you. One common point - all the problems I saw involve python packages that use some deep C++ integrated/compiled parts - opencv, tensorflow, scikit-;earn and so on. That would indeed imply that the problem is pandas. Maybe then simply manually upgrading pandas to latest available version will work (another thing to try @JavierLopezT ). Side-note that for 2.2 we **just** (2 days ago) - got rid of pandas (https://github.com/apache/airflow/pull/17575) and numpy (https://github.com/apache/airflow/pull/17594) as "core" airflow dependency. I think it does not change much in this case (they will simply not be installed by default when just "core" airflow is installed without any extras, but at least it will not be always installed, and if you install it separately (because your DAGs will need it), you will no longer be even "expected" to use the version specified in the constraints -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
