[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-591519540 The issue is the new version of `mysql-connector-python` 8.0.19, don't know exactly what it caused but 8.0.18 works. https://travis-ci.org/apache/airflow/jobs/654452458?utm_medium=notification&utm_source=github_status This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-591519540 The issue is the new version of `mysql-connector-python` 8.0.19, don't know exactly what it caused but 8.0.18 works. Failed tests: https://travis-ci.org/apache/airflow/jobs/654452458?utm_medium=notification&utm_source=github_status This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify
the mysql client library used in MySqlHook
URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933
```
root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import
mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; import json'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; from airflow.hooks.dbapi_hook import DbApiHook; import json'
[2020-02-11 10:31:54,392] {settings.py:179} INFO - settings.configure_orm():
Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=174
root@1b70ba426ae0:/opt/airflow# ls
/usr/local/lib/python3.6/dist-packages/mysql-vendor/
ls: cannot access '/usr/local/lib/python3.6/dist-packages/mysql-vendor/': No
such file or directory
```
(I did not remove the mysql-vendor package)
**EDIT:**
There it is 😁
```
root@1b70ba426ae0:/opt/airflow# ls
/usr/local/lib/python3.6/site-packages/mysql-vendor/
libcrypto.so.1.1 libssl.so.1.1
```
Everything seems to work fine on my local breeze env:
```
Branch name:master
Docker image:
apache/airflow:master-python3.6-ci
Airflow version:2.0.0.dev0
Python version: 3.6
DockerHub user: apache
DockerHub repo: airflow
Backend:mysql
```
I ran `./breeze --force-pull-images --force-build-images`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify
the mysql client library used in MySqlHook
URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933
```
root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import
mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; import json'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; from airflow.hooks.dbapi_hook import DbApiHook; import json'
[2020-02-11 10:31:54,392] {settings.py:179} INFO - settings.configure_orm():
Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=174
root@1b70ba426ae0:/opt/airflow# ls
/usr/local/lib/python3.6/dist-packages/mysql-vendor/
ls: cannot access '/usr/local/lib/python3.6/dist-packages/mysql-vendor/': No
such file or directory
```
(I did not remove the mysql-vendor package)
**EDIT:**
There it is 😁
root@1b70ba426ae0:/opt/airflow# ls
/usr/local/lib/python3.6/site-packages/mysql-vendor/
libcrypto.so.1.1 libssl.so.1.1
Everything seems to work fine on my local breeze env:
```
Branch name:master
Docker image:
apache/airflow:master-python3.6-ci
Airflow version:2.0.0.dev0
Python version: 3.6
DockerHub user: apache
DockerHub repo: airflow
Backend:mysql
```
I ran `./breeze --force-pull-images --force-build-images`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify
the mysql client library used in MySqlHook
URL: https://github.com/apache/airflow/pull/6576#issuecomment-584570933
```
root@1b70ba426ae0:/opt/airflow# python3 -c 'import hashlib; import
mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; import json'
root@1b70ba426ae0:/opt/airflow# python3 -c 'import mysql.connector; import
MySQLdb; from airflow.hooks.dbapi_hook import DbApiHook; import json'
[2020-02-11 10:31:54,392] {settings.py:179} INFO - settings.configure_orm():
Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=174
root@1b70ba426ae0:/opt/airflow# ls
/usr/local/lib/python3.6/dist-packages/mysql-vendor/
ls: cannot access '/usr/local/lib/python3.6/dist-packages/mysql-vendor/': No
such file or directory
```
(I did not remove the mysql-vendor package)
Everything seems to work fine on my local breeze env:
```
Branch name:master
Docker image:
apache/airflow:master-python3.6-ci
Airflow version:2.0.0.dev0
Python version: 3.6
DockerHub user: apache
DockerHub repo: airflow
Backend:mysql
```
I ran `./breeze --force-pull-images --force-build-images`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-78907 We are using `mysql-connector-python` in our airflow-plugins. But I would stay with `mysqlclient` if ssl would work with no cert. Here are some explanation on what the differences are between all those mysql client libs. https://stackoverflow.com/questions/43102442/whats-the-difference-between-mysqldb-mysqlclient-and-mysql-connector-python There is also a performance test 👍 (and `mysqlclient` is A LOT faster) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-36862 That is currently the only one I experienced. ``` .. note:: By default it connects to the database via the mysqlclient library. But you can also choose the mysql-connector-python library which lets you connect through ssl without any further ssl parameters required. ``` Because you cannot use pure ssl without any cert options in `mysqlclient`. This worked in early versions of mysql. https://stackoverflow.com/questions/7287088/ca-ssl-parameter-for-python-mysqldb-not-working-but-key-does And here is the open issue on stackoverflow: https://stackoverflow.com/questions/48441982/python-mysqldb-use-ssl-without-certificate This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-36862 That is currently the only one I experienced. ``` .. note:: By default it connects to the database via the mysqlclient library. But you can also choose the mysql-connector-python library which lets you connect through ssl without any further ssl parameters required. ``` Because you cannot use pure ssl without any cert options in `mysqlclient`. _Let me find some refs._ [This](https://stackoverflow.com/questions/3173650/python-mysqldb-ssl-connection) worked in early versions of mysql. https://stackoverflow.com/questions/7287088/ca-ssl-parameter-for-python-mysqldb-not-working-but-key-does And here is the open issue on stackoverflow: https://stackoverflow.com/questions/48441982/python-mysqldb-use-ssl-without-certificate This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-36862 That is currently the only one I experienced. ``` .. note:: By default it connects to the database via the mysqlclient library. But you can also choose the mysql-connector-python library which lets you connect through ssl without any further ssl parameters required. ``` Because you cannot use pure ssl without any cert options in `mysqlclient`. _Let me find some refs._ [This](https://stackoverflow.com/questions/3173650/python-mysqldb-ssl-connection) worked in early versions of mysql. https://stackoverflow.com/questions/7287088/ca-ssl-parameter-for-python-mysqldb-not-working-but-key-does ... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-36862 That is currently the only one I experienced. ``` .. note:: By default it connects to the database via the mysqlclient library. But you can also choose the mysql-connector-python library which lets you connect through ssl without any further ssl parameters required. ``` Because you cannot use pure ssl without any cert options in `mysqlclient`. _Let me find some refs._ [This](https://stackoverflow.com/questions/3173650/python-mysqldb-ssl-connection) worked in early versions of mysql. ... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553804892 At least I should add tests for the `client` specification. But after you connected with one of these you should be able to execute the same mysql code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
