ldacey commented on pull request #6850:
URL: https://github.com/apache/airflow/pull/6850#issuecomment-831989010
Recently added support for Snowflake so I thought I would share it in case
anyone else is using turbodbc and Airflow. The nice part about this is that my
data goes straight from the database to a partitioned parquet dataset without
worrying about types changing (due to pandas or reading in CSV data). I can use
a single "OdbcToAzureBlobOperator" for all of my SQL data sources.
Driver installation in my Dockerfile:
```
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
&& apt-get update && apt-get -y dist-upgrade \
&& curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list >
/etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y \
msodbcsql17 \
mssql-tools \
&& curl -SL
https://dev.mysql.com/get/downloads/mysql-connector-odbc-8.0.23-linux-glibc2.12-x86-64bit.tar.gz
| tar -zxC /opt \
&& cp
/opt/mysql-connector-odbc-8.0.23-linux-glibc2.12-x86-64bit/lib/libmyodbc8*
/usr/lib/x86_64-linux-gnu/odbc/ \
&&
/opt/mysql-connector-odbc-8.0.23-linux-glibc2.12-x86-64bit/bin/myodbc-installer
-d -a -n "MySQL ODBC 8.0 ANSI Driver" -t
"DRIVER=/usr/lib/x86_64-linux-gnu/odbc/libmyodbc8a.so;" \
&&
/opt/mysql-connector-odbc-8.0.23-linux-glibc2.12-x86-64bit/bin/myodbc-installer
-d -a -n "MySQL ODBC 8.0 Unicode Driver" -t
"DRIVER=/usr/lib/x86_64-linux-gnu/odbc/libmyodbc8w.so;" \
&& curl -SL
https://sfc-repo.azure.snowflakecomputing.com/odbc/linux/2.23.1/snowflake_linux_x8664_odbc-2.23.1.tgz
| tar -zxC /opt \
&& mv /opt/snowflake_odbc /usr/lib/x86_64-linux-gnu/odbc/ \
&& /usr/lib/x86_64-linux-gnu/odbc/snowflake_odbc/iodbc_setup.sh \
```
Sample of Extras from each database type:
Snowflake:
```
{"Driver": "SnowflakeDSIIDriver", "warehouse": "warehousename", "role":
"rolename"}
```
MS SQL:
```
{"Driver": "ODBC Driver 17 for SQL Server", "ApplicationIntent": "ReadOnly",
"TrustedConnection": "Yes"}
```
MySQL:
```
{"Driver": "MySQL ODBC 8.0 Unicode Driver", "Threading": 2, "INITSTMT": "set
session time_zone ='+00:00'"}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
