[
https://issues.apache.org/jira/browse/AIRFLOW-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17222171#comment-17222171
]
Richard Williamson commented on AIRFLOW-2075:
---------------------------------------------
Had some good discussion on this in Slack Apache Airflow Comunity #airflow-gcp
channel - thx for all the feedback!:
[https://apache-airflow.slack.com/archives/CCVDRN0F9/p1603544070099000] - quick
summary is distributed DB options are going to be tough fit currently - some
possible options CockroachDB?, Yugabyte?
([https://blog.yugabyte.com/part-1-deploying-a-distributed-sql-backend-for-apache-airflow-on-google-cloud/)|https://blog.yugabyte.com/part-1-deploying-a-distributed-sql-backend-for-apache-airflow-on-google-cloud/]
, Spanner? (can be tried again if we get sql_alchemy support, but initial
tests had some perf issues), Galera (Mysql8 version instead of Maria due to
locking support - probably will not be a good fit due to locking perf/issues,
but open for someone trying) - would really like to have a good option for this
for multi-region cloud support with 2.0 so please provide feedback!
[~potiuk] - is this the best place to track comments for this for now?
> Extract and create a data access layer
> --------------------------------------
>
> Key: AIRFLOW-2075
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2075
> Project: Apache Airflow
> Issue Type: Improvement
> Components: core
> Affects Versions: 2.0.0
> Reporter: Yang Pan
> Priority: Major
>
> Currently Airflow is tightly coupled with sqlalchemy.
> [https://github.com/search?p=2&q=%22from+sqlalchemy%22+repo%3Aapache%2Fincubator-airflow+path%3A%2Fairflow&type=Code&utf8=%E2%9C%93]
> While it's convenient for sqlalchemy compatible database, it makes non
> compatible ones, such as google cloud spanner, harder to integrate.
> It would be nice if the data access layer can be decoupled from the airflow
> core, and developers have option to use sqlalchemy or swap to talk to db
> directly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
