kaxil commented on pull request #18506: URL: https://github.com/apache/airflow/pull/18506#issuecomment-961510974
> > I am really really not sure about this. I still think we should "officially" support limited DBs and make sure they work 100% with all combinations > > Yeah. That would be ideal. However I believe we have quite "safe" test approach here. > > Happy to discus it further but I think we have quite a room for optimising the tests. > > I believe providers are not even supposed to know anything about the underlying DB. They will - at most - use Variables/Connections etc. I am not sure if there is a value on running Google Provider vs. MySQL or Postgres DB. I can't easily imagine any case that might casue error from Provider uses that will be actually db-dependent. I'd love to see what those could be? > > There might be some very specific exceptions (like some "core extensions") but those can be easily separated. > > I honestly can't remember (but of course I can't remember all cases) where we had a "Provider"-originated failed tests only causing Postges or MySQL or Sqlite failures (except the resources/temporary/intermittent errors). > > And that might even be an interesting step for the future "multi-tenancy" work possibly - any of the Providers should never touch the DB directly. I think they should only access the Variables/Connections etc. - generally the "internal APIs" exposed by Airflow. The "no direct access to db" might even be - in the future - a "criteria" to pass tests by a provider - so getting to the point where we can safely say "no direct metadata db access from providers" might be an important goal to achieve for all comunity providers (and it could be tested/enforced). > > For other, non-provider tests - I am quite certain (but I have not yet looked at it in detail) that we have a number of tests that are not even touching th DB / SQLAlachemy already (even in "core" of airflow). It's very likely we can easily separate some (or even a lot) of them and we could even be 100% sure of their "no DB" access. We could mock the sqlalchemy acces in the way that whenever it happens the test will faill. Such tests simply need not be run with different DBs. It's not just tests, it's DB Migrations that needs to be taken care of. We are creating some debt already if lots of "if.. mssql do this" .. if "mariab do Y". And while we already know that there are good number of issues with MySQL and MariaDB, I would not be in favor of officially supporting MariaDB. "best effort" is good tbh. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
