potiuk commented on PR #23971:
URL: https://github.com/apache/airflow/pull/23971#issuecomment-1184398617
Comment on that one (and I want to make it a "testing" ground a bit
@kazanzhy @eladkal ). We need to work out some ways to take care into account
that core-sql is shared by multiple providers. It solves one problem (i.e. how
to deliver new functionality faster) but it does not solve the problem of "how
to avoid backwards compatibility problems and cross-dependency problems).
I am not sure if we get any change here which will make backwards
compatibility problems, but it is likely we can be at least tempted to. And we
need to very consciously decide what we do and what's even more important, we
need to add some test harness to make sure we do not break such backwards
compatibility accidentally.
There are two possible approaches when we are tempted to add
backward-incompatible change
Solution 1) we do not care about backwards compatibility and we bump all (or
some) providers to use >= 2.0.0 of common-sql.
a) we upgrade some providers to >=2.0.0 - only those that do not work
without the breaking changes. Ten we somehow have to make sure that those
providers that do not have >= 2.0.0 still work even with 2.0.0. Some test
harness woudl be needed, ideally running all tests with the common-sql provider
1.*. And we have to make sure of that for all providers otherwise we will land
in the same place as 1b).
b) we upgarde all providers to >=2.0.0. Then we do not need test harness
to test <1.0.0, but it introduces strong coupling between providers -
esentially we need to upgrade all providers that use `common-sql together`. I
particularly do not like this solution - we have too many unrelated sql
providers here (22!). We cannot "bind them" together and force upgrading all of
them when one is upgraded (which will effectively happen if we do it this way).
Soluition 2) We VERY CAREFULLY make sure that core-sql providers is
backwards compatible. We only add new featiures, but do not add breaking
changes almost at all cost (we mightdeprecate some and then when the time comes
we implement 1) - either for all or some providers.
I thnk for now (unless someone has a better idea) we should default to "do
not do breaking changes" (Solution 2) ) unless we have a very good reason (and
then we likely go to 2).
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
