potiuk commented on PR #23971: URL: https://github.com/apache/airflow/pull/23971#issuecomment-1185253215
> I'm trying to say that TTL for data pipelines is not so long, but new features are important for all users Yeah. The whole thing about mixed-governance and ability to release older version of providers was behind that thinking (@eladkal was big proponent of that and his comments actually triggered the whole discssion) we are yet to see in the coming months how it plays out. I know Google is working on preparing 6.* and 7* cherry-picks with backwards compatibility fixes which will also make it possible for use to remove deprecations more aggreesively in newer versions of Google provider (and likely other big ones willl follow). I also thing we should be prepared for when we will want to remove deprecations in "common.sql" and have some scenarios prepared for that. I do not yet know how though :). Maybe we can find some crative ways to handle it without "coupling" the dependent providers - this "coupling" is what worries me most actually and finding a good solution for that would be nice. I thought a bit about it and I tihnk we have an interesting possibility. We could potentially vendor-in older versions of "common.sql" in the providers that require old versions OR we could change the package name for future common sql package and have "common-sql2". "common-sql3". While Python does not allow to have two versions of the same package installed at the same time, you **could** have "apache-airlfow-providers-common-sql" and "apache-airflow-prociders-comon-sql2" installed at the same time. There is nothing wrong with that. Or you could repackage old common.sql into a different package name and inline it in the provider that uses it. Both solution are viable and they recall very much what NPM does (where you can have multiple versions of same library installed at the same time and diffferent packages might use different versions). But if we limit it to only "small packages" (like common.sql is),. the overhead incurred (more space and memory used) might be actually quite acceptable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
