mobuchowski opened a new pull request, #44913:
URL: https://github.com/apache/airflow/pull/44913
This hook ensures API stability of common.compat provider by validating
method signatures between staged and committed versions of Python files
- detect removed methods
- prevent removal of existing arguments
- require default values for new arguments
Few things missing: how to handle imports? Important part of common.compat
provider is handling different import paths for different airflow version - for
example dataset->asset rename.
However, at some point we want to drop old version and remove old import.
Also, we can just import something for internal use in some method, it
should not be an API surface.
Another is nested classes/methods: this wouldn't detect change in
NoOpCollector:
```
def get_hook_lineage_collector():
# Dataset has been renamed as Asset in 3.0
if AIRFLOW_V_3_0_PLUS:
from airflow.lineage.hook import get_hook_lineage_collector
return get_hook_lineage_collector()
# HookLineageCollector added in 2.10
if AIRFLOW_V_2_10_PLUS:
return _get_asset_compat_hook_lineage_collector()
# For the case that airflow has not yet upgraded to 2.10 or higher,
# but using the providers that already uses `get_hook_lineage_collector`
class NoOpCollector:
"""
NoOpCollector is a hook lineage collector that does nothing.
It is used when you want to disable lineage collection.
"""
# for providers that support asset rename
def add_input_asset(self, *_, **__):
pass
def add_output_asset(self, *_, **__):
pass
# for providers that do not support asset rename
def add_input_dataset(self, *_, **__):
pass
def add_output_dataset(self, *_, **__):
pass
return NoOpCollector()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
