sjyangkevin commented on issue #54714: URL: https://github.com/apache/airflow/issues/54714#issuecomment-3698346447
Hi @Lee-W and @amoghrajesh , hope you enjoy the holiday season. I wanted to follow up on some questions as I found almost half of the list have been implemented, as mentioned in https://github.com/apache/airflow/issues/54714#issuecomment-3220353438. Therefore, I explored the codebase and re-organized the list a little bit. I will post specific question along with each module below. As discussed before, we will need to create a new code (e.g., AIR321) for changes applied in 3.1. Is there anyway I can verify which modules should be part of the new code, and which of those should not? Thanks! # airflow.utils module - [ ] airflow.utils.setup_teardown.BaseSetupTeardownContext → airflow.sdk.definitions._internal.setup_teardown.BaseSetupTeardownContext - [ ] airflow.utils.setup_teardown.SetupTeardownContext → airflow.sdk.definitions._internal.setup_teardown.SetupTeardownContext - [ ] airflow.utils.xcom.XCOM_RETURN_KEY → airflow.models.xcom.XCOM_RETURN_KEY - [ ] airflow.utils.task_group.TaskGroup → airflow.sdk.TaskGroup - [ ] airflow.utils.task_group.get_task_group_children_getter → airflow.sdk.definitions.taskgroup.get_task_group_children_getter - [ ] airflow.utils.task_group.task_group_to_dict → airflow.sdk.definitions.taskgroup.task_group_to_dict - [ ] **airflow.utils.timezone.*** (move everything under) → airflow.sdk.timezone - [ ] airflow.utils.decorators.remove_task_decorator → airflow.sdk.definitions._internal.decorators.remove_task_decorator - [ ] airflow.utils.decorators.fixup_decorator_warning_stack → airflow.sdk.definitions._internal.decorators.fixup_decorator_warning_stack # airflow.sensors module - [x] (AIR301) airflow.sensors.base.BaseSensorOperator → <strike>airflow.sdk.bases.sensor.BaseSensorOperator</strike> [airflow.sdk.BaseSensorOperator](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/suggested_to_update_3_0.rs#L303) - [x] (AIR311) airflow.sensors.base.PokeReturnValue → <strike>airflow.sdk.bases.sensor.PokeReturnValue</strike> [airflow.sdk.PokeReturnValue](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/suggested_to_update_3_0.rs#L303) - [x] (AIR311) airflow.sensors.base.poke_mode_only → <strike>airflow.sdk.bases.sensor.poke_mode_only</strike> [airflow.sdk.poke_mode_only](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/suggested_to_update_3_0.rs#L303) - [x] (AIR312) airflow.sensors.python.PythonSensor → airflow.providers.standard.sensors.python.PythonSensor - [x] (AIR312) airflow.sensors.bash.BashSensor → airflow.providers.standard.sensors.bash.BashSensor - [x] (AIR312) airflow.sensors.date_time.DateTimeSensor → airflow.providers.standard.sensors.date_time.DateTimeSensor - [x] (AIR312) airflow.sensors.date_time.DateTimeSensorAsync → airflow.providers.standard.sensors.date_time.DateTimeSensorAsync > `BaseSensorOperator`, `PokeReturnValue`, and `poke_mode_only` are moved directly under `airflow.sdk` instead of `airflow.sdk.bases.sensor`. **Question**: Should I update `module: "airflow.sdk"` to `module: "airflow.sdk.bases"`? I think `BaseSensorOperator` and `PokeReturnValue` are exposed from the `__init__.py` but `poke_mode_only` is not. Correct me if I am wrong. # airflow.secrets module - [ ] airflow.secrets.cache.SecretCache → airflow.sdk.execution_time.cache.SecretCache > There is an [airflow.secrets.cache.SecretCache](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs#L727) → `airflow.sdk.SecretCache` in AIR301. **Question**: Should we move it into `sdk` or `execution_time`? # airflow.notifications module - [x] (AIR301) airflow.notifications.basenotifier.BaseNotifier → airflow.sdk.bases.notifier.BaseNotifier # airflow.models module - [ ] airflow.models.abstractoperator.AbstractOperator → airflow.sdk.definitions._internal.abstractoperator.AbstractOperator - [ ] airflow.models.abstractoperator.NotMapped → airflow.sdk.definitions._internal.abstractoperator.NotMapped - [ ] airflow.models.abstractoperator.TaskStateChangeCallback → airflow.sdk.definitions._internal.abstractoperator.TaskStateChangeCallback - [x] (AIR311) airflow.models.param.Param → airflow.sdk.definitions.param.Param - [x] (AIR311) airflow.models.param.ParamsDict → airflow.sdk.definitions.param.ParamsDict - [ ] airflow.models.baseoperator.BaseOperator → airflow.sdk.bases.operator.BaseOperator - [x] (AIR311) airflow.models.baseoperator.chain → <strike>airflow.sdk.bases.operator.chain</strike> [airflow.sdk.chain](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/suggested_to_update_3_0.rs#L280) > `airflow.models.baseoperator.chain` is moved under `airflow.sdk.chain` under AIR311. **Question**: Should we move `airflow.models.baseoperator.chain` to `airflow.sdk.chain` or `airflow.sdk.bases.operator.chain`? # airflow.macros module - [ ] **airflow.macros.*** (move everything under) → airflow.sdk.execution_time.macros # airflow.io module - [ ] airflow.io.get_fs → airflow.sdk.io.get_fs - [ ] airflow.io.has_fs → airflow.sdk.io.has_fs - [ ] **airflow.io.attach → airflow.sdk.io.attach** - [ ] airflow.io.Properties → airflow.sdk.io.Properties - [x] (AIR311) airflow.io.path.ObjectStoragePath → airflow.sdk.ObjectStoragePath > There is an [airflow.io.store.attach](https://github.com/astral-sh/ruff/blob/77ad1076173144e9877494eec49fc5c77d2b8c4f/crates/ruff_linter/src/rules/airflow/rules/suggested_to_update_3_0.rs#L253) → `airflow.sdk.io.attach` in AIR311, not sure if it is the same as `airflow.io.attach` **Question**: wondering if `airflow.io.attach` is the same as `airflow.io.store.attach`. # airflow.hooks module - [x] (AIR312) airflow.hooks.filesystem.FSHook → airflow.providers.standard.hooks.filesystem.FSHook - [x] (AIR301) airflow.hooks.base.BaseHook → airflow.sdk.bases.hook.BaseHook # airflow.decorators module - [x] (AIR311) airflow.decorators.dag → airflow.sdk.dag - [x] (AIR311) airflow.decorators.task → airflow.sdk.task - [x] (AIR311) airflow.decorators.task_group → airflow.sdk.task_group - [x] (AIR311) airflow.decorators.base.DecoratedOperator → airflow.sdk.bases.decorator.DecoratedOperator -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
