amoghrajesh opened a new issue, #68214: URL: https://github.com/apache/airflow/issues/68214
### Body Airflow 3.2 added async `@task` support, and 3.3 added resumable tasks with `task_store.get/set` for checkpointing. These two features are currently independent, ie async tasks can't use task store for crash recovery because get/set are synchronous and would block the event loop. ### What to do? Add async counterparts `await task_store.aget(key)` and `await task_store.aset(key, value)` on TaskStoreAccessor, wired to an async method on BaseStoreBackend and its db backend implementation. This would enable stateful async tasks e.g. an async task paginating through an API that can checkpoint after each page and resume after a crash without restarting from page 1. #### Acceptance criteria - `TaskStoreAccessor` exposes `aget` and `aset` - `BaseStoreBackend` defines abstract `aget/aset` - `MetastoreStoreBackend` implements them using an async DB session - Async @task examples in docs updated to show the pattern ### Committer - [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
