kaxil opened a new pull request, #74608: URL: https://github.com/apache/airflow/pull/74608
The TaskFlow tutorial shows `@task(retries=3)` but doesn't say that a retry runs the whole function again from the start. For a task that works through a long list of items, such as copying files, a failure on the last file means the retry copies every file again, and nothing in the tutorial points to the way around it. This adds a short paragraph after the `retries` example in "Error Handling and Retries" that points to [resumable tasks](https://github.com/apache/airflow/blob/bf2de26c7b7ee1cd96f6086ed1f1c8140f55c825/airflow-core/docs/core-concepts/resumable-tasks.rst?plain=1#L83-L133) and the task state store. That section is where the tutorial first introduces retries, so the reader learns the restart behaviour and the fix in the same place. The resumable tasks page already covers this case with a `process_files` task that checkpoints `last_processed`. --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
