zhangyi-hu opened a new issue #17623: URL: https://github.com/apache/airflow/issues/17623
<!-- Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions. Don't worry if they're not all applicable; just try to include what you can :-) If you need to include code snippets or logs, please put them in fenced code blocks. If they're super-long, please use the details tag like <details><summary>super-long log</summary> lots of stuff </details> Please delete these comment blocks before submitting the issue. --> **Description** Currently, Airflow celery executor has this config in its default value, one day. As a user, I hope I can set this value by myself through Airflow easily. **Use case / motivation** When you spot a silent mistake, where the task is successful but the results are wrong, you find the bug in your ETL workflow and fixed the bug. However, you cannot easily rerun because the celery result backend has the results cached for one day by default. You can ask airflow the rerun that task for sure, however your celery worker will immediately return the cached results without actually doing anything. Instead of waiting for a day, you end up manually truncate the celery_metatask table. However, this hack can be avoided if we set that celery config as one hour instead of one day. **Are you willing to submit a PR?** Unfortunately, I don't have time for this recently. **Related Issues** I also believe when forcing rerun a dag, with celery executor, we should remove the celery task run history through the celery api. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
