potiuk commented on a change in pull request #11079:
URL: https://github.com/apache/airflow/pull/11079#discussion_r492672151
##########
File path: .github/workflows/delete_old_artifacts.yml
##########
@@ -1,11 +1,12 @@
name: 'Delete old artifacts'
on:
schedule:
- - cron: '0 * * * *' # every hour
+ - cron: '27 */6 * * *' # run every 6 hours
Review comment:
There is a bit of a problem - we cannot rate-limit the action and we
occasionally exhausted the limit now.
I did some back-of-the-envelope calculations:
The limit is 5000 API calls / hr. So if we have > 3000 artifacts to delete,
we are getting dangerously close to be able to exhaust our API calls within
single run.
We have (tops) ~ 200 builds a day with (tops) ~ 50 artifacts each (assume
intensive period and increasing number of artifacts) > 10.000 artifacts to
delete a day. Running it 4 times/day is ~ 2.500 artifacts to delete for each
run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
