potiuk commented on issue #14989:
URL: https://github.com/apache/airflow/issues/14989#issuecomment-808438815
> Using GitHub Action’s cache sounds like a cool idea. The only concern I
have is the cache can be pretty difficult to purge if something goes wrong
(maybe just me not knowing a good way to do it). Maybe we should have a “cache
busting” part to the key (e.g. cache-inventory-${{
hashFiles('constrant-3.8.txt') }}-${{ cacheKey }} where cacheKey is a variable
in the Action’s config) so we can force a cache miss when we need to.
In this case we will simply change the cache prefix. We've done that several
time. we can always turn it into:
```
key: cache-inventory-v2-${{ hashFiles('constraints-3.8.txt') }}
restore-key: cache-inventory-v2-
```
by changing the .yml file. No need for special variable for that -t can
simply be changed in ci.yml. And I think this cache will be self-healing. As I
understand it the inventory we are using now simply downloads URL and only if
it fails, it reads it from the cache @mik-laj might confirm it maybe?
> I just thought of another thing, does Action’s cache set file creation
date etc. correctly? Because if it does not, we won’t be able to fetch in new
documentation updates (say the `1.4` version of a documentation releases a fix
without changing the URL) since the files we retrieve from the cache will
always be newer.
Highly doubt it. Even if they do, this is not a big problem for us. First of
all the cache will always be secondary choice - so even if we have it from the
cache, the "URL one" will take precedence (Or this is at least how it works ).
And secondly the cache will get "eventually consistent". Every time we change
constraints (should happen regularly - every day or so) the cache will get
refreshed because while we will have it as secondary choice, the main one (URL
pull) will still get the latest version and override and then new version of
the cache will be produced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
