Hi Tristan, On Mon, 2020-10-05 at 17:08 +0900, Tristan Van Berkom wrote: > What do we foresee in terms of ability to interact with remote caches > in an RE context ? > > I remember older discussions where Sander had the idea that `bst > artifact` commands such as `list-artifacts` and `delete` etc, in > general, should assume we mean the remote cache instead of the local > one (I mostly recall disagreeing with this). > > But the point is, regardless of the particular shape of the future, we > need to have a better idea of what that shape is going to be if we're > going to have a meaningful conversation about how command line options > and configuration is to be expressed. > > Even if that future has not materialized completely, we need some hints > in order to design API stable interfaces in advance of things being > implemented. > > So, I guess my question is; if we have --no-fetch / --no-pull options > respected in the RE context, such that external CAS caches are > explicitly disabled and only the main RE CAS is consulted, and we have > a way to at least perform `bst artifact delete` on a remote CAS, e.g. a > remote CAS used for the configured RE cluster... then would it still be > reasonable to say that we actually *can* do this with the same user > facing interfaces in an RE context ?
If we add support for `bst artifact delete` with remote artifact servers, this would send a message to the Remote Asset server, not to the CAS server. I.e., this would delete the artifact reference, not necessarily the CAS blobs (the asset server might trigger pruning of unused blobs as well but that would be up to the server, not BuildStream). As (artifact) RA server configuration is independent of Remote Execution, I expect such a remote artifact delete operation to work exactly the same independent of whether the BuildStream project is configured for local or remote execution. However, with regards to repeating a build using remote execution, BuildStream will have to set `skip_cache_lookup` in the `ExecuteRequest` message, otherwise a result from the action cache may be reused even if all artifact references have been deleted beforehand. This means that if we want to support this, BuildStream might require a flag such as `--skip-cache-lookup=foo.bst`, which could be used for both local and remote execution. Cheers, Jürg
