gtristan commented on PR #2012: URL: https://github.com/apache/buildstream/pull/2012#issuecomment-2919195448
@harrysarson: > With the split-rule + compose approach (that requires the changes in this PR) we give the element that integrates the elements the choice on whether to remove the offending file, if perf isn't going into the disk image then it would make sense to be able to keep the other version of trace. Sure. I think that the approach you currently taking *is sensible*. That said, I'm not happy with the implementation you propose, and I've asked for more thinking in my above comment https://github.com/apache/buildstream/pull/2012#issuecomment-2900997466 to find an API that is more all around generally useful. One approach I've suggested there, would be to instead have a separate splitting at staging time (e.g. it could be `stage-include`/`stage-exclude`/`stage-orphans` kind of API to be applied at staging, before integration occurs). However, I rather like @abderrahim's line of thinking too: > FWIW, I've been thinking about this "filter while staging" idea. I feel that it makes more sense as a dependency configuration that is more generically applicable rather than being specific to compose. > > This would make it usable for other use cases as well. The way the code is structured, at least some duplication is needed in base Element classes to do the dependency configuration support, that said I agree it can make sense to do some kind of filtering at staging time for script, compose and build elements (probably with the same consistent API). Expressing the dependency on *partial artifacts* is also reminiscent of past suggestions from @sstriker, the idea you propose makes me wonder if just *having this data expressed* might leave some window open to improve *build avoidance* in the future (i.e. *"If the part of my dependency artifact which I use, has not changed, I don't need to rebuild"*), while this is a bit far fetched given the nature of cache keys, it's at least worth noting in the context of this discussion. Orthogonal to this (but somewhat relevant to this conflicting *"/usr/bin/perf"* program), something that I've been mulling over, is; *why do we limit ourselves to filtering* ? I think that elements like *compose* and *filter* which do these inclusions/exclusions, might also benefit from the ability to do *relocations*, i.e. renaming/relocating files on a per-split or per-file basis. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
