gtristan commented on PR #2012:
URL: https://github.com/apache/buildstream/pull/2012#issuecomment-2900997466
I think this needs some deeper thought.
Right now compose does:
```
stage dependencies in /
|
V
run integration commands
|
V
apply splits to resulting
filesystem after integration commands
|
V
place selected files into %{install-root}
```
The problem you are trying to solve, is that this approach conflicts with
overlaps, in the case that initially staged artifact dependency chains have
multiple artifacts which provide the same groups of files.
In what ways can we address this ?
* It seems to me that we would have this overlap problem even if it was a
build element and not a compose element, so the question of *why are these
files overlapping* is relevant
* Does it make more sense for the overlapping element to use the [overlap
whitelist](https://docs.buildstream.build/master/format_public.html#overlap-whitelist)
?
* If we address this in the way that you propose
* We render the integration command file capturing possibly useless in
cases with the proposed option added
* We are applying splits for different *reasons*
* It may be that what we want is to have a *different set* of
`include`/`exclude` elements in the staging process, that molds the input in
advance of integration commands and before the *original set* of
`include`/`exclude` rules are employed to curate the *output* of the compose
element.
* In the case of handling overlap errors raised in compose's
`Element.stage()` implementation, I wonder if it makes sense to carve out an
escape hatch of sorts ?
* I.e. if it is valid to have overlap errors enabled in a project, does it
still make sense to not enforce overlap whitelisting ?
* If we can deem that it is indeed a case worthy of a special case escape
hatch for the specific case of compose elements, would it make sense to
(optionally) catch the exception raised by
`Element.stage_dependency_artifacts()` and cause it to be non-fatal in this
case ?
* Would it change something if, for example, we handled the case where
integration commands are not going to run differently, and just do the
`Element.stage_dependency_artifacts()` with the `include/exclude/orphans`
options, directly into `%{install-root}`, and circumvent the issue in this case
?
If we're going to add configuration here, we should consider what will be
maximally useful for different possible use-cases.
Also, I think it would be good to avoid breaking expectations of setting
[overlap
warnings](https://docs.buildstream.build/master/buildstream.types.html#buildstream.types.CoreWarnings)
as [fatal
warnings](https://docs.buildstream.build/master/format_project.html#configurable-warnings)
(as mentioned above, it seems telling that this error you are trying to
circumvent would still be occurring if this were a build element and not a
compose element).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
