Hi,
On Fri, 2020-10-09 at 14:24 +0000, Benjamin Schubert wrote:
> Hey everyone,
>
> TLDR: I would like a way of allowing network access during builds on a
> per-element basis, to ease transition into BuildStream.
>
> I realize this has a potential to be highly controversial, but I think that
> the benefits in terms of ease of adoption makes it worth it.
>
In my opinion, this has the potential to cause more harm than good.
A good question is, if you are going to poke holes in the sandbox and
allow network access, what's the point of using BuildStream at all ?
The cache key loses a lot of meaning and no longer provides the kind of
guarantees it should, that at least the source code being built is
unique for the cache key...
If network access were "supported", this could lead us down a road
where we are requested to rebuild things when artifacts are too old
(for instance, I cannot build against this dependency, because the
random source code it downloaded from the internet only 3 months ago is
no longer valid for me to build against).
Mostly I worry that if we give users any opportunity to allow internet
access in builds, we almost guarantee that they will not do the minimal
legwork required to ensure their build works without network access.
Fixing your element to work without internet access can sometimes be a
pesky and annoying task, and it is one which only pays off in the long
run, which is where BuildStream provides your project real value. If
people have an escape hatch to this, they will take it - and then when
they suffer the consequences (only months or years later), they will be
right to blame us for our false promises regarding repeatability.
I think this proposal is essentially an easy way to arm users against
themselves, and I worry much more about the negative impacts this can
have than I worry about the initial workload it takes to adopt a tool
which forces you build your code repeatably and reliably.
Cheers,
-Tristan
> Problem
> -----------
> In some cases, it is very hard to completely sandbox a build, with
> some build systems absolutely wanting to go to the network. There are
> workaround to go over those, however, having to completely fix them before
> moving sizeable projects to BuildStream makes it very hard to adopt.
>
> In other cases, you might want access to network resources for testing
> elements
> that would require some network access
>
> I know and fully understand that having network access during a sandbox build
> removes many of the benefits of BuildStream (source caching, repeatable
> builds),
> but those can be addressed at an infrastructure level before being moved to
> work as it should in BuildStream.
>
> How would that look like?
> --------------------------
>
> I would envision a new variable option, similar to 'max-jobs', that would be
> transparent to the plugin, but used by BuildStream to open network access.
>
> How much work would we need to do for this?
> ------------------------------------------
>
> We already have mechanisms to open the sandbox when running a shell.
> We would need to make this possible for regular builds.
>
> This would also mean we need to be able to be able to set it on platform
> properties for remote execution.
>
> Mitigations
> -----------
>
> Some potential mitigations around this being 'bad' could be:
>
> - Optionally disable by default the pushing of such artifacts to remote
> caches,
> like we currently handle workspaces, with the ability to enable it either at
> the project level or the element level.
>
> Any thoughts?
>
> Thanks!
>
> Ben
