Bryce L Nordgren <[email protected]> writes: > I'm not saying you shouldn't. But the current build system finds out > about the breakage, so the maintainer can investigate and put the extra > dependency there. You have to understand that most package expressions > are probably created as a bare minimum, just adding the stuff the > builder/installer complains about. This works fine most of the time. > So in this case, where (up till now) just depending on python sufficed, > there's probably only python in the dependency list. For all we know, > the python maintainer will take care all the optional parts get built > anyway. If this changes in the future, I would like to find out > automatically. > > Well, I think two different cases are getting mixed up here. I'm talking > about a rebuild of the exact same python Nix expression, and you're > talking about the python Nix expression evolving over time. It's certainly > possible to treat the two cases differently, probably even > desirable.
That's not what I was talking about, the python expression didn't change in my example. Just a (made up) underlying graphics library. That triggers a python rebuild, which can't use the new gfx version, so it builds without bindings for it. Then a package on top of that gets triggered to rebuild, and its upstream configure/build script detects the missing bindings and fails. If that package had a "weak" dependency on python, it wouldn't rebuild, so its build scripts would not alarm us something is wrong. > > For instance, python code could have a "recompile only when the python > version is changed" semantics. Or "recompile whenever the python nix > expression changes in any way". Either of these would put a halt to cascading > pointlessness when a third generation dependency of the python > interpreter causes the same interpreter to be rebuilt; both would provide the > automatic checks you desire when something important changes > about the environment. That doesn't help for my example case as the python expression itself isn't upgraded or changed. So then the semantic would need to become "if python, or any of its inputs change", which sounds a lot like current behaviour. > > > > This sounds like a problem to me, one that the current semantics solve > by saying "if any inputs change, we want to make sure everything is > still fine (up to the level that upstream build/test scripts would > accept). > > The new semantics really only apply to these environments intended to serve > as an isolation layer between the host system and code which > executes in the environment. I realize none of them will provide perfect > isolation and you can all find exceptions to the rule. However, many > codes can run unchanged on Windows/Linux/Mac: in java's case, many compiled > binaries can run unchanged on Linux/Windows/Mac and it is actually > pretty rare to rebuild from source. A finished java app is commonly compiled > by a variety of JDK versions on a variety of different platforms. > In any case, the exact version of glibc (and a host of other 2nd, 3rd ... > generation dependencies) on the system is pretty irrelevant--windows > doesn't even have glibc at all! I agree on java. I think java builds a solid abstraction which really shields the host system off from packages using it. But I don't believe the same can be said about any other environment. > > The relevant question is: What is more common--that the isolation is > effective or that it isn't? So, let me quote you: > > "You have to understand that most package expressions are probably created as > a bare minimum, just adding the stuff the builder/installer > complains about." > > I would suggest that the "bare minimum" is to declare that a python package > depends on python with "recompile when the python version number > changes" semantics. If a particular package proves problematic, strengthen > the dependency semantics for that package to depend on a specific > python build. Then it's always rebuilt. Nope, the bare minimum is to just leave all detection to _upstream_ scripts. Choosing a "weak by default" path basically leads to bypassing all work done in those scripts (optimizing, using workarounds for certain versions of underlying deps) and saying "I know better, this package won't run any different on these new inputs". As I argued, weakening should only be applied when you (as a maintainer) know absolutely sure it is safe. You know all details about the upstream build system and the checks it performs, all (implicit) dependencies, and you check the changelog for changes before bumping a version. Weak dependency == manually managing dependencies. It should not be default in any case or template (except for java maybe, or binary proprietary packages) because I'm pretty sure most maintainers don't (and don't want to) investigate their package's build scripts and dependencies to that deep level. > > You still get a full rebuild every time the environment is upgraded, but you > don't rebuild everything whenever the same environment is > rebuilt. The same environment is never rebuilt. When something rebuilds, it means underlying inputs changed. I know that in a lot of cases, the change is small and isn't going to affect anything (like adding an extra man page), but in other cases, when (parent/grantparent) dependencies change, we really want to evaluate the effects these have on everything built on top of it. The problem with weak semantics is that you just cannot tell what underlying change _is_ important and which isn't. Think of when glibc added support for ipv6... With weak semantics, all packages that run _on_ python, or _on_ ruby wouldn't rebuild, because python and ruby themselves didn't upgrade version and their expressions stayed the same. Maybe packages check at runtime for ipv6 support, but it's perfectly reasonable for a package to not build/install ipv6 stuff (like a gui for entering addresses) when it detects it's not available at install time. So, this would mean adding an exception for glibc (force trigger rebuilds for weak deps when glibc is involved), or having maintainers specify that their package might use ipv6, and flag glibc versions with "ipv6 capable" so to trigger just those rebuilds. Sounds like a lot of work to me, and probably stuff will slip through. This too, is just an example, I'm not saying this has been spotted "in the wild", but the problem is that I can come up with many more examples that sound reasonable. This means maintainers would need to carefully watch what might influence their package (like ipv6 support in glibc) and meta-decorate their package with all these cases that validate a rebuild even on weak dependency. And even then, there are always things you don't think of up front. The only way to know is to find out what checks/decisions/macros upstream makes during build/installation. > Also anything with "native code" (linking directly against a system library, > subverting the isolation layer) should have a dependency > on the specific build of the library it links to, causing a rebuild of that > module as well as any other python modules which depend on it. As > you said, the Nix builder will fail if the system library is missing, > so the packager will be forced to add it. As my new example made clear, it's not just libraries at play here. Many decisions can be taken at build-time and it's a hell of a job to find out what/why for every package, just to specify when rebuilding can be skipped. Mathijs > > Bryce
pgpDlIlzUQz6y.pgp
Description: PGP signature
_______________________________________________ nix-dev mailing list [email protected] http://lists.science.uu.nl/mailman/listinfo/nix-dev
