Bryce L Nordgren <[email protected]> writes: > Hi list, > > I want to suggest something to see what you think. > > Nix currently supports a single meaning for its dependency relation, which > essentially can be summed up as "Depends on > exact build of". As in, GDAL <depends on exact build of> GEOS(hash here). > This is an enormously successful model, but it > does force you to completely omit real dependencies (and to trick Nix into > not detecting them). For instance, to avoid > recompiling all Java source to bytecode simply because some change triggers > the rebuilding of the java compiler (Note > we're not even talking about upgrading Java here: we're saying, i.e., a sound > library was upgraded so the same JDK was > rebuilt.), it has been suggested to NOT depend on the jdk and to refer to the > java interpreter only by an environment > variable so that Nix won't pick up the dependency (e.g., > $JAVA_HOME/bin/java). Likewise, "hash rewriting" has also been > suggested as a manual workaround for those situations where the "Depends on > exact build of" semantics are inappropriate. > Certainly "Depends on exact build of" will produce a workable result given > enough time and CPU cycles, but there are > situations where it is extremely wasteful. Consider the following additional > semantics: > > * "requires environment" : Requires the presense of the interpreter > software/VM (e.g. bash/python/perl/java/ruby) but > does not force the dependent package to be rebuilt (and stored again) if the > interpreter/VM is replaced. The target of > the dependency does not represent a specific build, but an environment > description. > * "implements environment" : Declares that the package is designed to provide > a specific environment. Two > implementations of Ruby 1.8.7 could be the reference implementation of Ruby > or Ruby-Enterprise. The target of the > dependency represents an environment description. > * "depends on executables from" : Refers to a Nix source expression instead > of a specific build. The dependency is on a > command line utility which has already been linked. The dependant package is > not rebuilt when the target is. Also: any > build instantiated from the target expression should be considered > interchangable. For example: a dependency on the > "dot" executable from GraphVis should not cause Doxygen to be rebuilt if a > rebuild of GraphVis is triggered; conversely, > any build of GraphVis (version X) present on the system should satisfy the > requirement. > > I understand that there is some resistance to these suggestions because they > appear too similar to the semantics present > in other package management systems (which completely lack "depends on exact > build of"). This is not an effort to make > Nix more "familiar" or to remove "depends on exact build of". It's intended > to eliminate wasteful and needless > rebuilding (while reducing the number of missed/masked dependencies) by > allowing the packager to correctly express > relationships between packages which are not "depends on exact build of". > Additionally, this would allow "nix-env" > "nix-build" and Hydra to correctly process these directives. > > Thoughts?
I think it's hard to find a good definition for a "weak" dependency. The thing I like about nix* is that it guarantees me I can get the exact same setup/environment/dependencies on my production env as I have on a dev box. Furthermore, it guarantees me that a certain configuration builds successfully, which (in the case where you run tests during a build) is also a very important thing. I'm planning to get rid of a dedicated CI server, which is basically just doing these rebuild-checks every time some source code changes. The current CI-server setup doesn't know anything about the runtime environment, so if I don't change any sources, but upgrade some library, I need to force the CI to rebuild the current sources. I agree a lot of these rebuilds are needless/wasteful. In the case of java for example, I'm pretty sure the VM provides a very good abstraction layer, so any changes "below" it, won't change anything for stuff that just needs a JVM. But the same thing cannot be said about Ruby (in all fairness: Ruby's main goal never was to be a VM) and most other environments. Also, some of these "language/platform environments" have optional components. Let's say some imaging library gets updated and (for example) python is not able to use it anymore (upstream decided to move the python-special-imaging-thingy to a separate package/egg). Currently, when python gets rebuilt, it won't be able to detect image-thingy headers for a supported version, so it silently decides to skip building just that small part. As a result, packages depending on python get rebuilt, and (if the upstream configure/build script does at least some small checks) fail, because there is something wrong. This is exactly what I want, because something changed and broke that package. Now, if "weak" dependencies get introduced, packages might just say "I require some python env". And as special-image-thingy was included (batteries included:) since day 1, no package ever bothered to specify "I require python, and it has to be built with the image thingy". So, as a result, all these packages will decide not to rebuild after this python change (not upgrade) and I would think they still work fine. The way other distributions try to fix this is by introducing special extra-version-number components which they can bump, to force signal all depending packages to rebuild. And advanced flags like "built with feature X", which depending packages then need to check/mandate. In other words, I'm afraid that your suggestion would quickly lead to hidden problems where stuff breaks without you knowing it. Also, it will lead to complexities to work around breakage and to manually override does-this-depend-and-need-a-rebuild decisions. Maintainers would need to be far more familiar with the packages they maintain, so they can intelligently apply "requires python env - no need to rebuild when lower deps change", while currently, they can just count on the upstream configure/build stuff to report problems. So while I agree with what you are trying to achieve, I'm almost certain it won't be possible to solve these issues elegantly without giving us hidden breakage, or needing maintainers to really spec out a lot of details about dependencies about their packages. The auto-detection "magic" that really impressed me when I came to nix, will be a lot less. I understand all this could be optional (--strict-rebuild-detection or something), but having different build strategies probably leads to a whole new other bunch of issues :) But in all, if there is a way, I'm not against it. Mathijs > > Bryce > > > > _______________________________________________ > nix-dev mailing list > [email protected] > http://lists.science.uu.nl/mailman/listinfo/nix-dev
pgpp1aSPnRPDT.pgp
Description: PGP signature
_______________________________________________ nix-dev mailing list [email protected] http://lists.science.uu.nl/mailman/listinfo/nix-dev
