2015-06-09 2:20 GMT+02:00 Jan Malakhovski <[email protected]>: > Hi. > > Was it by design that you didn't mention nix-dev in Cc? If not, then I > suggest forwarding this conversation there too. Feel free to quote my > comments when and if necessary. >
No, wasn't intentional. I'm used to mailing lists, that set the reply-to properly. Forwarded the previous messages. > > Herwig Hochleitner <[email protected]> writes: > > > thanks for your well thought-out response. You obviously have some > > experience from attempts to reform that area of nixpkgs. > > > > The problem is that people will say than "config is deprecated, use > >> overrides". > > > > How and why was config deprecated? Was there an ML thread? > > No idea. I have not seen an official statement about this, yet there was > this: https://github.com/NixOS/nixpkgs/pull/6968 > > > I kind of get what it is about. I think, the best way to use config is to > > have it determine the specific override of a package. See what I did with > > the wine package, recently. > > > > I don't think overrides are a proper replacement for config, because they > > are clunkyer and a part of config anyway. Plus, they suffer from the same > > shortcoming: Missing documentation, which can't easily be auto-generated. > > > > it was cut down with "do not add new config options, they are deprecated, > >> we have > >> overrides already". > > > > Interesting, do you have the ticket number handy? > > https://github.com/NixOS/nixpkgs/pull/6968 > > > Personally I have some problems with current practices: > >> > >> * I don't like the idea of using `null`s in package arguments of a > >> package expression as a way to specify features. I think this is ugly > >> and error prone. > >> > > > > Sure, to me, that is part of the problem with discoverability. > > > >> * Currently obsessive use of overrides generates more versions of > >> packages than needed. Do > >> > >> egrep -A 3 -B 6 ' .*override' all-packages.nix > >> > >> every override of a package in an argument set for another package > >> will likely to ask nix to compile another version. This is crazy. And > >> I see that craziness in action every time I rebuild my configuration, > >> 'cause I use a shitton of packages in `environment.systemPackages` on > >> my desktop machines. I haven't measured the CPU time and disk space > >> losses to this craziness, but I don't like the fact that these losses > >> exist at all. > > > > I'm not sure I follow. In my understanding, only needed overrides will be > > built. Also, changed config options will hopefully result in different > > compilations, same as overrides. What effect should they otherwise have? > > > > In fact, a better model of config, should also act as a guideline as to > > which package-trees should be precompiled. Right now, an entry in > > all-packages.nix needs to be created for every config (or override) that > > should be built by hydra. > > Suppose you have a package `foo` with option `barSupport` by default set > to `false` and a package `baz` depending on that `foo` with `barSupport` > set to `true`. If you do `baz = callPackages { foo = foo.override { > barSupport = true; }; }` then you'll produce two packages when you set > `environment.systemPackages = [ foo baz ]` in your `configuration.nix`, > but all you actually need to do is to set `barSupport` to `true` by > default on your system. > > > I think packages should assert that some options are set in their > >> dependencies > >> > > > > agreed > > This sentence was about that stuff right above. > > > that generates another version of dependency one would write > >> > >> somepackage_forced = somepackage.override { > >> dependency = if config.somepackage.thisWhateverSupport > >> then dependency.override { whateverSupport = true; } > >> else dependency; > >> }; > >> > >> which doesn't. > > > > Are you talking about if a package `bar` uses a dependency `libfoo`, > > configured differently from the rest of the system, while the rest of the > > system could just aswell be run with with `bar`'s libfoo? > > Yes. > > > Such information could be generated by bidirectional inferrence, but I > > believe that this shouldn't be done automatically, because it drives us > > towards the non-determinism of other OSes. Instead the user should be > able > > to inform themselves of packages duplicated because of different > > configurations, then they could consider using the most general > > configuration as the system default. > > Yep, that is a valid point. > > > In that light, I think `common.*` options are not sufficient, but if we > had > > a tool to find those overrides, maintainers would start to look for > common > > ground by themselves. > > Of course for that, a package needs to know, which options it requires in > > its dependecies. I believe gentoo use flags are a great source for that > > information and a better model than asserts. > > Yep, but having at least the `common` thingy would be awesome, wouldn't > it? :) > > > When one simply does > >> > >> nixpkgs.config.dependency.whateverSupport = true; > >> > >> or maybe even > >> > >> nixpkgs.config.common.whateverSupport = true; > >> > >> * Overrides are not fixed points and this is a serious problem for some. > >> See https://github.com/NixOS/nixpkgs/pull/7102. > > > > So if I understand correctly, this is about the `pkgs` parameter in > > config.packageOverrides (pkgs: ...) being the version before it being > > updated, while some users would expect it to iterate to a fixed point? > > If so, I'm not sure how this could even theoretically be solved without > > running the immediate risk of infinite override loops. > > For what use case is this a serious problem? > > I suppose one could just make `packageOverrides` to receive two parameters: > `pkgs` before (aka `super`) and `pkgs` after (aka `self`) overrides. > > > * For some strange reason it seems that overrides sometimes evaluate > >> stuff too early. I haven't been able debug this, but overriding all > >> the X11, GTK and QT packages (exactly those packages, not the packages > >> that use those packages) to `null`s in nixpkgs and then specifically > >> overriding some of the packages that use them shows that some of the > >> latter packages will try to evaluate X11, GTK or QT even when > >> explicitly overridden not to so. I guess that's because something in > >> the derivation get's stuck on the previous version of the argument, > >> but, as I said, I haven't been able to debug this yet. > >> > > > > Can you post the overrides, that you used when discovering this? > > Not now, but I plan to catch and document that the next time I play with > this. > > > And so now I maintain my own set of patches I rebase on top of master > >> once and then, that goes against the grain and simplifies all that > >> `nixpkgs.config`-style configuration stuff fairly significantly by > >> adding another version of callConfPackages (also removing a ton of > >> copy-paste from `all-packages.nix` as a side effect) > >> > >> > https://github.com/NixOS/nixpkgs/compare/master...oxij:call-conf-package > >> > >> and weep every time someone rewrites and expression for a package > >> dropping the support for `<feature>Support` options (e.g. most recently > >> `qemu` =/). > >> > > > > I like your approach there. Maybe, defaulted parameters could even be > used > > as a fallback to discover package config options. > > > > Said that, I agree that there are problems with this: > >> > >> * All that `somepackage_forced` is somewhat ugly. I would prefer to have > >> a way to describe and introspect assertions in the packages and > >> generate those *_forced thingies automatically. Or maybe even write > >> something like > >> > >> environment.systemPackages = [ somepackage.force_dependencies ]; > >> > > > > I created a couple of packages, that just generate their overrides > > privately. > > Do you use those _forced versions to get prebuilding by hydra? > > _forced and the related stuff from that section in my post is just a > concept. I have not implemented it yet. > > > * Package options need documentation (which the original message by > >> Herwig Hochleitner is about). This is totally doable even now with > >> reusing NixOS modules in nixpkgs: > >> > >> { dependency1, dependency2, ... }: > >> { config, ...} : > >> { options = { supportWhatever = mkOption ...; }; > >> assertions = [ ... ]; > >> config = mkDerivation { ... }; > >> }; > >> > >> and then fixing `callPackage` to grab only the `config` and move the > >> `options` to `meta`. And then collecting the latter by grabbing all > >> the stuff from the packages `nix-env -q` uses (one could even show the > >> options in nix-env this way). > >> > > > > That's what I meant by "use a schema similar to nixos modules". This > could > > be used to replace the fallback, if you want to document allowed values > or > > a description. > > > > But I agree that this is somewhat ugly. Especially because usually > >> all `supportWhatever` does is adds a flag to `configureFlags` and > >> another dependency to `buildInputs`. > >> > > > > Well, at the bottom needs to be something general, that can do it all. > > Sugar for this default case can be added afterwards. > > > > The proper solution is to implement a dual of `bulitins.functionArgs` > >> that _generates_ a function with arguments and their defaults set from > >> an attrset. > > > > > > +1, but that's not enough when you want docstrings and enum values. > > It kinda is. If I could generate a function from an attrset description > then, in principle, I could add any meta info to the `meta` of the > package it produces using the same attrset description. > > > So, I'm totally in favor of doing something to better the situation. And > > I'd ask anybody, so inclined, to explain to me how overrides are enough, > > with respect to the mentioned issues. > > > > I think further work on this should be framed as "enumerating and > > documenting pre-visioned overrides" > > > 2015-05-28 14:30 GMT+02:00 Jan Malakhovski <[email protected]>: > >> > >> And so now I maintain my own set of patches I rebase on top of master > >> once and then, that goes against the grain and simplifies all that > >> `nixpkgs.config`-style configuration stuff fairly significantly by > >> adding another version of callConfPackages (also removing a ton of > >> copy-paste from `all-packages.nix` as a side effect) > >> > >> > https://github.com/NixOS/nixpkgs/compare/master...oxij:call-conf-package > >> > >> and weep every time someone rewrites and expression for a package > >> dropping the support for `<feature>Support` options (e.g. most recently > >> `qemu` =/). > >> > > > > Are you talking about > > > https://github.com/NixOS/nixpkgs/commit/19c259161b987fcb5e1c3b7a87c746372ce017ee#commitcomment-11453831 > > per chance? > > Yep, that's the thing that made me weep that time. > > Cheers, > Jan >
_______________________________________________ nix-dev mailing list [email protected] http://lists.science.uu.nl/mailman/listinfo/nix-dev
