The situations you've described are basically the ones I'm in. Luckily the ocaml package manager I am using just does local installs the packages in the home dir. I actually think I have figured out why it wasn't building packages for me before, and think I can get around it so I can just use the ocaml package manager for fast development and once things become static make real Nix expressions from it.
/M Florian Friesdorf <f...@chaoflow.net> writes: > Hi Malcolm, > > Malcolm Matalka <mmata...@gmail.com> writes: >> Thanks for the complete response Marc. >> >> For the mean time, the current solution I'm working on is manually >> converting packages to Nix. The translation is pretty straight forward >> and I should be able to write a tool to automatically do it. It's not >> optimal but I have been able to get all of the packages I care about >> into my local nixpkgs repo and will submit them to the master repo once >> I'm satisfied with them. > > I like the approach of (semi-)automatically creating nix expressions. > > In my understanding (and I am pretty determined but happy to change it > based on good arguments): > > A set of nix expressions always produces the very same output. > > For me this means (so far) that ad-hoc resolving dependencies within nix > is not needed. I want to resolve dependencies ad-hoc when creating a new > set of nix expressions, but not when I install a package via nix. > > Therefore the resolver belongs outside of nix. > > If I want to install things ad-hoc in a nix system, I need to create a > bridge between a nix world where everything is determined and a fluent > world where things are installed as they are available. These fluent > things cannot reside in the nix store. Whether such dynamically managed > things belong into /var or a user's home is another discussion. > > For nodejs I was quit happy that I got such a dynamic profile via: > % npm config set prefix ~/.nodejs > > Now installing things "globally" end up in ~/.nodejs and the bins in > ~/.nodejs/.bin, which I included in my path. Perfect quick and dirty > installation of the view nodejs packages I need. > > For python we are aiming at something similar: generate nix expression > and check them into git for commonly needed packages, use the rest via > easy_install, but give the people the tool to create nix expressions for > whatever they need. > > Looking forward to your ocaml work! > > regards > forian > >> Marc Weber <marco-owe...@gmx.de> writes: >> >>> I created >>> hack-nix for Haskell, which dumps hackage. contains a brute force >>> dependency solver >>> >>> nixpkgs-ruby-overlay [1] which dumps rubyforge (which is quite usable, but >>> not perfect yet) >>> nixpkgs-python-overlay [2] which dumps PyPi (experimental, dependency >>> inforamtion is not complete enough) >>> does not do backtracking, if dependencies fails its you having to to >>> tell it "try lib-A version 2.0.0". >>> >>> They are special because they all work on a "dump of packages" creating >>> .nix derivations on the fly whereas cabal2nix creates .nix files very >>> close to what you find in nixpkgs. >>> >>> They all create kind of shell script you can source to augment the >>> environment variables, so that dependencies are found. >>> Thus you can have multiple "sets of packages" for different targen >>> porjects within the same user account without conflicts - however you >>> always have to load such an environment, eg by >>> >>> # run bash with ruby packgaes found in environment "name" >>> ruby-env-name bash >>> >>> Known additional universes: >>> - perl >>> - java, scala (ivy, maven, sbt) >>> - .... >>> >>> There are many ideas and ways to implement such. >>> Eg for scala/maven/ivy/... one way to think about it would be using the >>> store as "installation place", but not using much about the nix* tools >>> otherwise. >>> >>> then sbt build would just store everything in store. >>> Other ways are creating .nix files for a target on the fly - such as >>> sbt/ivy/mvn create-nix-derivations (which is close to what cabal2nix >>> does). >>> >>> The downside is that you may have to run a tool before you can succeed >>> with nixos-rebuild-switch, because not everything may be packaged. >>> >>> The perfect way would be including a SOT solver in nix, which hasn't >>> beeen done yet - and which was not favored by Eelco in the past (maybe >>> for good reason). Eg Eclipse plugin system works this way: the SAT >>> solver tries to find one working set of dependencies to satisfy the >>> setup you want - however the search space may be very big - which is why >>> I limit the simple brute force solver used by hack-nix by passing only a >>> subset of all packages found on hackage (latest versions & same manually >>> selected ones). >>> >>> While such a generic approach may seem perfect, there are these >>> downsides: >>> >>> - its harder to controls when rebuilds will take place, because small >>> changes in the pool may cause the solver arbitrarely choose a >>> different solution, otherwise its you having to force eg library-A >>> version should be 1.0 like thing. >>> >>> and such rebuilds are bad, because its easy to loos track about which >>> combinations actually work, because while constrtaints are fine, they >>> are never complete. >>> >>> Thus in any case there will be lots of maintainance effort. >>> >>> - its also hard and time consuming (for humans and the cpu) to evaluate >>> all solutions over and over again - which may not be the perfect end >>> user experience. Eg you do'nt want wont to wait 30secs for the >>> evaluation to finish just to install "gnu sed" >>> >>>> translation apps for all language package manager types? Specifically I >>>> am looking at opam, the new ocaml package manager. >>> I never used opam. >>> >>> Can you copy paste a package description with dependency information so >>> that we can get an idea about how it actually looks like? >>> >>> Cabal is kind of "static", but very complex. >>> Example: >>> http://hackage.haskell.org/packages/archive/darcs/2.8.3/darcs.cabal >>> scroll down to "build-depends" which depend on "flags" which are >>> automatiacally chosen depending on the ghc version available - but >>> flags are also used to enable/disable features or test cases >>> >>> for python and ruby there are .py or gemspec files. >>> The problem is that they may even run python or ruby code - thus >>> there may be packages whos dependencies may depend on configuration >>> options and or the system which makes it harder to to dump such info >>> into something which can be read and used by nix. >>> >>> [1] ruby: http://gitorious.org/nixpkgs-ruby-overlay >>> [2] python: http://gitorious.org/nixpkgs-python-overlay >>> >>> For ruby and Haskell I also have some code which can create package >>> descriptions for dev versions of packages which then can be read by the >>> code creating the derivations on the fly. >>> >>> And then there is stills the question: >>> Is it efficient to download 40.000 package descriptions if you need only >>> 10? The lazy behaviour of the native package managers for ruby (gem), >>> python (eg pip) etc somehow make this question obsolete. >>> >>> Just think about how many perl packages there are available. >>> >>> Well - you don't have to download 40.000 packages, cabal2nix only dumps >>> the the maintained packages which are selected manually. >>> Another solution would be making nix connect to a server to access a >>> versioned well known "dump state" about known packages. Then only those >>> package infos could be fetched which are required to fullfill would have >>> to be fetched. >>> >>> The ivy case may be not trivial cause you can configure dependencies in >>> a transient way which means if you have A < B < C you can configure C to >>> modify dependency A or such (I never fully had the time to learn about >>> all features). >>> >>> So the topic is hot - and there is still quite some work left to be >>> done. >>> >>> The cabal2nix way may work well if you need some packages only and is >>> easy to understand and to debug. >>> >>> I hope I was able to shed some light into current state I know about - >>> others may know more. >>> >>> Let's not forgett that eg most xorg packages are generated >>> automatically, too. The some additional manual work is required to make >>> everything build. >>> >>> There are more sub universes, such as gnome which is provides a central >>> download folder hirarchy you could use to dump package information from. >>> Nobody did it yet. >>> >>> Marc Weber >>> _______________________________________________ >>> nix-dev mailing list >>> nix-dev@lists.science.uu.nl >>> http://lists.science.uu.nl/mailman/listinfo/nix-dev >> _______________________________________________ >> nix-dev mailing list >> nix-dev@lists.science.uu.nl >> http://lists.science.uu.nl/mailman/listinfo/nix-dev _______________________________________________ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
