I think most of you probably know it already, but I've written a blog post about the reengineered implementation, that can be found here: http://sandervanderburg.blogspot.com/2014/10/deploying-npm-packages-with-nix-package.html
Second, I have some good and some bad news. Good news is that there were a few people lately, having problems with deploying certain packages with the old npm2nix (one time there was another cyclic dependency). I tested the same packages with the new npm2nix and they seem to deploy just fine without problems. So I think it that the new approach really solves some very nasty problems and accurately simulates most of NPM's way of managing dependencies. The only bad thing so far is that the new approach is a lot slower. I'm specifically talking about the time it takes to deploy something with nix-build or nix-env. The slowness is basically caused by the fact that for each dependency I want to include, I have to tell which shared dependencies have been provided already to correctly simulate the way version ranges are resolved. I need to run semver for this. Moreover, to also prevent infinite recursion because of cyclic dependencies, I use a expression generation trick. I think I already have a few optimizations in mind, but they are difficult to implement and I might break stuff again. For example, if I'm absolutely sure there are no cycles in a deployment, I can skip the dependency generation trick, for example. Question is: how to detect this? I don't think the slowness is problematic for private projects, but it might be too expensive to use that for the set of NPM packages part of Nixpkgs. On the other hand, I also think that in the future, most packages should be deployed privately in NPM projects. The main reason why I recommend this is that version range specifiers do not always resolve to their latest compatible versions. The actual version used is context specific -- if some project requires async 0.2.x, the latest conforming version is 0.2.9, but a shared dependency is 0.2.5, then 0.2.5 is used. That's why you have to reconsider the complete tree of dependencies for every project. The only packages that still need to be deployed from Nixpkgs IMHO are end-user utilities (command-line utilities) and libraries used by non-NPM projects. So I'm still thinking a bit about it and what I can do to make the performance better. In the worst case, I have to throw away the entire approach and invent something new, if it turns out that the performance is too problematic. What do you think? -- Sander On Tue, Sep 30, 2014 at 12:13 PM, Sander van der Burg <[email protected] > wrote: > Hi everybody, > > Since the sprint in Ljubljana until now, I've been working on npm2nix's > issues. I basically reengineered most of it (e.g. rewritten it to > JavaScript and modularized it a bit further) and I think the implementation > is fairly complete now. I've incorporated as much of Shea's functionaility > into the reengineered version as I could. > > The new implementation handles dependencies in such a way that its > behavior is closer to what NPM does. Moreover, it should also properly cope > with cyclic dependencies. The way dependencies are resolved is actually > much more advanced than the original npm2nix implementation. For example, > it uses semver to make matches between version ranges. > > In theory, also cyclic dependencies that match on version ranges should be > handled properly, although I haven't encountered them yet. > > Using the reengineered version is straight forward. You can obtain it from > my private Git fork: > > https://github.com/svanderburg/npm2nix/tree/reengineering > > I have updated the README.md file of npm2nix to elaborate a bit more about > the possible use cases of npm2nix. So I think it wouldn't be a bad idea to > read it first. > > Installation can be done by opening the git checkout, and running: > > $ nix-env -f default.nix -iA build > > Moreover, you don't need to update Nixpkgs. The new build function is > actually part of npm2nix. > > I've used the reengineered npm2nix on a private project for the company I > work for, as well as some other utilities and it seems to work fine for me. > > Hopefully, you can also try npm2nix on your projects to find out whether > there any additional issues. :) > > Furthermore, I'm still working a blog post to rationalize all the stuff, > which should give you better insights in the problem, details and the > choices I made. > > Let me know what you think! > > Regards, > > Sander > > > > >
_______________________________________________ nix-dev mailing list [email protected] http://lists.science.uu.nl/mailman/listinfo/nix-dev
