I believe strongly in getting this right as well, and have been dabbling with the gyp_addon system so far. With the removal of node-waf, and the new reliance of the developer to compile their modules, rather than the end-user, I see 3 new problems that open up for native module devs:
1. Devs have to now compile for each supported platform and architecture before publishing 2. Devs now have to figure out where to store these precompiled binaries (in the cloud? in the repo? where?) 3. Devs now have to figure out which binary to load at runtime, given the current arch and platform I've written a small module that attempts to solve #2 and #3, called node-bindings: https://github.com/TooTallNate/node-bindings It's a nice convention in my opinion that doesn't rely on npm's `bin-publish` feature. So that leaves problem #1. I like the "addon-gyp-toolchain" module idea. I hadn't considered that before, and that should help, though I'm not exactly sure how it would work (currently Windows addons require the node source tree, so that would need to be fixed I guess). I'm still envisioning some kind of Travis-CI-like cloud-based build system, that would receive a git-hook and recompile your module on the various platforms/archs and commit them back to your repo (or something), whenever a change happened in the `src` dir. If that was done right, it would make it painless for both the devs and the uses of the module. On Wed, Feb 1, 2012 at 3:14 PM, Isaac Schlueter <[email protected]> wrote: > There is a lot of interest in Node, and a lot of work to do. Several > of you have asked me lately how you can help in IRC and in person. > > In order to try to help match up the interest with the work, I'm going > to start occasionally making requests for help in specific areas where > I think improvements need to be made. In the past, these kinds of > calls have been very productive. > > You don't need permission. Just play around with the problem, and if > you make some headway, share it with the rest of the group. If you > find it's something you really can get excited about, then maybe it > can be your thing :) > > > A very problematic area right now is binary module compilation across > platforms, especially on Windows. > > In the past, we've deployed a node-waf program, which is a > lightly-customized fork of the "waf" build tool. This program has > worked ok, but it is fundamentally non-portable, and my hope is that > it will not have to be in 0.8. > > In the node source tree is a file called tools/gyp_addon. You can use > this to generate either a Makefile, or the appropriate MS Visual C > stuff to build an addon. There are some examples in test/addons/, but > the whole process is not very well documented, and requires a lot of > manual tweaking. > > To make matters even more hairy, there are a bunch of node programs > floating around in the wild that are using node-waf. We need to make > it as easy as possible to port these to a better future. > > > The goal: > > There should be a standalone `npm install addon-gyp-toolchain -g` or > something (probably ought to pick a better name) that sets up some > scripts. If you have this toolchain installed, we'll assume that you > also have "make" on unix, and Visual Studio on Windows. If you then > run the appropriate command in a project directory, it'll build an > addon using whichever method is appropriate for the current system. > > Once such a thing is in place, and it works reliably on Windows and > Unix, and is pretty easy and generalized, start making pull requests > to everybody to replace their wscripts with a gyp file. > > The most important thing is that it makes the process easier than it > is today. There's probably more API design, debugging, and picking > through bits, than actually writing new code. > > And, of course, it could be that my vision of this is way off. Your > task should be to make it easier to compile a bunch of C code into a > .node addon file in a way that works on Windows and Unix, and doesn't > completely suck. Don't be afraid to get creative. > > > > Why not just deploy gyp_addon with node? Because that would be > recreating the node-waf experience, which is terrible. This is a step > towards binary deployments (at least for operating systems and > platforms that can do such things reliably). It would not be terrible > to install it when node is configured with --debug or something, but > by default the msi and pkg installers should never include gyp. Users > who *want* install-time compilation will always be able to do this by > installing the toolchain and then opting out of binary builds. > > > Thanks for your help. >
