On Wednesday, January 23, 2013 11:34:17 AM UTC-7, Isaac Schlueter wrote: > > On Wed, Jan 23, 2013 at 9:13 AM, Jacob Groundwater > <[email protected]<javascript:>> > wrote: > > Is `index.js` considered a non-preferred method of doing package > imports? I > > really like it, but arguing a stable feature is pointless. I'm just > curious > > what the reasons were for including it, and what has changed since? > > > > FOR SCIENCE! > > index.js predates packages by quite a long time. It was the first > "load a packagey folder thing as if it was a single module" feature, > dating back to the 0.1.x days. It's not terrible. But personally I > prefer a main script in package.json, since you can name it something > more appropriate than "index.js", and somewhat like named functions, > it makes the debugging experience a bit nicer. >
package.json is, frankly, a terrible idea. Node.js is not and should not be in the business of knowing what a "package" is. That's something that exists several levels up in the abstraction hierarchy. Nearly the entire changelog of require() behavior from v0.4 to now is a history of how *bad* npm's design was initially. It also happens to be a story of how influential npm was, for better or worse, because nearly all of those patches was a result of npm's abuse of some feature or another. And the *apparent* attitude while such features as require.paths was being removed, and package.json being added, was "well if npm can't do it right then we can't trust anyone to do it right". Now why bring this up? I'll argue Node.js it right in the beginning, and that Node.js was at a * better* position to handle packages in v0.2 than it is now, so long as one had NODE_PATH properly exported, the same way one would export PATH. And it was always possible to use local modules, with some hacks. Exempting my own code, not once have I installed any module globally. I'll describe what can be done right further down. The only reason we're in such a good position now is because of node_modules - a replacement of commonly used behavior with NODE_PATH and require.paths. > On Wed, Jan 23, 2013 at 6:18 AM, Austin William Wright > <[email protected] <javascript:>> wrote: > >> "Objectively correct"? Wow. > > If you have a problem with my logic, the correct thing to do would be > > describe what's actually wrong. > > Fair enough. There are two major problems with your argument. > > The first is a gross category error. You are asserting that the > semantics of the require() function can be in any sense "objectively > correct", when in fact we are discussing complicated trade-offs > involving subjective values. > > Second, you are asserting that "correctness" is defined by adherence > to a spec that has nothing to do with Node, and never did. > I'm not trying to say that any specification canonically defines correct behavior. I'm instead saying two things: (1) There is an objectively correct way to design things, based on their meaning and semantics. This is chosen in part by paying due respect to subjective preference, i.e. just because one can't think of why you would ever want to use an "index.js" file, this does not mean that it is a bad idea. (2) Where a decision would otherwise be arbitrary, or of acceptable cost to the project, standardization is beneficial. You need not utilize existing standards, especially if the standard are poor or incorrect with respect to the application semantics. (If the specification were incorrect with respect to application semantics, then it follows that implementing the standard would not be an acceptable cost.) Note that the RequireJS functionality doesn't (or has no authority to) define which files get looked up when a module name is dereferenced. It's just a (rather ugly) API in two parts, defining (1) how to load a module, and (2) how to export a module body. Also, if some feature were indeed a bad idea, then *take it out* and call the next version 1.0. This is exactly what we have semver for. > > If you ignore the semantics of what > > operations like require() actually *mean* then you're going to run into > > problems, or stagnate growth. > > Where have you been the last 3 years? "Stagnating growth" isn't a > problem we have. We have the problems that come from *not* stagnating > growth. > Just because you can point to growth doesn't mean you can demonstrate a certain feature was responsible for it, or that you can excuse any decision that would negatively impact growth more than the alternative, i.e., a *post hoc ergo propter hoc*. I'm making any assertion that there's no growth, I am saying continued lack of deference for proper design for will have a negative impact all other things held equal. > > So yes, there's an "objectively correct" way to do things. And you MUST > NOT > > assume that the filesystem is the only way someone would want to > dereference > > a module. > > I don't assume that. But it IS the only way that someone is capable > of dereferencing a Node module. Look at the code. That's how it > works. If you want your modules to live somewhere else, mount it with > fuse. Apart from the buildin modules bundled in the binary, Node only > loads modules using the fs. > The point of is to make lives easier. Yes, you could do really costly things like custom filesystems or obscure linking. But those are hacks, and you shouldn't have to. > > But this response isn't unusual to see when people can't actually attack > > logic. > > Ad hominem. > On the contrary, I was saying "I don't blame you", albeit being needlessly snarky in doing so. > > The cost is the value of the next best alternative that must be given > up. > > What's being given up? A few extra CPU cycles, worst case? That can't be > it, > > The cost is time spent debugging programs you didn't write. That is > the cost I was referring to, not CPU cycles. (Unnecessary stat() > calls at startup are not ideal, but they're also not particularly > costly.) > I'm not seeing the cost here, to the point there's no way I could have known you were talking about debugging, or anything else you're referring to. I've never had a problem with locating the source files that a module is referring to, nor do I anticipate one. People aren't going to extend require() in some bizarre fashion unless it's a net benefit somehow. Especially if you're using a debugger, no matter how convoluted module dereferencing becomes, you still are handed off the correct source file. (Even if it is crazily uglified, as with compiled Coffeescript.) But for the sake of argument, maybe there is some case where this really will add a significant (measurable), marginal (on the margin) cost to development. Then, likely, the development process for your use-case was cheap and will become expensive, and as such the development process was likely chosen not because it was correct, but precisely because it was cheap, and wrongly chosen, though through no fault of your own, and as such we can fix whatever is causing the excess burden. > In the normal case, you have three types of things: > > - Builtin module: require('fs') > - Package main: require('request') > - Local module: require('./mine.js') > > Package mains and builtins are similar in their appearance, and when > you're working with your program, they're actually somewhat similar in > how you tend to think of them, as well. They're a thing "outside" the > current program that it depends on. This similarity in appearance is > still a bit annoying, but changing it would be overly destabilizing. > The fourth case, is depending on a local module within a file, and > that's what we're talking about here. > > So, say that the file is at node_modules/foo/lib/utils/bar.js. Right > now, you can do require('foo/lib/utils/bar.js'). It's pretty clear > what file you're pulling out: it's lib/util/bar.js from the foo > package. > > If we add directories.lib then foo/lib/utils/bar.js could become > require('foo/bar.js'). What is the advantage of this? What is the > cost? > > The advantage is that you can export more things from your package > slightly more easily. There's a feature designed explicitly for that, > so the API communicates that such a thing is a good idea. (Most in > the node community are of the opinion that it's *not* a good idea!) > > Another advantage is that you can have shorter strings in your > require() function. But you can also have that by just putting files > in the root of the foo package. That is a much simpler solution, and > simpler solutions are to be preferred unless they come along with a > cost that outweighs the value of their simplicity. What is the cost > of putting files in the root of your package? Clutter in your project > folder. But is that clutter a bad thing, necessarily, and exactly how > bad is it? The clutter is a motivation to build a more elegant > module. The API is communicating that such an approach is > ill-advised. (And this reflects the feelings of the node community.) > > The cost of directories.lib is that it's an extra step in figuring out > how node programs work. Consider a module baz that depends on foo. > In baz, there is the line `var Bar = require('foo/bar.js')`. I am > using baz, and debugging an issue that seems to stem from this Bar > thing. So, I want to look up its implementation. I type `npm explore > foo` to go into that folder, or `npm edit foo` to open it up in my > editor. Now what? Which file should I edit? There's a package.json > here, and a lib folder. No bar.js. In lib, there's a folder called > "exports" and a folder called "utils" and a folder called "browser" > and a folder called "ringo". Each have a file called "bar.js" in > them. Now I have to open up package.json, and understand it. > > (God help me if you're using "overlays", another feature that we have > flat out refused.) > Node.js by its nature defines what a module resolves to out-of-bounds from the file naming what is to be loaded. In Node.js, it's up to the external environment of the application to define which function "http" or "mysql" is, exactly. You seem to be of the opinion that this is partly a bad thing. Contrast this to how most frameworks and programming languages work: The module to be loaded is globally named somehow in a central namespace, defined in a system directory (maybe with an option to install things in a user directory), or both. The former functionality, defining modules out of bounds or otherwise without a central namespace, is far more powerful. This means that two applications can both refer to a "mysql" module, and it each resolves to a different module, no matter if it's byte-for-byte the same (actually the same thing as being the same module, since modules exist per-process), or different versions of each other, or completely different, competing projects. This exact behavior, though, is what enables us to do things like depend on certain versions of modules, a certain Git tree/commit, or a certain semver, or even more esoteric patterns. It allows each module to use a different name. It makes sure that your application always runs the code you endorse. (The exception being Node.js which is often installed system-wide. I don't see a particular problem with this, so long as Node.js properly marks breakage according to the semver specification, and especially considering usually people running Node.js tend to have root access anyways, though this is not an assumption we should make.) Now I may have been a bit confusing because I haven't actually thoroughly explained what I'm actually favoring. I'm not saying Node.js should add "lib/" to its search path (not necessarily, at least). I'm advocating that Node.js, wherever possible, should have less say in what a module name resolves to. You're arguing that this leads to unpredictable behavior. But what you're thereby arguing is that Node.js package management schemes should not be allowed to evolve towards better setups. That is, the cost of letting people define their own behavior is negative. Or another way, diversity and evolution is good in the long run. Nor do I believe your assertion that such freedom presents a significant development cost. If I want to find out which executable is going to be run when I type `git` or `ffmpeg`, I type `which ffmpeg`. That sounds like a much better solution than not letting people specify their own $PATH. (Obviously Node.js still has NODE_PATH, I refer to the more complex functionality I will describe shortly.) This probably means: Remove package.json and similar lookups from consideration, and replace index.js behavior. If, upon stat, the target is a directory, then recursively look for name+'/index' or maybe name+'/lib'. This would continue to permit symlinks e.g. index -> lib/ or index -> lib/module.js. This behavior is more flexible and just as readable as current solutions. I'd like to see people be able to replace the dereferencing functionality entirely. However I don't have any particular proposal to do so or how to do so. But the dereferencing process is nonetheless done within application space, and not globally (unlike, say, a URI), and while I don't anticipate anyone needing to extend the functionality beyond what I laid out, that possibility must not be eliminated. It gets much much worse if you use the "modules" feature of > package.json. Now it's not just a folder, but an arbitrary mapping of > module IDs to filenames. The "mapping" feature makes it so that I > can't even reliably read your code, since require("request") might > actually be something entirely different from the "request" module. > I'm not even sure why that exists. > > It may seem like I'm painting a bleak picture. I'm not. I'm actually > describing a real-world situation that I personally encountered trying > to build a real product for real customers at Joyent, back when npm > supported this feature you guys want to revive. > Supporting the "main" field is as far as we go. Why "main", but not > "directories.lib" or "modules" or "mapping", especially when we > already have "index.js" support? Well, having 25 files named > "index.js" is not very helpful in the debugger, and most packages were > already using the "main" field when we started putting > node_modules/package awareness into the node module loader. Only a > few were using modules or directories.lib, and they didn't have much > traction already. > > The API that provides a specific interface for "export one thing" > communicates that the correct way to build programs is with modules > that do one thing. This is widely acknowledged in the node community > as a best practice. > > Because of this, the experience of looking up a single "main" module > is much less crazy-making in practice, especially since it's often the > only .js file in the package. > > the current Node.js implementation currently has to synchronously stat() > for > > package.json, which should have been done away with long ago. > > Why? Because sync IO is "icky"? It's only icky if it prevents you > from servicing requests. At start-up time, it's actually much faster, > and gets you to a running program sooner. > I didn't mean synchronous operations, I meant all reliance on package.json, as described above. > > The solution is quite easy in practice: don't do require() after start > time, or suffer the consequences. > > > > As mentioned, require() may be extended to load things other than files. > > Take json, or coffeescript, for example. > > Those aren't examples of "things other than files". JSON and > coffeescript are files. > > > I think you suggested at one point > > that it'd be cool for require() to *load files off the Internet*. That's > not > > a good idea, anymore? > > The node team has rejected several proposals to do exactly that, and > I've been convinced that it's a bad idea, or at least, would require a > massive change to how Node works. > > Just because I think something would be neat doesn't mean it's > actually a good idea. I AM a hypocrite, after all. > > > >> > Cross-platform compatibility is a highly desirable feature. > >> > >> Cool story. > > > > Then help do something about it. > > "Cool story" is not a way of communicating agreement. It is a way of > dismissing a non sequitur. > I too am communicating my displeasure that you don't see the gravity of the argument. We made a case that having a certain style of require() arguments is a desirable feature because it would often allow programs to work cross-platform natively, without modification. This is a very desirable function for Node.js, being an ECMAScript based platform. The argument being made is precisely for cross-platform so I don't see how you can call it a non sequitur. If you don't see the use of cross-platform ES scripting then I'm not sure what to say. > "Desirability" is not an innate property of features. "Desire" is a > feeling a person has. Something is "desirable" if anyone desires it. > But really, the people in the Node community who care about being > compatible with pinf-js and ringo are a rounding error. > I use "desirable" to mean, "this feature is pretty darned important or useful function to a program or developer". > > So, desirable to whom? Who feels this "desire" sensation? Not enough > people to matter to me, I'm afraid, and certainly not enough to make > the benefits outweigh the costs. > Just because you don't develop cross-platform doesn't mean you ought to exclude people who do (me and the several others of us in this thread). Again, about the whole inhibiting growth thing. > > Seriously, if you are so convinced, prove it. Fork node, add this > feature, and then wait for the kudos to roll in. Or better yet, go > get Meteor and Vert.x to be compatible with Node, and let the CommonJS > experiment rest in peace. > Now I don't really like CommonJS at all, though admittedly it's better than any other thing I've come across. I'd like to see "return" values honored, but that's trivial enough it doesn't need a fork from me. But I can't see making progress on anything else as is, without development work in other areas first. Right now I'm stuck at convincing people that pragmatism isn't good. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
