Found the post with the clarification: https://groups.google.com/d/msg/nodejs/NBSZRyBl11E/KbBS1r6VSBQJ
For the record, at the time: 1) It was very clear that events could interleave with next ticks. 2) I talked about using nextTick to break long computations into chunks in the same thread and noone reacted by saying that there was anything wrong with it. Bruno On Friday, June 1, 2012 10:02:50 PM UTC+2, Bruno Jouhier wrote: > > Isaac, > > Stop saying that the child process API is the answer to our problem: you > don't know what our problem is and you are making up a solution that would > not work for us. > > process.nextTick is a non deteministic call and you want to turn it into a > deterministic one because its non-determinism is breaking your code. I call > this fixing a bug: your code is not robust enough to handle side effects > caused by events that may interleave between nextTick calls and you need to > eradicate these events to be on the safe side. > > I have always assumed that nextTick would be non deterministic and I've > designed my code to deal with this situation in a robust way. So your fix > is not helping me in any way. On the other hand, I've designed my code with > the assumption that nextTick would yield and let other events go through > (and I remember asking a question about the starvation issue on this > mailing list and getting a clarification from you saying that it did let > other events go through). Your "fix" is breaking this assumption. > > But you are in charge. So do as you like but be aware that this change is > not innocuous and that some people have relied on what you told them a > while ago and may be pissed off that you treat them so lightly. > > And BTW, I don't make much difference between your use case #1 and #2. My > use case is actually #2 and I would much prefer that my current nextTick > calls be treated with low priority that with higher priority as they are > today. So the 'onIdle' semantics are really what I'm looking for (as long > as it is efficiently implemented so that onIdle runs as fast as possible > when there is no I/O to process). Maybe we can come to an agreement in the > end. > > Bruno > > On Friday, June 1, 2012 9:19:03 PM UTC+2, Isaac Schlueter wrote: >> >> First of all, Mikeal Rogers is not a "core guy who doesn't write >> apps". Mikeal Rogers has written several apps with node, and has >> authored very few patches to node core. He has a startup, and has >> been building applications with Node since the 0.1 days. >> >> And yes, I work on the node core project, but I'm also a Joyent >> employee. My thoughts on this came from debugging node in production >> applications at scale, especially the memory leaks and HTTP errors >> that Voxer has seen on a regular basis. Also, some of these problems >> have manifested in the node programs that Joyent uses to run data >> centers for real customers, pushing loads of network traffic. >> >> Our complicated nextTick semantics are causing real problems for real >> applications. There is no ivory-tower-ism here. Node exists in the >> real world. That's why we need to make nextTick work properly for the >> cases where it's actually used. Complying with some notion of the >> "correct" meaning of "tick" and "next" is completely not a priority. >> >> I didn't ask for feedback as a joke. I asked for feedback because I >> wanted to know what problems we would encounter if we changed nextTick >> so that it actually works for its intended use case. The conclusion >> that I eventually drew (the plan for node v0.9) is informed by that >> feedback, and was not known at the outset. >> >> There were two use cases presented in objection to the proposal: >> 1. Using process.nextTick to "break up" CPU intensive operations. >> 2. Using process.nextTick as an idle listener. >> >> #2 is a valid use case for which there is no reasonable API at the >> moment, but nextTick is *not* an idle listener anyway, since it will >> frequently be fired in advance of pending IO. setTimeout(fn) is >> better, but we probably ought to implement setImmediate or something >> similar. That was new information, and is very useful. Thank you, >> everyone. >> >> #1 is not a valid use case. nextTick is horrible for this, I just >> can't put it any other way. Either the operations are fast enough to >> be done in a single thread, and a loop is fine, or they're not, and >> you need to put them into a child process. >> >> Meanwhile, we have lots of real world cases where nextTick is causing >> actual problems for real applications. It's adding latency and >> causing errors to be thrown. Phone conversations are interrupted by >> it, and error pages are showing up in web browsers and applications. >> It's an insidious bug that only manifests under load, and it must be >> fixed. >> >> Also, it seems that the documentation of process.nextTick needs to be >> improved along with this change, because there is widespread >> misunderstanding about how it ought to be used. The impression I'm >> getting is that a lot of people treat nextTick a bit like a background >> fork, and it's really not a good idea to use it that way. >> >> Are there other new use cases not already brought up in this thread? >> If so, please share them. They may make a big difference. >> >> For those who feel disenfranchised by this decision, please consider >> where I'm coming from. Half the people in node assumed that it worked >> one way, and the other half believed it worked the other way, and in >> fact, both were at least partly wrong. It's failing at its intended >> purpose, and causing http to be subtly broken in some cases, which is >> causing real world problems. >> >> We can make node much more reliable for high-traffic HTTP servers >> (which is what node is explicitly for), at the expense of making it >> slightly worse at some approaches to high-CPU use cases (which is what >> the child process API is for), and make node's internals simpler in >> the process. >> >> We can't please everyone. But please don't be so presumptuous to >> assume that this is somehow about library authors being out of touch >> with real world application developers. This is about choosing which >> real world application developers to please, and which developers are >> going to have to change their code. >> > On Friday, June 1, 2012 10:02:50 PM UTC+2, Bruno Jouhier wrote: > > Isaac, > > Stop saying that the child process API is the answer to our problem: you > don't know what our problem is and you are making up a solution that would > not work for us. > > process.nextTick is a non deteministic call and you want to turn it into a > deterministic one because its non-determinism is breaking your code. I call > this fixing a bug: your code is not robust enough to handle side effects > caused by events that may interleave between nextTick calls and you need to > eradicate these events to be on the safe side. > > I have always assumed that nextTick would be non deterministic and I've > designed my code to deal with this situation in a robust way. So your fix > is not helping me in any way. On the other hand, I've designed my code with > the assumption that nextTick would yield and let other events go through > (and I remember asking a question about the starvation issue on this > mailing list and getting a clarification from you saying that it did let > other events go through). Your "fix" is breaking this assumption. > > But you are in charge. So do as you like but be aware that this change is > not innocuous and that some people have relied on what you told them a > while ago and may be pissed off that you treat them so lightly. > > And BTW, I don't make much difference between your use case #1 and #2. My > use case is actually #2 and I would much prefer that my current nextTick > calls be treated with low priority that with higher priority as they are > today. So the 'onIdle' semantics are really what I'm looking for (as long > as it is efficiently implemented so that onIdle runs as fast as possible > when there is no I/O to process). Maybe we can come to an agreement in the > end. > > Bruno > > On Friday, June 1, 2012 9:19:03 PM UTC+2, Isaac Schlueter wrote: >> >> First of all, Mikeal Rogers is not a "core guy who doesn't write >> apps". Mikeal Rogers has written several apps with node, and has >> authored very few patches to node core. He has a startup, and has >> been building applications with Node since the 0.1 days. >> >> And yes, I work on the node core project, but I'm also a Joyent >> employee. My thoughts on this came from debugging node in production >> applications at scale, especially the memory leaks and HTTP errors >> that Voxer has seen on a regular basis. Also, some of these problems >> have manifested in the node programs that Joyent uses to run data >> centers for real customers, pushing loads of network traffic. >> >> Our complicated nextTick semantics are causing real problems for real >> applications. There is no ivory-tower-ism here. Node exists in the >> real world. That's why we need to make nextTick work properly for the >> cases where it's actually used. Complying with some notion of the >> "correct" meaning of "tick" and "next" is completely not a priority. >> >> I didn't ask for feedback as a joke. I asked for feedback because I >> wanted to know what problems we would encounter if we changed nextTick >> so that it actually works for its intended use case. The conclusion >> that I eventually drew (the plan for node v0.9) is informed by that >> feedback, and was not known at the outset. >> >> There were two use cases presented in objection to the proposal: >> 1. Using process.nextTick to "break up" CPU intensive operations. >> 2. Using process.nextTick as an idle listener. >> >> #2 is a valid use case for which there is no reasonable API at the >> moment, but nextTick is *not* an idle listener anyway, since it will >> frequently be fired in advance of pending IO. setTimeout(fn) is >> better, but we probably ought to implement setImmediate or something >> similar. That was new information, and is very useful. Thank you, >> everyone. >> >> #1 is not a valid use case. nextTick is horrible for this, I just >> can't put it any other way. Either the operations are fast enough to >> be done in a single thread, and a loop is fine, or they're not, and >> you need to put them into a child process. >> >> Meanwhile, we have lots of real world cases where nextTick is causing >> actual problems for real applications. It's adding latency and >> causing errors to be thrown. Phone conversations are interrupted by >> it, and error pages are showing up in web browsers and applications. >> It's an insidious bug that only manifests under load, and it must be >> fixed. >> >> Also, it seems that the documentation of process.nextTick needs to be >> improved along with this change, because there is widespread >> misunderstanding about how it ought to be used. The impression I'm >> getting is that a lot of people treat nextTick a bit like a background >> fork, and it's really not a good idea to use it that way. >> >> Are there other new use cases not already brought up in this thread? >> If so, please share them. They may make a big difference. >> >> For those who feel disenfranchised by this decision, please consider >> where I'm coming from. Half the people in node assumed that it worked >> one way, and the other half believed it worked the other way, and in >> fact, both were at least partly wrong. It's failing at its intended >> purpose, and causing http to be subtly broken in some cases, which is >> causing real world problems. >> >> We can make node much more reliable for high-traffic HTTP servers >> (which is what node is explicitly for), at the expense of making it >> slightly worse at some approaches to high-CPU use cases (which is what >> the child process API is for), and make node's internals simpler in >> the process. >> >> We can't please everyone. But please don't be so presumptuous to >> assume that this is somehow about library authors being out of touch >> with real world application developers. This is about choosing which >> real world application developers to please, and which developers are >> going to have to change their code. >> > On Friday, June 1, 2012 10:02:50 PM UTC+2, Bruno Jouhier wrote: > > Isaac, > > Stop saying that the child process API is the answer to our problem: you > don't know what our problem is and you are making up a solution that would > not work for us. > > process.nextTick is a non deteministic call and you want to turn it into a > deterministic one because its non-determinism is breaking your code. I call > this fixing a bug: your code is not robust enough to handle side effects > caused by events that may interleave between nextTick calls and you need to > eradicate these events to be on the safe side. > > I have always assumed that nextTick would be non deterministic and I've > designed my code to deal with this situation in a robust way. So your fix > is not helping me in any way. On the other hand, I've designed my code with > the assumption that nextTick would yield and let other events go through > (and I remember asking a question about the starvation issue on this > mailing list and getting a clarification from you saying that it did let > other events go through). Your "fix" is breaking this assumption. > > But you are in charge. So do as you like but be aware that this change is > not innocuous and that some people have relied on what you told them a > while ago and may be pissed off that you treat them so lightly. > > And BTW, I don't make much difference between your use case #1 and #2. My > use case is actually #2 and I would much prefer that my current nextTick > calls be treated with low priority that with higher priority as they are > today. So the 'onIdle' semantics are really what I'm looking for (as long > as it is efficiently implemented so that onIdle runs as fast as possible > when there is no I/O to process). Maybe we can come to an agreement in the > end. > > Bruno > > On Friday, June 1, 2012 9:19:03 PM UTC+2, Isaac Schlueter wrote: >> >> First of all, Mikeal Rogers is not a "core guy who doesn't write >> apps". Mikeal Rogers has written several apps with node, and has >> authored very few patches to node core. He has a startup, and has >> been building applications with Node since the 0.1 days. >> >> And yes, I work on the node core project, but I'm also a Joyent >> employee. My thoughts on this came from debugging node in production >> applications at scale, especially the memory leaks and HTTP errors >> that Voxer has seen on a regular basis. Also, some of these problems >> have manifested in the node programs that Joyent uses to run data >> centers for real customers, pushing loads of network traffic. >> >> Our complicated nextTick semantics are causing real problems for real >> applications. There is no ivory-tower-ism here. Node exists in the >> real world. That's why we need to make nextTick work properly for the >> cases where it's actually used. Complying with some notion of the >> "correct" meaning of "tick" and "next" is completely not a priority. >> >> I didn't ask for feedback as a joke. I asked for feedback because I >> wanted to know what problems we would encounter if we changed nextTick >> so that it actually works for its intended use case. The conclusion >> that I eventually drew (the plan for node v0.9) is informed by that >> feedback, and was not known at the outset. >> >> There were two use cases presented in objection to the proposal: >> 1. Using process.nextTick to "break up" CPU intensive operations. >> 2. Using process.nextTick as an idle listener. >> >> #2 is a valid use case for which there is no reasonable API at the >> moment, but nextTick is *not* an idle listener anyway, since it will >> frequently be fired in advance of pending IO. setTimeout(fn) is >> better, but we probably ought to implement setImmediate or something >> similar. That was new information, and is very useful. Thank you, >> everyone. >> >> #1 is not a valid use case. nextTick is horrible for this, I just >> can't put it any other way. Either the operations are fast enough to >> be done in a single thread, and a loop is fine, or they're not, and >> you need to put them into a child process. >> >> Meanwhile, we have lots of real world cases where nextTick is causing >> actual problems for real applications. It's adding latency and >> causing errors to be thrown. Phone conversations are interrupted by >> it, and error pages are showing up in web browsers and applications. >> It's an insidious bug that only manifests under load, and it must be >> fixed. >> >> Also, it seems that the documentation of process.nextTick needs to be >> improved along with this change, because there is widespread >> misunderstanding about how it ought to be used. The impression I'm >> getting is that a lot of people treat nextTick a bit like a background >> fork, and it's really not a good idea to use it that way. >> >> Are there other new use cases not already brought up in this thread? >> If so, please share them. They may make a big difference. >> >> For those who feel disenfranchised by this decision, please consider >> where I'm coming from. Half the people in node assumed that it worked >> one way, and the other half believed it worked the other way, and in >> fact, both were at least partly wrong. It's failing at its intended >> purpose, and causing http to be subtly broken in some cases, which is >> causing real world problems. >> >> We can make node much more reliable for high-traffic HTTP servers >> (which is what node is explicitly for), at the expense of making it >> slightly worse at some approaches to high-CPU use cases (which is what >> the child process API is for), and make node's internals simpler in >> the process. >> >> We can't please everyone. But please don't be so presumptuous to >> assume that this is somehow about library authors being out of touch >> with real world application developers. This is about choosing which >> real world application developers to please, and which developers are >> going to have to change their code. >> >
