Thanks for the detailed explanation. Much was over my head but I think the object vs bytes concept sunk in.
And I promise to try and find the time to run some odd versions. On Fri, Mar 29, 2013 at 4:29 PM, Isaac Schlueter <[email protected]> wrote: > On Fri, Mar 29, 2013 at 11:37 AM, Mark Hahn <[email protected]> wrote: > > Yes, I am guilty of waiting for stability and then trying it. That > makes me > > a leech. Considering the schedule pressure I'm under it's not possible > for > > me to test unstable. > > Oh, I think leech is too strong a word. Sorry, it wasn't my intent to > imply that there's anything *wrong* with waiting for it to be stable. > I'm sure there are a lot of people who wait for it to be not just > stable, but at least x.y.6 or something. > > The even/odd transition is still valuable. It means "the core team > and bleeding-edge experimenters can't find any more serious bugs, and > it's up to par on performance, so we're ready to commit to this API, > and you should try it out now, even if you don't upgrade for a while". > It's totally fine to wait as long as makes sense for your needs. > > That being said, I will probably try to convince you to try out the > unstable builds, because that makes my life a whole lot easier ;) > > > > On the empty-data question: What problem is the new behavior trying to > > solve? It seems like a random change. > > This is a great question! > > Basically, the requirement for an "empty push", comes from the need to > have a way to say "I'm not reading any more, but I don't have any more > data, and I won't be getting any unless you try to ask for it again, > so check back when you know I might have more". > > It's VERY rare that you'll need to say that. However, in core, we > have to support TLS, which is about the most crazy dance you've ever > imagined. Here's (in broad strokes) how it works: > > There's a "pair" of CryptoStreams. One of the two things in the pair > is an EncryptedStream, and the other is a CleartextStream. They are > attached at the hip, so to speak, and each is a duplex. > > Whenever you write() into the EncryptedStream, this sends data into > OpenSSL's machinery. Occasionally, this may cause data to pop *out* > of OpenSSL's machinery for the CleartextStream to consume. Likewise, > whenever you write() into the CleartextStream, this sends data into > the other side of the OpenSSL machine, which may cause data to pop out > on the encrypted side. > > From far away, this almost looks like it ought to be two Transform > streams. One of them you write crypto in and get clear out, and the > other you write clear in and get crypto out. However, TLS is not so > simple. There's all kinds of handshakes and other back-and-forth that > has to happen on the crypto side, without ever producing any cleartext > data. So, sometimes, you write to the EncryptedStream, and this > creates more data for the readable side of the EncryptedStream, but > *not* any data for the CleartextStream. > > So, even if you did it as two Transform streams, you'd still have the > attached-at-the-hip complications. > > So, once you've got the pair, you do something like this: > > socket.pipe(encrypted); > encrypted.pipe(socket); > > and then instead of using the TCP socket for your HTTP stuff, you use > the CleartextStream, which mostly looks just like a net.Socket object. > You write() plain data to it, and plain data comes out from the > readable side, so all is good. > > So, why the push('')? > > Well, every time we write() into one stream in the pair, we need to > trigger a check to OpenSSL to see if there's any more data for the > other side. We do this by calling stream.read(0), which can kick off > a _read() call. However, the Readable machinery is smart enough to > not call _read() if you're already reading and haven't pushed > anything. (Otherwise, you'd have to be careful to not accidentally > fs.read() the file while there's already a pending read, etc., and the > complications for implementing streams gets quite a bit worse.) > > Of course, you could get around this by doing > `stream._readableState.reading = false` but that's definitely too much > of a mingling of concerns. Touching those flags in the constructor is > mildly gross, but usually OK. Messing around with them in the midst > of operation is definitely crossing a line, even for core modules that > are good friends. > > So, `push('')` was implemented as a way to say "I have zero bytes of > data, and the read is completed, but this is not the EOF. It's YOUR > responsibility to try again later. I'm going to sit here and do > nothing until then." This situation does not exist for sockets, > files, or basically any other stream except TLS and some other bizarre > use-cases. > > Regarding whether or not "" is "no data", it really depends on whether > you're a byte stream or an object stream. In fact, your view of "" is > a great litmus test. If "" is relevant data, then you're an object > stream. That is, even if you don't emit "objects" per se, the nature > of the object is itself relevant; not just the bytes it represents. > From a byte stream's point of view, "" is "zero bytes", and thus "zero > data". > > Core streams have to support all the streaming APIs in core. Luckily > for you guys, this means that it'll probably cover all the streaming > APIs that you have as well, but of course, as we've seen, there are > always surprising edge cases that come up when people start using your > code for real things. > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
