> Yes, but why not do things the natural way:
This an imperative vs functional style mismatch. "the natural way" depends
on your background
```js
var fileStream = fs.read("/foo.txt")
var result = fs.write("/bar.txt", fileStream)
result(function (err) {
// entire fileStream has been flushed to disk
})
```
vs
```
var fileStream = fs.read("/foo.txt")
var writeStream = fs.write("/bar.txt")
function pump() {
fileStream.read(function (err, chunk) {
if (err) return cleanup()
writeStream.write(chunk, function (err) {
if (err) return cleanup()
pump()
})
})
}
```
The functional style is a different way of structuring streaming code that
assumes that you "never" just write data somewhere.
You move data from some streaming input to some streaming output in the 95%
case.
I agree there's value in having a helper with a { write: (chunk, callback)
} interface, just is not that important to spec.
>From looking at creationix's js-git code it appears he uses both styles.
With `tcpSocket.write(data)` and `tcpSocket.sink(stream)`
On Thu, Aug 15, 2013 at 2:59 PM, Bruno Jouhier <[email protected]> wrote:
>
>
> On Thursday, August 15, 2013 7:43:30 PM UTC+2, Raynos wrote:
>>
>> > * Would be nice to also include the writable side in the spec, with a
>> symmetrical API (a write / abort pair).
>>
>> For simplicity you can get away with a Writable stream to just be
>> `function (simpleStream) { /* consume it */ }`
>>
>
> Yes, but why not do things the natural way:
>
> When you need to read data, you "open" (or create) a readable stream, then
> you read from it, then you close it.
> When you need to write data, you "open" (or create) a writable stream,
> then you write from it, then you close it.
> Piping is just a loop that reads from a readable stream and writes to a
> writable stream.
>
> This is how you write things in C with standard I/O libraries. Why invent
> some weird APIs instead of following this simple model?
>
> And backpressure is no excuse. If your read and write functions are both
> callback based, backpressure works naturally through the event loop.
>
> Bruno
>
>
> Then you don't need a central `pipe()` or `pump()` implementation and each
> writable stream can implement it's own
>
>> consumption & back pressure algorithm.
>>
>> > * Spec does not say what item is, I'm assuming it can be a Buffer or a
>> string. Can it also be an object? why not?
>>
>> I believe it's an arbitrary value including another simple stream
>>
>>
>> On Thu, Aug 15, 2013 at 3:05 AM, Bruno Jouhier <[email protected]> wrote:
>>
>>> Isaac's message is clear. Simple streams won't make it into core.
>>>
>>> So the solution is probably in userland. Tim's module looks like a good
>>> starting point and if it gets popular it could become a de facto "simple
>>> stream" standard API.
>>>
>>> My comments on it:
>>>
>>> * Would be nice to also include the writable side in the spec, with a
>>> symmetrical API (a write / abort pair).
>>> * Why not rename abort close?
>>> * Spec does not say what item is, I'm assuming it can be a Buffer or a
>>> string. Can it also be an object? why not?
>>> * The "Related Interfaces" part should be left out. If we want it to be
>>> popular, we have to keep it as simple as possible.
>>>
>>> Streamline's streams interface is very similar and provides wrappers
>>> around most of node's steams (HTTP and TCP, client and server). It can also
>>> be used as source of inspiration (but I'll remap it to simple-streams if
>>> simple-streams catches up).
>>>
>>> Bruno
>>>
>>>
>>>
>>> On Wednesday, August 14, 2013 7:02:19 PM UTC+2, Eldar wrote:
>>>>
>>>> Node completely shutdowns because emitter.emit('error') triggers
>>>> exception when there is no listeners for error event.
>>>> The ".pipe()" listens for errors but doesn't change that behavior.
>>>>
>>>> So strictly speaking above server will shutdown on file error because
>>>> of uncaught exception
>>>> and leak on socket hang up, because "close" not an "error" event will
>>>> be emitted.
>>>>
>>>> среда, 14 августа 2013 г., 13:47:14 UTC+4 пользователь Floby написал:
>>>>>
>>>>> file.pipe(res) does clean the resources on error because node
>>>>> completely shuts down, doesn't it?
>>>>>
>>>>> On Tuesday, 13 August 2013 12:08:30 UTC+2, Eldar wrote:
>>>>>>
>>>>>> file.pipe(res)
>>>>>>
>>>>>> On response error (say client clicked close window button) the file
>>>>>> will remain open.
>>>>>> And vice versa on file error response will hang. So the correct way
>>>>>> to pipe is
>>>>>>
>>>>>> file.pipe(res)
>>>>>> file.on('error', res.destroy.bind(res))
>>>>>> res.on('error', file.destroy.bind(res))
>>>>>>
>>>>>> вторник, 13 августа 2013 г., 12:58:09 UTC+4 пользователь Floby
>>>>>> написал:
>>>>>>>
>>>>>>> ok.
>>>>>>> I didn't understand the part about " `.pipe()` doesn't cleanup any
>>>>>>> resources.
>>>>>>> Everything works fine until the streaming process completes
>>>>>>> successfully. "
>>>>>>>
>>>>>>> Is there an example where that becomes obvious?
>>>>>>>
>>>>>>> On Monday, 12 August 2013 18:05:02 UTC+2, Eldar wrote:
>>>>>>>>
>>>>>>>> If you want to create some function which accepts a stream and can
>>>>>>>> affect
>>>>>>>> it's state you should be able to destroy the given stream on demand
>>>>>>>> e.g. when something went wrong and/or you are not interested in
>>>>>>>> receiving data anymore.
>>>>>>>> Unfortunately that's impossible with current node readable streams
>>>>>>>> because they:
>>>>>>>>
>>>>>>>> 1. Support multiple consumers
>>>>>>>> 2. Don't have common cleanup interface
>>>>>>>>
>>>>>>>> So you can't touch another's stream because someone else may use it
>>>>>>>> and if you could you don't know how.
>>>>>>>>
>>>>>>>> The absence of ability to pass streams as arguments is a huge flow.
>>>>>>>> You can build almost nothing
>>>>>>>> without that. Now someone may wonder: but there are so many useful
>>>>>>>> modules about streams in npm!?
>>>>>>>> Right. They all rely on `.pipe()`. For some reason all kind of blog
>>>>>>>> posts, tutorials and even node's
>>>>>>>> own documentation teach that for passing readable you can do
>>>>>>>> `readable.pipe(writable)`.
>>>>>>>> E.g. http file server could be:
>>>>>>>>
>>>>>>>> ```javascript
>>>>>>>> var http = require('http')
>>>>>>>> var fs = require('fs')
>>>>>>>> http.createServer(function(**req**, res) {
>>>>>>>> res.writeHead(200, {'Content-Type': 'text/plain'})
>>>>>>>> fs.createReadStream(req.url.**sl**ice(1)).pipe(res)
>>>>>>>> }).listen(3000)
>>>>>>>> ```
>>>>>>>>
>>>>>>>> That's a lie. Exactly for the reasons above `.pipe()` doesn't cleanup
>>>>>>>> any resources.
>>>>>>>> Everything works fine until the streaming process completes
>>>>>>>> successfully.
>>>>>>>> But if something fails you get horrible leaks and hangs.
>>>>>>>> Almost every module doing stream processing have this flow. Obvious or
>>>>>>>> not.
>>>>>>>>
>>>>>>>> That all is sad. We need another, simpler API. Although I believe
>>>>>>>> incompatible changes should be made
>>>>>>>> to node core that is not what I am calling about (at least not at the
>>>>>>>> first place).
>>>>>>>> But we definitely should stop to expose node streams in userland
>>>>>>>> modules,
>>>>>>>> stop to extend "stream base classes", just stop to use it in userland.
>>>>>>>>
>>>>>>>> Now the constructive part. Tim Caswell proposed recently simple-stream
>>>>>>>> <https://github.com/creationix/js-git/blob/master/specs/simple-stream.md>
>>>>>>>> and build many things on top of it.
>>>>>>>>
>>>>>>>> Let's just peek this API as a standard and start using it.
>>>>>>>> Here is readable-to-simple-stream
>>>>>>>> <https://github.com/eldargab/stream-simple> converter which can be
>>>>>>>> used for dealing with node core.
>>>>>>>>
>>>>>>>> It also contains slightly more detailed version of original spec which
>>>>>>>> could be discussed.
>>>>>>>>
>>>>>>>> That's it.
>>>>>>>>
>>>>>>>> --
>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-*
>>> *Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines>
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to [email protected]
>>>
>>> To unsubscribe from this group, send email to
>>> nodejs+un...@**googlegroups.com
>>>
>>> For more options, visit this group at
>>> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en>
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to nodejs+un...@**googlegroups.com.
>>>
>>> For more options, visit
>>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>>> .
>>>
>>
>> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to [email protected]
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
