I've not yet needed streams which include null and undefined, but it seems
a shame to exclude them when everything else is available. That's the main
thing I'm trying to solve.
With the changes to push I'm trying to
separate concerns (one method for one purpose),
remove magic values (undefined and null),
make the interface more consistent between objectMode and non-objectMode
(allow a no-data push in neither or both)
and
allow streaming of arbitrary objects (null and undefined).
I'd not considered doing a direct writer.write(reader.read()) with no
intermediate null-check or unwrapping. Today this throws TypeError: invalid
data:
node -e 'process.stdout.write(process.stdin.read());'
Although I suppose if one controlled the streams such that read() was known
to not return null that would work.
I'd expect pipe (flow, actually) would continue checking for null as it
currently does and also prevent {} from being nested to [[[[[{}]]]]];
however, that forces callers of read() to be objectMode aware! :-(
With my changes to read I'm trying to:
make read(size) meaningful in objectMode to match non-objectMode,
and
allow null and undefined to be streamed.
On Friday, April 5, 2013 2:21:27 PM UTC-6, Isaac Schlueter wrote:
>
> That is a very complicated interface you're proposing.
>
> Also, your objectMode changes break the current symmetry, where the
> return value from read() is a valid argument to write(). Consider a
> chain of 5 objectMode passthrough streams piped to one another. You'd
> write {} and get [[[[[{}]]]]] out the other end!
>
> I think it's fine for null to be the magical EOF signal, and I'm not
> convinced that objectMode streams actually need a corresponding
> push('') mechanism. It's kind of a weird thing that leads to strange
> issues as it is in TLS, and no other byte stream actually should ever
> use this.
>
> I'm not sure what problem you're actually trying to solve. Can you
> back up and explain what you're trying to do, and what problems you're
> encountering with the interface as it is?
>
>
>
> On Thu, Apr 4, 2013 at 10:14 PM, Andrew Hart
> <[email protected]<javascript:>>
> wrote:
> > I have some ideas for modifications to the streams API, and I'd like
> some
> > feedback.
> >
> > First, thanks to isaacs for the detailed post around Readable.push('')
> and
> > TLS here https://groups.google.com/d/msg/nodejs/xzt5CLqIJe0/FBsneg30KVYJ
> > Thanks as well as participants in similar discussions here:
> > https://github.com/joyent/node/issues/4819
> > https://github.com/joyent/node/issues/5161
> >
> > Readable.push has had its argument overloaded:
> > null - end of data
> > undefined - end of data (undocumented)
> > '' - I'm not providing more data for now; you try again later
> > (non-objectMode only)
> >
> > 5161 proposes adding a NO_DATA object to this list for an objectMode
> version
> > of ''. I've implemented this and written some tests, but I'm not
> convinced
> > it is a good API.
> >
> > Push is being used for multiple things in a way that makes it impossible
> for
> > implementations which merely override the _read method in objectMode to
> send
> > null or undefined as part of the object stream. It would be possible to
> add
> > special objects NULL_DATA and UNDEFINED_DATA, but that further
> complicates
> > the interface.
> >
> > Proposal:
> > For backwards compatibility leave push as-is but flag it for removal
> > someday. Create new functions:
> > offerEOF() - equivalent to current push(null) and push(undefined)
> > offerLater() - like to push('') but works in objectMode as well; maybe
> this
> > could be removed entirely and TLS managed differently as I think is
> alluded
> > to in 4819.
> > offer(chunk) - normal case of current push but allows pushing anything
> when
> > in objectMode, including null and undefined
> > offerMany(arrayOfChunks) - equivalent to multiple calls to offer(chunk)
> but
> > might be more efficient and/or convenient when combined with the read
> > proposal
> >
> > Readable.read(n) compresses its argument to 0 or 1 in objectMode and
> returns
> > a single object or null if no object is available. This makes it
> impossible
> > for a consumer of a Readable to receive a null as part of the object
> stream.
> > It also makes it impossible to read multiple objects with a single call.
> >
> > Proposal:
> > Leave non-objectMode read alone. In objectMode:
> > read(0) behaves as it currently does
> > read() returns an array of all currently available objects, but returns
> null
> > in place of the empty array
> > read(size) counts objects like non-objectMode streams count bytes,
> returning
> > null or an array of the requested number of objects
> >
> > I'm assuming more flexibility in changing the interface here because of
> > objectMode's limitted documentation. If needed a new name could be
> chosen:
> > readEx, multiRead, etc.
> >
> > There is no general interface for changing streams options after
> > construction. For built-in streams or streams received from another
> module
> > it'd be nice to be able to request the stream change its options. This
> might
> > not be possible for some options on some streams. In a case I dealt with
> > recently, setting the highWaterMark for process.stdin might have helped,
> but
> > I did not create it.
> >
> > Proposal:
> > Stream.setOption(key, value) - request a stream change option key to new
> > value; returns value after modification, if any
> > Stream.getOption(key) - request the current value; may be useful for
> > tracking highWaterMark and querying for objectMode
> >
> > --
> > --
> > Job Board: http://jobs.nodejs.org/
> > Posting guidelines:
> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to [email protected]<javascript:>
> > To unsubscribe from this group, send email to
> > [email protected] <javascript:>
> > For more options, visit this group at
> > http://groups.google.com/group/nodejs?hl=en?hl=en
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups
> > "nodejs" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to [email protected] <javascript:>.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
