i'm not sure, but to mee it looks like there is a misunderstanding:
a transform is built out of a readable and a writable part. with .push you
put chunks in the internal buffer of the readable part, so it can inform
consumer and provide them the data. with done() you inform the writable
part, that the chunk is consumed, and it's ok to consume the next one. both
have verry little to do with each other.
so if you have to wait for 3 chunks before you can push something, then
it's ok to wait for them:
_transform = function(chunk,encoding,done) {
var that = this;
//first buffer them in your small own buffer
that.chunks.push(chunk)
// then check if you can proceed with transformation
if( that.chunks.length === 3){
myTransform(chunk,function(data) {
that.push(data);
}
}
// either you process or wait: at this point you have consumed the
incomming chunk, so call done()
done();
}
Am Mittwoch, 5. März 2014 01:17:37 UTC+1 schrieb Jeremy Hubble:
>
> Thanks! It seems to get tricky if there are iterations that may not
> necessarily callback. I tried a workaround by calling done before push
> data. This works as long as there are more than one chunks, but has
> problems if there is only one chunk is received.
>
> _transform = function(chunk,encoding,done) {
> var that = this;
> myTransform(chunk,function(data) {
> that.push(data);
> }
> done();
> }
>
> One workaround is to simply use a stack of done functions, and them call
> them all once some data is retrieved.
> On Tuesday, March 4, 2014 1:55:09 PM UTC-8, Timothy J Fontaine wrote:
>>
>> You can push as many times in the _transform stream as you like,
>> including 0, and you can set an interval that comes back later and pushes,
>> or you can opt to wait until _flush is called and then push.
>>
>> If you're implementing a protocol parser and may need multiple chunks
>> then you just consume the data, and then not push until you have enough
>> data to act upon, and then push.
>>
>> In other words it doesn't need to be a 1:1 map of _transform to .push,
>> but there does need to be a 1:1 map of _transform to done callbacks
>>
>>
>> On Tue, Mar 4, 2014 at 1:05 PM, Jeremy Hubble
>> <[email protected]>wrote:
>>
>>> Use case is an external service that may need more than one chunk of
>>> data to perform its operation.
>>>
>>> An analogy would be in some unix commands.
>>>
>>> For example:
>>> cat file.txt | grep someLongString
>>>
>>> In node, we get chunks at a time and need to return the result.
>>>
>>> Jeremy
>>>
>>>
>>>
>>>
>>> On Tuesday, March 4, 2014 12:33:40 PM UTC-8, mscdex wrote:
>>>>
>>>> On Tuesday, March 4, 2014 2:45:11 PM UTC-5, Jeremy Hubble wrote:
>>>>>
>>>>> Is there a way to construct a transform pipe such that it can receive
>>>>> multiple chunks before calling everything back?
>>>>>
>>>>>
>>>> What's the use case?
>>>>
>>> --
>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines:
>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to [email protected]
>>> To unsubscribe from this group, send email to
>>> [email protected]
>>> For more options, visit this group at
>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>
--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
---
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
