Ok, the deframer I use in smith (a node rpc protocol I designed) is at https://github.com/c9/smith/blob/master/smith.js#L150-191
It's a function that takes an onMessage callback and returns a function that consumes bytes. https://github.com/c9/smith/blob/master/smith.js#L85-94 To encode frames on the sending side, no state machine is required. Simply send the header and the body. https://github.com/c9/smith/blob/master/smith.js#L137-147 I use this over local stdio as well as over tcp sockets. It doesn't care at all that the chunk size ends up being. I unit test it by sending chunks from every size between 1 byte per chunk and the whole message in a single chunk. https://github.com/c9/smith/blob/master/tests/test-framer.js On Wed, Aug 22, 2012 at 9:47 AM, Tomithy Too <[email protected]> wrote: > Hi Tim, > > Thanks for your reply. I am wondering if you could provide me with some code > example of it. > > I would like to clarify that for my instance, the child process are running > in the same server as the master node.js process, ie the messages are not > transferred over a network, and if possible i prefer that it does not go > through the network stack. > > Cheers > Tomithy > > > > On Tuesday, August 21, 2012 9:20:31 PM UTC+8, Tim Caswell wrote: >> >> I implement my own protocols over stdio all the time. Node doesn't wait >> till 8k of data is queued before sending. You just need to treat the same >> as a TCP stream and not assume anything about the chunk sizes. The simplest >> framing protocol for binary data is to send a 4byte length header before >> each message. Then on the receiving end, implement an interruptable state >> machine parser. Have this parser emit message events with the original >> message bodies reconstructed and do you protocol/application logic from >> there. >> >> On Aug 21, 2012 5:31 AM, "Tomithy Too" <[email protected]> wrote: >>> >>> Hi Everyone, >>> >>> I am fairly new to Node.js and I have a hard time trying to figure this >>> one out. Here's I am trying to do: >>> >>> I have a node instance acting as a master program who will spawn other >>> long running applications (possible up to 100 programs) (that is not a >>> node.js instance) as worker applications. I would like my node instance to >>> be able to communicate with them via a REPL style, that is 1) node would >>> issue "work commands" to the worker applications, 2) and the worker >>> applications would be reply with "acknowledgements", "work done" or "error" >>> back to the node, so 3) node.js could effective schedule jobs and optimize >>> the worker applications. >>> >>> The problem here is that the streams between node and the child_process >>> are buffered by default to a length of 8192 bytes, which means that I have >>> to pad my message until it is of this length which is inelegant. I want a >>> much smaller buffer for message, which presumably would be give less >>> communications overhead, and I dont want to be counting characters to make >>> just they return in the same buffer at every message response. Making stream >>> emit a "data" on detection of a "\0" character would be idea. >>> >>> I have tried, various custom stdio options for child_process, to try to >>> modify the buffer length but it was to no avail. Help or pointers would be >>> very much appreciated! >>> >>> Cheers >>> Tomithy >>> >>> >>> >>> -- >>> Job Board: http://jobs.nodejs.org/ >>> Posting guidelines: >>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >>> You received this message because you are subscribed to the Google >>> Groups "nodejs" group. >>> To post to this group, send email to [email protected] >>> >>> To unsubscribe from this group, send email to >>> [email protected] >>> >>> For more options, visit this group at >>> http://groups.google.com/group/nodejs?hl=en?hl=en > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
