Thanks rubyonrailsx and Duy Nguyen, Redis pub/sub had been tried, but it quickly become the performance bottleneck.
The app we are working on requires high performance capability. It handles about 10k client socket stream simultaneously on a 8-core linux box. Averagely, each client stream send/receive 10 messages of 1kb in total buffer every seconds. By using node cluster, we have managed to greatly reduce message process lag and cpu usage. Now la is low. But the question we are facing is how to let 2 clients in different processes(workers) talk to each other efficiently. I think the exact thing I'm looking for is a tcp or udp based cross-process peer-to-peer node js implementation. BTW, regarding to Duy's question > Btw, I'm not sure about "duplicating transfer data" as you mentioned, initially I think data packet just has been forwarded literally, no duplication (Am I wrong here:-??). I haven't read node source of cluster, but if my assumption is correct. Nodejs has implement a js object serialization in its cluster implementation. In that way, cluster worker and master can talk to each other in js object. And data sent by process.send is serialised to buffer and sent to master. When the other end receive that buff, it parse the buffer back to js object. That's why when send a buffer instance, it will be serialised as an array. Regards, ty 2012/9/10 rubyonrailsx <[email protected]> > Did you consider remove the need for communication between service A and > service B . Instead use redis as message pipe? > > -- > rubyonrailsx > Sent with Sparrow <http://www.sparrowmailapp.com/?sig> > > On Monday, September 10, 2012 at 4:44 PM, Yi Tan wrote: > > Hi node mates: > > I'm looking for your advice about how to design and implement an mechanism > for real-time socket-based cross clustered-process data communication and > computation. > > [The server layout] > > clientA <--tcp socket--> node service A <--> DataModelA <--> unique radis > data store > > clientB <--tcp socket--> node service B <--> DataModelB <--> unique radis > data store > > * node service A and B are cluster on the same server > * 1 data model represent 1 client at the run time > * both node service A and B talk to the same redis data store > > [The function request] > > I need to implement a mechanism, in which: > 1. clients in difference processes can communication with each other > efficiently. > 2. there need to be a centralized place for data computation base on data > models represents difference clients > > the logic looks like the following diagram: > > clientA <--tcp socket--> node service A > | > socket pipe > | > v > clientB <--tcp socket--> node service B <--> DataModelA and B <--> unique > radis data store > > Do you know what is the best way to do this, or is there something already > been built > > Many thanks, > > ty > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
