Wow That is a lot of messages. I'd recommend trying to re-architect the solution you are building. It doesn't seem like a good idea to use dnode ( or even the node net module ) to pass that many messages on start.
If you want to use node, you'll probably need to bring in an add-on that links to a more robust messaging protocol. On Sun, Jul 1, 2012 at 4:11 PM, Dan Milon <[email protected]> wrote: > 30 million messages :/ > > > On 07/02/2012 02:11 AM, Marak Squires wrote: > >> Err, is ~30mil 30 milliseconds or 30 million messages? >> >> On Sun, Jul 1, 2012 at 4:10 PM, Marak Squires >> <[email protected]<mailto: >> marak.squires@gmail.**com <[email protected]>>> wrote: >> >> How far off are you from hitting your 30ms latency target? >> >> You could probably just use redis, or 0mq with a node.js driver. >> >> On Sun, Jul 1, 2012 at 3:45 PM, Dan Milon <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hey Marak, >> >> Thanks for the valuable info. >> The use case is: I have a few hooks that receive messages each >> one from a different source (rabbitMQ, http) and there is a >> central hook that receives all those messages, does some >> calculations and stores into a database. >> >> Truth is, our message rate is quite lower than 600 msg/s but i >> need to pass in some initial data which are ~30mil, so that >> means i will have to write separate code to do just this, >> without hook.io <http://hook.io>. >> >> >> I could also pack all that stuff into one process and get over >> it, but then i wouldn't make use of multicore etc. >> Do you know other communication/IPC libraries i could look >> into, or got any other suggestion? >> >> Thanks, >> danmilon. >> >> On 07/02/2012 01:35 AM, Marak Squires wrote: >> >> As of today, I've done literally no performance >> optimizations for hook.io <http://hook.io> <http://hook.io>. >> >> hook.io <http://hook.io> <http://hook.io>'s power is in >> >> API and discovery. >> >> If you require more then 666.66 messages per second out of >> the box, you should not be using dnode as your message >> transport. hook.io <http://hook.io> <http://hook.io> >> >> currently uses dnode as it's primary message transport. >> >> hook.io <http://hook.io> <http://hook.io> is intended to >> >> be multi-transport, so for high performance situations >> you'd want to use hook.io <http://hook.io> >> <http://hook.io> as a wrapper around a more robust >> solution like 0mq. >> >> If do you don't mind, I'd be interested in hearing what >> your intended use case for hook.io <http://hook.io> >> <http://hook.io> was which would require that throughput. >> Always good to get more input to shape the API. >> >> >> On Sun, Jul 1, 2012 at 3:18 PM, Dan Milon >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>> >> wrote: >> >> Hello, >> >> I am using hook.io <http://hook.io> <http://hook.io> >> >> for some project. >> Today i thought of measuring how fast hook.io >> <http://hook.io> <http://hook.io> can >> >> >> emit and receive messages, so i built a dead simple >> benchmark [1], >> and numbers are not that interesting. >> It took 15 secs to deliver 10k messages from one hook >> to another. >> >> I do not know the underlaying mechanisms hook.io >> <http://hook.io> <http://hook.io> >> >> >> uses for IPC (please enlighten me), but isn't this >> performance >> quite low? Or is there something wrong with the test? >> >> Thanks, >> Dan Milon. >> >> [1] >> https://gist.github.com/**3029842<https://gist.github.com/3029842> >> >> -- Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed >> to the Google >> Groups "nodejs" group. >> To post to this group, send email to >> [email protected] >> <mailto:nodejs@googlegroups.**com<[email protected]> >> > >> <mailto:nodejs@googlegroups.**com<[email protected]> >> >> <mailto:nodejs@googlegroups.**com <[email protected]>>> >> >> To unsubscribe from this group, send email to >> >> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]> >> >> <mailto:nodejs%2Bunsubscribe@**googlegroups.com<nodejs%[email protected]> >> > >> >> <mailto:nodejs%2Bunsubscribe@**googlegroups.com<nodejs%[email protected]> >> >> <mailto:nodejs%**252Bunsubscribe@googlegroups.**com<nodejs%[email protected]> >> >> >> >> >> For more options, visit this group at >> >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> -- Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to >> the Google >> Groups "nodejs" group. >> To post to this group, send email to >> [email protected] >> <mailto:nodejs@googlegroups.**com<[email protected]> >> > >> >> To unsubscribe from this group, send email to >> >> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]> >> >> <mailto:nodejs%2Bunsubscribe@**googlegroups.com<nodejs%[email protected]> >> > >> For more options, visit this group at >> >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> >> -- Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> <mailto:nodejs@googlegroups.**com <[email protected]>> >> To unsubscribe from this group, send email to >> >> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]> >> >> <mailto:nodejs%2Bunsubscribe@**googlegroups.com<nodejs%[email protected]> >> > >> For more options, visit this group at >> >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> >> >> >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** >> Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]> >> For more options, visit this group at >> http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> >> > > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: https://github.com/joyent/**node/wiki/Mailing-List-** > Posting-Guidelines<https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines> > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > nodejs+unsubscribe@**googlegroups.com<nodejs%[email protected]> > For more options, visit this group at > http://groups.google.com/**group/nodejs?hl=en?hl=en<http://groups.google.com/group/nodejs?hl=en?hl=en> > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
