Hi people! Mike, I guess the main approaches to take are:
- A distributed actor model - A service bus For the second option (started some month ago, but revamped inspired by your link about hook.io), I wrote: https://github.com/ajlopez/SimpleBus See some distributed tests https://github.com/ajlopez/SimpleBus/blob/master/test/client.js I have code for the first option, too, but I guess the second is the simplest to follow now, baby steps. My solution is a "pet project" but it could serve to explore your ideas. For the first option, you could use Federation. A question: You said a process A produces output to other process(es), said B. The process A knows in advance process B? In a message bus, process A emits a message, and process B (or B1, B2, and more) subscribe to some kind of message. So, processes A and B (B1, B2....) are loosely couple, and can be replaced, even in hot production, I guess. Angel "Java" Lopez @ajlopez gh:ajlopez On Mon, Feb 18, 2013 at 2:33 PM, MikeB_2012 <[email protected]> wrote: > Hi all. I'm an experienced data mining/machine learning algorithm > developer redirecting from programming/prototyping in Matlab (past decade) > to programming in js/node.js. I'm looking for guidance as to what node > packages I should be considering to achieve a certain functionality. > > The functionality I'm looking for is approximately as follows: > > a) an assortment of sensors puts data on a network. Each sensor has a > dedicated process that pre-processes sensor data. The processed data is > then transmitted to other processes; > b) there are several virtual 'layers' or 'groups'. Each layer has a > specific purpose or goal that is attained through the asynchronous > processing of the processed sensor data by one or more processes; > c) should any process fail/crash, the remaining processes are > unaffected; > d) similarly, processes can be added to the system as required without > affecting other processes; and > e) processes take input from other processes. For example, some > processes obviously will use the output of sensor pre-processing > processes. But some processes will simply be taking the output of other > processes. > > So you can picture this network of processes (a 'mesh' ?) with some > subsets of the mesh working towards common goals. The idea is vaguely > comparable to neural networks but with large amounts of feedback say, a > liquid state NN vice multi-layer feed forward NN, complex processes vice > simple transfer functions at the mesh nodes, and asynchrony. > Within the mesh, processes are born and die, but the whole operation > remains largely unaffected in the sense that other processes will only > lose/gain information but they won't fail (cease trying to process). The > latter is important because a nuance is that a given process will decide > for itself what information it will use, what other processes it will > 'listen' to and will have to adapt when changes occur. So processes must > be aware of each other. > > Anyways, I can fake portions of all this using Matlab (in un-real-time) > but node.js seems to offer the potential to actually implement the idea in > real-time, kind of, maybe. (I'm still learning so some of you may know > better.) I was optimistic with one node package, hook.io, but it has > shut down. I'm happy to fork it but would need lots of help and guidance. > Several others were suggested including: kue, rabbitmq, zero-mq, nssocket, > dnode, amino, and federation. But whereas I'd found a clear explanation > of hook.io (with illustrations to confirm it seemed like what I > wanted)<http://ejeklint.github.com/2011/09/23/hook.io-for-dummies-part-1-overview/>most > other packages need knowledge (eg. - reddis) that I don't yet have > ('federation' > being the exception <https://github.com/jacobgroundwater/federation>). > > So that's what I'm trying to do and it looks like js/node.js can help me > do it. Is that a correct conclusion? If so, are there any packages or > approaches that you might suggest? As always, any constructive guidance or > advice would be appreciated. > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
