Hey John, I created medea-clusterify[1] to handle this use case. It's using worker.send and process.send to do exactly as you describe. This is not ideal, but it works. The master process holds the Medea instance, and the workers communicate with it. This means the master process can bottleneck. However, you can still handle a lot more connections concurrently.
Doing a key-space partition is another way to handle this. To do that, you have to route traffic between processes based on owned key-space. There has been some discussion about this in a GitHub issue[2]. As discussed in this issue, cross-platform memory mapping would be a good way to not duplicate in-memory data structures per process, but that won't come until at least v2. LevelDB (and levelUP) restricts usage to a single process, as well, though you could build some multi-process support around it. [1] https://github.com/argo/medea-clusterify [2] https://github.com/argo/medea/issues/10 -- Kevin Swiber @kevinswiber https://github.com/kevinswiber On Apr 19, 2013, at 2:52 PM, John Fitzgerald <[email protected]> wrote: > When using these embedded databases, like node-levelup, how do you handle > cluster based apps? I'd love to use an embedded database, but also need to > run our express app on all 8 cores of the machine. > > I know I can process.send to children & master, but that seems like a clunky > way of interfacing. Creating a "db" process separate that responds over HTTP > and have the workers do requests against that is possible, but not sure if > there's a more common / clean way of implementing. > > > > > > > On Fri, Apr 19, 2013 at 10:36 AM, Ben Taber <[email protected]> wrote: > No guarantees that you can read a record before .load has been called, but > you can write before load. > > https://github.com/felixge/node-dirty#dirty-event-load-length > > To just open the file once on app init, wrap it in its own module, and then > require that into other modules. Something like below would work. > > -- db.js > var db = require('dirty')('./data.db'); > var isLoaded; > > db.on('load', function() { > isLoaded = true; > }); > > db.ready = function(cb) { > if (isLoaded) { > return cb(); > } > > db.on('load', cb); > } > > module.exports = db; > > -- otherFile.js > var db = require('./db.js'); > > db.ready(function() { > db.get(); > db.set(); > // etc > }) > > On Friday, April 19, 2013 9:58:15 AM UTC-6, Angelo Chen wrote: > Hi Ben, > > looks like, you have to put access code in db.on('load', ...) > > how to just open the database once at beginning, then use later? say > all the database code will be in mydao.js, exporting some CRUD > methods, so I can call from another js, dao.append({id:1, data:'d'}) > > in this append method, I have to wrap the function in a db.on('load', > function(rec){}) ? > > Angelo > > > On 4月19日, 下午10时07分, Ben Taber <[email protected]> wrote: > > There's dirty for super simple > > storage:https://github.com/felixge/node-dirty > > > > > > > > > > > > > > > > On Friday, April 19, 2013 2:59:28 AM UTC-6, Angelo Chen wrote: > > > > > Hi, > > > > > What are the options for embedded database? I use redis and mongodb for > > > now, but sometimes you made some small apps, and does not want to mix > > > data > > > with existing redis db or mongodb. it should be easier to install, now > > > I'm > > > looking at nosql,https://npmjs.org/package/nosql, also ejdb, > > >https://npmjs.org/package/ejdb, but seems you can not have your own data > > > file for a individual app. sqllite is another, but it's not json based, > > > any > > > suggestions? Thanks, > > > > > Angelo > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > John R. Fitzgerald > > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
