We actually use beanstalkd to queue up jobs to be written to disk (event logs to eventually be processed with Hadoop) - sounds sorta similar to what you are describing. Maybe just writing to flat files (or in our case - queuing in beanstalkd to be written to a flat file every few seconds) will be sufficient.
We use beanstalkd in between to effectively make writing to a flat file a non blocking call in our application. -- Chad Kouse On Friday, August 24, 2012 at 5:54 PM, Keith Rarick wrote: > On Fri, Aug 24, 2012 at 10:09 AM, David Leimbach <[email protected]> wrote: > > I don't need it to be in RAM if it's on disk, and RAM is precious on these > > nodes. > > > > > You should probably not be using beanstalkd for this. > > Keeping all jobs in memory is pretty fundamental to > how beanstalkd is intended to work. If you have more > data than fits in memory, consider using a database > such as postgres instead. > > -- > You received this message because you are subscribed to the Google Groups > "beanstalk-talk" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/beanstalk-talk?hl=en. > > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
