I second the flow-to-disk feature, this is a major problem with RabbitMQ today.
On Oct 27, 4:56 am, Eran Sandler <[email protected]> wrote: > Thanks for the details. It seems like there might be different needs if we > were to implement a flow-to-disk scenario, since that will require a > different way of scanning the file and loading the data. > > Eran > > On Mon, Oct 26, 2009 at 3:46 PM, Graham Barr <[email protected]> wrote: > > > On Oct 26, 2009 1:35am, Eran Sandler <[email protected]> wrote: > > > When a job enters, it goes into memory and binlog. When the job is marked > > as finished, it is removed from both memory and binlog, correct? > > > Close. the binlog is a log of events. binlog files are only ever > > appended to and only removed when they contain no events needed to > > recreate the current state. > > > So when a job is created/delayed/deleted etc an event is written to > > the log. A reference count is kept on each log file for each job > > created in that log file that has not yet been deleted. When that > > count reaches zero the file may be removed. A file may only be removed > > if all files before it have been removed. This is because reference > > counts are only kept for job creation and any file may contain a state > > change for any job previously created. > > > So when a job is deleted and event is logged into the binlog and the > > reference count on the file that created it is decremented. If that > > happens to be the oldest binlog file and the count goes to zero, then > > it will be removed, as will any file directly following it if the > > reference count for that file has already reached zero. > > > Graham. > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en -~----------~----~----~----~------~----~------~--~---
