On Jan 16, 1:45 pm, Ilya Grigorik <[email protected]> wrote:
> Hey guys, late to the party, but I'll add my 2 cents. :)
>
> I can appreciate the line of thought around framing + binary protocol
> from a performance standpoint, but personally I question the time vs
> benefit tradeoff (in terms of your own time). As it stands, you guys
> have already made beanstalkd one of the fastest job scheduling systems
> out there, and reliable! We've been running many instances of
> beanstalk for almost a year now and never stumbled into speed as a
> problem.
My particular thing on the framing (more than the binary protocol, but
as Kieth pointed out, they go together very well) is not so much
performance (execution speed) as reliability in code. There are just
fewer potential issues with a framed message, since less code is
needed for handling such. If it was strictly about speed, I would
totally concede this point :)
>
> 1) New 'page-to-disk' option is awesome, as it gives us disaster
> recovery, and now I'm wondering: what about persistence proper.
> Namely, beanstalkd is pure memory at the moment, but what about
> crossing that threshold? I realize that this is exactly orthogonal to
> the performance track, but I would love to have beanstalk be able to
> swap out large sets of jobs to disk once the memory is full. Perhaps
> you can even use the current persister as a starting point.
>
> Here's the use case: I have ~90 million jobs which need to be
> scheduled, which adds up to ~30GB of memory. Now, this is not
> outrageous, but it's definitely a non-trivial investment (one machine,
> or a cluster). In reality though, most of those jobs are far away in
> the future (+6 hours), so there is no reason why they should be kept
> in memory.. If they could instead go to disk, and be brought into
> memory as needed, that would be nice.
>
Intersting, I like this.
Regards,
Erich
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"beanstalk-talk" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/beanstalk-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
