Hi, I just found beanstalkd is quite simple but very useful. Thank you for such a great contribution.
I'm not familiar with the design and implementation of beanstalkd because I just went through the code yesterday. So if I made any mistake or invalid consumption, please correct me. I noticed that beanstalkd uses normal read/write style file IO on the binlog and keeps all the job data in the memory, including the bookmarking information and body data. If binlog is activated, is it a good idea to map the binlog file into memory directly? The advantage of this approach is: 1) No double cache of the job. Just leave it in the binlog and map it to memory, the VFS laryer will manage it for beanstalkd. The current approach caches the job in heap memory, which is backed up by swap space, it also caches the job in the VFS layer of file system. 2) The number of system call can be decreased, so that the latency of single operation might be better. 3) Binlog can be used as the default option. And even the memory cannot hold all the jobs, that's fine. Let the VFS layer to do the cache management. Varnish uses the same approach and it's reported with good performance number. Refer: http://varnish-cache.org/wiki/ArchitectNotes The only disadvantage I can though of is if the OS swapped the memory page, due to the single thread implementation of beanstalkd, all operations by different client connections may be affected... We can tune the memory settings of process to avoid it for low latency environment or disable the binlog at all. -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
