I work in development of a large fleet management software. We have thousands and thousands of trackers sending messages every few seconds. This is our structure today:
- gateways this guy talk direct with the tracker or with third party sofware that talk with the tracker. they receive and parse messages to a format that we expect, and, send messages to trackers when need. every tracker brand has its own instance of tracker and we have several instances of the same brand running to load balance / fail safe. - handlers handlers are used to direct the traffic to our internal services, databases, filters, etc... the communication between the gateways and the handlers today is built in rabbit, but, i want to change it to beanstalk. every gateway type have a isolated queue and every income message is directed to this queue to be processed later by the handlers. not really a problem here. the tricky part is when we need to send messages back to trackers. the trackers can connect in any gateway of his kind. we plan to have a isolated queue to each tracker and everytime we send a messsage, we enqueue direct in the tracker queue. when the tracker connect into the gateway, the gateway does the bind to the tracker queue, so every message that is there is delivered to the tracker. and if the tracker choose to change the server, not a problem, the messages are in the tracker queue and not in the gateway queue. this could be done with a database, but involves locking, stale process, etc... 2014-08-19 17:07 GMT-03:00 Chad Kouse <[email protected]>: > 1) Yes > 2) No > (these are both afaik) > > 3/4 I don’t know > > I don’t know that I would suggest using so many tubes.. Maybe if you > describe your problem a bit we can help determine if there is any better > way to solve it. > > -- > Chad Kouse > Sent with Airmail > > On August 19, 2014 at 2:10:56 PM, Diego Bernardes ( > [email protected]) wrote: > > 1) is beanstalk single thread? > don't know why, but all the time i get a fixed 25% cpu usage here and i > have 4 cores > > 2) does beanstalkd have any hard limit on tube quantity? > i need to create individual tubes to each individual gps tracker that i > have (100k+) > > 3) whats the min size of a tube? > in rabbitmq a empty queue uses 30kb of memory, so, 100k queues gonna use > almost 3gb of memory only to queues... > > 4) is beanstalk being used in any big company? > dont want to migrate a large system to beanstalk and suffer from a project > being discontinued or something like this. > i think beanstalk is awesome and does a amazing jog in what it propose. > > --- > > rabbitmq is discarted because the issue with the queue quantity. redis was > a alternative, but, need too much logic and lua scripts to do a reliable > queue the way we need. beanstalk just fit perfect our needs. > it is faster or similar to redis speed and much faster than rabbitmq. > > -- > You received this message because you are subscribed to the Google Groups > "beanstalk-talk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/beanstalk-talk. > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/beanstalk-talk. For more options, visit https://groups.google.com/d/optout.
