I use beanstalk queues for database updates on one of my projects to
avoid DDoS'ing my web server.
I wrote a server side daemon that reads messages out of a queue that
look like this:
{'table': {'id': 123, 'field1', 'value', 'field2', 'value'}}
It is pretty straight forward to use the YAML support built into the
standard beanstalk clients to serialize your structures from your source
language. I use a ruby reader on the server, and ruby and perl clients
(one of my users may have implemented a java client recently as well).
If you are using Rails on your server, it is really easy to convert the
table name to model with code like:
data = record-from-beanstalk-message-ybody
table = 'users'
model = table.singularize.classify.constantize
record = model.find(data['id'])
data.delete :id
record.update_attributes(data)
To help reduce client/server transactions, I also implemented support of
nested, relational lookups so the clients do not need to look up the IDs
of related tables. E.g.,
{'reports':
{'id': 123, 'notes': 'some notes', 'users': {'name': 'fox'}}}
In this example, the user record would be looked up, and user_id would
be set in the report record.
Good luck in your project.
IF
On 02/09/2014 01:29 PM, Piotr Koryśko wrote:
> Hi all,
> I just found out about beanstalkd. I got lot insert/update queries per
> second in my database, which my PostgreSQL (with PgBouncer) can't handle
> (statement timeouts). I can afford to have this data available to read with
> some delay.
> Can I use beanstalked to create queries queue, how can I achive that? The
> documentation is quite poor.
>
--
You received this message because you are subscribed to the Google Groups
"beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/beanstalk-talk.
For more options, visit https://groups.google.com/groups/opt_out.
