Hello all,
I have a queue (I am using redis) with things pilling up. Sometimes things
gets pilled up real fast and sometimes real slow.
I have a ruby script that loops forever and empty the queue, one item at a
time.
When there is less than 5000 in the queue, I want only one process.
When there is between 5000 and 9999 items in the queue, I want two process
until the queue is down to 4000 or less.
When there is between 10000 and 14999 items in the queue, I want three
process until the queue is down to 8000 or less.
and so on...
I want the number of processes to go up and down depending on how many
things are in the queue. Is it possible with god? If not, is there an
alternative or a way around?
Here is my god config file that was hoping to work but, I cannot use :stop
in transition (E [2012-09-06 14:00:18] ERROR: Invalid state :stop. Must be
one of the symbols :init, :up, :start, :restart)
rails_env = ENV['RAILS_ENV'] || "production"
rails_bin = `which rails`.chomp
rails_root = ENV['RAILS_ROOT'] ||
File.dirname(File.dirname(File.realdirpath(__FILE__)))
require File.expand_path('environment.rb', File.dirname(__FILE__))
num_process = 5
num_process.times do |num|
God.watch do |w|
w.name = "process_queued_track_hits-#{num}"
# w.start = "#{rails_root}/script/process_queued_track_hits.sh
${rails_env}"
w.start = "exec #{rails_bin} runner -e '#{rails_env}'
'Bonus.process_queued_track_hits'"
w.env = {
'RAILS_ROOT' => rails_root,
'RAILS_ENV' => rails_env
}
w.dir = rails_root
w.interval = 30.seconds
# restart if memory gets too high
w.transition(:up, :restart) do |on|
on.condition(:memory_usage) do |c|
c.above = 350.megabytes
c.times = 2
end
on.condition(:cpu_usage) do |c|
c.above = 50.percent
c.times = 5
end
on.condition(:cpu_usage) do |c|
c.above = 80.percent
c.times = 2
end
end
# determine the state on startup
w.transition(:init, { true => :up, false => :start }) do |on|
on.condition(:process_running) do |c|
c.running = true
end
end
# determine when process has finished starting
w.transition([:start, :restart], :up) do |on|
on.condition(:process_running) do |c|
c.running = true
c.interval = 5.seconds
end
# failsafe
on.condition(:tries) do |c|
c.times = 5
c.transition = :start
c.interval = 5.seconds
end
end
# start if process is not running
w.transition(:up, :start) do |on|
on.condition(:process_running) do |c|
c.running = false
end
end
w.stop_if do |on|
on.condition(:lambda) do |c|
c.interval = 30.seconds
c.lambda = lambda do
$redis.llen("track_hits_queue") < num * 4000
end
end
end
w.transition([:stop, :init], :start) do |on|
on.condition(:lambda) do |c|
c.interval = 30.seconds
c.lambda = lambda do
$redis.llen("track_hits_queue") > num * 5000
end
end
end
end
end
If not possible, I can pass a parameter to
Bonus.process_queued_track_hits(#{num
* 5000}) so that all process always run and my loop check if its time to
start emptying the queue or not.
Thanks.
--
You received this message because you are subscribed to the Google Groups
"god.rb" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/god-rb/-/_3SBvaBPsWYJ.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/god-rb?hl=en.
