Hello community,
here is the log from the commit of package rubygem-delayed_job for
openSUSE:Factory checked in at 2015-09-27 08:40:16
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/rubygem-delayed_job (Old)
and /work/SRC/openSUSE:Factory/.rubygem-delayed_job.new (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "rubygem-delayed_job"
Changes:
--------
--- /work/SRC/openSUSE:Factory/rubygem-delayed_job/rubygem-delayed_job.changes
2015-02-11 16:45:19.000000000 +0100
+++
/work/SRC/openSUSE:Factory/.rubygem-delayed_job.new/rubygem-delayed_job.changes
2015-09-27 08:39:42.000000000 +0200
@@ -1,0 +2,26 @@
+Fri Sep 25 04:28:48 UTC 2015 - [email protected]
+
+- updated to version 4.1.1
+ see installed CHANGELOG.md
+
+ 4.1.1 - 2015-09-24
+ ==================
+ * Fix shared specs for back-ends that reload objects
+
+-------------------------------------------------------------------
+Wed Sep 23 04:29:11 UTC 2015 - [email protected]
+
+- updated to version 4.1.0
+ see installed CHANGELOG.md
+
+ 4.1.0 - 2015-09-22
+ ==================
+ * Alter `Delayed::Command` to work with or without Rails
+ * Allow `Delayed::Worker.delay_jobs` configuration to be a proc
+ * Add ability to set destroy failed jobs on a per job basis
+ * Make `Delayed::Worker.new` idempotent
+ * Set quiet from the environment
+ * Rescue `Exception` instead of `StandardError` in worker
+ * Fix worker crash on serialization error
+
+-------------------------------------------------------------------
Old:
----
delayed_job-4.0.6.gem
New:
----
delayed_job-4.1.1.gem
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ rubygem-delayed_job.spec ++++++
--- /var/tmp/diff_new_pack.uv6hMd/_old 2015-09-27 08:39:42.000000000 +0200
+++ /var/tmp/diff_new_pack.uv6hMd/_new 2015-09-27 08:39:42.000000000 +0200
@@ -24,7 +24,7 @@
#
Name: rubygem-delayed_job
-Version: 4.0.6
+Version: 4.1.1
Release: 0
%define mod_name delayed_job
%define mod_full_name %{mod_name}-%{version}
++++++ delayed_job-4.0.6.gem -> delayed_job-4.1.1.gem ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/CHANGELOG.md new/CHANGELOG.md
--- old/CHANGELOG.md 2014-12-22 14:22:31.000000000 +0100
+++ new/CHANGELOG.md 2015-09-24 23:40:11.000000000 +0200
@@ -1,3 +1,17 @@
+4.1.1 - 2015-09-24
+==================
+* Fix shared specs for back-ends that reload objects
+
+4.1.0 - 2015-09-22
+==================
+* Alter `Delayed::Command` to work with or without Rails
+* Allow `Delayed::Worker.delay_jobs` configuration to be a proc
+* Add ability to set destroy failed jobs on a per job basis
+* Make `Delayed::Worker.new` idempotent
+* Set quiet from the environment
+* Rescue `Exception` instead of `StandardError` in worker
+* Fix worker crash on serialization error
+
4.0.6 - 2014-12-22
==================
* Revert removing test files from the gem
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/README.md new/README.md
--- old/README.md 2014-12-22 14:22:31.000000000 +0100
+++ new/README.md 2015-09-24 23:40:11.000000000 +0200
@@ -1,3 +1,8 @@
+**If you're viewing this at https://github.com/collectiveidea/delayed_job,
+you're reading the documentation for the master branch.
+[View documentation for the latest release
+(4.0.6).](https://github.com/collectiveidea/delayed_job/tree/v4.0.6)**
+
Delayed::Job
============
[][gem]
@@ -58,14 +63,30 @@
rails generate delayed_job:active_record
rake db:migrate
+For Rails 4.2, see [below](#rails-42)
+
Development
===========
In development mode, if you are using Rails 3.1+, your application code will
automatically reload every 100 jobs or when the queue finishes.
You no longer need to restart Delayed Job every time you update your code in
development.
-Rails 4
-=======
-If you are using the protected_attributes gem, it must appear before
delayed_job in your gemfile.
+Rails 4.2
+=========
+Set the queue_adapter in config/application.rb
+
+```ruby
+config.active_job.queue_adapter = :delayed_job
+```
+
+See the [rails
guide](http://guides.rubyonrails.org/active_job_basics.html#setting-the-backend)
for more details.
+
+Rails 4.x
+=========
+If you are using the protected_attributes gem, it must appear before
delayed_job in your gemfile. If your jobs are failing with:
+
+ ActiveRecord::StatementInvalid: PG::NotNullViolation: ERROR: null value
in column "handler" violates not-null constraint
+
+then this is the fix you're looking for.
Upgrading from 2.x to 3.0.0 on Active Record
============================================
@@ -103,9 +124,17 @@
device.deliver
```
-handle_asynchronously can take as options anything you can pass to delay. In
-addition, the values can be Proc objects allowing call time evaluation of the
-value. For some examples:
+## Parameters
+
+`#handle_asynchronously` and `#delay` take these parameters:
+
+- `:priority` (number): lower numbers run first; default is 0 but can be
reconfigured (see below)
+- `:run_at` (Time): run the job after this time (probably in the future)
+- `:queue` (string): named queue to put this job in, an alternative to
priorities (see below)
+
+These params can be Proc objects, allowing call-time evaluation of the value.
+
+For example:
```ruby
class LongTasks
@@ -282,6 +311,20 @@
end
```
+To set a per-job default for destroying failed jobs that overrides the
Delayed::Worker.destroy_failed_jobs you can define a destroy_failed_jobs?
method on the job
+
+```ruby
+NewsletterJob = Struct.new(:text, :emails) do
+ def perform
+ emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
+ end
+
+ def destroy_failed_jobs?
+ false
+ end
+end
+```
+
To set a default queue name for a custom job that overrides
Delayed::Worker.default_queue_name, you can define a queue_name method on the
job
```ruby
@@ -384,6 +427,14 @@
It is possible to disable delayed jobs for testing purposes. Set
`Delayed::Worker.delay_jobs = false` to execute all jobs realtime.
+Or `Delayed::Worker.delay_jobs` can be a Proc that decides whether to execute
jobs inline on a per-job basis:
+
+```ruby
+Delayed::Worker.delay_jobs = ->(job) {
+ job.queue != 'inline'
+}
+```
+
You may need to raise exceptions on SIGTERM signals,
`Delayed::Worker.raise_signal_exceptions = :term` will cause the worker to
raise a `SignalException` causing the running job to abort and be unlocked,
which makes the job available to other workers. The default for this option is
false.
Here is an example of changing job parameters in Rails:
Files old/checksums.yaml.gz and new/checksums.yaml.gz differ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/delayed_job.gemspec new/delayed_job.gemspec
--- old/delayed_job.gemspec 2014-12-22 14:22:31.000000000 +0100
+++ new/delayed_job.gemspec 2015-09-24 23:40:11.000000000 +0200
@@ -4,12 +4,12 @@
spec.description = 'Delayed_job (or DJ) encapsulates the common pattern
of asynchronously executing longer tasks in the background. It is a direct
extraction from Shopify where the job table is responsible for a multitude of
core tasks.'
spec.email = ['[email protected]']
spec.files = %w[CHANGELOG.md CONTRIBUTING.md LICENSE.md README.md
Rakefile delayed_job.gemspec]
- spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*')
+ spec.files += Dir.glob('{contrib,lib,recipes,spec}/**/*')
spec.homepage = 'http://github.com/collectiveidea/delayed_job'
spec.licenses = ['MIT']
spec.name = 'delayed_job'
spec.require_paths = ['lib']
spec.summary = 'Database-backed asynchronous priority queue system --
Extracted from Shopify'
spec.test_files = Dir.glob('spec/**/*')
- spec.version = '4.0.6'
+ spec.version = '4.1.1'
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/backend/base.rb
new/lib/delayed/backend/base.rb
--- old/lib/delayed/backend/base.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/backend/base.rb 2015-09-24 23:40:11.000000000 +0200
@@ -10,7 +10,7 @@
def enqueue(*args) # rubocop:disable CyclomaticComplexity
options = args.extract_options!
options[:payload_object] ||= args.shift
- options[:priority] ||= Delayed::Worker.default_priority
+ options[:priority] ||= Delayed::Worker.default_priority
if options[:queue].nil?
if options[:payload_object].respond_to?(:queue_name)
@@ -32,7 +32,7 @@
new(options).tap do |job|
Delayed::Worker.lifecycle.run_callbacks(:enqueue, job) do
job.hook(:enqueue)
- Delayed::Worker.delay_jobs ? job.save : job.invoke_job
+ Delayed::Worker.delay_job?(job) ? job.save : job.invoke_job
end
end
end
@@ -63,6 +63,12 @@
end
end
+ attr_reader :error
+ def error=(error)
+ @error = error
+ self.last_error = "#{error.message}\n#{error.backtrace.join("\n")}" if
self.respond_to?(:last_error=)
+ end
+
def failed?
!!failed_at
end
@@ -93,7 +99,7 @@
hook :before
payload_object.perform
hook :success
- rescue => e
+ rescue Exception => e # rubocop:disable RescueException
hook :error, e
raise e
ensure
@@ -139,6 +145,12 @@
end
end
+ def destroy_failed_jobs?
+ payload_object.respond_to?(:destroy_failed_jobs?) ?
payload_object.destroy_failed_jobs? : Delayed::Worker.destroy_failed_jobs
+ rescue DeserializationError
+ Delayed::Worker.destroy_failed_jobs
+ end
+
def fail!
update_attributes(:failed_at => self.class.db_time_now)
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/backend/shared_spec.rb
new/lib/delayed/backend/shared_spec.rb
--- old/lib/delayed/backend/shared_spec.rb 2014-12-22 14:22:31.000000000
+0100
+++ new/lib/delayed/backend/shared_spec.rb 2015-09-24 23:40:11.000000000
+0200
@@ -161,7 +161,7 @@
job = described_class.enqueue(CallbackJob.new)
expect(job.payload_object).to
receive(:perform).and_raise(RuntimeError.new('fail'))
- expect { job.invoke_job }.to raise_error
+ expect { job.invoke_job }.to raise_error(RuntimeError)
expect(CallbackJob.messages).to eq(['enqueue', 'before', 'error:
RuntimeError', 'after'])
end
@@ -450,6 +450,34 @@
end
end
+ describe 'destroy_failed_jobs' do
+ context 'with a SimpleJob' do
+ before(:each) do
+ @job = described_class.enqueue SimpleJob.new
+ end
+
+ it 'is not defined' do
+ expect(@job.destroy_failed_jobs?).to be true
+ end
+
+ it 'uses the destroy failed jobs value on the payload when defined' do
+ expect(@job.payload_object).to
receive(:destroy_failed_jobs?).and_return(false)
+ expect(@job.destroy_failed_jobs?).to be false
+ end
+ end
+
+ context 'with a job that raises DserializationError' do
+ before(:each) do
+ @job = described_class.new :handler => '---
!ruby/struct:GoingToRaiseArgError {}'
+ end
+
+ it 'falls back reasonably' do
+ expect(YAML).to receive(:load_dj).and_raise(ArgumentError)
+ expect(@job.destroy_failed_jobs?).to be true
+ end
+ end
+ end
+
describe 'yaml serialization' do
context 'when serializing jobs' do
it 'raises error ArgumentError for new records' do
@@ -500,6 +528,7 @@
Delayed::Worker.max_run_time = 1.second
job = Delayed::Job.create :payload_object => LongRunningJob.new
worker.run(job)
+ expect(job.error).to_not be_nil
expect(job.reload.last_error).to match(/expired/)
expect(job.reload.last_error).to match(/Delayed::Worker\.max_run_time
is only 1 second/)
expect(job.attempts).to eq(1)
@@ -513,6 +542,7 @@
it 'marks the job as failed' do
Delayed::Worker.destroy_failed_jobs = false
job = described_class.create! :handler => '---
!ruby/object:JobThatDoesNotExist {}'
+ expect_any_instance_of(described_class).to
receive(:destroy_failed_jobs?).and_return(false)
worker.work_off
job.reload
expect(job).to be_failed
@@ -535,6 +565,7 @@
Delayed::Worker.max_attempts = 1
worker.run(@job)
@job.reload
+ expect(@job.error).to_not be_nil
expect(@job.last_error).to match(/did not work/)
expect(@job.attempts).to eq(1)
expect(@job).to be_failed
@@ -617,6 +648,10 @@
end
context 'and we want to destroy jobs' do
+ after do
+ Delayed::Worker.destroy_failed_jobs = true
+ end
+
it_behaves_like 'any failure more than Worker.max_attempts times'
it 'is destroyed if it failed more than Worker.max_attempts times' do
@@ -624,6 +659,13 @@
Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
end
+ it 'is destroyed if the job has destroy failed jobs set' do
+ Delayed::Worker.destroy_failed_jobs = false
+ expect(@job).to receive(:destroy_failed_jobs?).and_return(true)
+ expect(@job).to receive(:destroy)
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
+ end
+
it 'is not destroyed if failed fewer than Worker.max_attempts times' do
expect(@job).not_to receive(:destroy)
(Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
@@ -641,15 +683,35 @@
it_behaves_like 'any failure more than Worker.max_attempts times'
- it 'is failed if it failed more than Worker.max_attempts times' do
- expect(@job.reload).not_to be_failed
- Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
- expect(@job.reload).to be_failed
+ context 'and destroy failed jobs is false' do
+ it 'is failed if it failed more than Worker.max_attempts times' do
+ expect(@job.reload).not_to be_failed
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
+ expect(@job.reload).to be_failed
+ end
+
+ it 'is not failed if it failed fewer than Worker.max_attempts times'
do
+ (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job)
}
+ expect(@job.reload).not_to be_failed
+ end
end
- it 'is not failed if it failed fewer than Worker.max_attempts times' do
- (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job) }
- expect(@job.reload).not_to be_failed
+ context 'and destroy failed jobs for job is false' do
+ before do
+ Delayed::Worker.destroy_failed_jobs = true
+ end
+
+ it 'is failed if it failed more than Worker.max_attempts times' do
+ expect(@job).to receive(:destroy_failed_jobs?).and_return(false)
+ expect(@job.reload).not_to be_failed
+ Delayed::Worker.max_attempts.times { worker.reschedule(@job) }
+ expect(@job.reload).to be_failed
+ end
+
+ it 'is not failed if it failed fewer than Worker.max_attempts times'
do
+ (Delayed::Worker.max_attempts - 1).times { worker.reschedule(@job)
}
+ expect(@job.reload).not_to be_failed
+ end
end
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/command.rb new/lib/delayed/command.rb
--- old/lib/delayed/command.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/command.rb 2015-09-24 23:40:11.000000000 +0200
@@ -6,15 +6,19 @@
end
end
require 'optparse'
+require 'pathname'
module Delayed
class Command # rubocop:disable ClassLength
attr_accessor :worker_count, :worker_pools
+ DIR_PWD = Pathname.new Dir.pwd
+
def initialize(args) # rubocop:disable MethodLength
@options = {
:quiet => true,
- :pid_dir => "#{Rails.root}/tmp/pids"
+ :pid_dir => "#{root}/tmp/pids",
+ :log_dir => "#{root}/log"
}
@worker_count = 1
@@ -37,11 +41,14 @@
@options[:max_priority] = n
end
opt.on('-n', '--number_of_workers=workers', 'Number of unique workers
to spawn') do |worker_count|
- @worker_count = worker_count.to_i rescue 1 # rubocop:disable
RescueModifier
+ @worker_count = worker_count.to_i rescue 1
end
opt.on('--pid-dir=DIR', 'Specifies an alternate directory in which to
store the process ids.') do |dir|
@options[:pid_dir] = dir
end
+ opt.on('--log-dir=DIR', 'Specifies an alternate directory in which to
store the delayed_job log.') do |dir|
+ @options[:log_dir] = dir
+ end
opt.on('-i', '--identifier=n', 'A numeric identifier for the worker.')
do |n|
@options[:identifier] = n
end
@@ -114,18 +121,19 @@
end
def run(worker_name = nil, options = {})
- Dir.chdir(Rails.root)
+ Dir.chdir(root)
Delayed::Worker.after_fork
- Delayed::Worker.logger ||= Logger.new(File.join(Rails.root, 'log',
'delayed_job.log'))
+ Delayed::Worker.logger ||= Logger.new(File.join(@options[:log_dir],
'delayed_job.log'))
worker = Delayed::Worker.new(options)
worker.name_prefix = "#{worker_name} "
worker.start
rescue => e
- Rails.logger.fatal e
STDERR.puts e.message
- exit 1
+ STDERR.puts e.backtrace
+ ::Rails.logger.fatal(e) if rails_logger_defined?
+ exit_with_error_status
end
private
@@ -142,5 +150,21 @@
worker_count = (worker_count || 1).to_i rescue 1
@worker_pools << [queues, worker_count]
end
+
+ def root
+ @root ||= rails_root_defined? ? ::Rails.root : DIR_PWD
+ end
+
+ def rails_root_defined?
+ defined?(::Rails.root)
+ end
+
+ def rails_logger_defined?
+ defined?(::Rails.logger)
+ end
+
+ def exit_with_error_status
+ exit 1
+ end
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/message_sending.rb
new/lib/delayed/message_sending.rb
--- old/lib/delayed/message_sending.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/message_sending.rb 2015-09-24 23:40:11.000000000 +0200
@@ -31,8 +31,10 @@
module ClassMethods
def handle_asynchronously(method, opts = {})
- aliased_method, punctuation = method.to_s.sub(/([?!=])$/, ''), $1 #
rubocop:disable PerlBackrefs
- with_method, without_method =
"#{aliased_method}_with_delay#{punctuation}",
"#{aliased_method}_without_delay#{punctuation}"
+ aliased_method = method.to_s.sub(/([?!=])$/, '')
+ punctuation = $1 # rubocop:disable PerlBackrefs
+ with_method = "#{aliased_method}_with_delay#{punctuation}"
+ without_method = "#{aliased_method}_without_delay#{punctuation}"
define_method(with_method) do |*args|
curr_opts = opts.clone
curr_opts.each_key do |key|
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/recipes.rb new/lib/delayed/recipes.rb
--- old/lib/delayed/recipes.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/recipes.rb 2015-09-24 23:40:11.000000000 +0200
@@ -38,17 +38,17 @@
desc 'Stop the delayed_job process'
task :stop, :roles => lambda { roles } do
- run "cd #{current_path};#{rails_env} #{delayed_job_command} stop #{args}"
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} stop
#{args}"
end
desc 'Start the delayed_job process'
task :start, :roles => lambda { roles } do
- run "cd #{current_path};#{rails_env} #{delayed_job_command} start
#{args}"
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} start
#{args}"
end
desc 'Restart the delayed_job process'
task :restart, :roles => lambda { roles } do
- run "cd #{current_path};#{rails_env} #{delayed_job_command} restart
#{args}"
+ run "cd #{current_path} && #{rails_env} #{delayed_job_command} restart
#{args}"
end
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/syck_ext.rb new/lib/delayed/syck_ext.rb
--- old/lib/delayed/syck_ext.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/syck_ext.rb 2015-09-24 23:40:11.000000000 +0200
@@ -29,7 +29,7 @@
# Constantize the object so that ActiveSupport can attempt
# its auto loading magic. Will raise LoadError if not successful.
name.constantize
- "Struct::#{ name }"
+ "Struct::#{name}"
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/tasks.rb new/lib/delayed/tasks.rb
--- old/lib/delayed/tasks.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/tasks.rb 2015-09-24 23:40:11.000000000 +0200
@@ -19,7 +19,7 @@
:min_priority => ENV['MIN_PRIORITY'],
:max_priority => ENV['MAX_PRIORITY'],
:queues => (ENV['QUEUES'] || ENV['QUEUE'] || '').split(','),
- :quiet => false
+ :quiet => ENV['QUIET']
}
@worker_options[:sleep_delay] = ENV['SLEEP_DELAY'].to_i if
ENV['SLEEP_DELAY']
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/lib/delayed/worker.rb new/lib/delayed/worker.rb
--- old/lib/delayed/worker.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/lib/delayed/worker.rb 2015-09-24 23:40:11.000000000 +0200
@@ -39,6 +39,7 @@
self.delay_jobs = DEFAULT_DELAY_JOBS
self.queues = DEFAULT_QUEUES
self.read_ahead = DEFAULT_READ_AHEAD
+ @lifecycle = nil
end
reset
@@ -97,13 +98,29 @@
end
def self.lifecycle
- @lifecycle ||= Delayed::Lifecycle.new
+ # In case a worker has not been set up, job enqueueing needs a lifecycle.
+ setup_lifecycle unless @lifecycle
+
+ @lifecycle
+ end
+
+ def self.setup_lifecycle
+ @lifecycle = Delayed::Lifecycle.new
+ plugins.each { |klass| klass.new }
end
def self.reload_app?
defined?(ActionDispatch::Reloader) &&
Rails.application.config.cache_classes == false
end
+ def self.delay_job?(job)
+ if delay_jobs.is_a?(Proc)
+ delay_jobs.arity == 1 ? delay_jobs.call(job) : delay_jobs.call
+ else
+ delay_jobs
+ end
+ end
+
def initialize(options = {})
@quiet = options.key?(:quiet) ? options[:quiet] : true
@failed_reserve_count = 0
@@ -112,7 +129,9 @@
self.class.send("#{option}=", options[option]) if options.key?(option)
end
- plugins.each { |klass| klass.new }
+ # Reset lifecycle on the offhand chance that something lazily
+ # triggered its creation before all plugins had been registered.
+ self.class.setup_lifecycle
end
# Every worker has a unique name which by default is the pid of the
process. There are some
@@ -121,7 +140,7 @@
# it crashed before.
def name
return @name unless @name.nil?
- "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue
"#{@name_prefix}pid:#{Process.pid}" # rubocop:disable RescueModifier
+ "#{@name_prefix}host:#{Socket.gethostname} pid:#{Process.pid}" rescue
"#{@name_prefix}pid:#{Process.pid}"
end
# Sets the name of the worker.
@@ -181,7 +200,8 @@
# Do num jobs and return stats on success/failure.
# Exit early if interrupted.
def work_off(num = 100)
- success, failure = 0, 0
+ success = 0
+ failure = 0
num.times do
case reserve_and_run_one_job
@@ -190,7 +210,7 @@
when false
failure += 1
else
- break # leave if no work could be done
+ break # leave if no work could be done
end
break if stop? # leave if we're exiting
end
@@ -200,18 +220,18 @@
def run(job)
job_say job, 'RUNNING'
- runtime = Benchmark.realtime do
+ runtime = Benchmark.realtime do
Timeout.timeout(max_run_time(job).to_i, WorkerTimeout) {
job.invoke_job }
job.destroy
end
job_say job, format('COMPLETED after %.4f', runtime)
- return true # did work
+ return true # did work
rescue DeserializationError => error
- job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}"
+ job.error = error
failed(job)
- rescue => error
+ rescue Exception => error # rubocop:disable RescueException
self.class.lifecycle.run_callbacks(:error, self, job) {
handle_failed_job(job, error) }
- return false # work failed
+ return false # work failed
end
# Reschedule the job in the future (when a job fails).
@@ -236,7 +256,7 @@
say "Error when running failure callback: #{error}", 'error'
say error.backtrace.join("\n"), 'error'
ensure
- self.class.destroy_failed_jobs ? job.destroy : job.fail!
+ job.destroy_failed_jobs? ? job.destroy : job.fail!
end
end
end
@@ -268,7 +288,7 @@
protected
def handle_failed_job(job, error)
- job.last_error = "#{error.message}\n#{error.backtrace.join("\n")}"
+ job.error = error
job_say job, "FAILED (#{job.attempts} prior attempts) with
#{error.class.name}: #{error.message}", 'error'
reschedule(job)
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/metadata new/metadata
--- old/metadata 2014-12-22 14:22:31.000000000 +0100
+++ new/metadata 2015-09-24 23:40:11.000000000 +0200
@@ -1,7 +1,7 @@
--- !ruby/object:Gem::Specification
name: delayed_job
version: !ruby/object:Gem::Version
- version: 4.0.6
+ version: 4.1.1
platform: ruby
authors:
- Brandon Keepers
@@ -15,7 +15,7 @@
autorequire:
bindir: bin
cert_chain: []
-date: 2014-12-22 00:00:00.000000000 Z
+date: 2015-09-24 00:00:00.000000000 Z
dependencies:
- !ruby/object:Gem::Dependency
name: activesupport
@@ -84,6 +84,7 @@
- spec/autoloaded/instance_clazz.rb
- spec/autoloaded/instance_struct.rb
- spec/autoloaded/struct.rb
+- spec/daemons.rb
- spec/delayed/backend/test.rb
- spec/delayed/command_spec.rb
- spec/delayed/serialization/test.rb
@@ -117,7 +118,7 @@
version: '0'
requirements: []
rubyforge_project:
-rubygems_version: 2.4.4
+rubygems_version: 2.4.8
signing_key:
specification_version: 4
summary: Database-backed asynchronous priority queue system -- Extracted from
Shopify
@@ -126,6 +127,7 @@
- spec/autoloaded/instance_clazz.rb
- spec/autoloaded/instance_struct.rb
- spec/autoloaded/struct.rb
+- spec/daemons.rb
- spec/delayed/backend/test.rb
- spec/delayed/command_spec.rb
- spec/delayed/serialization/test.rb
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/autoloaded/instance_struct.rb
new/spec/autoloaded/instance_struct.rb
--- old/spec/autoloaded/instance_struct.rb 2014-12-22 14:22:31.000000000
+0100
+++ new/spec/autoloaded/instance_struct.rb 2015-09-24 23:40:11.000000000
+0200
@@ -1,5 +1,6 @@
module Autoloaded
- class InstanceStruct < ::Struct.new(nil)
+ InstanceStruct = ::Struct.new(nil)
+ class InstanceStruct
def perform
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/autoloaded/struct.rb
new/spec/autoloaded/struct.rb
--- old/spec/autoloaded/struct.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/autoloaded/struct.rb 2015-09-24 23:40:11.000000000 +0200
@@ -1,6 +1,7 @@
# Make sure this file does not get required manually
module Autoloaded
- class Struct < ::Struct.new(nil)
+ Struct = ::Struct.new(nil)
+ class Struct
def perform
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/daemons.rb new/spec/daemons.rb
--- old/spec/daemons.rb 1970-01-01 01:00:00.000000000 +0100
+++ new/spec/daemons.rb 2015-09-24 23:40:11.000000000 +0200
@@ -0,0 +1,2 @@
+# Fake "daemons" file on the spec load path to allow
spec/delayed/command_spec.rb
+# to test the Delayed::Command class without actually adding daemons as a
dependency.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/delayed/command_spec.rb
new/spec/delayed/command_spec.rb
--- old/spec/delayed/command_spec.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/delayed/command_spec.rb 2015-09-24 23:40:11.000000000 +0200
@@ -2,6 +2,129 @@
require 'delayed/command'
describe Delayed::Command do
+ let(:options) { [] }
+ let(:logger) { double('Logger') }
+
+ subject { Delayed::Command.new options }
+
+ before do
+ allow(Delayed::Worker).to receive(:after_fork)
+ allow(Dir).to receive(:chdir)
+ allow(Logger).to receive(:new).and_return(logger)
+ allow_any_instance_of(Delayed::Worker).to receive(:start)
+ allow(Delayed::Worker).to receive(:logger=)
+ allow(Delayed::Worker).to receive(:logger).and_return(nil, logger)
+ end
+
+ shared_examples_for 'uses --log-dir option' do
+ context 'when --log-dir is specified' do
+ let(:options) { ['--log-dir=/custom/log/dir'] }
+
+ it 'creates the delayed_job.log in the specified directory' do
+ expect(Logger).to receive(:new).with('/custom/log/dir/delayed_job.log')
+ subject.run
+ end
+ end
+ end
+
+ describe 'run' do
+ it 'sets the Delayed::Worker logger' do
+ expect(Delayed::Worker).to receive(:logger=).with(logger)
+ subject.run
+ end
+
+ context 'when Rails root is defined' do
+ let(:rails_root) { Pathname.new '/rails/root' }
+ let(:rails) { double('Rails', :root => rails_root) }
+
+ before do
+ stub_const('Rails', rails)
+ end
+
+ it 'runs the Delayed::Worker process in Rails.root' do
+ expect(Dir).to receive(:chdir).with(rails_root)
+ subject.run
+ end
+
+ context 'when --log-dir is not specified' do
+ it 'creates the delayed_job.log in Rails.root/log' do
+ expect(Logger).to
receive(:new).with('/rails/root/log/delayed_job.log')
+ subject.run
+ end
+ end
+
+ include_examples 'uses --log-dir option'
+ end
+
+ context 'when Rails root is not defined' do
+ let(:rails_without_root) { double('Rails') }
+
+ before do
+ stub_const('Rails', rails_without_root)
+ end
+
+ it 'runs the Delayed::Worker process in $PWD' do
+ expect(Dir).to receive(:chdir).with(Delayed::Command::DIR_PWD)
+ subject.run
+ end
+
+ context 'when --log-dir is not specified' do
+ it 'creates the delayed_job.log in $PWD/log' do
+ expect(Logger).to
receive(:new).with("#{Delayed::Command::DIR_PWD}/log/delayed_job.log")
+ subject.run
+ end
+ end
+
+ include_examples 'uses --log-dir option'
+ end
+
+ context 'when an error is raised' do
+ let(:test_error) { Class.new(StandardError) }
+
+ before do
+ allow(Delayed::Worker).to receive(:new).and_raise(test_error.new('An
error'))
+ allow(subject).to receive(:exit_with_error_status)
+ allow(STDERR).to receive(:puts)
+ end
+
+ it 'prints the error message to STDERR' do
+ expect(STDERR).to receive(:puts).with('An error')
+ subject.run
+ end
+
+ it 'exits with an error status' do
+ expect(subject).to receive(:exit_with_error_status)
+ subject.run
+ end
+
+ context 'when Rails logger is not defined' do
+ let(:rails) { double('Rails') }
+
+ before do
+ stub_const('Rails', rails)
+ end
+
+ it 'does not attempt to use the Rails logger' do
+ subject.run
+ end
+ end
+
+ context 'when Rails logger is defined' do
+ let(:rails_logger) { double('Rails logger') }
+ let(:rails) { double('Rails', :logger => rails_logger) }
+
+ before do
+ stub_const('Rails', rails)
+ end
+
+ it 'logs the error to the Rails logger' do
+ expect(rails_logger).to receive(:fatal).with(test_error)
+ subject.run
+ end
+ end
+ end
+ end
+
describe 'parsing --pool argument' do
it 'should parse --pool correctly' do
command = Delayed::Command.new(['--pool=*:1', '--pool=test_queue:4',
'--pool=mailers,misc:2'])
@@ -40,13 +163,13 @@
expect(Dir).to receive(:mkdir).with('./tmp/pids').once
[
- ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> []}],
- ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> ['test_queue']}],
- ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> ['test_queue']}],
- ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> ['test_queue']}],
- ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> ['test_queue']}],
- ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> %w[mailers misc]}],
- ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :queues
=> %w[mailers misc]}]
+ ['delayed_job.0', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => []}],
+ ['delayed_job.1', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => ['test_queue']}],
+ ['delayed_job.2', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => ['test_queue']}],
+ ['delayed_job.3', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => ['test_queue']}],
+ ['delayed_job.4', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => ['test_queue']}],
+ ['delayed_job.5', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => %w[mailers misc]}],
+ ['delayed_job.6', {:quiet => true, :pid_dir => './tmp/pids', :log_dir
=> './log', :queues => %w[mailers misc]}]
].each do |args|
expect(command).to receive(:run_process).with(*args).once
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/message_sending_spec.rb
new/spec/message_sending_spec.rb
--- old/spec/message_sending_spec.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/message_sending_spec.rb 2015-09-24 23:40:11.000000000 +0200
@@ -62,6 +62,7 @@
class FairyTail
attr_accessor :happy_ending
def self.princesses; end
+
def tell
@happy_ending = true
end
@@ -118,5 +119,25 @@
end.not_to change(fairy_tail, :happy_ending)
end.to change { Delayed::Job.count }.by(1)
end
+
+ it 'does delay when delay_jobs is a proc returning true' do
+ Delayed::Worker.delay_jobs = ->(_job) { true }
+ fairy_tail = FairyTail.new
+ expect do
+ expect do
+ fairy_tail.delay.tell
+ end.not_to change(fairy_tail, :happy_ending)
+ end.to change { Delayed::Job.count }.by(1)
+ end
+
+ it 'does not delay the job when delay_jobs is a proc returning false' do
+ Delayed::Worker.delay_jobs = ->(_job) { false }
+ fairy_tail = FairyTail.new
+ expect do
+ expect do
+ fairy_tail.delay.tell
+ end.to change(fairy_tail, :happy_ending).from(nil).to(true)
+ end.not_to change { Delayed::Job.count }
+ end
end
end
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/performable_method_spec.rb
new/spec/performable_method_spec.rb
--- old/spec/performable_method_spec.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/performable_method_spec.rb 2015-09-24 23:40:11.000000000 +0200
@@ -68,7 +68,7 @@
story = Story.create
expect(story).to receive(:error).with(an_instance_of(Delayed::Job),
an_instance_of(RuntimeError))
expect(story).to receive(:tell).and_raise(RuntimeError)
- expect { story.delay.tell.invoke_job }.to raise_error
+ expect { story.delay.tell.invoke_job }.to raise_error(RuntimeError)
end
it 'delegates failure hook to object' do
@@ -98,7 +98,7 @@
story = Story.create
expect(story).to receive(:error).with(an_instance_of(Delayed::Job),
an_instance_of(RuntimeError))
expect(story).to receive(:tell).and_raise(RuntimeError)
- expect { story.delay.tell }.to raise_error
+ expect { story.delay.tell }.to raise_error(RuntimeError)
end
it 'delegates failure hook to object' do
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/sample_jobs.rb new/spec/sample_jobs.rb
--- old/spec/sample_jobs.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/sample_jobs.rb 2015-09-24 23:40:11.000000000 +0200
@@ -1,4 +1,5 @@
-class NamedJob < Struct.new(:perform)
+NamedJob = Struct.new(:perform)
+class NamedJob
def display_name
'named_job'
end
@@ -22,11 +23,12 @@
cattr_accessor :runs
@runs = 0
def perform
- raise 'did not work'
+ raise Exception, 'did not work'
end
end
-class CustomRescheduleJob < Struct.new(:offset)
+CustomRescheduleJob = Struct.new(:offset)
+class CustomRescheduleJob
cattr_accessor :runs
@runs = 0
def perform
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/spec/worker_spec.rb new/spec/worker_spec.rb
--- old/spec/worker_spec.rb 2014-12-22 14:22:31.000000000 +0100
+++ new/spec/worker_spec.rb 2015-09-24 23:40:11.000000000 +0200
@@ -154,4 +154,22 @@
@worker.say(@text, Delayed::Worker.default_log_level)
end
end
+
+ describe 'plugin registration' do
+ it 'does not double-register plugins on worker instantiation' do
+ performances = 0
+ plugin = Class.new(Delayed::Plugin) do
+ callbacks do |lifecycle|
+ lifecycle.before(:enqueue) { performances += 1 }
+ end
+ end
+ Delayed::Worker.plugins << plugin
+
+ Delayed::Worker.new
+ Delayed::Worker.new
+ Delayed::Worker.lifecycle.run_callbacks(:enqueue, nil) {}
+
+ expect(performances).to eq(1)
+ end
+ end
end
