Actually solved it. It turns out it was a red herring: the S3 error is actually just a transient issue caused by the DJ worker being on a different host. The real issue is that I had the last_error column of my delayed_jobs table as string rather than text, causing this to die with a PostgresSQL exception because of last_error being too long, and the S3 write is never re-attempted.
Changing the column type to text made things work fine because the S3 error goes away after a few seconds. On Oct 27, 3:40 am, Mark Wang <[email protected]> wrote: > Hi all, > > I'm implementing a simple photo upload test app that uses paperclip > 2.3.5 for the upload, S3 for storage, and delayed_job + ImageMagick > for photo resizing. > > It works fine on my local development machine, but on heroku (I've got > a worker running), the worker dies with this error: > > "last_error" = 'The specified key does not exist. > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/error.rb:38:in > `raise'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/base.rb:72:in > `request'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/base.rb:88:in > `get'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/object.rb: > 134:in `value'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/paperclip-2.3.5/lib/paperclip/storage/ > s3.rb:134:in `to_file'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/paperclip-2.3.5/lib/paperclip/ > attachment.rb:219:in `reprocess!'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/app/models/photo.rb:29:in `regenerate_styles!'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/lib/photo_job.rb:3:in `perform'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:217:in > `invoke_job'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:92:in > `run_with_lock'' > /usr/ruby1.8.7/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:91:in > `run_with_lock'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:155:in > `reserve_and_run_one_job'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:154:in > `each'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:154:in > `reserve_and_run_one_job'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:201:in > `work_off'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:200:in > `times'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/job.rb:200:in > `work_off'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/worker.rb: > 28:in `start'' > /usr/ruby1.8.7/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/worker.rb: > 27:in `start'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/worker.rb: > 24:in `loop'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/lib/delayed/worker.rb: > 24:in `start'' > /disk1/home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5- > b4c7f19aace0/mnt/vendor/plugins/delayed_job/tasks/tasks.rb:13 > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:636:in > `execute'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:631:in > `execute'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:597:in > `invoke_with_call_chain'' > /usr/ruby1.8.7/lib/ruby/1.8/monitor.rb:242:in `synchronize'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:590:in > `invoke_with_call_chain'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in > `invoke_task'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in > `top_level'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in > `top_level'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in > `standard_exception_handling'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in > `top_level'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in > `standard_exception_handling'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'' > /home/slugs/321931_787afbf_9e88-aba30188-0172-47bf-bcf5-b4c7f19aace0/ > mnt/.bundle/gems/ruby/1.8/gems/rake-0.8.7/bin/rake:31 > /usr/ruby1.8.7/bin/rake:19:in `load'' > /usr/ruby1.8.7/bin/rake:19', "run_at" = '2010-10-27 10:31:15.149811', > "updated_at" = '2010-10-27 10:31:10.155015' WHERE > ("delayed_jobs"."id" = 1) > > (See full trace by running task with --trace) > > Browsing the contents of my S3 account, it looks like the original > photo is successfully written to S3 by paperclip running on the web > host, but the worker host dies because of that S3 "key not found" > exception. I've made sure :path to Paperclip is defined, so absolute > filesystem paths that differ from host to host aren't an issue. > > Does anyone know off the top of their heads what else might be causing > it? > > Thanks in advance for any help! > > Mark -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
