Vacuum has been run, (took quite a while). Yeah, I see now that the JobManager actually tries to update all the JobSandbox rows in the transaction, so 60 seconds was pretty low.
I am trying 10 minutes now and see how that goes. I am using postgress by the way. Thanks for the help, I really appreciate it. -- Josh. On Wed, Jul 13, 2011 at 8:29 PM, Scott Gray <scott.g...@hotwaxmedia.com> wrote: > Not sure what db you're using but it probably wouldn't hurt to run a vacuum > on the table to speed up processing. > > By the way, I'm pretty sure the default timeout is 60 seconds so you might > want to try something a little larger :-) > > Regards > Scott > > On 14/07/2011, at 2:58 PM, Josh Jacobson wrote: > >> I tried 60 seconds for timeout but that didn't work. I guess Ill >> double it now and keep trying. >> >> I have about 260,000 pending jobs, and nothing is getting done. >> >> I know what you mean about purgeOldjobs. That service is crashed now >> and I deleted old jobs from the database by hand. I was up to 2.6 >> million rows. Ofbiz was pretty much unusable. >> >> If you have any other suggestions I'd love Yo hear them. >> >> On Wednesday, July 13, 2011, Scott Gray <scott.g...@hotwaxmedia.com> wrote: >>> Ah okay, that is entirely dependent on the number of jobs and the speed the >>> server can process them. As a side note I would keep a close eye on the >>> purgeOldJobs service, when it starts falling over (transaction timeout >>> again) then the number of rows in the table will increase quickly which in >>> turn will slow down polling. >>> >>> In general the whole persisted jobs implementation is a bit fragile, >>> especially when dealing with a large number of jobs. I've wanted to >>> replace it with something like quartz for a while but haven't had the time. >>> >>> Regards >>> Scott >>> >>> On 14/07/2011, at 2:10 PM, Josh Jacobson wrote: >>> >>>> Thanks again. I actually meant a suggestion for the transaction >>>> timeout. In any case I am grateful for your explanation. >>>> >>>> >>>> On Wednesday, July 13, 2011, Scott Gray <scott.g...@hotwaxmedia.com> wrote: >>>>> As best I can tell there shouldn't be any need to increase the interval >>>>> between polls since the interval timer doesn't actually start until the >>>>> previous poll has completed (see JobPoller.run()) so I can't see how a >>>>> small interval would cause any backlog problems. >>>>> >>>>> I'm guessing if there is any lock contention then it's probably caused by >>>>> the executing jobs trying to update their respective rows while the >>>>> poller is holding a table lock. So from that point of view I guess >>>>> increasing the interval could reduce the amount of contention between the >>>>> executing jobs and the next poll. >>>>> >>>>> Regards >>>>> Scott >>>>> >>>>> On 14/07/2011, at 1:02 PM, Josh Jacobson wrote: >>>>> >>>>>> Scott, >>>>>> >>>>>> Thanks! That is very precise advise. Do you have a suggestion on >>>>>> interval time? 60 seconds? 120? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> On Wed, Jul 13, 2011 at 5:34 PM, Scott Gray <scott.g...@hotwaxmedia.com> >>>>>> wrote: >>>>>>> That configuration is for the frequency of job polls. There isn't any >>>>>>> ability to specify the transaction timeout via configuration so you'll >>>>>>> need to modify the code directly: >>>>>>> JobManager.java (line 148): >>>>>>> beganTransaction = TransactionUtil.begin(); >>>>>>> needs to be changed to use TransactionUtil.begin(int) >>>>>>> >>>>>>> Regards >>>>>>> Scott >>>>>>> >>>>>>> HotWax Media >>>>>>> http://www.hotwaxmedia.com >>>>>>> >>>>>>> On 14/07/2011, at 12:23 PM, Josh Jacobson wrote: >>>>>>> >>>>>>>> Brett, >>>>>>>> >>>>>>>> Before I start trying to run the jobs manually, I want to give your >>>>>>>> suggestion a try. I think I know where to configure the job polling >>>>>>>> transaction time (I believe it's the poll-db-millis="20000" value on >>>>>>>> the framework/service/config/serviceengine.xml. >>>>>>>> >>>>>>>> However, I still don't know what to increase it to. I understand that >>>>>>>> we wouldn't want to make it bigger than the default polling interval. >>>>>>>> Do you know what the default interval between polling is? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> On Wed, Jul 13, 2011 at 12:31 PM, Brett Palmer >>>>>>>> <brettgpal...@gmail.com> wrote: >>>>>>>>> I meant removing finished jobs. If you have thousands of pending >>>>>>>>> jobs then >>>>>>>>> you will have the same problem I mentioned in my first email. One >>>>>>>>> resolution will be to increase the job poller transaction time. In >>>>>>>>> the >>>>>>>>> ofbiz version I was using there was not a way to configure the poller >>>>>>>>> transaction time. It just used the default time. I had to create a >>>>>>>>> patch >>>>>>>>> to allow this to happen. >>>>>>>>> >>>>>>>>> In the patch you had to be careful to not increase the transaction >>>>>>>>> time >>>>>>>>> greater than the frequency of the job poller. Otherwise you get into >>>>>>>>> a lock >>>>>>>>> situation where one job poller is still running within a transaction >>>>>>>>> and >>>>>>>>> another poller starts. This didn't create a huge problem but the >>>>>>>>> second job >>>>>>>>> poller would usually lock and then time out. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Brett >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jul 13, 2011 at 1:15 PM, Josh Jacobson >>>>>>>>> <josh.s.jacob...@gmail.com>wrote: >>>>>>>>> >>>>>>>>>> Brett, >>>>>>>>>> >>>>>>>>>> > >
