Re: JobManager/JobPoller issues

2019-04-07 Thread Scott Gray
Job prioritization is committed in r1857071, thanks everyone who provided their thoughts. Regards Scott On Sat, 16 Mar 2019, 22:30 Scott Gray, wrote: > Patch available at https://issues.apache.org/jira/browse/OFBIZ-10865 > > Reviews welcome, I probably won't have time to commit it for a few

Re: JobManager/JobPoller issues

2019-03-16 Thread Scott Gray
Patch available at https://issues.apache.org/jira/browse/OFBIZ-10865 Reviews welcome, I probably won't have time to commit it for a few weeks so no rush. By the way I was amazed to notice that jobs are limited to 100 jobs per poll with a 30 second poll time, seems extremely conservative. They

Re: JobManager/JobPoller issues

2019-02-25 Thread Jacques Le Roux
+1 Jacques Le 25/02/2019 à 21:15, Mathieu Lirzin a écrit : Hello Scott, Scott Gray writes: I'm working on implementing the priority queue approach at the moment for a client. All things going well it will be in production in a couple of weeks and I'll report back then with a patch.

Re: JobManager/JobPoller issues

2019-02-25 Thread Mathieu Lirzin
Hello Scott, Scott Gray writes: > I'm working on implementing the priority queue approach at the moment for a > client. All things going well it will be in production in a couple of > weeks and I'll report back then with a patch. Sounds great! -- Mathieu Lirzin GPG: F2A3 8D7E EB2B 6640 5761

Re: JobManager/JobPoller issues

2019-02-25 Thread Scott Gray
Hi Jacques, I'm working on implementing the priority queue approach at the moment for a client. All things going well it will be in production in a couple of weeks and I'll report back then with a patch. Regards Scott On Tue, 26 Feb 2019 at 03:11, Jacques Le Roux wrote: > Hi, > > I put this

Re: JobManager/JobPoller issues

2019-02-25 Thread Jacques Le Roux
Hi, I put this comment there with OFBIZ-10002 trying to document why we have 5 as hardcoded value of /max-threads/ attribute in /thread-pool/ element (serviceengine.xml). At this moment Scott already mentioned[1]: /Honestly I think the topic is generic enough that OFBiz doesn't need to

Re: JobManager/JobPoller issues

2019-02-06 Thread Nicolas Malin
Hello Scott, On a customer project we use massively the job manager with an average of one hundred thousand job per days. We have different cases like, huge long jobs, async persistent job, fast regular job. The mainly problem that we detect has been (as you notified) the long jobs that

Re: JobManager/JobPoller issues

2019-02-04 Thread Scott Gray
Hi Taher, I say that it isn't a drop-in replacement solely because it is unbounded whereas the current implementation appears to depend on the queue being bounded by the number set in the serviceengine.xml thread-pool.jobs attribute. The main concern I have about an unbounded queue is the

Re: JobManager/JobPoller issues

2019-02-04 Thread Jacopo Cappellato
On Wed, Jan 30, 2019 at 8:59 PM Scott Gray wrote: > [...] 2. The advice on the number of threads to use doesn't seem good to me, it > assumes your jobs are CPU bound when in my experience they are more likely > to be I/O bound while making db or external API calls, sending emails etc. > [...]

Re: JobManager/JobPoller issues

2019-02-03 Thread Taher Alkhateeb
Hi Scott, It seems we have some issues currently with our job scheduler [1] which seems to be some sort of memory leak. We are also experiencing some performance issues and other anomalies. It seems like a good time to perhaps revisit the whole thing. Are you suggesting to replace

Re: JobManager/JobPoller issues

2019-01-31 Thread Shi Jinghai
Hi Scott, Perhaps we can use multiple service engines (pools), each pool run a set of specific services, fast or slow. I created an issue "Enable a service to run by a specific service engine"[1] about 2 years ago, not sure whether it works in your case. Kind Regards, Shi Jinghai [1]