Re: TaskManager as an ExecutorService

2014-01-08 Thread Peter
Thanks Gregg, I appreciate your support, I've been playing around with your suggestion, it's good advise, will share changes shortly. The intent of present efforts is to increase the reliability and predictibility of the codebase while maintaining backward compatibility. However after the code

Re: TaskManager as an ExecutorService

2014-01-08 Thread Gregg Wonderly
From the very early days of Jini, I have not had good luck with ServiceDiscoveryManager. I actually have perhaps 3 different version of this behavior coded into various applications in different ways, because SDM just did not seem to provide sane behavior for me, ever. I am not sure that I hav

Fixing ServiceDiscoveryManager WAS: Re: TaskManager as an ExecutorService

2014-01-06 Thread Peter
I recently updated RetryTask to implement RunnableFuture (it had very similar methods), it also no longer implements Task.runAfter which has been pushed down to subclasses that use it (aka those in ServiceDiscoveryManager) it wouldn't take much effort to add functionality to RetryTask to allow l

Re: TaskManager as an ExecutorService

2014-01-06 Thread Peter
Gregg, Are you able to help out with ServiceDiscoveryManager? Fixing SDM would make a huge stability improvement. Guava has an interface called ListenableFuture, which sounds like a possible candidate, listeners are registered and notified in the event of failure or success. The Guava site cl

Re: TaskManager as an ExecutorService

2014-01-05 Thread Gregg Wonderly
I think the better choice is to not try and handle failure with retry at all. Instead, we should use API which allows the dependent task to know whether it’s dependent has completed or failed. It can then report failure if its dependent failed, or submit its own work to be executed in the que

Re: TaskManager as an ExecutorService

2014-01-04 Thread Peter
For a moment, lets consider how we might fix TaskManager, while retaining the existing Task.runAfter method and how the fix would impact TaskManager's users. TaskManager, like most thread pools has a queue. Before a task is removed from the queue for execution, it is asked if it should "runAfte

Re: TaskManager as an ExecutorService

2014-01-04 Thread Peter
The problem is with TaskManager's public api method Task.runAfter. This is well documented in River-344. The fix requires changing every class that uses it As a thread pool TaskManager is correct provided that no ordering dependencies exist between tasks. TaskManager doesn't compare to Doug L

Re: TaskManager as an ExecutorService

2014-01-04 Thread Greg Trasuk
I’d like you to make a reasonable case for why TaskManager needs to be replaced, requiring changes to many other classes that depend on TaskManager, rather than stating what the problem is with TaskManager and seeking to fix it, which would only affect TaskManager and not require modifying and

Re: TaskManager

2011-07-08 Thread Patricia Shanahan
On 7/7/2011 2:26 PM, Mike Sobolewski wrote: I'm curious about why the load factor option is desirable. Suppose you have 30 tasks waiting, and you have could afford 30 threads for the function, then why not allow 30 threads? The load factor has the effect of limiting the throughput even when you

Re: TaskManager

2011-07-07 Thread Mike Sobolewski
The object that created an instance of TaskManger knows what is in its manger queue. However, tasks are usually coming from outside. They can be recorded initially before appending them to the queue, but later the order can be still updated. For me a simple reordering based on a task priority

Re: TaskManager

2011-07-07 Thread Patricia Shanahan
Generally, tasks in the same TaskManager instance were all created by the same object, and their class is, or could be, an inner class in that object, and therefore they *could* talk to each through methods in that object. I'm not sure the current runAfter mechanism is right for all the uses to w

Re: TaskManager

2011-07-07 Thread Gregg Wonderly
Can we institute a "call back" mechanism to handle this issue in a way that allows the "dependent" code to be run from the "after" task so that the serialization of the work is explicitly specified? Having a failed announcement come back might be appropriate too. It seems like runAfter() does

incubator mail archive - was: Re: TaskManager

2011-07-05 Thread Zsolt Kúti
On Wed, 29 Jun 2011 10:06:02 -0400 "Christopher Dolan" wrote: > P.S. can someone add a link to the Incubator mail archives to the web > page? They are quite hard to find... > http://mail-archives.apache.org/mod_mbox/incubator-river-dev/ > http://mail-archives.apache.org/mod_mbox/incubator-riv

Re: TaskManager

2011-07-05 Thread Patricia Shanahan
The different thread pool implementations allow different constructor parameters. Perhaps a factory pattern? I think it would make run time selection of an alternative implementation easier, but make it harder for a class to specify parameters. On the other hand, I'm not sure how informed and

Re: TaskManager

2011-07-05 Thread Tom Hobbs
Would it be possible t extract the whole thing into an interface? Have the river default implementation something similar to what we have now (at least in behaviour if not implementation) but allow users to swap in their own task execution implementations that may or may not throttle in some way.

Re: TaskManager

2011-07-01 Thread Patricia Shanahan
On 7/1/2011 1:14 PM, Gregg Wonderly wrote: ... If the system throughput is not meeting my needs, I'll apply admission controls, or "add hardware" to manage the issue. Anything else done by river only serves as a "limit" that can be very frustrating to debug and deal with. I would like to make a

Re: TaskManager

2011-07-01 Thread Gregg Wonderly
On 7/1/2011 2:44 AM, Patricia Shanahan wrote: On 6/30/2011 11:38 PM, Dan Creswell wrote: On 1 July 2011 03:22, Gregg Wonderly wrote: On 6/30/2011 9:02 PM, Patricia Shanahan wrote: ... loadLevel permits specification of a ratio between threads and tasks, so that threads are only created up to

Re: TaskManager

2011-07-01 Thread Dan Creswell
Inline On 1 July 2011 08:44, Patricia Shanahan wrote: > On 6/30/2011 11:38 PM, Dan Creswell wrote: >> >> On 1 July 2011 03:22, Gregg Wonderly  wrote: >>> >>> On 6/30/2011 9:02 PM, Patricia Shanahan wrote: > > ... loadLevel permits specification of a ratio between threads and tasks,

Re: TaskManager

2011-07-01 Thread Patricia Shanahan
On 6/30/2011 11:38 PM, Dan Creswell wrote: On 1 July 2011 03:22, Gregg Wonderly wrote: On 6/30/2011 9:02 PM, Patricia Shanahan wrote: ... loadLevel permits specification of a ratio between threads and tasks, so that threads are only created up to e.g. one third of the number of tasks. I'm not

Re: TaskManager

2011-06-30 Thread Dan Creswell
; page? They are quite hard to find... >>> http://mail-archives.apache.org/mod_mbox/incubator-river-dev/ >>> http://mail-archives.apache.org/mod_mbox/incubator-river-user/ >>> http://mail-archives.apache.org/mod_mbox/incubator-river-commits/ >>> >>> &g

Re: TaskManager

2011-06-30 Thread Gregg Wonderly
incubator-river-dev/ http://mail-archives.apache.org/mod_mbox/incubator-river-user/ http://mail-archives.apache.org/mod_mbox/incubator-river-commits/ -Original Message- From: Peter Firmstone [mailto:j...@zeus.net.au] Sent: Sunday, June 26, 2011 5:18 PM To: dev@river.apache.org Subject: R

Re: TaskManager

2011-06-30 Thread Patricia Shanahan
x/incubator-river-commits/ -Original Message- From: Peter Firmstone [mailto:j...@zeus.net.au] Sent: Sunday, June 26, 2011 5:18 PM To: dev@river.apache.org Subject: Re: TaskManager I think the way to look at this is to examine the non-trivial runAfter implementations. If they have common eleme

RE: TaskManager

2011-06-29 Thread Christopher Dolan
g/mod_mbox/incubator-river-user/ http://mail-archives.apache.org/mod_mbox/incubator-river-commits/ -Original Message- From: Peter Firmstone [mailto:j...@zeus.net.au] Sent: Sunday, June 26, 2011 5:18 PM To: dev@river.apache.org Subject: Re: TaskManager > I think the way to look at this

Re: TaskManager

2011-06-26 Thread Peter
The porter mail thread had a number of conversations about threading and resource issues you may find useful. The threading model in River is quite old and will certainly benefit from a good refactoring. +1 Peter. - Original message - > On 6/26/2011 3:18 PM, Peter Firmstone wrote: > >

Re: TaskManager

2011-06-26 Thread Patricia Shanahan
On 6/26/2011 3:18 PM, Peter Firmstone wrote: I think the way to look at this is to examine the non-trivial runAfter implementations. If they have common elements that can be helped by some utility classes, write those classes. If not, deal with each individually. We are now committed to 1.5, s

Re: TaskManager

2011-06-26 Thread Peter Firmstone
I think the way to look at this is to examine the non-trivial runAfter implementations. If they have common elements that can be helped by some utility classes, write those classes. If not, deal with each individually. We are now committed to 1.5, so if we did not have runAfter we could repla

Re: TaskManager

2011-06-26 Thread Patricia Shanahan
On 6/26/2011 1:58 PM, Peter Firmstone wrote: Patricia Shanahan wrote: On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting commen

Re: TaskManager

2011-06-26 Thread Peter Firmstone
Patricia Shanahan wrote: On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting comment from Bob Scheifler: Over the years we've

Re: TaskManager

2011-06-26 Thread Patricia Shanahan
On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting comment from Bob Scheifler: Over the years we've been slowly eliminating use