Peter Firmstone wrote:
Patricia Shanahan wrote:
Peter Firmstone wrote:
So it would seem that the runAfter() method is superfluous and
unnecessary in most, if not all cases? Especially if the caller is
making the decision about when tasks are placed on the queue.
We do need to confirm that in our current implementations, the
callers are taking responsibility for the queue placement. Then try
to discovery why we needed to have a dependency check by the
TaskManager.
It sounds like for absolute concurrency the design needs to be
totally revised, perhaps the responsibility of determining if the
Task is ready to be executed should be placed solely on the Task
implementation itself, but not by passing in the list of all Tasks on
the queue, but with a simple:
Do you have a lot of bug reports indicating the concurrency design is
basically broken?
No, not really, don't think it's full of bugs, just not as concurrent as
it should be, I'd like to see it plateau under pressure, rather than
degrade significantly. I'm also a bit puzzled why the caller,
TaskManager and Task all share some responsibility (but not all
responsibility) for managing dependency ordering correctness.
I agree with the "plateau under pressure" objective, and think
degradation should be investigated. That said, there are some things
that may be beyond our control. For example, the current tendency in
operating system memory management is to keep time slicing all runnable
threads even if it means thrashing, rather than picking a subset that
can run, not crawl, and periodically changing the subset.
In a few months, when I have had time to learn my way around both the
specification and the implementation, I would be happy to participate
in a liveness and ordering review. I've never done it for software,
but some of the intellectual tools used for proving memory order
properties in large multiprocessor designs were borrowed from
distributed software engineering.
Sounds like you've got some good experiences to share.
Yes, but please remember I want to function primarily as a Java
programmer. I have spent decades as a project leader, or architect, or
most recently as an academic researcher. Been there, done that.
Programming is the part of computing I enjoy most, I'm retired, and I'm
doing this for fun.
I am opposed to moving the queuing and run after organization work
from TaskManager to its callers.
I understand, if based on the work involved for each implementing
class. To reduce the work substantially, a class implementing Task
(eg:: TaskHelper) could be provided for the developer implementing a
Task to encapsulate, to delegate the dependency management to
internally, rather than extending an abstract class. This reduces the
work back to mostly implementing one method, run(), but leaves the
implementer free to do something else too.
It does have some merit:
* Very small lists and references are fast, with small lock windows
and good concurrency.
* The caller doesn't talk to TaskManager, only Task, coupling is
reduced.
The caller still needs to control which tasks share a set of threads,
and the characteristics of the set of threads, such as the rules for the
number of threads in the set. How would this work if the caller does not
create and talk to a TaskManager or equivalent?
* The caller doesn't have to be responsible for the safety of
placing Tasks on a queue.
Wouldn't the caller would have to be fully responsible, in the sense of
not adding a Task until it can be run in any order relative to all tasks
in the queue?
* Any number of initiators can add dependencies to a Task.
* TaskManager can assume all Tasks received are ready to run.
* The Task implementer can encapsulate a TaskHelper, to delegate
responsibility, for dependencies and correctness.
It is work that has to be done, and the choice is between doing it in
one class, or doing it in each class that needs to both permit some
parallelism and yet maintain some ordering among tasks. It is far less
work to make it both efficient and correct if it only needs to be done
in one class.
But I digress, it's only a suggestion and it might not be the optimum
solution, so feel free to do something else.
I think there is an approach that gives us the immediate benefits of
fixing the obvious performance issues in TaskManager without eliminating
the possibility of moving in the direction you are suggesting if it
seems desirable later.
Patricia
