Benjamin Mahler created MESOS-8587:
--------------------------------------
Summary: Introduce a parallel for each loop (and other parallel
algorithms).
Key: MESOS-8587
URL: https://issues.apache.org/jira/browse/MESOS-8587
Project: Mesos
Issue Type: Improvement
Components: libprocess
Reporter: Benjamin Mahler
Consider the following code:
{code}
SomeProcess::func()
{
foreach (const Item& item, items) {
// Perform some const work on item.
}
}
{code}
When {{items}} becomes very large, this code would benefit from some
parallelism. With a parallel loop construct, we could improve the performance
of this type of code significantly:
{code}
SomeProcess::func()
{
foreach_parallel (items, [=](const Item& item) {
// Perform some const work on item.
});
}
{code}
Ideally, this could enforce const-access to the current Process for safety. An
implementation of this would need to do something like:
# Split the iteration of {{items}} into 1 <= N <= num_worker_threads segments.
# Spawn N-1 additional temporary execution Processes (or re-use from a pool)
# Dispatch to these N-1 additional processes for them to perform their segment
of the iteration.
# Perform the 1st segment on the current Process.
# Have the current Process block to wait for the others to finish. (note need
to avoid deadlocking the worker threads here!)
This generalizes to many other algorithms rather than just iteration. It may be
good to align this with the C++ Parallelism TS, which shows how many of the C++
algorithms have potential for parallel counterparts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
