Hi,
just wanted to point that if you are interested in a "framework" for
submit jobs to a thread pool, I can point to
https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.h
https://github.com/uclouvain/openjpeg/blob/master/src/lib/openjp2/thread.c
which is a port in C I've done from the equivalent C++ code of GDAL
(https://github.com/OSGeo/gdal/blob/master/port/cpl_worker_thread_pool.h).
It has a pthread and Win32 implementation. It could be easily extracted
from libopenjp2 (pending a opj_ -> grass_ renaming to avoid conflicts if
both are combined)
The high level API is the opj_thread_pool_* one.
Probably not super fancy, but serves my need well. The user is
responsible for selecting the number of threads and splitting the work
load in jobs that are queued to the thread pool and consumed by the
threads as soon as they are no longer busy.
Even
Le 14/10/2022 à 11:02, Moritz Lennert a écrit :
Am 09.10.2022 20:45 schrieb Brad ReDacted:
Those variables would be...? Is this documented somewhere? If not, can
we get it documented?
https://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly
OpenMP can do far more than just loop "unrolling", these days. Tasking
sections to run concurrent is also quite trivial. It can also offload
to GPU, etc. Check out the v4.5+ spec. It's pretty impressive. I
believe it can do most of what pthreads does, but you certainly lose
control of implementation details. Some compilers have an omp library
while others convert to pthreads.
I do find myself rewriting algorithms so that OpenMP can handle them.
It doesn't seem to handle nested loops with breaks very well and I'm
not entirely sure why.
On 10/9/2022 10:43 AM, William Hargrove wrote:
Can still run GRASS outside the shell by setting all of the
environment variables appropriately ...
OpenMP just works by "unrolling" all of the determinate loops, i.e.,
the ones that iterate a fixed number of times. No speedups to
anything else.
Speedup from OpenMP will be limited, depending on the number of
determinate loops present, and how much of the load they represent.
pthreads are totally flexible, but the programmer has to specify
everything, very carefully ...
But pthreads can speed up lots of stuff outside of determinate loops
...
HTH,
Bill H.
On 10/9/2022 12:37 PM, Brad ReDacted wrote:
Hello,
I'm working on adding parallelism to modules, but debugging is
turning out to be a logistical nightmare:
Why do I not get any reporting from GCC option
'-fsanitize=address|thread"?
I am also having trouble getting the profiler to work properly
inside GRASS (I assume due to shell?). The gmon.out file produced
has no usable data.
OpenMP is extremely poorly supported by most tools. valgrind with
helgrind reports a lot of nonsense. I can't seem to get the Intel
linux tools to work properly, either.
BTW, we are supporting both pthreads and OpenMP. While this isn't
an issue in most cases, there can be races and deadlocks if not
handled properly. Pthreads aren't entirely portable. OpenMP is.
However, pthreads gives us a more control. May I suggest using
OpenMP for most modules and reserve Pthreads to libraries, etc? Or
should we start moving away from pthreads?
Any suggestions would be greatly appreciated!
--
Best Regards,
-Brad
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
