I was looking into assembling matrices with threads. I have a coloring to avoid conflicts.
Turning off all the logging seems way overkill and for methods that can get called in a thread then we could use PETSC_HAVE_THREADSAFTEY (thingy) to protect logging functions. So one can still get timings for the whole assembly process, just not for MatSetValues. Few people are going to do this. I don't think it will be a time sink, and if it is we just revert back to saying 'turn logging off'. I don't see a good argument for insisting on turning off logging, it is pretty important, if we just say that we are going to protect methods as needed. It is not a big deal, I am just exploring this idea. It is such a basic concept in shared memory sparse linear algebra that it seems like a good thing to be able to support and have in an example to say we can assemble matrices in threads (not that it is a great idea). We have all the tools (eg, coloring methods) that it is just a matter of protecting code a few methods. I use DMPlex MatClosure instead of MatSetValues and this is where I die now with non-thread safe code. We have an idea, from Jed, on how to fix it. Anyway, thanks for your help, but I think we should hold off on doing anything until we have some consensus that this would be a good idea to put some effort into getting a thread safe PETSc that can support OMP matrix assembly with a nice compact example. Thanks again, Mark On Mon, Apr 13, 2020 at 5:44 PM Junchao Zhang <[email protected]> wrote: > Mark, > I saw you had "--with-threadsaftey --with-log=0". Do you really want to > call petsc from multiple threads (in contrast to letting petsc call > other libraries, e.g., BLAS, doing multithreading)? If not, you can > drop --with-threadsaftey. > I have https://gitlab.com/petsc/petsc/-/merge_requests/2714 that should > fix your original compilation errors. > > --Junchao Zhang > > On Mon, Apr 13, 2020 at 2:07 PM Mark Adams <[email protected]> wrote: > >> https://www.mcs.anl.gov/petsc/miscellaneous/petscthreads.html >> >> and I see this on my Mac: >> >> 14:23 1 mark/feature-xgc-interface-rebase *= ~/Codes/petsc$ >> ../arch-macosx-gnu-O-omp.py >> >> >> >> =============================================================================== >> Configuring PETSc to compile on your system >> >> >> =============================================================================== >> =============================================================================== >> >> >> Warning: PETSC_ARCH from environment does not match >> command-line or name of script. >> >> Warning: Using from command-line or >> name of script: arch-macosx-gnu-O-omp, ignoring environment: >> arch-macosx-gnu-g >> >> =============================================================================== >> >> >> TESTING: configureLibraryOptions from >> PETSc.options.libraryOptions(config/PETSc/options/libraryOptions.py:37) >> >> >> >> ******************************************************************************* >> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for >> details): >> >> ------------------------------------------------------------------------------- >> Must use --with-log=0 with --with-threadsafety >> >> ******************************************************************************* >> >> >> On Mon, Apr 13, 2020 at 2:54 PM Junchao Zhang <[email protected]> >> wrote: >> >>> >>> >>> >>> On Mon, Apr 13, 2020 at 12:06 PM Mark Adams <[email protected]> wrote: >>> >>>> BTW, I can build on SUMMIT with logging and OMP, apparently. I also >>>> seem to be able to build with debugging. Both of which are not allowed >>>> according the the docs. I am puzzled. >>>> >>> What are "the docs"? >>> >>>> >>>> On Mon, Apr 13, 2020 at 12:05 PM Mark Adams <[email protected]> wrote: >>>> >>>>> I think the problem is that you have to turn off logging with openmp >>>>> and the (newish) GPU timers did not protect their timers. >>>>> >>>>> I don't see a good reason to require logging be turned off with OMP. >>>>> We could use PETSC_HAVE_THREADSAFETY to protect logs that we care about >>>>> (eg, in MatSetValues) and as users discover more things that they want to >>>>> call in an OMP thread block, then tell them to turn logging off and we >>>>> will >>>>> fix it when we can. >>>>> >>>>> Any thoughts on the idea of letting users keep logging with openmp? >>>>> >>>>> On Mon, Apr 13, 2020 at 11:40 AM Junchao Zhang < >>>>> [email protected]> wrote: >>>>> >>>>>> Yes. Looks we need to include petsclog.h. Don't know why OMP >>>>>> triggered the error. >>>>>> --Junchao Zhang >>>>>> >>>>>> >>>>>> On Mon, Apr 13, 2020 at 9:59 AM Mark Adams <[email protected]> wrote: >>>>>> >>>>>>> Should I do an MR to fix this? >>>>>>> >>>>>>
