> On Dec 7, 2022, at 11:56 PM, Jed Brown <[email protected]> wrote:
>
> It isn't always wrong to link threaded BLAS. For example, a user might need
> to call threaded BLAS on the side (but the application can only link one) or
> a sparse direct solver might want threading for the supernode.
Indeed, the user asked specifically for their work flow if configure could,
based on additional configure argument, ensure that they did not get a threaded
BLAS; they did not ask that configure never give a threaded BLAS or even that
it give a non-threaded BLAS by default.
> We could test at runtime whether child threads exist/are created when calling
> BLAS and deliver a warning.
How does one test for this? Some standard Unix API for checking this?
>
> Barry Smith <[email protected]> writes:
>
>> There would need to be, for example, some symbol in all the threaded BLAS
>> libraries that is not in the unthreaded libraries. Of at least in some of
>> the threaded libraries but never in the unthreaded.
>>
>> BlasLapack.py could check for the special symbol(s) to determine.
>>
>> Barry
>>
>>
>>> On Dec 7, 2022, at 4:47 PM, Mark Lohry <[email protected]> wrote:
>>>
>>> Thanks, yes, I figured out the OMP_NUM_THREADS=1 way while triaging it, and
>>> the --download-fblaslapack way occurred to me.
>>>
>>> I was hoping for something that "just worked" (refuse to build in this
>>> case) but I don't know if it's programmatically possible for petsc to tell
>>> whether or not it's linking to a threaded BLAS?
>>>
>>> On Wed, Dec 7, 2022 at 4:35 PM Satish Balay <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>> If you don't specify a blas to use - petsc configure will guess and use
>>>> what it can find.
>>>>
>>>> So only way to force it use a particular blas is to specify one [one way
>>>> is --download-fblaslapack]
>>>>
>>>> Wrt multi-thread openblas - you can force it run single threaded [by one
>>>> of these 2 env variables]
>>>>
>>>> # Use single thread openblas
>>>> export OPENBLAS_NUM_THREADS=1
>>>> export OMP_NUM_THREADS=1
>>>>
>>>> Satish
>>>>
>>>>
>>>> On Wed, 7 Dec 2022, Mark Lohry wrote:
>>>>
>>>>> I ran into an unexpected issue -- on an NP-core machine, each MPI rank of
>>>>> my application was launching NP threads, such that when running with
>>>>> multiple ranks the machine was quickly oversubscribed and performance
>>>>> tanked.
>>>>>
>>>>> The root cause of this was petsc linking against the system-provided
>>>>> library (libopenblas0-pthread in this case) set by the update-alternatives
>>>>> in ubuntu. At some point this machine got updated to using the threaded
>>>>> blas implementation instead of serial; not sure how, and I wouldn't have
>>>>> noticed if I weren't running interactively.
>>>>>
>>>>> Is there any mechanism in petsc or its build system to prevent linking
>>>>> against an inappropriate BLAS, or do I need to be diligent about manually
>>>>> setting the BLAS library in the configuration stage?
>>>>>
>>>>> Thanks,
>>>>> Mark
>>>>>
>>>>
