On 28 July 2010 23:04, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
> On Jul 28, 2010, at 8:40 PM, Lisandro Dalcin wrote:
>
>> On 28 July 2010 21:33, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>
>>> On Jul 28, 2010, at 6:03 PM, Lisandro Dalcin wrote:
>>>
>>>> Could any of you tell me exactly the intention of
>>>> "--with-dynamic-loading" in the context of PETSc?
>>>>
>>>> I tend to think that dynamic loading of user-defined stuff should
>>>> always be supported if dlopen() or equivalent calls are available. For
>>>> example, I'm able to build Python from sources with static libraries,
>>>> but I can still use Python .so extension modules loaded via dlopen().
>>>> So usage of dynamic loading is somewhat orthogonal to static/shared
>>>> libraries.
>>>>
>>>> If PETSc-builtin Vec/Mat/KSP/etc subtypes implementations could be
>>>> built on separate .so files intended to be loaded at runtime,
>>>
>>> ? ?This is "in theory" possible but requires changes to the "buildsystem"
>>> that compiles code and sticks it into libraries.
>>>
>>
>> I understand. This is somewhat hard to make it robust across platforms.
>>
>>>> then
>>>> --with-dynamic-loading would ask for such kind of build. And then a
>>>> normal build would be just like now (let say, single petsc.so library
>>>> with everything bundled). However, regardless of the way PETSc would
>>>> be built, users should still be able to dlopen() modules and register
>>>> custom subtypes at runtime, even if PETSc was built with static
>>>> libraries!!
>>>
>>> ? We should be able to support this with a small number of changes.
>>> Currently the flag PETSC_USE_DYNAMIC_LIBRARIES is used for two purposes:
>>>
>>> 1) that KSPRegisterDynamic() eliminates the function pointer so PETSc
>>> registers the library and string name and not directly the function
>>> pointer. Then the functions are accessed via sym()
>>>
>>> 2) the function registration stuff that keeps the registered function
>>> pointers and libraries (if PETSC_USE_DYNAMIC_LIBRARIES is off then managing
>>> the library access is turned off.
>
> ? We actually already have a PETSC_HAVE_DYNAMIC_LIBRARIES so I have changed
> PetscFListFind() etc to always build the dynamic library access parts when
> dynamic libraries work even if PETSc is not built with dynamic libraries.
> ?Now you can dynamically load types with static PETSc (let me know if you
> have problems).
>
>
>>>
>>
>> I never fully understood the rationale of the current implementation.
>> IMHO, PETSc should NEVER dlopen()&dlsym() itself, and the function
>> pointer should NEVER be eliminated. Every time you lookup for a
>> XXXCreate_YYY routine, then you first try to find it in the DLL
>> library list (by default empty, unless users append to it). If the
>> search success, it means the user wanted to override the PETSc
>> implementation. If the search fails (remember I'm proposing
>> libpetsc.so not being in the library list), then just use the stored
>> function pointer.
>
> ?The rationale is that we have intended for well over a decade to separate
> the "interface" part of PETSc from the implementation. This means all the
> particular instances would go into a different (or several different) dynamic
> libraries from libpetsc.[so/a] We did all the logic for dynamic loading etc
> but never separate the two parts of PETSc into independent parts.
>
Ah! OK. I understand now.
>
>>
>>> ?If we introduce PETSC_HAVE_DLOPEN_OR_WINDOWS_VERSION that we use in the
>>> underlying code then I think we will get the effect you want.
>>>
>>
>> Yes, but I still think that we should review the usages of
>> XXXRegister()/XXXRegisterDynamic() and the somewhat nonsensical
>> auto-dlopen of libpetsc.so
>>
>>
>> Ah! Let me add something. Regarding dynamic loading, we could do more
>> or less the same that Python does for loading extension modules.
>> Instead of dlsym() onto libs for every XXXCreate_YYY routine, we
>> should instead lookup right after dlopen() lib<name>.so for a
>> PetscInitialize_<name>() and call it (anyway, PETSc currently does
>> something like this). That entry-point routine would be in charge of
>> calling any XXXRegister() for user-provided subtypes of Vec/Mat/KSP,
>> etc.
>
> ? We want the flexibility of not requiring the user to actually register
> their function, just stick the function into the library and then use it.
> -ksp_type libraryname.so:mycreatefunction this is why we also look for the
> raw symbol.
>
Yep, you are right, that's very convenient.
>
>> The only thing that I do not like about this is coupling the file
>> name lib<name>.so to the PetscInitialize_<name>() symbol. Anyway, we
>> could implement this more or less independently o all the current
>> stuff, think about a PetscImportModule("modname"), then we are free to
>> decide the filename convention <prefix?><name><suffix?>.so and the
>> naming convention for the PetscInitialize_<name>() entry point...
>>
> ?I cannot remember why I had the function name depend on the library name but
> I know I had a good reason for it.
>
PETSc dlopen()'s libraries with RTLD_GLOBAL. Then the entry points
should have different names to avoid collisions in the global symbol
namespace.
--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
