Indeed, experimenting using some combination of AddDllDirectory(), SetDllDirectory() and LoadLibraryEx() was in my plan for next week.
Thanks On 16/03/17 11:59, Mark Bestley wrote: > Yes the calling process can alter where these indirect DLLs are loaded > from and so should be able to be done in UFFI and directly from Smalltalk > > The OS will load the DLL specified then it runs DllMain and doing that > causes the OS to load any DLLs that it calls > > MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says > > With both implicit and explicit linking, Windows first searches for > "known DLLs", such as Kernel32.dll and User32.dll. Windows then searches > for the DLLs in the following sequence: > The directory where the executable module for the current process is > located. > The current directory. > The Windows system directory. The GetSystemDirectory function retrieves > the path of this directory. > The Windows directory. The GetWindowsDirectory function retrieves the > path of this directory. > The directories listed in the PATH environment variable. > > > This can be changed using the call LoadLibraryEx instead of LoadLibrary > Flags can be set to change the directories used > > https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx > > > > Also see AddDllDirectory > https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx > > > Mark > > On 15/03/2017 17:22, Raffaello Giulietti wrote: >> Hi Dimitris, >> >> I don't get your point about proper C coding, C performance or Pharo >> crashes. It is not the focus of this thread. >> >> How dependencies between libraries are resolved at load-time is dictated >> by the OS. If the OS offers a working mechanism to specify folders that >> the OS uses to search for libraries at load-time, well, I think that >> Pharo, hence the UFFI, should offer it as well and in a more fashionable >> format. >> UFFI is quite elegant in many respects but I wouldn't call it a >> minimalist design (zero hand holding, if I understand you correctly). >> But I'm happy the extra elegance is there because it makes life more >> comfortable. >> >> Again, I still have to experiment with SetDllDirectory() in the case of >> Windows, so I'm not in a position to say something authoritative yet. >> >> >> >> >> >> >> >> On 2017-03-15 17:41, Dimitris Chloupis wrote: >>> Please note the moment you use a FFI of any language you step to a C >>> territory. VMs tend not to touch the code because the code is already >>> in machine code format. >>> >>> There is zero reason for UFFI to worry about the dependencies of a >>> library loaded, this must happen at the library level that UFFI does >>> not focus on. >>> >>> What you describe here is standard practice because the vast majority >>> of libraries depend on other libraries. In the end UFFI or any FFI in >>> any language is not there to replace proper C coding but rather to >>> allow you to utilize C libraries. You cannot hope to achieve >>> performance comparable to C if you try to use a language that is not C >>> to do standard things like dependency management. >>> >>> If the library fail to load it wont be because of UFFI it will be >>> because the library is buggy at handling its dependencies. That means >>> that the library will be buggy even if you use it from C directly. >>> >>> Even you do not know what you doing at C level, you going to crash >>> Pharo countless times anyway and the least of your worries will be >>> handling dependencies which far more obvious than some other nasty C >>> bugs. >>> >>> I think its a great idea that UFFI does zero hand holding. This way it >>> gives the clear message that if you do not understand C then you >>> should not be using the UFFI in the first place. >>> >>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano >>> <esteba...@gmail.com >>> <mailto:esteba...@gmail.com>> wrote: >>> >>> >>> > On 15 Mar 2017, at 15:44, Raffaello Giulietti >>> <raffaello.giulie...@lifeware.ch >>> <mailto:raffaello.giulie...@lifeware.ch>> >>> wrote: >>> > >>> > On 2017-03-15 15:20, Esteban Lorenzano wrote: >>> >> >>> >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti >>> <raffaello.giulie...@lifeware.ch >>> <mailto:raffaello.giulie...@lifeware.ch>> >>> wrote: >>> >>> >>> >>> Hi Esteban, >>> >>> >>> >>> I understand this is the current status of the UFFI, so I can >>> certainly use the workarounds discussed below. >>> >>> >>> >>> But I hope UFFI will soon offer a more "declarative" way to >>> specify the search folders, kind of LD_LIBRARY_PATH mechanism but >>> in the UFFI. >>> >> >>> >> that will NOT solve the problem of dependencies of libraries >>> indirectly references. >>> >> There is no way to do that (indirect reference solving) on an >>> executing program that I know. >>> >> So is not “current status of UFFI”: it will never provide that. >>> >> >>> > >>> > Do you mean that SetDllDirectory() has no influence when invoked >>> between Pharo's start and the LoadLibrary() call done by the UFFI? >>> >>> I really have no idea. You can try to add it to your app. >>> >>> Esteban >>> >>> > >>> > >>> > >>> > >>> >> All that, unless someone come and says: "look, Esteban is like >>> this”. >>> >> >>> >> I spent last three weeks (not full time of course) trying to do >>> exactly that for our current linux vm, to force libgit2 to use the >>> version of libssh2 we want (and not the version on system)… and I >>> failed. I solved some cases, but not all. So at the end I opted to >>> change zeroconf to define LD_LIBRARY_PATH before calling pharo. >>> Only way I found I can force the lookup paths. >>> >> >>> >> cheers, >>> >> Esteban >>> >> >>> >>> >>> >>> >>> >>> >>> >>> Greetings >>> >>> Raffaello >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 2017-03-15 14:52, Esteban Lorenzano wrote: >>> >>>> Hi, >>> >>>> >>> >>>> UFFI cannot do what you want. >>> >>>> If I understand well, you have: >>> >>>> >>> >>>> Pharo -> libA.dll -> libB.dll >>> >>>> >>> >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how >>> libA calls libB… and is not possible for us to influence it more >>> than the predefined platform ways. >>> >>>> >>> >>>> One posible workaround (just possible, no idea if it will >>> work) is to perform a load of libB before is required by libB. >>> Then hopefully the loader will understand is same library and will >>> answer same handler (but not sure, because if it discriminates by >>> path… then you’re in the same position). Anyway, if you want to >>> try it, it would be something like this: >>> >>>> >>> >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’. >>> >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’. >>> >>>> >>> >>>> then you can try to execute functions of libA… >>> >>>> >>> >>>> cheers, >>> >>>> Esteban >>> >>>> >>> >>>> >>> >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti >>> <raffaello.giulie...@lifeware.ch >>> <mailto:raffaello.giulie...@lifeware.ch>> >>> wrote: >>> >>>>> >>> >>>>> On 2017-03-15 13:51, Ben Coman wrote: >>> >>>>>> >>> >>>>>> >>> >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti >>> >>>>>> >>> <raffaello.giulie...@lifeware.ch >>> <mailto:raffaello.giulie...@lifeware.ch> >>> >>>>>> >>> <mailto:raffaello.giulie...@lifeware.ch >>> <mailto:raffaello.giulie...@lifeware.ch>>> >>> wrote: >>> >>>>>> >>> >>>>>> Hi Ben, >>> >>>>>> >>> >>>>>> my understanding is that SetDllDirectory only affects the >>> search >>> >>>>>> path of libraries that are loaded at *run-time* with >>> LoadLibrary. >>> >>>>>> >>> >>>>>> What I'm asking for is a mechanism that works at >>> *load-time*, when a >>> >>>>>> library I'm accessing directly depends on another one >>> which I'm not >>> >>>>>> targeting directly. >>> >>>>>> >>> >>>>>> >>> >>>>>> I don't quite follow. I'm not intimate with Windows >>> dynamic loading, so >>> >>>>>> these questions are an opportunity for me to learn... >>> >>>>>> >>> >>>>>> You can't mean when Pharo loads, because it wouldn't know >>> of OTHER.DLL ? >>> >>>>>> >>> >>>>>> Do you mean that LoadLibrary is not called explicitly by >>> YOUR.DLL before >>> >>>>>> its uses the function from OTHER.DLL? >>> >>>>>> >>> >>>>> >>> >>>>> During the build of YOUR.DLL, you can specify that it should >>> depend on OTHER.LIB (kind of binary which lists function headers >>> and static data declarations for OTHER.DLL). Linking resolution is >>> then done at load-time of YOUR.DLL, not by invoking >>> LoadLibrary("OTHER.DLL"). >>> >>>>> >>> >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means >>> of LoadLibrary(), I guess), Windows (not the UFFI) also attempts >>> to load OTHER.DLL because of the linking information gathered >>> during the build. Notice that the linking information does *not* >>> include a specific path for OTHER.DLL. >>> >>>>> >>> >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib >>> because they are included by default. But of course, Windows knows >>> where to find the corresponding .dll.) >>> >>>>> >>> >>>>> OTHER.DLL is searched using the strategy described in the >>> MSDN page you mention. My expectation is/was that UFFI has/had an >>> API for specifying the search paths where to find OTHER.DLL in a >>> platform-independent way. In other words, I would expect the UFFI >>> to do the call of SetDllDirectory() or whatever is needed on >>> Windows for me or to use other mechanisms on other platforms. >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time? >>> >>>>>> But still, YOUR.DLL it not loaded until called via Pharo >>> FFI, so neither >>> >>>>>> is OTHER.DLL, >>> >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL >>> with the >>> >>>>>> same module name >>> >>>>>> is already loaded in memory, the system uses the loaded >>> DLL, no matter >>> >>>>>> which directory it is in. >>> >>>>>> The system does not search for the DLL." >>> >>>>>> >>> >>>>>> So maybe before calling any of YOUR.DLL, >>> >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL) >>> >>>>>> and later when you call into YOUR.DLL, >>> >>>>>> OTHER.DLL is already loaded. >>> >>>>>> >>> >>>>> >>> >>>>> Yes, this might work but is rather contrived. >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>>> Otherwise it would seems(?) your options are either Manifests >>> >>>>>> or Dynamic-Link Library Redirection, both done outside of >>> Pharo. >>> >>>>>> >>> >>>>> >>> >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? >>> The only thing I know is where the .dll and the corresponding .lib >>> + .h are. >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> Thanks for your interest >>> >>>>> Raffaello >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>>> But I'll try whether SetDllDirectory also affects >>> load-time searches. >>> >>>>>> >>> >>>>>> Anyway, I hope future versions of the UFFI to offer >>> setting the >>> >>>>>> search paths more easily. >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> Greetings >>> >>>>>> Raffaello >>> >>>>>> >>> >>>>> >>> >>>>> >>> >>>> >>> >>>> >>> >>> >>> >>> >>> >> >>> >> >>> > >>> > >>> >>> >> > >
