I meant to reply to part of what's below, but never got around to it.
On 2/13/2015 6:07 PM, Kyle Brady wrote:
> ...
> This is true, I was viewing the headers as an unfortunate necessity of the
> C backend. We could make it work without them, but the pain probably
> wouldn't be worth it. Actually... thinking about it, it may not be too bad
> - we already need the extern declarations in the Chapel source. What would
> stop us from just putting those as extern declarations in the C code? That
> could possible get us down to a 'require' only being valid for libraries
> (if we drop .c/.o) which helps with the static vs dynamic and file
> extensions.
>
> module foo {
> require "gmp", "dynamic";
> require "hwloc", "static";
> ..
> }
I can see the attractiveness here, but note that this could be difficult
to make work in practice. The Gnu linker has command line syntax that
will allow you to force use of a specific library file (via
-l:FILENAME), but as far as I know other linkers (Darwin) don't have
this ability. And you can't necessarily get around that limitation by
doing things like '... -Bdynamic -lgmp -Bstatic ...', because it can be
hard or impossible to know that to the left of those three options the
linker was in static mode in the first place. If it was actually in
dynamic mode then those three options end up switching the mode to
static. Depending on system setup, command line driver in use, etc. the
linker's default can be either static or dynamic. In an ideal world we'd
have a way to change the mode and then restore the previous mode
(inventing syntax, something like: '... --Bpush-mode --Bdynamic -lgmp
--Bpop-mode ...'), but that doesn't exist.
The most trustworthy and portable method for making sure we link in
specific kinds of library files is to only create the kind we want in
the first place, and then use the vanilla linker -l option to request
them. This is what we're doing now. It isn't perfect, of course, and
we've had issues with it in the past, but I haven't seen anything else
that looks better and is actually doable.
A secondary comment is that I'm not sure we want to be specifying
libraries (shared or dynamic ) at the Chapel source level anyway. That
seems like mixing functionality and mechanism. The functionality is that
we need specific symbols (representing contractually agreed-upon
operations) to be defined. Which library is to provide those symbols and
whether they're brought into the program when it's linked or when it's
executed isn't something that should be decided at compile time.
greg
>> To be clear, though are you counterproposing that the user would be
>> responsible for setting up things like LIBRARY_PATH and
>> CPATH/C_INCLUDE_PATH in their environment to satisfy external dependences
>> that libraries have on specific files?
> Yeah, I see no real solution to that. This case only comes up if you have
> a odd/non-system install of the library anyways. In that case you probably
> have a makefile that has access to locations anyways.
>
> -Kyle
>
> On 2/13/15, 4:02 PM, "Brad Chamberlain" <[email protected]> wrote:
>
>> Hi Kyle --
>>
>> Thanks for the notes.
>>
>>>> Q: Do you prefer overloading 'use' or some other syntax? (must propose
>>>> an alternative, if that's your choice)
>>> I'd prefer to not overload use to mean very different things depending
>>> on
>>> what is in the string. I think it would also prevent us from giving very
>>> helpful error messages at times. What really like to do is place
>>> attributes onto modules or functions indicating what resource it
>>> requires,
>>> but in the meantime:
>>>
>>> module Regexp {
>>> requires {
>>> lib("re2"); // switched based off of --static/--dynamic perhaps? or
>>> just default to dynamic
>>> lib("bonus1", "static");
>>> lib("bonus2", "dynamic");
>>> // These three are all specific to the C back end
>>> include("re2.h", "<stdlib.h>");
>>> // I dislike these two even more as they lock us into the C backend
>>> compile("extra.cpp");
>>> link("extra2.o");
>>> }
>>>
>>> ...
>>> }
>> I think 'requires' is a pretty good keyword in place of where I'm
>> currently overloading 'use' (which I feel so-so about, at best).
>>
>> To my tastes, the rest of it feels unnecessarily verbose. By analogy, I
>> like that the 'chpl' command line permits one to write:
>>
>> chpl foo.chpl bar.h bar.c baz.h baz.o
>>
>> rather than requiring:
>>
>> chpl foo.chpl --include "bar.h baz.h" --compile "bar.c" --link "baz.o"
>>
>> (and, in fact, I've implemented my use-based prototype by calling the
>> same
>> routines that handle the command-line arguments in my current
>> implementation which is what made it so easy). To that end, I'd prefer:
>>
>> requires "bar.h", "bar.c", "baz.h", "baz.o";
>>
>> over the more verbose/explicit form you sketch above. I don't have a
>> good
>> story for the static/dynamic library case, though, apart from writing
>> "libfoo.a", "libbar.so", and/or "-lbaz" (where I agree with you, this
>> latter may either suggest following --static/--dynamic or defaulting to
>> dynamic).
>>
>>
>> W.r.t. your concern about locking into the C back-end... I don't think
>> that's the case. Whatever the back-end is, one could specify that there
>> are additional .c/.o/... files that need to be linked into the binary to
>> make it work. For example, it seems the current LLVM back-end supports
>> additional files like this on the command-line. Similarly, I could
>> imagine supporting the ability to specify Fortran files to implement an
>> interface if we had interoperability with Fortran (I think some have
>> postulated that we probably already do for simple cases...?). I don't
>> think it locks us into any specific Chapel compiler back-end so much as
>> it
>> suggests that we're willing to invoke some compiler/tool on behalf of the
>> user in building their binary.
>>
>> It may be that a given implementation of the Chapel compiler, or given
>> back-end, may not be willing/able to handle a particular type of
>> requirement, but I think that's arguably a fragility in the library
>> implementation itself rather than in the feature that we're discussing
>> here.
>>
>> (Note that, if anything, the ".h" case locks us into C as a back-end
>> language more than the others, since it effectively says "Add a #include
>> for this .h file to the generated Chapel code". I'm realizing that I'm
>> actually not sure what the LLVM back-end does with this case... maybe
>> clangs it? Or maybe just drops it on the floor?)
>>
>> In the short-term, I think it's more important for us to support
>> interoperating with libraries in a way that doesn't require users to
>> specify additional dependences on the command-line rather than to worry
>> overmuch about forward portability of those libraries...
>>
>>
>>
>>> 1. Not a fan of shell variables at all, and I think they have no place
>>> in
>>> being there.
>>>
>>> 2. I don't think -I and -L should be be specified inside of Chapel code.
>>> They represent search paths that can not be known on a users machine. -l
>>> is handled with a lib() directive in the example above.
>>>
>> As the implementer, I'd be happy not to worry about either of these
>> cases.
>> :)
>>
>> To be clear, though are you counterproposing that the user would be
>> responsible for setting up things like LIBRARY_PATH and
>> CPATH/C_INCLUDE_PATH in their environment to satisfy external dependences
>> that libraries have on specific files? (assuming the files were not in
>> the standard system locations?)
>>
>> I was thinking that that would be more onerous than what I was proposing,
>> but now I'm thinking it's not so different. Specifically, the example
>> I've been using as motivation is that, to compile FFTW programs today,
>> these are the extra compiler options that a user has to pass to 'chpl':
>>
>> fftw3.h -I$FFTW_DIR/include -L$FFTW_DIR/lib -lfftw3
>>
>> So I'd been thinking "How can the author of FFTW.chpl make it as trivial
>> as possible for a Chapel user to compile against it?" where my answer was
>> going to be something like:
>>
>> use "ffw3.h", "-I$FFTW_DIR/include" "-L$FFTW_DIR/lib" "-lfftw3";
>>
>> but in retrospect, this still requires the user to set something in their
>> environment (FFTW_DIR rather than LIBRARY_PATH/CPATH), so maybe it's
>> morally no better and yours has the benefit of being simpler to
>> implement.
>> The only slight downside is setting two env. vars. rather than one, in
>> the
>> case that a non-system version of the library is going to be used. But
>> the simplicity seems worth that.
>>
>> In which case, maybe the above simply becomes:
>>
>> requires "fftw3.h", "-lfftw3"
>> or:
>> requires "fftw3.h", "libfftw3.a"
>> or:
>> requires "fftw3.h", "libfftw3.so"
>>
>> depending on where we land on the static/dynamic/library question...
>>
>>
>>> 3. Handled w/ "static" vs "dynamic" in the example. Different platforms
>>> have different extensions (which might not practically matter for us..)
>>> but I still think we should avoid mentioning the exact file extension
>>> for
>>> libraries.
>> Are there cases other than Microsoft's .lib/.dll that you're thinking of?
>>
>> Thanks Kyle,
>> -Brad
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
> _______________________________________________
> Chapel-developers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/chapel-developers
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Chapel-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-developers
