Re: FFI proposal: allow some control over the scope of C header files
Duncan Coutts: On Fri, 2006-04-21 at 09:32 -0400, Manuel M T Chakravarty wrote: I think we'd want to be able to specify that a C header file not escape a module boundary and probably we'd also want to be able to ask that it not escape a package boundary (though this may be beyond the H' spec since Haskell does not talk about packages). The H98 standard already specifies a NOINLINE pragma for any function: http://haskell.org/onlinereport/pragmas.html The simplest solution is to ensure that all Haskell compilers implement this pragma properly for foreign imported functions. If you want finer control over where inlining takes place, then maybe the pragma should be extended to provide that finer control. I don't think we need to generalise the problem to all function inlinings. There are specific practical problems caused by inlining foreign calls that are not a problem for ordinary Haskell functions. Inlining of foreign functions causes extra problems, but generally inlining is a concern; so, if we can use the same mechanisms, we get a simpler language. Besides, the standard so far doesn't cover command line options at all. So, there is the more general question of whether it should. I don't think we need to specify the command line interface. The required headers can be put in the module. That's ok with me. I was just pointing out that many of the problems and/or lack of understanding of users that we are seeing has to do with the use of command line options. We simply cannot address this unless the standard covers command line options. So some syntax off the top of my head: foreign import cheader module-local foo/bar.h I think there are 3 possibilities for the C header escape/scope setting (which should probably be manditory rather than optional): module-local package-local (extension for compilers that have a notion of a package) global Is this additional complexity really necessary or would the use of NOINLINE pragmas not suffice? It's really in a library context where you want to restrict the inlining of foreign functions, but there the foreign functions are probably not much used inside the library itself, but mainly exported, so I doubt that you would get much of a performance loss by just tagging all foreign imported functions that you don't want to escape as NOINLINE. What I really want is for the issue of header scope to be something that can be checked by the compiler. As a distro packager I see far too many people getting it wrong because they don't understand the issue. If we could declare the intended scope of the header files then 1. people would think about and 2. if they got it wrong it'd be checkable because the compiler would complain. Whether or not the compiler can check for wrong use, seems to me independent of whether we use inline pragmas or any other syntax. GHC could very well check some of these things today. It just doesn't. Do you propose to make such checks mandatory in the standard? As it is at the moment people don't know they're doing anything dodgy until some user of their package gets a mysterious gcc warning and possibly a segfault. If we just tell everyone that they should use NOINLINE then they won't and they'll still get it wrong. The reason for some specific syntax rather than using NOINLINE is that the compiler will be able to track the header files needed by each module. So we can avoid the situation where a call gets made outside the scope of its defining header file - either by automatically #including the header file in the right place, or by complaining if the user does not supply the header (eg by putting it in the .cabal file). So it's not the general issue of inlining but the specific problem of what C header files are required to compile what modules. The ideal situation I imagine is that the scope of the headers can be checked automatically so that the compiler or cabal will complain to a library author that their private header file needs to be marked as local to the package/module or included in the library package file and installed with the package. We are having two issues here: (1) Specification of which functions need what headers and whether these functions can be inlined. (2) Let the compiler spot wrong uses of header files. These two issues are largely independent. Re (1), I dislike new syntax (or generally any additions to the language) and prefer using existing mechanisms as far as possible. The reason is simply that Haskell is already very complicated. Haskell' will be even more complicated. Hence, we must avoid any unnecessary additions. Re (2), I am happy to discuss what kind of checks are possible, but I am worried that it'll be hard to check for everything without assistance from cabal, which I don't think will be part of Haskell'. Re the concern about wrong use: FFI programming is a
Re: FFI proposal: allow some control over the scope of C header files
On Sun, 2006-04-23 at 17:26 -0400, Manuel M T Chakravarty wrote: Duncan Coutts: On Fri, 2006-04-21 at 09:32 -0400, Manuel M T Chakravarty wrote: I think we'd want to be able to specify that a C header file not escape a module boundary and probably we'd also want to be able to ask that it not escape a package boundary (though this may be beyond the H' spec since Haskell does not talk about packages). The H98 standard already specifies a NOINLINE pragma for any function: http://haskell.org/onlinereport/pragmas.html The simplest solution is to ensure that all Haskell compilers implement this pragma properly for foreign imported functions. If you want finer control over where inlining takes place, then maybe the pragma should be extended to provide that finer control. I don't think we need to generalise the problem to all function inlinings. There are specific practical problems caused by inlining foreign calls that are not a problem for ordinary Haskell functions. Inlining of foreign functions causes extra problems, but generally inlining is a concern; so, if we can use the same mechanisms, we get a simpler language. True, though with a special case mechanism we can make automatic checks possible/easier. Besides, the standard so far doesn't cover command line options at all. So, there is the more general question of whether it should. I don't think we need to specify the command line interface. The required headers can be put in the module. That's ok with me. I was just pointing out that many of the problems and/or lack of understanding of users that we are seeing has to do with the use of command line options. We simply cannot address this unless the standard covers command line options. Under my hypothetical scheme the ghc command line method would be equivalent to putting it in the module and could be checked the same way. What I really want is for the issue of header scope to be something that can be checked by the compiler. As a distro packager I see far too many people getting it wrong because they don't understand the issue. If we could declare the intended scope of the header files then 1. people would think about and 2. if they got it wrong it'd be checkable because the compiler would complain. Whether or not the compiler can check for wrong use, seems to me independent of whether we use inline pragmas or any other syntax. GHC could very well check some of these things today. It just doesn't. Do you propose to make such checks mandatory in the standard? That'd be nice, though I can see that it is more work. We are having two issues here: (1) Specification of which functions need what headers and whether these functions can be inlined. (2) Let the compiler spot wrong uses of header files. These two issues are largely independent. Yes, ok. Re (1), I dislike new syntax (or generally any additions to the language) and prefer using existing mechanisms as far as possible. The reason is simply that Haskell is already very complicated. Haskell' will be even more complicated. Hence, we must avoid any unnecessary additions. Sure. Re (2), I am happy to discuss what kind of checks are possible, but I am worried that it'll be hard to check for everything without assistance from cabal, which I don't think will be part of Haskell'. I think it can be checked without cabal. Outline: suppose we use a module level granularity (I know jhc proposes to use a finer granularity) so we track which C header files are needed to compile which modules. A FFI decl specifying a header file makes that module need that header. Then transitively each module that imports that module needs that header too. We can only stop the header leaking out of the module/package by specifying NOINLINE on the imported function (or using some additional syntax s I originally suggested). So now it's easy to check what headers are needed to to compile any module. Then we probably need to rely on an external mechanism (eg cabal or the user) to make sure that all these headers are available - but at least we can check that the user has done it right. So it's at this point that issue (1) (2) become related. If we say that a header file transitively infects every client module then it effectively bans private header files and so we need some mechanism to limit the scope of header files to allow them again (like NOINLINE). Eg with c2hs, I think that in theory we should be installing every .h file that c2hs generates for each module with the library package. I've never seen anyone actually do that (Cabal's c2hs support doesn't do that for example). Re the concern about wrong use: FFI programming is a minefield. We will never be able to make it safe. So, I am reluctant to complicate the language just to make it (maybe) a little safer. What IMHO will be far more effective is a good tutorial on FFI
