Re: FFI: c/c++ struct on stack as an argument or return value
Simon,
I finally managed to implement that for major NCG backends.
Phabricator revision is here: https://phabricator.haskell.org/D252
Here is a link to the review you did before:
https://github.com/Yuras/ghc/commit/7295a4c600bc69129b6800be5b52c3842c9c4e5b
I don't have implementation for mac os x86, ppc and sparc. Are they
actively used today? I don't have access to hardware to test them.
Do you think it has it's own value without exposing to haskell FFI? What
is the minimal feature set I should implement to make it merged?
Thanks,
Yuras
On Sat, 2014-06-14 at 18:57 +0300, Yuras Shumovich wrote:
> Hello,
>
> I implemented support for returning C structures by value in cmm for
> x86_64 (most likely it works only on linux). You can find it here:
> https://github.com/Yuras/ghc/commits/cmm-cstruct
>
> It supports I8, I16, I32, I64, F_, D_ cmm types, and requires special
> annotation. For example:
>
> #include "Cmm.h"
>
> #define MyStruct struct(CInt, I8, struct(I8, CInt))
>
> cmm_test(W_ i)
> {
> CInt i1;
> I8 i2, i3;
> float32 i4;
> (i1, i2, i3, i4) = ccall c_test(W_TO_INT(i)) MyStruct;
> return (TO_W_(i1), TO_W_(i2), TO_W_(i3), i4);
> }
>
> (See "test" directory for full examples.)
>
>
> Do you think it is right approach?
> Could anyone review the code please?
>
> And the last thing, I need mentor for this project. Is anyone interested?
>
> Thanks,
> Yuras
>
> On Tue, 2014-03-18 at 21:30 +, Simon Marlow wrote:
> > So the hard parts are:
> >
> > - the native code generators
> > - native adjustor support (rts/Adjustor.c)
> >
> > Everything else is relatively striaghtforward: we use libffi for
> > adjustors on some platforms and for GHCi, and the LLVM backend should be
> > quite easy too.
> >
> > I would at least take a look at the hard bits and see whether you think
> > it's going to be possible to extend these to handle struct args/returns.
> > Because if not, then the idea is a dead end. Or maybe we will need to
> > limit the scope to make things easier (e.g. only integer and pointer
> > fields).
> >
> > Cheers,
> > Simon
> >
> > On 18/03/2014 17:31, Yuras Shumovich wrote:
> > > Hi,
> > >
> > > I thought I have lost the battle :)
> > > Thank you for the support, Simon!
> > >
> > > I'm interested in full featured solution: arguments, return value,
> > > foreign import, foreign export, etc. But it is too much for me to do it
> > > all at once. So I started with dynamic wrapper.
> > >
> > > The plan is to support structs as arguments and return value for dynamic
> > > wrapper using libffi;
> > > then implement native adjustors at least for x86_64 linux;
> > > then make final design decision (tuple or data? language pragma? union
> > > support? etc);
> > > and only then start working on foreign import.
> > >
> > > But I'm open for suggestions. Just let me know if you think it is better
> > > to start with return value support for foreign import.
> > >
> > > Thanks,
> > > Yuras
> > >
> > > On Tue, 2014-03-18 at 12:19 +, Simon Marlow wrote:
> > >> I'm really keen to have support for returning structs in particular.
> > >> Passing structs less so, because working around the lack of struct
> > >> passing isn't nearly as onerous as working around the lack of struct
> > >> returns. Returning multiple values from a C function is a real pain
> > >> without struct returns: you have to either allocate some memory in
> > >> Haskell or in C, and both methods are needlessly complex and slow.
> > >> (though allocating in Haskell is usually better.) C++ code does this all
> > >> the time, so if you're wrapping C++ code for calling from Haskell, the
> > >> lack of multiple returns bites a lot.
> > >>
> > >> In fact implementing this is on my todo list, I'm really glad to see
> > >> someone else is planning to do it :-)
> > >>
> > >> The vague plan I had in my head was to allow the return value of a
> > >> foreign import to be a tuple containing marshallable types, which would
> > >> map to the appropriate return convention for a struct on the current
> > >> platform. Perhaps allowing it to be an arbitrary single-constructor
> > >> type is better, because it allows us to use a type that has a Storable
> > >> instance.
> > >>
> > >> Cheers,
> > >> Simon
> > >>
> > >
> > >
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
Hello,
just as another data point: I'd also find this functionality useful. I've
run into the problem when trying to bind to C libraries that have functions
with struct parameters being passed by value, or returned (the idea being
that under the hood these structs often end up split up across registers).
In the library I was working with, colors were implemented as RGBA
structs which were passed and returned by value all over the place.
-Iavor
On Sat, Jun 14, 2014 at 8:57 AM, Yuras Shumovich
wrote:
> Hello,
>
> I implemented support for returning C structures by value in cmm for
> x86_64 (most likely it works only on linux). You can find it here:
> https://github.com/Yuras/ghc/commits/cmm-cstruct
>
> It supports I8, I16, I32, I64, F_, D_ cmm types, and requires special
> annotation. For example:
>
> #include "Cmm.h"
>
> #define MyStruct struct(CInt, I8, struct(I8, CInt))
>
> cmm_test(W_ i)
> {
> CInt i1;
> I8 i2, i3;
> float32 i4;
> (i1, i2, i3, i4) = ccall c_test(W_TO_INT(i)) MyStruct;
> return (TO_W_(i1), TO_W_(i2), TO_W_(i3), i4);
> }
>
> (See "test" directory for full examples.)
>
>
> Do you think it is right approach?
> Could anyone review the code please?
>
> And the last thing, I need mentor for this project. Is anyone interested?
>
> Thanks,
> Yuras
>
> On Tue, 2014-03-18 at 21:30 +, Simon Marlow wrote:
> > So the hard parts are:
> >
> > - the native code generators
> > - native adjustor support (rts/Adjustor.c)
> >
> > Everything else is relatively striaghtforward: we use libffi for
> > adjustors on some platforms and for GHCi, and the LLVM backend should be
> > quite easy too.
> >
> > I would at least take a look at the hard bits and see whether you think
> > it's going to be possible to extend these to handle struct args/returns.
> > Because if not, then the idea is a dead end. Or maybe we will need to
> > limit the scope to make things easier (e.g. only integer and pointer
> > fields).
> >
> > Cheers,
> > Simon
> >
> > On 18/03/2014 17:31, Yuras Shumovich wrote:
> > > Hi,
> > >
> > > I thought I have lost the battle :)
> > > Thank you for the support, Simon!
> > >
> > > I'm interested in full featured solution: arguments, return value,
> > > foreign import, foreign export, etc. But it is too much for me to do it
> > > all at once. So I started with dynamic wrapper.
> > >
> > > The plan is to support structs as arguments and return value for
> dynamic
> > > wrapper using libffi;
> > > then implement native adjustors at least for x86_64 linux;
> > > then make final design decision (tuple or data? language pragma? union
> > > support? etc);
> > > and only then start working on foreign import.
> > >
> > > But I'm open for suggestions. Just let me know if you think it is
> better
> > > to start with return value support for foreign import.
> > >
> > > Thanks,
> > > Yuras
> > >
> > > On Tue, 2014-03-18 at 12:19 +, Simon Marlow wrote:
> > >> I'm really keen to have support for returning structs in particular.
> > >> Passing structs less so, because working around the lack of struct
> > >> passing isn't nearly as onerous as working around the lack of struct
> > >> returns. Returning multiple values from a C function is a real pain
> > >> without struct returns: you have to either allocate some memory in
> > >> Haskell or in C, and both methods are needlessly complex and slow.
> > >> (though allocating in Haskell is usually better.) C++ code does this
> all
> > >> the time, so if you're wrapping C++ code for calling from Haskell, the
> > >> lack of multiple returns bites a lot.
> > >>
> > >> In fact implementing this is on my todo list, I'm really glad to see
> > >> someone else is planning to do it :-)
> > >>
> > >> The vague plan I had in my head was to allow the return value of a
> > >> foreign import to be a tuple containing marshallable types, which
> would
> > >> map to the appropriate return convention for a struct on the current
> > >> platform. Perhaps allowing it to be an arbitrary single-constructor
> > >> type is better, because it allows us to use a type that has a Storable
> > >> instance.
> > >>
> > >> Cheers,
> > >> Simon
> > >>
> > >
> > >
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
Hello,
I implemented support for returning C structures by value in cmm for
x86_64 (most likely it works only on linux). You can find it here:
https://github.com/Yuras/ghc/commits/cmm-cstruct
It supports I8, I16, I32, I64, F_, D_ cmm types, and requires special
annotation. For example:
#include "Cmm.h"
#define MyStruct struct(CInt, I8, struct(I8, CInt))
cmm_test(W_ i)
{
CInt i1;
I8 i2, i3;
float32 i4;
(i1, i2, i3, i4) = ccall c_test(W_TO_INT(i)) MyStruct;
return (TO_W_(i1), TO_W_(i2), TO_W_(i3), i4);
}
(See "test" directory for full examples.)
Do you think it is right approach?
Could anyone review the code please?
And the last thing, I need mentor for this project. Is anyone interested?
Thanks,
Yuras
On Tue, 2014-03-18 at 21:30 +, Simon Marlow wrote:
> So the hard parts are:
>
> - the native code generators
> - native adjustor support (rts/Adjustor.c)
>
> Everything else is relatively striaghtforward: we use libffi for
> adjustors on some platforms and for GHCi, and the LLVM backend should be
> quite easy too.
>
> I would at least take a look at the hard bits and see whether you think
> it's going to be possible to extend these to handle struct args/returns.
> Because if not, then the idea is a dead end. Or maybe we will need to
> limit the scope to make things easier (e.g. only integer and pointer
> fields).
>
> Cheers,
> Simon
>
> On 18/03/2014 17:31, Yuras Shumovich wrote:
> > Hi,
> >
> > I thought I have lost the battle :)
> > Thank you for the support, Simon!
> >
> > I'm interested in full featured solution: arguments, return value,
> > foreign import, foreign export, etc. But it is too much for me to do it
> > all at once. So I started with dynamic wrapper.
> >
> > The plan is to support structs as arguments and return value for dynamic
> > wrapper using libffi;
> > then implement native adjustors at least for x86_64 linux;
> > then make final design decision (tuple or data? language pragma? union
> > support? etc);
> > and only then start working on foreign import.
> >
> > But I'm open for suggestions. Just let me know if you think it is better
> > to start with return value support for foreign import.
> >
> > Thanks,
> > Yuras
> >
> > On Tue, 2014-03-18 at 12:19 +, Simon Marlow wrote:
> >> I'm really keen to have support for returning structs in particular.
> >> Passing structs less so, because working around the lack of struct
> >> passing isn't nearly as onerous as working around the lack of struct
> >> returns. Returning multiple values from a C function is a real pain
> >> without struct returns: you have to either allocate some memory in
> >> Haskell or in C, and both methods are needlessly complex and slow.
> >> (though allocating in Haskell is usually better.) C++ code does this all
> >> the time, so if you're wrapping C++ code for calling from Haskell, the
> >> lack of multiple returns bites a lot.
> >>
> >> In fact implementing this is on my todo list, I'm really glad to see
> >> someone else is planning to do it :-)
> >>
> >> The vague plan I had in my head was to allow the return value of a
> >> foreign import to be a tuple containing marshallable types, which would
> >> map to the appropriate return convention for a struct on the current
> >> platform. Perhaps allowing it to be an arbitrary single-constructor
> >> type is better, because it allows us to use a type that has a Storable
> >> instance.
> >>
> >> Cheers,
> >> Simon
> >>
> >
> >
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On Wed, 2014-03-19 at 12:12 +, Simon Marlow wrote: > On 18/03/2014 11:42, Yuras Shumovich wrote: > > On Tue, 2014-03-18 at 12:37 +1100, Manuel M T Chakravarty wrote: > >>> > >>> Library implementation can't generate native dynamic wrapper, it has to > >>> use slow libffi. > >> > >> When we first implemented the FFI, there was no libffi. Maintaining the > >> adjustor code for all platforms is a PITA; hence, using libffi was a > >> welcome way to improve portability. > > > > Do you think we can remove native adjustors? I can prepare a patch. > > I believe native adjustors are sufficiently faster than libffi to want > to keep them; at least that was what I found the last time I checked. > It might be the case that either libffi is now faster, or we could > optimise our libffi support enough that it isn't worth keeping native > adjustors. I'd love it if that happened. > > The benchmark I think I used is in nofib/smp/callback002. callback002 is 10-15% faster with native adjustors then with libffi (x86_64 linux, BuildFlavour = perf, mode=slow). Not a huge difference, but it is noticeable. Native adjustors: 6.53user 0.15system 0:06.69elapsed 99%CPU (0avgtext+0avgdata 2436maxresident)k 0inputs+8outputs (0major+894minor)pagefaults 0swaps <> With "UseLibFFIForAdjustors=YES" in mk/build.mk: 7.18user 0.10system 0:07.29elapsed 99%CPU (0avgtext+0avgdata 2440maxresident)k 0inputs+8outputs (0major+893minor)pagefaults 0swaps <> However, as Sergei Trofimovich mentioned, we use native adjustors only on i386 and x86_64. So rts/AdjustorAsm.S and half of rts/Adjustor.c seems to be a dead code. Unless someone has "UseLibFFIForAdjustors=NO" in his mk/build.mk. Thanks, Yuras > > Cheers, > Simon > > > > It requires minor changes to cache ffi_cif structure. On desugar phase > > for each wrapper we can generate fresh global variable to store cif > > pointer and pass it to createAdjustor. > > > >> > >>> > >>> From my point of view, at this point it is more important to agree on > >>> the next question: do we want such functionality in ghc at all? I don't > >>> want to waste time on it if nobody wants to see it merged. > >> > >> I still don’t see the benefit in further complicating an already murky > >> corner of the compiler. Moreover, for this to make sense, it would need to > >> work on all supported platforms. Unless you are volunteering to implement > >> it on multiple platforms, this would mean, we’d use libffi for most > >> platforms anyway. This brings me to my original point, a library or tool > >> is the better place for this. > > > > OK, I don't buy it, but I see your point. > > > >> > >> Manuel > >> > >> PS: I’d happily accept language-c-inline patches for marshalling structs. > >> > > > > > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://www.haskell.org/mailman/listinfo/ghc-devs > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On 18/03/2014 11:42, Yuras Shumovich wrote: On Tue, 2014-03-18 at 12:37 +1100, Manuel M T Chakravarty wrote: Library implementation can't generate native dynamic wrapper, it has to use slow libffi. When we first implemented the FFI, there was no libffi. Maintaining the adjustor code for all platforms is a PITA; hence, using libffi was a welcome way to improve portability. Do you think we can remove native adjustors? I can prepare a patch. I believe native adjustors are sufficiently faster than libffi to want to keep them; at least that was what I found the last time I checked. It might be the case that either libffi is now faster, or we could optimise our libffi support enough that it isn't worth keeping native adjustors. I'd love it if that happened. The benchmark I think I used is in nofib/smp/callback002. Cheers, Simon It requires minor changes to cache ffi_cif structure. On desugar phase for each wrapper we can generate fresh global variable to store cif pointer and pass it to createAdjustor. From my point of view, at this point it is more important to agree on the next question: do we want such functionality in ghc at all? I don't want to waste time on it if nobody wants to see it merged. I still don’t see the benefit in further complicating an already murky corner of the compiler. Moreover, for this to make sense, it would need to work on all supported platforms. Unless you are volunteering to implement it on multiple platforms, this would mean, we’d use libffi for most platforms anyway. This brings me to my original point, a library or tool is the better place for this. OK, I don't buy it, but I see your point. Manuel PS: I’d happily accept language-c-inline patches for marshalling structs. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On Tue, 18 Mar 2014 20:31:19 +0300 Yuras Shumovich wrote: > Hi, > > I thought I have lost the battle :) > Thank you for the support, Simon! > > I'm interested in full featured solution: arguments, return value, > foreign import, foreign export, etc. But it is too much for me to do it > all at once. So I started with dynamic wrapper. > > The plan is to support structs as arguments and return value for dynamic > wrapper using libffi; > then implement native adjustors at least for x86_64 linux; On a positive side there is only 2 arches supporting native adjustors that actually work these days: ArchHasAdjustorSupport = $(if $(findstring $(TargetArch_CPP),i386 x86_64),YES,NO) > then make final design decision (tuple or data? language pragma? union > support? etc); > and only then start working on foreign import. > > But I'm open for suggestions. Just let me know if you think it is better > to start with return value support for foreign import. > > Thanks, > Yuras -- Sergei signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
So the hard parts are: - the native code generators - native adjustor support (rts/Adjustor.c) Everything else is relatively striaghtforward: we use libffi for adjustors on some platforms and for GHCi, and the LLVM backend should be quite easy too. I would at least take a look at the hard bits and see whether you think it's going to be possible to extend these to handle struct args/returns. Because if not, then the idea is a dead end. Or maybe we will need to limit the scope to make things easier (e.g. only integer and pointer fields). Cheers, Simon On 18/03/2014 17:31, Yuras Shumovich wrote: Hi, I thought I have lost the battle :) Thank you for the support, Simon! I'm interested in full featured solution: arguments, return value, foreign import, foreign export, etc. But it is too much for me to do it all at once. So I started with dynamic wrapper. The plan is to support structs as arguments and return value for dynamic wrapper using libffi; then implement native adjustors at least for x86_64 linux; then make final design decision (tuple or data? language pragma? union support? etc); and only then start working on foreign import. But I'm open for suggestions. Just let me know if you think it is better to start with return value support for foreign import. Thanks, Yuras On Tue, 2014-03-18 at 12:19 +, Simon Marlow wrote: I'm really keen to have support for returning structs in particular. Passing structs less so, because working around the lack of struct passing isn't nearly as onerous as working around the lack of struct returns. Returning multiple values from a C function is a real pain without struct returns: you have to either allocate some memory in Haskell or in C, and both methods are needlessly complex and slow. (though allocating in Haskell is usually better.) C++ code does this all the time, so if you're wrapping C++ code for calling from Haskell, the lack of multiple returns bites a lot. In fact implementing this is on my todo list, I'm really glad to see someone else is planning to do it :-) The vague plan I had in my head was to allow the return value of a foreign import to be a tuple containing marshallable types, which would map to the appropriate return convention for a struct on the current platform. Perhaps allowing it to be an arbitrary single-constructor type is better, because it allows us to use a type that has a Storable instance. Cheers, Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
Hi, I thought I have lost the battle :) Thank you for the support, Simon! I'm interested in full featured solution: arguments, return value, foreign import, foreign export, etc. But it is too much for me to do it all at once. So I started with dynamic wrapper. The plan is to support structs as arguments and return value for dynamic wrapper using libffi; then implement native adjustors at least for x86_64 linux; then make final design decision (tuple or data? language pragma? union support? etc); and only then start working on foreign import. But I'm open for suggestions. Just let me know if you think it is better to start with return value support for foreign import. Thanks, Yuras On Tue, 2014-03-18 at 12:19 +, Simon Marlow wrote: > I'm really keen to have support for returning structs in particular. > Passing structs less so, because working around the lack of struct > passing isn't nearly as onerous as working around the lack of struct > returns. Returning multiple values from a C function is a real pain > without struct returns: you have to either allocate some memory in > Haskell or in C, and both methods are needlessly complex and slow. > (though allocating in Haskell is usually better.) C++ code does this all > the time, so if you're wrapping C++ code for calling from Haskell, the > lack of multiple returns bites a lot. > > In fact implementing this is on my todo list, I'm really glad to see > someone else is planning to do it :-) > > The vague plan I had in my head was to allow the return value of a > foreign import to be a tuple containing marshallable types, which would > map to the appropriate return convention for a struct on the current > platform. Perhaps allowing it to be an arbitrary single-constructor > type is better, because it allows us to use a type that has a Storable > instance. > > Cheers, > Simon > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On 18/03/2014 01:37, Manuel M T Chakravarty wrote: Yuras Shumovich : I think the compiler is the right place. It is impossible to have efficient implementation in a library. For dynamic wrapper (foreign import "wrapper" stuff) ghc generates piece of executable code at runtime. There are native implementations for a number of platforms, and libffi is used as a fall back for other platforms (see rts/Adjustor.c). AFAIK it is done that way because libffi is slower then native implementation. Library implementation can't generate native dynamic wrapper, it has to use slow libffi. When we first implemented the FFI, there was no libffi. Maintaining the adjustor code for all platforms is a PITA; hence, using libffi was a welcome way to improve portability. Making the adjustor code more complicated by adding more functionality doesn’t sound like a good plan to me. Besides, there are other overheads in addition to the actual marshalling in FFI calls and most of the time we are calling out to library functions for which the FFI call overhead is only a small portion of the runtime. > i think the crux of Manuel's point is mainly that any good proposal has to at least give a roadmap to support on all the various platforms etc etc I don't think you are expecting detailed schedule from me. Passing structure by value is possible on all platforms ghc supports, and it can be implemented for any particular platform if somebody is interested. From my point of view, at this point it is more important to agree on the next question: do we want such functionality in ghc at all? I don't want to waste time on it if nobody wants to see it merged. I'm really keen to have support for returning structs in particular. Passing structs less so, because working around the lack of struct passing isn't nearly as onerous as working around the lack of struct returns. Returning multiple values from a C function is a real pain without struct returns: you have to either allocate some memory in Haskell or in C, and both methods are needlessly complex and slow. (though allocating in Haskell is usually better.) C++ code does this all the time, so if you're wrapping C++ code for calling from Haskell, the lack of multiple returns bites a lot. In fact implementing this is on my todo list, I'm really glad to see someone else is planning to do it :-) The vague plan I had in my head was to allow the return value of a foreign import to be a tuple containing marshallable types, which would map to the appropriate return convention for a struct on the current platform. Perhaps allowing it to be an arbitrary single-constructor type is better, because it allows us to use a type that has a Storable instance. Cheers, Simon I still don’t see the benefit in further complicating an already murky corner of the compiler. Moreover, for this to make sense, it would need to work on all supported platforms. Unless you are volunteering to implement it on multiple platforms, this would mean, we’d use libffi for most platforms anyway. This brings me to my original point, a library or tool is the better place for this. Manuel PS: I’d happily accept language-c-inline patches for marshalling structs. On Sat, 2014-03-15 at 00:37 -0400, Carter Schonwald wrote: I'm not opposing that, in fact, theres a GHC ticket discussing some stuff related to this (related to complex numbers). i think the crux of Manuel's point is mainly that any good proposal has to at least give a roadmap to support on all the various platforms etc etc On Sat, Mar 15, 2014 at 12:33 AM, Edward Kmett wrote: I don't care enough to fight and try to win the battle, but I just want to point out that Storable structs are far more brittle and platform dependent than borrowing the already correct platform logic for struct passing from libffi. I do think the existing FFI extension made the right call under the 32 bit ABIs that were in use at the time it was defined. That said, with 64-bit ABIs saying that 2 32-bit ints should be passed in a single 64 bit register, you wind up with large chunks of third party APIs we just can't call out to directly any more, requiring many one-off manual C shims. -Edward On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald < carter.schonw...@gmail.com> wrote: indeed, its very very easy to do storable instances that correspond to the struct type you want, the ``with`` function in http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils actually gets you most of the way there! On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < c...@cse.unsw.edu.au> wrote: Yuras, I’m not convinced that the compiler is the right place for this kind of functionality. In fact, when we designed the Haskell FFI, we explicit decided against what you propose. There are a few reasons for this. Firstly, compilers are complex beasts, and secondly, it takes a long time until a change in the
Re: FFI: c/c++ struct on stack as an argument or return value
On Tue, 2014-03-18 at 12:37 +1100, Manuel M T Chakravarty wrote: > > > > Library implementation can't generate native dynamic wrapper, it has to > > use slow libffi. > > When we first implemented the FFI, there was no libffi. Maintaining the > adjustor code for all platforms is a PITA; hence, using libffi was a welcome > way to improve portability. Do you think we can remove native adjustors? I can prepare a patch. It requires minor changes to cache ffi_cif structure. On desugar phase for each wrapper we can generate fresh global variable to store cif pointer and pass it to createAdjustor. > > > > > From my point of view, at this point it is more important to agree on > > the next question: do we want such functionality in ghc at all? I don't > > want to waste time on it if nobody wants to see it merged. > > I still don’t see the benefit in further complicating an already murky corner > of the compiler. Moreover, for this to make sense, it would need to work on > all supported platforms. Unless you are volunteering to implement it on > multiple platforms, this would mean, we’d use libffi for most platforms > anyway. This brings me to my original point, a library or tool is the better > place for this. OK, I don't buy it, but I see your point. > > Manuel > > PS: I’d happily accept language-c-inline patches for marshalling structs. > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
Yuras Shumovich : > I think the compiler is the right place. It is impossible to have > efficient implementation in a library. > > For dynamic wrapper (foreign import "wrapper" stuff) ghc generates piece > of executable code at runtime. There are native implementations for a > number of platforms, and libffi is used as a fall back for other > platforms (see rts/Adjustor.c). AFAIK it is done that way because libffi > is slower then native implementation. > > Library implementation can't generate native dynamic wrapper, it has to > use slow libffi. When we first implemented the FFI, there was no libffi. Maintaining the adjustor code for all platforms is a PITA; hence, using libffi was a welcome way to improve portability. Making the adjustor code more complicated by adding more functionality doesn’t sound like a good plan to me. Besides, there are other overheads in addition to the actual marshalling in FFI calls and most of the time we are calling out to library functions for which the FFI call overhead is only a small portion of the runtime. >> i think the crux of Manuel's point is mainly that any good proposal >> has to >> at least give a roadmap to support on all the various platforms etc >> etc > > I don't think you are expecting detailed schedule from me. Passing > structure by value is possible on all platforms ghc supports, and it can > be implemented for any particular platform if somebody is interested. > > From my point of view, at this point it is more important to agree on > the next question: do we want such functionality in ghc at all? I don't > want to waste time on it if nobody wants to see it merged. I still don’t see the benefit in further complicating an already murky corner of the compiler. Moreover, for this to make sense, it would need to work on all supported platforms. Unless you are volunteering to implement it on multiple platforms, this would mean, we’d use libffi for most platforms anyway. This brings me to my original point, a library or tool is the better place for this. Manuel PS: I’d happily accept language-c-inline patches for marshalling structs. > On Sat, 2014-03-15 at 00:37 -0400, Carter Schonwald wrote: >> I'm not opposing that, in fact, theres a GHC ticket discussing some stuff >> related to this (related to complex numbers). >> >> i think the crux of Manuel's point is mainly that any good proposal has to >> at least give a roadmap to support on all the various platforms etc etc >> >> >> On Sat, Mar 15, 2014 at 12:33 AM, Edward Kmett wrote: >> >>> I don't care enough to fight and try to win the battle, but I just want to >>> point out that Storable structs are far more brittle and platform dependent >>> than borrowing the already correct platform logic for struct passing from >>> libffi. >>> >>> I do think the existing FFI extension made the right call under the 32 bit >>> ABIs that were in use at the time it was defined. That said, with 64-bit >>> ABIs saying that 2 32-bit ints should be passed in a single 64 bit >>> register, you wind up with large chunks of third party APIs we just can't >>> call out to directly any more, requiring many one-off manual C shims. >>> >>> -Edward >>> >>> >>> >>> >>> On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald < >>> carter.schonw...@gmail.com> wrote: >>> indeed, its very very easy to do storable instances that correspond to the struct type you want, the ``with`` function in http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils actually gets you most of the way there! On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < c...@cse.unsw.edu.au> wrote: > Yuras, > > I’m not convinced that the compiler is the right place for this kind of > functionality. In fact, when we designed the Haskell FFI, we explicit > decided against what you propose. There are a few reasons for this. > > Firstly, compilers are complex beasts, and secondly, it takes a long > time until a change in the compiler goes into production. Hence, as a > general rule, it is advisable to move complexity from the compiler into > libraries as this reduces compiler complexity. Libraries are less complex > and changes can be rolled out much more quickly (it’s essentially a > Hackage > upload versus waiting for the next GHC and Haskell Platform release). > > Thirdly, we have got the Haskell standard for a reason and modifying the > compiler implies a language extension. > > The design goal for the Haskell FFI was to provide the absolute minimum > as part of the language and compiler, and to layer additional conveniences > on top of that in the form of libraries and tools. > > Have you considered the library or tool route? > > Manuel > > Yuras Shumovich : >> Hi, >> >> Right now ghc's FFI doesn't suppo
Re: FFI: c/c++ struct on stack as an argument or return value
Manuel, I think the compiler is the right place. It is impossible to have efficient implementation in a library. For dynamic wrapper (foreign import "wrapper" stuff) ghc generates piece of executable code at runtime. There are native implementations for a number of platforms, and libffi is used as a fall back for other platforms (see rts/Adjustor.c). AFAIK it is done that way because libffi is slower then native implementation. Library implementation can't generate native dynamic wrapper, it has to use slow libffi. > On Sat, 2014-03-15 at 00:17 -0400, Carter Schonwald wrote: > > indeed, its very very easy to do storable instances that correspond to the > > struct type you want, > > > > the ``with`` function in > > http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils > > actually gets you most of the way there! Not sure I understand. `with` can be used to provide C function with pointer to C structure. How can it help when C function requires C structure to be passed by value? > i think the crux of Manuel's point is mainly that any good proposal > has to > at least give a roadmap to support on all the various platforms etc > etc I don't think you are expecting detailed schedule from me. Passing structure by value is possible on all platforms ghc supports, and it can be implemented for any particular platform if somebody is interested. From my point of view, at this point it is more important to agree on the next question: do we want such functionality in ghc at all? I don't want to waste time on it if nobody wants to see it merged. Thanks, Yuras On Sat, 2014-03-15 at 00:37 -0400, Carter Schonwald wrote: > I'm not opposing that, in fact, theres a GHC ticket discussing some stuff > related to this (related to complex numbers). > > i think the crux of Manuel's point is mainly that any good proposal has to > at least give a roadmap to support on all the various platforms etc etc > > > On Sat, Mar 15, 2014 at 12:33 AM, Edward Kmett wrote: > > > I don't care enough to fight and try to win the battle, but I just want to > > point out that Storable structs are far more brittle and platform dependent > > than borrowing the already correct platform logic for struct passing from > > libffi. > > > > I do think the existing FFI extension made the right call under the 32 bit > > ABIs that were in use at the time it was defined. That said, with 64-bit > > ABIs saying that 2 32-bit ints should be passed in a single 64 bit > > register, you wind up with large chunks of third party APIs we just can't > > call out to directly any more, requiring many one-off manual C shims. > > > > -Edward > > > > > > > > > > On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald < > > carter.schonw...@gmail.com> wrote: > > > >> indeed, its very very easy to do storable instances that correspond to > >> the struct type you want, > >> > >> the ``with`` function in > >> http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils > >> actually gets you most of the way there! > >> > >> > >> > >> > >> On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < > >> c...@cse.unsw.edu.au> wrote: > >> > >>> Yuras, > >>> > >>> I’m not convinced that the compiler is the right place for this kind of > >>> functionality. In fact, when we designed the Haskell FFI, we explicit > >>> decided against what you propose. There are a few reasons for this. > >>> > >>> Firstly, compilers are complex beasts, and secondly, it takes a long > >>> time until a change in the compiler goes into production. Hence, as a > >>> general rule, it is advisable to move complexity from the compiler into > >>> libraries as this reduces compiler complexity. Libraries are less complex > >>> and changes can be rolled out much more quickly (it’s essentially a > >>> Hackage > >>> upload versus waiting for the next GHC and Haskell Platform release). > >>> > >>> Thirdly, we have got the Haskell standard for a reason and modifying the > >>> compiler implies a language extension. > >>> > >>> The design goal for the Haskell FFI was to provide the absolute minimum > >>> as part of the language and compiler, and to layer additional conveniences > >>> on top of that in the form of libraries and tools. > >>> > >>> Have you considered the library or tool route? > >>> > >>> Manuel > >>> > >>> Yuras Shumovich : > >>> > Hi, > >>> > > >>> > Right now ghc's FFI doesn't support c/c++ structures. > >>> > > >>> > Whenever we have foreign function that accepts or returns struct by > >>> > value, we have to create wrapper that accepts or returns pointer to > >>> > struct. It is inconvenient, but actually not a big deal. > >>> > > >>> > But there is no easy workaround when you want to export haskell > >>> function > >>> > to use it with c/c++ API that requires structures to be passed by value > >>> > (Usually it is a callback in c/c++ API. You can't change it's > >>> signature, > >>> > and if it doesn'
Re: FFI: c/c++ struct on stack as an argument or return value
I'm not opposing that, in fact, theres a GHC ticket discussing some stuff related to this (related to complex numbers). i think the crux of Manuel's point is mainly that any good proposal has to at least give a roadmap to support on all the various platforms etc etc On Sat, Mar 15, 2014 at 12:33 AM, Edward Kmett wrote: > I don't care enough to fight and try to win the battle, but I just want to > point out that Storable structs are far more brittle and platform dependent > than borrowing the already correct platform logic for struct passing from > libffi. > > I do think the existing FFI extension made the right call under the 32 bit > ABIs that were in use at the time it was defined. That said, with 64-bit > ABIs saying that 2 32-bit ints should be passed in a single 64 bit > register, you wind up with large chunks of third party APIs we just can't > call out to directly any more, requiring many one-off manual C shims. > > -Edward > > > > > On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald < > carter.schonw...@gmail.com> wrote: > >> indeed, its very very easy to do storable instances that correspond to >> the struct type you want, >> >> the ``with`` function in >> http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils >> actually gets you most of the way there! >> >> >> >> >> On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < >> c...@cse.unsw.edu.au> wrote: >> >>> Yuras, >>> >>> I’m not convinced that the compiler is the right place for this kind of >>> functionality. In fact, when we designed the Haskell FFI, we explicit >>> decided against what you propose. There are a few reasons for this. >>> >>> Firstly, compilers are complex beasts, and secondly, it takes a long >>> time until a change in the compiler goes into production. Hence, as a >>> general rule, it is advisable to move complexity from the compiler into >>> libraries as this reduces compiler complexity. Libraries are less complex >>> and changes can be rolled out much more quickly (it’s essentially a Hackage >>> upload versus waiting for the next GHC and Haskell Platform release). >>> >>> Thirdly, we have got the Haskell standard for a reason and modifying the >>> compiler implies a language extension. >>> >>> The design goal for the Haskell FFI was to provide the absolute minimum >>> as part of the language and compiler, and to layer additional conveniences >>> on top of that in the form of libraries and tools. >>> >>> Have you considered the library or tool route? >>> >>> Manuel >>> >>> Yuras Shumovich : >>> > Hi, >>> > >>> > Right now ghc's FFI doesn't support c/c++ structures. >>> > >>> > Whenever we have foreign function that accepts or returns struct by >>> > value, we have to create wrapper that accepts or returns pointer to >>> > struct. It is inconvenient, but actually not a big deal. >>> > >>> > But there is no easy workaround when you want to export haskell >>> function >>> > to use it with c/c++ API that requires structures to be passed by value >>> > (Usually it is a callback in c/c++ API. You can't change it's >>> signature, >>> > and if it doesn't provide some kind of "void* userdata", then you are >>> > stuck.) >>> > >>> > I'm interested in fixing that. I'm going to start with 'foreign import >>> > "wrapper" ...' stuff. >>> > >>> > Calling conventions for passing c/c++ structures by value are pretty >>> > tricky and platform/compiler specific. So initially I'll use libffi for >>> > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see >>> > rts/Adjustor.c). It will allow me to explore design space without >>> > bothering about low level implementation details. Later it could be >>> > implemented for native (non-libffi) adjustors. >>> > >>> > Is anybody interested it that? I appreciate any comments/ideas. >>> > >>> > Right now I don't have clear design. It would be nice to support plain >>> > haskell data types that are 1) not recursive, 2) has one constructor >>> and >>> > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. >>> Any >>> > ideas are welcome. >>> > >>> > An example how to use libffi with structures: >>> > http://www.atmark-techno.com/~yashi/libffi.html#Structures >>> > >>> > Thanks, >>> > Yuras >>> > >>> > >>> > ___ >>> > ghc-devs mailing list >>> > ghc-devs@haskell.org >>> > http://www.haskell.org/mailman/listinfo/ghc-devs >>> >>> ___ >>> ghc-devs mailing list >>> ghc-devs@haskell.org >>> http://www.haskell.org/mailman/listinfo/ghc-devs >>> >> >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://www.haskell.org/mailman/listinfo/ghc-devs >> >> > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
I don't care enough to fight and try to win the battle, but I just want to point out that Storable structs are far more brittle and platform dependent than borrowing the already correct platform logic for struct passing from libffi. I do think the existing FFI extension made the right call under the 32 bit ABIs that were in use at the time it was defined. That said, with 64-bit ABIs saying that 2 32-bit ints should be passed in a single 64 bit register, you wind up with large chunks of third party APIs we just can't call out to directly any more, requiring many one-off manual C shims. -Edward On Sat, Mar 15, 2014 at 12:17 AM, Carter Schonwald < carter.schonw...@gmail.com> wrote: > indeed, its very very easy to do storable instances that correspond to the > struct type you want, > > the ``with`` function in > http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils > actually gets you most of the way there! > > > > > On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < > c...@cse.unsw.edu.au> wrote: > >> Yuras, >> >> I’m not convinced that the compiler is the right place for this kind of >> functionality. In fact, when we designed the Haskell FFI, we explicit >> decided against what you propose. There are a few reasons for this. >> >> Firstly, compilers are complex beasts, and secondly, it takes a long time >> until a change in the compiler goes into production. Hence, as a general >> rule, it is advisable to move complexity from the compiler into libraries >> as this reduces compiler complexity. Libraries are less complex and changes >> can be rolled out much more quickly (it’s essentially a Hackage upload >> versus waiting for the next GHC and Haskell Platform release). >> >> Thirdly, we have got the Haskell standard for a reason and modifying the >> compiler implies a language extension. >> >> The design goal for the Haskell FFI was to provide the absolute minimum >> as part of the language and compiler, and to layer additional conveniences >> on top of that in the form of libraries and tools. >> >> Have you considered the library or tool route? >> >> Manuel >> >> Yuras Shumovich : >> > Hi, >> > >> > Right now ghc's FFI doesn't support c/c++ structures. >> > >> > Whenever we have foreign function that accepts or returns struct by >> > value, we have to create wrapper that accepts or returns pointer to >> > struct. It is inconvenient, but actually not a big deal. >> > >> > But there is no easy workaround when you want to export haskell function >> > to use it with c/c++ API that requires structures to be passed by value >> > (Usually it is a callback in c/c++ API. You can't change it's signature, >> > and if it doesn't provide some kind of "void* userdata", then you are >> > stuck.) >> > >> > I'm interested in fixing that. I'm going to start with 'foreign import >> > "wrapper" ...' stuff. >> > >> > Calling conventions for passing c/c++ structures by value are pretty >> > tricky and platform/compiler specific. So initially I'll use libffi for >> > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see >> > rts/Adjustor.c). It will allow me to explore design space without >> > bothering about low level implementation details. Later it could be >> > implemented for native (non-libffi) adjustors. >> > >> > Is anybody interested it that? I appreciate any comments/ideas. >> > >> > Right now I don't have clear design. It would be nice to support plain >> > haskell data types that are 1) not recursive, 2) has one constructor and >> > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any >> > ideas are welcome. >> > >> > An example how to use libffi with structures: >> > http://www.atmark-techno.com/~yashi/libffi.html#Structures >> > >> > Thanks, >> > Yuras >> > >> > >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org >> > http://www.haskell.org/mailman/listinfo/ghc-devs >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://www.haskell.org/mailman/listinfo/ghc-devs >> > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
indeed, its very very easy to do storable instances that correspond to the struct type you want, the ``with`` function in http://hackage.haskell.org/package/base-4.6.0.1/docs/Foreign-Marshal-Utils.htmlforeigh.marshal.utils actually gets you most of the way there! On Sat, Mar 15, 2014 at 12:00 AM, Manuel M T Chakravarty < c...@cse.unsw.edu.au> wrote: > Yuras, > > I’m not convinced that the compiler is the right place for this kind of > functionality. In fact, when we designed the Haskell FFI, we explicit > decided against what you propose. There are a few reasons for this. > > Firstly, compilers are complex beasts, and secondly, it takes a long time > until a change in the compiler goes into production. Hence, as a general > rule, it is advisable to move complexity from the compiler into libraries > as this reduces compiler complexity. Libraries are less complex and changes > can be rolled out much more quickly (it’s essentially a Hackage upload > versus waiting for the next GHC and Haskell Platform release). > > Thirdly, we have got the Haskell standard for a reason and modifying the > compiler implies a language extension. > > The design goal for the Haskell FFI was to provide the absolute minimum as > part of the language and compiler, and to layer additional conveniences on > top of that in the form of libraries and tools. > > Have you considered the library or tool route? > > Manuel > > Yuras Shumovich : > > Hi, > > > > Right now ghc's FFI doesn't support c/c++ structures. > > > > Whenever we have foreign function that accepts or returns struct by > > value, we have to create wrapper that accepts or returns pointer to > > struct. It is inconvenient, but actually not a big deal. > > > > But there is no easy workaround when you want to export haskell function > > to use it with c/c++ API that requires structures to be passed by value > > (Usually it is a callback in c/c++ API. You can't change it's signature, > > and if it doesn't provide some kind of "void* userdata", then you are > > stuck.) > > > > I'm interested in fixing that. I'm going to start with 'foreign import > > "wrapper" ...' stuff. > > > > Calling conventions for passing c/c++ structures by value are pretty > > tricky and platform/compiler specific. So initially I'll use libffi for > > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see > > rts/Adjustor.c). It will allow me to explore design space without > > bothering about low level implementation details. Later it could be > > implemented for native (non-libffi) adjustors. > > > > Is anybody interested it that? I appreciate any comments/ideas. > > > > Right now I don't have clear design. It would be nice to support plain > > haskell data types that are 1) not recursive, 2) has one constructor and > > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any > > ideas are welcome. > > > > An example how to use libffi with structures: > > http://www.atmark-techno.com/~yashi/libffi.html#Structures > > > > Thanks, > > Yuras > > > > > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://www.haskell.org/mailman/listinfo/ghc-devs > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
Yuras, I’m not convinced that the compiler is the right place for this kind of functionality. In fact, when we designed the Haskell FFI, we explicit decided against what you propose. There are a few reasons for this. Firstly, compilers are complex beasts, and secondly, it takes a long time until a change in the compiler goes into production. Hence, as a general rule, it is advisable to move complexity from the compiler into libraries as this reduces compiler complexity. Libraries are less complex and changes can be rolled out much more quickly (it’s essentially a Hackage upload versus waiting for the next GHC and Haskell Platform release). Thirdly, we have got the Haskell standard for a reason and modifying the compiler implies a language extension. The design goal for the Haskell FFI was to provide the absolute minimum as part of the language and compiler, and to layer additional conveniences on top of that in the form of libraries and tools. Have you considered the library or tool route? Manuel Yuras Shumovich : > Hi, > > Right now ghc's FFI doesn't support c/c++ structures. > > Whenever we have foreign function that accepts or returns struct by > value, we have to create wrapper that accepts or returns pointer to > struct. It is inconvenient, but actually not a big deal. > > But there is no easy workaround when you want to export haskell function > to use it with c/c++ API that requires structures to be passed by value > (Usually it is a callback in c/c++ API. You can't change it's signature, > and if it doesn't provide some kind of "void* userdata", then you are > stuck.) > > I'm interested in fixing that. I'm going to start with 'foreign import > "wrapper" ...' stuff. > > Calling conventions for passing c/c++ structures by value are pretty > tricky and platform/compiler specific. So initially I'll use libffi for > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see > rts/Adjustor.c). It will allow me to explore design space without > bothering about low level implementation details. Later it could be > implemented for native (non-libffi) adjustors. > > Is anybody interested it that? I appreciate any comments/ideas. > > Right now I don't have clear design. It would be nice to support plain > haskell data types that are 1) not recursive, 2) has one constructor and > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any > ideas are welcome. > > An example how to use libffi with structures: > http://www.atmark-techno.com/~yashi/libffi.html#Structures > > Thanks, > Yuras > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On Fri, Mar 14, 2014 at 2:00 PM, Yuras Shumovich wrote:
> On Fri, 2014-03-14 at 09:08 -0400, Edward Kmett wrote:
> > I spent some time hacking around on this from a library perspective when
> I
> > had to interoperate with a bunch of Objective C on a 64-bit mac as many
> of
> > the core library functions you need to FFI out to pass around pairs of
> Int32s
> > as a struct small enough by the x64 ABI to get shoehorned into one
> > register, and as I was programmatically cloning Objective C APIs via
> > template haskell I couldn't use the usual clunky C shims.
>
> Was it related to language-c-inline package?
It was an exercise in serial yak shaving brought about by writing a
realtime GPU-based Metropolis light transport raytracer... er.. nevermind.
=)
> So if nothing else, you can at least take this as a vote of confidence
> that
> > your idea isn't crazy. =)
> >
> > I'd also be happy to answer questions if you get stuck or need help.
>
> Thank you, Edward
>
> Since there is at least one person how is interested in, I'll start
> asking questions. Please let me know when I become too noisy :)
>
> For now I'm focused on desugaring phase. Right now
>
> type Fn = CInt -> CInt -> IO ()
> foreign import ccall "wrapper" f :: Fn -> IO (FunPtr Fn)
>
> is desugared into
>
> f :: Fn -> IO (FunPtr Fn)
> f hsFunc = do
> sPtr <- newStablePtr hsFunc
> createAdjustor sPtr staticWrapper ...
>
> Here staticWrapper -- address of C function. It will dereference the
> sPtr, cast to StgClosure* and call with appropriate arguments. All the
> arguments are primitive C types (int, char, pointer, etc), so it is easy
> to convert them to corresponding haskell types via rts_mkInt, rts_mkChar
> etc.
>
> But I want to allow argument to be C structs.
>
> data CStruct {
> i :: CInt,
> j :: CInt
> }
> type Fn = CStruct -> IO ()
> foreign import ccall "wrapper" f :: Fn -> IO (FunPtr Fn)
>
> Looks like it is impossible to instantiate CStruct from C function. Is
> it true? Is it easy to add such functionality?
>
> The only solution I see is to flatten CStruct before creating StablePtr:
>
> f :: Fn -> IO (FunPtr Fn)
> f hsFunc = do
> sPtr <- newStablePtr $ \i j -> hsFunc (CStruct i j)
> createAdjustor sPtr staticWrapper ...
>
> Does it make sense? It will add performance overhead because of
> additional indirection. Better ideas are welcome.
>
Not sure. This is a much lower level (and more correct .. and likely
faster) approach than I was taking.
I'd just built all my functions in a way that would cache the resulting
ffi_prep_cif for each signature using typeclass magic. I had to do some
allocations on each ffi_call though as well for struct avalues though, so
I'm guessing you'd have to do at least that much.
> Thanks,
> Yuras
>
> >
> > -Edward
> >
> >
> > On Fri, Mar 14, 2014 at 7:50 AM, Yuras Shumovich >wrote:
> >
> > >
> > > Hi,
> > >
> > > Right now ghc's FFI doesn't support c/c++ structures.
> > >
> > > Whenever we have foreign function that accepts or returns struct by
> > > value, we have to create wrapper that accepts or returns pointer to
> > > struct. It is inconvenient, but actually not a big deal.
> > >
> > > But there is no easy workaround when you want to export haskell
> function
> > > to use it with c/c++ API that requires structures to be passed by value
> > > (Usually it is a callback in c/c++ API. You can't change it's
> signature,
> > > and if it doesn't provide some kind of "void* userdata", then you are
> > > stuck.)
> > >
> > > I'm interested in fixing that. I'm going to start with 'foreign import
> > > "wrapper" ...' stuff.
> > >
> > > Calling conventions for passing c/c++ structures by value are pretty
> > > tricky and platform/compiler specific. So initially I'll use libffi for
> > > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see
> > > rts/Adjustor.c). It will allow me to explore design space without
> > > bothering about low level implementation details. Later it could be
> > > implemented for native (non-libffi) adjustors.
> > >
> > > Is anybody interested it that? I appreciate any comments/ideas.
> > >
> > > Right now I don't have clear design. It would be nice to support plain
> > > haskell data types that are 1) not recursive, 2) has one constructor
> and
> > > 3) contains only c/c++ types. But it doesn't work with c/c++ unions.
> Any
> > > ideas are welcome.
> > >
> > > An example how to use libffi with structures:
> > > http://www.atmark-techno.com/~yashi/libffi.html#Structures
> > >
> > > Thanks,
> > > Yuras
> > >
> > >
> > > ___
> > > ghc-devs mailing list
> > > ghc-devs@haskell.org
> > > http://www.haskell.org/mailman/listinfo/ghc-devs
> > >
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
On Fri, 2014-03-14 at 09:08 -0400, Edward Kmett wrote:
> I spent some time hacking around on this from a library perspective when I
> had to interoperate with a bunch of Objective C on a 64-bit mac as many of
> the core library functions you need to FFI out to pass around pairs of Int32s
> as a struct small enough by the x64 ABI to get shoehorned into one
> register, and as I was programmatically cloning Objective C APIs via
> template haskell I couldn't use the usual clunky C shims.
Was it related to language-c-inline package?
> So if nothing else, you can at least take this as a vote of confidence that
> your idea isn't crazy. =)
>
> I'd also be happy to answer questions if you get stuck or need help.
Thank you, Edward
Since there is at least one person how is interested in, I'll start
asking questions. Please let me know when I become too noisy :)
For now I'm focused on desugaring phase. Right now
type Fn = CInt -> CInt -> IO ()
foreign import ccall "wrapper" f :: Fn -> IO (FunPtr Fn)
is desugared into
f :: Fn -> IO (FunPtr Fn)
f hsFunc = do
sPtr <- newStablePtr hsFunc
createAdjustor sPtr staticWrapper ...
Here staticWrapper -- address of C function. It will dereference the
sPtr, cast to StgClosure* and call with appropriate arguments. All the
arguments are primitive C types (int, char, pointer, etc), so it is easy
to convert them to corresponding haskell types via rts_mkInt, rts_mkChar
etc.
But I want to allow argument to be C structs.
data CStruct {
i :: CInt,
j :: CInt
}
type Fn = CStruct -> IO ()
foreign import ccall "wrapper" f :: Fn -> IO (FunPtr Fn)
Looks like it is impossible to instantiate CStruct from C function. Is
it true? Is it easy to add such functionality?
The only solution I see is to flatten CStruct before creating StablePtr:
f :: Fn -> IO (FunPtr Fn)
f hsFunc = do
sPtr <- newStablePtr $ \i j -> hsFunc (CStruct i j)
createAdjustor sPtr staticWrapper ...
Does it make sense? It will add performance overhead because of
additional indirection. Better ideas are welcome.
Thanks,
Yuras
>
> -Edward
>
>
> On Fri, Mar 14, 2014 at 7:50 AM, Yuras Shumovich wrote:
>
> >
> > Hi,
> >
> > Right now ghc's FFI doesn't support c/c++ structures.
> >
> > Whenever we have foreign function that accepts or returns struct by
> > value, we have to create wrapper that accepts or returns pointer to
> > struct. It is inconvenient, but actually not a big deal.
> >
> > But there is no easy workaround when you want to export haskell function
> > to use it with c/c++ API that requires structures to be passed by value
> > (Usually it is a callback in c/c++ API. You can't change it's signature,
> > and if it doesn't provide some kind of "void* userdata", then you are
> > stuck.)
> >
> > I'm interested in fixing that. I'm going to start with 'foreign import
> > "wrapper" ...' stuff.
> >
> > Calling conventions for passing c/c++ structures by value are pretty
> > tricky and platform/compiler specific. So initially I'll use libffi for
> > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see
> > rts/Adjustor.c). It will allow me to explore design space without
> > bothering about low level implementation details. Later it could be
> > implemented for native (non-libffi) adjustors.
> >
> > Is anybody interested it that? I appreciate any comments/ideas.
> >
> > Right now I don't have clear design. It would be nice to support plain
> > haskell data types that are 1) not recursive, 2) has one constructor and
> > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any
> > ideas are welcome.
> >
> > An example how to use libffi with structures:
> > http://www.atmark-techno.com/~yashi/libffi.html#Structures
> >
> > Thanks,
> > Yuras
> >
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://www.haskell.org/mailman/listinfo/ghc-devs
> >
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: FFI: c/c++ struct on stack as an argument or return value
I spent some time hacking around on this from a library perspective when I had to interoperate with a bunch of Objective C on a 64-bit mac as many of the core library functions you need to FFI out to pass around pairs of Int32s as a struct small enough by the x64 ABI to get shoehorned into one register, and as I was programmatically cloning Objective C APIs via template haskell I couldn't use the usual clunky C shims. What I was doing was just using libffi with a lot of work to cache the results of ffi_prep_cif for each signature. It worked reasonably well for my purposes, but my need for it vanished and I abandoned the code in the middle of refactoring it for grander things. So if nothing else, you can at least take this as a vote of confidence that your idea isn't crazy. =) I'd also be happy to answer questions if you get stuck or need help. -Edward On Fri, Mar 14, 2014 at 7:50 AM, Yuras Shumovich wrote: > > Hi, > > Right now ghc's FFI doesn't support c/c++ structures. > > Whenever we have foreign function that accepts or returns struct by > value, we have to create wrapper that accepts or returns pointer to > struct. It is inconvenient, but actually not a big deal. > > But there is no easy workaround when you want to export haskell function > to use it with c/c++ API that requires structures to be passed by value > (Usually it is a callback in c/c++ API. You can't change it's signature, > and if it doesn't provide some kind of "void* userdata", then you are > stuck.) > > I'm interested in fixing that. I'm going to start with 'foreign import > "wrapper" ...' stuff. > > Calling conventions for passing c/c++ structures by value are pretty > tricky and platform/compiler specific. So initially I'll use libffi for > that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see > rts/Adjustor.c). It will allow me to explore design space without > bothering about low level implementation details. Later it could be > implemented for native (non-libffi) adjustors. > > Is anybody interested it that? I appreciate any comments/ideas. > > Right now I don't have clear design. It would be nice to support plain > haskell data types that are 1) not recursive, 2) has one constructor and > 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any > ideas are welcome. > > An example how to use libffi with structures: > http://www.atmark-techno.com/~yashi/libffi.html#Structures > > Thanks, > Yuras > > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://www.haskell.org/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
FFI: c/c++ struct on stack as an argument or return value
Hi, Right now ghc's FFI doesn't support c/c++ structures. Whenever we have foreign function that accepts or returns struct by value, we have to create wrapper that accepts or returns pointer to struct. It is inconvenient, but actually not a big deal. But there is no easy workaround when you want to export haskell function to use it with c/c++ API that requires structures to be passed by value (Usually it is a callback in c/c++ API. You can't change it's signature, and if it doesn't provide some kind of "void* userdata", then you are stuck.) I'm interested in fixing that. I'm going to start with 'foreign import "wrapper" ...' stuff. Calling conventions for passing c/c++ structures by value are pretty tricky and platform/compiler specific. So initially I'll use libffi for that (it will work when USE_LIBFFI_FOR_ADJUSTORS is defined, see rts/Adjustor.c). It will allow me to explore design space without bothering about low level implementation details. Later it could be implemented for native (non-libffi) adjustors. Is anybody interested it that? I appreciate any comments/ideas. Right now I don't have clear design. It would be nice to support plain haskell data types that are 1) not recursive, 2) has one constructor and 3) contains only c/c++ types. But it doesn't work with c/c++ unions. Any ideas are welcome. An example how to use libffi with structures: http://www.atmark-techno.com/~yashi/libffi.html#Structures Thanks, Yuras ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
