Thanks for all your links. I'm learning a lot.
> You may find Tamas' experiences with various Lisp implementations useful:
>
> http://tkpapp.blogspot.com/2010/05/upgraded-array-element-types-and-pinned.html
This is indeed interesting, but the idea of keeping track of all these
implementations is very daunting.
That's CFFI's job ;)
> Some Lisp implementations have an alternative to pinning arrays, which
> is instead to allocate the array in memory that the GC promises never
> to move ("static arrays"). The following library wraps that
> functionality:
>
> http://gitorious.org/iolib/static-vectors/
That is especially interesting; I wish that I had know about it. After looking
at SBCL's source,
It appears that SBCL does real pinning, not just suspending GC (at least for
x86, the only
platform where you worry about threads).
> My sense is that this is the right way to go: it's safer (no calling
> malloc() and free()), and it's more scalable in parallel (no need to
> interrupt or pause the GC).
Here's my argument why copying to malloc'ed memory is the most sensible
compromise,
after messing with this issue for a few years (and passing Lisp arrays
directly):
* it uses the hard work that the authors of CFFI have done to understand
various Lisp
implementations. CFFI has a broad user base and is here to stay. Things
like
gitorious.org/iolib/static-vectors/ and FFA are impressive, but they depend
on one or two
dedicated people digging through many compilers' source code and often
inferring
unpublished internals.
* It's what the Lisp compiler writers expect you to do, by providing an FFI
and malloc.
* In an age of 64 bit machines, insufficient address space for malloc should
be much less
of a problem. I just allocated and freed 1 billion bytes on my laptop in
both CCL and SBCL.
* The penalty of copying isn't too bad. Copying is O(N), and the things you
do in foreign memory are usually
O(N^a) or AxO(N) for A>>1. For instance, matrix inversion is N^(3/2) for
an N element array.
The exception might be some sort of very simple IO with no processing, but
why is that being done in FFI-land?
* If a implementation is slow to copy into foreign memory, then it will be slow
for everything else, and the copying will be the least of one's worries.
-John
_______________________________________________
cffi-devel mailing list
cffi-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/cffi-devel
