Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Aug 8, 2017, at 3:00 AM, Catonano  wrote:
> 
> Matt,
> 
> 2017-07-15 1:17 GMT+02:00 Matt Wette  >:
> 
> Hi All,
> 
> I am working on a ffi-helper: a program that will read in a C dot-h file and 
> generate a Guile
> 
> I would be intrigued by the possibility to wrrite Gnome apps in Guile scheme
> 
> I know that there's a proejct for thhat, it's G-golf
> https://www.gnu.org/software/g-golf/ 
> 
> But I undertsand it's in its very early stage
> 
> Could there be any overlap between your ffi helper and G-golf ? 
> 
> Could your ffi helper work on te GObject introspection thing ?
> 
> Also, there's a library provided by Postgresql. Cwebber's guile-squee uses 
> that to pilot Postgresql from Scheme
> 
> A nice wrap would be cool !
> 
> Thanks !

Much of gobject got sucked in when compiling librsvg.ffi.  Supporting g-object 
was tough but it does compile.   I’m guessing this could help with the  
g-object stuff.  However, I’m not a GOOPS programmer, so someone else would 
have to work on the GOOPS layer.

Re: ffi-helper: looking for C libraries to try

[Catonano]

Matt,

2017-07-15 1:17 GMT+02:00 Matt Wette :

>
> Hi All,
>
> I am working on a ffi-helper: a program that will read in a C dot-h file
> and generate a Guile


I would be intrigued by the possibility to wrrite Gnome apps in Guile scheme

I know that there's a proejct for thhat, it's G-golf
https://www.gnu.org/software/g-golf/

But I undertsand it's in its very early stage

Could there be any overlap between your ffi helper and G-golf ?

Could your ffi helper work on te GObject introspection thing ?

Also, there's a library provided by Postgresql. Cwebber's guile-squee uses
that to pilot Postgresql from Scheme

A nice wrap would be cool !

Thanks !

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 15, 2017, at 6:15 AM, Matt Wette  wrote:
> 
> 
>> On Jul 14, 2017, at 9:35 PM, Matt Wette  wrote:
>> 
>> 
>>> On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  
>>> wrote:
>>> 
>>> On 2017-07-15 01:17, Matt Wette wrote:
 I tried git2 but those headers are too
 broken.
>>> 
>>> What do you mean by "too broken"? Maybe it will be easier in the long run 
>>> to maintain our own version of the headers than manually bind everything. 
>>> Even if it requires to create a shim layer in scheme to have a proper set 
>>> of bindings that schemer will enjoy. Also, git is a good candidate since 
>>> there are some tests already in guile-git [1].
>> 
>> Maybe another bug in my CPP.  
>> 
>> (define-ffi-module (libgit2)
>> #:include '("git2.h")
>> #:inc-dirs '("/opt/local/include")
>> #:library '("git2")
>> )
>> 
>> mwette$ guild compile-ffi libgit2.ffi 
>> (unknown):1: not found: “common.h"
>> 
> 
> I claim libgit2 headers are broken.  The standard says, essentially, “how 
> headers are located is implementation defined”.
> 
> The file …/include/git2.h states:
> #include “git2/annotated_commit.h”
> #include “git2/common.h”
> …
> 
> But the file include/git2/annotated_commit.h states:
> #include “common.h”
> 
> Note this is “common.h" not “git2/common.h” How does gcc find common.h?  
> The directory include/git2 is not in the include path.  
> These are the ugly, undocumented, non-standard items that scare me.

I found a reference for include file searching in gcc: 
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html

I have returned to working on libgit2.  It turns out glib uses this, and 
include_next as well.
So I have added this behavior plus include_next.  Test/debug of the fyi-helper 
goes on …

I have been able too compile-ffi, almost everything from cairo, gdbm, sqlite3.  
Now working on rsvp and libgit2. 

Not supported so far is va-args.  That will require some help from the user.  
(Libffi does not support va-args.)

Matt

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 14, 2017, at 9:35 PM, Matt Wette  wrote:
> 
> 
>> On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  
>> wrote:
>> 
>> On 2017-07-15 01:17, Matt Wette wrote:
>>>  I tried git2 but those headers are too
>>> broken.
>> 
>> What do you mean by "too broken"? Maybe it will be easier in the long run to 
>> maintain our own version of the headers than manually bind everything. Even 
>> if it requires to create a shim layer in scheme to have a proper set of 
>> bindings that schemer will enjoy. Also, git is a good candidate since there 
>> are some tests already in guile-git [1].
> 
> Maybe another bug in my CPP.  
> 
> (define-ffi-module (libgit2)
>  #:include '("git2.h")
>  #:inc-dirs '("/opt/local/include")
>  #:library '("git2")
>  )
> 
> mwette$ guild compile-ffi libgit2.ffi 
> (unknown):1: not found: “common.h"
> 

I claim libgit2 headers are broken.  The standard says, essentially, “how 
headers are located is implementation defined”.

The file …/include/git2.h states:
#include “git2/annotated_commit.h”
#include “git2/common.h”
…

But the file include/git2/annotated_commit.h states:
#include “common.h”

Note this is “common.h" not “git2/common.h” How does gcc find common.h?  
The directory include/git2 is not in the include path.  
These are the ugly, undocumented, non-standard items that scare me.

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 15, 2017, at 5:37 AM, Matt Wette  wrote:
> 
> 
>> On Jul 15, 2017, at 1:24 AM, Amirouche  wrote:
>> 
>> 
>> 
>> Le 15/07/2017 à 06:35, Matt Wette a écrit :
 On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  
 wrote:
 
 On 2017-07-15 01:17, Matt Wette wrote:
>  I tried git2 but those headers are too
> broken.
 What do you mean by "too broken"? Maybe it will be easier in the long run 
 to maintain our own version of the headers than manually bind everything. 
 Even if it requires to create a shim layer in scheme to have a proper set 
 of bindings that schemer will enjoy. Also, git is a good candidate since 
 there are some tests already in guile-git [1].
>>> Maybe another bug in my CPP.
>>> 
>>> (define-ffi-module (libgit2)
>>>  #:include '("git2.h")
>>>  #:inc-dirs '("/opt/local/include")
>>>  #:library '("git2")
>>>  )
>>> 
>>> mwette$ guild compile-ffi libgit2.ffi
>>> (unknown):1: not found: “common.h"
>>> 
>>> 
>> With the following .ffi file:
>> 
>> (define-ffi-module (libgit2)
>> #:include '("git2.h")
>> #:inc-dirs '("/usr/include" "/usr/include/git2" 
>> "/usr/include/x86_64-linux-gnu/"  "/usr/include/linux/")
>> #:library '("git2")
>> )
>> 
>> I now have the following output:
>> 
>> $ guild compile-ffi libgit2/libgit2.ffi
>> TODO: compile-ffi args, arrays, extern variables
>> (unknown):1: parse failed at state 59, on input "__attribute__"
>> (unknown):1: C99 parse error
>> 
> 
> No surprise that it doesn’t work right now on non-Mac.  Add  #:inc-help 
> ‘((“__builtin” “__attribute__(X)=“))
> 
> My goal is release something that works in October 2017.  That may be an 
> ambitious goal.
> 
> Matt

When I created the C parser, I tried to put in hooks that would help avoid the 
highly system-dependent, compiler-dependent rat’s nest under /usr/include.  
With the FFI I will probably not be able to avoid /usr/include.  So, I will 
likely have to deal with passing flags from guild down to the translator in 
order to deal with these things.  In addition, the Mac has non-standard 
features like int foo() adm(“foo”);

Matt

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 15, 2017, at 1:24 AM, Amirouche  wrote:
> 
> 
> 
> Le 15/07/2017 à 06:35, Matt Wette a écrit :
>>> On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  
>>> wrote:
>>> 
>>> On 2017-07-15 01:17, Matt Wette wrote:
   I tried git2 but those headers are too
 broken.
>>> What do you mean by "too broken"? Maybe it will be easier in the long run 
>>> to maintain our own version of the headers than manually bind everything. 
>>> Even if it requires to create a shim layer in scheme to have a proper set 
>>> of bindings that schemer will enjoy. Also, git is a good candidate since 
>>> there are some tests already in guile-git [1].
>> Maybe another bug in my CPP.
>> 
>> (define-ffi-module (libgit2)
>>   #:include '("git2.h")
>>   #:inc-dirs '("/opt/local/include")
>>   #:library '("git2")
>>   )
>> 
>> mwette$ guild compile-ffi libgit2.ffi
>> (unknown):1: not found: “common.h"
>> 
>> 
> With the following .ffi file:
> 
> (define-ffi-module (libgit2)
>  #:include '("git2.h")
>  #:inc-dirs '("/usr/include" "/usr/include/git2" 
> "/usr/include/x86_64-linux-gnu/"  "/usr/include/linux/")
>  #:library '("git2")
>  )
> 
> I now have the following output:
> 
> $ guild compile-ffi libgit2/libgit2.ffi
> TODO: compile-ffi args, arrays, extern variables
> (unknown):1: parse failed at state 59, on input "__attribute__"
> (unknown):1: C99 parse error
> 

No surprise that it doesn’t work right now on non-Mac.  Add  #:inc-help 
‘((“__builtin” “__attribute__(X)=“))

My goal is release something that works in October 2017.  That may be an 
ambitious goal.

Matt

Re: ffi-helper: looking for C libraries to try

[Amirouche]

Le 15/07/2017 à 06:35, Matt Wette a écrit :

On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  wrote:

On 2017-07-15 01:17, Matt Wette wrote:

   I tried git2 but those headers are too
broken.

What do you mean by "too broken"? Maybe it will be easier in the long run to 
maintain our own version of the headers than manually bind everything. Even if it 
requires to create a shim layer in scheme to have a proper set of bindings that schemer 
will enjoy. Also, git is a good candidate since there are some tests already in guile-git 
[1].

Maybe another bug in my CPP.

(define-ffi-module (libgit2)
   #:include '("git2.h")
   #:inc-dirs '("/opt/local/include")
   #:library '("git2")
   )

mwette$ guild compile-ffi libgit2.ffi
(unknown):1: not found: “common.h"



With the following .ffi file:

(define-ffi-module (libgit2)
  #:include '("git2.h")
  #:inc-dirs '("/usr/include" "/usr/include/git2" 
"/usr/include/x86_64-linux-gnu/"  "/usr/include/linux/")

  #:library '("git2")
  )

I now have the following output:

$ guild compile-ffi libgit2/libgit2.ffi
TODO: compile-ffi args, arrays, extern variables
(unknown):1: parse failed at state 59, on input "__attribute__"
(unknown):1: C99 parse error

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 14, 2017, at 5:44 PM, Amirouche Boubekki  
> wrote:
> 
> On 2017-07-15 01:17, Matt Wette wrote:
>>   I tried git2 but those headers are too
>> broken.
> 
> What do you mean by "too broken"? Maybe it will be easier in the long run to 
> maintain our own version of the headers than manually bind everything. Even 
> if it requires to create a shim layer in scheme to have a proper set of 
> bindings that schemer will enjoy. Also, git is a good candidate since there 
> are some tests already in guile-git [1].

Maybe another bug in my CPP.  

(define-ffi-module (libgit2)
  #:include '("git2.h")
  #:inc-dirs '("/opt/local/include")
  #:library '("git2")
  )

mwette$ guild compile-ffi libgit2.ffi 
(unknown):1: not found: “common.h"

Re: ffi-helper: looking for C libraries to try

[Matt Wette]

> On Jul 14, 2017, at 5:55 PM, Amirouche  wrote:
> 
> Can you provide guidance regarding how to run compile-ffi, here what I've 
> done so far:
> 
> $ git clone https://git.savannah.nongnu.org/git/nyacc.git
> $ cd nyacc
> $ git checkout c99dev
> $ cd examples/nyacc/lang/c99
> $ source env.sh
> 
> Then:
> 
> $ guild compile-ffi cairo/cairo.ffi
> 
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;   or pass the --no-auto-compile argument to disable.
> ;;; compiling /home/amirouche/.guix-profile/bin/guild
> ;;; compiled 
> /home/amirouche/.cache/guile/ccache/2.2-LE-8-3.A/gnu/store/1pzfigry5bnh3n146w0ib77vkd2g6jdc-guile-2.2.2/bin/guild.go
> guild: unknown script "compile-ffi"
> Try `guild help' for more information.
> 
> I have guile-bytestructures
> 

Hmm.  Yea, I goofed on env.sh.  This seems to work for me:

#!/bin/sh
# source this file: $ . env.sh
topdir=`(cd ../../../..; pwd)`
if [ "X$GUILE_LOAD_PATH" = "X" ]; then
 GUILE_LOAD_PATH=$topdir/module:$topdir/examples:$topdir/examples/nyacc/lang/c99
else
 
GUILE_LOAD_PATH=$topdir/module:$topdir/examples:$topdir/examples/nyacc/lang/c99:$GUILE_LOAD_PATH
fi;
export GUILE_LOAD_PATH

mwette$ guild compile-ffi cairo/cairo.ffi
TODO: compile-ffi args, arrays, extern variables

Re: ffi-helper: looking for C libraries to try

[Amirouche]

Le 15/07/2017 à 01:17, Matt Wette a écrit :

Hi All,

I am working on a ffi-helper: a program that will read in a C dot-h file and 
generate a Guile dot-scm file which
defines a module to provide hooks into the associated C library.  My goal is to 
have something to release ~Oct 2017.

I am looking for suggestions for libraries (w/ includes of course) to use as 
test cases to flush-out and debug my code.  I have worked on cairo and now 
working on gdbm.  I tried git2 but those headers are too broken.

If you have suggestions for test cases, please post.  They should be commonly 
used.
(I use MacPorts and would want them to be installable in that context.)

Status:

I have now been able to compile-ffi the following on my Mac. I’m sure bugs 
remain.

(define-ffi-module (cairo cairo)
  #:pkg-config "cairo"
  #:include '(“cairo.h" "cairo-pdf.h" "cairo-svg.h")

  ;; the following are bent pipe to scm-module
  #:export (make-cairo-unit-matrix)
  )

(define (make-cairo-unit-matrix)
  (make-cairo_matrix_t #(1.0 0.0 0.0 1.0 0.0 0.0)))

I convert the above “cairo.ffi” file to “cairo.scm” using the following command

mwette$ guild compile-ffi cairo/cairo.ffi


Can you provide guidance regarding how to run compile-ffi, here what 
I've done so far:


$ git clone https://git.savannah.nongnu.org/git/nyacc.git
$ cd nyacc
$ git checkout c99dev
$ cd examples/nyacc/lang/c99
$ source env.sh

Then:

$ guild compile-ffi cairo/cairo.ffi

;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;   or pass the --no-auto-compile argument to disable.
;;; compiling /home/amirouche/.guix-profile/bin/guild
;;; compiled 
/home/amirouche/.cache/guile/ccache/2.2-LE-8-3.A/gnu/store/1pzfigry5bnh3n146w0ib77vkd2g6jdc-guile-2.2.2/bin/guild.go

guild: unknown script "compile-ffi"
Try `guild help' for more information.

I have guile-bytestructures

Thanks!

Re: ffi-helper: looking for C libraries to try

[Amirouche Boubekki]

On 2017-07-15 01:17, Matt Wette wrote:

Hi All,

I am working on a ffi-helper: a program that will read in a C dot-h
file and generate a Guile dot-scm file which
defines a module to provide hooks into the associated C library.  My
goal is to have something to release ~Oct 2017.

I am looking for suggestions for libraries (w/ includes of course) to
use as test cases to flush-out and debug my code.  I have worked on
cairo and now working on gdbm.  I tried git2 but those headers are too
broken.


What do you mean by "too broken"? Maybe it will be easier in the long 
run to maintain our own version of the headers than manually bind 
everything. Even if it requires to create a shim layer in scheme to have 
a proper set of bindings that schemer will enjoy. Also, git is a good 
candidate since there are some tests already in guile-git [1].


https://gitlab.com/guile-git



If you have suggestions for test cases, please post.  They should be
commonly used.
(I use MacPorts and would want them to be installable in that context.)


There is wiredtiger for which I am the maintainer. I will work on it. 
Since there is a test suite it will be easy to make it work. Sadly there 
is not support for it in macports.


By the way, nyacc is missing installation instruction.


Status:

I have now been able to compile-ffi the following on my Mac. I’m sure
bugs remain.

(define-ffi-module (cairo cairo)
 #:pkg-config "cairo"
 #:include '(“cairo.h" "cairo-pdf.h" "cairo-svg.h")

 ;; the following are bent pipe to scm-module
 #:export (make-cairo-unit-matrix)
 )

(define (make-cairo-unit-matrix)
 (make-cairo_matrix_t #(1.0 0.0 0.0 1.0 0.0 0.0)))

I convert the above “cairo.ffi” file to “cairo.scm” using the following 
command


mwette$ guild compile-ffi cairo/cairo.ffi

The above generates 397 FFI declarations in cairo.scm, a file which is
about 6000 lines long, compared to the original “cairo.ffi” which is
less than 10 lines long.

Some generated code:

;; typedef struct _cairo_device cairo_device_t;
(define-fh-pointer-type cairo_device_t*)

;; union _cairo_path_data_t {
;;   struct {
;; cairo_path_data_type_t type;
;; int length;
;;   } header;
;;   struct {
;; double x, y;
;;   } point;
;; };
(define cairo_path_data_t-desc
 (bs:union
   (list `(header
,(bs:struct
   (list `(type ,cairo_path_data_type_t-desc)
 `(length ,int
 `(point ,(bs:struct (list `(y ,double) `(x ,double)))
(export cairo_path_data_t-desc)
(define-fh-bytestructure-type/p cairo_path_data_t 
cairo_path_data_t-desc)

(define union-_cairo_path_data_t cairo_path_data_t)

;; typedef enum _cairo_path_data_type {
;;   CAIRO_PATH_MOVE_TO,
;;   CAIRO_PATH_LINE_TO,
;;   CAIRO_PATH_CURVE_TO,
;;   CAIRO_PATH_CLOSE_PATH,
;; } cairo_path_data_type_t;
(define-fh-enum-type cairo_path_data_type_t
 '((CAIRO_PATH_MOVE_TO . 0)
   (CAIRO_PATH_LINE_TO . 1)
   (CAIRO_PATH_CURVE_TO . 2)
   (CAIRO_PATH_CLOSE_PATH . 3))
 )

;; typedef void (*cairo_destroy_func_t)(void *data);
(define (wrap-cairo_destroy_func_t proc) ;; => pointer
(ffi:procedure->pointer ffi:void proc (list '*))
)
(export wrap-cairo_destroy_func_t)

;; cairo_status_t cairo_device_set_user_data(cairo_device_t *device, 
const
;; cairo_user_data_key_t *key, void *user_data, 
cairo_destroy_func_t

;; destroy);
(define cairo_device_set_user_data
 (let ((~f (ffi:pointer->procedure
 ffi:int
 (lib-func "cairo_device_set_user_data")
 (list '* '* '* '*
   (lambda (device key user_data destroy)
 (let ((~device (unwrap-cairo_device_t* device))
   (~key (unwrap-cairo_user_data_key_t* key))
   (~user_data (unwrap~pointer user_data))
   (~destroy (unwrap-cairo_destroy_func_t destroy)))
   (wrap-cairo_status_t
 (~f ~device ~key ~user_data ~destroy))
(export cairo_device_set_user_data)


--
Amirouche ~ amz3 ~ http://www.hyperdev.fr