Here is a problem with CMU Lisp I found, when
trying to port a connection of LISP
to the C bison parser to CMU Lisp.
It looks like the garbage collector
sometimes destroys values that have been
just passed from C to LISP via callback's.
My best guess is that it is moving
the reference to an extern C string before
converting it to a LISP string.
The problem is VERY hard to reproduce with a small example
(it always happens right after some garbage collection, but
only very few GC's trigger it). Even minimal changes in the small
example code below often make the problem vanish
(I'm using the binary of release 19a on a Suse 9.1 Linux distribution).
With the full parser the problem occurs much more often
than with the small example below,
probably due to more cons'ing in the actions.
The following C program "cmutest.c" simulates random
actions done by a bison parser (C). The parser is called
from LISP, and its actions use a callback function
lispcall to construct datastructures on a LISP stack.
-----------------------------------------------------
parse(s, lispcall)
char s[] ;
int (*lispcall) () ;
{
// in reality actions will
// depend on bison parsing s
// instead here are some random actions
int i;
int j;
// do 10000 times:
for (j = 1; j < 10000; j++) {
// push 100 x "AA" on parser_stack
for (i = 1; i < 100; i++) {
lispcall("ACTION2","AA");
}
for (i = 1; i < 100; i++) {
// pop them again, empty string is ignored
lispcall("ACTION1","");
}
}
return(0);
}
--------------------------------------------
The corresponding LISP Program "cmutest.lisp" is:
---------------------------------------------------
(in-package "COMMON-LISP-USER")
(use-package :alien)
(use-package :c-call)
(defun parser (text)
(parse text (callback lispcall)))
(def-alien-routine parse int
(s c-string)
(lispcall (* (function int c-string c-string)))
)
(defvar parser_stack nil)
(defun action1 (value)
(declare (ignore value))
(pop parser_stack)
)
(defun action2 (value)
(push value parser_stack)
)
(def-callback lispcall (int (actionname c-string) (argument c-string))
;; this should not happen, but does
(when (string= "" actionname) (break))
;; execute one of action1/ action2
(funcall (intern actionname "COMMON-LISP-USER") argument)
0)
---------------------------------------------------
The exact procedure how I get this error is as follows:
I compile the C program with
gcc -fPIC -c cmutest.c -o cmutest.o
ld -shared -o cmutest.so cmutest.o
Then I start cmulisp, load the foreign file cmutest.so, load
the source file, compile it (compile-file), and leave cmulisp again.
Then I start cmulisp again, load the foreign file cmutest.so
and then the compiled cmutest.x86f file. Finally I call
(parser "dummystring")
The C program calls lispcall, which dispatches to the
different actions (here only two, in the real program there are
a lot more) by converting the actionname it gets to
a function (with intern). It alternates (10000 times)
100 pushes of the string "AA" on the global parser_stack (action2)
with 100 pops (action1). Now, after some call
of the garbage collector, the program will stop with
Error in KERNEL:%COERCE-TO-FUNCTION: the function || is undefined.
[Condition of type UNDEFINED-FUNCTION]
indicating that an empty string was intern'ed.
In tests with the full parser, often (break) was called,
but I couldn't reproduce this for the small example.
Very small changes to the program make the error go away in
the small example (e.g. I tried a copy-seq on the actionname before
using it). Some changes also trigger the (break).
Even removing the "(when (string= "" actionname) (break))"
line makes it go away. Unfortunately, all these changes
do not help when I try the full parser. I always get callback's
that return an empty string, even though all callback's have constant
non-empty strings on the C side. The string does not seem to
be overwritten in C, I have written testfunctions that
positively show that putting the string in a C-variable,
and using an alien-call to C after the error occurs,
shows that the string is still unchanged from C's perspective.
Maybe the warning
(INTERN ACTIONNAME "COMMON-LISP-USER")
; --> ACTIONNAME ALIEN::LOCAL-ALIEN ALIEN::NATURALIZE
; ==>
; (IF (ZEROP #) NIL (C-CALL::%NATURALIZE-C-STRING ALIEN))
; Warning: This is not a (VALUES &OPTIONAL BASE-STRING &REST T):
when compiling "cmutest.lisp" is a hint, but I interpret it just
as: the compiler cannot determine, that actionname will never
be nil (which should not be a problem).
Any ideas?
Gerhard
