Upon investigating further, the problem seems to be a bad interaction
between block compilation and memoization. I am using the memoization
facility as defined in Norvig's PAIP book (included below). The basic
gist is that memo returns a memoized version of a function, memoize
"redefines" the function to be the memoized version, and defun-memo
combines these operations.
I find that I don't yet have a good enough mental model of the
compilatoin process to understand how this would interact with block
compilations. It seems that if the memoization and redefinition both
happened before the code was compiled, it should be OK. Also, since
the function memo is itself compiled, it seems that the closures it
puts out should also be compiled, at least assuming that the function
it gets passed is compiled. Can anyone help clear these issues up,
and suggest possible bad interactions between memoization and inlining
and/or block compilation, and how to avoid them?
Cheers,
rif
(defmacro defun-memo (fn args &body body)
"Define a memoized function."
`(memoize (defun ,fn ,args . ,body)))
(defun memo (fn &key (key #'first) (test #'eql) name)
"Return a memo-function of fn."
(let ((table (make-hash-table :test test)))
(setf (get name 'memo) table)
#'(lambda (&rest args)
(let ((k (funcall key args)))
(multiple-value-bind (val found-p)
(gethash k table)
(if found-p val
(setf (gethash k table) (apply fn args))))))))
(defun memoize (fn-name &key (key #'first) (test #'eql))
"Replace fn-name's global definition with a memoized version."
(clear-memoize fn-name)
(setf (symbol-function fn-name)
(memo (symbol-function fn-name)
:name fn-name :key key :test test)))
