Re: unwanted recompilation with :CFFI-GROVEL-FILE

[JTK]

Thank you for the suggestion to upgrade ASDF, Stelian.

I upgraded my builtin SBCL asdf.fasl from 3.3.4 to 3.3.4.1 (git version) with a 
manual replacement of fasl.

This 

- fixes my toy test case on OSX+SBCL (no more unnecessary recompile)

- does not fix osicat on OSX+SBCL

- but both osicat and toy test case work in ccl64+OSX with no ASDF fixes needed.

- but osicat works without reloading in SBCL+Linux64, again without an ASDF 
upgrade.

So it looks like it could be some kind of glitch in asdf or OSX SBCL (up to 
1.4.9).

JK



> On Apr 14, 2020, at 5:40 PM, Stelian Ionescu  wrote:
> 
>> 
>> Hello,
>> 
>> Sorry if this is the wrong place to ask ...
>> 
>> I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ 
>>    and  
>> https://github.com/osicat/osicat  ) in 
>> SBCL on OSX.
>> 
>> whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the 
>> cffi-grovel-files and their dependencies.
>> 
>> Is this normal, or is there some way to make it behave like asdf usually 
>> does, that it recompiles only when a file has changed?
> 
> This sounds very much like a bug in ASDF that was fixed a while ago. I 
> suggest upgrading to the latest version and retrying.
> 
> --
> Stelian Ionescu

Re: unwanted recompilation with :CFFI-GROVEL-FILE

[JTK]

It turns out to be an intermediate-version SBCL bug, fixed by upgrading to 
2.0.3.  

Sorry for this additional email, but I thought it better to close the issue out.



> On Apr 15, 2020, at 12:36 PM, JTK  wrote:
> 
> 
> Thank you for the suggestion to upgrade ASDF, Stelian.
> 
> I upgraded my builtin SBCL asdf.fasl from 3.3.4 to 3.3.4.1 (git version) with 
> a manual replacement of fasl.
> 
> This 
> 
> - fixes my toy test case on OSX+SBCL (no more unnecessary recompile)
> 
> - does not fix osicat on OSX+SBCL
> 
> - but both osicat and toy test case work in ccl64+OSX with no ASDF fixes 
> needed.
> 
> - but osicat works without reloading in SBCL+Linux64, again without an ASDF 
> upgrade.
> 
> So it looks like it could be some kind of glitch in asdf or OSX SBCL (up to 
> 1.4.9).
> 
> JK
> 
> 
> 
>> On Apr 14, 2020, at 5:40 PM, Stelian Ionescu > > wrote:
>> 
>>> 
>>> Hello,
>>> 
>>> Sorry if this is the wrong place to ask ...
>>> 
>>> I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ 
>>>    and  
>>> https://github.com/osicat/osicat  ) in 
>>> SBCL on OSX.
>>> 
>>> whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the 
>>> cffi-grovel-files and their dependencies.
>>> 
>>> Is this normal, or is there some way to make it behave like asdf usually 
>>> does, that it recompiles only when a file has changed?
>> 
>> This sounds very much like a bug in ASDF that was fixed a while ago. I 
>> suggest upgrading to the latest version and retrying.
>> 
>> --
>> Stelian Ionescu
> 

Re: unwanted recompilation with :CFFI-GROVEL-FILE

[Stelian Ionescu]

> Hello,
> 
> Sorry if this is the wrong place to ask ...
> 
> I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ and 
> https://github.com/osicat/osicat ) in SBCL on OSX.
> 
> whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the 
> cffi-grovel-files and their dependencies.
> 
> Is this normal, or is there some way to make it behave like asdf usually 
> does, that it recompiles only when a file has changed?

This sounds very much like a bug in ASDF that was fixed a while ago. I suggest 
upgrading to the latest version and retrying.

--
Stelian Ionescu

Re: unwanted recompilation with :CFFI-GROVEL-FILE

[JTK]

Sorry for all these emails, but this trivial case reproduces the recompilation, 
so it isn’t osicat specific.

https://github.com/jetmonk/test-grovel 





> On Apr 14, 2020, at 4:31 PM, JTK  wrote:
> 
> 
> A followup:  this behavior stops when I comment out 
>  (:cffi-wrapper-file "wrappers" :soname "libosicat”) 
> in osicat’s asd file.
> 
> I understand that wrappers are to generate functions from C macros, 
> necessitating a C compilation step to build a dynamic library that is then 
> loaded.
> 
> For example in libosicat, the following line appears to turn a C macro into a 
> function
> 
>  (defwrapper* "s_isreg" ("int" :boolean) ((mode ("mode_t" mode)))
>   "return S_ISREG(mode);”)
> 
> 
> Apparently wrappers.lisp becomes wrappers.processed-wrapper-file, which, 
> despite not changing upon
> reload of osicat system, is recompiled anyway, resulting in a new 
> wrappers.fasl.
> 
> The dynamic library resulting from this (libosicat.dylib on OSX) is not 
> rebuilt, however.
> 
> In short, :CFFI-WRAPPER-FILE seems to trigger unnecessary recompilation of an 
> intermediate lisp
> file processed from the original lisp source, but not creation of this 
> intermediate file, nor the recompilation
> of the dynamic library.
> 
> Using the latest git version of CFFI doesn’t seem to help.
> 
> 
> 
> 
> 
>> On Apr 14, 2020, at 1:06 PM, JTK > > wrote:
>> 
>> 
>> Hello,
>> 
>> Sorry if this is the wrong place to ask ...
>> 
>> I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ 
>>    and  
>> https://github.com/osicat/osicat  ) in 
>> SBCL on OSX.
>> 
>> whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the 
>> cffi-grovel-files and their dependencies.
>> 
>> Is this normal, or is there some way to make it behave like asdf usually 
>> does, that it recompiles only when a file has changed?
>> 
>> I know that this issue has 3 components (asdf, cffi, and osicat), but I 
>> thought I’d first check whether this is an unavoidable feature of using 
>> grovel files.
>> If not,  I can try to fix osicat.   
>> 
>> Here are some of the compilation messages: https://pastebin.com/GKArFCWc 
>>  
>> 
>> JK
>> 
>> 
>> 
> 

Re: unwanted recompilation with :CFFI-GROVEL-FILE

[JTK]

A followup:  this behavior stops when I comment out 
 (:cffi-wrapper-file "wrappers" :soname "libosicat”) 
in osicat’s asd file.

I understand that wrappers are to generate functions from C macros, 
necessitating a C compilation step to build a dynamic library that is then 
loaded.

For example in libosicat, the following line appears to turn a C macro into a 
function

 (defwrapper* "s_isreg" ("int" :boolean) ((mode ("mode_t" mode)))
  "return S_ISREG(mode);”)


Apparently wrappers.lisp becomes wrappers.processed-wrapper-file, which, 
despite not changing upon
reload of osicat system, is recompiled anyway, resulting in a new wrappers.fasl.

The dynamic library resulting from this (libosicat.dylib on OSX) is not 
rebuilt, however.

In short, :CFFI-WRAPPER-FILE seems to trigger unnecessary recompilation of an 
intermediate lisp
file processed from the original lisp source, but not creation of this 
intermediate file, nor the recompilation
of the dynamic library.

Using the latest git version of CFFI doesn’t seem to help.





> On Apr 14, 2020, at 1:06 PM, JTK  wrote:
> 
> 
> Hello,
> 
> Sorry if this is the wrong place to ask ...
> 
> I’m using the ‘osicat’ package ( https://common-lisp.net/project/osicat/ 
>    and  
> https://github.com/osicat/osicat  ) in SBCL 
> on OSX.
> 
> whenever I do ‘(asdf:load-system “osicat”) it seems to recompile the 
> cffi-grovel-files and their dependencies.
> 
> Is this normal, or is there some way to make it behave like asdf usually 
> does, that it recompiles only when a file has changed?
> 
> I know that this issue has 3 components (asdf, cffi, and osicat), but I 
> thought I’d first check whether this is an unavoidable feature of using 
> grovel files.
> If not,  I can try to fix osicat.   
> 
> Here are some of the compilation messages: https://pastebin.com/GKArFCWc 
>  
> 
> JK
> 
> 
>