Re: hu.dwim.zlib broke; broken operation-done-p

[Attila Lendvai]

> If you generate the Lisp files, do you need to check in the .spec files?

there's a strong argument for keeping the spec files: if the
cffi/c2ffi generator itself gets a bug fix/upgrade, then the lisp
files would need to be regenerated for those fixes/upgrades to take
effect, and that's only possible if the spec files are available.

which, BTW, currently happens automagically due to that unlucky
component -> system dependency on the "cffi/c2ffi-generator" system.

thanks for the review!

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
The real voyage of discovery consists not in seeking new landscapes,
but in looking with new eyes.

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

>: Attila

> what if i make the .spec generation a standalone operation that needs
> to be explicitly run by the library author?
>
I think that's indeed the right thing to do.

> it's a bit more burden for
> the lib author because he needs to keep track of changes to the
> configuration and/or the .h file, but that would implicitly fix one of
> the bugs.
>
He needs to, anyway, as he writes the wrappers for the library.

> https://github.com/cffi/cffi/commit/d59331b050fb31f3a3f9103f302890be176b77ce
LGTM.

> and shall i do the same with the next stage, namely the .lisp file
> generation? for some reason i think it's better to keep that automatic
> and integrated into the normal flow of the build process, but i'm not
> fully convinced about that either.
>
Generating the Lisp file in advance is good if it allows you to avoid
loading some of the code at build time. It might take more space. Or
you could save the Lisp files and drop the spec files.

> if i did that, it would annull the last remaining issue, namely the
> current unlucky
>
> (load-op ,(find-system "cffi/c2ffi-generator"))
>
> dependency.
>
Indeed, that would be slightly nicer on ASDF.

> but the price would be high: either users would need to regenerate
> .lisp files by hand each time they pull changes to the .spec file
> (this is unacceptable), or all libs would need to check in also the
> generated .lisp files into their repos (this is less of a
> constraint/burden, but still...).
>
> any thoughts?
>
If you generate the Lisp files, do you need to check in the .spec files?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
- We're all different.
- I'm not!

Re: hu.dwim.zlib broke; broken operation-done-p

[Attila Lendvai]

> One solution is to create a new file with the correct timestamp, that
> is either a copy of the existing spec file or a new generated one.


ok, here's an idea that i'd be grateful if i could bounce off of you:

what if i make the .spec generation a standalone operation that needs
to be explicitly run by the library author? it's a bit more burden for
the lib author because he needs to keep track of changes to the
configuration and/or the .h file, but that would implicitly fix one of
the bugs.

in fact i have already pushed this change:

https://github.com/cffi/cffi/commit/d59331b050fb31f3a3f9103f302890be176b77ce

and shall i do the same with the next stage, namely the .lisp file
generation? for some reason i think it's better to keep that automatic
and integrated into the normal flow of the build process, but i'm not
fully convinced about that either.

if i did that, it would annull the last remaining issue, namely the
current unlucky

(load-op ,(find-system "cffi/c2ffi-generator"))

dependency.

but the price would be high: either users would need to regenerate
.lisp files by hand each time they pull changes to the .spec file
(this is unacceptable), or all libs would need to check in also the
generated .lisp files into their repos (this is less of a
constraint/burden, but still...).

any thoughts?

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
Education is life. Life is education.

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

One solution is to create a new file with the correct timestamp, that
is either a copy of the existing spec file or a new generated one.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
For followers of most ideologies (openly religious or not), toleration is
a concession of defeat. For libertarians, it is victory itself.


On Thu, Feb 1, 2018 at 7:54 PM, Attila Lendvai  wrote:
>> My theory is that that you're failing to re-generate the .spec when
>> it's present, even though your dependencies say it will be
>> re-generated. Therefore, ASDF decides that it's out-of-date and must
>> be re-generated again the next time over, etc. In other words, you lie
>> to ASDF, and ASDF punishes you right back.
>
>
> that was indeed a lurking bug. excellent remote debugging skills,
> thanks! :)
>
> i pushed the fix. now after i touch'd the appropriate spec files, it
> doesn't want to run c2ffi anymore, but it's still regenerating the
> lisp file unconditionally.
>
> i ran out of hacking steam for today. i'll look into it tomorrow, but
> further remote debugging magic is welcome of course... :)
>
> i'm kinda lost for now. i have no plan of action, so to say... :)
>
> --
> • attila lendvai
> • PGP: 963F 5D5F 45C7 DFCD 0A39
> --
> Liberty is a demand. Tyranny is submission.

Re: hu.dwim.zlib broke; broken operation-done-p

[Attila Lendvai]

> My theory is that that you're failing to re-generate the .spec when
> it's present, even though your dependencies say it will be
> re-generated. Therefore, ASDF decides that it's out-of-date and must
> be re-generated again the next time over, etc. In other words, you lie
> to ASDF, and ASDF punishes you right back.


that was indeed a lurking bug. excellent remote debugging skills,
thanks! :)

i pushed the fix. now after i touch'd the appropriate spec files, it
doesn't want to run c2ffi anymore, but it's still regenerating the
lisp file unconditionally.

i ran out of hacking steam for today. i'll look into it tomorrow, but
further remote debugging magic is welcome of course... :)

i'm kinda lost for now. i have no plan of action, so to say... :)

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
Liberty is a demand. Tyranny is submission.

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

On Tue, Jan 30, 2018 at 5:01 PM, Attila Lendvai  wrote:
> if you issue the following (available in quicklisp):
>
> (asdf:load-system :hu.dwim.zlib)
>
> then for the first time it should generate a lisp file, which then gets
> compiled and loaded.
>
> issuing it for the second time shouldn't do anything, but since some
> revisions it regenerates the lisp file every time.
>
> the relevant code is somewhere around here:
>
> https://github.com/cffi/cffi/blob/master/src/c2ffi/asdf.lisp#L135
>
> i tried to trace the INPUT-FILES, OUTPUT-FILES, PERFORM,
> OPERATION-DONE-P methods, but i don't see anything wrong.
>
> one unusual thing is that the output of GENERATE-LISP-OP goes into the
> src/ directory, not to the usual fasl output dir. same applies to
> GENERATE-SPEC-OP.
>
> any ideas how to debug this? or what to look at?
>
I can reproduce the issue using asdf 3.3.1.3 from master (the tag
hadn't been pushed, so I pushed it, but the code was already there).

(plan-actions (nth-value 1 (operate 'load-op :hu.dwim.zlib)))

starts with:
((# . #)
 (# . #)
 (# . #) (# . #)

So somehow the first action above is considered not done. I suppose
it's a bug in c2ffi.

Looking at the c2ffi source, I don't see anything obviously wrong,
though you shouldn't the (the (not null) (find-system ...)) since
find-system will error out for you if the system is not found (to
avoid error, add optional argument nil, as in find-class).

My theory is that that you're failing to re-generate the .spec when
it's present, even though your dependencies say it will be
re-generated. Therefore, ASDF decides that it's out-of-date and must
be re-generated again the next time over, etc. In other words, you lie
to ASDF, and ASDF punishes you right back.

To make ASDF happy, you might instead want to have two disjoint
operation modes, one to generate the spec files, and one to use it.
Normal use mode wouldn't try to generate the file, but instead bork
with a useful error message telling you how to generate it and submit
it upstream.

Another strategy to make ASDF happy would be to detect whether the
file exists as part of input-files and output-files. If it exists,
it's in input-files and not output-files; if not, it's in output-files
and not input-files. There's something I don't like about this
strategy, but it might work (somehow I can't swear it won't bite you
back somehow at some point).

Good luck!

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
There is only one thing more harmful to society than an elected official
forgetting the promises he made in order to get elected; that's when he
doesn't forget them.  — John McCarthy

hu.dwim.zlib broke; broken operation-done-p

[Attila Lendvai]

if you issue the following (available in quicklisp):

(asdf:load-system :hu.dwim.zlib)

then for the first time it should generate a lisp file, which then gets
compiled and loaded.

issuing it for the second time shouldn't do anything, but since some
revisions it regenerates the lisp file every time.

the relevant code is somewhere around here:

https://github.com/cffi/cffi/blob/master/src/c2ffi/asdf.lisp#L135

i tried to trace the INPUT-FILES, OUTPUT-FILES, PERFORM,
OPERATION-DONE-P methods, but i don't see anything wrong.

one unusual thing is that the output of GENERATE-LISP-OP goes into the
src/ directory, not to the usual fasl output dir. same applies to
GENERATE-SPEC-OP.

any ideas how to debug this? or what to look at?

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“For an instant I think I saw. I saw the loneliness of man as a
gigantic wave which had been frozen in front of me, held back by the
invisible wall of a metaphor.”
— Carlos Castaneda (1925–1998), 'Journey to Ixtlan' (1972)