Re: Method combination and ASDF

[Faré]

On Thu, Apr 25, 2024 at 9:11 PM Faré  wrote:
>
> Funny, but ASDF 1 at some point (commit aa52ad22 1.128 to 1.636) up was 
> defining its own method combination, so users could write their own methods 
> without overwriting ASDF's (but still overwriting each other's).
>
> In practice it was only used for two :around methods for perform, with one 
> being renamed perform-with-restarts in the end. It may also have helped with 
> portability to gcl, that was added at the same time.
>
Uh, I meant *removing* that method combination may have helped with
portability to gcl.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“Why is there only one Monopolies Commission?”
— Lord Sutch

Re: Method combination and ASDF

[Faré]

Funny, but ASDF 1 at some point (commit aa52ad22 1.128 to 1.636) up was
defining its own method combination, so users could write their own methods
without overwriting ASDF's (but still overwriting each other's).

In practice it was only used for two :around methods for perform, with one
being renamed perform-with-restarts in the end. It may also have helped
with portability to gcl, that was added at the same time.

-#f


On Thu, Apr 25, 2024, 19:33 Robert Goldman  wrote:

> On 24 Apr 2024, at 3:39, Didier Verna wrote:
>
> > François-René ÐVB Rideau écrivait:
> >
> >> What are the methods defined by asdf-flv?
> >
> >   In order to support file local variables, ASDF-FLV does this:
> >
> > (defmethod asdf:perform :around
> > ((operation asdf:load-op) (file asdf:cl-source-file))
> >   "Establish new dynamic bindings for file-local variables."
> >   (progv *file-local-variables*
> >   (mapcar #'symbol-value *file-local-variables*)
> > (call-next-method)))
> >
> > (defmethod asdf:perform :around
> > ((operation asdf:compile-op) (file asdf:cl-source-file))
> >   "Establish new dynamic bindings for file-local variables."
> >   (progv *file-local-variables*
> >   (mapcar #'symbol-value *file-local-variables*)
> > (call-next-method)))
> >
> >
> > The problem, of course, is that loading ASDF-FLV will override any
> > existing method with the same signature, and conversely, anything else
> > loaded afterward will override ASDF-FLV.
> >
> > ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that
> > subclass, but that would defeat its pervasiveness and make it more
> > difficult to have multiple extensions working together. Hence Robert's
> > idea of mixins. My own idea was to have exported operations use a
> > special method combination allowing multiple methods with the same
> > signature.
>
> The one thing that worries me about that is that I don't know how well all
> the supported lisp implementations handle defining new method combinations.
>
> But otherwise I agree that this would be an appropriate thing to do.
>
>

Re: Method combination and ASDF

[Faré]

1. I once implemented a more general mechanism for binding syntax
variables around compilation, which must be part of the syntax-control
branch (that probably needs a lot of love rebasing it on top of the
latest ASDF). Syntax variables include *readtable*, *read-base*,
*read-default-float-format*, etc., as well as any user-defined
variables to similar purposes in user extensions.

2. However, note this only has guaranteed semantics if the effects of
those variables are at read-time / macroexpansion-time / compile-time.
Features such as linking (on ECL, MKCL) or fasl concatenation (on
about every other platform) are incompatible with relying on effects
from binding variables around individual lisp or fasl files. Of
course, you could declare "my system and any system that depends on it
is incompatible with linking and fasl concatenation".

3. As for having bindings around compiling individual files, there is
already the around-compile hook, that you could use or override or
define methods for, to bind variables around such compilation. You
could use this interface, and make it more extensible if you want.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“Those who cannot remember the past are condemned to repeat it.”
— George Santayana

On Wed, Apr 24, 2024 at 4:51 AM Didier Verna  wrote:
>
> François-René ÐVB Rideau écrivait:
>
> > What are the methods defined by asdf-flv?
>
>   In order to support file local variables, ASDF-FLV does this:
>
> (defmethod asdf:perform :around
> ((operation asdf:load-op) (file asdf:cl-source-file))
>   "Establish new dynamic bindings for file-local variables."
>   (progv *file-local-variables*
>   (mapcar #'symbol-value *file-local-variables*)
> (call-next-method)))
>
> (defmethod asdf:perform :around
> ((operation asdf:compile-op) (file asdf:cl-source-file))
>   "Establish new dynamic bindings for file-local variables."
>   (progv *file-local-variables*
>   (mapcar #'symbol-value *file-local-variables*)
> (call-next-method)))
>
>
> The problem, of course, is that loading ASDF-FLV will override any
> existing method with the same signature, and conversely, anything else
> loaded afterward will override ASDF-FLV.
>
> ASDF-FLV could subclass ASDF:CL-SOURCE-FILE and specialize only on that
> subclass, but that would defeat its pervasiveness and make it more
> difficult to have multiple extensions working together. Hence Robert's
> idea of mixins. My own idea was to have exported operations use a
> special method combination allowing multiple methods with the same
> signature.
>
> --
> Resistance is futile. You will be jazzimilated.
>
> Jazz site:   http://www.didierverna.com
> Other sites: http://www.didierverna.info

Re: Method combination and ASDF

[Faré]

What are the methods defined by asdf-flv?

IIRC, POIU also has a few :around methods (it also doesn't properly support
ASDF 3.3 phases).

-#f


On Tue, Apr 23, 2024, 17:24 Robert Goldman  wrote:

> An issue that came up discussing ASDF-FLV with Didier has to do with
> outside modification to exported ASDF generic functions. I think it's
> unambiguously reasonable to say "if you mess with unexported functions,
> generic or otherwise, you're on your own, and could break things
> arbitrarily."
>
> But I don't think ASDF has established clear expectations about extending
> methods of exported ASDF generic functions. I think there's been an
> unspoken assumption that if ASDF generic functions are to be extended,
> particularly by :around, :before, and :after methods, there should be
> some novel class introduced that will be used to select extensions. So, for
> example, it's fine to wrap or write new methods for perform, but not for
> existing component and operation classes. Doing so means that a change one
> might think appropriate for one's own library bleeds out into other code
> that one has no way of predicting, and also has the potential to collide
> with ASDF's own code or other people's ASDF extensions.
>
> Those are good reasons for this convention -- no modification without
> introducing new classes for method dispatch -- but ASDF-FLV is an example
> of a system that wants to introspect about the entire running lisp image.
> I'd be interested in hearing what people think about this kind of case.
> Should we introduce, for example, an ASDF object that extenders could add
> mixins to in order to enable modifications to ASDF behavior that aren't
> limited to a single library?
>
> Please post any thoughts you have about this idea.
>

Re: Loading only fasl files

[Faré]

You may be interested in deliver-asd-op to create a system that consists in
a fasl that you load.

On Sun, Apr 21, 2024, 13:42 Greg Bennett  wrote:

> Hello from Greg Bennett who is trying to debug some compilation issues
> in Allegro.
>
> I have what I believe to be a completely compiled project, consisting of
> several systems,
> and I would like to load only its fasl files. Specifically I do not want
> to re-compile the source
> files which would replace those fasls.
>
> I have looked at the manual in the hope that load-bundle-op might work,
> but I see that
> this requires *compiling* the source into a single fasl.
>
> Perhaps (probably!) I'm missing something.
>
> Any advice/assistance much appreciated.
>
> Cheers
> Greg
>

Re: Readtable not bound to standard around compilation

[Faré]

rpg:>> Given that Quicklisp and SBCL already refuse to update their
bundled ASDF versions, because of warnings about deprecated behavior,
I'm reluctant to donate any of my unpaid time to fixing this: it's a
strong disincentive to making any improvements to ASDF, as opposed to
just fixing bugs around the edges.
On the other hand, the whole point of ASDF 2 and later was that by
making ASDF upgradable (and with ASDF 3, it's now automatically
self-upgradable), users shouldn't have to care as much which version
of ASDF their implementation and package distribution system do or
don't provide: "just" install the latest ASDF in e.g.
~/common-lisp/asdf/ and things will "just work". If the new ASDF is so
much better, eventually the implementors and distributors should
follow.

si:> Is it time for ASDF 4 ? There's tons of stuff I'd like to delete or change.
It's always time for ASDF 4, and never time for ASDF 4. The main
question is: is someone crazy enough to sink in the time to do it, the
emotional energy to fight half the community, etc.

If only Bazel didn't fuck up their namespace system, the solution
could have been "just use Bazelisp".

If and when someone volunteers to do ASDF 4 (if ever), there are
plenty of suggestions in the asdf/TODO file, in addition to the issues
on gitlab and the old launchpad. Good luck!

—♯ƒ • François-René Rideau • Chief Scientist, MuKn.com
“With freedom, no more One True Scale to rank people. Everyone pick his own.
Why vie for a society of equals, when everyone can be superior?”

Re: Readtable not bound to standard around compilation

[Faré]

Oh, and as for binding variables around files, this has subtly different
meanings when compiling a file, loading its fasl or loading its source. And
if done wrong it can easily break fasl concatenation or linking on ecl and
mkcl, where you just can't bind around the loading of individual fasls,
whereas with source loading you can't not bind, so now the user is
responsible to make sure his code works both ways and the binding only
really matters at compile-time—which kind of is already a development
constraint, just unarticulated, unenforced, and undetected until a subtle
bug hits someone.

In the end you can only protect the developer from himself so much, when
you can't change the semantics of CL itself across twenty implementations
many unmaintained.

On Thu, Feb 22, 2024, 11:16 Faré  wrote:

> In the past, to introduce changes a fraction as compatibility-breaking
> (e.g. making utf-8 the default, or changing the internals of OPERATION), I
> would write up an explanation why the change is desired that anyone can
> read, discuss on the mailing list, write the code but keep it unmerged or
> disabled, make an announcement for the intended future breaking change, go
> over all of Quicklisp with Anton Vodonosov's cl-test-grid, locate all the
> offenders there, and send patches to each and everyone, wait a year if the
> old behavior used to be officially documented and supported, only a few
> weeks if it was a forbidden internals dependency, run the test again, give
> a last warning to those who didn't merge the patches, sometimes fork the
> systems left unmaintained to a new home, then finally make the change and
> announce it. And be ready to handle all the angry authors of proprietary
> code not in Quicklisp that you failed to previously contact.
>
> For syntax-control (see my branch, that needs much love merging with
> 3.3.7), we never reached that point, if only because we were never
> satisfied with the complexity of the code (speaking for myself at least, I
> suspect for RPG also to a point, but he can opine otherwise if needed).
> There are many special variables beside *readtable* that control syntax
> (e.g. some systems try to make double-float the default—with "interesting"
> side effects to other systems compiled later, more subtle than the big
> obvious breakage when the readtable is wrong or corrupted). And users
> define more such variables as they extend the language. So now you want to
> maintain a dynamic list of symbols, some in packages not yet defined, that
> you want to track, save and restore. Next your users will want to bind the
> values of some of these variables around some systems, modules or
> components. The code is big and messy, and it isn't even obviously "the
> right thing" overall, though a lot of elements of it are, and every piece
> is needed somehow.
>
> There's also the question of whether the default readtable should be THE
> "standard" readtable (immutable on some implementations, not on others), or
> some shared mutable readtable copied from it (will break many CCL
> extensions), or from the magic initial readtable that is not standard and
> that some will mutate (the most backward compatible option, so probably the
> one ASDF should pick, though it's ugly, unless you want to wrestle some
> more with the community).
>
> Should we merge the whole messy thing that solves the entire problem? Only
> the small and clean part that is obviously right but clearly insufficient?
> Each time we make such a change, we have to pay a lot to overcome a big
> incompatibility hurdle. If the solution isn't fully satisfactory, then we
> are setting ourselves up for yet another round (or several) of such costly
> change(s).
>
> I'm out of the Common Lisp community (moved to Gerbil Scheme), but whoever
> wants to tackle this challenge for good better makes sure they (and the
> community!) are ready to pay the price for the transition, and multiple
> times so if they don't get it right on the first try. Oh, and once you take
> on the job, you'll be hated by some part of the community whether you
> subsequently make the change or don't.
>
> Good luck!
>
> On Thu, Feb 22, 2024, 10:15 Robert Goldman  wrote:
>
>> I think this is still true, but... we cannot be discussing ASDF 3.2.1
>> here. It was released almost 7 years ago, and for whatever reason Zach
>> refuses to update. The current version is 3.3.7
>>
>> Please get a more recent ASDF and try again. I *believe* that this
>> behavior is still in place: Faré proposed the "syntax control" patch to fix
>> this, but we decided we had no easy path to introducing it, because it
>> would break other code (admittedly not of good style) that relies on the
>> old behavior of having the re

Re: Readtable not bound to standard around compilation

[Faré]

In the past, to introduce changes a fraction as compatibility-breaking
(e.g. making utf-8 the default, or changing the internals of OPERATION), I
would write up an explanation why the change is desired that anyone can
read, discuss on the mailing list, write the code but keep it unmerged or
disabled, make an announcement for the intended future breaking change, go
over all of Quicklisp with Anton Vodonosov's cl-test-grid, locate all the
offenders there, and send patches to each and everyone, wait a year if the
old behavior used to be officially documented and supported, only a few
weeks if it was a forbidden internals dependency, run the test again, give
a last warning to those who didn't merge the patches, sometimes fork the
systems left unmaintained to a new home, then finally make the change and
announce it. And be ready to handle all the angry authors of proprietary
code not in Quicklisp that you failed to previously contact.

For syntax-control (see my branch, that needs much love merging with
3.3.7), we never reached that point, if only because we were never
satisfied with the complexity of the code (speaking for myself at least, I
suspect for RPG also to a point, but he can opine otherwise if needed).
There are many special variables beside *readtable* that control syntax
(e.g. some systems try to make double-float the default—with "interesting"
side effects to other systems compiled later, more subtle than the big
obvious breakage when the readtable is wrong or corrupted). And users
define more such variables as they extend the language. So now you want to
maintain a dynamic list of symbols, some in packages not yet defined, that
you want to track, save and restore. Next your users will want to bind the
values of some of these variables around some systems, modules or
components. The code is big and messy, and it isn't even obviously "the
right thing" overall, though a lot of elements of it are, and every piece
is needed somehow.

There's also the question of whether the default readtable should be THE
"standard" readtable (immutable on some implementations, not on others), or
some shared mutable readtable copied from it (will break many CCL
extensions), or from the magic initial readtable that is not standard and
that some will mutate (the most backward compatible option, so probably the
one ASDF should pick, though it's ugly, unless you want to wrestle some
more with the community).

Should we merge the whole messy thing that solves the entire problem? Only
the small and clean part that is obviously right but clearly insufficient?
Each time we make such a change, we have to pay a lot to overcome a big
incompatibility hurdle. If the solution isn't fully satisfactory, then we
are setting ourselves up for yet another round (or several) of such costly
change(s).

I'm out of the Common Lisp community (moved to Gerbil Scheme), but whoever
wants to tackle this challenge for good better makes sure they (and the
community!) are ready to pay the price for the transition, and multiple
times so if they don't get it right on the first try. Oh, and once you take
on the job, you'll be hated by some part of the community whether you
subsequently make the change or don't.

Good luck!

On Thu, Feb 22, 2024, 10:15 Robert Goldman  wrote:

> I think this is still true, but... we cannot be discussing ASDF 3.2.1
> here. It was released almost 7 years ago, and for whatever reason Zach
> refuses to update. The current version is 3.3.7
>
> Please get a more recent ASDF and try again. I *believe* that this
> behavior is still in place: Faré proposed the "syntax control" patch to fix
> this, but we decided we had no easy path to introducing it, because it
> would break other code (admittedly not of good style) that relies on the
> old behavior of having the readtable setting leak out of one file into
> another. See https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/86
>
> Any non-backwards compatible modification to ASDF -- even to the extent of
> raising a deprecation warning -- has led to temper-tantrums in the CL
> community...
>
> On 21 Feb 2024, at 22:14, sc...@sympoiesis.com wrote:
>
> Hi all! I just ran into something surprising. This is with ASDF 3.2.1,
> packaged with Quicklisp. I am using Named-Readtables. I had '*readtable*'
> set to a nonstandard readtable, then did quickload of a system unrelated to
> the one that defines and uses that readtable. The compilation failed; after
> I did '(in-readtable :common-lisp)' and tried again, it succeeded.
>
> A quick glance at the ASDF source code shows that it binds '*readtable*'
> to a standard readtable in some cases, such as to read a '.asd' file, but
> not in 'uiop/lisp-build/compile-file*', nor in
> 'asdf/lisp-action:perform-lisp-compilation'. Wouldn't it make sense to do
> that?
>
> -- Scott
>
>

Re: historical question

[Faré]

On Thu, Feb 1, 2024 at 12:35 PM Robert Goldman  wrote:
> I'm open to an ASDF history section being added to the manual, or a separate 
> ASDF history web page being added that would be linked from index.html.
>
> But I'm not open to doing this myself! I would accept a PR.
>
> Actually, isn't this information available in some of the ILC papers? If so, 
> isn't that sufficient?

A historical writeup is right up my alley. I think it can bind
together the information that was in the ILC papers.

Is a doc/history.md page alright, similar to the best_practices.md document?

Should I remove the information from the index.html page as I put it
in history.md, leaving a link to the new document instead? I'm
thinking particularly of the pseudo-changelog at the bottom of the
page, but also part of the documentation section, and the discussion
of obsolete alternatives (a lot of it currently commented out, but I
intend to move it all to the new document and uncomment the stuff
there, maybe in a different subsection).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“Each party only has to LOOK better than the other before the vote.
After, it can BE worse. Then the other only has to compete with THAT.
Lather, rinse, repeat...”

Re: historical question

[Faré]

I think we should add this link, as well as a link to the original
README in git (which I suppose subsumes the original defsystem
proposal link in the archived email) somewhere in doc/index.html.

Looking at said doc/index.html, I see that a lot of references to
historical ASDF alternatives were commented out—why not keep them, if
only in a separate paragraph or section? Or move them somewhere deep
in the manual? I believe they have historical value, too.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“Men never do evil so completely and cheerfully as when they do it
from religious conviction.”  ― Blaise Pascal (1623–1662)
On Thu, Feb 1, 2024 at 9:34 AM cage  wrote:
>
> On Thu, Feb 01, 2024 at 10:38:29AM +0100, Rudolf Schlatte wrote:
>
> Hi!
>
> [...]
>
> > If you need a definitive answer, you could ask Dan Barlow, he's on the
> > fediverse at @d...@brvt.telent.net
>
> The funny thing  is that I realized  that the person I  talked  with on
> the fediverse in the last days, is Him! 
>
> He  provided a  link  that I  pasted  below because  I  think is  very
> interesting for an historical perspective of ASDF development.
>
> https://sourceforge.net/p/cclan/mailman/message/2011712/
>
> Bye!
> C.

Re: historical question

[Faré]

The initial revision of ASDF was only 418 lines long, compared to 14130 today.
The key ideas of ASDF is entirely present in those 418 lines—even though
in many ways, the algorithm used is unredeemably buggy (see the ASDF 3 paper
and/or my blog posts on the matter).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“The only difference between a military and a terrorist often is that the
former lives off taxes taken by force from the population, whereas the latter
only aspires to it.”

On Wed, Jan 31, 2024 at 12:08 PM Faré  wrote:
>
> The initial commit in git is:
>
> commit c58257d94bfae7d5a2f527fc1f2d587a0fef8222
> Author: Daniel Barlow <>
> Date:   Wed Aug 1 17:52:49 2001 +
>
> Initial revision
>
> If you want to find out when he actually started working on it, you
> should ask Daniel which day he started working on it—or maybe dig the
> archives of some old forums (sbcl-devel? comp.lang.lisp?).
>
> In any case, you'll have to wait two more years for the 25th
> anniversary of ASDF. But this year, you can celebrate the 11th
> anniversary of ASDF 3! Wow. Time flies.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> “I have never killed any one, but I have read some obituary notices with great
> satisfaction.”  — Clarence Darrow
>
>
> On Wed, Jan 31, 2024 at 11:33 AM cage  wrote:
> >
> > Hi!
> >
> > Some  person  (including  me)  was wondering  when  the  ASDF  project
> > started; on  the fediverse someone  guessed that the  25th anniversary
> > could be near. Is that true?
> >
> > Bye!
> > C.

Re: historical question

[Faré]

The initial commit in git is:

commit c58257d94bfae7d5a2f527fc1f2d587a0fef8222
Author: Daniel Barlow <>
Date:   Wed Aug 1 17:52:49 2001 +

Initial revision

If you want to find out when he actually started working on it, you
should ask Daniel which day he started working on it—or maybe dig the
archives of some old forums (sbcl-devel? comp.lang.lisp?).

In any case, you'll have to wait two more years for the 25th
anniversary of ASDF. But this year, you can celebrate the 11th
anniversary of ASDF 3! Wow. Time flies.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“I have never killed any one, but I have read some obituary notices with great
satisfaction.”  — Clarence Darrow


On Wed, Jan 31, 2024 at 11:33 AM cage  wrote:
>
> Hi!
>
> Some  person  (including  me)  was wondering  when  the  ASDF  project
> started; on  the fediverse someone  guessed that the  25th anniversary
> could be near. Is that true?
>
> Bye!
> C.

Re: Request for assistance: GitLab actions

[Faré]

Would it be that the images were created with the exact same
Dockerfile recipe in different environments? Docker could be doing the
caching wrong because it ignores the environment when caching. A
solution is to modify at least one byte in the Dockerfile—when I
generate Dockerfile's, I include a sha256 of the environmental
contents as a value to be printed while generating the image.

Then again, yours could be a completely different problem.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“Once upon a time, there was a person whose life was so good
there was no story to tell about it.”  — Foer


On Fri, Jan 26, 2024 at 12:03 PM Robert Goldman  wrote:
>
> I have created a new docker image for Allegro CL version 11.0 (express/free), 
> and put it on DockerHub: 
> https://hub.docker.com/repository/docker/rpgoldman/allegro11express/general
>
> I have tried to add regression and upgrade tests using this Docker Image -- 
> see Regression test and Upgrade test.
>
> Unfortunately, these tests don't work! They print the right information at 
> the head of the job:
>
> Regression test: [allegro, rpgoldman/allegro11express, modern] :
>
> But they pull the wrong docker image:
>
> Using Docker executor with image 
> containers.common-lisp.net/cl-docker-images/allegro:latest ...
> Authenticating with credentials from job payload (GitLab Registry)
> Pulling docker image 
> containers.common-lisp.net/cl-docker-images/allegro:latest ...
> Using docker image 
> sha256:d0fc75f61208fa39497f5ae2188d6d70d4b65ae513105dbdf80f85b41a45ffdc for 
> containers.common-lisp.net/cl-docker-images/allegro:latest with digest 
> containers.common-lisp.net/cl-docker-images/allegro@sha256:e16f3a55a2d829732a438c363dc6905754c46ca595e868377ed61faeeda44fdb
>  ...
>
> Does anyone on the list understand GitLab pipelines well enough to tell me 
> why my image variable binding in "Regression test" and "Upgrade test" doesn't 
> clobber the value from the template? Or, even more oddly, why it seems to 
> half-way clobber it, giving the right info print, but pulling the wrong image?
>
> Sorry to bother the list, but I'm at my wits end and the debugging tools and 
> documentation are somewhere between horrible and non-existent.
>
> Thanks for any advice,
>
> R

Re: Past discussion on ASDF loading

[Faré]

Do you have multiple versions of ASDF installed? e.g. one from Allegro and
another from your source-registry or system-registry ? ASDF always upgrades
itself first thing to avoid Big Problems otherwise.

If that's what you're experiencing, I recommend the right after you
(require "asdf") and configure it (if you do), you should explicitly
(asdf:upgrade-asdf) with whatever warning need be muffled, and then you'll
be good.

If there's more than that, please send instructions to reproduce the issue.

-#f

On Tue, Jan 2, 2024, 16:47 Robert Goldman  wrote:

> I am finding that ASDF loads itself multiple times, unnecessarily, at
> least on Allegro 11. E.g., it reloads when I simply invoke 
> (asdf:component-pathname
> (asdf:find-system "foo")) for one of my systems.
>
> This is annoying because I get redefinition style warnings every time I do
> this, which makes it hard to tell when a system loads cleanly.
>
> I have a vague memory of a discussion that addressed the question of why
> this happens. Does anyone recall the answer, or have a pointer to the
> discussion?
>
> Interestingly, when I load ASDF and then invoke 
> (asdf/component:component-operation-times
> (asdf:find-system "asdf")) the only time listed is for define-op
>
> Thanks,
> R
>

Re: Removal of feature from UIOP library

[Faré]

That's probably my bad for never completing and testing this feature.

IIRC, it was originally meant to ensure that the optimization settings
used by ASDF should not be affected by those used outside of it, nor
affect them, so as to avoid weird side-effects wherein the behavior of
a system depends on which other systems you did or did not build
before it, which was causing "interesting" effects at times.

No, I'm not volunteering to fix it—I haven't actively used CL in years
(apart from my stumpwm configuration and some non-shell scripts I
haven't rewritten in Gerbil Scheme yet).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“The real danger is that one day machines *will* become intelligent,
but we’ll lack the mental equipment to notice.”
— Tiresias, in J.-P. Petit, “Run, Robot, Run”


On Tue, Dec 12, 2023 at 11:44 AM Robert Goldman  wrote:
>
> Issue 48 on the ASDF GitLab repo concerns a bug in the 
> with-optimization-settings macro in UIOP's lisp-build.lisp
>
> I did some investigation and as far as I can tell:
>
> There's no clear description of what this is supposed to do
> There's no use of it anywhere in the ASDF + UIOP code base
> It has never worked -- at least when I tried to test it, the tests failed all 
> over the place.
>
> Rather than trying to write a new implementation out of whole cloth, my Merge 
> Request to address this issue simply rips this macro and some supporting 
> functions and variables out of UIOP. Doing so breaks no tests (unlike trying 
> to test it).
>
> I feel a little uncomfortable removing functionality, but as I said, I cannot 
> find any evidence that this has ever been used.
>
> Absent any pushback -- and likely even with pushback, unless that pushback is 
> accompanied by a MR that supplies a working implementation, I will make that 
> merge, and this feature will vanish from UIOP.
>
> Best,
> R

Re: ASDF config file under Windows

[Faré]

If you save an image without using uiop's save-image and/or calling its
hooks, then you'll save the ASDF configuration from the time you build your
image. I've seen many people do that (e.g. stumpwm). Sad.

On Sun, Jun 4, 2023, 15:20 Robert Goldman  wrote:

> Ah, I see. I think some of my work colleagues were doing something in
> their CL init files that must have loaded some ASDF systems, because they
> were seeing configuration happen before they got into interactive contact
> with the REPL.
>
> Thank you very much for clarifying, Faré!
>
>
> On 4 Jun 2023, at 14:11, Faré wrote:
>
> ASDF configuration is lazy. It doesn't happen during load, but the first
> time ASDF is invoked, if not done by the user after loading ASDF.
>
> On Sun, Jun 4, 2023, 15:00 Robert Goldman  wrote:
>
>> TBQH, I am not as sure as I should be. I try to avoid all the
>> configuration that happens while ASDF loads, so that I configure only
>> *after* the load is complete, at a point where I can watch what goes on.
>> As I have said before, I don't like the configuration that uses config
>> files, environment variables or magic directories, because it's impossible
>> to debug (by the time ASDF is loaded and can be instrumented, it's too
>> late).
>>
>> I believe the right thing would be for you to figure out how to add an
>> entry to the variable asdf:*default-source-registries* that is a
>> function that will collect the appropriate configuration information. But I
>> don't claim to fully understand source-registry.lisp
>> <https://gitlab.common-lisp.net/asdf/asdf/-/blob/9c7f1b76ca8e820c020a3ec3aeb4b507ab15aa34/source-registry.lisp>
>> .
>>
>> On 4 Jun 2023, at 3:29, Marco Antoniotti wrote:
>>
>> Sure.
>>
>> I am not privy to ASDF bootstrap.  Where would you start?
>>
>> MA
>>
>>
>> On Sun, Jun 4, 2023 at 12:41 AM Robert P. Goldman 
>> wrote:
>>
>>> The next step would be to add an ASDF plug-in so that ASDF can read its
>>> configuration from a CLAD set up…. :-)
>>>
>>> --
>>> Robert P. Goldman
>>>
>>> On June 3, 2023 at 16:56:53, Marco Antoniotti (
>>> marco.antonio...@unimib.it) wrote:
>>>
>>>> We.  You need a CL to load ASDF, don’t you? :)
>>>>
>>>> CLAD is minimal and barebones.  You can load it from two files (just
>>>> because the DEFPACKAGE is separate) and it’s there.  Look ma! No ASDF! :)
>>>> (Or UIOP)
>>>>
>>>> But I see your point.  Time to add a clad-setup.lisp file.
>>>>
>>>> Cheers
>>>>
>>>> MA
>>>>
>>>>
>>>>
>>>> On Sat, 3 Jun 2023 at 23:10, Robert Goldman 
>>>> wrote:
>>>>
>>>>> I do like the idea of CLAD, but in this case isn't there a Catch-22?
>>>>> You need to configure ASDF in order to load CLAD, so you can't use CLAD to
>>>>> configure ASDF, can you?
>>>>>
>>>>> On 3 Jun 2023, at 10:57, Marco Antoniotti wrote:
>>>>>
>>>>> Ahem.  Shameless plug: https://gitlab.common-lisp.net/mantoniotti/CLAD
>>>>>
>>>>> Cheers
>>>>>
>>>>> MA
>>>>>
>>>>>
>>>>> On Sat, Jun 3, 2023 at 5:28 PM Robert Goldman 
>>>>> wrote:
>>>>>
>>>>>> Dear Greg,
>>>>>>
>>>>>> Thanks for the kind words! I am not sure why I didn't see your post
>>>>>> on ASDF-devel, but I didn't. And I have seen several requests for
>>>>>> moderation, so the list seems live. In case my response is interesting to
>>>>>> the group, I am responding to the list as well.
>>>>>>
>>>>>> A couple of disclaimers: (1) I haven't used Windows in more than a
>>>>>> decade, and (2) I don't use these configuration files. I find I'm happier
>>>>>> to keep all of my lisp configuration in lisp configuration files (
>>>>>> clinit.cl, .sbclrc, etc. -- indeed I point all of those at one
>>>>>> single lisp-config.lisp file), instead of having to hunt through
>>>>>> multiple files in multiple locations for this information. So take 
>>>>>> anything
>>>>>> I say with more than a grain of salt (given your cardiologist approves!).
>>>>>>
>>>>>> That said, these config files are placed according to the XDG
>>>>>> standard which... as far as I 

Max version constraints

[Faré]

Dear Robert & asdf-develers,

I was discussing max version constraints and the havoc they wroke on
Haskell land, yet the legitimate use they have in toplevel projects
integrating many libraries, and figured that maybe the solution could
be a difference in treatment between max version constraints found in
a toplevel asd file (the one you directly call operate on) vs ones
found in subsequently loaded .asd files: a broken constraint at the
toplevel would cause an error, whereas a constraint broken outside the
toplevel would only issue a warning—a warning that would suggest
either removing the constraint (because its value outside the toplevel
is dubious or advisory only) or updating it (since the toplevel works
without it), or adding it to the toplevel too (because the toplevel
doesn't actually work without it). The warning can be hushed the usual
way if desired, and both the warning and error would inherit from a
common condition.

Technically, to implement this, the main methods of load-asd and
operate would bind a special variable to the path of the .asd file (or
its truename* ?) if the binding is currently NIL, and the constraint
checker would behave differently depending on whether the
component-path of the current system matches it or not. (This is a
rough idea that has not been tested.)

Would this solution work for you? It seems to me that it should make
everyone happy.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“All problems in computer science can be solved by another level of
indirection.”  — David Wheeler

Re: ADG: handling functions defined after a reference

[Faré]

IIRC, at ITA we loaded all the source code of QPX as source code in a
first pass before we compiled it, to get all the functions resolved
(happily, there weren't any macro-defining macros requiring more than
one level of such loading), due to the code having been defined
without such a clean build in mind. I don't remember for sure, but I
suspect we did that also when we used ADG on that code-base to detect
the dependencies. Maybe we even ran ADG twice? In the end, we also
generated fine-grained dependency reports so we could later split the
files in multiple parts that had a proper dependency order, and even
automatically did the file-splitting based on dependencies with
toplevel-form granularity. (The output from this project was dropped
for political reasons, though, sigh).

Sorry, my memories are growing dim, and I don't have access to this
codebase anymore. I might know people who have access and might be
able to dig it out (or then again might not—the code might have been
archived with the old revision control system when ITA was absorbed by
Google, and no one is going to get paid to fight bureaucracy to get to
ancient code; and it probably was in a branch which will make
archaeology harder).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
“In reviewing the history of the English Government, its wars and its taxes,
a bystander, not blinded by prejudice nor warped by interest, would declare
that taxes were not raised to carry on wars, but that wars were raised
to carry on taxes.”  — Thomas Paine, “Rights of Man”




On Sat, Dec 31, 2022 at 12:43 AM Robert Dodier  wrote:
>
> Hi, thanks for your help so far.
>
> I have noticed that asdf-dependency-grovel doesn't handle functions
> which are defined after a reference. E.g. suppose a file says (defun f
> () (g)) and G hasn't been defined yet. Then a file that's loaded
> afterwards says (defun g ()). If the loading order of the files were
> reversed, ADG would detect the dependency, but as it stands, ADG sees
> that there is a new symbol G but doesn't output a dependency for that.
>
> It seems like the problem might be originating from dependencies being
> detected at the time that the reference is detected. It seems like if
> the definitions and references were all generated, and then all of the
> references matched with definitions after everything is loaded, then
> ADG would detect out-of-order references. Just guessing there.
>
> Do you see an easy way to get ADG to detect out-of-order dependencies?
> The reason I ask is that the practical example I am interested in
> (Maxima) has a lot of those.
>
> Thanks for any insights,
>
> Robert

Re: asdf-dependency-grovel tests mostly passing now

[Faré]

Congratulations for resurrecting ADG.

> I wonder why the appropriate dependency wasn't detected, but I'm not
> too worried about it. ADG is still useful even if it misses these.
>
There is no guarantee when the types defined by deftype are or aren't
going to be expanded and checked, unless you explicitly (eval-when
(:compile-toplevel :load-toplevel :execute) ... (typep my-value
my-type) ...)
Implementation-dependent optimization settings might affect type checks.

> There are various questionable aspects of that, which I'll point out,
> although they don't really bother me too much -- I'm not responsible
> for ADG.
>
You've been the most responsible person for ADG in the last 10 years.

> (1) I was able to get ADG working only with this particular
> combination of SBCL (2.1.6) and ASDF (3.1.5). There might be other
> workable combinations, however, some or all later versions of ASDF
> trigger errors in ADG.
>
It *should* work all the way to ASDF 3.2.1. For 3.3.0 and later,
you'll need to refactor things to support DEFINE-OP or probably, a way
to instrument OPERATE rather than modify either operation or component
classes.

> (3) It appears the handlers file is supposed to be loaded by ASDF,
> which, I guess, is supposed to scan the handlers directory, but
> something in that process didn't work, so it's loaded by hand.
>
The code in ADG hails from the ASDF 1 era. It might be possible to fix
the grovel-handlers code—but it's probably best to junk it.

> ADG constructs a new .asd file from the inferred dependencies (and
> writes it into the ASDF output cache -- maybe that could be mentioned
> in a readme or something) and I present it here as a kind of trophy.
>
Making ASDF work with the output from ADG on a thousand-file project
is what started me fixing significant ASDF bugs and ultimately
becoming ASDF maintainer and writing ASDF 2 and 3. Notably: ASDF 1
running time was cubic in the number of components, leading to minutes
to process the generated file, instead of a fraction of a second.

—♯ƒ • François-René Rideau • Co-Founder and President, MuKn.io
“Time is really the only capital that any human being has and the thing that
he can least afford to waste or lose.”  — Thomas A. Edison (1847–1931)

Re: progress on asdf-dependency-grovel

[Faré]

> I tried it again with ASDF 3.3 + SBCL 2.1.1, and I can load adg
> successfully, with a couple of (push "path/to/adg/"
> asdf:*central-registry*) to help it along.
I strongly recommend using the ASDF2-era source-registry instead.

> However, when I try to run
> the unit tests, I get an error from ASDF about "failed to mark it
> done". There is also a warning about "proper dependencies" as well.
>
I see that the way that perform calls load-instrumented-systems that
calls load-asd is very incompatible with asdf 3.3. I don't understand
what this whole thing is about, but it smells very wrong to me.
Presumably, these shenanigans should instead be replaced by some
instrumentation of define-op (that failed to exist before asdf 3.3).
And dependency-op should possibly be also replaced by the same
instrumentation of load-source-op.

>Deprecated recursive use of (ASDF/OPERATE:OPERATE 'ASDF/LISP-ACTION:LOAD-OP
>'("test-serial-system")) while visiting
>(ASDF-DEPENDENCY-GROVEL:DEPENDENCY-OP "test-serial"
> "asdf-dependency-grovel-test/serial")
>- please use proper dependencies instead
>
Yup, this is very incompatible with ASDF 3.3, that both offers and
requires much better ways to do things.

—♯ƒ • François-René Rideau • Co-Founder and President, MuKn.io
“The fool considers that what he doesn't understand is either extremely stupid,
or extremely intelligent, pending on how others react to it.”  — Nassim Taleb

Re: progress on asdf-dependency-grovel

[Faré]

> I tried a later version (3.2 or 3.3, I forget which) and adg fails to
> load. I think I'll stick w/ the last known working version until I
> sort out getting it to run ...
>
It loads perfectly for me:
sbcl --eval "'(#.(require :asdf) #.(in-package :asdf) #.(upgrade-asdf)
#.(load-system :asdf-dependency-grovel))" --quit
No error.

>> If that still fails, rather than the asdf plumbing itself, can you
>> identify if the dependency instrumentation is still working on the
>> latest SBCL?
>
> How would I determine if the dependency instrumentation is working? It
> is not working in the sense that the known dependencies are not
> detected -- do you mean something else?
>
See if fine-grain-instrumented-load works with the expected
side-effects, emitting the suitable signal-user, signal-provider, and
other signal-* events.

At this point, you have to read the source code. I don't believe
anyone is left to help you. But if you have specific questions, I can
read the source code too and maybe remember more context. (NB: I
didn't write that code, but I oversaw an intern who used it, and
tweaked it a little bit to package it better.)

> Okay, there is a warning about an internal SB-something function --
> I'll look at whether the relevant code can be updated.

Again, good luck, and happy holidays!

—♯ƒ • François-René Rideau • Co-Founder and President, MuKn.io
“If all values are relative, then cannibalism is a matter of taste.”
— Leo Strauss

Re: progress on asdf-dependency-grovel

[Faré]

On Fri, Dec 16, 2022 at 2:08 AM Robert Dodier  wrote:
>
> I've managed to get asdf-dependency-grovel to load. There is a comment
> in adg/tests/grovel-test.lisp that ASDF 3.1 is required. Okay, I have
> ASDF 3.1.5 laying around so I loaded that. Now SBCL can load adg.
>
Or later. I recommend testing with the latest ASDF.

> The test script adg/tests/run-tests.sh runs (with modifications) but
> all of the tests fail -- it appears that no dependencies are
> identified. I enabled *DEBUG-TRACE* in variables.lisp, and I get some
> output which suggests something is happening, which is encouraging.
> Faré, can you say anything about what one should expect to see in the
> debug output if stuff is working correctly? Can you see anything amiss
> in the output shown below?
>
This output doesn't help me. Are the files now visited by the perform
method in order? It looks like that might be the case, but make sure.

If that still fails, rather than the asdf plumbing itself, can you
identify if the dependency instrumentation is still working on the
latest SBCL? IIRC, adg relies on some low-level SBCL interfaces that
might have bitrotten.

Sorry, I haven't used that in production in over ten years and have no
time to dive into CL code at the moment.

Good luck!

—♯ƒ • François-René Rideau • Co-Founder and President, MuKn.io
“Welfare: when you love strangers so much you're willing to have government
steal money from another stranger to help them out.” http://bit.ly/ahVPBj

Fwd: Lisp file and/or ASDF dependency analysis; trying to load asdf-dependency-grovel

[Faré]

-- Forwarded message -
From: Faré 
Date: Thu, Dec 15, 2022, 04:02
Subject: Re: Lisp file and/or ASDF dependency analysis; trying to load
asdf-dependency-grovel
To: Robert Dodier 


asdf-dependency-grovel predates asdf 2, and probably bitrotted with asdf 3.
My guess is that its dependency-op needs a prepare-dependency-op the same
way that load-source-op has a prepare-load-source-op.
The entire class hierarchy could probably be lifted the same way, except of
course for the perform method.

Sorry I have no time to dig into the code. Hope this helps.


On Thu, Dec 15, 2022, 02:08 Robert Dodier  wrote:

> Hi, I'm interested in analyzing dependencies among a collection of
> Lisp files, or analyzing dependencies stated in a .asd file, or both.
>
> (1) Towards the goal of analyzing dependencies among Lisp files, I'm
> trying to load asdf-dependency-grovel, and I'm not having any luck. I
> obtained a tar.gz of the current version from Gitlab, from the project
> https://gitlab.common-lisp.net/xcvb/asdf-dependency-grovel. I found it
> necessary to patch tests/run-tests.sh to get anywhere (patch below)
> and I get the following error at this point:
>
> 
> $ sh tests/run-tests.sh
> This is SBCL 2.1.6, an implementation of ANSI Common Lisp.
>
> WARNING: redefining UIOP/UTILITY:PARSE-BODY in DEFUN
> WARNING:
>Deprecated recursive use of (ASDF/OPERATE:OPERATE
> 'ASDF/LISP-ACTION:LOAD-OP
>'("test-serial-system")) while visiting
>(ASDF-DEPENDENCY-GROVEL:DEPENDENCY-OP "test-serial"
> "asdf-dependency-grovel-test/serial")
>- please use proper dependencies instead
>
> debugger invoked on a SIMPLE-ERROR in thread
> #:
>   Just performed compiling # "test-serial-system" "package"> but failed to mark it done
> 
>
> Can anyone offer any advice about the "failed to mark it done" error?
> or the bit about "deprecated recursive use"?
>
> (2) About looking at dependencies stated in a .asd file, I have a .asd
> file which ASDF is refusing to load because it has a circular
> dependency. Is there a way to get ASDF to tell me what that circular
> dependency is? Failing that, can someone suggest any tools to extract
> a directed graph from a .asd file? I suppose it wouldn't be too hard
> to do it myself but no need to reinvent the wheel.
>
> I'm thinking I could find a graph analysis library to extract any
> cycles, if I had a directed graph in hand. I don't suppose someone has
> already automated looking for cycles in .asd files?
>
> Thanks in advance for any advice, I appreciate your help.
>
> best,
>
> Robert Dodier
>
> PS. Here's my patch for asdf-dependency-grovel.
>
> $ diff -u tests/run-tests.sh-original tests/run-tests.sh
> --- tests/run-tests.sh-original 2022-12-13 22:39:50.174826971 -0800
> +++ tests/run-tests.sh 2022-12-13 22:43:41.675974917 -0800
> @@ -5,7 +5,10 @@
>
>  ADG_TEST_DIR="$(dirname $0)"
>
> -$LISP --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval '(uiop:quit
> (asdf-dependency-grovel-tester:test-result))'
> +$LISP --eval '(require (quote asdf))'\
> + --eval '(push
> "/home/robert/by-others/asdf-dependency-grovel-master/"
> asdf:*central-registry*)'\
> + --eval '(push
> "/home/robert/by-others/asdf-dependency-grovel-master/tests/"
> asdf:*central-registry*)'\
> + --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval '(uiop:quit
> (asdf-dependency-grovel-tester:test-result))'
>  STATUS=$?
>  rm -rf asdf-dependency-grovel-tmp-*
>  #$LISP --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval
> '(asdf-dependency-grovel-tester:check-base-deps)' --eval '(uiop:quit)'
>
>

Re: Define a Simple Echo-Op

[Faré]

>
> > ASDF as it exists now is not capable of running anything in parallel.
> Faré has an experimental project to do this, called POIU, but I
> > can't speak for its status. Until very recently, SBCL was incapable of
> compiling files in parallel, and still today, lots of code does
> > not-thread-safe stuff at load-time, so I'm not convinced it's
> practically possible to make ASDF parallel via shared-memory threaded
> > concurrency. I believe POIU compiles each component in a separate
> process, then loads them sequentially.
>
> I see, that's good to know. At least it's an implementation limitation
> and not a design limitation I guess.
>
POIU compiles files in a fork, on supported implementations that it knows
how to fork on (SBCL, CCL with a patch, CLISP, ACL). It used to work well,
but it wasn't updated to take into account the new planning infrastructure
of ASDF 3.3, and requires some love. It looks like it's still working, but
it's probably not doing all the right things across build phases. Good luck
untangling it.

-#f

Not watching asdf anymore

[Faré]

Dear ASDF developers,

just writing this email to tell you that I'm no longer watching ASDF's
gitlab, and at some point I may stop reading this mailing list. I'm
happy to see that ASDF is in good hands, but the current activity is
already too much for me. Between running a company and having jumped
ship to Gerbil Scheme, I have stopped caring enough about CL and ASDF.

If you have questions about why I did things a particular way in ASDF,
or how to resolve some conundrum, I'll be happy to reply to questions,
but then be sure to send them to my personal email.

Happy hacking!

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The meta-Turing test counts a thing as intelligent if it seeks to apply
Turing tests to objects of its own creation.
— Lew Mammel, Jr.

Re: Rejiggering the branches

[Faré]

On Tue, Jul 13, 2021 at 9:34 AM Attila Lendvai  wrote:
>>
>>
>> Would the "stable" branch be any different from the "release" branch?
>> If it's actually a not-so-stable development branch for 3.3 while a
>> separate branch contains development for 3.4, then maybe indeed
>> calling branches v3.3 and v3.4 make more sense.
>>
>
> +1
>
> what i would do:
>
> one branch that holds the bleeding edge. i'd call it main, just to go with 
> the flow.
> branches for ASDF versions (down to the desired resolution, probably 
> major.minor), so that you can easily cherry pick or backport fixes into them. 
> a new version-branch is forked off of main whenever a release happens.
> optionally a stable *tag* as an indirection to the latest release. it 
> communicates which specific git revision is it that the maintainer considers 
> the stable state at any moment in time. it comes handy e.g. in CI scripts 
> that want to check out the latest ASDF release, etc...
>
> note though that this last point requires force-pushing the stable tag, which 
> i have done before, but i'm not completely sure it results in a slick 
> workflow. the main question is whether or not a git fetch/pull silently and 
> automatically updates the tag in the local repo.
>
Nah, a tag is supposed to never change. The mechanism for a "tag that
changes" is called... a branch. And that's precisely why we have a
release branch. And yes, if we have version-based branches v1, v2,
v3.0, v3.1, v3.2, v3.3, v3.4... then it makes sense to force-push to
the release branch when the release "jumps" from v3.3 to v3.4... or to
"merge" the latest release into master or main before to make a new
release off of it so there is no need for force.

—♯ƒ • François-René Rideau • Co-Founder and CEO, MuKn.io
Love thy neighbor as thyself, but choose your neighborhood.
— Louise Beal

Re: Rejiggering the branches

[Faré]

Would the "stable" branch be any different from the "release" branch?
If it's actually a not-so-stable development branch for 3.3 while a
separate branch contains development for 3.4, then maybe indeed
calling branches v3.3 and v3.4 make more sense.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The knowable universe is everything, as far as we can know.

On Mon, Jul 12, 2021 at 2:13 PM Martin Simmons  wrote:
>
> > On Mon, 12 Jul 2021 19:52:01 +0200, Rudolf Schlatte said:
> > Cancel-Lock: sha1:dqYu7Py9JNAyZJWALyW1kLx3PD8=
> >
> > "Robert Goldman" 
> > writes:
> >
> > > If stable seems bad, is there another name we could use to avoid 
> > > renaming? Like maint for "maintenance"?
> > >
> > > I don't love maint, because it's too close to main, and it seems like 
> > > main has an edge in familiarity if not in meaningfulness.
> > >
> > > legacy?
> > >
> > > Unless we can come up with something better than stable, it seems like 
> > > the least-worst alternative. But there's all week to come up with 
> > > something better!
> > >
> >
> > In the first email you said that the purpose of that branch was to
> > permit continuation of the 3.3 release series, so maybe call the branch
> > "v3.3"?  That way, there can be multiple such branches without resorting
> > to "stable", "oldstable" etc. names.
>
> Yes, that's the kind of name I meant.
>
> Or include the stableness in the name with something like "stable/3.3"
> (c.f. FreeBSD).
>
> --
> Martin Simmons
> LispWorks Ltd
> http://www.lispworks.com/
>

Re: System cleanup

[Faré]

Considering that SBCL upgraded to 3.1.5 in July 2015, I think you
should be pretty safe assuming that your users' ASDF is more recent
than 3.1.4.

What more, even Xach seems to have miraculously seen the light: one
months and one week ago, he updated the fallback ASDF in Quicklisp
from 2.26 to 3.2.1! That was the first update of ASDF in Quicklisp
since 2012, to a version then-current, to a version 4 years old. By
linear extrapolation, we should expect the next update to happen in
2030, to a version from 2022. I admit that I was thinking his next
move would be to downgrade ASDF to 1.369 (the last version before I
took over).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
To avoid criticism, do nothing, say nothing, and be nothing. – Elbert Hubbard

On Thu, Mar 18, 2021 at 1:00 PM Didier Verna  wrote:
>
>
>   Hi,
>
> I would like to remove some old workaround code from a couple of
> systems, wrt to ASDF 3.1.4 bugs (see below). Is it safe to do so now, or
> are these "bugs" still lurking around?
>
> The relevant parts are as follows:
>
>   :defsystem-depends-on
>   (:net.didierverna.clon.setup/termio
>#+sbcl ;; BUG in ASDF 3.1.4: d-d-o can't deal dependency expanding to NIL
>(:feature :sbcl (:require :sb-grovel))
>#+(or allegro clisp lispworks)
>(:feature (:or :allegro :clisp :lispworks) :cffi-grovel))
>
> and
>
>   :components (;; bug in ASDF 3.1.4: cannot deal with conditionally defined
>;; component class!
>#+sbcl (sb-grovel:grovel-constants-file "sbcl/constants"
>:package :net.didierverna.clon :if-feature :sbcl)
>#+(or allegro clisp lispworks)
>(:cffi-grovel-file "cffi/constants"
> :if-feature (:or :allegro :clisp :lispworks))
>
> Thank you!

Re: A CFFI -ASDF integration bug

[Faré]

I suppose this is what the previous setup was trying to avoid, but the
perform method on load-source-op could compile the .so file if needed.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Corollaries to the Law of Bitur-Camember: The political process destroys the
value of all known resources that are up for grabs. The socialist process of
systematically denying legitimacy to property rights applies the political
process universally and destroys the value of all available resources.


On Thu, Aug 27, 2020 at 2:34 PM Ilya Perminov  wrote:
>
> I finally got some time to work on this and found a complication -
> load-source-op logically depends on generated .so files. Currently
> process-op produces .so files and load-source-op depends on it. If I
> move .c to .so compilation from process-op to compile-op,
> load-source-op will have to depend on compile-op somehow, which is
> obviously wrong.
>

Re: A CFFI -ASDF integration bug

[Faré]

On Tue, Aug 4, 2020 at 6:13 PM Ilya Perminov  wrote:
> The files are the same. compile-op does not touch them at all. They
> are just its fake output-files. Is it a good idea to compile a .c to
> .o/.so in a compile-op? Its doc string says ""Operation for compiling
> a Lisp file to a FASL".
I don't see why there's any issue compiling a .c to a .o in a
compile-op, the operation is for compiling in general. Maybe the doc
string can be amended to "Operation for compiling source files or
preprocessed source files to object files, fasls, etc.".

If they are not otherwise needed at a previous step, I would compile
the files during compile-op, as a defmethod perform :before (or
:after) on the suitable class. If needed at a previous step, I would
copy them to a different filename.

Apologies for likely being the author of this bug.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Social peace will come when the socialists will love the poor
more than they hate the rich. — Not Golda Meir

Re: A CFFI -ASDF integration bug

[Faré]

On Fri, Jul 31, 2020 at 1:18 PM Ilya Perminov  wrote:
> As a result inputs and outputs of the ops look like this:
> process-op:
>   input: wrapper-file
>   output: bindings-file.lisp file.c FILE.O FILE.SO
>
> compile-op:
>   input: bindings-file.lisp
>   output: bindings-file.fasl FILE.O FILE.SO
>
Are these FILE.O and FILE.SO two different files with the same name,
or the same file reported twice?

In BOTH cases, you'll want to have DIFFERENT NAMES for the two sets of
outputs. If the files are the same, then either you should compile
them only during the compile-op, or, if not possible, you should copy
the output to the new file with a different name.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The world is a comedy to those that think; a tragedy to those that feel.
— Horatio Walpole, 4th Earl of Orford (1717–1797)

Re: A CFFI -ASDF integration bug

[Faré]

On Tue, Aug 4, 2020 at 1:03 PM Ilya Perminov  wrote:
>
> I think protobuf and CFFI structure their operations in a very similar way - 
> process-op is analogous to proto-to-lisp, it takes a "specification" file and 
> generates a lisp file and some other files. protobuf generates lisp(fasl) 
> files only, so it does not need to do anything special to support bundle 
> operations. CFFI's process-op generates some .o and .so files that a  bundle 
> operation may need. The current implementation adds .o and .so files to 
> outputs of compile-op and it causes the problem I described.
> I do not know what methods need to be defined on process-op to make bundle 
> operations to pick up its output files. From my very limited understanding of 
> ASDF I do not think there is a way to do it. Method "component-depends-on ((o 
> gather-operation) (s system))" determines input-files of a bundle-op. The 
> method returns dependencies of one operation only (e.g. compile-op), but in 
> case of CFFI's wrapper-file we need output files of two operations: 
> process-op and compile-op.
>
The intended design is that you only take the output of compile-op,
and that process-op is implicitly involved as a dependency to compute
said output, but that process-op's own outputs are not included. If
process-op does the compilation and you want its outputs, you lose.

That said, I haven't looked at the code recently, nor do I intend to,
being too busy trying to make my startup survive and feed my family.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A word for the epoch of free software and universal publishing:
voluntocracy n 1. governance by those who do the work. 2. the volunteers
who do the work. — Aubrey Jaffer, http://swissnet.ai.mit.edu/~jaffer/

Re: A CFFI -ASDF integration bug

[Faré]

I'm not developing ASDF anymore (unless for hire) but I believe the
CFFI toolchain has a new maintainer, who might be willing to devote
cycles to that (or at least to merging a patch to CFFI).

Note that if the code that builds stuff and the code that tracks the
dependencies disagree, the right solution is to make the dependencies
match the actual build, not to make it lie better: make sure that
files are attributed as the output-files of the action that creates
it, and as the input-files of the other actions.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Due to circumstances beyond your control, you are master of your fate
and captain of your soul.



On Fri, Jul 31, 2020 at 1:18 PM Ilya Perminov  wrote:
>
> Hi,
>
> I found a ASDF-related CFFI bug a couple of days ago. Can anyone think of a 
> good way of fixing it?
>
> A method in grovel/asdf.lisp adds output files of process-op to output files 
> of compile-op:
>
> ;;; Declare the .o and .so files as compilation outputs,
> ;;; so they get picked up by bundle operations.
> #.(when (version<= "3.1.6" (asdf-version))
> '(defmethod output-files ((op compile-op) (c wrapper-file))
>   (destructuring-bind (generated-lisp lib-file c-file o-file) 
> (output-files 'process-op c)
> (declare (ignore generated-lisp c-file))
> (multiple-value-bind (files translatedp) (call-next-method)
>   (values (append files (list lib-file o-file)) translatedp)
>
> As a result inputs and outputs of the ops look like this:
> process-op:
>   input: wrapper-file
>   output: bindings-file.lisp file.c FILE.O FILE.SO
>
> compile-op:
>   input: bindings-file.lisp
>   output: bindings-file.fasl FILE.O FILE.SO
>
> The problem is that process-op generates file.o and file.so before it 
> generates bindings-file.lisp. Thus compile-op gets re-executed all the time, 
> because its output files file.o and file.so are always older than its input 
> file bindings-file.lisp. And an execution of the compile-op does not change 
> anything, because it does not really generate .o and .so.
> One way to fix it would be to "touch" .o and .so to get the right order of 
> file modification times, but there may be a better way.
>
> CFFI bug report: https://bugs.launchpad.net/cffi/+bug/1889491
>
> Thanks,
> Ilya

Re: How to tell if we're being loaded by ASDF?

[Faré]

One hacky way is to check whether *asdf-session* is nil or not. For
more precision, check the top of the visiting-action-list of the
*asdf-session* and see if its component has the pathname of your lisp
file (from (uiop:current-lisp-file-pathname)

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The main reason Santa is so jolly is because
he knows where all the bad girls live.


On Sun, Mar 8, 2020 at 6:23 PM Robert Dodier  wrote:
>
> Hi, this is a bit off the wall, but maybe there is a way to do it. A
> web search didn't seem to find anything about it.
>
> Is there a way to know that ASDF is loading stuff?
>
> I have this situation. A file might be loaded by ASDF or it might be
> loaded by hand. If it's being loaded by ASDF, then I'd like to do some
> stuff differently.
>
> Why might this file be loaded by hand, why not always use ASDF? Well,
> it's not a Lisp file, it's a Maxima file being loaded by some
> maxima-asdf hackery. I can't count on maxima-asdf working for anybody
> else so I'm expecting that people will load it by hand instead of
> ASDF. But if it is being loaded by ASDF, then I'd like to know that.
>
> Thanks a lot for any info,
>
> Robert Dodier
>

Re: I should know this... finding INPUT-FILES

[Faré]

1. output-translations is for output-files, not for input-files—except
of course that input-files often will in turn call output-files on a
previous operation that it depends on.

2. as to output-files, it is the outermost method that applies the
output translations—when the second value is NIL; the translations are
assumed to already have been applied or subsumed when the second value
is T.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Foolishness is rarely a matter of lack of intelligence or even lack of
information.  — John McCarthy


On Fri, Jan 24, 2020 at 3:56 PM Robert Goldman  wrote:
>
> I'm looking at the documentation for INPUT-FILES and OUTPUT-FILES and I see 
> this:
>
>   (defgeneric output-files (operation component)
> (:documentation "Methods for this function return two values: a list of 
> output files
> corresponding to this action, and a boolean indicating if they have already 
> been subjected
> to relevant output translations and should not be further translated.
>
> Methods on PERFORM *must* call this function to determine where their outputs 
> are to be located.
> They may rely on the order of the files to discriminate between outputs.
> "))
>   (defgeneric input-files (operation component)
> (:documentation "A list of input files corresponding to this action.
>
> Methods on PERFORM *must* call this function to determine where their inputs 
> are located.
> They may rely on the order of the files to discriminate between inputs.
> "))
>
> Missing in the above is an explanation of how to tell INPUT-FILES whether to 
> apply output translations or not.
>
> Is this left to the author of the relevant PERFORM method? Or is there 
> something one should do when writing a PERFORM method or an operator 
> definition?

Re: ASDF dependency graph or complexity score

[Faré]

I updated the POIU README with remarks based on our discussion so far.
After thinking a bit more, I suspect it is not *that* hard to
completely fix POIU and extract (and display) a precise ASDF
dependency graph from it. But it's still some amount of serious work
and a walk through the internals or ASDF and POIU.

NB: I'm talking about POIU, not UIOP. Even I got confused in my
previous messages at times. Oops.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The idea is not to confront bad ideas but to come up with good ideas.
Otherwise, your enemies define the game and you are the loyal opposition.
— Terence McKenna


On Tue, Dec 24, 2019 at 12:17 PM Robert Goldman  wrote:
>
> What kind of dependency graph are you looking to make? Just the systems, or 
> also the individual system components?
>
> And do you want a visible representation of this graph, or just some set of 
> data structures?
>
> I assume the former, since you should be able to use the system components 
> themselves to get all the dependency links.
>
> If it's the former, the easiest thing might be to use CL-DOT and write the 
> necessary methods to create a graph.
>
> You might need to get ASDF to compute the plan for LOAD-OP as a way to ensure 
> that all the required system definitions are loaded.
>
> With respect to test systems, I have usually found it best to have my 
> systems, say "foo", have "foo/test" related systems, delegate the test-op for 
> "foo" to "foo/test" using in-order-to, and then only "foo/test" needs to 
> depend on the test library -- "foo" itself doesn't need it.
>
> HtH,
>
> R
>
> On 23 Dec 2019, at 19:08, Jay wrote:
>
> Thanks.
>
> I will follow up with Rob later.
>
> Anyway, thanks for help in the past. I will liaise with Rob to figure out
> the best way forward.
>
> Jay
>
> Faré
> 6:52 PM (14 minutes ago)
> to me
>
> I guess, I will have to step up at some point. I have a tool that's built
>
> with CL. I'll start small from the easier tasks and work my way up the
> graph one you highlighted above.
>
> Then, I will be able to tackle the graph thing, as you said it's no easy
>
> fit.
>
> Makes sense.
>
> Are the TODO you mentioned ordered in some priority list or tagged with
>
> easier/medium/hard.
>
> Unhappily not. Give it a look and/or give a look at the issues on
> launchpad, and/or ask Robert, for guidance. Sorry I don't have time to
> sort out the issues right now.
>
> And the repo is still the version on gitlab right? Does Rob hang over at
>
> #lisp irc?
>
> Yes the repo is on gitlab, but the issues are still mostly on launchpad.
>
> Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
>
> —♯ƒ • François-René ÐVB Rideau •Reflection•
> http://fare.tunes.org
> "Floating point numbers are like sandpiles; every time you move one
> you lose a little sand and pick up a little dirt"
> — Vic Vissotsky
>
>
> On Mon, Dec 23, 2019 at 6:52 PM Faré  wrote:
>
> I guess, I will have to step up at some point. I have a tool that's
>
> built with CL. I'll start small from the easier tasks and work my way up
> the graph one you highlighted above.
>
> Then, I will be able to tackle the graph thing, as you said it's no easy
>
> fit.
>
> Makes sense.
>
> Are the TODO you mentioned ordered in some priority list or tagged with
>
> easier/medium/hard.
>
> Unhappily not. Give it a look and/or give a look at the issues on
> launchpad, and/or ask Robert, for guidance. Sorry I don't have time to
> sort out the issues right now.
>
> And the repo is still the version on gitlab right? Does Rob hang over at
>
> #lisp irc?
>
> Yes the repo is on gitlab, but the issues are still mostly on launchpad.
>
> Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
>
> —♯ƒ • François-René ÐVB Rideau •Reflection•
> http://fare.tunes.org
> "Floating point numbers are like sandpiles; every time you move one
> you lose a little sand and pick up a little dirt"
> — Vic Vissotsky
>
>
> On Fri, Nov 29, 2019 at 11:33 AM Faré  wrote:
>
> I could not follow up in the past as I had to relocate between cities
>
> (central to west coast) so that caused a lot of changes in my schedule.
> What is the state of the asdf contributors you mentioned last time -- the
> developers taking over asdf development albeit gradually.
>
> No one has really stepped forward. It's still Robert Goldman
> maintaining things, while I have moved away from active maintenance
> and am only reviewing patches and giving advice, sometimes making
> small edits to documentation.
>
> I just sta

Re: PATCH: SIGNAL a condition on test failures, for use with ASDF:TEST-SYSTEM

[Faré]

> > Note that my recommended solution is for a test library to create a
> > report file with a test-report-op (that always succeeds at creating a
> > report file, but the report file may indicate failure), and the
> > test-op just depends on the test-report-op, and signals and error if
> > it didn't indicate success.
>
> What branch of the ASDF repository is this in? These are the only two
> mentions of test-report-op in master:
>
There is no branch of ASDF doing this, and so far as I know you don't
need one: just define your own subclass of system for which test-op
has a suitable method. And if you're using my advanced proposal and
the inherited methods don't suit you, just skip the
(call-next-method).

> I do not understand how test-report-op would help in making a
> generic version of the test script I posted:
> https://gitlab.common-lisp.net/uri-template/uri-template2/blob/master/run-tests.lisp
>
The two are orthogonal. What test-report-op would do is help avoid
re-testing systems that haven't changed, if and when you do massive
incremental testing of lots of systems.

Your script looks mostly good. I would use lisp-invocation instead of
ros, but I can see merit to ros, too.

—♯ƒ • François-René Rideau
The state is the coldest of all cold monsters. Coldly it lies, too; and this
lie creeps from its mouth: "I, the state, am the people."  — Nietzsche

Re: long-description

[Faré]

On Tue, Feb 26, 2019 at 8:30 AM Will Mengarini  wrote:
>
> * Robert Goldman  [19-02/25=Mo 15:37 -0600]:
> > Even if so, why not just add `:documentation-pathname`,
> > and then no one will have to worry about type errors?
>
> +1
>
Because adding a new slot means a new ASDF version with its own
feature and systems having to guard use of that slot with #+asdf3.4 to
avoid breaking on older versions of ASDF. I don't think this feature
is worth it. Actually, I'm not even sure the feature is worth this
discussion that I started, that looks more and more like bikeshedding.
My apologies for starting it.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Prior to capitalism, the way people amassed great wealth was by looting,
plundering and enslaving their fellow man. Capitalism made it possible to
become wealthy by serving your fellow man. ― Walter E. Williams

long-description

[Faré]

I've seen the pattern of using
  :long-description
  #.(uiop:read-file-string
 (uiop:subpathname *load-pathname* "README.md"))
spread among CL libraries.

I see it only as a waste of kilobytes of data (quadrupled on 32-bit
unicode lisps such as SBCL).

I'm told it's because Quickdocs likes it this way.

Since there is not (yet?) any type enforcement on the value of that
field, can we instead agree for an alternate format, wherein the field
would instead contain the pathname of the long-description file,
relative to the (asdf:system-source-directory) ? Thus you'd use:
 :long-description #p"README.md"
And Quickdocs and other documentation tools would do the right thing from there.

Maybe I'm just an old-school code bummer but I'm always saddened by a
waste of bytes.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
I have never killed any one, but I have read some obituary notices with great
satisfaction. — Clarence Darrow

Re: ASDF upgrade script fails on cmucl/Linux

[Faré]

I believe this is an issue with ASDF upgrade not working on CMUCL due
to CLOS issues.

The workaround is to use the install-asdf.lisp script to overwrite
CMUCL's builtin ASDF with a newer one. — Although I'm not sure whether
the script will work if ASDF can't upgrade itself.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Every great truth begins as heresy and ends as superstition.
— Thomas Henry Huxley


On Tue, Feb 12, 2019 at 3:51 PM Robert Goldman  wrote:
>
> I'm getting a failure of the ASDF upgrade tests on CMUCL since the UIOP 
> dependencies mod:
>
> Upgrading ASDF from version 3.1.7 to version 3.3.2.13
>
> TEST ABORTED: Error while trying to load definition for system test-asdf from 
> pathname /var/lib/jenkins/workspace/asdf-upgrade/test/test-asdf.asd: don't 
> recognize component type :MODULE
>
> This looks like the key bit of the backtrace:
>
> 10: (ERROR
>  ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR
>  :NAME "test-asdf"
>  :PATHNAME #P"/var/lib/jenkins/workspace/asdf-upgrade/test/test-asdf.asd"
>  :CONDITION #)
> 11: ("LAMBDA (.PV-CELL. .NEXT-METHOD-CALL. O S)"
>  #)
> 12: (SIGNAL #)
> 13: (ERROR
>  ASDF/SESSION:FORMATTED-SYSTEM-DEFINITION-ERROR
>  :FORMAT-CONTROL "don't recognize component type ~S"
>  :FORMAT-ARGUMENTS (:MODULE))
> 14: (ASDF/SESSION:SYSDEF-ERROR "don't recognize component type ~S" 
> :MODULE)[:OPTIONAL]
> 15: (ASDF/PARSE-DEFSYSTEM:PARSE-COMPONENT-FORM
>  NIL
>  (:MODULE "test-asdf" :PATHNAME NIL)
>  :PREVIOUS-SERIAL-COMPONENT NIL)[:OPTIONAL]
> 16: (LISP::SLOLOAD
>  # "/var/lib/jenkins/workspace/asdf-upgrade/test/test-asdf.asd">)
> 17: (LISP::INTERNAL-LOAD
>  #P"/var/lib/jenkins/workspace/asdf-upgrade/test/test-asdf.asd"
>  #P"/var/lib/jenkins/workspace/asdf-upgrade/test/test-asdf.asd"
>  :ERROR :SOURCE
>  :UTF-8)
>
> TBH, I'm inclined to regard 3.1.7 as so old that I don't have to think about 
> it, but I'm open to contradiction. If I don't hear any, I'll prune that from 
> the set of origin versions to test for upgrades, and move on.
>
> Best,
>
> R

Re: clisp testing on linux

[Faré]

Yes, I am on Linux amd64 using
make -f Makefile t l=clisp t=test-encodings.script

On Debian, using GNU CLISP 2.49.92 (2018-02-18) everything works fine.
On NixOS, using GNU CLISP 2.49.60+ (2017-06-25) I get the error
POSIX:COPY-FILE: illegal keyword/value pair :METHOD, :RENAME in argument list.
The allowed keywords are NIL

That's weird, because I've been using these arguments to copy-file
since the very first ASDF 3 prerelease (2.27) in 2013 and never had
this issue before. I'll blame the NixOS build script for failing to
configure clisp properly.

—♯ƒ • François-René ÐVB Rideau •ReflectionCybernethics•
http://fare.tunes.org
A cuddle a day keeps the shrink away

On Tue, Dec 18, 2018 at 6:49 PM Robert Goldman  wrote:
>
> Are you getting that error on the same test (test-encodings.script), or is it 
> in a different test script?
>
> It would be interesting if it's on the same test, because I am not seeing 
> this error, and maybe it's causing the segmentation fault. Would you post the 
> context of the error?
>
> Thanks!
> R
>
> On 18 Dec 2018, at 17:43, Faré wrote:
>
> Well, CLISP is known for its deterministic but chaotic segfaults (i.e.
> sensitive to the slightest perturbation in input), probably due to
> some GC bug somewhere, which would explain your failure.
>
> Interestingly, I experience a different failure using GNU CLISP
> 2.49.60+ (2017-06-25) on NixOS 18.09.1228.a4c4cbb613c, I instead get:
> TEST ABORTED:
> POSIX:COPY-FILE: illegal keyword/value pair :METHOD, :RENAME in argument list.
> The allowed keywords are NIL
>
> This suggests that uiop/filesystem:rename-file-overwriting-target has
> bitrotten on the latest CLISP. Any volunteer to fix that in a
> backward-compatible way? I'll pass.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection•
> http://fare.tunes.org
> We must do away with all newspapers. A revolution cannot be accomplished
> with freedom of the press. — Ernesto "Che" Guevara
>
> On Tue, Dec 18, 2018 at 3:58 PM Robert Goldman rpgold...@sift.info wrote:
>
> The latest tweak was followed by a Jenkins (linux) test failure for clisp on 
> test-encodings.script, with a segmentation fault.
>
> I cannot reliably replicate this, and I don't see any reason to assume that 
> this is ASDF's fault, rather than clisp's.
>
> I'm not sure that there is anything I can do about this, but thought I would 
> ask for suggestions:
>
> Test TEST-FILE-ENCODING-U8: should be UTF-8
> ; Registering system test-file-encoding-u8
>
> *** - handle_fault error2 ! address = 0x55f6cc8871d0 not in 
> [0x55db332bd000,0x55db336926d8) !
> SIGSEGV cannot be cured. Fault address = 0x55f6cc8871d0.
> GC count: 203
> Space collected by GC: 304757408
> Run time: 3 915662
> Real time: 4 586182
> GC time: 0 693431
> Permanently allocated: 165368 bytes.
> Currently in use: 8753400 bytes.
> Free space: 1903262 bytes.
> Segmentation fault (core dumped)
> Using clisp, test-encodings.script failed

Re: wild-package-inferred-system

[Faré]

On Tue, Dec 4, 2018 at 9:21 PM Hugo Ishimaru  wrote:
> I have made an extension of package-inferred-system that interprets
> star `*' and globstar `**' in a package name, which I will use in the
> future for my library.
> https://github.com/privet-kitty/wild-package-inferred-system
> (My motivation is to eliminate the need for manually creating
> `foo/.../all'-type subsystems that exist only to use-reexport all
> .lisp files.)
>
I'm not convinced this is a good idea, but then again I don't
understand your use case.

> In accordance with the ASDF Best Practices by Faré, I let you know the
> non-exported facilities of ASDF I had to used: PRIMARY-SYSTEM-P,
> *DEFPACKAGE-FORMS*, *PACKAGE-INFERRED-SYSTEMS*, AROUND-COMPILE-HOOK,
> RELATIVE-PATHNAME.
>
You should ask Robert, who is maintainer, but yes, it makes sense to
export these from package ASDF.

> I would appreciate any suggestion or correction for the design and
> implementation of this extension, of course.
>
A very cursory glance finds nothing wrong with your implementation. I
don't love the mismatch between your treatment of wild and the
completely unportable treatment of wild by underlying implementations,
but that's not your fault.

Also, why don't you use package-inferred-system for your system? :-)

Would you be interested in becoming ASDF co-maintainer and slowly take
over from Robert and I?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
I don't want to belong to any club that will accept me as a member — Groucho

Re: clisp testing on linux

[Faré]

Well, CLISP is known for its deterministic but chaotic segfaults (i.e.
sensitive to the slightest perturbation in input), probably due to
some GC bug somewhere, which would explain your failure.

Interestingly, I experience a different failure using GNU CLISP
2.49.60+ (2017-06-25) on NixOS 18.09.1228.a4c4cbb613c, I instead get:
TEST ABORTED:
POSIX:COPY-FILE: illegal keyword/value pair :METHOD, :RENAME in argument list.
The allowed keywords are NIL

This suggests that uiop/filesystem:rename-file-overwriting-target has
bitrotten on the latest CLISP. Any volunteer to fix that in a
backward-compatible way? I'll pass.

—♯ƒ • François-René ÐVB Rideau •ReflectionCybernethics•
http://fare.tunes.org
We must do away with all newspapers. A revolution cannot be accomplished
with freedom of the press. — Ernesto "Che" Guevara


On Tue, Dec 18, 2018 at 3:58 PM Robert Goldman  wrote:
>
> The latest tweak was followed by a Jenkins (linux) test failure for clisp on 
> test-encodings.script, with a segmentation fault.
>
> I cannot reliably replicate this, and I don't see any reason to assume that 
> this is ASDF's fault, rather than clisp's.
>
> I'm not sure that there is anything I can do about this, but thought I would 
> ask for suggestions:
>
> Test TEST-FILE-ENCODING-U8: should be UTF-8
> ; Registering system test-file-encoding-u8
>
> *** - handle_fault error2 ! address = 0x55f6cc8871d0 not in 
> [0x55db332bd000,0x55db336926d8) !
> SIGSEGV cannot be cured. Fault address = 0x55f6cc8871d0.
> GC count: 203
> Space collected by GC: 304757408
> Run time: 3 915662
> Real time: 4 586182
> GC time: 0 693431
> Permanently allocated: 165368 bytes.
> Currently in use: 8753400 bytes.
> Free space: 1903262 bytes.
> Segmentation fault (core dumped)
> Using clisp, test-encodings.script failed

Re: why does ASDF ask to please only define system/test?

[Faré]

On Tue, Dec 11, 2018 at 2:09 PM Mark H. David  wrote:
> It seems that any system Y associated with a name X must have its name be of 
> the form X/Y.  For example, when you build "cl-ppcre", you get this warning:
>
> Please only define "cl-ppcre" and secondary systems with a name starting with 
> "cl-ppcre/" (e.g. "cl-ppcre/test") in that file.
>
> I've seen this complaint for quite a few systems already. What's the need for 
> this, and is it really worth nagging users of all these systems that have 
> existed in many cases for years and have worked perfectly well without 
> following the new convention? What great functionality are we getting for 
> this?
>
"Worked perfectly well", except that
1- if you wanted to find a system not properly named, you had to make
sure to have read the relevant .asd file beforehand.
2- if someone defined two systems with the same name in two different
files, then ASDF 2.014 could loop infinitely, and even 2.016 or later
might survive but behave in unstable way depending on how changes may
cause one system to be loaded instead of the other, then reloaded,
etc.

With ASDF 2.27 (ASDF 3 pre-release), a solution was offered to define
multiple systems in one .asd file in a safe way, and the old way was
deprecated ever since, though a warning has only been issued since
ASDF 3.2.0.

ASDF is still backward-compatible. You can hush the warning if you
want—though it importantly tells you which systems could use a new
maintainer.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A true intellectual is a man who, after reading a book and being convinced by
its arguments, will shoot someone or, more likely, order someone shot.
— John McCarthy

Re: Timing compilation

[Faré]

>   Hello,
>
> I would like to collect information about the time it takes to compile
> an ASDF system (possibly also load it), dependencies excluded. I'm
> thinking there's probably a way to do this by :around'ing compile-op, or
> something like that, but if someone already has a clear view on how to
> make this happen, I'd be grateful for the time spared!
>
Simplest  method:
1- load the dependencies, possibly using (asdf:operate :prepare-op s)
2- (time (asdf:make s))

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
I used to like the government, but that was before it got big and popular.

Re: Extracting a UIOP's manual

[Faré]

This looks very helpful indeed. Can you submit a MR on
gitlab.common-lisp.net/asdf/asdf ?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Fraud is the homage that force pays to reason.  — Charles Curtis

On Thu, Aug 16, 2018 at 3:23 AM Hugo Ishimaru  wrote:
>
> Hi all,
>
> I have added the UIOP's manual to index.html and integrated the
> generation into the main Makefile just like that of ASDF (rsync is not
> tested):
> https://github.com/privet-kitty/asdf/commit/907ab8bea2cb28392fe178107e559af8a1d0e70d
> I wish this will be helpful to you. (I'll do a merge request if you
> need, of course)
>
> Regards,
> Hugo Ishimaru
>
> 2018-08-15 15:59 GMT+02:00 Hugo Ishimaru :
> > Hi all,
> >
> > I have extracted a UIOP's manual from docstrings:
> > https://privet-kitty.github.io/etc/uiop.html
> > It's not perfect in any means as docstring isn't written in a markup
> > langugage, but will be at least a comprehensive reference. The relevant fork
> > of ASDF is in Github:
> > https://github.com/privet-kitty/asdf/tree/uiop-man/uiop/doc
> > The greater part of the generation conforms to that of Alexandria, which
> > depends on SBCL (sb-texinfo).
> >
> > I send this mail as it states
> >> Help wanted extracting working documentation from UIOP's docstrings.
> > in README.md though I might not understand a historical context.
> >
> > Regards
> > Hugo Ishimaru
> >
>

Re: bootstrapping asdf

[Faré]

> I think we should probably clear out all the MRs except the syntax 
> rationalization, then release bug fix 3.3.3, and then move to get the syntax 
> branch integrated as a release candidate for 3.4.
>
> I think the syntax isolation is a big enough change in behavior that it 
> merits being 3.4
>
I wholeheartedly agree. There's just !99 left, which contains at least
one important fix (defining call-without-redefinition-warnings in
asdf.asd, not just uiop.asd, since we won't be loading uiop.asd
anymore). After that, I believe we should release 3.3.3 as a
bug-fixing milestone, where the 3.3 release gets really stable (also,
half way to 6.6.6). And then move on with 3.4.0 that would include
syntax-control !86 which I believe is ready, with its conservative
take on using the initial readtable by default.

The other MRs out there are not ready: the Genera one !84 awaits input
from someone who actually develops on Genera and could fix the CLOS
issues with ASDF 3.3 there. The !83 on process-status is a WIP that
the late author, Elias Pipping won't make further progress on. :-(
RIP.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
You think that by not releasing your in-house software, you save
engineering resources. You only get yourself out of the loop.

Re: Is anyone actually using the deferred warnings code?

[Faré]

On Wed, May 30, 2018 at 6:01 PM Robert Goldman  wrote:
>
> AFAICT, it is disabled by default, and it is not documented in the ASDF 
> manual.
>
> So is this code even live, except for in the test scripts?
>
> It was pretty painful to get this to work because it relies on unexported 
> (and hence unstable) SBCL internals.
>
> My guess is that this was originally done as a way to dispense with 
> WITH-COMPILATION-UNIT and hence potentially parallelize the compilations.
>
> If no one is even using it, I am tempted to move it to be a contrib and drop 
> it from the tests and maintenance as part of core ASDF. Maybe it would be 
> better as a part of a POIU extension to ASDF, and relieve me of dealing with 
> these SBCL compiler internals?
>

Sorry I only see this message now.

I haven't used deferred warnings in quite some time (since QRes was
cancelled), but I can comment on the design.

The goal was to make undefined-function warnings usable in an
incremental (not just parallel) context.

Suppose a file A uses a function F, that is (supposed to be) defined
in file B. Further, there is no dependency from A to B and indeed
there may be a dependency from B to A, due to e.g. macros in A being
used in B.

If B fails to define F, then SBCL will issue a warning at the end of
the "compilation unit", which since ASDF 1 has always been "the entire
ASDF build". Now, if we're in an incremental build, and A was
previously compiled, then A.fasl is loaded, SBCL doesn't even register
that a function was used before it was defined, and no warning is
issued at all. In the case of a parallel build with no dependency
between A and B, then, depending on the compilation unit situation,
either there would always be a spurious warning or never a warning at
all compiling A, and never a warning afterwards. This made those
warnings less than useful at ITA.

Some hacker at ITA got the idea of saving the context for these
warnings (maybe James Knight? he usually had all the clever ideas). It
was first implemented for SBCL by Douglas Katzman, and I made it part
of XCVB, POIU, and later ASDF 3. This was really nice to have at ITA.

However, when I proposed making it the default in ASDF 3, the proposal
was immediately met with a huge backlash from the community, so the
feature remained disabled, and may have bitrotten. One thing (that
I've done) is sending patches to hundreds of projects to fix their
builds that depended on unsupported or deprecated ASDF behavior that
was refactored away, when the system authors will welcome the patches.
Another thing that I learned not to try (after the XCVB debacle) is
try to maintain software patches against the wishes of the authors.

I apologize for never documenting it. My bad. With the poor reception,
I wasn't motivated.

Moreover, for QRes we had this configuration layer whereby the
third-party libraries we depended on were usually compiled with many
more warnings disabled than when building our code, where we disabled
some warnings but enabled others. The code is part of the quux code
dump on qitab, but would be pain to document.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Anyone who goes to a psychiatrist ought to have his head examined.
— Samuel Goldwyn

bootstrapping asdf

[Faré]

In case anyone takes over maintenance of ASDF in the future, I wrote
this mind dump on some of its more stuble "magic" behavior:
https://fare.livejournal.com/190738.html

Thanks to Eric Timmons for the prompting...

PS: there are relatedly a few bug fixes in 3.3.2.5 — thanks and
congratulations again to Robert Goldman for his careful reviewing and
testing.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
It is deplorable that many people think that the best way to improve the world
is to forbid something. However, they're morally more advanced than the people
who think the best way to improve the world is to kill somebody.
— John McCarthy

Re: POIU is bitrotten

[Faré]

Sorry for the breakage. Completing the update of POIU to work with ASDF 3.3.x
is not high enough in my todo list to happen any time soon,
but since you expressed interest, I pushed a few tiny fixes, but more
importantly,
I updated the README and the test.lisp to document how you should be able
to test it.

Issue:
To be able to do a differential test, you may want to try an older POIU
with an older ASDF. I tested POIU 1.31.1 to work with ASDF 3.1.7,
but using ASDF 3.1.7 may require overriding your implementation's ASDF
using tools/install-asdf.lisp from the asdf source repository.
Yet, you will want to use the updated test.lisp that I just posted with
instructions in the README to test that old POIU.
For the test to work, though, I needed a patch to closer-mop:
https://github.com/pcostanza/closer-mop/pull/10

But the differential test will only help you to understand
how POIU used to work with an old ASDF.
The deeper problem is that ASDF's internals changed *a lot* since 3.1.7,
notably so as to better support multi-stage builds in 3.3.0.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
If once you have paid him the Dane-geld / You never get rid of the Dane.
— Rudyard Kipling


On Tue, Jul 24, 2018 at 6:14 PM Robert Goldman  wrote:
>
> Hi, Faré --
>
> I had a look at POIU, because I thought it would be interesting to fix
> this.  I got pretty badly bogged down, though, because there's a really
> high barrier to entry in trying to run the tests.
>
> The tests require that I have your exscribe system and its full
> dependency tail.  I tried to collect it, but after getting down to about
> the third level of transitive dependency, I gave up (partly this is
> because ASDF is not helping me figure out what the full set of
> dependencies is -- each time I grab a new dependency I may discover
> others that are needed).
>
> Would it be possible to shove the relevant git submodules into the POIU
> repo?  Or somehow otherwise make the test runnable to anyone who checks
> out the repo?
>
> Thanks,
> r
>
>
> On 24 Jun 2018, at 12:49, Faré wrote:
>
> > In case anyone cares (I suppose no one does), POIU, the parallel
> > operator on independent units, that parallelizes compilation (but not
> > loading) of ASDF component, with various options regarding determinism
> > or non-determinism, is current bitrotten and not working on ASDF
> > 3.3.2.2, and I disabled its functionality. I don't have the time or
> > interest to fix it (I haven't been using CL at work for five years,
> > and have stopped using it for new personal projects).
> >
> > The other option I know to parallelize a CL build is bazelisp (the
> > Lisp support for bazel.build), which I would bet is also bitrotten,
> > since I haven't touched it in years, either. [An earlier option I
> > worked on, XCVB, has bitrotten long ago.]
> >
> > I am still available for advice in case someone wants to fix things
> > —
> > better ask before the information gets evicted to long-term storage.
> >
> > —♯ƒ • François-René ÐVB Rideau •Reflection•
> > http://fare.tunes.org
> > Superstition brings bad luck. — Saul Gorn

Re: A thing I should know myself....

[Faré]

Sorry for a late reply.

UIOP has these utilities that can help you:
(uiop:find-symbol* :unimplemented-stub :foo nil)
(uiop:match-condition-p #(unimplemented-stub foo) (make-condition
'simple-warning))
(setf uiop:*uninteresting-conditions* '(#(unimplemented-stub foo)))

Also, ASDF has the around-compile hook that is the recommended place
to locally set those things:
(defun ignoring-unimplemented-sub (f) (let
((uiop:*uninteresting-conditions* '(#(unimplemented-stub foo
(funcall f)))
(defsystem ... :around-compile ignoring-unimplemented-sub ...)

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Do NOT question authority — they don't know either.

On Sun, Jun 24, 2018 at 8:41 AM Robert Goldman  wrote:
>
> On 22 Jun 2018, at 23:55, Stas Boukarev wrote:
>
> On Fri, Jun 22, 2018 at 5:13 PM Robert Goldman  wrote:
>>
>> I have a library that provides DEF-UNIMPLEMENTED as a macro for defining 
>> stub functions. When you compile a file with unimplemented functions, you 
>> get a warning of the type FOO:UNIMPLEMENTED-STUB in my library FOO.
>>
>> I'd like to put in an asdf system definition a file spec something like this:
>>
>> (:file "file-with-stubs"
>>   :method (:around (o c)
(handler-bind ((foo:unimplemented-stub
>>   #'(lambda (c)
>>   (print c)
>>   (muffle-warning c))
>> (call-next-method)))
>>
>> but, of course, the package foo doesn't exist when this is read (although I 
>> could put (asdf:load-system "foo") upstream of the enclosing defsystem).
>>
>> This isn't a case that's nicely consistent with Faré's hack for translating 
>> strings or keyword symbols, nor does it seem easy to use find-symbol for 
>> this purpose.
>
> You could still use FIND-SYMBOL:
> (handler-bind ((error (lambda (c) (when (typep c (find-symbol x :foo)) 
> (a))
>
> That's a good point, and effectively what I ended up doing. But it's 
> certainly not pleasing, because we end up doing our own type dispatch, on top 
> of that which is built into CL with handler-bind. Still, this might be the 
> best I can do.
>
> thanks,
> r

Re: Deferred warnings broken on SBCL 1.4.7

[Faré]

Also, if you fix reify-warnings, you may have to fix unreify-warnings with it.
For a test, try to (uiop:enable-deferred-warnings-check) before you
build software.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Passive hope is wishful thinking, a poison of the mind.
Active hope is creative passion, the mover of the universe.

On Wed, May 30, 2018 at 4:12 PM Robert Goldman  wrote:
>
> Whoops.  Looks like this doesn't work on older SBCLs.  I'll fix that
> now.
>
> R
>
>
> On 30 May 2018, at 15:08, Robert Goldman wrote:
>
> > I have just pushed a merge request and topic branch for this.
> >
> > See https://gitlab.common-lisp.net/asdf/asdf/merge_requests/95
> >
> > I'm pretty scared about this -- we are getting in there and rooting
> > around in SBCL internals in ways that seem almost guaranteed to break
> > again later.  But for now, I think it's fixed.
> >
> > Comments welcome -- especially comments involving a nicer rewrite of
> > what I wrote.
> >
> > Best,
> > R
> >
> > On 30 May 2018, at 12:13, Faré wrote:
> >
> >> On Wed, May 30, 2018 at 12:53 PM Eric Timmons 
> >> wrote:
> >>> Somewhat related, I was curious why ASDF doesn't use Gitlab CI to
> >>> automatically run tests. It probably wouldn't have helped in this
> >>> particular case since the root cause was a change outside ASDF, but
> >>> it's still nice for things like merge requests.
> >>
> >> Well, there's about ten different scriptable implementations on three
> >> major
> >> platforms, plus multiple versions of each compiler and maybe extra
> >> architectures to test for some of them. And then there are the
> >> software
> >> licensing issues for those operating systems and the proprietary
> >> implementations. Finally, a complete test set would probably include
> >> non-regression with cl-test-grid.
> >>
> >> Testing ASDF is a major undertaking. And even when all tests pass on
> >> all
> >> platforms, we have bad surprises, as demonstrated by the bumpy
> >> releases 3.2
> >> and 3.3.
> >>
> >> That said, even one image running tests on one implementation without
> >> cl-test-grid would be an obvious improvement over the current
> >> situation.
> >>
> >> Not being the maintainer anymore, nor paid to do it anymore, not
> >> using it
> >> for work or for fun anymore, I'd say "patches welcome".
> >>
> >> —♯ƒ • François-René ÐVB Rideau
> >> •Reflection•
> >> http://fare.tunes.org
> >> The state is the coldest of all cold monsters. Coldly it lies, too;
> >> and this
> >> lie creeps from its mouth: "I, the state, am the people."  —
> >> Nietzsche

Re: Deferred warnings broken on SBCL 1.4.7

[Faré]

On Wed, May 30, 2018 at 12:53 PM Eric Timmons  wrote:
> Somewhat related, I was curious why ASDF doesn't use Gitlab CI to
> automatically run tests. It probably wouldn't have helped in this
> particular case since the root cause was a change outside ASDF, but
> it's still nice for things like merge requests.

Well, there's about ten different scriptable implementations on three major
platforms, plus multiple versions of each compiler and maybe extra
architectures to test for some of them. And then there are the software
licensing issues for those operating systems and the proprietary
implementations. Finally, a complete test set would probably include
non-regression with cl-test-grid.

Testing ASDF is a major undertaking. And even when all tests pass on all
platforms, we have bad surprises, as demonstrated by the bumpy releases 3.2
and 3.3.

That said, even one image running tests on one implementation without
cl-test-grid would be an obvious improvement over the current situation.

Not being the maintainer anymore, nor paid to do it anymore, not using it
for work or for fun anymore, I'd say "patches welcome".

—♯ƒ • François-René ÐVB Rideau •Reflection•
http://fare.tunes.org
The state is the coldest of all cold monsters. Coldly it lies, too; and this
lie creeps from its mouth: "I, the state, am the people."  — Nietzsche

Re: Deferred warnings broken on SBCL 1.4.7

[Faré]

Oops. Can you provide a patch? If possible one that uses #. to test what
symbols are present and does the right thing?
There are a few examples of #+sbcl #.( in filesystem.lisp and image.lisp.

—♯ƒ • François-René ÐVB Rideau •Reflection•
http://fare.tunes.org
Government — If you think the problems we create are bad,
just wait until you see our solutions.

On Tue, May 29, 2018 at 11:51 PM Eric Timmons  wrote:

> Looks like SBCL 1.4.7 changed the slots of
> sb-c::compiler-error-context (in particular enclosing-source ->
> %enclosing-source, source -> %source, and original-source was
> removed). As a result, deferred warnings are broken. Attached is the
> output of `./run-tests.sh sbcl test-deferred-warnings.script`

> -Eric

Re: asdf-announce post from nim...@gentoo.org requires approval

[Faré]

Chema Alonso asked:

> Where can I download the 3.3.2 tarball?
>
> Normally I get it from here:
>
> https://common-lisp.net/project/asdf/archives/
>
> But I can't find it right now.
>
> Thanks.
>
I took the liberty of running "make archive publish-archive" from the
release branch. The tarballs should now all be present.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The hacker: someone who figured things out and made something cool happen.
— Alan Schmitt

Re: Problems with DEFSYSTEM-DEPENDS-ON

[Faré]

On Mon, Apr 9, 2018 at 12:22 PM, Robert Goldman  wrote:
> On 9 Apr 2018, at 11:17, Attila Lendvai wrote:
>
>>> A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might
>>> cause
>>> errors in its present form.
>>
>>
>>
>> i've learned, painfully, that indiscriminate ignore-errors will almost
>> always bite you back (in the form of wasted debugging time), no matter
>> how innocent they look.
>>
>>
>>> This really is more a QL issue than an ASDF one (although it illustrates
>>> an
>>> issue with wrapping errors, IMO).
>>
>> while this is true, the implementation of QL requires a reliable way
>> to hook into the internal state of various versions of ASDF (namely,
>> into the situation when ASDF is looking for a system, and not finding
>> it will lead to an error without QL intervention). if i understand it
>> correctly, this is the crux of this issue.
>
> Sure, and I am happy to try to support this, but not to the extent of
> recovering a copy of ASDF 2.x and trying to run it.
>
> The problem is that I don't know when the missing component condition was
> added to ASDF, and doing this right would involve checking the enclosed
> condition to see if it's a missing component error.  I know how to do that
> for a modern ASDF, but I don't know how to handle ASDFs that are too old to
> have this condition class.  And I don't fee like it's my job to think about
> that: I think it's perverse to continue trying to use ASDF 2.
>
ASDF 2.26 is totally unsupported at this point. No implementation uses
anything less than 3.1.2 (the first stable release in the 3.1 series,
from May 2014). Many essential packages require 3.1.x or later. It is
a waste of time to try to get 2.26 running

If Quicklisp wants to be conservative, I would recommend requiring ior
providing ASDF 3.1.7, which is the last in the 3.1 series and pretty
stable, from March 2016, which is two years ago.

The "official" policy of ASDF was always to not support anything that
had already been superseded 2 years ago or more. 3.1.7 should be the
oldest supported ASDF release.

That said, what my opinions seem to be negatively taken into account
by Xach, so there.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Evolution competitively selects stable cooperative patterns.

Re: Building from clean on Allegro Windows

[Faré]

I couldn't reproduce with AllegroExpress 10.1 on Linux amd64.
So I gave a closer look on the Windows VM, rm'ing all the files in
build/results/
before I retried, and looked at the minimal logs for the issue.

The warnings were a false lead: Allegro does issue them, but they are actually
caught by the build system, so were not the issue.
The real issue was that allegro was looking for alisp.exe.dxl instead
of alisp.dxl.
I thus modified run-tests.sh so it will handle this case with more care.

All the tests then passed except test-multiple (which works for me on Linux).
Unsurprisingly, I see test-multiple uses ln -s and has an ugly
workaround on Windows
that doesn't work on sbcl, either. I moved it from using ln -s to using copies,
which modified the behavior in ways I admit I don't fully comprehend.
I documented the parts I did understand.

test-multiple exercises a really dark corner of the design, anyway,
one that we're trying to deprecate, badly-named secondary systems.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
You may easily play a joke on a man who likes to argue — agree with him.
— Edgar Waston Howe


On Sat, Apr 7, 2018 at 3:22 PM, Robert Goldman  wrote:
> Got the same issue:
>
> Warning: FIRST-FEATURE is defined more than once as `operator' in file
> build\asdf.lisp.
> Warning: IMPLEMENTATION-TYPE is defined more than once as `operator' in
> file build\asdf.lisp.
> Warning: *IMPLEMENTATION-TYPE* is defined more than once as `variable'
> in file build\asdf.lisp.
> Warning: OPERATING-SYSTEM is defined more than once as `operator' in
> file build\asdf.lisp.
> Warning: ARCHITECTURE is defined more than once as `operator' in file
> build\asdf.lisp.
>
> I'll see if I can figure out what causes this. EVAL-WHEN is a likely
> candidate.
>
> More as I have news,
> r

Re: "deprecated recursive use" warning

[Faré]

On Wed, Mar 21, 2018 at 7:25 AM, Attila Lendvai  wrote:
> yeah, it feels like a lot of pain. it would be nice if there was a
> fork-like API in ASDF for implementing such exec'd compilation, but
> then i guess ASDF itself has no clue which /foo/bin/ directory has the
> underlying lisp binary, and/or which lisp to launch at all, let alone
> how...
>
Well, if you want to fork+exec a subprocess, then asdf:run-program does it,
and lisp-invocation can find the lisp binary,
at least on a lot of implementations, and/or with user help.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Where did the water from the Deluge come from, and where did it go?

Re: follow-up about handling .info file and index

[Faré]

Dear Robert and Robert,

here is a complement to Robert Goldman's excellent response.
It includes some style hints for the future of ASDF and its extensions.

>: Robert Dodier
> What I finally settled on is this. When the operation is COMPILE-OP,
> the .info file is copied to same location where the index fasl will go
> (because the index has some code to read the .info and it assumes the
> .info is in the same place as the fasl). In the .asd file, I say
> (defsystem-depends-on "info-index") and (:info-index "foo-index") for
> the component foo-index.lisp. This seems to be working as intended.
>
If your index is just a lisp file,
why do you even need to subclass cl-source-file, to begin with?
And if you do, why do you need override any method or initform?
Let the CL source files be compiled the normal way!

As to locating the .info file, while not just use the recommended
asdf:system-relative-pathname ?

I admit ASDF doesn't have a good story for standalone software delivery,
beside "hardwire everything in the image".
However your method strikes me as worse in every respect.

For a solution that requires all libraries to use suitable abstractions,
see in quux/lisp/qres-build/pathnames.lisp in the quux snapshot tarball
(that you can find on the QITAB page)
how we used to do it at ITA Software's QRes team.
The code in asdf/uiop/configuration.lisp including XDG support
is also a good start.
Making from it a solution adopted by the Common Lisp community at large
is an exercise left to the masochist.

NB: If you're that masochist, please contact me as I'm pushing for
something broadly similar in gerbil-scheme's clan libraries;
see utils/path-config.ss for the current version,
good enough for my current application but probably not
a well-defined general solution.

> (1) In the class definition for INFO-INDEX, it
> seems to be necessary to intone (type :initform "lisp").
>
This is not necessary. You are confused. See Robert Goldman's response.

It is inherited indeed, and is defined in lisp-action.lisp as follows:

  (defclass cl-source-file (source-file)
((type :initform "lisp"))

Now, my latest woes with asdf/bundle revealed that this pattern,
that I had been relying on since ASDF2,
isn't friendly with respect to code upgrade,
and particularly not friendly in case of a bug in a value
that must be fixed in a latter version.

A better pattern, that is now used for some methods
(notably gather-type and bundle-type in asdf/bundle.lisp),
is instead to define a method:

(defmethod file-type ((_ info-index)) "lisp")

I recommend you should use this pattern for all code going forward.
Except of course, you don't need to define a new method if you don't
need to override the behavior from the super-class.

However, note that this pattern is NOT currently in use
in asdf/component.lisp or asdf/lisp-action.lisp -- yet.
I would recommend that a future ASDF maintainer should in the future
replace all current occurrences of static slots :initform with this style
but that is a backward incompatible move that requires
a proper transition period:
first, announce the deprecation of the old style;
second, update all relevant ASDF extensions in Quicklisp;
third, bug known companies suspected to have private ASDF extensions;
finally, after a suitable timeout (one or two years since deprecation),
make the change in ASDF itself.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Don't worry about what anybody else is going to do. The best way to predict
the future is to invent it. Really smart people with reasonable funding can
do just about anything that doesn't violate too many of Newton's Laws!
— Alan Kay, 1971

Re: Testing for ASDF 3.3.2 and beyond?

[Faré]

On Sat, Mar 17, 2018 at 2:52 PM, Robert Goldman  wrote:
> Thank you very much, Anton.  Question: is the inner-conditional-test failure
> on SBCL 1.3.21 not a regression?  I just loaded this system and tested it on
> my mac with SBCL 1.4.3, and it worked fine, so I'm inclined to treat this is
> not a problem.  Also, the inner conditional system has a readme saying it's
> not maintained and please not to use it.
>
> What about reversi on CCL 1.9?  I guess if I understand correctly, reversi
> must have passed on CCL 1.10 and 1.11, so probably I shouldn't worry about
> this, either.
>
> Fare, what do you think?  Should I release this version, or wait a little
> longer and see if we can get the syntax fix in, as well?
>
I think we're good for 3.3.2. There are often a few non-reproducible
failures and failures on older implementations, and I see nothing
worrisome there, even though I haven't tried to rerun the crashes and
timeouts in case their work the second time around.

> Maybe I should release this as 3.3.2, and make the syntax-fixed version of
> ASDF be 3.4, since (for some people) it might be a bigger change.
>
Yes, that might be prudent.

When you make a 3.4, be sure to test cl-test-grid with a branch that
includes the version update, because that's how you'll find problems
with deprecated functions (for which you should make sure to bump the
expiration date properly).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Whatever is worth doing at all is worth doing well.  — Chesterfield

Re: Testing for ASDF 3.3.2 and beyond?

[Faré]

Dear Anton,

can you try your test suite again against 3.3.1.7 ? I think we're
mostly ready to release 3.3.2 this time, with its many bugs fixes (and
bug fix fixes).

Can you also try the branch made by Robert for syntax-control + a copy
of the standard readtable as default?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Wir Mathematiker sind alle ein bisschen meschugge.
(We mathematicans are all a bit crazy). — Lev Landau

Re: Best Practice for an ASDF Variable Like *compile-file-failure-behaviour*

[Faré]

On Fri, Mar 9, 2018 at 7:32 PM, Mark H. David  wrote:
> Actually, I see these lines in the file asdf.lisp in my sbcl distribution
> (SBCL 1.4):
>
>   (defmethod operate :around (operation component  keys
>verbose
> (on-warnings
> *compile-file-warnings-behaviour*)
> (on-failure
> *compile-file-failure-behaviour*))
>
> Could someone please ELI5 how to hook into that for on-failure?
>
These are there for backward compatibility with ASDF 1.
You are of course free to use them, and
I'm in no position to deprecate anything anymore.
But that's not what I would recommend as best practice.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The highest of minds / Often have built for themselves / The tallest of jails.

Re: Best Practice for an ASDF Variable Like *compile-file-failure-behaviour*

[Faré]

On Fri, Mar 9, 2018 at 5:34 PM, Robert Goldman  wrote:
> Are you just using this for yourself? If so, a simple
>
> (let ((asdf:*compile-file-failure-behaviour* :warn))
> (asdf:load-system "my system"))
>
> will suffice.
>
Yup.

> Alternatively, you could put something like this in the .asd file:
>
> (defmethod operate :around ((operation load-op) (system (asdf:find-system
> "my-system")))
>(let ((asdf:*compile-file-failure-behaviour* :warn))
>  (call-next-method)))
>
> The above most emphatically has not been tested, so it might be wrong.
>
As a rule of thumb, you should never define :around methods for
operate, for they do NOT do what you might naively believe they do:
1- they are only called on the top-level system, and/or on systems
loaded directly by a defsystem-depends-on
2- they wrap around not just the system at hand, but all its
transitive dependencies.

The working approach to changing variables for a system, no more no
less, is an :around-compile hook for your system.

But the correct approach is to NOT modify
asdf:*compile-file-failure-behaviour* but instead to catch
specifically the warnings that you want to ignore, using
*uninteresting-conditions* and the with-muffled-compiler-conditions
implicit in compile-file* and thus perform compile-op. See notably the
variable *usual-uninteresting-conditions*.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Intentions are not so much direct descriptions of actual mind processes as
Schelling points in the way social animals communicate.

Re: Testing for ASDF 3.3.2 and beyond?

[Faré]

On Fri, Mar 2, 2018 at 1:05 PM, Robert Goldman  wrote:
> So as I see it we have three options for the *shared-readtable*
>
> Your original option -- the "initial-ish" readtable (since we can't control
> when ASDF is loaded)
Usually ASDF is loaded before any significant other software is. If
the user insists in modifying the initial readtable early, he's
probably on to something.

> My proposal: a copy of the standard readtable. We should consider what might
> happen in this case if ASDF is reloaded or upgraded. I don't know if option
> #1 has this same issue.
In either case, definitely use defvar rather than defparameter, unless
you're also going to clear-registered-systems at every update.

> An unmodifiable standard readtable. I don't believe that this is portably
> possible, is it? There's no ANSI method (unless I've overlooked something)
> to make a readtable unmodifiable, and there's no way to access the standard
> readtable portably (since it isn't required to be unique).
>
It's not portable for unmodifiability to be enforced, but it's
certainly non-conformant to try to modify even when the
implementations fails silently later rather than catches you
immediately. So far as I know, only SBCL, ECL, Allegro, CMUCL make it
unmodifiable. Maybe file bugs for other relevant implementations?

> I believe my proposal, #2, is the most portable of all of the solutions, but
> I am still open to correction.
>
For some definition of "portable". I know that some systems rely on
e.g. CCL or MCL extensions #$ or #_ — but they could be made to use
the *initial-readtable* that we definitely are going to save anyway.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A good answer is one that solves the asker's problem,
not one that (necessarily) fits his expectations.
Actually, if the asker has been seriously looking for a solution,
and did not find any, then there's a good deal of chance
that a good answer won't fit his expectations! (At least, not all of them.)

Re: Testing for ASDF 3.3.2 and beyond?

[Faré]

>> Maybe it's an artefact of SBCL using too much memory *while compiling*
>> and would go away if you used e.g. POIU to compile inside forks.
>
> How?
>
"Just" load POIU right after you load ASDF, and before you load anything else.
https://gitlab.common-lisp.net/qitab/poiu

> The test order is alphabetical (by projects, and then inside a project
> its ASDF systems are sorted alphabetically too).
>
> Fasls compiled by previous tests are reused by next tests.
>
OK, this *might* explain some cases of "succeeds the second time around".
I suggest always making a second try after a timeout or out-of-memory error.

> Yes, I plan to start syntax-control testing soon.
>
Thanks!

> You might be interested to know that ACL binds *readtable* on startup to an 
> unmodifieble readtable
> (which is a violation of CLHS http://clhs.lisp.se/Body/02_aac.htm :
> "The initial readtable is distinct from the standard readtable. It is 
> permissible for a conforming program to modify the initial readtable.")
>
> The most visible result of this is puri and more than 150 other systems in 
> Quicklisp depending on it, failing to load.
>
Well, that's a good argument for fixing puri & al, regardless of what
option ASDF picks in the future.
And maybe a good argument for making ASDF strict all the time on all
implementations that allow it.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Death is only a milestone — albeit one that is dropped on you
from a very great height
— Terry Pratchett.

Re: Patch: Add support for Mezzano

[Faré]

On Thu, Mar 1, 2018 at 1:33 AM, Peter Housel  wrote:
> Enclosed is a patch (which can be applied using git am) that adds ASDF 
> support for the Mezzano Common Lisp operating system.
>
Please make it a merge request on gitlab.common-lisp.net/asdf/asdf

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Failure is the opportunity to begin again, more intelligently.
— Henry Ford, who had two flops before founding Ford Motor Co.

Re: package-inferred-systems and primary-system-name

[Faré]

Dear Eric, Robert,

>:Eric
> If a have a package-inferred-systems "a" and "a/b/c", the following
> code used to return "a":
>
> (primary-system-name (find-system "a/b/c"))
>
> But after commit 069cd2a6 it returns nil.
>
Thanks for finding that bug. Sorry we didn't have regression tests for that.

> The root cause right now is that system-source-file returns nil for
> the inferred systems. The easiest fix would be to have it return the
> asd file for the primary system. That's probably not *technically*
> correct since the system isn't actually defined in that file, but it's
> probably correct in the principle of least surprise sense.
>
This is probably technically correct enough, since that's the file that,
when it was loaded, implicitly defined the system, and that, if modified,
is likely to cause the system to be redefined.

The "obvious" solution is then that in package-inferred-system.lisp, instead of
:source-file nil
the defsystem form in sysdef-package-inferred-system-search should say
:source-file ,(system-source-file top)

>: Robert
> Looking over the change, I believe the issue is that, unfortunately, the "/"
> character is being used for two different purposes. In the "slashy" systems,
> it is used to identify subsystems, but the use in package-inferred systems
> is subtly different, because the location of their definitions as systems is
> different (indeed the package-inferred systems don't have explicit
> definitions). Also, I don't know that multiple slashes are ("a/b/c") are
> really supported in the case of systems that are defined in a.asd, but I
> haven't checked.
>
> I agree with you, though, that it's reasonable to treat a package-inferred
> system "a/b/c" as having "a" as its primary system name.
>
> I believe that this is the diff that caused the change in that commit:
>
> -  (component (primary-system-name (coerce-name (component-system
> system-designator))
> +  (component (let* ((system (component-system system-designator))
> +(source-file (physicalize-pathname
> (system-source-file system
> +   (and source-file
> +(equal (pathname-type source-file) "asd")
> +(pathname-name source-file))
>
> But I confess that I don't know the rationale for that change, so I don't
> know what collateral damage there will be to changing it.
>
I recommend against changing this function. It plays an important role
in supporting systems that are not named after the file they are
defined in.

> If you are going to patch this, will you please make a test case? I believe
> it could be easily added to package-inferred-system-test.script. I will be
> happy to help, if you would like.
>
Yes, that's the place where a test case would fit.

> It looks like you could add a separate branch to the etypecase with the
> logic special to package-inferred-system.
>
I think it's better to correctly set the system-source-file.

> If you have access to the cl.net gitlab, then a merge request would be a
> great way to supply a patch, but if that doesn't work for you, sending a git
> patch or just a regular old diff patch would also be fine.
>
Eric, will you send a MR?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
The trouble with opportunity is that it always comes disguised as hard work.
— Herbert V. Prochnow

Re: test-bundle

[Faré]

> (defmethod mark-operation-done :after ((o load-bundle-op) (c system))
> (mark-operation-done (find-operation o 'load-op) c)))
>
> Thanks for explaining that. That said, can you explain why we do this,
> instead of marking load-bundle-op as done? I guess because loading the
> bundle is intended to be equivalent to loading the system, so that if we
> load a bundle for a system, S, we want other systems that depend on S to
> know that it has been loaded.
>
It's in addition, not instead of, and considering how asdf 3 works as
opposed to 1 & 2, it's probably not effective unless you recurse for
all dependencies.

> Related to this, the first parameter to FIND-OPERATION is described as a
> "context," but there's nowhere an explanation of what a context is meant to
> be and in what way another operation can be a context.
>
find-operation once made sense, back when I was trying to support the
crazy and actually buggy ASDF 1 behavior (notably used by asdf-ecl) of
carrying extra build information in operation objects. This was
definitely deprecated last year in ASDF 3.2 or 3.3, although 3.3 also
opened the way to doing it in a non-buggy way in the future if you
want. I don't recommend that: it will be super complex to get right
for little short-term benefit; but you may have to it or something
just as hard, anyway, if you want to properly support
cross-compilation, for instance.

What you do with find-operation depends on your long-term plans. You
could get rid of the context and use make-operation everywhere instead
— that's what the current implementation does, anyway (it used to try
to preserve the operation initargs, even though
component-operation-times used to drop them).

> I read this as "find an operation by making it. Ignore the context
> argument." So I'm not entirely sure why it exists (presumably because it's
> idempotent, and MAKE-OPERATION is not), or why it has the context argument.
> I guess that you foresaw the need for a context at some point, but never
> ended up using it?
>
The context used to be (from ASDF 1, before refactoring into
find-operation) another operation from which you extracted the
initargs. In the future, it might be reborn, after some kind of
canonicalization, for e.g. optimization settings, cross-compilation,
etc.. Or that context could be moved to a different object — but that
would mean a new API incompatible with the current one for ASDF
extensions. Cross-compilation will require something somewhat
backward-incompatible, anyway.

> As I said above, I don't yet understand why that method exists, so I don't
> know what would be the implications of removing it.
>
I believe that before ASDF 3, this method was intended hat you could
indeed load a bundle and the system would therefore assume that the
system is loaded and not recurse into loading the individual files of
the bundle. This is definitely not working anymore (if it ever did) —
you'd have to recursively mark all objects as loaded with a suitable
timestamp. Also, considering that ECL abandoned the idea of a
*load-system-operation* (which that was supporting), the point of it
is moot. I recommend either recursing or removing the method over
hushing the warning. Note that, in the context of upgrade, removing
the method requires conditionally defining an empty method (or using
the MOP to remove it) in a suitable when-upgrading form.

> Yes, we could make that dependency warning into a defined condition and
> muffle it in this context. I just need to understand a little better the
> implications of doing that.
>
My current recommendation would be to just remove the method: wrap it
in when-upgrading with proper version spec, and empty out its body.

> So, even after the last email, I'm inclined to hold up a release until I
> understand this bug and kill it.
>
Good luck.

A lot of ASDF is historically-motivated cruft. Just like CL itself.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
In a centralized society social status is a zero-sum game along a single
hierarchy. That's why politics can never help the weak, only swap roles.

Re: test-bundle

[Faré]

On Tue, Feb 20, 2018 at 4:44 PM, Robert Goldman <rpgold...@sift.info> wrote:
> I still see these "dependency not done" warnings in test-bundle on SBCL and
> other implementations (I haven't checked them all, but at least my big
> three: SBCL, Allegro, and CCL).
>
I can see them too.

> This seems like a bug somewhere, right? And it must be an ASDF bug, probably
> in the definition of LOAD-OP for bundles.
>
I believe it's all due to this method:

  (defmethod mark-operation-done :after ((o load-bundle-op) (c system))
(mark-operation-done (find-operation o 'load-op) c)))

Said method should probably somehow hush these specific warnings,
or first recurse into all dependencies and mark them done, or be removed.
You decide.

While it's indeed a cosmetic bug to issue the warning,
I believe the underlying logic is sound.

> So, even after the last email, I'm inclined to hold up a release until I
> understand this bug and kill it.
>

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Toleration is not about respecting other people's ideas.
We have every right to fight ideas we think are stupid.
Toleration is about respecting other people's persons.
We have every duty to respect even persons we think are stupid.
— Faré

Re: Issues with package location information on SBCL

[Faré]

On Tue, Feb 20, 2018 at 3:35 PM, Robert Goldman  wrote:
> This does seem to illustrate an issue with the current "export everything
> that's in UIOP" strategy.
>
> Should we consider changing this policy?
>
1- UIOP reexports everything that individual subpackages of UIOP
export. I believe that's a simple and good policy. If a helper must
remain private, don't export it in the subpackage.
2- UIOP/PACKAGE is special, in that it is defined by defpackage (for
bootstrap reasons), and that therefore to portably ensure
upgradability, the list of symbols it exports MAY NEVER CHANGE, EVER.
No adds, no deletes, no renames. No change. If you need any change,
define and export a different package. Or use defpackage to define an
empty package or constant package, then define-package to import-from
and reexport from it.

> Or, if we have "internal" functions that we don't want to be visible through
> UIOP/DRIVER, should we simply not export them from the sub-packages and use
> :import-from to move them among the sub-packages?
>
Yes, that's the idea so far.

Also, we've moved symbols within UIOP in the past, and the
UIOP/subpackage names aren't stable. If you're using a utility in
UIOP, use UIOP: as a symbol prefix, don't use the symbol from its
current subpackage.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Statism is the secular version of salvation through faith: it doesn't
matter what bureaucrats do, only that they do it with good intentions.

Re: Test-multiple

[Faré]

> ;;; Warning: Computing just-done stamp in plan NIL for action
> (PREPARE-SOURCE-OP
>
> "test-multiple-too"
>"file2"), but
> dependency (LOAD-SOURCE-OP
>
> "test-multiple-too"
>
> "file1") wasn't done yet!
>
> Note the warning -- this is a warning message I'm coming to really hate:
>
> it has data that looks almost like a type error. If NIL is OK for the PLAN,
> then we should have a conditional in the warning format string that just
> prunes the plan out of the warning message if it's NIL. If NIL is not ok,
> then we need to explain that it isn't, and there should probably be a
> different and earlier warning.
>
Indeed, the warning message should use ~@[in plan ~A~].

> It assumes that the user will understand what a just-done stamp is, and how
> it fits into a plan. This really asks too much.
>
This is indeed a very bad error message for users to see.
Admittedly, it's usually a symptom of a bug in the build system (or an
extension thereof), so users ideally never see them.

> It doesn't tell the user what's really wrong. And, I suspect in this case
> it's sort of ASDF's fault, and it's sort of the fault of the system -- the
> system has a bad name (it is a sub-system without a "slashy name"), and
> somehow when we have that, we get these warnings. If that's the case, then
> we should give the user the information about the root cause and suppress
> these warnings, which are just noise.
>
I don't know what the root cause is here, but this looks like the same
bug that is supposed to be fixed, and the fix works for me. Does
Jenkins use some fancy filesystem setup that confuses the timestamp
checking?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Politics is the only profession that does without learning, probably because
those who suffer from mistakes are not the same as those who make them.
— Achille Tournier, Pensées d'automne

Re: Issues with package location information on SBCL

[Faré]

On Mon, Feb 19, 2018 at 7:03 PM, Eric Timmons  wrote:
> Glad to help! I've also opened the following bug on SBCL to let them know
> about it: https://bugs.launchpad.net/sbcl/+bug/1750466.
>
Thanks!

> Also, I checked that nothing else in ASDF uses `parse-define-package-form`,
> but I somehow missed until just now that it's actually exported from uiop
> =/. Is there any concern about another library using it? If so, I can try to
> fix it in such a way that the output of `parse-define-package-form` is
> unchanged.
>
I wouldn't worry about that: grepping through quicklisp systems reveals no user.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
It costs the nation millions to keep Gandhi living in poverty — Sarojini Naidu

Re: Test-multiple

[Faré]

test-multiple works for me with asdf 3.3.1.4, mkcl 1.1.10.19-2dbfa99
on Linux 4.14 x64.

This is all long gone from my mental cache. The test could be better
commented, but I suppose the purpose can be extracted by looking at
its history then looking at related commits, bugs, bug fix commits,
mailing-list messages, etc. A starting point:
git log --stat test/test-multiple.*

Apparently, it tests support for what is now considered misnamed
secondary systems, but was once a kind-of-supported feature, seen in
the wild, with nasty consequences sometimes (e.g. infinite loop with
quicklisp until relevant fix).

A variable not being rebound is a test that a file hasn't been reloaded.

I'd rather not add comments, but I'll review them gladly.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Everyone hates a martyr. It's no wonder martyrs were burnt at a stake.
— E.W. Howe, "Country Town Sayings", p.7



On Mon, Feb 19, 2018 at 5:21 PM, Robert Goldman <rpgold...@sift.info> wrote:
> Faré ---
>
> Would you please add some comments to test-multiple?  I got a failure on
> that with MKCL under jenkins on linux, but cannot replicate that failure
> running it myself.
>
> There's no comment saying what this is supposed to test, other than the
> name, which suggests that it's about testing where there are ... multiple
> systems defined (incorrectly) in one .asd file? the same systems defined in
> multiple .asd files?
>
> The test checks to make sure (I believe) that a variable is not rebound when
> we ask to reload a system, but not how this pertains to correct ASDF
> function.
>
> thanks!
> r
>

Re: Issues with package location information on SBCL

[Faré]

Thanks a lot for fixing this issue! I opened a MR based on it:
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/92

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
To be an anarchist only means that you believe that aggression is not
justified, and that states necessarily employ aggression. And, therefore,
that states, and the aggression they necessarily employ, are unjustified.
It's quite simple, really. It's an ethical view, so no surprise
it confuses utilitarians.
—- N. Stephan Kinsella


On Sat, Feb 17, 2018 at 12:30 AM, Eric Timmons  wrote:
> I've started testing ASDF 3.3.1.3 with my group's code on SBCL 1.4.4
> and noticed some issues with uiop:define-package (due to commit
> 8281e011).
>
> First, when compiling ASDF I get 456 compilation notes
> (https://pastebin.com/NnRUKGWe). I get the same notes when using
> uiop:define-package in our code as well. I honestly think this is an
> issue of SBCL being over aggressive. It's also odd because if the
> recording of the source location is removed, then the notes aren't
> produced. It appears this started happening in SBCL 1.4.1 (potentially
> due to commit af5e2ed1e).
>
> The volume of notes (especially when using package-inferred-system)
> can drown out real issues. I'll likely report this as an issue to SBCL
> since the keywords *should* be constant and the source transform for
> apply doesn't seem to be preserving that. However, it could also be
> fixed in ASDF by changing the apply to a funcall in the define-package
> expansion (patch 2).
>
> Second, the comma on (sb-c:source-location) seems to cause the source
> location for the package to always point to the body of the
> define-package macro. Patch 1 changes this so that form is evaluated
> after the macro is expanded. I didn't test this on anything earlier
> than SBCL 1.4.4, but I don't believe this behavior is version
> dependent.
>
> -Eric

Re: Testing for ASDF 3.3.2 and beyond?

[Faré]

Anton,

thanks a lot for the test! asdf 3.3.1.3 looks good to go.

Robert,

I had no trouble loading clws or lime with clisp, cl-rrt or
cl-sat.minisat.test or inner-conditional-test with sbcl (using asdf
3.3.1.3 from master).

The ecl failures I could reproduce, but I'm not worried, as
lisp-executable is not supported with recent asdf, and a reproducible
out-of-memory error on one system :trivia.balland2006.enabled.test
isn't outrageous.

CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap clisp -I -x "(and '(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:clws)) ())"
CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap clisp -I -x "(and '(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:lime)) ())"

CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap sbcl --eval "(and '(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:cl-rrt)) ())"
CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap sbcl --eval "(and '(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:cl-sat.minisat.test )) ())"
CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap sbcl --eval "(and '(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:inner-conditional-test )) ())"

CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap ecl --eval "'(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:lisp-executable-example)#.(quit))"
CL_SOURCE_REGISTRY='(:source-registry
:ignore-inherited-configuration)' rlwrap ecl --eval "'(#.(load
\"/home/fare/cl/asdf/build/asdf.lisp\") #.(in-package :asdf)
#.(upgrade-asdf) #.(load \"~/quicklisp/setup.lisp\") #.(ql:quickload
:trivia.balland2006.enabled.test )#.(quit))"

Often failures in cl-test-grid are "just" the result of using too
little memory, or batching system loads, or some other reason, and
have to be retried.

I can try to activate that Windows VM and run the all the tests there,
if you want...

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Pain is inevitable. Suffering is optional.



On Fri, Feb 16, 2018 at 11:42 AM, Robert Goldman <rpgold...@sift.net> wrote:
> Thanks for sending that.  Had a quick look.  One nice thing is that there's
> a very limited number of regressions.  I'll look at those when I can.
>
> Faré: I didn't believe it was possible to downgrade ASDF, but we see this
> here in a couple of cases for ECL.  ECL is trying to reload "prebuilt-asdf".
> I think we can ignore these failures on ECL. They just should not do that,
> and it's not really and ASDF test failure if they load a conflicting version
> of ASDF, breaking our upgrade method.
>
> clisp I refuse to care about, since it's effectively abandonware, unless you
> are willing to build from source, which I am not.  Certainly clisp 2.49 is
> abandonware.k
>
> That leaves only the SBCL and ACL failures for us to worry about  I'm
> pretty busy this weekend, but I will have a look.
>
> Best,
> Robert
>
>
> On 15 Feb 2018, at 22:11, Anton Vodonosov wrote:
>
>> Robert, what delay are you apologizing for? I'm aware only of the delay
>> from my side. :)
>>
>>
>>
>> The results for ASDF 3.3.1.3:
>> <https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-72.html>
>>
>> Lisps tested so far:
>>
>>
>>
>> abcl-1.5.0-fasl43-linux-x86
>>
>> acl-10.0s-linux-x86
>>
>> ccl-1.11-r16635-f96-linux-x86
>>
>> clisp-2.49-unix-x86
>>
>> ecl-16.1.2-unknown-linux-x86-bytecode
>>
>> ecl-16.1.2-unknown-linux-x86-lisp-to-c
>>
>> sbcl-1.3.21-linux-x86
>>
>>
>>
>> Best regards,
>>
>> \- Anton
>>
>>
>>
>>
>>
>> 14.02.2018, 22:02, "Robert Goldman" <rpgold...@sift.net>:
>>
>>> OK, as I said, sorry about the delay. Anton, in place of Fare's #3 below,
>>> will you please just test what's in the
>>> `syntax-control-based-on-standard-syntax` branch? The comparison between 2
>>> an

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

>: Attila

> what if i make the .spec generation a standalone operation that needs
> to be explicitly run by the library author?
>
I think that's indeed the right thing to do.

> it's a bit more burden for
> the lib author because he needs to keep track of changes to the
> configuration and/or the .h file, but that would implicitly fix one of
> the bugs.
>
He needs to, anyway, as he writes the wrappers for the library.

> https://github.com/cffi/cffi/commit/d59331b050fb31f3a3f9103f302890be176b77ce
LGTM.

> and shall i do the same with the next stage, namely the .lisp file
> generation? for some reason i think it's better to keep that automatic
> and integrated into the normal flow of the build process, but i'm not
> fully convinced about that either.
>
Generating the Lisp file in advance is good if it allows you to avoid
loading some of the code at build time. It might take more space. Or
you could save the Lisp files and drop the spec files.

> if i did that, it would annull the last remaining issue, namely the
> current unlucky
>
> (load-op ,(find-system "cffi/c2ffi-generator"))
>
> dependency.
>
Indeed, that would be slightly nicer on ASDF.

> but the price would be high: either users would need to regenerate
> .lisp files by hand each time they pull changes to the .spec file
> (this is unacceptable), or all libs would need to check in also the
> generated .lisp files into their repos (this is less of a
> constraint/burden, but still...).
>
> any thoughts?
>
If you generate the Lisp files, do you need to check in the .spec files?

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
- We're all different.
- I'm not!

Testing for ASDF 3.3.2 and beyond?

[Faré]

Dear Anton,

can you run the below tests, in order or priority?

1- Can you test what is currently in master, a.k.a. 3.3.1.3, as a
release candidate for 3.3.2? It has been too long since 3.3.1 was
released with several bugs that have impacted Quicklisp users.

2- Can you test what is currently in the syntax-control branch as a
release candidate for 3.3.3 or 3.4.0? We want to merge syntax control,
but it's a significant enough change that we don't want it at the same
time as the bug fixes. Also...

3- Can you test what is currently in the syntax-control branch as a
release candidate for 3.3.3 or 3.4.0, but with the following addition
just after you load asdf but before you start using it: (defparameter
uiop:*shared-readtable* (copy-readtable nil)) ? Indeed, we want to see
what breaks if we disable extensions implementation-specific reader
extensions. Test most important on CCL. I don't expect much breakage
on other implementations, but it may exist, too.

4- While you're at it, can you also run the test, at least on sbcl,
with (defparameter uiop:*shared-readtable* uiop:*standard-readtable*))
? This will detect what breaks when we make the default readtable
read-only.

The thing is, we really want to have this syntax control, but we also
want to preserve backward compatibility, and we can't make asdf
stricter until every client is fixed (famous last word; of course we
failed at exactly that in 3.3.1 — we could build correctly, but would
also spuriously rebuild if secondary systems were misnamed; fixed in
3.3.1.3).

https://gitlab.common-lisp.net/asdf/asdf/blob/syntax-control/doc/syntax-control.md
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/86
http://blog.quicklisp.org/2018/01/build-failures-with-asdf-331.html

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A friend once asked me if I had ever considered terrorism as a means for
political change. I replied that yes, I had considered it... and rejected it.
Because it only causes change for the worse.
Killing innocent people does not promote a culture of peaceful interaction.

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

One solution is to create a new file with the correct timestamp, that
is either a copy of the existing spec file or a new generated one.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
For followers of most ideologies (openly religious or not), toleration is
a concession of defeat. For libertarians, it is victory itself.


On Thu, Feb 1, 2018 at 7:54 PM, Attila Lendvai  wrote:
>> My theory is that that you're failing to re-generate the .spec when
>> it's present, even though your dependencies say it will be
>> re-generated. Therefore, ASDF decides that it's out-of-date and must
>> be re-generated again the next time over, etc. In other words, you lie
>> to ASDF, and ASDF punishes you right back.
>
>
> that was indeed a lurking bug. excellent remote debugging skills,
> thanks! :)
>
> i pushed the fix. now after i touch'd the appropriate spec files, it
> doesn't want to run c2ffi anymore, but it's still regenerating the
> lisp file unconditionally.
>
> i ran out of hacking steam for today. i'll look into it tomorrow, but
> further remote debugging magic is welcome of course... :)
>
> i'm kinda lost for now. i have no plan of action, so to say... :)
>
> --
> • attila lendvai
> • PGP: 963F 5D5F 45C7 DFCD 0A39
> --
> Liberty is a demand. Tyranny is submission.

Re: Multiple processing compiling the same file

[Faré]

> I suspect this would be one binary for each permutation of the optimization
> settings used times the number of top-level entry points.  Right?  That 
> number is
> much larger than the number of /tmp directories I need just to automatically
> compile before running.
>
No, use cl-launch/dispatch to create a single executable for all entry-points.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
I'm only a stupid AI, but my creator is a real genius!

Re: Multiple processing compiling the same file

[Faré]

>: Jim Newton

> One difficulty about your build-then-deliver suggestion is that my local
> machine is running mac-os, and the cluster is
> running linux.   I don’t think I can build linux executables on my mac.
>
Your build does not have to be "local": pick one random Linux machine,
have it do the compilation, and when it's done, your entire cluster is
ready to start from the compiled executable.

The advantage is that you don't have ugly build race conditions as above.

ASDF's program-op and/or cl-launch will help you build and deliver a
single executable for all your needs. You can even use cl-launch's
multicall capabilities so the same executable has multiple functions.

> For example,what if different hosts want to run the same source code but
> with different optimization settings?
> This is a real possibility, as some of my processes are running with
> profiling (debug 3) and collecting profiling results,
> and others are running super optimized (speed 3) code to try to find the
> fastest something-or-other.
>
Then have one output-translations per optimization setting, and
produce two binaries with different names.

> I don’t even know whether it is possible create the .asd files so that
> changing a optimization declaration will trigger
> everything depending on it to be recompiled.  And If I think i’ve written my
> .asd files as such, how would I know
> whether they are really correct?
>
You need to configure optimization settings in your build script,
after you load asdf and before you use it.
See for instance the snapshot of quux on qitab for how we did it at ITA
(in qres-build).

> It is not the case currently, but may very well be in the future that I want
> different jobs in the cluster running different
> git branches of my code code.  That would be a nightmare to manage if I try
> to share fasl files.
>
Indeed. Build multiple binaries each with its own output-translations,
then distribute the binaries under different names.

> Basel sounds interesting, but I don’t really see the advantage of building
> in parallel when it only
> takes a few seconds to build, but half a day to execute.
>
A split second is better than a few seconds, but yes, if you're the
only user, the cost of setting it up is probably not worth it.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
I'm a polyatheist — there are many gods I don't believe in. — Dan Fouts

Re: Multiple processing compiling the same file

[Faré]

(Sorry for delayed response)

>>>: Jim Newton
>>> If I run several sbcl processes on different nodes in my compute cluster, 
>>> it might happen that
>>> two different runs notice the same file needs to be recompiled (via asdf),
>>> and they might try to compile it at the same time.  What is the best way to 
>>> prevent this?
>>>
You mean that this machines share the same host directory? Interesting.

"Normal" rules of ASDF compile to a temporary file and rename the
output at the end,
thus providing some kind of race resistance. But for
backward-compatibility reasons,
this requires every extension to manually follow a protocol for ASDF
to remain robust.

>>> I see in the asdf documentation that there is an asdf:*user-cache* variable 
>>> whose
>>> value is the path name of the directory where asdf compiles into.
>>> Would it be advisable for my to arrange so that asdf:*user-cache*
>>> is a function of the pid and hostname and perhaps thread-id (if such a 
>>> thing exists) to avoid such collisions?
>>>
That's an option. It is expensive, though: it means no sharing of fasl
files between hosts. If you have cluster of 200 machines, that means
200x the disk space.

What about instead building your application as an executable and
delivering that to the cluster?

My rule of thumb is that there is one home directory per human, and
the human is only interactively building one thing at a time (and/or
can set several accounts and/or $HOME variants for as many
"personalities"). Thus you only need one fasl cache for interactive
compilation. If you want non-interactive deployment, use tools like
bazel, nix, etc., to build your software deterministically.

>>> Or is there some better way to handle this which is build into asdf?
>
You can have different ASDF_OUTPUT_TRANSLATIONS or
asdf:*output-translations-parameter*
on each machine, or you can indeed have the user cache depend on
uiop:hostname and more.

The Right Thing™ is still to build and test then deploy, rather than
deploy then build.
Using Bazel, you might even be able to build in parallel on your cluster.

>>: pjb
>> I had requested that ASDF includes the hostname (or machine-instance), in
>> the built path for the cache.
>> Unfortunately, for some reason, the maintainers of ASDF thought it was a
>> good read to remove it.
>> There you are!
I still think it's a bad idea. If your $HOME is shared by many
machines, you probably want what's in $HOME to be shared, too. Go
build in /var/tmp or use Bazel or whatever. Or use uiop:hostname in
your ASDF configuration.

On Tue, Jan 23, 2018 at 7:51 AM, Jim Newton  wrote:
> Apparently, this approach seems to work.   I’m not sure if it is the best
> approach.
> Here is what my code looks like.  It creates a directory in /tmp/ and
> asdf:load-system
> seems to compile the .fasl files into there.
>
>
> (require :asdf)
> (require :sb-posix)
> (let ((home (directory-namestring (user-homedir-pathname)))
>   (uid (sb-posix:getuid))
>   (pid  (sb-posix:getpid)))
>   (setf asdf::*user-cache* (ensure-directories-exist (format nil
> "/tmp~A~D/~D/" home uid pid
>
I still don't understand why your use case uses deploy-then-build
rather than build-then-deploy.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A child of five would understand this. Send someone to fetch a child of five.
— Groucho Marx

Re: spurious reloads with systems not following the foo/bar naming convention?

[Faré]

On Tue, Jan 30, 2018 at 1:18 PM, Attila Lendvai  wrote:
>> I believe this is the bug that was fixed in 3.3.1.3.
>
> FYI, there's no such tag pushed into the official repo.
> https://gitlab.common-lisp.net/asdf/asdf/tags
>
Robert and I failed to push the 3.3.1.2 and 3.3.1.3 tags. This is now fixed.
But the code was already there in master, and should contain a bug
to the issue you're experiencing (with my apologies again).

> using HEAD, the test case i sent doesn't fail anymore, but our own
> issue still prevails. it's a large environment with many systems, but
> the symptom looks the same, although for us due to cxml.
>
If the issue is the c2ffi issue, see the other thread.

> i couldn't easily test with an older version of ASDF because my SBCL
> already ships with 3.3.0.1 (preventing an easy downgrade), but it used
> to work until recently.
>
One weird trick: use the install-asdf script to overwrite your SBCL's
asdf with an older (or newer) version.

> after i edit cxml.asd to follow the foo/bar convention, i see a few of
> these, that may or may not be the cause of the spurious reload that i
> still see:
>
> WARNING:
>   Computing just-done stamp in plan NIL for action
>   (ASDF/LISP-ACTION:COMPILE-OP "iolib/syscalls" "ffi-types"), but
>   dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/syscalls"
>   "ffi-types") wasn't done yet!
>
> WARNING:
>Computing just-done stamp in plan NIL for action
>(ASDF/LISP-ACTION:COMPILE-OP "iolib/os" "ffi-types"), but
>dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/os" "ffi-types")
>wasn't done yet!
>
> WARNING:
>Computing just-done stamp in plan NIL for action
>(ASDF/LISP-ACTION:COMPILE-OP "iolib/sockets" "grovel"), but
>dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/sockets" "grovel")
>wasn't done yet!
>
> please note that this is not a burning issue for us, it's just a
> friendly heads up with the hopes that it's useful.
>
Thanks for raising the issue while ASDF is still in my L3 cache.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
A word for the epoch of free software and universal publishing:
voluntocracy n 1. governance by those who do the work. 2. the volunteers
who do the work. — Aubrey Jaffer, http://swissnet.ai.mit.edu/~jaffer/

Re: "deprecated recursive use" warning

[Faré]

On Tue, Jan 30, 2018 at 4:53 PM, Attila Lendvai  wrote:
>> I haven't used CFFI in a while.
>
> TL;DR: is this a sane fix?
>
> https://github.com/cffi/cffi/commit/4b9b06f15912e823581b1aeb8a0d5c2ef11f702d
>
(the (not null) ...) is redundant around find-system without the nil
optional argument.

Also, having a component depend on a system violates a weak tacit
invariant in ASDF, though it should only matter for bundle operations,
and only for things required at link-time, which shouldn't be the case
here. Still, something to keep in mind.

> now, whether the generator code (extra dependencies) is needed is
> decided by whether or not the generated lisp file is up-to-date.
>
... except that ASDF only considers it "up-to-date" if its date is
later than that of all dependencies, including the compiled objects
for c2ffi, now that staging is done properly.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Eric Schmidt rule of management: always assume that
the other person is as intelligent as you are.

Re: hu.dwim.zlib broke; broken operation-done-p

[Faré]

On Tue, Jan 30, 2018 at 5:01 PM, Attila Lendvai  wrote:
> if you issue the following (available in quicklisp):
>
> (asdf:load-system :hu.dwim.zlib)
>
> then for the first time it should generate a lisp file, which then gets
> compiled and loaded.
>
> issuing it for the second time shouldn't do anything, but since some
> revisions it regenerates the lisp file every time.
>
> the relevant code is somewhere around here:
>
> https://github.com/cffi/cffi/blob/master/src/c2ffi/asdf.lisp#L135
>
> i tried to trace the INPUT-FILES, OUTPUT-FILES, PERFORM,
> OPERATION-DONE-P methods, but i don't see anything wrong.
>
> one unusual thing is that the output of GENERATE-LISP-OP goes into the
> src/ directory, not to the usual fasl output dir. same applies to
> GENERATE-SPEC-OP.
>
> any ideas how to debug this? or what to look at?
>
I can reproduce the issue using asdf 3.3.1.3 from master (the tag
hadn't been pushed, so I pushed it, but the code was already there).

(plan-actions (nth-value 1 (operate 'load-op :hu.dwim.zlib)))

starts with:
((# . #)
 (# . #)
 (# . #) (# . #)

So somehow the first action above is considered not done. I suppose
it's a bug in c2ffi.

Looking at the c2ffi source, I don't see anything obviously wrong,
though you shouldn't the (the (not null) (find-system ...)) since
find-system will error out for you if the system is not found (to
avoid error, add optional argument nil, as in find-class).

My theory is that that you're failing to re-generate the .spec when
it's present, even though your dependencies say it will be
re-generated. Therefore, ASDF decides that it's out-of-date and must
be re-generated again the next time over, etc. In other words, you lie
to ASDF, and ASDF punishes you right back.

To make ASDF happy, you might instead want to have two disjoint
operation modes, one to generate the spec files, and one to use it.
Normal use mode wouldn't try to generate the file, but instead bork
with a useful error message telling you how to generate it and submit
it upstream.

Another strategy to make ASDF happy would be to detect whether the
file exists as part of input-files and output-files. If it exists,
it's in input-files and not output-files; if not, it's in output-files
and not input-files. There's something I don't like about this
strategy, but it might work (somehow I can't swear it won't bite you
back somehow at some point).

Good luck!

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
There is only one thing more harmful to society than an elected official
forgetting the promises he made in order to get elected; that's when he
doesn't forget them.  — John McCarthy

Re: Including uiop and not asdf in a built application

[Faré]

> :Robert
> Am I correct in thinking that Dave's way of building monolithic bundles of
> either fasls or source code are, at least potentially, a baby version of
> cross-compilation? It seems like these are interesting specifically because
> they could be loaded into different images (otherwise, it's not clear to me
> why it would be better to build a monolithic FASL than just build an image).
>
Indeed, (monolithic) bundles are compiled in one image, loaded in another.
Same implementation, but different image. But then, as much may apply
to regular fasls; and this matters when e.g. some #+ compilation happens
based on user-configured features.

> In that case, since this would effectively be cross-compilation (albeit a
> trivial case of it), it's not surprising that the logic for dealing with
> built-in dependencies like ASDF can go awry.
>
Indeed.

> In which case delivering with Docker might be the better approach! ;-)
>
Did you mean Bazel? :-)

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
What is the shortest joke? — Communism
What is the longest joke? — 70 years of Communism in Russia

Re: Syntax control code

[Faré]

On Sun, Jan 7, 2018 at 6:56 AM, Erik Huelsmann  wrote:
> gitlab is back up now.
>
Thanks a lot, Erik! Congrats for drive-by bugfixing!

So the official URLs are:
Syntax control merge request:
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/86
Current version of the document in the syntax-control branch:
https://gitlab.common-lisp.net/asdf/asdf/blob/syntax-control/doc/syntax-control.md
Final location of the document, once the branch is merged:
https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/syntax-control.md

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Microphones, liberating singers from having to be loud to be heard, gave back
to each language a music that matches its own prosody.

Re: Syntax control code

[Faré]

It's in doc/syntax-control.md in the syntax-control branch (MR !86 on gitlab).
Unhappily, gitlab.common-lisp.net seems to be down right now:
https://gitlab.common-lisp.net/asdf/asdf
If symptom persists, you may have to use my github backup in the meantime.
https://github.com/fare/asdf/blob/syntax-control/doc/syntax-control.md

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
>From a programmer's point of view, the user is a peripheral that types
when you issue a read request. — P. Williams


On Sun, Jan 7, 2018 at 5:24 AM, 73budden .  wrote:
> Hi! Where the document is found?
>
> 2018-01-06 3:53 GMT+03:00, Robert Goldman :
>> I just pushed an edit of syntax-control.md in which I try to capture the
>> terminology.
>>
>> Status: several Allegro failures break for me on test-syntax-control.
>> Results from Linux:
>>
>> build/results/allegro8_64-test.text
>> build/results/allegro8_64_s-test.text
>> build/results/allegromodern8_64-test.text
>> build/results/allegromodern8_64_s-test.text
>> build/results/allegromodern_64-test.text
>> build/results/allegromodern_64_s-test.text
>>
>> These failures seem to be due to NAMED-READTABLES not working properly
>> on these platforms, rather than on anything ASDF itself does.
>>
>> Concern:  As I was reading over syntax-control.md, it was brought home
>> to me that the ASDF shared syntax is initialized to the *initial syntax*
>> on the host implementation, rather than the standard syntax of ANSI CL.
>> My understanding is that this is done for backwards-compatibility with
>> some QL systems that assume they have access to extended syntax from
>> some implementation(s).  I'm concerned that this will create a
>> maintenance headache going forward just so someone *else* can avoid
>> making some minor clean-up.  Should we just suck it up and make the
>> shared syntax start out with the initial syntax?  Why not break it now,
>> and save ourselves trouble later?  Also, it seems like "initial syntax"
>> is not well-defined, even on a single implementation, since ASDF might
>> be loaded at arbitrary times, possibly after modifications to the
>> "initial initial" readtable.  Finally, going forward, people will be
>> yelling at *us* if implementations change their initial syntax.
>>
>> Unless there's a really important reason to keep this, I think we should
>> kill it.
>>
>>
>

One bug down, one bug up

[Faré]

Dear future maintainers of ASDF,

I have great news for you if you were looking for an opportunity to
learn about the guts of ASDF: a new bug that will cover a lot of
ground, yet is relatively well-understood, and isn't urgent at all. I
wouldn't make it anyone's first bug, but once you have some
experience, it might be great bug to get really into it.

Context: I did this interactive debugging session, but it went quite
slow and after two hours we only narrowed down the root issue: we
needed the primary-system-* functions to distinguish between
"syntactic" and "semantic" primariness, since the two notions aren't
identical in the existing codebase, and my just ignoring the issue
caused these spurious rebuilds. I once again failed to properly record
the 2.5 hour interaction, but one of the participants might have done
so, though it's not as valuable as I'd have hoped. Then,
It took me the next morning to slowly develop a bug fix. See
explanation in the bug and code in the MR:
https://bugs.launchpad.net/asdf/+bug/1739514
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/91

Now now now, while thinking very hard about the correct fix to the
above bug, and especially about the up-to-date-p condition in
find-system, I realized that there was a lingering bug in it, of the
kind order as the bug that broke ASDF 2, except in a much rarer
situation: if you update the defsystem-depends-on of a loaded system a
that could cause a change in how the a.asd file would be evaluated;
then this should trigger a reload of a.asd when you next operate on a;
but it won't because the timestamp is not propagated, because
find-system fails to integrate well enough with the dependency graph
usage and construction (which integration is new with 3.3).
https://bugs.launchpad.net/asdf/+bug/1741083

This seems to be like it's "just" a variant at the meta-level of the
famous bug that broke ASDF 2 (see my 25 page paper on the history of
ASDF about that). Except that now the infrastructure to propagate
timestamps, what more across evaluation phases, already exists;
therefore fixing this bug should mostly "just" use the existing
infrastructure. Then again, it could possibly break the universe
again, though I bet it won't: I believe there should be "just" an
update of find-system and the creation of a new operation class
prepare-define-op.

In any case, not my problem anymore, because it's clearly not a
regression. But a great opportunity for whoever steps up and fixes
this bug and whatever else they find. You better step forward while
I'm still here to answer questions: considering how hard it was for me
to page back the crucial elements of the algorithm into memory after
just one year, don't wait.

PS: Another exercise for a future ASDF maintainer would be to do the
boring work and fix all the 300+ systems on Quicklisp that trigger the
bad-system-name warning, one by one. This is also a great exercise for
getting to know what kind of clients are around. And for fixing them
according to my best_practices document.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Every great truth begins as heresy and ends as superstition.
— Thomas Henry Huxley

Re: ASDF debugging on Hangouts?

[Faré]

OK, I'm setting the day and time for the next ASDF debugging session
to Tuesday January 2nd 2018 at 14:00 EST (19:00 UTC). Please send me a
private email if you can make it, I'll add you to a Google Calendar
event with a Hangout invitation.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
When it comes to giving, some men stop at nothing. — Saul Gorn


On Wed, Dec 27, 2017 at 6:06 PM, Faré <fah...@gmail.com> wrote:
> On Wed, Dec 27, 2017 at 10:33 AM, Robert Goldman <rpgold...@sift.info> wrote:
>> On 4 Dec 2017, at 21:56, Faré wrote:
>>> https://bugs.launchpad.net/asdf/+bug/1739514
>>> Basically, ASDF fails spuriously rebuilds misnamed secondary systems
>>> and/or things that depend on them, instead of just issuing a warning
>>> as intended.
>>
>> Another editing query here.  Looks like it might be either "ASDF fails to 
>> rebuild" or "ASDF spuriously rebuilds misnamed..."
>>
> Yes, it's "ASDF spuriously rebuilds misnamed...".
>
> Note that if it were a simple circular dependency, it would have just
> broken the build and be very visible and I'd have caught it last year.
>
> Without having looked at the code or tried to debug it yet, my working
> hypothesis is that having "foo" in "foo.asd" depend on "bar" also
> defined in "foo.asd" causes dependency from (define-op "foo") to
> (define-op . "bar"), which is always in need of build because there is
> no "bar.asd" and the associated timestamp is therefore NIL. If that is
> correct, the backward-compatible solution would be to make sure that
> "bar" remembers that it was defined in "foo.asd", so that it gets the
> timestamp from "foo.asd", and the next time around, if there is no
> up-to-date "bar.asd", ASDF falls back on looking at the previously
> loaded "foo.asd" or a more up-to-date version of it.
>
> Note that if "bar" is defined in both "foo.asd" and "bar.asd" you'll
> still have a mighty bug. Therefore people should still fix their code
> to properly name secondary system, ASDF should still issue a warning
> when they are misnamed, and this warning should still be upgraded to a
> cerror then an error when all of Quicklisp is fixed (>300 systems).

Re: ASDF debugging on Hangouts?

[Faré]

On Wed, Dec 27, 2017 at 10:33 AM, Robert Goldman <rpgold...@sift.info> wrote:
> On 4 Dec 2017, at 21:56, Faré wrote:
>
>> Fixing the same potential issue with the (more stable but still
>> evolvable) code in the core of ASDF will require applying the same
>> solution to action.lisp all files that define new operations.
>
> Will you please clarify this sentence? It looks like some editing error cost
> us some key words.  Did you originally write "to action.lisp" and then
> decide it had to change to "to all files that define new operations"?
>
Oops, I meant to action.lisp AND all files that define new operations.
Fixing action.lisp is trivial: change all the operation slots to methods.
But *first* you need to change all clients to do the same, probably over a year
(doing it in the other order doesn't work).

Alternatively, demand that suitable
update-instance-for-redefined-class methods be written; but it's just
as much work, and much more fragile. It's also slightly more backward
compatible, especially if are aggressive bumping
*oldest-forward-compatible-asdf-version*, but the fragility means that
if someone makes an API call with an operation created before upgrade
then things might go wrong (the operate :around method will catch that
in the common case; but your uncommon case will be "interesting" to
debug).

>> https://bugs.launchpad.net/asdf/+bug/1739514
>> Basically, ASDF fails spuriously rebuilds misnamed secondary systems
>> and/or things that depend on them, instead of just issuing a warning
>> as intended.
>
> Another editing query here.  Looks like it might be either "ASDF fails to 
> rebuild" or "ASDF spuriously rebuilds misnamed..."
>
Yes, it's "ASDF spuriously rebuilds misnamed...".

Note that if it were a simple circular dependency, it would have just
broken the build and be very visible and I'd have caught it last year.

Without having looked at the code or tried to debug it yet, my working
hypothesis is that having "foo" in "foo.asd" depend on "bar" also
defined in "foo.asd" causes dependency from (define-op "foo") to
(define-op . "bar"), which is always in need of build because there is
no "bar.asd" and the associated timestamp is therefore NIL. If that is
correct, the backward-compatible solution would be to make sure that
"bar" remembers that it was defined in "foo.asd", so that it gets the
timestamp from "foo.asd", and the next time around, if there is no
up-to-date "bar.asd", ASDF falls back on looking at the previously
loaded "foo.asd" or a more up-to-date version of it.

Note that if "bar" is defined in both "foo.asd" and "bar.asd" you'll
still have a mighty bug. Therefore people should still fix their code
to properly name secondary system, ASDF should still issue a warning
when they are misnamed, and this warning should still be upgraded to a
cerror then an error when all of Quicklisp is fixed (>300 systems).

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Rocky's Lemma of Innovation Prevention
Unless the results are known in advance, funding agencies will
reject the proposal.

Re: ASDF debugging on Hangouts?

[Faré]

Why not make a backup and update all of quicklisp? Mixing and matching
software of vastly different versions can never be supported. This is
a hell of your own making.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Amateurs talk strategy. Professionals talk logistics. — old military saying


On Tue, Dec 26, 2017 at 2:31 PM, Ken Tilton <kentil...@gmail.com> wrote:
> Well, the cffi loaded by ql is:
> beta.quicklisp.org/archive/cffi/2015-09-23/cffi_0.16.1.tgz
>
> The list of beta releases at https://www.quicklisp.org/beta/releases.html
> shows 0.19.0
>
> Uninstalling and reinstalling pulled down cffi_0.16.1 again.
>
> I'll have to look at a manual override.
>
>
>
> On Tue, Dec 26, 2017 at 2:05 PM, Ken Tilton <kentil...@gmail.com> wrote:
>>
>> Thanks for the info. But Ceramic just built on the same windows laptop
>> under ccl64. Maybe the asdf AllegroCL provides is getting in the way? I will
>> look also at tossing cffi just to get a fresh start.
>>
>> -kt
>>
>> On Tue, Dec 26, 2017 at 1:55 PM, Faré <fah...@gmail.com> wrote:
>>>
>>> >> Error: OPERATION instances must only be created through
>>> >> MAKE-OPERATION.
>>> >>
>>> >> [condition type: FORMATTED-SYSTEM-DEFINITION-ERROR]
>>> >
>>> >
>>> > Is that an ASDF issue? Ceramic? ACL? cffi-grovel (the system being
>>> > built
>>> > when the error is thrown)?
>>> >
>>> You need a fresher version of cffi-grovel. Update your quicklisp
>>> and/or make sure you don't have an antique version of cffi in your
>>> local-projects and/or source-registry.
>>>
>>> > Does quicklisp use its own copy of ASDF for stability?
>>> >
>>> quicklisp provides an antique asdf 2.26 that is only loaded in even
>>> more antique implementations. All maintained implementations have been
>>> providing asdf 3.1 or later for over a year.
>>>
>>> —♯ƒ • François-René ÐVB Rideau •Reflection•
>>> http://fare.tunes.org
>>> C is a DSL for turning low-level byte arrays into security advisories.
>>>
>>
>>
>>
>> --
>> Kenneth Tilton
>> http://tiltontec.com/
>
>
>
>
> --
> Kenneth Tilton
> http://tiltontec.com/

Re: ASDF debugging on Hangouts?

[Faré]

>> Error: OPERATION instances must only be created through MAKE-OPERATION.
>>
>> [condition type: FORMATTED-SYSTEM-DEFINITION-ERROR]
>
>
> Is that an ASDF issue? Ceramic? ACL? cffi-grovel (the system being built
> when the error is thrown)?
>
You need a fresher version of cffi-grovel. Update your quicklisp
and/or make sure you don't have an antique version of cffi in your
local-projects and/or source-registry.

> Does quicklisp use its own copy of ASDF for stability?
>
quicklisp provides an antique asdf 2.26 that is only loaded in even
more antique implementations. All maintained implementations have been
providing asdf 3.1 or later for over a year.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
C is a DSL for turning low-level byte arrays into security advisories.

Re: Relativising FASL Cache Paths

[Faré]

On Sun, Dec 17, 2017, 06:08 Nicolas Hafner  wrote:

> On 17/12/17 12:02, 73budden . wrote:
>
> Nicolas, can you resolve source locations at SWANK level? E.g.
> functions like SWANK/SBCL::SOURCE-FILE-SOURCE-LOCATION might be
> appropriate for patching: you take the location that SBCL gives you,
> but then redirect SWANK to other place. I guess this is not the only
> place you have to patch, it would require patching private functions
> and it might not work at all. But if that works for you please let me
> know. I'm also maintaining a thing a bit similar to portacle and it
> looks like I would suffer from the same issue.
>
> That sounds like a really nasty workaround. I'd much rather try to get
> ASDF to use logical pathnames.
>
Logical pathnames are never the solution, especially not if portability
means anything, and if SBCL is targeted in particular.

Too many systems have file names that just can't be valid logical pathnames
(that are limited strictly to single-case letters, digits, and dashes, plus
a dot only to separate the similarly-limited type). Underscores, mixed
case, wrong case, plus sign, dot, and whatever else are not allowed, yet
may be implied by the .asd file.

Good luck convincing all authors in Quicklisp of renaming their files and
directories (though this renaming could conceivably be done automatically
after the fact, combined with large logical pathname mapping tables, for
atrocious performance and horrible bugs when that fails; good luck with
that kluge).

I recommend aiming for a true solution, which implies adding the capability
of overriding the pathname to SBCL's compile-file then UIOP's compile-file*
and ASDF's compile-op, and/or non-compliant lax pass-through logical
pathnames a la CCL.

-#f

Re: Relativising FASL Cache Paths

[Faré]

Source location via logical pathnames works great on CCL. However, on SBCL,
they run afoul of SBCL's strict enforcement of the standard limitations to
logical pathnames, and you will have quite a hard time with them if you
succeed at all.

One easy but fragile and ugly way to make it work with SBCL would be to
fake logical pathnames using a fixed symlink in $TMPDIR.

More correct solutions that require more hacking include: adding an extra
argument to compile-file by hacking both SBCL and ASDF, to let you override
the default source location stored in a file; or maybe if you're hacking
SBCL, add a non standard logical pathname option. Etc.

My recommendation would be to strive to upstream your patch to whichever
software you hack, and to that end, to speak with the maintainers about
what they're willing to commit.

-#f

On Fri, Dec 15, 2017, 07:32 Nicolas Hafner <shinm...@tymoon.eu> wrote:

> I see, thanks a lot for the hint.
>
> I've just stumbled upon another complicating factor though, which is
> source location information. Apparently it is not possible to manually
> relocate source information after it has been loaded from a FASL in SBCL.
> However, if the Lisp files were compiled using a logical pathname, logical
> pathname translations could be used to fix that up as required.
>
> Is there a way to transform the input/output files to logical pathnames to
> facilitate this, or would that be too much to ask for?
>
> On 15/12/17 11:44, Faré wrote:
>
> The design of ASDF is that you should properly initialize the
> output-translations. The usual way is to use
> ~/.config/common-lisp/asdf-output-translations.conf, but since in your
> case you support the directory moving from one instantiation to the
> next, it is probably better to call
> asdf:initialize-output-translations at startup with a proper argument
> (using properly computed absolute pathnames), just after you (require
> "asdf") (which I assume is possible and yielding no older than 3.1.2)
> but before you compile anything with it. Untested template that might
> work:
>
> (asdf:initialize-output-translations
>   `(:output-translations
>   (,(uiop:wilden *portacle-directory*) ,(uiop:wilden
> (uiop:subpathname *cache-top* "portacle/")))
>   (t (uiop:wilden (uiop:subpathname *cache-top* "other/")))
> :ignore-inherited-configuration))
>
> Note that you may choose to use the :inherit-configuration instead.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> If soldiers are not to cross international boundaries, goods must do so. 
> Unless
> the Shackles can be dropped from trade, bombs will be dropped from the sky.
> — Otto T. Mallery
>
>
> On Fri, Dec 15, 2017 at 4:41 AM, Nicolas Hafner <shinm...@tymoon.eu> 
> <shinm...@tymoon.eu> wrote:
>
> Hello everyone.
>
> I have a rather particular problem that I have so far not been able to solve
> on my own without brittle workarounds. Specifically, I'm looking for a way
> to ensure that the directory paths stored in the FASL cache directory are,
> if possible, truncated according to some other path (by enough-pathname or a
> similar facility).
>
> The reason I want this is for Portacle. The Portacle directory should be
> relocatable and you should even be able to take it with you on a flash drive
> to run on other machines. However, this usually means that the absolute
> position of the directory on the file system will change, even if the Lisp
> files and FASLs within it remain at the same relative locations.
>
> I would thus like to stop ASDF from recompiling everything when the
> directory is moved absolutely, and instead rely on paths relative to
> Portacle's own "root" directory if possible.
>
> My current approach involves setting
> asdf/output-translations::*output-translation-function* to a modified
> version that attempts to relativise the path as seen here. However, this
> sometimes leads to issues as the translated path might be relative and other
> components down the line seem to expect absolute paths.
>
> Any advice on how to proceed on this would be appreciated.
>
> Sincerely, Nicolas Hafner
>
>
>

Re: Relativising FASL Cache Paths

[Faré]

The design of ASDF is that you should properly initialize the
output-translations. The usual way is to use
~/.config/common-lisp/asdf-output-translations.conf, but since in your
case you support the directory moving from one instantiation to the
next, it is probably better to call
asdf:initialize-output-translations at startup with a proper argument
(using properly computed absolute pathnames), just after you (require
"asdf") (which I assume is possible and yielding no older than 3.1.2)
but before you compile anything with it. Untested template that might
work:

(asdf:initialize-output-translations
  `(:output-translations
  (,(uiop:wilden *portacle-directory*) ,(uiop:wilden
(uiop:subpathname *cache-top* "portacle/")))
  (t (uiop:wilden (uiop:subpathname *cache-top* "other/")))
:ignore-inherited-configuration))

Note that you may choose to use the :inherit-configuration instead.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
If soldiers are not to cross international boundaries, goods must do so. Unless
the Shackles can be dropped from trade, bombs will be dropped from the sky.
— Otto T. Mallery


On Fri, Dec 15, 2017 at 4:41 AM, Nicolas Hafner  wrote:
> Hello everyone.
>
> I have a rather particular problem that I have so far not been able to solve
> on my own without brittle workarounds. Specifically, I'm looking for a way
> to ensure that the directory paths stored in the FASL cache directory are,
> if possible, truncated according to some other path (by enough-pathname or a
> similar facility).
>
> The reason I want this is for Portacle. The Portacle directory should be
> relocatable and you should even be able to take it with you on a flash drive
> to run on other machines. However, this usually means that the absolute
> position of the directory on the file system will change, even if the Lisp
> files and FASLs within it remain at the same relative locations.
>
> I would thus like to stop ASDF from recompiling everything when the
> directory is moved absolutely, and instead rely on paths relative to
> Portacle's own "root" directory if possible.
>
> My current approach involves setting
> asdf/output-translations::*output-translation-function* to a modified
> version that attempts to relativise the path as seen here. However, this
> sometimes leads to issues as the translated path might be relative and other
> components down the line seem to expect absolute paths.
>
> Any advice on how to proceed on this would be appreciated.
>
> Sincerely, Nicolas Hafner

Re: ASDF + QL maintenance needed.

[Faré]

On Mon, Dec 11, 2017 at 2:13 PM, Faré <fah...@gmail.com> wrote:
> ASDF needs volunteers to replace its retiring maintainers.
>
> One easy way to start, and a good filter for people non ready for the
> job, is the simple and boring yet essential task of looking at
> Quicklisp build failures and either fixing other people's build files,
> or fixing ASDF, depending on who introduced the bug (often a
> combination of the two).
>
> Recent Quicklisp build report:
> http://report.quicklisp.org/2017-12-11/failure-report.html
>
> I've started these two pull requests:
> https://github.com/marijnh/Postmodern/pull/115
> https://github.com/zkat/chanl/pull/12
Also
https://gitlab.common-lisp.net/f2cl/f2cl/merge_requests/4

Here is another interesting breakage, that calls for an active new
ASDF maintainer:
https://github.com/Shinmera/qtools/issues/25#issuecomment-350873853
qt-libs tries to hot-patch commonqt to work around the way it builds and loads
a C library, and ASDF 3.3 broke that. While a new set of workarounds
is possible,
a real fix, i.e. really teaching ASDF about C libraries, will require
a maintainer
with a good vision and the will to make things work --- and that's not me.
Or you can all migrate to bazel.

My offers to help a new maintainer get started still holds, so far.
But my memory of ASDF won't hold indefinitely:

> If there are candidate maintainers, we could do some more of these together.
> This would build up:
> * Good practice for dealing with build issues in quicklisp
> * Knowledge of what is or isn't the contract provided by ASDF 3.3.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Those who hammer their guns into plows will plow for those who do not.
— Thomas Jefferson

ASDF + QL maintenance needed.

[Faré]

ASDF needs volunteers to replace its retiring maintainers.

One easy way to start, and a good filter for people non ready for the
job, is the simple and boring yet essential task of looking at
Quicklisp build failures and either fixing other people's build files,
or fixing ASDF, depending on who introduced the bug (often a
combination of the two).

Recent Quicklisp build report:
http://report.quicklisp.org/2017-12-11/failure-report.html

I've started these two pull requests:
https://github.com/marijnh/Postmodern/pull/115
https://github.com/zkat/chanl/pull/12

If there are candidate maintainers, we could do some more of these together.
This would build up:
* Good practice for dealing with build issues in quicklisp
* Knowledge of what is or isn't the contract provided by ASDF 3.3.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
If instead of teaching other people what government should be and should do,
you'd teach yourself what government actually is and does do,
you'd be a libertarian.

Re: ASDF debugging on Hangouts?

[Faré]

My second attempt at a live session was also ultimately a failure:
there were interruptions, the rhythm was slow with lots of side issues,
the microphone was unplugged at one point, etc.,
and it all lasted way too many hours with lots of down time.
Happily, none of the two anonymous spectators stayed long anyway, so
no one missed much.

That said, I battled and vanquished an interesting conundrum of bugs,
all of them
directly or indirectly related to code upgrade:

* Simplest and most obvious issue: ASDF 3.3.1 renamed STAMP< to
TIMESTAMP< because the API changed; this caused a naming conflict with
LOCAL-TIME:TIMESTAMP<. Solution: in WORKOUT-TIMER's
UIOP:DEFINE-PACKAGE, use the :MIX clause with LOCAL-TIME ahead of
UIOP, instead of :USE. The backtrace pointed to the correct issue, the
ASDF changelog and/or commit messages explained what it was about,
this was obvious to find out and fix.
https://gitlab.common-lisp.net/frideau/workout-timer/commit/0e3f0104ddb26524f8445ddcf5e34dafe43aa8dd

* I had somehow compiled my SBCL with an old script that hadn't set
the SB-LINKABLE-RUNTIME feature; but CFFI-TOOLCHAIN failed to tell me,
instead failing in a cryptic way. Solution: have CFFI proactively
check for feature presence and issue a useful error message if absent.
(This is loosely related to SBCL, because CFFI failed to detect that
SBCL hadn't been upgraded to be compiled with this recent feature I
added.) This issue was compounded by the fact that, while believing
that it was an ASDF bug and trying to check at what point the bug
appeared, other issues kept cropping up because of the bug below, and
the fact that downgrading ASDF is itself quite tricky, unlike
upgrading which is trivial.
https://github.com/cffi/cffi/pull/127

* ASDF stored metadata in the :initform of operation class slots, but
then changes due to code upgrade were not visible because the slot was
already initialized (yet upgrade previously worked, once, due to
change of :allocation to :class). Solution: use defmethod, not slots.
This is a deep issue. At root was my failing to take measure of how
:initform would ultimately interact with software upgrade, when I
originally wrote this code. And yet it worked well enough so far.
Fixing the same potential issue with the (more stable but still
evolvable) code in the core of ASDF will require applying the same
solution to action.lisp all files that define new operations. This
will impact backward compatibility and will require yet another round
of going through Quicklisp fixing everything (not that many systems
define new operations, but some do), and making big announcements. ---
Good task for a candidate future maintainer: well defined and
predictable, and yet will get you around the core code of ASDF.
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/89

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
If you don't like yourself, you *can't* like other people.
— Robert Heinlein, "Time Enough For Love"


On Mon, Dec 4, 2017 at 11:29 AM, Faré <fah...@gmail.com> wrote:
> Well, after realizing one hour into the debugging session that I
> needed to click on a button "Start Broadcast" to go live, I'm going to
> reschedule the event, starting at 14:00 EST (19:00 UTC). Sorry.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> How small of all that human hearts endure
> That part which laws or kings can cause or cure! — Samuel Johnson
>
>
> On Wed, Nov 29, 2017 at 7:24 PM, Faré <fah...@gmail.com> wrote:
>> After a kernel downgrade, I have painfully managed to get streaming to
>> Youtube Live Events working.
>> https://www.youtube.com/my_live_events
>>
>> I'm tentatively scheduled an event at 10:00 EST (15:00 UTC) on next
>> Monday December 4th 2017.
>> https://www.youtube.com/watch?v=1kq-73Cjn08
>>
>> I'll be using Hangouts on Air, and will send a message to these lists
>> with the link before I go live,
>> and also posting the link on my twitter https://twitter.com/Ngnghm
>>
>> If you have requests for an alternate time slot, I'm open to moving the 
>> session.
>>
>> —♯ƒ • François-René ÐVB Rideau •Reflection• 
>> http://fare.tunes.org
>> What one person receives without working for, another person must work for
>> without receiving. — Adrian Rogers

Re: ASDF debugging on Hangouts?

[Faré]

Well, after realizing one hour into the debugging session that I
needed to click on a button "Start Broadcast" to go live, I'm going to
reschedule the event, starting at 14:00 EST (19:00 UTC). Sorry.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
How small of all that human hearts endure
That part which laws or kings can cause or cure! — Samuel Johnson


On Wed, Nov 29, 2017 at 7:24 PM, Faré <fah...@gmail.com> wrote:
> After a kernel downgrade, I have painfully managed to get streaming to
> Youtube Live Events working.
> https://www.youtube.com/my_live_events
>
> I'm tentatively scheduled an event at 10:00 EST (15:00 UTC) on next
> Monday December 4th 2017.
> https://www.youtube.com/watch?v=1kq-73Cjn08
>
> I'll be using Hangouts on Air, and will send a message to these lists
> with the link before I go live,
> and also posting the link on my twitter https://twitter.com/Ngnghm
>
> If you have requests for an alternate time slot, I'm open to moving the 
> session.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> What one person receives without working for, another person must work for
> without receiving. — Adrian Rogers

Re: ASDF debugging on Hangouts?

[Faré]

After a kernel downgrade, I have painfully managed to get streaming to
Youtube Live Events working.
https://www.youtube.com/my_live_events

I'm tentatively scheduled an event at 10:00 EST (15:00 UTC) on next
Monday December 4th 2017.
https://www.youtube.com/watch?v=1kq-73Cjn08

I'll be using Hangouts on Air, and will send a message to these lists
with the link before I go live,
and also posting the link on my twitter https://twitter.com/Ngnghm

If you have requests for an alternate time slot, I'm open to moving the session.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
What one person receives without working for, another person must work for
without receiving. — Adrian Rogers