Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-09 Thread Robert Goldman
I think Xach and I are trying to cope with essentially the same problem: 
lack of resources for a community project.


On the one hand, Xach wants to stick with and old ASDF because it's a 
lot of trouble to maintain it, and he's afraid of breakage.


OTOH, we don't have the resources to maintain backwards-compatibility in 
ASDF (though we try not to break things gratuitously).  Keeping ancient 
unmaintained software running at the cost of introducing complexity into 
ASDF is something I simply can't do (when I transfer maintenance to 
Microsoft, they'll have that covered!).


Faré has worked heroically (with admirable assistance from Anton 
Vodonosv) to keep from breaking libraries and offering patches, but at 
the end of the day, if there's no one available to merge patches into 
new releases of libraries, that can't be a reason to veto introduction 
of a fix into ASDF.  If there's no one available to change `foo-test` 
into `foo/test` in `crusty-library.asd` that's not really our fault (and 
really, should people be using `crusty-library.asd` if there isn't?). 
Naturally, that has the potential to make Xach's life miserable.  It's 
not my fault or his.


When there's stuff like `crusty-library` out there, there will be a 
certain amount of misery to be had, and there will always be a tension 
between QL and ASDF about who gets to enjoy it.


I wish that I could coordinate more effectively with Xach, but I really 
don't know what I can do to make this work better.


Best,
r



On 9 Apr 2018, at 12:13, Faré wrote:

On Mon, Apr 9, 2018 at 12:22 PM, Robert Goldman  
wrote:

On 9 Apr 2018, at 11:17, Attila Lendvai wrote:

A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it 
might

cause
errors in its present form.




i've learned, painfully, that indiscriminate ignore-errors will 
almost
always bite you back (in the form of wasted debugging time), no 
matter

how innocent they look.


This really is more a QL issue than an ASDF one (although it 
illustrates

an
issue with wrapping errors, IMO).


while this is true, the implementation of QL requires a reliable way
to hook into the internal state of various versions of ASDF (namely,
into the situation when ASDF is looking for a system, and not 
finding
it will lead to an error without QL intervention). if i understand 
it

correctly, this is the crux of this issue.


Sure, and I am happy to try to support this, but not to the extent of
recovering a copy of ASDF 2.x and trying to run it.

The problem is that I don't know when the missing component condition 
was
added to ASDF, and doing this right would involve checking the 
enclosed
condition to see if it's a missing component error.  I know how to do 
that
for a modern ASDF, but I don't know how to handle ASDFs that are too 
old to
have this condition class.  And I don't fee like it's my job to think 
about

that: I think it's perverse to continue trying to use ASDF 2.


ASDF 2.26 is totally unsupported at this point. No implementation uses
anything less than 3.1.2 (the first stable release in the 3.1 series,
from May 2014). Many essential packages require 3.1.x or later. It is
a waste of time to try to get 2.26 running

If Quicklisp wants to be conservative, I would recommend requiring ior
providing ASDF 3.1.7, which is the last in the 3.1 series and pretty
stable, from March 2016, which is two years ago.

The "official" policy of ASDF was always to not support anything that
had already been superseded 2 years ago or more. 3.1.7 should be the
oldest supported ASDF release.

That said, what my opinions seem to be negatively taken into account
by Xach, so there.

—♯ƒ • François-René ÐVB Rideau •Reflection• 
http://fare.tunes.org

Evolution competitively selects stable cooperative patterns.


Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-09 Thread Faré
On Mon, Apr 9, 2018 at 12:22 PM, Robert Goldman  wrote:
> On 9 Apr 2018, at 11:17, Attila Lendvai wrote:
>
>>> A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might
>>> cause
>>> errors in its present form.
>>
>>
>>
>> i've learned, painfully, that indiscriminate ignore-errors will almost
>> always bite you back (in the form of wasted debugging time), no matter
>> how innocent they look.
>>
>>
>>> This really is more a QL issue than an ASDF one (although it illustrates
>>> an
>>> issue with wrapping errors, IMO).
>>
>> while this is true, the implementation of QL requires a reliable way
>> to hook into the internal state of various versions of ASDF (namely,
>> into the situation when ASDF is looking for a system, and not finding
>> it will lead to an error without QL intervention). if i understand it
>> correctly, this is the crux of this issue.
>
> Sure, and I am happy to try to support this, but not to the extent of
> recovering a copy of ASDF 2.x and trying to run it.
>
> The problem is that I don't know when the missing component condition was
> added to ASDF, and doing this right would involve checking the enclosed
> condition to see if it's a missing component error.  I know how to do that
> for a modern ASDF, but I don't know how to handle ASDFs that are too old to
> have this condition class.  And I don't fee like it's my job to think about
> that: I think it's perverse to continue trying to use ASDF 2.
>
ASDF 2.26 is totally unsupported at this point. No implementation uses
anything less than 3.1.2 (the first stable release in the 3.1 series,
from May 2014). Many essential packages require 3.1.x or later. It is
a waste of time to try to get 2.26 running

If Quicklisp wants to be conservative, I would recommend requiring ior
providing ASDF 3.1.7, which is the last in the 3.1 series and pretty
stable, from March 2016, which is two years ago.

The "official" policy of ASDF was always to not support anything that
had already been superseded 2 years ago or more. 3.1.7 should be the
oldest supported ASDF release.

That said, what my opinions seem to be negatively taken into account
by Xach, so there.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Evolution competitively selects stable cooperative patterns.



Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-09 Thread Robert Goldman

On 9 Apr 2018, at 11:17, Attila Lendvai wrote:

A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it 
might cause

errors in its present form.



i've learned, painfully, that indiscriminate ignore-errors will almost
always bite you back (in the form of wasted debugging time), no matter
how innocent they look.


This really is more a QL issue than an ASDF one (although it 
illustrates an

issue with wrapping errors, IMO).



while this is true, the implementation of QL requires a reliable way
to hook into the internal state of various versions of ASDF (namely,
into the situation when ASDF is looking for a system, and not finding
it will lead to an error without QL intervention). if i understand it
correctly, this is the crux of this issue.


Sure, and I am happy to try to support this, but not to the extent of 
recovering a copy of ASDF 2.x and trying to run it.


The problem is that I don't know when the missing component condition 
was added to ASDF, and doing this right would involve checking the 
enclosed condition to see if it's a missing component error.  I know how 
to do that for a modern ASDF, but I don't know how to handle ASDFs that 
are too old to have this condition class.  And I don't fee like it's my 
job to think about that: I think it's perverse to continue trying to use 
ASDF 2.


Best,
r



Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-09 Thread Attila Lendvai
> A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might cause
> errors in its present form.


i've learned, painfully, that indiscriminate ignore-errors will almost
always bite you back (in the form of wasted debugging time), no matter
how innocent they look.


> This really is more a QL issue than an ASDF one (although it illustrates an
> issue with wrapping errors, IMO).


while this is true, the implementation of QL requires a reliable way
to hook into the internal state of various versions of ASDF (namely,
into the situation when ASDF is looking for a system, and not finding
it will lead to an error without QL intervention). if i understand it
correctly, this is the crux of this issue.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“A general State education is a mere contrivance for molding people to
be exactly like one another; and as the mold in which it casts them is
that which pleases the dominant power in the government […]; it
establishes a despotism over the mind, leading by a natural tendency
to one over the body.”
— John Stuart Mill (1806–1873), 'On Liberty' (1869)



Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-09 Thread Robert Goldman

On 7 Apr 2018, at 15:56, Robert Dodier wrote:


Thanks to everyone for their interest in this topic.

In reference to
https://github.com/quicklisp/quicklisp-client/pull/128/, it is only a
minimal stop-gap measure which I found could fix the immediate problem
which I encountered. I know very little about ASDF and Quicklisp, so I
certainly had no conscious intent to assert that one condition or
another must prevail. Any improvements by more knowledgeable people
would be welcomed by me.

All the best,
Robert Dodier


A cheesy fix would simply be to wrap it in `IGNORE-ERRORS`.  But it 
might cause errors in its present form.


This really is more a QL issue than an ASDF one (although it illustrates 
an issue with wrapping errors, IMO).


Best,
r


Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-07 Thread Robert Dodier
Thanks to everyone for their interest in this topic.

In reference to
https://github.com/quicklisp/quicklisp-client/pull/128/, it is only a
minimal stop-gap measure which I found could fix the immediate problem
which I encountered. I know very little about ASDF and Quicklisp, so I
certainly had no conscious intent to assert that one condition or
another must prevail. Any improvements by more knowledgeable people
would be welcomed by me.

All the best,
Robert Dodier



Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-07 Thread Robert Goldman
OK, I think I understand now.  This happens when we need quicklisp not 
just to load, but to install and load, defsystem depends on systems.


I just looked at Robert Dodier's bug fix, 
https://github.com/quicklisp/quicklisp-client/pull/128/ and I'm a little 
concerned that it could raise an error if `asdf::missing-requires` isn't 
implemented on the error condition that is signaled by ASDF (and I don't 
see why it would be guaranteed to be implemented there).  Shouldn't this 
check the type of `(asdf::error-condition c)`? Or if there's something 
about the conditions under which this handler is invoked that guarantees 
that the call to `missing-requires` will not error out, I don't see it, 
so it's probably worthy of a comment.


My guess is that if you checked for the `error-condition` being a 
subtype of `missing-component` that would work in modern ASDF.  I have 
no idea whether it would work in ASDF 2 and I'm afraid that I don't have 
the time for the ASDF archaeology required to figure out how to 
"past-proof" this code.


HtH,
Best,
r


On 6 Apr 2018, at 4:20, Mark Evenson wrote:


On Apr 2, 2018, at 18:23, Robert Goldman  wrote:

On 1 Apr 2018, at 7:57, Mark Evenson wrote:

On Apr 1, 2018, at 14:20, Attila Lendvai att...@lendvai.name wrote:

The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will 
be
satisfied by QL:QUICKLOAD no longer seems to be working in 
asdf-3.3.1.


FTR, here's the history of this issue:

https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822

https://github.com/quicklisp/quicklisp-client/issues/108

Wow! Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in

https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the 
preferred way forward from ASDF’s perspective?


"From ASDF's perspective," this is all new to me, since it was filed 
as a bug against Quicklisp, and as far as I know, never raised as an 
issue for ASDF. I could use some help here:


• What's a minimal error case using quickload alone?
	• What's a minimal case that arises with using ASDF as the entry 
point?
It seemed like there was one where if Quicklisp is up and running, 
and you use asdf:load-system to load a system, this can also happen.
Something I can type into a REPL verbatim is what I would like to 
see.


Not sure how to distinguish between your two requests for quickload 
alone versus ASDF as an entry point


A minimal case would be the following ASDF definition

--—depends.asd---

(defsystem depends
  :in-order-to ((test-op (test-op "depends/t"

(defsystem depends/t
  :defsystem-depends-on (prove-asdf)
  :depends-on (prove)
  :components ((:test-file "depends-test.lisp")))

——depends-test.lisp——

(in-package :cl-user)
(prove:plan 1)
(prove:pass "A test that always passes")
(prove:finalize)

--

(ql:quickload :depends) should pick up the depends/t secondary system 
to install PROVE from the network, which is needed to provide a CLOS 
for the TEST-FILE component.


Component "prove-asdf" not found, required by NIL
 0: (CONDITIONS::CONDITIONS-ERROR :INVISIBLEP T 
ASDF/FIND-COMPONENT:MISSING-DEPENDENCY (:REQUIRED-BY NIL :REQUIRES 
"prove-asdf"))
  1: (ERROR ASDF/FIND-COMPONENT:MISSING-DEPENDENCY :REQUIRED-BY NIL 
:REQUIRES "prove-asdf")
  2: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL "prove-asdf" 
NIL)

  3: ((SUBFUNCTION 1 ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION))
…

For ASDF3 alone, as long as PROVE is installed, there is no problem.





Also, sounds like though this is an issue on all lisps, not just ABCL 
as the first post suggested


Yes, this issue effects all Common Lisp implementations.   I don’t 
think I even mentioned ABCL in my first message, so other than being 
an ABCL maintainer, I don’t see how you got that impression.



Communications between ASDF and QL have been difficult since Zach 
dropped off this list (and, to be fair, I have never joined up to 
read quicklisp-devel, if there is such a thing).


Yes, we are certainly dealing with the resistance of Quicklisp to 
deprecate ASDF2 in favor of ASDF3, for which I neither really know nor 
want to go into the history thereof.  Rather than pointing fingers, 
and spreading blame, I am trying to find some compromise that works 
for both the ASDF and Quicklisp maintainers, as without getting 
ql:quickload to somehow include :defsystem-depends-on declarations as 
recognized load dependencies in the currently stable ASDF3, it means 
this useful feature for ASDF extensiblity is effectively unusable for 
inter-system cooperation within Quicklisp.


In the January 2018 Quicklisp systems, there are 103 references to 
prove-asdf, so this issue effects quite a bit of the current Quicklisp 
distributed ecosystem for that use case alone.


As I read the Quicklisp issues and 

Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-06 Thread Mark Evenson


> On Apr 2, 2018, at 18:23, Robert Goldman  wrote:
> 
> On 1 Apr 2018, at 7:57, Mark Evenson wrote:
> 
> On Apr 1, 2018, at 14:20, Attila Lendvai att...@lendvai.name wrote:
> 
> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
> satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.
> 
> FTR, here's the history of this issue:
> 
> https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822
> 
> https://github.com/quicklisp/quicklisp-client/issues/108
> 
> Wow! Holy stale complications, batman!
> 
> Robert apparently suggested something (apparently) much simpler in
> 
> https://github.com/quicklisp/quicklisp-client/pull/128
> 
> but without any commentary from Zach on that approach.
> 
> Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred 
> way forward from ASDF’s perspective?
> 
> "From ASDF's perspective," this is all new to me, since it was filed as a bug 
> against Quicklisp, and as far as I know, never raised as an issue for ASDF. I 
> could use some help here:
> 
>   • What's a minimal error case using quickload alone?
>   • What's a minimal case that arises with using ASDF as the entry point?
> It seemed like there was one where if Quicklisp is up and running, and you 
> use asdf:load-system to load a system, this can also happen.
> Something I can type into a REPL verbatim is what I would like to see.

Not sure how to distinguish between your two requests for quickload alone 
versus ASDF as an entry point

A minimal case would be the following ASDF definition

--—depends.asd---

(defsystem depends
  :in-order-to ((test-op (test-op "depends/t"

(defsystem depends/t
  :defsystem-depends-on (prove-asdf)
  :depends-on (prove)
  :components ((:test-file "depends-test.lisp")))

——depends-test.lisp——

(in-package :cl-user)
(prove:plan 1)
(prove:pass "A test that always passes")
(prove:finalize)

--

(ql:quickload :depends) should pick up the depends/t secondary system to 
install PROVE from the network, which is needed to provide a CLOS for the 
TEST-FILE component.

Component "prove-asdf" not found, required by NIL
 0: (CONDITIONS::CONDITIONS-ERROR :INVISIBLEP T 
ASDF/FIND-COMPONENT:MISSING-DEPENDENCY (:REQUIRED-BY NIL :REQUIRES 
"prove-asdf"))
  1: (ERROR ASDF/FIND-COMPONENT:MISSING-DEPENDENCY :REQUIRED-BY NIL :REQUIRES 
"prove-asdf")
  2: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL "prove-asdf" NIL)
  3: ((SUBFUNCTION 1 ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION))
…

For ASDF3 alone, as long as PROVE is installed, there is no problem.


> Also, sounds like though this is an issue on all lisps, not just ABCL as the 
> first post suggested

Yes, this issue effects all Common Lisp implementations.   I don’t think I even 
mentioned ABCL in my first message, so other than being an ABCL maintainer, I 
don’t see how you got that impression.


> Communications between ASDF and QL have been difficult since Zach dropped off 
> this list (and, to be fair, I have never joined up to read quicklisp-devel, 
> if there is such a thing).

Yes, we are certainly dealing with the resistance of Quicklisp to deprecate 
ASDF2 in favor of ASDF3, for which I neither really know nor want to go into 
the history thereof.  Rather than pointing fingers, and spreading blame, I am 
trying to find some compromise that works for both the ASDF and Quicklisp 
maintainers, as without getting ql:quickload to somehow include 
:defsystem-depends-on declarations as recognized load dependencies in the 
currently stable ASDF3, it means this useful feature for ASDF extensiblity is 
effectively unusable for inter-system cooperation within Quicklisp.  

In the January 2018 Quicklisp systems, there are 103 references to prove-asdf, 
so this issue effects quite a bit of the current Quicklisp distributed 
ecosystem for that use case alone.

As I read the Quicklisp issues and pull-requests, Quicklisp would be willing to 
accept a “minimally invasive” patch if it would support asdf-2.26 as well as 
ASDF3. 


So, to put things more succintly, given the choice between Quicklisp pulls 
[122][] or [128][], and given that we have advanced to asdf-3.3.1 since these 
requests were issued, what would be the preferred manner to patch Quicklisp 
that would be the most forward-looking for future ASDF3 compatibility so that 
Quicklisp might continue to work with :DEFSYSTEM-DEPENDS-ON clauses like it 
used to?

[122]: https://github.com/quicklisp/quicklisp-client/pull/122
[128]: https://github.com/quicklisp/quicklisp-client/pull/128






Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-02 Thread Robert Goldman

On 1 Apr 2018, at 7:57, Mark Evenson wrote:


On Apr 1, 2018, at 14:20, Attila Lendvai  wrote:

The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will 
be
satisfied by QL:QUICKLOAD no longer seems to be working in 
asdf-3.3.1.


FTR, here's the history of this issue:

https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822

https://github.com/quicklisp/quicklisp-client/issues/108



Wow!  Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in

  https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the 
preferred way forward from ASDF’s perspective?


"From ASDF's perspective," this is all new to me, since it was filed as 
a bug against Quicklisp, and as far as I know, never raised as an issue 
for ASDF.  I could use some help here:


1. What's a minimal error case using `quickload` alone?
2. What's a minimal case that arises with using ASDF as the entry point? 
It seemed like there was one where if Quicklisp is up and running, and 
you use `asdf:load-system` to load a system, this can also happen.


Something I can type into a REPL verbatim is what I would like to see.

Also, sounds like though this is an issue on all lisps, not just ABCL as 
the first post suggested


Communications between ASDF and QL have been difficult since Zach 
dropped off this list (and, to be fair, I have never joined up to read 
quicklisp-devel, if there is such a thing).


Best,
R

Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-01 Thread Mark Evenson


> On Apr 1, 2018, at 14:20, Attila Lendvai  wrote:
> 
>> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
>> satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.
> 
> FTR, here's the history of this issue:
> 
> https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822
> 
> https://github.com/quicklisp/quicklisp-client/issues/108


Wow!  Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in 

  https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred 
way forward from ASDF’s perspective?





Re: Problems with DEFSYSTEM-DEPENDS-ON

2018-04-01 Thread Attila Lendvai
> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
> satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.

FTR, here's the history of this issue:

https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822

https://github.com/quicklisp/quicklisp-client/issues/108

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“When a country is well governed, poverty and a mean condition are
things to be ashamed of. When a country is ill governed, riches and
honor are things to be ashamed of.”
— Confucius (551–479 BC), 'The Analects', Chapter VIII 
(邦有道貧且賤焉恥也,邦無道富且貴焉恥也。)