Question about version in submitting patches

[Mark Evenson]

I’m a little unsure of whether the “Committee for Ongoing and Perpetual ASDF 
maintenance” (hi Robert!) wishes us to include the results of 
“” in submitted patches.

I have a small ABCL-specific patch dealing with UIOP:PARSE-UNIX-NAMESTRING when 
loading system definitions from zip archives for which I have used bump-version 
to denote as version “3.3.4.0.1”.  I’ve not quite finished my testing to ensure 
that previous versions of ABCL work well with it, but when I do, do you wish me 
to include the use of “bump-version” with the patch or is that something the 
Committee prefers to do on its own?

yours,
Mark


-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Jenkins expertise?

[Mark Evenson]

> On Oct 16, 2020, at 20:13, Eric Timmons  wrote:
> 
> "Robert Goldman"  writes:
> 
>> In mine, I simply set the environment variables `ASDF_TEST_LISPS` and 
>> the variables that point to the lisp implementations.  Then I call `make 
>> test-all-no-upgrades-no-stop`.
> 
> Just to clarify, it sounds from this that you don't actually care if the
> upgrade tests are run? Do you ever want to run them via CI? As it
> currently stands they're the slowest jobs, so finding some way to reduce
> the number of times they're run would be a win.


Some tests are better than no tests.

The CLF advocates a merge of [!146][] as we then we can adjust runners as
needed for capacity and capability.  The default runners are Linux.  Eric
Timmons is setting up a macOS runner which can be used.  Windows runners are
further in the future.

Among other positive attributes, this establishes a [non-dotfile
artifact][file:ci-gitlab.yml] which runs the CI sequence, so is ammendable to
further pull requests.

For commercial implementations, we are still working out the details that would
be conformant with the licensing conditions.

[!144]: 

[file:ci-gitlab.yml]: 



-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

ASDF CI (was Re: Jenkins expertise?)

[Mark Evenson]

> On Oct 16, 2020, at 06:21, Eric Timmons  wrote:

[…]

> I'm a huge fan of Gitlab CI and was getting tired of running all tests
> locally whenever I made a change, so I started a Gitlab CI configuration
> for ASDF at
> https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/146 and also
> modified it a bit more today. The recent modifications are marking the
> upgrade tests as `allow_failure: true` (since in a previous email it
> sounded like the previous Jenkins config didn't even run them),
> increasing some timeouts in tests (I don't know what happened, but some
> tests started failing under CCL when run on cl.net runners), reordering
> things such that all regression tests ran first, and addressing a bug in
> the latest SBCL that was triggered by the cl.net infrastructure.
> 
> If you want to take some radically different approach than I did, that's
> great! I just wanted to make sure everyone knew that an 80% solution
> exists and didn't waste their time coming up with something
> substantially similar. I think the biggest things left on !146 are
> deciding how often to run the tests (every push? only on MRs?) and
> adding more runners for licensed implementations or OSes.

Wow!  This looks like much more than 80%, and its working today!

> 
> Speaking of OSes, my research group has a Mac mini we've been using for
> our own CI purposes. By no means is it a speed demon, but it's lightly
> used so I can set up another runner on it for ASDF.
> 
> As for the licensed implementations: I'm not sure of what the cl.net
> infrastructure looks like and how paranoid you want to be about
> protecting the license, but it seems the easiest way forward would be to
> add a new VM based runner with the implementation and license already
> installed in the VM, register that runner so that only the asdf/asdf
> project can use it, and tag it with the implementation name. We can then
> add a job that is triggered only if it's run in the asdf/asdf project
> and have it require that tag.
> 
> As I said in another message, this would mean that forks can't use that
> runner (and MR tests are, unfortunately, run in the CI context of the
> fork). If it's OK for master to occasionally break, the changes from
> forks could be merged directly to master and breakage dealt with after
> that. If you want master to never break because of licensed
> implementations, then someone with write bits on asdf/asdf would need to
> pull the branch from the fork into asdf/asdf first.
> 
> Either way, the person pulling into asdf/asdf would have to do a sanity
> check to make sure the license isn't being exfiltrated, but I don't
> think that's any change from the previous approach (unless you were
> previously running the tests without external network access!).

Thanks for the suggested way forward on Allegro/LispWorks.  We (i.e. CLF)
certainly want to be fairly thorough on protecting the licenses for the
commercial implementations.  If we do a good enough job with ASDF, we would
like to try to provide them as a potential service for any
gitlab.common-lisp.net project.

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Jenkins expertise?

[Mark Evenson]

> On Oct 14, 2020, at 19:34, Robert Goldman  wrote:
> 
> That would be great.  I have a Jenkins set up on a multi-core box at SIFT 
> that for now runs only Allegro, SBCL and CCL.  It's hard for me to make it 
> accessible to anyone else, though, because Jenkins is such a security 
> nightmare: we keep access only to our VPN.

Are all the artifacts necessary to run the Jenkins pipeline in the ASDF
repository?  I haven’t used Jenkins in something like a decade, so I have long
forgotten anything I may have remembered, but will start figuring things out
tomorrow.

Running on Allegro, SBCL, CCL, ECL, and ABCL should be quite doable.  I have
yet to hit LispWorks up for a license, but will make the appropiate polite
request when I know more about how to keep the licensing secure for Allegro in
the Jenkins instance.

We should be able to provide somewhat of a progress report tomorrow around this
time as to how far we actually got to running the ASDF CI under Jenkins.

best,
Mark

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Jenkins expertise?

[Mark Evenson]

> On Oct 14, 2020, at 18:58, Jason Miller  wrote:
> 
> I have very little experience with gitlab, but I do have have experience with 
> configuring Jenkins, and the gitlab-plugin looks reasonably straightforward 
> as these things go.  If nobody else chimes-in I can probably get it setup.


I think in order to get ASDF CI back to running ASAP, we should host a special
Jenkins instance on common-lisp.net.  Once ASDF is back to a more stable
development cycle with CI like it was working, we can refactor the knowledge
for commerical instances into the more generic Gitlab runner.  Also, we would
be free to hand the ASDF Jenkins instance as much oompf as it needs to return
results in an acceptable wall-clock time.

As such, I would like to take Jason up on his kind offer.  

@jason: could you do a Jenkins instance as a Docker container?  This would be
the fastest route to getting something working in the CLF infrastructure.

It would be great to coordinate things on the Freenode #common-lisp.net
channel.  For synchronous response from me, I will be working with Dave Cooper
there tomorrow, Thursday, October 15 at 0800 UTC; otherwise just leave a
message with @Colleen, and we’ll get back to ya.

yers in CONS,
Mark

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Apologies for slow progress...

[Mark Evenson]

> On Aug 1, 2020, at 22:00, Robert Goldman  wrote:
> 
> I'm afraid I have been very busy at work, but also the linux box I have been 
> using as my Jenkins platform had a disk failure, taking it out of action.

[…]

> If anyone has hosting suggestions, please let me know: otherwise I'm afraid 
> we'll be limping along till September, which is unfortunate, since we have a 
> bunch of fixes and bug reports, and I would very much like to be able to get 
> these wrapped up into a bug fix release.
> 
> I hope you all are staying safe and healthy!


We would potentially be interested in hosting the ASDF CI on common-lisp.net 
with gitlab
CI.  We would have to work out the kinks with using commercial CL
implementations including reaching out to Allegro and LispWorks for proper
licensing.  Providing CI containers for tuned for ANSI CL implementations has
long been on our desired features, so this would line up well with where we
want to push the services offered by common-lisp.net.

It would take an unknown amount of work to provide this service to ASDF 
provided by an
unspecified number of volunteers, but given the importance of ASDF to everyone,
I believe we could attract a decent amount of help.  If there is a decent
alternative with Travis or some other service, and just getting things running
again rather than being a guinea pig for new services is your priority, then
please pursue that path.

We have a CLF board meeting this Wednesday at 1900 UTC where I could advance 
such
an idea, and we could coordinate on reaching out to the commercial vendors for
(hopefully free) licensing term details.

CLF board meeting attendence is more or less open to those who wish to attend
via GOOG meet.  Either drop me a mail, or stop by #common-lisp.net to get the
connection details.

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: [PATCH] Add a TEST-OP-TEST-FAILURE condition for test libraries to sub-class

[Mark Evenson]

> On Sep 25, 2019, at 21:06, Vladimir Sedach  wrote:
> 
> 
> Mark Evenson  writes:
> 
>> Each flavor of testing framework needs to write a simple adaptor
>> that returns a condition containing a boolean indicating success or
>> failure and the testing framework specific results.

> Unfortunately, it has the same downsides. There is only a subset of
> the test libraries supported. The code depends on the internal
> workings of the test libraries and is often out of date. The projects
> cannot be used as-is to test arbitrary systems (with cl-test-grid,
> you have to manually add configuration for each system to
> testsuites/testsuites.lisp, asdf-test-harness seems to depend on
> emotiq configuration files).
> 
> This approach has not worked well in the past 8 years (cl-test-grid
> was started in 2011), and I do not think it is viable.
> 
> What you have also done in asdf-test-harness, adding an
> ASDF-TEST-SYSTEM-FAILURE condition to ASDF, is a better approach. I
> have shown how this can be used by test libraries (with working
> implementations for FiveAM³ and rove⁴). This removes the need for
> asdf-test-harness, cl-test-grid, etc. to each have to implement
> custom adapters for every different test library, completely
> eliminating the dependency between test harnesses and particular test
> libraries. The test libraries themselves do not need any new
> dependencies either.
> 
> This decouples the systems and will make projects like
> asdf-test-harness much easier to extend and maintain.

Thanks for taking the time to understand my code enough to cogently critique
the two approaches.

As I understand it, both approaches share the need to add a condition to ASDF
that indicates that an invocation of an ASDF test-op has failed.

The difference is that you propose getting changes to all the various test
framework to signal this error condition with a subclass that contains
meaningful information for that given framework.  Whereas, I (and
cl-test-grid)--who was in a hurry to get something working without being able
to wait for modifications to the test suites--needs to write an adaptor for
each test suite.  And consequently these adaptors will be often be “brittle”
depending on implementation details that may not be constant over time to work.

If this is indeed an accurate summary, then I would endorse your patch to ASDF
as necessary for both approaches with the following suggestions for
improvements:

1.  Have a slot in your base condition class TEST-OP-TEST-FAILURE in which  one
can record the ASDF component which caused the failure.  It is probably
possible to dig this information out of the stack, but that will be messy. This
would also allow for distinguishing when multiple TEST-OP-TEST-FAILURES are
signaled from a single ASDF:TEST-OP invocation, as will be the case when one
“chains” test invocation over many ASDF systems.

2.  Provide an implementation of the subclass of TEST-OP-TEST-FAILURE that
contains the basic structure of a reporter class for the  information that
should be present in all test frameworks, namely the total number of tests run,
the number of failed tests, the identities of the failed tests, and a slot for
a human readable error message, along with a reporter function that displays
this information.  Having an implementation class to work from would make it
easier for test frameworks to adapt.

3.  Go ahead and define the subclass of this condition when no tests have been
run.

4.  As for adoption by test framework your strategy, we will have the problem
that a given test framework won’t want to adopt the conditions because it isn’t
in the version of ASDF they are using, or can easily get a hold of.  To solve
this, we might somehow define the code within the ASDF source tree so that one
can make a standalone ASDF system (“ASDF-TEST-CONDITIONS” or some such) that
one may include separately from actually upgrading ASDF.

Sincerely, 
Mark Evenson

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: [PATCH] Add a TEST-OP-TEST-FAILURE condition for test libraries to sub-class

[Mark Evenson]

> On Sep 15, 2019, at 05:07, Vladimir Sedach  wrote:
> 
> Hello!
> 
> I recently wrote a script to run FiveAM tests for one of my
> libraries on many different implementations on your own machine:
> https://gitlab.common-lisp.net/uri-template/uri-template2/blob/master/run-tests.lisp
> 
> It would be really nice if I did not have to copy-paste that script
> to my other libraries, and instead could contribute a generalized
> version to Roswell (on which the script is based) and have it work
> for any project using any test library.
> 
> What I would like to be able to do, for any system:
> 
> (handler-case (asdf:test-system "system")
>  (asdf:test-op-test-failure (condition)
>(princ condition uiop:*stderr*)
>(uiop:quit 1)))

[Last year for Emotiq][emotiq] I implemented such a proposal as 
[asdf-test-harness][] to the maturity needed for us to use as part of 
Continuout Integration for all our commits.

Each flavor of testing framework needs to write a simple adaptor that returns a 
condition containing a boolean indicating success or failure and the testing 
framework specific results.  

It’s been a while so please press me on the claim that my code would be more 
mature, as it has been enough time that I don’t remember writing the code at 
the moment…

[emotiq]: https://github.com/easye/emotiq/
[asdf-test-harness]:  https://github.com/easye/asdf-test-harness/

Re: ASDF 3.3.3 is Released

[Mark Evenson]

> On Mar 27, 2019, at 17:49, Robert Goldman  wrote:
> 
> In honor of the impending 2019 European Lisp Symposium, today we release ASDF 
> 3.3.3, the third bugfix release for the 3.3 release series.
> 
> We urge implementations that are currently bundling previous versions of ASDF 
> -- and especially those bundling 3.3.0 or 3.3.1 -- to upgrade to 3.3.3 at 
> their earliest convenience.

[…]

There doesn’t seem to be a tag or branch marker for 3.3.3 (or 3.3.3.0) in the 
[gitlab.common-listp.net repository][1], or am I missing something?

Congratulations on the release!

[1]: https://gitlab.common-lisp.net/asdf/asdf.git]




-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Introspecting test suite passing/failing

[Mark Evenson]

> On Jun 6, 2018, at 22:41, Mark Evenson  wrote:
> 
> 
> 
>> On Jun 6, 2018, at 21:08, Robert Goldman  wrote:
>> 
>> Will you please send me the example (with ancillary files), so that I can 
>> see exactly what's going wrong?
> 


 wget 
https://raw.githubusercontent.com/emotiq/asdf-test-harness/master/eg/lisp-unit-example.lisp



-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."






lisp-unit-example.lisp
Description: Binary data

Re: Introspecting test suite passing/failing

[Mark Evenson]

> On Jun 6, 2018, at 21:08, Robert Goldman  wrote:
> 
> Will you please send me the example (with ancillary files), so that I can see 
> exactly what's going wrong?

[…]

Attaching 

wget 
https://raw.githubusercontent.com/emotiq/asdf-test-harness/master/eg/fiveam-asdf-example.asd

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."






fiveam-asdf-example.asd
Description: Binary data

Re: Introspecting test suite passing/failing

[Mark Evenson]

> On Jun 6, 2018, at 19:14, Robert Goldman  wrote:

[…]

> If you get a chance, can you eyeball [my example code to try to use 
> FIVEAM-ASDF][2] to tell me if that looks like the correct usage?
> 
> [2]: 
> https://github.com/emotiq/asdf-test-harness/blob/master/eg/fiveam-asdf-example.asd
> 
> Various versions of ASDF-3.1.x (SBCL, CCL, ABCL) are failing with problems 
> about symbols:
> 
> Error while trying to load definition for system
> fiveam-asdf-example from pathname
> /Users/evenson/work/asdf-test-harness/eg/fiveam-asdf-example.asd:
> 
> EXPORT ASDF/INTERFACE::FIVEAM-TESTER-SYSTEM causes
> name-conflicts in # between the
> following symbols:
> ASDF/INTERFACE::FIVEAM-TESTER-SYSTEM,
> ASDF/USER::FIVEAM-TESTER-SYSTEM
> [Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]
> 
> I had problems like this, too -- it's because of the fact that when the 
> defsystem form is first read, before the :defsystem-depends-on is loaded, the 
> symbol named "FIVEAM-TESTER-SYSTEM" gets interned in ASDF/USER and then when 
> it's later exported from ASDF/INTERFACE (used by ASDF/USER) it collides with 
> the earlier-read symbol.
> 
> I believe that the correct fix for this is to use any new symbols (like 
> fiveam-tester-system) in the keyword package, so
> 
> :class :fiveam-tester-system
> 
> and then when the defsystem form is processed after the defsystem 
> dependencies (in this case, fiveam-asdf), ASDF will look for keyword symbols 
> in the current package.
> 
> Give that a try and see if it fixes your problem.

Unfortunately, it doesn’t help to specify a keyword, but one gets a new error:

Error while trying to load definition for system
asdf-test-harness-example from pathname
/Users/evenson/work/asdf-test-harness/eg/asdf-test-harness-example.asd:

   The following symbols need to be imported to # 
before they can be exported 
 from that package:
(:HARNESS-TEST):

I think with some futzing around with EVAL-WHEN around EXPORT statements we can 
get this to work.

I’m coming to the end of my day with a Common Lisp Foundation board meeting, so 
I will probably get back to this tomorrow.  

best,
Mark

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Introspecting test suite passing/failing

[Mark Evenson]

> On Jun 5, 2018, at 18:44, Robert Goldman  wrote:
> 
> I have pushed the system fiveam-asdf, which supports integration between ASDF 
> and the FIVEAM test library, to the contribs directory in the ASDF repo. 
> Please have a look.
> 
> Be warned! It is old, and not being broke, hasn't been fixed. It 
> inappropriately is housed in the ASDF package and inappropriately exports 
> extensions from that package.
> 
> But I believe it's still useful as an example of how to raise conditions when 
> the test operation goes wrong. Catching those exceptions can be used to cause 
> a build to fail in a CI system, typically by running lisp in batch mode and 
> having it exit with a nonzero error code if the test operation fails.
> 

Robert, 

Thanks ever so much for [releasing the fiveam-asdf code][1]. 

[1]: https://gitlab.common-lisp.net/asdf/asdf/tree/master/contrib/fiveam-asdf

I seemingly misunderstand how to use the :CLASS argument to ASDF:DEFSYSTEM, as 
I cannot quite get your code to work.  

If you get a chance, can you eyeball [my example code to try to use 
FIVEAM-ASDF][2] to tell me if that looks like the correct usage?

[2]: 
https://github.com/emotiq/asdf-test-harness/blob/master/eg/fiveam-asdf-example.asd

Various versions of ASDF-3.1.x (SBCL, CCL, ABCL) are failing with problems 
about symbols:

Error while trying to load definition for system
fiveam-asdf-example from pathname
/Users/evenson/work/asdf-test-harness/eg/fiveam-asdf-example.asd:

   EXPORT ASDF/INTERFACE::FIVEAM-TESTER-SYSTEM causes
   name-conflicts in # between the
   following symbols:
 ASDF/INTERFACE::FIVEAM-TESTER-SYSTEM,
 ASDF/USER::FIVEAM-TESTER-SYSTEM
   [Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]

Introspecting test suite passing/failing

[Mark Evenson]

We use ASDF to encapsulate the building and testing many systems under an
automated test runner.

For a given system, ASDF:TEST-SYSTEM always returns boolean truth as long as
the invocation of the underlying test suite succeeds.

This means there is no programatic way to provide a boolean as to whether all
the tests passed or not on the invocation of a test suite to the invoker of
ASDF:TEST-SYSTEM.

It seems that this issue has been raised before, as the ASDF manual documents
TEST-OP 

 with:

> The results of this operation are not defined by ASDF. It has proven difficult
> to define how the test operation should signal its results to the user in a 
> way
> that is compatible with all of the various test libraries and test techniques
> in use in the community, and given the fact that ASDF operations do not return
> a value indicating success or failure. For those willing to go to the effort,
> we suggest defining conditions to signal when a test-op fails, and storing in
> those conditions information that describes which tests fail.

Is this still the best current practice to introspect the situation of failing
tests?  Can someone point me to an example implementation of this technique?


-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Problems with DEFSYSTEM-DEPENDS-ON

[Mark Evenson]

> On Apr 2, 2018, at 18:23, Robert Goldman <rpgold...@sift.info> wrote:
> 
> On 1 Apr 2018, at 7:57, Mark Evenson wrote:
> 
> On Apr 1, 2018, at 14:20, Attila Lendvai att...@lendvai.name wrote:
> 
> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
> satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.
> 
> FTR, here's the history of this issue:
> 
> https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822
> 
> https://github.com/quicklisp/quicklisp-client/issues/108
> 
> Wow! Holy stale complications, batman!
> 
> Robert apparently suggested something (apparently) much simpler in
> 
> https://github.com/quicklisp/quicklisp-client/pull/128
> 
> but without any commentary from Zach on that approach.
> 
> Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred 
> way forward from ASDF’s perspective?
> 
> "From ASDF's perspective," this is all new to me, since it was filed as a bug 
> against Quicklisp, and as far as I know, never raised as an issue for ASDF. I 
> could use some help here:
> 
>   • What's a minimal error case using quickload alone?
>   • What's a minimal case that arises with using ASDF as the entry point?
> It seemed like there was one where if Quicklisp is up and running, and you 
> use asdf:load-system to load a system, this can also happen.
> Something I can type into a REPL verbatim is what I would like to see.

Not sure how to distinguish between your two requests for quickload alone 
versus ASDF as an entry point

A minimal case would be the following ASDF definition

--—depends.asd---

(defsystem depends
  :in-order-to ((test-op (test-op "depends/t"

(defsystem depends/t
  :defsystem-depends-on (prove-asdf)
  :depends-on (prove)
  :components ((:test-file "depends-test.lisp")))

——depends-test.lisp——

(in-package :cl-user)
(prove:plan 1)
(prove:pass "A test that always passes")
(prove:finalize)

--

(ql:quickload :depends) should pick up the depends/t secondary system to 
install PROVE from the network, which is needed to provide a CLOS for the 
TEST-FILE component.

Component "prove-asdf" not found, required by NIL
 0: (CONDITIONS::CONDITIONS-ERROR :INVISIBLEP T 
ASDF/FIND-COMPONENT:MISSING-DEPENDENCY (:REQUIRED-BY NIL :REQUIRES 
"prove-asdf"))
  1: (ERROR ASDF/FIND-COMPONENT:MISSING-DEPENDENCY :REQUIRED-BY NIL :REQUIRES 
"prove-asdf")
  2: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL "prove-asdf" NIL)
  3: ((SUBFUNCTION 1 ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION))
…

For ASDF3 alone, as long as PROVE is installed, there is no problem.


> Also, sounds like though this is an issue on all lisps, not just ABCL as the 
> first post suggested

Yes, this issue effects all Common Lisp implementations.   I don’t think I even 
mentioned ABCL in my first message, so other than being an ABCL maintainer, I 
don’t see how you got that impression.


> Communications between ASDF and QL have been difficult since Zach dropped off 
> this list (and, to be fair, I have never joined up to read quicklisp-devel, 
> if there is such a thing).

Yes, we are certainly dealing with the resistance of Quicklisp to deprecate 
ASDF2 in favor of ASDF3, for which I neither really know nor want to go into 
the history thereof.  Rather than pointing fingers, and spreading blame, I am 
trying to find some compromise that works for both the ASDF and Quicklisp 
maintainers, as without getting ql:quickload to somehow include 
:defsystem-depends-on declarations as recognized load dependencies in the 
currently stable ASDF3, it means this useful feature for ASDF extensiblity is 
effectively unusable for inter-system cooperation within Quicklisp.  

In the January 2018 Quicklisp systems, there are 103 references to prove-asdf, 
so this issue effects quite a bit of the current Quicklisp distributed 
ecosystem for that use case alone.

As I read the Quicklisp issues and pull-requests, Quicklisp would be willing to 
accept a “minimally invasive” patch if it would support asdf-2.26 as well as 
ASDF3. 


So, to put things more succintly, given the choice between Quicklisp pulls 
[122][] or [128][], and given that we have advanced to asdf-3.3.1 since these 
requests were issued, what would be the preferred manner to patch Quicklisp 
that would be the most forward-looking for future ASDF3 compatibility so that 
Quicklisp might continue to work with :DEFSYSTEM-DEPENDS-ON clauses like it 
used to?

[122]: https://github.com/quicklisp/quicklisp-client/pull/122
[128]: https://github.com/quicklisp/quicklisp-client/pull/128

Re: Problems with DEFSYSTEM-DEPENDS-ON

[Mark Evenson]

> On Apr 1, 2018, at 14:20, Attila Lendvai  wrote:
> 
>> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
>> satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.
> 
> FTR, here's the history of this issue:
> 
> https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822
> 
> https://github.com/quicklisp/quicklisp-client/issues/108


Wow!  Holy stale complications, batman!

Robert apparently suggested something (apparently) much simpler in 

  https://github.com/quicklisp/quicklisp-client/pull/128

but without any commentary from Zach on that approach.

Given asdf-3.3 is out, and recent sbcl’s ship with it, which is the preferred 
way forward from ASDF’s perspective?

Problems with DEFSYSTEM-DEPENDS-ON

[Mark Evenson]

The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be
satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1.

It used to be the case that to use the [PROVE testing framework][1], it was
sufficient to place a

   …
   :defsystem-depends-on (:prove-asdf)
   …

clause in the secondary system to be tested, and then upon a QL:QUICKLOAD of 
this system,
the dependency on PROVE was then resolved via a network download from Quicklisp.


[1]: https://github.com/fukamachi/prove

I believe this was working with asdf-3.2, but testing that assumption is a 
little hard as
asdf-3.3 refuses to degrade itself with a downgrade, and all my systems are 
running asdf-3.3.

Maybe something in Quicklisp changed as well?

LOGICAL-PATHNAMES output locations

[Mark Evenson]

In spite of Faré's resistance to supporting LOGICAL-PATHNAMEs in ASDF, I
would ask for help in using the current capabilities documented for
ASDF-3.2.

To wit, I would like to implement my own OUTPUT-TRANSLATIONS for
LOGICAL-PATHNAMES [as is allowed (but discouraged)][1].

[1]:
https://www.common-lisp.net/project/asdf/asdf.html#Using-logical-pathnames

I tried to define a (massively inefficient) :BEFORE COMPILE-OP which
replaces all LOGICAL-PATHNAME instances with a call to CL:TRUENAME but
that doesn't seem to work:

```
(defsystem abcl/build
  :version "2.0.0"
  :description "Build ABCL from a Lisp.  Downloads necessary build-time
tools to local cache."
  :in-order-to ((test-op (test-op abcl/build/t)))
  :perform (compile-op :before (op c)
   (when (typep c 'source-file)
 (with-slots (absolute-pathname)
 c
   (when (typep absolute-pathname 'logical-pathname)
 (setf absolute-pathname
   (truename absolute-pathname))
  :components ((:module package
:pathname #p"SYS:build;"
:components ((:file "package")))
```

What is the right well to specify my own OUTPUT-TRANSLATIONS mechanism?


The disfavor of LOGICAL-PATHANME references in DEFSYSTEM form is fully
understandable from the observation that there is no conforming way to
ensure that the "right" translations are in place.  But, I have come up
with a use case in abcl where LOGICAL-PATHNAMEs would be very useful,
and since it is "my" implementation, I can guarantee that appropiate
translations are in place at runtime.  My use case involves locating
source code referenced by ASDF in different locations depending on
whether the source is being actively developed, and is on a writable
filesystem, or whether the source is being loaded from a read-only JVM
source, i.e. a "jar" archive.  Using LOGICAL-PATHNAMEs within `abcl.asd`
allows me to configure the runtime behavior quite nicely via suitable
DSL parameter to the ASDF:INITIALIALIZE-SOURCE-REGISTRY.  Running the
[currently-in-asdf jar translation code on the SOURCE-FILE][2] is all I
need to do here to get a valid output location for COMPILE-OP.

[2]:
https://gitlab.common-lisp.net/asdf/asdf/blob/master/output-translations.lisp#L166

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Running asdf-3.3 aka 'plan' with ABCL

[Mark Evenson]

After a bit of analysis, I have [my first issue to report][1] with
running asdf-3.3 (aka [the plan branch][2]) on [ABCL][3].

The problem stems from the new--and quite sensible--restriction that any
[code in a ASDF:PERFORM cannot call into another ASDF:PERFORM operation][4].

QUICKLISP-ABCL's ability to encapsulate the load of Quicklisp in an ASDF
definition has been quite useful to me, and I'd hate to abandon it.  Of
course I could stuff the code ensuring Quicklisp's local installation in
an the ABCL specific package, but maybe there is another way to
encapsulate this situation in ASDF?  Otherwise, it would seem that ASDF
loses a little functionality here with the current asdf-3.3, which
possibly might be the tradeoff for a more deterministic plan over all
operations.  But it would be nice to "allow allowing" here…

[1]: https://gitlab.common-lisp.net/mevenson/asdf/issues/1
[2]: https://gitlab.common-lisp.net/asdf/asdf/tree/plan
[3]:
https://gitlab.common-lisp.net/mevenson/asdf/uploads/3039ebfe40dc111a3ab6d061186a16fe/asdf-3.3.diff
[4]:
https://gitlab.common-lisp.net/asdf/asdf/blob/plan/doc/best_practices.md#more-elaborate-testing


-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Impact on modifying the current planning code (was Re: Plans for ASDF 3.3)

[Mark Evenson]

On 5/4/17 18:41, Stelian Ionescu wrote:
[…]

> ASDF currently (for Lisp) doesn't support
>> a maximum version number, only a minimum.  I keep meaning to fix that,
>> but never get around to it.
> 
> I believe this would be doubling down on the error of specifying versions in 
> ASDF.

But as long as we have a notion of ASDF:VERSION it should certainly have
a possible upper bound capability for its assertions, whatever your
local mapping to such assertions may be…


-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Impact on modifying the current planning code (was Re: Plans for ASDF 3.3)

[Mark Evenson]

On 5/4/17 17:08, Faré wrote:
> Dear Mark,
> 
> thanks for your request. I'm not sure I understand how your ASDF
> dependencies do or don't map to MVN entities, though. I also don't
> understand the maven model very well.

[…]
> 
> In case (a), it probably should be a special subclass of SYSTEM. In
> case (b) a component with a single file pathname should be fine. In
> case (c) I would probably also have some subclass of SYSTEM do it, and
> use one or many secondary systems.

Exactly what I needed to know. I'll rebase my inheritance off of
ASDF:SYSTEM instead of ASDF:COMPONENT, working off your 'plan' branch.
More questions when I have an implementation to kick around…

[…]

> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> Anyone who says he can see through women is missing a lot. — Groucho Marx
> 


-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Impact on modifying the current planning code (was Re: Plans for ASDF 3.3)

[Mark Evenson]

On 5/1/17 03:45, Faré wrote:
> I took the content of this thread and edited it into a blog post:
> http://fare.livejournal.com/188940.html
>
> Robert, can you tell me what the plan should be for ASDF 3.3, the
> "plan" (phase separation) branch and the "syntax-control" branch?

Thanks for the detailed write-up, Faré, as such exposition from your
ASDF wrangling helps me greatly in understanding why things are the
way they are in ASDF.  Bonus points for the snark in "Common Lisp is a
hippie language that disallows disallowing", at which I am still
chuckling.

In the JVM ecosystem, the standard way to identify packaged binary
artifacts ("jar files") and their dependencies is via the Maven
["coordinate system" in the POM model][1] which associates every
artifact with a group-id:artifact-id:version triple.  An example of
such a triple would be ["org.abcl:abcl:1.4.0" ][2].  The use of the
POM model has been adopted by almost (?) all other current "Java the
platform" build systems, including Clojure's Leiningen and ABCL's
extensions to ASDF, [abcl-asdf][3].

[1]: https://maven.apache.org/pom.html
[2]:
https://search.maven.org/#artifactdetails%7Corg.abcl%7Cabcl%7C1.4.0%7Cjar
[3]:
https://gitlab.common-lisp.net/abcl/abcl/blob/master/contrib/abcl-asdf/abcl-asdf.lisp

Unfortunately, the current implementation of ABCL-ASDF took a naive
shortcut that needs fixing, as it works on a per-dependency basis as
opposed to properly sorting and reducing declared dependencies to a
minimal working set for a given ASDF system definition.  Alan
Ruttenberg has [proposed a short term fix][5] that we are currently
experimenting with, but the proper way forward as I understand it,
would be to push more of this dependency logic into ASDF where the
"plan" phase would be able to properly reason about dependencies.

[4]:
https://mailman.common-lisp.net/pipermail/armedbear-devel/2017-April/003810.html
[5]:
https://mailman.common-lisp.net/pipermail/armedbear-devel/2017-April/003838.html

An example to illuminate the problem as I understand it:

Currently, when ABCL encounters the following definition

(asdf:defsystem :log4j
  :defsystem-depends-on (abcl-asdf)
  :components ((:module log4j.jar
:components ((:mvn "log4j/log4j/1.2.17")

a request is made to the Maven code to satisfy the dependency for
"log4j/log4j/1.2.17" and add all transitive dependencies to the JVM's
classpath.  This graph is discovered at request time by parsing all
the POM models from the network.  Issuing this request at an ABCL repl
just returned the following for me:


"/Users/evenson/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/evenson/.m2/repository/javax/mail/mail/1.4.3/mail-1.4.3.jar:/Users/evenson/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/evenson/.m2/repository/org/apache/geronimo/specs/geronimo-jms_1.1_spec/1.0/geronimo-jms_1.1_spec-1.0.jar"

which means a request for a logging framework dependency
(`log4j-1.2.17.jar`) has also pulled in an SMTP implementation
(`mail-1.4.3.jar`) amongst other things (a common problem with the
JVM, apparently not easily solved as the "resolution" of the packaging
in jar files is too coarse grained).  ABCL-ASDF currently only creates
a single ASDF:MVN object that represents the entire request, not
representing the other binary artifacts it introduces in a way that is
in any way amendable to "planning".

I figure that the right way forward would be to create additional
ASDF:MVN objects for each discrete dependency in-memory that would not
have a corresponding ASDF:MVN definition in a file on the filesystem.
To continue the example, the ABCL-ASDF code would create the
discovered "javax.mail/mail/1.4.3",
"javax.activation/activation/1.4.3", and
"org.apache.geronimo.specs/gerionimo-jms/1.1" ASDF:MVN objects, using
them to reason about what to load.

Questions for consideration by the ASDF community:

1.  As I understand it, my problem doesn't have the problems of
redefining ASDF behavior during ASDF's load phase that Faré wishes
to eliminate in asdf-3.3.  If I were to slog through asdf-3.2.x to
implement the planning which I need would such an effort be
impacted in any way that you can foresee by what you need to change
for asdf-3.3?

2.  Does anyone know if there an existing analogy for the ASDF:MVN
component in an ASDF extension that I could profitably study?
Currently, ABCL-ASDF hackishily neuters the ASDF:COMPONENT
association with a pathname, mainly because in the current
implementation a given ASDF:MVN component results in one or more
jar archives.  Does creating additional ASDF:MVN (a subclass of
ASDF:COMPONENT) instances in the in-memory ASDF that aren't
reflected in an *.asd file raise any problems that anyone is aware
of?

3.  In the last few months, I think I remember that there has been
discussion around the possible use of ASDF to locate and download
shared objects for CFFI 

Re: ASDF Best Practices

[Mark Evenson]

On 4/1/17 06:27, Faré wrote:
> I've started an ASDF Best Practices document as a response to all the
> ugly stuff I saw while debugging backward incompatibilities introduced
> by ASDF 3.3. It is currently in my plan branch:
> 
> https://gitlab.common-lisp.net/asdf/asdf/blob/plan/doc/best_practices.md

Very cool, and a needed resource to get some idea of the state of Faré's
brain.

One thing I learned from reading this document is that my use of symbols
to name and refer to ASDF systems is probably wrong, as it contradicts
the best practice of using strings for system identity.  I like using
symbols because

(asdf:defsystem democracry
   :depends-on (civil-society press/freedom)
…

looks much less cluttered than

(asdf:defsystem "capitalism"
   :depends-on ("profit/increasing" "internal-contradictions")
…

While I can see problems stemming from what package the "democracry"
symbol gets interned within, I assume that ASDF:DEFSYSTEM goes to great
lengths to ensure that it is processed in a reasonable manner to guard
against such problems, therefore system designators as symbols should be
more or less the same as system designators as strings.  Using strings
as system designators seems to work fine with asdf-3.2.0.

Just to be clear, could someone please illuminate my understanding here
a bit:  why are strings a better practice than symbols here?



-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Is the :PERFORM stanza still valid?

[Mark Evenson]

On 2016/12/8 18:28, Robert Goldman wrote:
> Hi, Mark --
> 
> The issue is in PROVE-ASDF.  Replacing line 50 of asdf.lisp
> 
> (asdf:perform (make-instance 'asdf:load-source-op) c)
> 
> with
> 
> (asdf:perform (asdf:make-operation 'asdf:load-source-op) c)
> 
> should solve the problem.
> 
> LMK if that doesn't fix the problem.

Thanks!  That works.

Created an [upstream pull request][34].

[34]: https://github.com/fukamachi/prove/pull/34

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Is the :PERFORM stanza still valid?

[Mark Evenson]

If the perform stanza still the way to add operable test systems in
ASDF?  It would be nice to update the manual if this is not the case.

For an ASDF test operation for a test system defined as:

(asdf:defsystem plato-test
  :depends-on (:plato
   :prove)
  :components ((:module "t"
:components
((:test-file "plato"
  :description "Test system for plato"
  :defsystem-depends-on (:prove-asdf)
  :perform (test-op (op c)
(uiop:symbol-call :prove-asdf 'run-test-system c)
(asdf:clear-system c)))

I am getting the error:

OPERATION instances must only be created through MAKE-OPERATION.
   [Condition of type ASDF/FIND-SYSTEM:FORMATTED-SYSTEM-DEFINITION-ERROR]

Restarts:
 0: [SKIP-TEST-FILE] Skip this test file.
 1: [SKIP-ALL-TEST-FILES] Give up all test files.
 2: [RETRY] Retry # on #.
 3: [ACCEPT] Continue, treating # on
# as having been successful.
 4: [RETRY] Retry ASDF operation.
 5: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting
the configuration.
 --more--

Backtrace:
  0: (#
#
#)
  1: (APPLY #
(#
#))
  2: (SYSTEM::RUN-HOOK SYSTEM::*INVOKE-DEBUGGER-HOOK*
#
#)
  3: (INVOKE-DEBUGGER
#)
  4: (ERROR ASDF/FIND-SYSTEM:FORMATTED-SYSTEM-DEFINITION-ERROR
:FORMAT-CONTROL "OPERATION instances must only be created through
MAKE-OPERATION." :FORMAT-ARGUMENTS NIL)
  5: (ASDF/FIND-SYSTEM:SYSDEF-ERROR "OPERATION instances must only be
created through MAKE-OPERATION.")
  6: (ASDF/OPERATION::CHECK-OPERATION-CONSTRUCTOR)
  7: (# #)
  8: (APPLY # (#))
  9: (#
#)
 10: (APPLY #
# NIL)
 11: (# #)
 12: (APPLY # (#))
 13: (# #)
 14: (APPLY # # NIL)
 15: (# ASDF/LISP-ACTION:LOAD-SOURCE-OP)
 16: (APPLY # (ASDF/LISP-ACTION:LOAD-SOURCE-OP))
 17: (MAKE-INSTANCE ASDF/LISP-ACTION:LOAD-SOURCE-OP)
 18: (PROVE.ASDF:RUN-TEST-SYSTEM #)
 19: (APPLY PROVE.ASDF:RUN-TEST-SYSTEM (#))

This is with ABCL trunk using ASDF:ASDF-VERSION "3.1.7.40".

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Patch for ABCL development versions

[Mark Evenson]

I'm unsure of the current ASDF work flow for patch submissions, but I
guessed that y'all would like [merge requests][57].

Without this patch, Elias' recent work on UIOP/RUN-PROGRAM won't be
utilized on any ABCL which is built from development source.

ABCL uses the convention that development versions start appending
strings separated via #\- characters to the primary value returned by
CL:LISP-IMPLEMENTATION-VERSION (e.g. '1.5.0-dev'. Such values cause the
UIOP/UTILITY:PARSE-VERSION function to return nil, meaning that this is
not a suitable conditional for whether LAUNCH-PROGRAM is invoked.

This patch uses the value of UIOP/OS:IMPLEMENTATION-IDENTIFIER to
identify version. Unknown whether this would work on MKCL/ECL, which
might simplify the code path here.

[57]: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/57

Please let me know what I can do to conform to y'alls current style
conventions, test coverage, etc. for such a patch submission.  This
patch is necessary for [my backporting of Elias's work to JDK56][1]

[1]: http://abcl.org/trac/ticket/422

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Misnamed secondary systems

[Mark Evenson]

> On 18 Nov 2016, at 14:40, Faré <fah...@gmail.com> wrote:
> 
> On Fri, Nov 18, 2016 at 8:36 AM, Mark Evenson <even...@panix.com> wrote:
>>> I'd like to forbid such misnamed systems.
>>> Now a quick grepping through Quicklisp (see latest update to my ql-test)
>>> finds 233 .asd files with such misnamed secondary systems.
>>> Obviously it will take time to clean up the mess,
>>> so for after the next release, I'd like to signal a full WARNING
>>> when the condition is detected, and at some point,
>>> make that a CERROR, then later an ERROR.
>> 
>> I object on the grounds of widespread adoption.  At least it will leave me 
>> on the current ASDF for a long time.
> 
> What's wrong with issuing a WARNING until said adopting is down 95% ?

I have a substantial use of secondary systems in my personal code that will
take a long time to unwind.  Since it was an advertised feature of ASDF3, I
expect to be around for the lifetime of that version.

As an implementor, I will patch ABCL’s ASDF3 to muffle such warnings, but to
remove behavior without a bit longer warning to my user base seems
unacceptable.

Please put it in ASDF4.

Sorry for being harsh, and terse, but if you are asking for opinions, I happen
to have a strong one here.

With respect,
Mark

Re: Misnamed secondary systems

[Mark Evenson]

> On 18 Nov 2016, at 14:19, Faré  wrote:
> 
> Starting with ASDF 3 (2013), ASDF officially supports "secondary systems",
> i.e. additional systems named in an asd, beside the main one.
> For the additional systems to be findable by ASDF, though, their name
> must be of the form "foo/bar" where "foo.asd" is the name of the file,
> and "foo" is called the primary system name.
> 
> ASDF1 and ASDF2 would let you define those systems, as well as
> other systems of arbitrary names, but would be incapable of finding them.
> By badly copying each other, Lispers have developed a lot of misnamed
> secondary system the name of which doesn't match the pattern.
> 
> I'd like to forbid such misnamed systems.
> Now a quick grepping through Quicklisp (see latest update to my ql-test)
> finds 233 .asd files with such misnamed secondary systems.
> Obviously it will take time to clean up the mess,
> so for after the next release, I'd like to signal a full WARNING
> when the condition is detected, and at some point,
> make that a CERROR, then later an ERROR.
> 
> Cleaning up this mess will also prevent a subtle class of errors,
> such as when two .asd files define each other's systems
> (see test case test-mutual-redefinition.script in asdf/test).


I object on the grounds of widespread adoption.  At least it will leave me on 
the current ASDF for a long time.

Re: Recent disappearance of asdf:bundle-system.

[Mark Evenson]

On 2016/9/11 17:09, Faré wrote:
> 1- While the trivial convenience function bundle-system was removed,
> the underlying functionality still exists. The function was ill-named
> legacy of dubious value. Do ABCL users actually use this function as
> such?

[Sorry for the late reply]

As far as I know, no one actively uses BUNDLE-SYSTEM for ABCL, instead
using the [ASDF-JAR ABCL contrib][1], which allows a system to
(optionally recursively) package all dependencies of an ASDF system into
a single binary artifact (a jar file containing all the code).

If I understand things conceptually correctly, if their were a stable
BUNDLE-SYSTEM, we would move the code for ASDF-JAR to use those interfaces.

[1]: http://abcl.org/trac/browser/trunk/abcl/contrib/asdf-jar

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Recent disappearance of asdf:bundle-system.

[Mark Evenson]

> On Sep 11, 2016, at 08:31, Jean-Claude Beaudoin 
>  wrote:
> 
> Hello ASDF devs,
> 
> I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF.
> MKCL is/was a user of that function as a convenient entry point to the ASDF 
> bundle facility.

[…] 

For what it is worth, ABCL had an implementation for BUNDLE-SYSTEM that, while 
suffering from a little bitrot, constructed a “monolithic” FASL from all the 
compilation units of a give ASDF definition.  If BUNDLE-SYSTEM gets 
resurrected, I would re-work ABCL’s implementation to be more conformant.
 

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."

Re: Simple (?) Windows question

[Mark Evenson]

On 2016/3/21 21:15, Robert Goldman wrote:
> Is there some way that a CL implementation can know whether it is
> running under a Unix-alike like Cygwin or MinGW?

Another point to consider in finding the Right Thing here:

Since ABCL runs on the JVM and the JVM is a Windows "native" executable,
there is no difference for ABCL whether it has been invoked under cygwin
or not.  Of course, since we have a fair about of specializaion  in
ABCL's PATHNAME implementation to smooth over the differences between
Windows and UNIX representations, notably in the normalization of
NAMESTRING results, the problems you are trying to engineer around here
don't show up in ABCL from what I can tell.

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Understanding UIOP/RUN-PROGRAM

[Mark Evenson]

In a potentially memory constrained environment, I need to portably
(across sbcl and ccl at least) process a potentially large (multiple
GiB) stream of bytes output from a Linux command.  For the curious,
"process" here means encrypt with a block cipher and push to the
network; whereas the UNIX command is the output of "btrfs send".

I wanted to confirm with ASDF developers that as far as I can tell from
wrangling with UIOP:RUN-PROGRAM, it isn't going to do what I want
because there is no "asynchronous" mode.  UIOP:RUN-PROGRAM cannot be
"run in the background" like SBCL's SB-EXT:RUN-PROGRAM with :wait nil
that allows the output of the sub-process to be gathered and processed
in blocks.  So, it would always be the case that
UIOP/RUN-PROGRAM::SLURP-INPUT-STREAM has "gone through" all the
(potentially huge) output of the sub-process before UIOP:RUN-PROGRAM can
return.

Note that I'm not complaining about UIOP:RUN-PROGAM: it certainly
bridges a lot of implementation specific details of the use case for
"Execute this command; let me process its output".  I'm just trying to
affirm my understanding of UIOP and (regrettable) need to move off to do
my own CCL/SBCL implementation for my needs.

If my understanding is correct, the UIOP maintainers may wish to
emphasize the synchronous nature of UIOP:RUN-PROGRAM in its docstring.

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Re: Understanding UIOP/RUN-PROGRAM

[Mark Evenson]

On 2015/9/29 17:34, Faré wrote:
> On Tue, Sep 29, 2015 at 11:03 AM, Mark Evenson <even...@panix.com> wrote:
>> I wanted to confirm with ASDF developers that as far as I can tell from
>> wrangling with UIOP:RUN-PROGRAM, it isn't going to do what I want
>> because there is no "asynchronous" mode.
> UIOP:RUN-PROGRAM itself is synchronous only, but its internal
> UIOP::%RUN-PROGRAM is asynchronous on platforms that support it...
> which does not include ABCL at this time, but does include CCL and SBCL.
> The interface is under-documented, and which exact subset of features
> it supports depends on the underlying implementation;
> but obviously it works well enough to portably implement the synchronous
> UIOP:RUN-PROGRAM.

Is their any informal ASDF policy on the stability unexported symbols
like UIOP::%RUN-PROGRAM?  I suspect y'all reserve the right to yank the
rug at any point, right?  Given that UIOP::%RUN-PROGRAM abstracts the
Lisp implementations capable of asynchronous operation, I guess it is a
little more stable.

[…]
> If you only care about CCL and SBCL, you can use UIOP::%RUN-PROGRAM.
> 
> If you want a portable solution that requires an external library, try
> IOLIB/OS:CREATE-PROCESS.

The "external library" library part of IOLIB always tends to make me shy
away.

@stellian and @fare:  thanks for the advice.

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."

Introspecting source registry

[Mark Evenson]

With asdf-3.1.4, how do I introspect the current state of source
registry configuration?  I need to debug a remote user's ASDF
configuration on a machine which I don't have access to, so I would like
to have ASDF output the current source registry configuration search
state.  Problematically, the behavior documented in section 8.1 doesn't
seem to be working as described on the user's machine.  And this
behavior seemingly changes between patch level ASDF releases
especially on marginal platforms like Windows, digging up the correct
version of the manual isn't so easy either.

[1]:
https://common-lisp.net/project/asdf/asdf.html#Controlling-where-ASDF-searches-for-systems

The contents of ASDF/SOURCE-REGISTRY:*SOURCE-REGISTRY* provides a
hashtable of systems that ASDF has located, but doesn't help me figure
out where ASDF *might* have searched for things.  For output
translations the return value of (ASDF:INITIALIZE-OUTPUT-TRANSLATIONS)
gives me enough information for output translations, so it would be nice
to have something equivalent for the source registry.  Since I suppose
that all of the behavior specified in section 8.1 is not easily
transcribed as a s-expr, my request may not be trivial to implement, but
in lieu of some similar mechanism, I am really having problems with
debugging ASDF behavior in a particular instance.

Additionally, what is the behavior of ASDF when the $XDG_* environment
variables are not set, as is seemingly the case on a stock Windows
environment?  Are these configuration steps just skipped?

-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: manual version

[Mark Evenson]

On 2015/4/3 14:29, Robert P. Goldman wrote:

 Seems like we should encourage the implementations to make the manual
 available to their users.

 ABCL has [always included the manual in the source distribution][1].

 Would you also like it in the binary distribution?

 [1]: http://abcl.org/trac/browser/trunk/abcl/doc/asdf


 
 That sounds like a good idea, as long as it's not to much trouble.

The ASDF manual shipped with the [abcl-1.3.2 binary release][1]

[1]: http://abcl.org/releases/1.3.2/


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: ASDF configuration and cache

[Mark Evenson]

On 2015/4/29 10:03, Faré wrote:
 Dear Mark,
 
 ~/.config/common-lisp/source-registry.conf.d/ is being used because
 you're compiling on Unix. If you were using Windows, it would be using
 something else based on LOCALAPPDATA, not USERPROFILE.

Hmmm.  Do you mean compiling [ABCL] on Unix?  That is not the case, as
the ABCL being used has been compiled on the Windows host.  And the
creation of the WAR artifact for production is also happening on the
Windows host.

While running on Windows, everything *is* being compiled under Cygwin.
 Maybe that is the cause?

 For reliable builds, especially ones that create cross-platform
 objects, we recommend you don't use either, but explicitly pass a
 proper (or null) overriding configuration to the initialization
 functions initialize-source-registry and
 initialize-output-translations.

I'm a little unsure what you are suggesting here, perhaps because I am
using a strategy outside of the normal use of ASDF.  This strategy can
roughly be described as follows:  At build time for the WAR, I need to
locate all the ASDF systems in a running ABCL image.  This allows me to
package both Quicklisp available systems with local ones in a consistent
manner that freezes their versions, and doesn't rely on access to the
network at production time.  Then I iterate across all the dependencies
performing an rsync of all the source artifacts into  a designated build
directory that is subsequently packaged in the WAR deployment artifact.
 Here, I assume that the .asd file lies at the top of the source
artifact file hierarchy, an assumption which seems to largely work with
the ASDF systems I am utilizing at the penalty of including sub .asd
systems twice (since they are the same system, this inefficiency doesn't
really matter although it should be cleaned up).  At runtime, I use
ASDF:INITIALIZE-SOURCE-REGISTRY to explicitly refer to the directory
containing the source artifacts, which is now located within the WAR
artifact.  ABCL pathnames are able to specify locations within the zip
file which constitutes the WAR artifact.  After this manual
configuration of the source registry, the systems are available by the
usual ASDF:LOAD-SYSTEM mechanism.

Under this scheme is there an application of proper (or null)
overriding configuration that you recommend using?

Best,
Mark

-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: ASDF configuration and cache

[Mark Evenson]

On 2015/4/29 11:24, Faré wrote:
 On Wed, Apr 29, 2015 at 2:00 AM, Mark Evenson even...@panix.com wrote:
 On 2015/4/29 10:03, Faré wrote:
 Dear Mark,

 ~/.config/common-lisp/source-registry.conf.d/ is being used because
 you're compiling on Unix. If you were using Windows, it would be using
 something else based on LOCALAPPDATA, not USERPROFILE.

 Hmmm.  Do you mean compiling [ABCL] on Unix?  That is not the case, as
 the ABCL being used has been compiled on the Windows host.  And the
 creation of the WAR artifact for production is also happening on the
 Windows host.

 While running on Windows, everything *is* being compiled under Cygwin.
  Maybe that is the cause?

 If you have :unix in your *features*, then ASDF decides that ABCL is
 running on Unix. Is that not a correct criteria? Are you enabling
 :unix because of cygwin?

Please re-explain your reasons for thinking that ABCL is mistakenly
using file:%USERPROFILE%/.config/ as we're not pushing :UNIX to
*FEATURES* as far as I can see:

CIMS-dev-01:~/work/abcl$ uname -a
CYGWIN_NT-6.1 CIMS-dev-01 1.7.35(0.287/5/3) 2015-03-04 12:09 x86_64 Cygwin
CIMS-dev-01:~/work/abcl$ ./abcl.bat
VM settings:
Max. Heap Size (Estimated): 4.00G
Ergonomics Machine Class: client
Using VM: Java HotSpot(TM) 64-Bit Server VM

Armed Bear Common Lisp 1.4.0-dev
Java 1.8.0_40 Oracle Corporation
Java HotSpot(TM) 64-Bit Server VM
Low-level initialization completed in 0.742 seconds.
Startup completed in 5.988 seconds.
;  Loading
jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/abcl-contrib.abcl
...
;   Loading
jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/asdf.abcl
...
;   Loaded
jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/asdf.abcl
(7.577 seconds)
Using probed value of abcl-contrib:
'C:/Users/markeven/work/abcl/dist/abcl-contrib.jar'.
Added
jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/quicklisp/
to ASDF.
Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/mvn/
to ASDF.
Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/jss/
to ASDF.
Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/jfli/
to ASDF.
Added
jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/asdf-jar/ to
ASDF.
Added
jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/asdf-install/ to
ASDF.
Added
jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/abcl-asdf/
to ASDF.
;  Loaded
jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/abcl-contrib.abcl
(7.694 seconds)
Loading C:\Users\markeven\.abclrc completed in 7.697 seconds.
Type :help for a list of available commands.
CL-USER(1): (find :unix *features*)
NIL
CL-USER(2): *features*
(:ASDF-PACKAGE-SYSTEM :ASDF3.1 :ASDF3 :ASDF2 :ASDF :OS-WINDOWS
 :ASDF-UNICODE :ABCL-BUNDLE-OP-SUPPORTED :DRAKMA-NO-SSL
 :HUNCHENTOOT-NO-SSL :X86-64 :WINDOWS :ARMEDBEAR :ABCL :COMMON-LISP
 :ANSI-CL :CDR6 :MOP :PACKAGE-LOCAL-NICKNAMES)


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: manual version

[Mark Evenson]

On 3/24/15 03:12, Robert P. Goldman wrote:
 DJ wrote:
 How do I know whether the asdf manual I am looking at matches the
 version of asdf I have installed?

 
 Seems like we should encourage the implementations to make the manual
 available to their users.

ABCL has [always included the manual in the source distribution][1].

Would you also like it in the binary distribution?

[1]: http://abcl.org/trac/browser/trunk/abcl/doc/asdf


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: [Asdf-devel] startup times and initialize-source-registry

[Mark Evenson]

On 21 Aug 2014, at 02:36, Faré fah...@gmail.com wrote:

[…] 

 The trick here is in this new stop-at-asd flag, which here defaults to
 t and isn't configurable, but which should default to nil and be
 configurable, for backward compatibility. Its effect is that recursing
 into subdirectories stops if a .asd file is found in the toplevel
 directory. This saves a lot of recursing, and would save even more if
 a .asd file of symlink to one exists at the top of a git hierarchy.
 But this is incompatible with a lot of existing code, and so the
 transition will be long and painful if this is adopted.

If you proposing that the stop-at-asd property would be somehow configurable in
the DEFSYSTEM form, like:

(asdf:defsystem :foo
   :contains-interior-asdf-defintions
   :components …

then please ensure that this is present when/if you introduce this change to
ASDF.  But I get the feeling that in order to speed things up, you weren’t
intending to parse the DEFSYSTEM form in your search.

If you are proposing that the user would have to do explicitly do some sort of
configuration “for this instance of a user using asdf with this Lisp
implementation”, I won’t be so happy because:

1)  This sort of configuration hasn’t been necessary before, so we will
introduce complexity in ASDF deployment for efficiency in using CL as a
scripting language which is something I don’t currently use (Admittedly because
my platform, ABCL, based on the JVM, is just not going to ever have reasonable
startup times.  Although there are systems that keep a JVM “warmed up” for
firing such one-off commands to, and for specialised JVM there are memory
mapped solutions for faster startup).

2)  I am using a system (lsw2) not in Quicklisp that has many such “interior”
ASDF definitions. Usually when systems get in this state it is because they are
big enough that nobody has time to package them correctly, so they tend to stay
that way.  If I can’t put a flag in the top-level system, I’m going to run into
problems when users haven’t done the per-user-per-lisp configuration.

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.






___
Asdf-devel mailing list
Asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [Asdf-devel] ASDF 3.1.2.9 now available

[Mark Evenson]

On 17 Jul 2014, at 04:43, Anton Vodonosov avodono...@yandex.ru wrote:

 I've run cl-test-grid tests on several lisps.
 
 No regressions detected.
 
 The lisps tested:
 abcl-1.2.1-fasl42-linux-x86
 abcl-1.3.1-fasl42-linux-x86

abcl-1.4.0-dev has been updated to this ASDF.

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.






___
Asdf-devel mailing list
Asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] proposal to split ASDF in half

[Mark Evenson]

On Mar 13, 2014, at 15:42, Robert P. Goldman rpgold...@sift.info wrote:

[…] 

 I believe Faré and I discussed this issue in our first ASDF paper at
 ILC.  For an even more radical proposal, see McDermott's earlier ILC
 paper about his chunking mechanism, which allows programmers to specify
 integrity constraints even for individual data structures.

Suggestion: please put URI de-referencable bibliographic resources like this in 
the ASDF documentation.

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.

Re: [asdf-devel] Alternate default lisp system location

[Mark Evenson]

I smell a need for a CDR, c'nest pas?

Tersely pecked on my iPad

On Mar 13, 2014, at 18:42, Faré fah...@gmail.com wrote:

 : dherring
 : sionescu
 : rpgoldman
 : sionescu
 
 For essential infrastructure like what ASDF claims to be, I expect major
 changes to happen less than once every 5 to 10 years.
 Daniel, if you want changes only every 5 to 10 years, do like
 janderson: try an upgrade every 5 years, find that it breaks your code
 somehow and downgrade back to the old version without further
 investigation. Then maybe 10 years from now you'll get the system
 dependency bug fixed.
 
 You can expect whatever you want, but unless somebody is paid full-time
 to work on ASDF, it's not going to happen.
 
 Agreed.  Tell me: am I to piss our contributors off by refusing to
 accept their patches, in order to make happy the people who contribute
 only complaints? New features may simply be the price users pay to have
 bugs getting fixed.
 
 The force vs force-not message is a good example. A feature was added
 at the request of a user without considering all the use cases and now
 we discover that it's poorly thought out. When you say this has always
 been my impression..., it means that nobody ever even wrote down a
 rationale for that new feature so you don't *know*, you have
 impressions.
 force and force-not were done well. The interaction between the two
 was not well thought out, but obviously no one relied on the
 interaction before force-not was implemented, so that was backward
 compatible, and the recent change keeps the previously useful uses
 working, so it is arguably, too.
 
 It's not like my changes were done in secret. I did it and explained
 what I did, and even mentioned at the time I didn't understand which
 setting should take precedence over the other. I only apologize for
 not understanding then the objection by Robert, though he also didn't
 care enough to submit a patch, and no one else even complained.
 
 The more backwards compatibility, the better.  Projects like glibc
 have developed significant infrastructure to enable transparent
 improvements (see the ABI compliance checker, DSO symbol versioning,
 etc.).
 
 See above. That kind of backwards-compatibility is very difficult and
 burdensome.
 
 We spend a great deal of time maintaining backwards compatibility.
 Consider how much work was spent making the bug-fix coming from
 PREPARE-OP from breaking previous OPERATION subclasses.  Similarly, as
 someone who still uses ASDF:*CENTRAL-REGISTRY*, maintaining that is
 substantial backwards compatibility.
 
 The backwards-compatibility is not complete, though. When we're talking
 about glibc, we're talking about versioning functions and keeping the
 old entry points bug-to-bug compatible for ever while the new version of
 that function is simply an addition.
 Issuing a warning that OPERATION is being directly subclassed is not
 backwards-compatibility.
 We do care about backward compatibility, a lot. Yes, there are
 sometimes bugs in our backward-compatibility support. And then we
 painfully fix them, as in the case of OPERATION above. Are we perfect?
 No. Do we have symbol versioning for backwarder compatibility? No. The
 libc has additional issues and solutions because it's all about binary
 compatibility. We do not offer binary compatibility. We offer source
 compatibility. We consider binary compatibility is backward.
 
 Every breaking change inflicts cost on a fraction of the existing
 userbase, in exchange for some proposed benefit to future users.  Every
 time I have to debug breakage and change something or redesign my workflow
 to maintain existing capability, it encourages me to explore other, more
 stable or better designed options...
 
 Sometimes good ideas fade after a month or two of reflection.  Some
 survive the test because the benefit truly outweighs the cost.  When that
 is the case, it is often helps to give the community time to prepare and
 minimize the number of times the community must change.  So propose the
 change, allow a long RFC window, allow users to obtain test
 implementations (while still promoting the stable branch), wait a while
 for several changes to collect before pushing them into major new
 releases, etc.
 
 I agree that an RFC-like process would be useful, instead of jumping in
 and implementing new features, as long as it's not too lengthy.
 
 It has, in fact, been a long time since ASDF's last release, October
 2013.  During that time, there have been a very large number of tagged
 versions available to the community.
 
 I don't think that this community can afford the time, nor can it muster
 the interest, to deal with such an RFC process.
 
 What community ?
 If someone wants a RFC process, he's welcome to write a specification
 for ASDF that all the currently documented or undocumented features
 implicitly relied upon by every piece of software in Quicklisp, and
 develop an independent reimplementation of ASDF based on that
 specification.
 
 

Re: [asdf-devel] asdf loading error error

[Mark Evenson]

On 2/23/14, 17:27, Robert Goldman wrote:
 Faré wrote:
 On Sat, Feb 22, 2014 at 3:26 PM, Cyrus Harmon ch-l...@bobobeach.com wrote:
 While trying to load and asdf system that generates an error (that’s 
 another story, presumably an abcl-contrib problem), I get the following 
 error:

 Error while trying to load definition for system abcl-cdk from
 pathname /Users/sly/projects/abcl-cdk/abcl-cdk.asd:
There is no applicable method for the generic function 
 #STANDARD-GENERIC-FUNCTION ASDF/SYSTEM:BUILTIN-SYSTEM-P {4DE3AFA9} when 
 called with arguments ((NIL)).
[Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]

 There's an error while reporting a warning (warning about your system
 having a bad version string).
 I suggest you add this method in system.lisp after defclass system:
  (defmethod builtin-system-p ((x null)) nil)
 Does it help?

 Robert, do you want me to commit that?
 
 I'd like to understand this patch better before we push it.
 
 Why is it that having a method on NIL is right?  Doesn't this just mask a bug 
 where we are incorrectly treating NIL as if it's a system designator?
 
 What's the cause for trying to ask if NIL is a built in system?  Presumably 
 some system lookup returned NIL, right?  If so, why wasn't that trapped as an 
 error?

The error is coming from the [ABCL-ASDF contrib which defines an
MVN-COMPONENT class descending from ASDF:COMPONENT][1].  This class is
used to represent a Maven dependency which is resolved to a binary JVM
artifact on the distributed Maven POM graph specified by three
coordinates:  GROUP-ID, ARTIFACT-ID, and VERSION.

In my implementation, I have (probably mistakenly) tried to overload the
ASDF:COMPONENT VERSION slot to contain a version inteligble to both
Maven and ASDF.  Unfortunately, the Maven version can basically contain
any string, not having a meaningfully defined ordering (i.e. you can
only determine if versions are different, not if one preceded another)
which conflicts with ASDF requirements for strict semantic versioning
(if I am using that term correctly).  In my defense, I think this was
working this way with ASDF 1, before ASDF 2's requirements came into being.

Currently, ABCL-ASDF cheats by stuffing a possibly non-conforming ASDF
version in the COMPONENT VERSION slot via the ASDF:ENSURE-MVN-VERSION
cleanup function, so that the form

   (:mvn org.openscience.cdk/cdk-bundle :version 1.5.6-SNAPSHOT)

produces this error while

(:mvn org.openscience.cdk/cdk-bundle/1.5.6-SNAPSHOT)

will correctly reference the correct Maven artifact, sneaking by ASDF's
checking for a valid version.

The right way forward would be to re-write the MVN-COMPONENT class not
to use the inherited VERSION slot, but another for its purposes.

But in the meantime, users get a baffling error when they use a non-ASDF
compliant slot value.


[1]:
http://abcl.org/trac/browser/trunk/abcl/contrib/abcl-asdf/abcl-asdf.lisp


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

Re: [asdf-devel] BUNDLE-OP now working on abcl-1.3.0-dev

[Mark Evenson]

On Feb 12, 2014, at 16:57, Robert Goldman rpgold...@sift.net wrote:
 
 Thank you very much, Mark -- I will get a micro-patch for this installed into 
 ASDF for the new release.

ASDF 3.1.0.65 works with ABCL trunk, which has just been committed as 
[r14626][].

Sorry that it is taking me so long to close out the ABCL issues.  Next up:  
pathname translation (wrt. case).

[r14626]: http://abcl.org/trac/changeset/14626 

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.

Re: [asdf-devel] ASDF walkthrough

[Mark Evenson]

Excellent idea! Please let me know when you will be doing this. Currently, I 
would possibly profit from an explanation of the kinds of transversal ASDF 
does over the system of systems it needs to do, but that is not too sharply 
defined  as a question yet.

Tersely pecked on my Nexus 5

On Jan 3, 2014 8:10 AM, =?ISO-8859-1?Q?Far=E9?= fah...@gmail.com wrote:

 Dear Lisp hackers, 

 I'm considering recording a walk through the ASDF sources. 

 I'd like to have an interactive session over Google Hangout 
 with one or a few people, explaining the current code in asdf/defsystem 
 (i.e. not going into uiop, except to minimally explain what asdf uses). 
 That should take about 2 hours. 

 To enjoy the walkthrough, you need to be somewhat fluent in Common Lisp, 
 and somewhat familiar with how to use ASDF, though not necessarily with 
 either extending it or hacking its internals. 

 If at least one person is familiar with ASDF 2 and has questions, 
 that could be interesting, or that could be a separate session. 

 Please respond me to this email (privately or publicly) if you're interested. 

 —♯ƒ • François-René ÐVB Rideau •ReflectionCybernethics• 
 http://fare.tunes.org 
 We are always in anarchy. But we pay a hefty price maintaining the illusion 
 that we aren't, and another one being misled by the illusion. 

Testing ASDF with ABCL under Windows (was Re: [asdf-devel] Releasing asdf 3.1.1 ?)

[Mark Evenson]

On Dec 20, 2013, at 22:34, Mark Evenson even...@panix.com wrote:

[…]

 It isn’t quite clear how to run the tests under Windows.  I am using Cygwin, 
 so I thought I could just ensure that ‘abcl’ is in my path and symlinked to 
 ‘abcl.bat  
 
   cygwin-bash$ sh run-tests.sh abcl 
 
 but I get an “access is denied” message, which is odd as I can invoke `abcl` 
 directly.  Probably some faulty understanding on my part of cygwin v. DOS 
 scripting.  

A couple of notes on running ASDF tests under Windows:

1.  The “Access is denied” message comes from problems with mismatched NTFS 
permissions.  I think it was the result of using “hg clone in the Windows 
System Administrator” role which creates directories don’t automatically have 
all necessary permissions for writes.  Or something.  The solution for me was 
to use the “Edit” dialog accessed “Properties  Security” popup from Windows 
Explorer to recursively set “Full Control” for “Everyone” from the root `asdf` 
folder.  Those who wish to contribute a more nuanced understanding of what 
Windows is trying to protect against are invited to explain in a more 
articulate manner the more proper set of restricted permissions to grant.

2.  Using cygwin, one cannot get ‘test/run-tests.sh’ to use the ‘abcl.bat’ 
Windows batch script to invoke ABCL, as it fails with problems in escaping the 
VERTICAL_LINE (#\|) from some being interpreted as a pipe operation by some 
part of the invocation chain.  The solution is to use the Bourne-shell ‘abcl’ 
invocation script.

More soon…

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.

Re: [asdf-devel] Releasing asdf 3.1.1 ?

[Mark Evenson]

On Dec 19, 2013, at 17:23, Mark Evenson even...@panix.com wrote:

[…]
 
 I’ll take a look at  the ABCL problem in the next day, reporting back to the 
 list if I have a fix

Unfortunately, I have come to the end of a long day for me without much to 
report as I have spent most of my non-work time mucking with getting a 
virtualized version of Windows running/patched/configured in which to test the 
ABCL ASDF failures.

It isn’t quite clear how to run the tests under Windows.  I am using Cygwin, so 
I thought I could just ensure that ‘abcl’ is in my path and symlinked to 
‘abcl.bat  

cygwin-bash$ sh run-tests.sh abcl 

but I get an “access is denied” message, which is odd as I can invoke `abcl` 
directly.  Probably some faulty understanding on my part of cygwin v. DOS 
scripting.  

So, if someone can please provide a quick sketch of how to run the ASDF tests 
under Windows, I can try pick this up tomorrow.

And if abcl is the only thing holding up a release, y’all might as well pull 
the trigger without the Bear, as the social aspect of holidays with family will 
place additional stress on my spare time.



-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.

Re: [asdf-devel] Releasing asdf 3.1.1 ?

[Mark Evenson]

On Dec 19, 2013, at 17:20, Robert Goldman rpgold...@sift.net wrote:

 Faré wrote:
 Dear Robert,
 
 do you think we can release this year?
 All tests pass for me on Linux, and I believe all tests were passing
 for Dave Cooper on Windows. There has been very light development this
 past month, with no intent for further development ahead, but notable
 new features and bug fixes since the last release in October.
 
 —♯ƒ • François-René ÐVB Rideau •ReflectionCybernethics• 
 http://fare.tunes.org
 A student who changes the course of history is probably taking an exam.
 
 
 I would be happy to make a release.  I would like to have the bundle op fixed 
 before we do so, if possible.  bundle-op doesn't work (fails tests) on Mac 
 OSX for ABCL and ECL.
 
 I *suspect* these are both due to bugs in the implementations.  ABCL I am 
 cautiously optimistic about getting a fix.  ECL, I am pessimistic, since I 
 haven't heard about anyone taking it over since Juanjo stepped aside as ECL 
 project lead.

I’ll take a look at  the ABCL problem in the next day, reporting back to the 
list if I have a fix

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.

[asdf-devel] Test for ASDF system in jar archive with ABCL

[Mark Evenson]

The ABCL contribs are all defined via ASDF in a jar file, so the test is simply
to try to load one of them, here JSS.

Not all downstream packaging systems include the abcl-contrib.jar, so I try to
guard against a false negative with an UNLESS clause in the abcl-contrib
REQUIRE, but in my testing by removing abcl-contrib.jar this didn’t seem to 
work correctly.


-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.






test-jar-pathname.script
Description: Binary data

[asdf-devel] Patch for UIOP/UTILITY:ENSURE-FUNCTION

[Mark Evenson]

Without the attached patch, I believe the use of a function for output 
translations is hosed from post asdf-3.0.2 onwards. 

I think ABCL is the only implementation actively using the ability to specify 
an output translation as a function, as indeed Faré expressly introduced this 
for us.   ABCL uses a specialized output translation to translate the location 
of FASLs for ASDF systems stored in jar files (like ABCL’s contribs) which 
can’t otherwise be expressed in the translation DSL.   

I have [committed an asdf-3.0.3.0.1 including this patch][r14583] to ABCL 
trunk, which I will sync in the near future to ASDF master if this is indeed 
the correct way to patch the problem.  

[r14583]: http://abcl.org/trac/changeset/14583

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.






ensure-function.diff
Description: Binary data

[asdf-devel] Patch for UIOP/UTILITY:ENSURE-FUNCTION

[Mark Evenson]

Without the attached patch, I believe the use of a function for output 
translations is hosed from post asdf-3.0.2 onwards. 

I think ABCL is the only implementation actively using the ability to specify 
an output translation as a function, as indeed Faré expressly introduced this 
for us.   ABCL uses a specialized output translation to translate the location 
of FASLs for ASDF systems stored in jar files (like ABCL’s contribs) which 
can’t otherwise be expressed in the translation DSL.   

I have [committed an asdf-3.0.3.0.1 including this patch][r14583] to ABCL 
trunk, which I will sync in the near future to ASDF master if this is indeed 
the correct way to patch the problem.  

[r14583]: http://abcl.org/trac/changeset/14583

-- 
A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now.






ensure-function.diff
Description: Binary data

Re: [asdf-devel] [armedbear-devel] ASDF-JAR vs ASDF3 deployment support

[Mark Evenson]

On 4/4/13 12:44 AM, Anton Vodonosov wrote:
[…]
 General note about deployment CL systems.
 
 Often the deployment package should contain not only compiled code, 
 but also resources: web libraries and frameworks often contain .css,
 .html, .javascript; bordeaux-threads uses version.lisp-sexp files,
 and so on.
 
 In addition to the #. reader macro as used in bordeaux-threads,
 applications/libraries access their resources during run-time using
 asdf:system-relative-pathname or similar functions. In other words,
 such libraries assume presence during run-time of their full source
 control checkout .
 
 A general purpose deployment solutions should account this aspect.

 One way to deploy applications with full content of the dependencies
 code and resources:
  1. install quicklisp in a custom directory,
  2.  (ql:quickload :my-application)
  3. copy the directory of my-application, the custom quicklisp
 directory and probably prebuild .fals files to server.

 A little bit more docs and example of this approach is here:
 https://github.com/avodonosov/heroku-buildpack-cl2

In my opinion of the right way, system definitions should contain a
comprehensive listing of all the resources in the DEFSYSTEM form.
Anything referenced by ASDF:SYSTEM-RELATIVE-PATHNAME or as a target of a
CL:SHARPSIGN-DOT reader macro should be declared as a STATIC-FILE ASDF
component.  If this were somehow universally the case, then we could
reliably use the machinery of ASDF3 to transverse its components, and
determine both the source and the compiled components.

But because universal distribution (i.e. for the majority of users of
a given system) has more-or-less subsumed by Quicklisp, there is no real
constraint on the developers of system definitions to ensure that all
components are so enumerated:  loading the system from Quicklisp works
so nothing is broken, right?

After Xach, Anton probably has the most practical experience with
packaging ASDF definitions for deployment due to his pioneering work on
CL-TEST-GRID.  I find Anton comments to be a succinct summary of the
issues involved in packaging a given ASDF system in Quicklisp.  But when
facing deployment for an ASDF system that is not in Quicklisp, but has
dependencies on Quicklisp systems, Anton's approach won't work so well.
 One would have to find the .asd definition on the filesystem, then
transverse its sub-directories to find the source files.  This has the
problem that there is no guarantee that the components of an asd
definition exist in subdirectories, as theoretically the PATHNAME could
point to an ancestor.  And even if one discarded this possiblity, often
directory hierarchies share multiple ASDF definitions, so one would
package source components that aren't really source components.

So, assuming that deployment outside of Quicklisp from an ASDF
definition would be a greater good for the community, we seem to be
stuck at a bootstrap problem.  I can't think of a way to force ASDF
authors to enumerate their static components, but until such enumeration
is widespread, I can't demonstrate its utility by developing an
ASDF/BUNDLE operation.

Any ideas?


-- 

A screaming comes across the sky.  It has happened before, but there is
nothing to compare it to now.




___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ABCL and UIOP

[Mark Evenson]

On 3/26/13 1733 , Mark Evenson wrote:

I meant that I couldn't figure out how to reproduce the state you get
ABCL into, but it seems the answer is that one tries to QL:QUICKLOAD
:UIOP from abcl-1.2.0-dev.


We managed to get this fixed in [ABCL][r14449].

[r14449]: http://trac.common-lisp.net/armedbear/changeset/14449

Now to contemplate ASDF-JAR as part of ASDF/BUNDLE…


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ABCL and UIOP

[Mark Evenson]

On 3/26/13 0039 , Faré wrote:

Well, in this case, it looks like it's a bug in ABCL:

[1] UIOP/BACKWARD-DRIVER(14): (find-symbol LOAD-ASDF-DEBUG-UTILITY
:uiop/utility)
UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY
:INTERNAL
[1] UIOP/BACKWARD-DRIVER(15): (find-symbol LOAD-ASDF-DEBUG-UTILITY
:uiop/driver)
UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY
:EXTERNAL
[1] UIOP/BACKWARD-DRIVER(17): [1] UIOP/BACKWARD-DRIVER(17): (unexport
(find-symbol LOAD-ASDF-DEBUG-UTILITY :uiop/driver) :uiop/driver)
Error loading /home/tunes/cl/asdf/build/asdf.lisp at line 5302 (offset 252109)
#THREAD interpreter {7E6AB533}: Debugger invoked on condition of
type PACKAGE-ERROR
   The symbol UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY is not accessible
in package UIOP/DRIVER
Restarts:
   0: ABORT Return to debug level 1.


[…]

Thanks for the confirmation.  We're tracking this as ticket [311][].

[311]: http://trac.common-lisp.net/armedbear/ticket/311


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ABCL and UIOP

[Mark Evenson]

On 3/26/13 0039 , Faré wrote:

Well, in this case, it looks like it's a bug in ABCL:

[1] UIOP/BACKWARD-DRIVER(14): (find-symbol LOAD-ASDF-DEBUG-UTILITY
:uiop/utility)
UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY
:INTERNAL
[1] UIOP/BACKWARD-DRIVER(15): (find-symbol LOAD-ASDF-DEBUG-UTILITY
:uiop/driver)
UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY
:EXTERNAL
[1] UIOP/BACKWARD-DRIVER(17): [1] UIOP/BACKWARD-DRIVER(17): (unexport
(find-symbol LOAD-ASDF-DEBUG-UTILITY :uiop/driver) :uiop/driver)
Error loading /home/tunes/cl/asdf/build/asdf.lisp at line 5302 (offset 252109)
#THREAD interpreter {7E6AB533}: Debugger invoked on condition of
type PACKAGE-ERROR
   The symbol UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY is not accessible
in package UIOP/DRIVER
Restarts:
   0: ABORT Return to debug level 1.


Hmmm.  How did you get into the state where 
UIOP/UTILITY::LOAD-ASDF-DEBUG-UTILITY is :INTERNAL? asdf.lisp:1017 
explicitly exports this symbol from what I see.


From ABCL trunk I only see it as an :EXTERNAL symbol:

CL-USER (lisp-implementation-version)
1.2.0-dev
Java_HotSpot(TM)_64-Bit_Server_VM-Oracle_Corporation-1.7.0_17-b02
x86_64-Mac_OS_X-10.8.3
CL-USER (asdf:asdf-version)
2.32
CL-USER (find-symbol LOAD-ASDF-DEBUG-UTILITY :uiop/driver)
UIOP/UTILITY:LOAD-ASDF-DEBUG-UTILITY
:EXTERNAL
CL-USER (find-symbol LOAD-ASDF-DEBUG-UTILITY :uiop/utility)
UIOP/UTILITY:LOAD-ASDF-DEBUG-UTILITY
:EXTERNAL
CL-USER


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] Pre ASDF-2.27 failing to compile on ABCL

[Mark Evenson]

On 1/11/13 5:53 PM, Faré wrote:

Dear Lisp hackers,

I've completed yet another major pass of refactoring of ASDF.


[…]

Coolness!  Keep it coming!

Failing to compiled the concatenated fasl from ASDF master tip (local :

   UNDEFINED-FUNCTION: ;COMMON-LISP:CELL-ERROR-NAME triggers 
autoloading of org.armedbear.lisp.cell_error_name ...

; Autoloaded org.armedbear.lisp.cell_error_name (0.0050 seconds)
   The function ASDF/PACKAGE::WHEN-FISHY is undefined.


Have I missed some packaging in the implemenation (abcl-1.2.0-dev)?

ABCL trunk currently has some problems with autoloading (some) 
COMMON-LISP functions via SETF expanders, which I am in the process of 
fixing.


; SLIME 2012-11-23
CL-USER (lisp-implementation-version)
1.2.0-dev
Java_HotSpot(TM)_64-Bit_Server_VM-Sun_Microsystems_Inc.-1.6.0_37-b06
amd64-SunOS-5.11

Thanks,
Mark even...@panix.com



___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ECL tests failing on Mac OS X Lion [Re: asdf 2.23 released]

[Mark Evenson]

On 7/17/12 7:49 PM, Robert Goldman wrote:
[…]


OK, I checked.  This /is/ a bug with MacPorts, and they are working on it.


I have a MacPOrts commit bit, so Lisp implementers should feel free to 
bug me with advice and/or patches…


I'll take a stab at ecl-12.7.1 for MacPorts in the near future to see if 
that improves things.





___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] asdf-bundle

[Mark Evenson]

On 5/3/12 17:55 , Faré wrote:

asdf-bundle is now working (hopefully) on SBCL and CCL as well as ECL,
by concatenating fasls. It won't work for systems where CFFI creates .so.

Try it!
(require :asdf)
(require :asdf-bundle)
(asdf:oos 'asdf::load-fasl-op :your-favorite-system)

It creates a your-favorite-system.system.fasl in the fasl cache.


[…]


Others: you're welcome to contribute support for your favorite implementations.


ABCL has [ASDF-JAR][1] in ABCL-CONTRIB, which will recursively package 
all the necessary components to run an ASDF system into a jar archive.


We'll look at supporting the ASDF-BUNDLE interface when I get a chance, 
although we are doing somewhat different things here.


[1]: 
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/contrib/asdf-jar/README.markdown


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] asdf and windows vs unix on abcl

[Mark Evenson]

On Oct 20, 2011, at 18:00 , Faré wrote:

 ABCL is special in that it can run on either Windows or Unix.
 
 The problem is that ASDF 2.017 determines which of Windows or Unix at
 compile-time, which causes confusion when ABCL's builtin ASDF is
 compiled on Windows and used on Unix or vice-versa. And I can tell
 your binary was definitely produced on Windows.
 
 (I found this issue while testing cl-launch. I also noticed that you
 fixed your ext:*command-line-argument-list* albeit in a necessarily
 incompatible way.)
 
 I suppose the solution is that the test of unixness vs windowsness
 should happen at runtime. Crazy.

We place :WINDOWS and :UNIX in *FEATURES* depending on where we end up 
executing, so forms like 

(when (find :unix *features*)
   (run-process ls))

should do the right sorts of things.  If you have other suggestions about how 
to manage this, please make them.

Mark even...@panix.com


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] Disabling built-in output translation

[Mark Evenson]

On 5/17/11 15:28 , Faré wrote:
[…]

Instead of disabling translations, you ought to add some specific translation
(more specific than the default jar translation) that will tell ASDF that
indeed for this or that jar, the fasls are stored in the jar,
with an entry like:
(:output-translations
   (#pjar:file:/path/to/my.jar!/**/*.*)
   :inherit-output-translations)

Hope this helps.


I think you meant

(:output-translation
(#pjar:file:/path/to/my.jar!/**/*.*)
:inherit-configuration)

which does indeed help.


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

On 6/10/11 00:57 , Faré wrote:

On 6/10/11 00:57 , Faré wrote:
 Please try 2.016.1 and see if it satisfies you.

Yes, that works for [ABCL and I have pushed][1].

[1]: http://trac.common-lisp.net/armedbear/changeset/13319

 Pathnames are a big FAIL of CL.

So thinks Stellian as well, so IOLIB tries to get away from.  I admire 
CL Pathnames given the different types of abstractions that early 1980s 
filesystems had to normalize, but certainly that is scant consolation 
for dealing with them in the 2010s.  Certainly it seems that the 
behavior of COMPILE-FILE-PATHNAME not matching expectations of 
MERGE-PATHNAMES is a CL FAIL.  With a little more specification on how 
to handle the implementation-specific parts, I think things would be 
better off.   At this point, the code of ASDF2 for handling these 
differences is probably the most advanced--and widely tested--stab at 
such abstraction.  It some point in the future one might consider 
breaking it out into a separate library, as it would undoubtedly have 
wide utility outside of ASDF.


Thanks again for the help,
Mark

--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

On 6/10/11 00:57 , Faré wrote:

On 6/10/11 00:57 , Faré wrote:
 Please try 2.016.1 and see if it satisfies you.

Yes, that works for [ABCL and I have pushed][1].

[1]: http://trac.common-lisp.net/armedbear/changeset/13319

 Pathnames are a big FAIL of CL.

So thinks Stellian as well, so IOLIB tries to get away from.  I admire 
CL Pathnames given the different types of abstractions that early 1980s 
filesystems had to normalize, but certainly that is scant consolation 
for dealing with them in the 2010s.  Certainly it seems that the 
behavior of COMPILE-FILE-PATHNAME not matching expectations of 
MERGE-PATHNAMES is a CL FAIL.  With a little more specification on how 
to handle the implementation-specific parts, I think things would be 
better off.   At this point, the code of ASDF2 for handling these 
differences is probably the most advanced--and widely tested--stab at 
such abstraction.  It some point in the future one might consider 
breaking it out into a separate library, as it would undoubtedly have 
wide utility outside of ASDF.


Thanks again for the help,
Mark

--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

Stellian's [normalization to ANSI semantics of 
COMPILE-FILE-PATHNAME*][1] has the unfortunate effect of breaking ABCL's 
translation of systems packaged in jar files.


[1]: 
http://common-lisp.net/gitweb?p=projects/asdf/asdf.git;a=commit;h=ed4cd32932e937724b0b28d1c20ed6abe5d58dc0;js=1


I have applied the subsequent patch to the asdf-2.016 packaged with ABCL 
to restore our needed functionality but am  unhappy at the necessity of 
introducing a runtime implementation conditional, as sure a sign as 
anything that I'm doing it wrong,  and can't really request that ASDF 
patch itself in this manner.  Maybe if I explain things as I have 
analyzed them, someone else can suggest a better way forward.


Recall that ABCL uses the presence of a cons in the DEVICE component to 
indicate that a Pathname represents an entry in a jar archive.  A 
pathname located within a jar archive is not a writable location, so we 
introduced ABCL specific ASDF output translation rules which translate 
such a source location to an appropriately differentiated output 
location in the ASDF cache hierarchy.  This results in ASDF making the 
following kind of call in preparation of the output file location it 
will pass to COMPILE-FILE:


  (compile-file-pathname* path with a DEVICE :output-file path 
without a DEVICE)


Stellian's patch correctly fixes behavior of COMPILE-FILE-PATHNAME* to 
use pathname components in input-file if none are specified in 
output-file.  For ABCL systems in a jar, this results in returning an 
output location within the jar file, which being a non-valid output 
location because it is not writable causes ASDF to error out in 
compiling the system.


If there were some mechanism to indicate that the invocation of 
APPLY-OUTPUT-TRANSLATIONS had really finished so that no further 
processing was necessary, we could possibly plausibly short-circuit this 
call to COMPILE-FILE-PATHAME*.  I suppose my recursive invocation of 
APPLY-OUTPUT-TRANSLATIONS in the ABCL-specific TRANSLATE-JAR-PATHNAME 
adds the missing TYPE to the pathname which other code paths don't have 
at this point, which is why the call to COMPILE-FILE-PATHNAME* is 
necessary.


ANSI allows the addition of implementation specific arguments to 
COMPILE-FILE-PATHNAME, so we could maybe add a :strip-device for ABCL 
but this seems even less elegant.


Any suggestions?

@@ -3523,7 +3525,16 @@
--- a/asdf.lisp Thu Jun 09 06:03:20 2011 +
+++ b/asdf.lisp Thu Jun 09 15:00:45 2011 +0200
 (defun* compile-file-pathname* (input-file rest keys key output-file 
allow-other-keys)

   (if (absolute-pathname-p output-file)
-  (apply 'compile-file-pathname (lispize-pathname input-file) keys)
+  ;;; If the default ABCL rules for translating from a jar path to
+  ;;; a non-jar path have been affected, no further computation of
+  ;;; the output location is necessary.
+  (if (and (find :abcl *features*)
+   (pathname-device input-file) ; input-file is in a jar
+   (not (pathname-device output-file)) ; output-file is not 
in a jar

+   (equal (pathname-type input-file) lisp)
+   (equal (pathname-type output-file) abcl))
+  output-file
+  (apply 'compile-file-pathname (lispize-pathname input-file) 
keys))

   (apply-output-translations
(apply 'compile-file-pathname
   (truenamize (lispize-pathname input-file))

--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

On 6/9/11 16:37 , Faré wrote:
[…]


Can you write a simple test to be added to test/asdf-pathnames.script,


I'll work on that.  Can I just create a new top-level form or do you 
prefer I wedge it into TEST-COMPONENT-PATHNAMES with an #+abcl/#-abcl 
conditional?



and/or send me an example such that I can reproduce the bug at home?
Maybe just a trace output of the relevant functions:
(trace translate-pathname* and compile-file-pathname* apply-output-translations)
(add any function there necessary to show where it breaks).


A trace for the JSS system packaged in abcl-contrib.jar:

CL-USER (asdf:compile-system :jss :force t)
  0: (ASDF:APPLY-OUTPUT-TRANSLATIONS 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl)
1: (ASDF::TRANSLATE-PATHNAME* 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl 
#Pjar:file:/**/*.jar!/**/*.* #FUNCTION ASDF::TRANSLATE-JAR-PATHNAME 
{66EF7D74} NIL #Pjar:file:/**/*.jar!/**/*.*)
  2: (ASDF:APPLY-OUTPUT-TRANSLATIONS 
#P/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl)
3: (ASDF::TRANSLATE-PATHNAME* 
#P/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl 
#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
NIL #P/___jar___file___root___/**/*.*)
3: TRANSLATE-PATHNAME* returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  2: APPLY-OUTPUT-TRANSLATIONS returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
1: TRANSLATE-PATHNAME* returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  0: APPLY-OUTPUT-TRANSLATIONS returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  0: (ASDF:APPLY-OUTPUT-TRANSLATIONS 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl)
1: (ASDF::TRANSLATE-PATHNAME* 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.abcl 
#Pjar:file:/**/*.jar!/**/*.* #FUNCTION ASDF::TRANSLATE-JAR-PATHNAME 
{66EF7D74} NIL #Pjar:file:/**/*.jar!/**/*.*)
  2: (ASDF:APPLY-OUTPUT-TRANSLATIONS 
#P/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl)
3: (ASDF::TRANSLATE-PATHNAME* 
#P/___jar___file___root___/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl 
#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
NIL #P/___jar___file___root___/**/*.*)
3: TRANSLATE-PATHNAME* returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  2: APPLY-OUTPUT-TRANSLATIONS returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
1: TRANSLATE-PATHNAME* returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  0: APPLY-OUTPUT-TRANSLATIONS returned 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl
  0: (ASDF:COMPILE-FILE-PATHNAME* 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/jss/packages.lisp 
:OUTPUT-FILE 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl 
:OUTPUT-FILE 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl)
  0: COMPILE-FILE-PATHNAME* returned 
#Pjar:file:/Users/evenson/work/abcl/dist/abcl-contrib.jar!/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/Users/evenson/work/abcl/dist/abcl-contrib.jar/jss/packages.abcl


And the subsequent COMPILE-FILE fails as it can't write to the jar file 
returned by COMPILE-FILE-PATHNAME*.



Note that when I (describe #pjar:file:///foo/bar.jar!/baz/quux.lisp)
I get:
#Pjar:file:/foo/bar.jar!/baz/quux.lisp is an object of type
EXTENSIONS:JAR-PATHNAME:
   HOST NIL
   DEVICE   (#P/foo/bar.jar)
   DIRECTORY(:ABSOLUTE baz)
   NAME quux
   TYPE lisp
   VERSION  NIL

I believe that a pathname instead of a namestring
makes that non-conformant. Sigh.


You mean the list with pathname in the device field?  Yes, ABCL is 
non-conformant in this manner in order to support addressing entries in 
jar files.



Note that if requires you could keep a cache of the pathname
in addition (or underlying store?) to any namestring in the 

Re: [asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

On 6/9/11 6:44 PM, Faré wrote:

Note that when I (describe #pjar:file:///foo/bar.jar!/baz/quux.lisp)
I get:
#Pjar:file:/foo/bar.jar!/baz/quux.lisp is an object of type
EXTENSIONS:JAR-PATHNAME:
   HOST NIL
   DEVICE   (#P/foo/bar.jar)
   DIRECTORY(:ABSOLUTE baz)
   NAME quux
   TYPE lisp
   VERSION  NIL

I believe that a pathname instead of a namestring
makes that non-conformant. Sigh.


You mean the list with pathname in the device field?  Yes, ABCL is
non-conformant in this manner in order to support addressing entries in jar
files.


Note that if requires you could keep a cache of the pathname
in addition (or underlying store?) to any namestring in the device.


On re-reading CLHS, I retract my admission that ABCL is non-conformant. 
 From the CLHS glossary a valid pathname device [is a]  a string, nil, 
:unspecific, or some other object defined by the implementation to be a 
valid pathname device. ABCL is using the some other object ability 
here. The might description in CLHS 19.2.2.4.2 offers guidance as to 
the allowed types, but doesn't put a firm requirement.  Or is this 
understanding incorrect somehow?  Note I am trying to use conformant 
here in the manner prescribed by CLHS 1.5, not in the informal sense of 
what a friendly CL implementation should do.


The two main design reasons for allowing a list of pathnames in our 
device component:  they allow pathnames representing URI locations so 
our jars may be loaded over the network, and they allow us to reference 
jar archives stored within archives which is important for ABCL fasls. 
We could plausibly convert the device pathname to namestrings, as every 
URI has an suitably encoded string representation, but without the 
ability to use a cons in the device slot we would lose the ability to 
refer to a nested entry.



You could be conformant if the device component were (/foo/bar.jar)
instead of (#P/foo/bar.jar).


But have the device component be a cons violates the first sentence of 
CLHS 19.2.2.4.2 that you are presumably referencing.  Or how do I 
misunderstand?


[…]


At this point can you trace COMPILE-FILE-PATHNAME, too?
Since the output-file satisfies absolute-pathname-p,
the branch taken by COMPILE-FILE-PATHNAME* should be to just call
COMPILE-FILE-PATHNAME, at which point I expect it to *not*
merge the device of the source file.
OH I GET IT! (I think.) The bug is that
(1) COMPILE-FILE-PATHNAME uses MERGE-PATHNAMES
   instead of ASDF:MERGE-PATHNAMES* (as you would, I assume).
(2) Your absolute pathnames have a host and device of NIL,
   instead of a non-nil host device of say FILE and LOCALHOST
   or say  and .


Yes, it is our use of NIL here that is causing problems.

What I was going to suggest is that the TRANSLATE-JAR-PATHNAME function 
return a pathname with an :UNSPECIFIC device component.  This should 
allow the COMPILE-FILE-PATHNAME merge algorithm to do what we want, but 
I see you have incorporated this into your next message, which I will 
reply to.



Sigh. I would have suggested fixing things by having
ABCL's pathname scheme not use NIL for HOST and DEVICE in such cases,
but who knows what other interesting (positive or negative) consequences
that might or might not have. It could be a big win or a big headache.


I think we should probably insert :UNSPECIFIC for more cases than where 
we do from MAKE-PATHNAME.


[…]


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF 2.016 breaks ABCL translations for jar files

[Mark Evenson]

On 6/9/11 7:06 PM, Faré wrote:

Thanks for thinking with me on this.


OK. I believe the following definition might make
each of Stelian, you and I happy. Can you try it?

(defun* compile-file-pathname*
 (input-filerest keyskey output-fileallow-other-keys)
   (if (absolute-pathname-p output-file)
   (apply 'compile-file-pathname
  (make-pathname :host nil :device nil
 :defaults (lispize-pathname input-file))
  keys)
   (apply-output-translations
(apply 'compile-file-pathname
   (truenamize (lispize-pathname input-file))
   keys


Yes, this works fine with ABCL.

But for implementations that need a device (like Windows using device 
for the drive letter), won't this result in stripping the device out of 
all output-file locations?


As I mentioned in the previous message, I think the better fix is to 
have TRANSLATE-JAR-PATHNAME do something like the following (untested):


index a8339f7..0395f56 100755
--- a/asdf.lisp
+++ b/asdf.lisp
@@ -3569,12 +3569,14 @@ effectively disabling the output translation 
facility.

  (root (format nil /___jar___file___root___/~@[~A/~]
(and (find :windows *features*)
 (pathname-device p)
-(apply-output-translations
- (merge-pathnames*
-  (relativize-pathname-directory source)
-  (merge-pathnames*
-   (relativize-pathname-directory (ensure-directory-pathname p))
-   root)
+(make-pathname :defaults
+   (apply-output-translations
+(merge-pathnames*
+ (relativize-pathname-directory source)
+ (merge-pathnames*
+  (relativize-pathname-directory 
(ensure-directory-pathname p))

+  root)))
+   :device :unspecific)))

  -
  Compatibility mode for ASDF-Binary-Locations


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] Disabling built-in output translation

[Mark Evenson]

ASDF contains some specialized code for ABCL that enable the translation 
of ASDF systems whose source is packaged into jar files to locate the 
corresponding object files in ASDF's user cache.  In trying to figure 
out the correct mechanism to allow for binary ASDF distribution under 
ABCL that would include both the source and the fasls in the same jar 
file, I need to disable the jar file translation mechanism.


I expected that invoking DISABLE-OUTPUT-TRANSLATIONS would clear this 
built-in translation mechanism, but it doesn't seem to do this.   Only 
invoking CLEAR-OUTPUT-TRANSLATIONS seems to actually change the output 
translation map.  To wit:


ASDF (disable-output-translations)
((T T) 
(#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
T) (#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*) 
(#Pjar:file:/**/*.jar!/**/*.* #FUNCTION TRANSLATE-JAR-PATHNAME 
{771931F8}) (T 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*))

ASDF (output-translations)
((#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
T) (#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*) 
(#Pjar:file:/**/*.jar!/**/*.* #FUNCTION TRANSLATE-JAR-PATHNAME 
{771931F8}) (T T) (T 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*))

ASDF (disable-output-translations)
((T T) 
(#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
T) (#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*) 
(#Pjar:file:/**/*.jar!/**/*.* #FUNCTION TRANSLATE-JAR-PATHNAME 
{771931F8}) (T 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*))

ASDF (output-translations)
((#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.* 
T) (#P/___jar___file___root___/**/*.* 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*) 
(#Pjar:file:/**/*.jar!/**/*.* #FUNCTION TRANSLATE-JAR-PATHNAME 
{771931F8}) (T T) (T 
#P/Users/evenson/.cache/common-lisp/abcl-0.26.0-dev-fasl37-macosx-java/**/*.*))

ASDF (clear-output-translations)
; No value
ASDF (output-translations)
NIL
ASDF

Should DISABLE-OUTPUT-TRANSLATIONS actually push that (T T) list into 
the beginning of that (OUTPUT-TRANSLATION) list or does the API expect 
that the user use it something like


ASDF (setf (output-translations) (disable-output-translations))

Should ASDF under ABCL strip out the jar translation mechanism as well, 
or should the ASDF API expect that the user would actual issue a 
(CLEAR-OUTPUT-TRANSLATIONS)?  I confess that I am a bit confused when 
one would ever use DISABLE-OUTPUT-TRANSLATIONS as opposed to 
CLEAR-OUTPUT-TRANSLATIONS.


Once I understand the ASDF API here a bit better, I will work on the 
necessary ABCL specific ASDF patches if necessary.


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] ABCL packaging ASDF system in jars (was Re: Disabling built-in output translation)

[Mark Evenson]

On 5/17/11 3:28 PM, Faré wrote:
[…]

Hope this helps.


I'm gonna need a bit more time to chew over the ASDF code, but thanks 
for the general direction.  I must confess that in spite of the 
reasonable looking documentation and having contributed the function 
translation code to ASDF, its whole output translation API has never 
really gelled in my understanding as a totality.  Maybe I can contribute 
some examples to the texinfo file when the fuzz in my understanding 
resolves a bit.



BTW, for cl-launch and XCVB, I indeed am looking for a way to create
bundles from compiled stuff. How do I create a jar from ABCL and a set
of lisp files, precompiled or not?


A reasonable packaging mechanism for ABCL plus compiled stuff composed 
of ASDF system definitions that works well in a Java ecosystem is 
precisely the itch I'm in the process of scratching for ABCL right now. 
 The base unit of packaging in Java is the jar file, which is nothing 
more than a ZIP archive with (possibly) some Java-the-language specific 
metadata.  Currently, one can package an ASDF definition in a jar file, 
push the location of the asd file into ASDF:*CENTRAL-REGISTRY* using the 
[ABCL JAR-PATHNAME conventions][1] for which a subsequent 
ASDF:LOAD-SYSTEM will do the right thing.


[1]: 
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/doc/design/pathnames/jar-pathnames.markdown


For instance, if you were to package up cl-ppcre installed via 
[quicklisp][10^9monkey-wants-this] in a jar file via


  unix$ cd ~/quicklisp/dists/quicklisp/software  jar cfv 
/tmp/cl-ppcre-2.0.3.jar cl-ppcre-2.0.3


[10^9monkey-wants-this]: http://www.quicklisp.org

one could subsequently enable an ASDF controlled load of this system via

  CL-USER (push jar:file:/tmp/cl-ppcre-2.0.3.jar!/cl-ppcre-2.0.3/ 
asdf:*central-registry)


When loaded, ASDF will compile the system to the user cache.

Some problems here that I'm working through:

1)  You can't immediately load FASL out of the jar.  In my current 
hackish way—i.e. without comprehending your advice yet—one has first 
ASDF compile the system with output translations disabled, and then


 (defmethod asdf:perform ((o asdf:compile-op) (c asdf:cl-source-file)))
 (setf (asdf::output-translations) '((t t

in the target JVM to load the ABCL FASLS from the jar.

2) One has to specify the absolute path on the local filesystem (or 
potentially via a URI) for the jar, which makes things a bit fragile in 
the typical Java ecosystem usage which shuffles jars around like win32 
DLLs (or, to be fair, Unix dynamic libraries) relying merely on the 
filename to keep things straight.  My current insight into a way around 
would be to define another PATHNAME extension in ABCL for CLASSPATH 
entries, i.e. classpath:cl-ppcre-2.0.3.jar!/cl-ppcre-2.0.3/ would 
refer to the named jar in the JVM CLASSPATH if it exists.


3)  ABCL [has a bug][#149] that currently prevents ASDF systems located 
in the top-level entry of a JAR from being accessed.


[#149]: http://trac.common-lisp.net/armedbear/ticket/149

4)  The extremely nice use of [JSS][jss] and [ABCLD's slight 
modification][abcld] of ASDF to also refer to jar files to dynamically 
load into the JVM probably needs to be rewritten, otherwise we run into 
the situation whereby we have jars (i.e. the packaged Java code) within 
the ASDF packaged jar which A) needs changes within ABCL to completely 
work and B) would be rather inefficient in that the naive implementation 
 each request for a new entry in the JAR within a JAR would require a 
complete reseek through the enclosed ZIP file.


[jss]: http://code.google.com/p/lsw2/source/browse/#svn%2Ftrunk%2Fjss
[abcld]: 
http://code.google.com/p/abcl-dynamic-install/source/browse?repo=abcld


5)  A fear of mine: if I enable all this, I presume that people would 
start going around creating 'abcl.jar' files with different inclusions 
of different ASDF packagings.  Without a real smart dynamic 
introspection system that essentially solves the problems in JVM's 
classpath we would end up with, at the minimum, a rather hostile 
situation for the end user.  Put 'abcl.jar' in your classpath.  I 
did, and it still didn't load Maxima.  Well, what's the checksum of 
your abcl.jar? c48d359a23ee  Oh, you need 846f78c279cb.  Ideally, I 
would like to come up with a mechanism that would require that 
'abcl.jar' come from official ABCL packaging, but would somehow be 
able to introspect the JVM classpath to include ASDF definitions.


Comments solicited.

--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF 2.003

[Mark Evenson]

On 6/24/10 11:51 PM, Faré wrote:

Can ABCL upgrade its ASDF to 2.003 or whichever is latest in the
release branch (if possible NOT the master branch)?

That would be great. It should fix several bugs that may have hit ABCL
users since then. Xach at least reported that upgrading ASDF on ABCL
made his software work.

As usual, please test on your specific implementation with a few
systems before you commit, and please report any bug.


Committed in [ABCL svn r12765][1].

Attached are the ABCL specific modifications to asdf-2.003, namely 
differentiation of the output locations by ABCL FASL version and the 
Java platform we're running under.


[1]: http://trac.common-lisp.net/armedbear/changeset/12765


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.
--- /Users/evenson/work/asdf/asdf.lisp	2010-06-25 09:12:46.0 +0200
+++ src/org/armedbear/lisp/asdf.lisp	2010-06-25 09:39:38.0 +0200
@@ -2392,7 +2392,8 @@
 (defparameter *architecture-features*
   '((:x86-64 :amd64 :x86_64 :x8664-target)
 (:x86 :i686 :i586 :pentium3 :i486 :i386 :pc386 :iapx386 :x8632-target :pentium4)
-:hppa64 :hppa :ppc64 (:ppc32 :ppc :powerpc) :sparc64 :sparc))
+:hppa64 :hppa :ppc64 (:ppc32 :ppc :powerpc) :sparc64 :sparc
+:java-1.4 :java-1.5 :java-1.6 :java-1.7))
 
 (defun lisp-version-string ()
   (let ((s (lisp-implementation-version)))
@@ -2424,7 +2425,8 @@
 #+lispworks (format nil ~...@[~a~] s
 (when (member :lispworks-64bit *features*) -64bit))
 ;; #+sbcl (format nil ~a-fasl~d s sb-fasl:+fasl-file-version+) ; fasl-f-v is redundant
-#+(or armedbear cormanlisp mcl sbcl scl) s
+#+armedbear (format nil ~a-fasl~a s system::*fasl-version*)
+#+(or cormanlisp mcl sbcl scl) s
 #-(or allegro armedbear clisp clozure cmu cormanlisp digitool
   ecl gcl lispworks mcl sbcl scl) s))
 
___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] translate-jar-pathname : The value NIL is not of type (OR PATHNAME STRING FILE-STREAM).

[Mark Evenson]

On 4/6/10 1:24 AM, james anderson wrote:

given:

[1] CL-USER(69): (lisp-implementation-type)
Armed Bear Common Lisp
[2] CL-USER(70): (lisp-implementation-version)
0.19.1
[3] CL-USER(73): asdf::*asdf-version*
1.666

asdf fails to compile alexandria.
translate-jar-pathname applies namestring to the null device
component of a pathname and fails.



Acknowledged and reproduced as a bug in somewhere in ABCL.

ABCL built with the attached patch does not exhibit this error.  The 
patch represents the current state branch in progress that fixes a 
number of things dealing with Pathnames in addition to introducing 
support for URLs as Pathnames.  It's not immediately clear how to 
separate out the fix for the ASDF2 problem, so I'm planning on spending 
what time I've got pushing this patch towards a point to push to trunk. 
 If someone has time to help diagnose the specific part that fixes 
ASDF2, I'd be happy to sponsor that commit to ABCL trunk.


Once I graft my patch to the ABCL trunk, I will tackle adding ASDF2 into 
the ABCL distribution.


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


abcl-url-pathname-20100410a.patch.gz
Description: GNU Zip compressed data
___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] Patches for ASDF2 for translation function

[Mark Evenson]

On 4/5/10 8:59 PM, Faré wrote:
 Dear Mark,

 I merged your patch in, after rearranging it a little bit. Tell me how
 it's working for you...


Seems to work just fine in my tests.  Thanks for including this!


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On 3/29/10 8:25 AM, Faré wrote:
 Dear Mark,

 Fare's suggestion that I use an output translation based on the jar
 pathname doesn't quite work, because in our current implementation,
 the pathname of the jar is stored in DEVICE, separate from the rest
 of the jar pathname. I extended PATHNAME-MATCH-P to match jars
 correctly, but I don't see a possible extension of TRANSLATE-PATHNAME.

 Couldn't you extend the ABCL pathname matching algorithm to allow for
 wildcards and all in the device component?

Wildcards do work in DEVICE but you can't currently use unused results 
of the wildcard matches on the DEVICE components to carry over to the 
translation of the DIRECTORY component.  I suppose that's what you were 
suggesting anyways.  There isn't an obviously easy way to implement this 
in ABCL due to the structure of our code and I need to think through the 
various permutations, but that doesn't mean that its implementation 
isn't a win for the end user.  I'll see what I can do.

 So I would ask the ASDF developers to consider extending the
 output translation DSL to allow something like

   (initialize-output-translations
 '(:output-translations :ignore-inherited-configuration
(#pjar:file:/**/*.jar!/**/*.* :function 
 SYS::ASDF-JAR-OUTPUT-TRANSLATE)))

 I suggest that either you make (:function foo) a valid second value
 for an output-translations list, or you make said list accept akey
 function argument in addition to its two current positional arguments.
 But having weird non-standard vararg convention for such lists is
 probably a bad API. The default function would be TRANSLATE-PATHNAME
 and the calling convention would be the same.

I'd favor the first proposal, as making :FUNCTION a real key argument 
would imply that the second positional parameter is optional, whereas it 
is actually being replaced by the (:FUNCTION ARG) parameter.  When I get 
time to revisit the patch, I'll implement your first proposal.

[…]

Thanks,
Mark

-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On Mar 29, 2010, at 10:39 AM, Faré wrote:
[…]

 So I would ask the ASDF developers to consider extending the
 output translation DSL to allow something like
 
  (initialize-output-translations
'(:output-translations :ignore-inherited-configuration
   (#pjar:file:/**/*.jar!/**/*.* :function
 SYS::ASDF-JAR-OUTPUT-TRANSLATE)))
 
 I suggest that either you make (:function foo) a valid second value
 for an output-translations list, or you make said list accept akey
 function argument in addition to its two current positional arguments.
 But having weird non-standard vararg convention for such lists is
 probably a bad API. The default function would be TRANSLATE-PATHNAME
 and the calling convention would be the same.
 
 I'd favor the first proposal, as making :FUNCTION a real key argument would
 imply that the second positional parameter is optional, whereas it is
 actually being replaced by the (:FUNCTION ARG) parameter.  When I get time
 to revisit the patch, I'll implement your first proposal.
 
 I'd favor either a keyword argument or :function as a magic marker to
 be inserted in FIRST position, followed by a function designator (and
 additionally, parameters to curry?). Unless of course, you
 symmetrically accept a FUNCTION in first position to denote a
 recognize if this matching rule applies.


I would think that we want to preserve the current ability for the user to 
vistually scan down the list of translations to determine what should match, so 
I would favor going for your second proposal.  It doesn't make sense that there 
be a match predicate paired with a translation pathname right?  So the two new 
possibilities for output translation lists would be:

  (/**/foo/*.* (:function FOO-OUTPUT-TRANSLATION))
  ((:function BAR-PATHNAME-P) (:function BAR-OUTPUT-TRANSLATION))


--

A screaming comes across the sky.  It has happened before, but there is 
nothing to compare to it now.






___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ABCL testing issue

[Mark Evenson]

On 3/17/10 4:15 PM, james anderson wrote:
[…]

 The specific bug you are encountering involves abcl-0.18.1 not being
 able to handle the renaming of a FASL, as ASDF2 compiles to
 asdf-tmp.XXX and then renames to asdf-LISP_IMPLEMENTATION.XXX.
 Rather embaressing for us, really, so [we fixed it fast][1]

 i'll wait for that. please advise when it happens.

[ABCL-0.19.1 has been released][1] (there was no public announcement of 
ABCL-0.19.0).

[1]: http://common-lisp.net/project/armedbear/


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

 != Keyword.IO)
 error(new LispError(Direction must be :INPUT, :OUTPUT, or 
:IO.));
-try {
-return new FileStream(pathname, namestring.getStringValue(),
-  elementType, direction, ifExists,
-  externalFormat);
-}
-catch (FileNotFoundException e) {
-return NIL;
-}
-catch (IOException e) {
-return error(new StreamError(null, e));
+
+if (pathname.isJar())  {
+if (direction != Keyword.INPUT) {
+error(new FileError(Only direction :INPUT is supported 
for jar files., pathname));
+}
+try { 
+return new JarStream(pathname, namestring.getStringValue(),
+ elementType, direction, ifExists,
+ externalFormat);
+} catch (IOException e) {
+return error(new StreamError(null, e));
+}
+} else {
+try {
+return new FileStream(pathname, 
namestring.getStringValue(),
+  elementType, direction, ifExists,
+  externalFormat);
+}
+catch (FileNotFoundException e) {
+return NIL;
+}
+catch (IOException e) {
+return error(new StreamError(null, e));
+}
 }
 }
 };
diff -r a9813719dfba src/org/armedbear/lisp/JarStream.java
--- /dev/null   Thu Jan 01 00:00:00 1970 +
+++ b/src/org/armedbear/lisp/JarStream.java Thu Mar 25 06:20:00 2010 +0100
@@ -0,0 +1,150 @@
+/*
+ * JarStream.java
+ *
+ * Copyright (C) 2010 Mark Evenson
+ * $Id: FileStream.java 12422 2010-02-06 10:52:32Z mevenson $
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ *
+ * As a special exception, the copyright holders of this library give you
+ * permission to link this library with independent modules to produce an
+ * executable, regardless of the license terms of these independent
+ * modules, and to copy and distribute the resulting executable under
+ * terms of your choice, provided that you also meet, for each linked
+ * independent module, the terms and conditions of the license of that
+ * module.  An independent module is a module which is not derived from
+ * or based on this library.  If you modify this library, you may extend
+ * this exception to your version of the library, but you are not
+ * obligated to do so.  If you do not wish to do so, delete this
+ * exception statement from your version.
+ */
+
+package org.armedbear.lisp;
+
+import static org.armedbear.lisp.Lisp.*;
+
+import java.io.File;
+import java.io.InputStream;
+import java.io.Reader;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.io.BufferedReader;
+
+/** 
+ * Stream interface for an entry in a jar pathname.
+ * 
+ * This only supports reading from the stream.
+ */
+public final class JarStream extends Stream
+{
+private final Pathname pathname;
+private final InputStream input;
+private final Reader reader;
+private final int bytesPerUnit;
+
+public JarStream(Pathname pathname, String namestring,
+  LispObject elementType, LispObject direction,
+  LispObject ifExists, LispObject format)
+throws IOException
+{
+super(Symbol.JAR_STREAM);
+Debug.assertTrue(direction == Keyword.INPUT);
+Debug.assertTrue(pathname.name != NIL);
+isInputStream = true;
+
+super.setExternalFormat(format);
+
+this.pathname = pathname;
+this.elementType = elementType;
+
+this.input = pathname.getInputStream();
+if (elementType == Symbol.CHARACTER || elementType == 
Symbol.BASE_CHAR) {
+isCharacterStream = true;
+bytesPerUnit = 1;
+InputStreamReader isr = new InputStreamReader(input);
+this.reader = (Reader) new BufferedReader(isr);
+initAsCharacterInputStream(this.reader

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On 3/25/10 7:34 AM, Faré wrote:
[…]

 [1]:
 http://trac.common-lisp.net/armedbear/browser/trunk/abcl/src/org/armedbear/lisp/asdf-abcl.lisp

 Do you want me to merge that into upstream ASDF with #+abcl conditions?

Not yet: it doesn't make sense until I figure out what works with ASDF2 
(that patch only works with ASDF 1.3).

 Note: If ABCL supports matching and translating such paths,
 there could be ASDF-Output-Translation rules by default with ABCL
 in the idea of
 (#p/**/*.jar/**/*.* (:home #p**/*.jar/**/*.*)

An avenue that I will explore.  Thanks.

 It would be perhaps cleaner to have the binary locations machinery of ASDF2
 react to not being able to write to the Pathname derived from the location
 of the .asd file in an extensible manner. This might be useful for users
 of Lisps other than ABCL who don't have permission to write to the system
 ASDF location for instance.  My current problem is that the :BEFORE for
 PERFORM specialized on COMPILE-OP SOURCE-FILE contains an
 ENSURE-DIRECTORIES-EXIST which by default is derived from the .asd
 Pathname.  Is there a way to per-system customization of the output
 location?

 How do you propose to do that?

Not sure, which is why I was floating the idea for comments, but if I 
can run a user-defined function after the .asd has been parsed to 
programatically implement the following, that might work.  I need to 
read up on what is present in the currently defined mechanism.

 Thinking quickly for an ASDF2 algorithm:

 1.  Is the directory containing the .asd writable?  If so, continue
 normally.

[…]

 Juanjo of ECL was thinking of a different delivery mechanism, where
 compiling a ASDF system would yield another ASDF system of the same
 interchangeable name, but the content of which would be just one or
 several binary objects to load.

 Perhaps that's what you really want to be using?

That's worth investigating as well.

Thanks for the input.


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] Patch for ASDF2 to use arbitrary output translation function

[Mark Evenson]

On 3/25/10 2:39 PM, Mark Evenson wrote:
[…]


I'm working on an implementation of this on your git version as a
proof of concept, but haven't gotten much further than getting your
output configuration to accept the new syntax.  I assume that
actually applying the function will be easy, but I am not sure.


[…]

Attached please find a patch to implement the use of an arbitrary output 
translation function in ASDF2.


An example of the use would be the directive

   (#pjar:file:/**/*.jar!/**/*.* :function translate-jar-pathname)

which will call the function TRANSLATE-JAR-PATHNAME on a match of the 
first element of the list.  The function will be invoked with two 
arguments, the first being the matching wildcard, and the second being 
the pathname to be translated.  The function returns the translation.


Hopefully, after review and critique, we can can get something like this 
in ASDF2.


With the attached patch to abcl trunk, I can successfully use ASDF2 to 
load ASDF systems in jar files, with the compiled files being output to 
a translation under :USER-CACHE.



--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.
diff -r a9813719dfba doc/design/pathnames/jar-pathnames.markdown
--- a/doc/design/pathnames/jar-pathnames.markdown   Tue Mar 23 13:59:08 
2010 +0100
+++ b/doc/design/pathnames/jar-pathnames.markdown   Thu Mar 25 16:18:18 
2010 +0100
@@ -3,10 +3,10 @@
 
 Mark Evenson
 Created:  09 JAN 2010
-Modified: 16 MAR 2010 
+Modified: 25 MAR 2010 
 
-Notes towards sketching an implementation of jar: references to be
-contained in Common Lisp `PATHNAMEs` within ABCL.  
+Notes towards an implementation of jar: references to be contained
+in Common Lisp `PATHNAME`s within ABCL.
 
 Goals
 -
@@ -51,17 +51,23 @@
 6.  References jar:URL for all strings URL that java.net.URL can
 resolve works.
 
-7.  Make jar pathnames work as a valid argument for OPEN.
+7.  Make jar pathnames work as a valid argument for OPEN with
+:DIRECTION :INPUT.
 
 8.  Enable the loading of ASDF systems packaged within jar files.
 
+9.  Enable the matching of jar pathnames with PATHNAME-MATCH-P
+
+(pathname-match-p 
+  jar:file:/a/b/some.jar!/a/system/def.asd
+  jar:file:/**/*.jar!/**/*.asd)  
+== t
+
 Status
 --
 
-As of svn r12501, all the above goals have been implemented and tested
-*except* for:
-
-7.  Make jar pathnames work as a valid argument for OPEN.
+As of svn r125??, all the above goals have been implemented and
+tested.
 
 
 Implementation
@@ -81,7 +87,7 @@
 PATHNAME representing a JAR on the filesystem, or a SimpleString
 representing a URL.
 
-*   a PATHNAME occuring in the list in the DEVICE of a JAR PATHNAME is
+*   A PATHNAME occuring in the list in the DEVICE of a JAR PATHNAME is
 known as a DEVICE PATHNAME.
 
 *   If the DEVICE is a String it must be a String that successfully
diff -r a9813719dfba doc/design/pathnames/url-pathnames.markdown
--- a/src/org/armedbear/lisp/BuiltInClass.java  Tue Mar 23 13:59:08 2010 +0100
+++ b/src/org/armedbear/lisp/BuiltInClass.java  Thu Mar 25 16:18:18 2010 +0100
@@ -181,6 +181,9 @@
   public static final LispClass FILE_STREAM =
 addClass(Symbol.FILE_STREAM,
  new StructureClass(Symbol.FILE_STREAM, list(SYSTEM_STREAM)));
+  public static final LispClass JAR_STREAM =
+addClass(Symbol.JAR_STREAM,
+ new StructureClass(Symbol.JAR_STREAM, list(SYSTEM_STREAM)));
   public static final LispClass CONCATENATED_STREAM =
 addClass(Symbol.CONCATENATED_STREAM,
  new StructureClass(Symbol.CONCATENATED_STREAM, 
list(SYSTEM_STREAM)));
@@ -233,6 +236,8 @@
 FIXNUM.setCPL(FIXNUM, INTEGER, RATIONAL, REAL, NUMBER, CLASS_T);
 FILE_STREAM.setCPL(FILE_STREAM, SYSTEM_STREAM, STREAM,
STRUCTURE_OBJECT, CLASS_T);
+JAR_STREAM.setCPL(JAR_STREAM, SYSTEM_STREAM, STREAM,
+   STRUCTURE_OBJECT, CLASS_T);
 FLOAT.setDirectSuperclass(REAL);
 FLOAT.setCPL(FLOAT, REAL, NUMBER, CLASS_T);
 FUNCTION.setDirectSuperclass(CLASS_T);
diff -r a9813719dfba src/org/armedbear/lisp/FileStream.java
--- a/src/org/armedbear/lisp/FileStream.javaTue Mar 23 13:59:08 2010 +0100
+++ b/src/org/armedbear/lisp/FileStream.javaThu Mar 25 16:18:18 2010 +0100
@@ -286,11 +286,6 @@
 else {
 return type_error(first, Symbol.PATHNAME);
 }
-if (pathname.isJar()) {
-error(new FileError(Direct stream input/output on entries in 
JAR files no currently supported.,
-pathname));
-}
-
 final LispObject namestring = checkString(second);
 LispObject elementType = third;
 LispObject direction = fourth;
@@ -300,16 +295,30 @@
 if (direction != Keyword.INPUT  direction != Keyword.OUTPUT 
 direction != Keyword.IO)
 error

Re: [asdf-devel] [armedbear-devel] Patch for ASDF2 to use arbitrary output translation function

[Mark Evenson]

Right: I'll make sure that such an ability exists when we finally  
include ASDF2 in ABCL.


Tersely written from my iPod


On Mar 25, 2010, at 5:17 PM, Alan Ruttenberg  
alanruttenb...@gmail.com wrote:

 Sorry, haven't had a chance to look closely at asdf2, but wanted to
 point out that in certain contexts, like running applets, writing to
 any file system is prohibited. For those cases, it would be nice if it
 was super-easy to set a flag that says: if there isn't a compiled
 version, or if it is stale, don't try to compile - just load source.

 -Alan

 On Thu, Mar 25, 2010 at 8:38 AM, Mark Evenson even...@panix.com  
 wrote:
 On 3/25/10 2:39 PM, Mark Evenson wrote:
 […]

 I'm working on an implementation of this on your git version as a
 proof of concept, but haven't gotten much further than getting your
 output configuration to accept the new syntax.  I assume that
 actually applying the function will be easy, but I am not sure.

 […]

 Attached please find a patch to implement the use of an arbitrary  
 output
 translation function in ASDF2.

 An example of the use would be the directive

   (#pjar:file:/**/*.jar!/**/*.* :function translate-jar-pathname)

 which will call the function TRANSLATE-JAR-PATHNAME on a match of  
 the first
 element of the list.  The function will be invoked with two  
 arguments, the
 first being the matching wildcard, and the second being the  
 pathname to be
 translated.  The function returns the translation.

 Hopefully, after review and critique, we can can get something like  
 this in
 ASDF2.

 With the attached patch to abcl trunk, I can successfully use ASDF2  
 to load
 ASDF systems in jar files, with the compiled files being output to a
 translation under :USER-CACHE.


 --
 A screaming comes across the sky.  It has happened before, but there
 is nothing to compare to it now.

 ___
 armedbear-devel mailing list
 armedbear-de...@common-lisp.net
 http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel




___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On 3/23/10 4:53 PM, Faré wrote:
[…]

 I hope ABCL has fixed its merge-pathnames woes.

[…]

As I reported last week, ASDF2 works with ABCL from trunk and the 
upcoming abcl-0.19 release, for which I have fixed a couple issues with 
our pathname support.

Are there current failures with ABCL's MERGE-PATHNAMES that you are 
referring to, or is this referring to the state of ABCL a couple weeks 
ago?  If there are known problems, please help me identify them so we 
can fix them or at least record the information in our issue tracker.


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On 3/24/10 2:52 PM, Faré wrote:
 Sorry, I haven't tried recompiling the latest ABCL from source and
 don't know if any pathname issue remains.

 Have you tried downloading the latest ASDF and running its test suite?
 Do you pass all tests? I ought to include more tests about the
 output-translations configuration and the source-registry
 configuration...

Yes, and I just retried with ABCL trunk r12570 with ADSF 1.661 with no 
test failures.


-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] [armedbear-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

On 3/24/10 4:54 PM, Erik Huelsmann wrote:
 Hi Alan,

 On Wed, Mar 24, 2010 at 3:36 PM, Alan Ruttenberg
 alanruttenb...@gmail.com  wrote:
 Is ASDF2 what gets loaded when you require 'asdf in trunk, or some
 other action required to use it?
 -Alan

 I think we will need to update our ASDF now that the new ASDF2 is
 available. What you get when you use REQUIRE is a very old version of
 ASDF1.

Right:  we still include ASDF 1.3 with ABCL.

I'm looking at upgrading ABCL trunk to the new (aka ASDF2) version, but 
right now ASDF2 cannot load systems from jar files so I haven't put it 
on trunk just yet.

One can load ASDF2 from ASDF quite easily, for those wishing to 
experiment along with me.

-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.

___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ABCL testing issue

[Mark Evenson]

On Mar 17, 2010, at 3:55 PM, ] wrote:

 http://ec2-174-129-63-37.compute-1.amazonaws.com/test/log/20100316T233422/abcl-output.txt
 
 
 Testing: test-force.script
 Armed Bear Common Lisp 0.18.1
 Java 1.6.0_0 Sun Microsystems Inc.
 OpenJDK Client VM
 Low-level initialization completed in 1.985 seconds.
 Startup completed in 3.591 seconds.
 
 Caught ERROR while processing --eval option (load test-
 force.script):
   Failed to find asdf-abcl.lisp in /ebs/test/asdf/tmp/asdf-abcl.abcl.
 Using abcl , test-force.script failed
 
 This looks like a failure in ABCL's make-pathname or merge-pathnames 
 mechanism.
 
 Mark, can you comment?


abcl-0.18.1 is not going to work with ASDF2.  You currently need
to compile ABCL from a post svn r12551 commit.  The necessary fixes
have been backported to the upcoming  (any day now) abcl-0.19
release.

The specific bug you are encountering involves abcl-0.18.1 not being
able to handle the renaming of a FASL, as ASDF2 compiles to
asdf-tmp.XXX and then renames to asdf-LISP_IMPLEMENTATION.XXX.
Rather embaressing for us, really, so [we fixed it fast][1]

[1]: http://trac.common-lisp.net/armedbear/changeset/12550

[…]

--

A screaming comes across the sky.  It has happened before, but there is 
nothing to compare to it now.






___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] 1.636

[Mark Evenson]

On 3/16/10 12:10 AM, james anderson wrote:

 most everything now has equivalent results.[1]
 abcl now loads asdf, but the test fails anomalously. looking...


 abcl fails in connection with a make-pathname operation of the sort

  (make-pathname :directory '(:relative) :name
 file :type :unspecific :host nil :device nil)

I just fixed the failure of that form in [abcl trunk in r122549][1].

[1]: http://trac.common-lisp.net/armedbear/changeset/12549

[…]

 abcl has a distinct dislike for :unspecific pathname components.

 if the 1.636 code for merge-pathnames* is patched for abcl to always
 use #'ununspecific, abcl constructs all systems and succeeds with the
 same 1280/2080 directory/file matches as most of the other
 implementations.

I'll try to fix what I can in ABCL's implementation of :UNSPECIFC to get 
ASDF running.

-- 
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.


___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] Patches for ABCL against asdf-1.641

[Mark Evenson]

Attached are patches in 'abcl-asdf.patch' to get asdf-1.641 to work 
against ABCL for the run-tests.sh.


Unfortunately, you will need to build [ABCL from trunk][1] using at 
least svn r12550, because I had to patch ABCL to work with ASDF.  And 
you'll need to apply the 'abcl-translate-pathname.patch'.


[1]: svn://common-lisp.net/project/armedbear/svn/trunk/abcl


Notes:

1.  The fugliness of the conditional around ASDF:GET-UID works around a 
bug in the ABCL compiler.  We are in the process of [analyzing the 
error][2].


[2]: http://trac.common-lisp.net/armedbear/ticket/89

2.  There is some undiagnosed problem in translating the binary location 
for the new ASDF2 ~/.cache conventions, as ABCL seems to collapses 
everything into a single directory.  Somehow, the default 
*output-translations* have a /**/**/*.* where there should be 
/**/*.*.  I'm not sure if this is a problem in 
abcl-translate-pathname.patch, which is why I didn't apply this to 
ABCL trunk.


3. ASDF's run-tests.sh seems to ignore the flags setting, which seems 
to be broken on the ASDF side.


4.  The changes for ASDF look quite interesting.  However I would 
advocate that you should make ASDF2 work *exactly* like ASDF1 out of the 
box, meaning you shouldn't do any of the subsumed ASDF-BINARY-LOCATIONS 
stuff without being asked to by configuration.


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.
149c149,150
  :fmakunbound '(#:perform #:explain #:output-files #:operation-done-p)
---
  :fmakunbound '(#:perform #:explain #:output-files #:operation-done-p
 #:component-relative-pathname)
253c254
   (subseq VERSION:1.640 (1+ (length VERSION
---
   (subseq VERSION:1.641 (1+ (length VERSION
573,575c574
   (si:getenv x)
   #+abcl
   (ext:getenv x))
---
   (si:getenv x))
658,660c657,658
   (parse-integer (read-line stream))
   ;; (handler-case (parse-integer (read-line stream))
   ;;   (error () (error Unable to find out user ID)))
---
   (handler-case (parse-integer (read-line stream))
 (error () (error Unable to find out user ID)))
698a697,699
 (defun lispize-pathname (input-file)
   (make-pathname :type lisp :defaults input-file))
 
1490c1491
  (output (compile-file-pathname (first input) :type :fasl)))
---
  (output (compile-file-pathname (lispize-pathname (first input)) 
 :type :fasl)))
1524,1528c1525,1530
   #-:broken-fasl-loader
   (list #-ecl (compile-file-pathname (component-pathname c))
 #+ecl (compile-file-pathname (component-pathname c) :type :object)
 #+ecl (compile-file-pathname (component-pathname c) :type :fasl))
   #+:broken-fasl-loader (list (component-pathname c)))
---
   (let ((p (lispize-pathname (component-pathname c
 #-:broken-fasl-loader
 (list #-ecl (compile-file-pathname p)
   #+ecl (compile-file-pathname p :type :object)
   #+ecl (compile-file-pathname p :type :fasl))
 #+:broken-fasl-loader (list p)))
1551c1553
   :collect (let ((output (compile-file-pathname i)))
---
   :collect (let ((output (compile-file-pathname (lispize-pathname 
 i
2041,2044c2043
 #+abcl
 (ext:run-shell-command command :output *verbose-out*)
 
 #-(or openmcl clisp lispworks allegro scl cmu sbcl ecl abcl)
---
 #-(or openmcl clisp lispworks allegro scl cmu sbcl ecl)
2515c2514
   (truenamize (make-pathname :type lisp :defaults input-file))
---
   (truenamize (lispize-pathname input-file))
diff -r 1be31c2fbbe1 src/org/armedbear/lisp/pathnames.lisp
--- a/src/org/armedbear/lisp/pathnames.lisp Tue Mar 16 16:18:26 2010 +0100
+++ b/src/org/armedbear/lisp/pathnames.lisp Tue Mar 16 16:56:08 2010 +0100
@@ -232,6 +232,10 @@
(append (reverse match) 
(translate-directory-components-aux
 src (cdr from) (cdr to) case
+   (when (and (null src) 
+  (eq (car from) :wild-inferiors)
+  (eq (car to) :wild-inferiors))
+ (return-from translate-directory-components-aux nil))
(when (null src) ;; SRC is NIL and we're still here: error exit
  (throw 'failed-match))
 
___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

[asdf-devel] ABCL trunk r12551 works with asdf-1.643

[Mark Evenson]

Thanks for applying my patches to ASDF, as it now appears that ABCL 
trunk (svn r12551 and later) runs pretty well with ASDF-1.643.  The 
previous problem I noted with the binary locations getting collapsed 
was our problem which I fixed in ABCL.


Attached is a (trivial) patch so that 'run-tests.sh' actually uses the 
flags argument.


ABCL now passes the ASDF test suite:

-#---
Using abcl  --noinit
Ran 21 tests:
  21 passing and 0 failing
all tests apparently successful
-#---

After discussion on IRC and further thought, I no longer advocate that 
ASDF2 not make the relocating the FASLs the default.  There are 
certainly decent arguments for it, and we do want progress in ASDF2, 
right?  Instead, I would advise that you make a section in the (really 
excellent looking) manual that explicates the changes that an ASDF1 
user—both with and without using ASDF-BINARY-LOCATIONS—should expect in 
ASDF2.  And I would include the *LOAD-TRUENAME*/*LOAD-PATHNAME* issue 
(i.e. use a conditionalized ASDF:SYSTEM-DEFINITION-PATHNAME invocation) 
in that section.


Otherwise, ASDF2 looks quite cool.  What's your timeframe for an 
official release?  I'd include it as-is in ABCL, but don't want to chase 
all the release candidates to ASDF2.


--
A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now.
diff --git a/test/run-tests.sh b/test/run-tests.sh
index 2716f03..5308fe3 100755
--- a/test/run-tests.sh
+++ b/test/run-tests.sh
@@ -182,6 +182,7 @@ if [ -z $command ] ; then
 else
 create_config
 mkdir -p results
+command=$command $flags
 echo $command
 thedate=`date +%Y-%m-%d`
 do_tests $command $eval 21 | \
___
asdf-devel mailing list
asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel