Re: [asdf-devel] Ordering of system :depends-on elements
On Mon, Jul 8, 2013 at 9:36 PM, Anton Vodonosov wrote: > If given system name "mcclim/freetype" we want > ASDF to search for mcclim.asd, then the extension hook > should be named *sysdef-file-search-functions* instead > of *system-definition-search-functions*. > Gratuitously changing the hook variable name would only force an even more abrupt transition without obviously bringing any benefit. It's not like the old semantics is hopelessly broken and we want to force people away from it with aggravation (as was the case with the conditional semantics of :if-component-fails). It's more like we introduced a much needed new feature, and it is subtly incompatible with some previous ASDF extensions (or singular key extension, namely quicklisp). > That would allow the the strategy of fallback to the > main system name to be encapsulated in the main ASDF. > It is a feature of ASDF3 (started in the last year of ASDF2) that ASDF supports systems that are not backed by files, but simply defined in the lisp image. The feature is notably used by ASDF itself, and could be used by any system that needs to be defined for dependency purpose, and may or may not be upgraded to a version from source when available. > And still have the code organization reflecting what is really going on: > ASDF tries to load system "mcclim/freetype" > and asks the extensions for find the "mcclim.asd" file. > What if it's not in a file? Shouldn't each extension be responsible for itself? > Of course, deploying this solution today is complicated - all clients > must attach to new hook. So probably it is easier to just duplicate > the fallback policy in the clients. > As far as I can tell, there is one and only one client that matters — quicklisp. And matters it does. > P.S. If it is necessary to support people who keep several > system definitions in a single .asd file, then it makes sense > to apply the same strategy to dash separated system names: > search for "mcclim-freetype" may fallback to search in mcclim.asd > I don't know if it's necessary, but it has been common practice for a long time, except lacking any supported way to name them in such a way that ASDF would find them, unless they had been somehow found already — until ASDF3. Except as people start to use the new ASDF3 feature, something has to be done in ASDF and/or Quicklisp for Quicklisp to support it. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Any sufficiently advanced misquotation is indistinguishable from an original statement. — John McCarthy, misquoted
Re: [asdf-devel] Ordering of system :depends-on elements
Anton Vodonosov writes: > 08.07.2013, 01:34, "Zach Beane" : >> A few projects in quicklisp work something like this: >> >> ;;; foo.asd >> >> (defsystem foo ...) >> >> (defsystem foo-extra ...) >> >> ;;; bar.asd >> >> (defsystem bar :depends-on (:foo-extra :foo)) >> >> With asdf 2, (asdf:load-system "bar") seems to work fine, I guess >> because asdf 2 does the equivalent of find-system on the elements from >> right-to-left. >> >> With asdf 3, it doesn't seem to work fine, I guess because asdf 3 does >> the equivalent of find-system on the elements from left-to-right. >> >> Are those guesses correct? >> >> What's the best way to have a system definition that works equally well >> in asdf2 and asdf3 in this kind of situation? > > If we surround the non-findable system with the main system like this: > > (defsystem bar :depends-on (:foo :foo-extra :foo)) > > will it work? > > :) > > It's a dirty workaround (worksurround), but the level of dirtiness is the same > as the original: have a system not findable by asdf:find-system due to > difference between the system name and it's .asd file name; solve it > by relying on the order ASDF loads dependencies, having a findable > system name first. Ooh, that is so delightfully dirty that even if it doesn't work, I admire the twisted kind of mind that would come up with it. Nice! I'll give it a try. Zach
Re: [asdf-devel] Ordering of system :depends-on elements
08.07.2013, 01:34, "Zach Beane" : > A few projects in quicklisp work something like this: > > ;;; foo.asd > > (defsystem foo ...) > > (defsystem foo-extra ...) > > ;;; bar.asd > > (defsystem bar :depends-on (:foo-extra :foo)) > > With asdf 2, (asdf:load-system "bar") seems to work fine, I guess > because asdf 2 does the equivalent of find-system on the elements from > right-to-left. > > With asdf 3, it doesn't seem to work fine, I guess because asdf 3 does > the equivalent of find-system on the elements from left-to-right. > > Are those guesses correct? > > What's the best way to have a system definition that works equally well > in asdf2 and asdf3 in this kind of situation? If we surround the non-findable system with the main system like this: (defsystem bar :depends-on (:foo :foo-extra :foo)) will it work? :) It's a dirty workaround (worksurround), but the level of dirtiness is the same as the original: have a system not findable by asdf:find-system due to difference between the system name and it's .asd file name; solve it by relying on the order ASDF loads dependencies, having a findable system name first.
Re: [asdf-devel] Ordering of system :depends-on elements
If given system name "mcclim/freetype" we want ASDF to search for mcclim.asd, then the extension hook should be named *sysdef-file-search-functions* instead of *system-definition-search-functions*. That would allow the the strategy of fallback to the main system name to be encapsulated in the main ASDF. And still have the code organization reflecting what is really going on: ASDF tries to load system "mcclim/freetype" and asks the extensions for find the "mcclim.asd" file. Of course, deploying this solution today is complicated - all clients must attach to new hook. So probably it is easier to just duplicate the fallback policy in the clients. P.S. If it is necessary to support people who keep several system definitions in a single .asd file, then it makes sense to apply the same strategy to dash separated system names: search for "mcclim-freetype" may fallback to search in mcclim.asd
Re: [asdf-devel] Ordering of system :depends-on elements
Faré writes: > On Mon, Jul 8, 2013 at 11:42 AM, Zach Beane wrote: >> Faré writes: >> >>> As for stripping the /, currently it's done by the individual functions in >>> *system-definition-search-functions* by calling primary-system-name. >> >> I think this could be moved into search-for-system-definition by >> wrapping coerce-name in primary-system-name. >> > One issue I see with this is that some s-f-s-d handlers might be > sensitive to the full name rather than to the primary system name, and > always loading base on the primary name only drops the information > about which specific subsystem one is looking for. It is theoretically possible that some new handler written in the past few months may be adversely affected by searching only for primary system names via *s-d-s-f*. It is concretely the case that ALL handlers written prior to the introduction of this feature have been broken by it. And some of those handlers, e.g. the one in Quicklisp, have been widely deployed. Zach
Re: [asdf-devel] Ordering of system :depends-on elements
On Mon, Jul 8, 2013 at 11:42 AM, Zach Beane wrote: > Faré writes: > >> As for stripping the /, currently it's done by the individual functions in >> *system-definition-search-functions* by calling primary-system-name. > > I think this could be moved into search-for-system-definition by > wrapping coerce-name in primary-system-name. > One issue I see with this is that some s-f-s-d handlers might be sensitive to the full name rather than to the primary system name, and always loading base on the primary name only drops the information about which specific subsystem one is looking for. The current solution is for each s-d-s-f function to do its own thing. This is most flexible, but requires all clients of s-d-s-f to be updated to support this feature. One solution proposed by Zach is that if and when we realize that normalizing to primary system name is always the right thing to do, this can be moved up the call chain — but is it the case? Zach seems to think so, but at least I'm not convinced. Another solution, that Zach proposed in a previous email, was to search for the full name, then the primary name as a fallback. This is somewhat quirky and makes the semantics of system naming more complex, but probably works automatically in the most cases. Yet another solution might be to always look for the primary name first and load the system definition, then look for the full name if different. This makes the semantics more regular, though a bit more expensive. I admit picked the current solution was the most straightforward to implement, and most open to future improvements. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org What's funny with equality is how everyone has a different idea of it. —#f
Re: [asdf-devel] About some new system slots
On Mon, Jul 8, 2013 at 12:25 PM, Robert Goldman wrote: > 1. I believe that the value of a *human-readable* piece of meta-data here > should be obvious. Particularly if you get a version of a library from some > source (tarball, quicklisp), that has a bug you wish to fix. It would be > nice if you didn't have to track down the VCS for yourself (particularly if > there are multiple possible candidates in DVCS world). But for this all we > need is an arbitrary string such as "git repository > git://common-lisp.net/projects/asdf/asdf.git". This requires no action on > the part of the ASDF developers. Note that Faré's format meets this > criterion, as well. > Yes, and because darcs, svn, git, and possibly more, can all be done over http, it isn't sufficient to have a URL, and/or that would require some magic autodetection. Explicit specification of the VCS seems better to me. > 2. It is an open question whether *machine-readable* meta-data would be > useful. My previous response was directed at machine-readable meta-data. > This could be either (a) something that prescribes enough of a format to > ensure happily formatted information (e.g., for CL-TEST-GRID), or (b) > something that would support automated access to the version control > repository encoded in the system description. As I said, it's not clear that > this is needed right now. My proposal would aim at this, but is probably > premature. > At the very least, machine readable means you can automate the push-button task of downloading the upstream source of a system and/or all its relevant dependencies when given a tarball, or a lisp image where the system is loaded. Admittedly, if the system is already in quicklisp, quicklisp can possibly give you the answer already, so this might be redundant. On the other hands, systems need not be in quicklisp. In any case, this was one feature requested by users, that I find passably useful, and that had been done badly in non-standard ways in the past in various systems, so I at least introduced the slot and a proposed format for it, but pending an actual documented proposal, nothing is enforced about the value in this slot as of yet. Of course, I'm not the one to decide anymore. I'm just here to assist the new maintainer in understanding previous decisions. Good luck, Robert! —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Everything is interesting if you go into it *deeply* enough. — Richard Feynman
Re: [asdf-devel] About some new system slots
Didier Verna wrote: > Faré wrote: > >> Although, it isn't clear yet how source-control may or may not specify >> the VCS used: should we prefix the URL with a git: or svn: or darcs: >> or some such? ASDF uses: :source-control (:git >> "git://common-lisp.net/projects/asdf/asdf.git") > > Personally, I would not complicate this slot and just go for a simple > URL pointing to a VCS related web page. If you start specifying more > information there, you will end up with a big mess (which VCS, which > access backend, which access type (r/o r/w) etc.). > > That's just me of course. > I can think of two answers to this: 1. I believe that the value of a *human-readable* piece of meta-data here should be obvious. Particularly if you get a version of a library from some source (tarball, quicklisp), that has a bug you wish to fix. It would be nice if you didn't have to track down the VCS for yourself (particularly if there are multiple possible candidates in DVCS world). But for this all we need is an arbitrary string such as "git repository git://common-lisp.net/projects/asdf/asdf.git". This requires no action on the part of the ASDF developers. Note that Faré's format meets this criterion, as well. 2. It is an open question whether *machine-readable* meta-data would be useful. My previous response was directed at machine-readable meta-data. This could be either (a) something that prescribes enough of a format to ensure happily formatted information (e.g., for CL-TEST-GRID), or (b) something that would support automated access to the version control repository encoded in the system description. As I said, it's not clear that this is needed right now. My proposal would aim at this, but is probably premature.
Re: [asdf-devel] Ordering of system :depends-on elements
Faré writes: > As for stripping the /, currently it's done by the individual functions in > *system-definition-search-functions* by calling primary-system-name. I think this could be moved into search-for-system-definition by wrapping coerce-name in primary-system-name. Zach
Re: [asdf-devel] Ordering of system :depends-on elements
If quicklisp can't use / in names, it's probably a bug, since such names have always been accepted by ASDF (though not actually used before). Nevertheless, a workaround is for a system do have the dependency on foo both before and after that on the thing that is loaded when loading foo.asd: :depends-on (foo ... foo-extra ... foo) That's ugly, and probably an abuse. As for stripping the /, currently it's done by the individual functions in *system-definition-search-functions* by calling primary-system-name. I suppose that could be changed, but the backward-compatible way would be to (let ((name ...) #+asdf3 (name (primary-system-name name))) ...) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Foreign aid might be defined as a transfer from poor people in rich countries to rich people in poor countries. — Douglas Casey On Mon, Jul 8, 2013 at 11:11 AM, Zach Beane wrote: > Zach Beane writes: > >>> So can you make suggestions about how to be quicklisp-compatible while >>> making it possible to have multiple findable system definitions in a >>> single file? >> >> Hmm, I think it might be workable if, when a system with a given slashed >> name is not found, the *system-definition-search-functions* chain was >> called recursively with the truncated name, instead of having each >> function in *s-d-s-f* have to know how to truncate on its own. >> >> E.g. imagine I had something like this: >> >> (push (lambda (name) (print (list 'trace name)) nil) >> *system-definition-search-functions*) >> >> Then it would be nice if the ASDF internals led to putput like this: >> >> (find-system "foo/bar") >> >> (TRACE "foo/bar") >> (TRACE "foo") >> >> => # >> >> Does it work that way now? I don't think it does. > > Actually, another option could be to ONLY call the *s-d-s-f* chain on > the truncated name. That would preclude those functions from > implementing some new funky logic for slashed names, but I don't think > that's really much of an issue. > > Zach
Re: [asdf-devel] Ordering of system :depends-on elements
Zach Beane writes: >> So can you make suggestions about how to be quicklisp-compatible while >> making it possible to have multiple findable system definitions in a >> single file? > > Hmm, I think it might be workable if, when a system with a given slashed > name is not found, the *system-definition-search-functions* chain was > called recursively with the truncated name, instead of having each > function in *s-d-s-f* have to know how to truncate on its own. > > E.g. imagine I had something like this: > > (push (lambda (name) (print (list 'trace name)) nil) > *system-definition-search-functions*) > > Then it would be nice if the ASDF internals led to putput like this: > > (find-system "foo/bar") > > (TRACE "foo/bar") > (TRACE "foo") > > => # > > Does it work that way now? I don't think it does. Actually, another option could be to ONLY call the *s-d-s-f* chain on the truncated name. That would preclude those functions from implementing some new funky logic for slashed names, but I don't think that's really much of an issue. Zach
Re: [asdf-devel] About some new system slots
Robert Goldman wrote: > The advantage of having the pair, of course, is that a simple URL does > not necessarily identify the form of VCS. But what would be the use for this information? I mean, having this information as part of the system desciption. -- Resistance is futile. You will be jazzimilated. Lisp, Jazz, Aïkido: http://www.didierverna.info
Re: [asdf-devel] About some new system slots
Didier Verna wrote: > Faré wrote: > >> Although, it isn't clear yet how source-control may or may not specify >> the VCS used: should we prefix the URL with a git: or svn: or darcs: >> or some such? ASDF uses: :source-control (:git >> "git://common-lisp.net/projects/asdf/asdf.git") > > Personally, I would not complicate this slot and just go for a simple > URL pointing to a VCS related web page. If you start specifying more > information there, you will end up with a big mess (which VCS, which > access backend, which access type (r/o r/w) etc.). > > That's just me of course. > The advantage of having the pair, of course, is that a simple URL does not necessarily identify the form of VCS. My preference would be to a. document a proposed format (e.g., Faré's two-element list of keyword + URL -- possibly keyword + ancillary info to permit some hypothetical VCS that does not use URLs); b. check this format when parsing DEFSYSTEM, issuing a STYLE-WARNING when the meta-data doesn't comply; c. hope people provide useful information here, but not worry about it. Cheers, r
Re: [asdf-devel] Ordering of system :depends-on elements
Robert Goldman writes: > Zach Beane wrote: >> Faré writes: >> >>> A solution that is both backward-compatible and forward compatible is >>> >>> 1- to rename the systems so they follow the foo/bar convention for >>> naming systems in foo.asd, so that ASDF3 can find them. This solves >>> forward compatibility >> >> If anyone considering fixing mcclim is reading this, please don't use a >> slash-named system. It's incompatible with Quicklisp. >> >> Zach > > Can you explain the incompatibility? > > It would be hugely useful to be able to map multiple systems to a > single file, as your McCLIM example illustrates. I think it illustrates the problems of putting multiple systems in the same file, not the utility. > But, of course, we want to maximize quicklisp compatibility. > > So can you make suggestions about how to be quicklisp-compatible while > making it possible to have multiple findable system definitions in a > single file? Hmm, I think it might be workable if, when a system with a given slashed name is not found, the *system-definition-search-functions* chain was called recursively with the truncated name, instead of having each function in *s-d-s-f* have to know how to truncate on its own. E.g. imagine I had something like this: (push (lambda (name) (print (list 'trace name)) nil) *system-definition-search-functions*) Then it would be nice if the ASDF internals led to putput like this: (find-system "foo/bar") (TRACE "foo/bar") (TRACE "foo") => # Does it work that way now? I don't think it does. Zach
Re: [asdf-devel] Ordering of system :depends-on elements
Zach Beane wrote: > Faré writes: > >> A solution that is both backward-compatible and forward compatible is >> >> 1- to rename the systems so they follow the foo/bar convention for >> naming systems in foo.asd, so that ASDF3 can find them. This solves >> forward compatibility > > If anyone considering fixing mcclim is reading this, please don't use a > slash-named system. It's incompatible with Quicklisp. > > Zach Can you explain the incompatibility? It would be hugely useful to be able to map multiple systems to a single file, as your McCLIM example illustrates. But, of course, we want to maximize quicklisp compatibility. So can you make suggestions about how to be quicklisp-compatible while making it possible to have multiple findable system definitions in a single file? thanks, Robert
Re: [asdf-devel] Ordering of system :depends-on elements
Faré writes: > A solution that is both backward-compatible and forward compatible is > > 1- to rename the systems so they follow the foo/bar convention for > naming systems in foo.asd, so that ASDF3 can find them. This solves > forward compatibility If anyone considering fixing mcclim is reading this, please don't use a slash-named system. It's incompatible with Quicklisp. Zach
Re: [asdf-devel] About some new system slots
Faré wrote: > Although, it isn't clear yet how source-control may or may not specify > the VCS used: should we prefix the URL with a git: or svn: or darcs: > or some such? ASDF uses: :source-control (:git > "git://common-lisp.net/projects/asdf/asdf.git") Personally, I would not complicate this slot and just go for a simple URL pointing to a VCS related web page. If you start specifying more information there, you will end up with a big mess (which VCS, which access backend, which access type (r/o r/w) etc.). That's just me of course. -- Resistance is futile. You will be jazzimilated. Lisp, Jazz, Aïkido: http://www.didierverna.info
