Re: follow-up about handling .info file and index

[Robert Goldman]

On 21 Mar 2018, at 14:09, Robert Dodier wrote:


Thanks for your comments, Robert. I've revised the INFO-INDEX class
and made it a lot simpler (and still working the same). For what it's
worth, I've pasted it below.

I noticed that UIOP:COPY-FILE doesn't seem to behave well if told to
copy a file to itself -- it clobbers the file. That seems like
unfriendly behavior; it seems better that either it should complain or
do nothing.


My guess (Faré's is the authorized word) is that it's difficult to tell 
when you are copying a file onto itself, the reason being that telling 
when two pathnames co-refer is problematic.


I guess we could add an :if-exists argument (defaulting to :overwrite 
for backwards compatibility), and make :supersede and :error be options.


We can't do :append, because Faré has used some implementations' 
built-ins for efficiency.  If you add a feature request to the 
launchpad, or (even better) a merge request to the common-lisp.net 
gitlab, I'll see about getting it in there.


Best,
r



Thanks for your help, I think I'm in good shape on this front now.

best,
Robert Dodier

PS.
;; info-index.lisp -- ASDF component type for Maxima documentation 
index

;; copyright 2018 by Robert Dodier
;; I release this work under terms of the GNU General Public License

(require 'asdf)
(require 'uiop)

(in-package :asdf)

(defclass info-index (cl-source-file) ())

;; An info index file is a Lisp source file, which is compiled
;; just the same as an ordinary Lisp file, with the additional
;; step of copying the .info to the same location to where the
;; compiler output will go.

(defmethod perform ((o compile-op) (c info-index))
  (let*
((system-name (component-name (component-system c)))
 (info-name (make-pathname :name system-name :type "info"))
 (info-in-file (merge-pathnames info-name (first (input-files o 
c
 (info-out-file (merge-pathnames info-name (first (output-files o 
c)

;; INFO-IN-FILE and INFO-OUT-FILE should be different,
;; but just to be safe, silently refuse to copy file to itself.
(unless (uiop:pathname-equal info-in-file info-out-file)
  (uiop:copy-file info-in-file info-out-file))
(call-next-method)))

Re: follow-up about handling .info file and index

[Robert Dodier]

Thanks for your comments, Robert. I've revised the INFO-INDEX class
and made it a lot simpler (and still working the same). For what it's
worth, I've pasted it below.

I noticed that UIOP:COPY-FILE doesn't seem to behave well if told to
copy a file to itself -- it clobbers the file. That seems like
unfriendly behavior; it seems better that either it should complain or
do nothing.

Thanks for your help, I think I'm in good shape on this front now.

best,
Robert Dodier

PS.
;; info-index.lisp -- ASDF component type for Maxima documentation index
;; copyright 2018 by Robert Dodier
;; I release this work under terms of the GNU General Public License

(require 'asdf)
(require 'uiop)

(in-package :asdf)

(defclass info-index (cl-source-file) ())

;; An info index file is a Lisp source file, which is compiled
;; just the same as an ordinary Lisp file, with the additional
;; step of copying the .info to the same location to where the
;; compiler output will go.

(defmethod perform ((o compile-op) (c info-index))
  (let*
((system-name (component-name (component-system c)))
 (info-name (make-pathname :name system-name :type "info"))
 (info-in-file (merge-pathnames info-name (first (input-files o c
 (info-out-file (merge-pathnames info-name (first (output-files o c)
;; INFO-IN-FILE and INFO-OUT-FILE should be different,
;; but just to be safe, silently refuse to copy file to itself.
(unless (uiop:pathname-equal info-in-file info-out-file)
  (uiop:copy-file info-in-file info-out-file))
(call-next-method)))

Re: follow-up about handling .info file and index

[Robert Dodier]

On Tue, Mar 20, 2018 at 7:30 PM, Faré  wrote:

> If your index is just a lisp file,
> why do you even need to subclass cl-source-file, to begin with?
> And if you do, why do you need override any method or initform?
> Let the CL source files be compiled the normal way!

Yes, well, the only specific thing is to copy the .info file to the
place where the fasl goes, then it punts to whatever cl-source-file
says to do.

> As to locating the .info file, while not just use the recommended
> asdf:system-relative-pathname ?

Well, the info index is something that's used in non-ASDF contexts in
Maxima, and I didn't want to change it. I agree that
asdf:system-relative-pathname would be the right way to go, if I were
able to start over.

> NB: If you're that masochist, please contact me as I'm pushing for
> something broadly similar in gerbil-scheme's clan libraries;

Well, I do endure a lot of self-inflicted suffering, but unfortunately
I already have an unending to-do list for Maxima ...

>> (1) In the class definition for INFO-INDEX, it
>> seems to be necessary to intone (type :initform "lisp").
>>
> This is not necessary. You are confused. See Robert Goldman's response.

Yup, I will revise the code in light of Robert's comments.

> However, note that this pattern is NOT currently in use
> in asdf/component.lisp or asdf/lisp-action.lisp -- yet.
> I would recommend that a future ASDF maintainer should in the future
> replace all current occurrences of static slots :initform with this style

I dunno -- I think I'm going to play it safe and use the old pattern,
if it's necessary at all ... I'm a pretty casual ASDF user, I don't
know if I can invest the time to figure the old vs new stuff.

All the best,
Robert Dodier

Re: follow-up about handling .info file and index

[Faré]

Dear Robert and Robert,

here is a complement to Robert Goldman's excellent response.
It includes some style hints for the future of ASDF and its extensions.

>: Robert Dodier
> What I finally settled on is this. When the operation is COMPILE-OP,
> the .info file is copied to same location where the index fasl will go
> (because the index has some code to read the .info and it assumes the
> .info is in the same place as the fasl). In the .asd file, I say
> (defsystem-depends-on "info-index") and (:info-index "foo-index") for
> the component foo-index.lisp. This seems to be working as intended.
>
If your index is just a lisp file,
why do you even need to subclass cl-source-file, to begin with?
And if you do, why do you need override any method or initform?
Let the CL source files be compiled the normal way!

As to locating the .info file, while not just use the recommended
asdf:system-relative-pathname ?

I admit ASDF doesn't have a good story for standalone software delivery,
beside "hardwire everything in the image".
However your method strikes me as worse in every respect.

For a solution that requires all libraries to use suitable abstractions,
see in quux/lisp/qres-build/pathnames.lisp in the quux snapshot tarball
(that you can find on the QITAB page)
how we used to do it at ITA Software's QRes team.
The code in asdf/uiop/configuration.lisp including XDG support
is also a good start.
Making from it a solution adopted by the Common Lisp community at large
is an exercise left to the masochist.

NB: If you're that masochist, please contact me as I'm pushing for
something broadly similar in gerbil-scheme's clan libraries;
see utils/path-config.ss for the current version,
good enough for my current application but probably not
a well-defined general solution.

> (1) In the class definition for INFO-INDEX, it
> seems to be necessary to intone (type :initform "lisp").
>
This is not necessary. You are confused. See Robert Goldman's response.

It is inherited indeed, and is defined in lisp-action.lisp as follows:

  (defclass cl-source-file (source-file)
((type :initform "lisp"))

Now, my latest woes with asdf/bundle revealed that this pattern,
that I had been relying on since ASDF2,
isn't friendly with respect to code upgrade,
and particularly not friendly in case of a bug in a value
that must be fixed in a latter version.

A better pattern, that is now used for some methods
(notably gather-type and bundle-type in asdf/bundle.lisp),
is instead to define a method:

(defmethod file-type ((_ info-index)) "lisp")

I recommend you should use this pattern for all code going forward.
Except of course, you don't need to define a new method if you don't
need to override the behavior from the super-class.

However, note that this pattern is NOT currently in use
in asdf/component.lisp or asdf/lisp-action.lisp -- yet.
I would recommend that a future ASDF maintainer should in the future
replace all current occurrences of static slots :initform with this style
but that is a backward incompatible move that requires
a proper transition period:
first, announce the deprecation of the old style;
second, update all relevant ASDF extensions in Quicklisp;
third, bug known companies suspected to have private ASDF extensions;
finally, after a suitable timeout (one or two years since deprecation),
make the change in ASDF itself.

—♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
Don't worry about what anybody else is going to do. The best way to predict
the future is to invent it. Really smart people with reasonable funding can
do just about anything that doesn't violate too many of Newton's Laws!
— Alan Kay, 1971

Re: follow-up about handling .info file and index

[Robert Goldman]

On 20 Mar 2018, at 12:31, Robert Dodier wrote:


Hi, I was asking recently about how to handle a .info file and its
index (CL) in ASDF.

What I finally settled on is this. When the operation is COMPILE-OP,
the .info file is copied to same location where the index fasl will go
(because the index has some code to read the .info and it assumes the
.info is in the same place as the fasl). In the .asd file, I say
(defsystem-depends-on "info-index") and (:info-index "foo-index") for
the component foo-index.lisp. This seems to be working as intended.

I've pasted the code I devised as a PS below. If anyone has any
comments about it, I would be interested to hear it.

I have a couple of specific questions about the code, which you can
see in the comments. (1) In the class definition for INFO-INDEX, it
seems to be necessary to intone (type :initform "lisp"). Is that
actually necessary? If so, why is that? I would have expected (on
little reason) that the parent class CL-SOURCE-FILE would supply that.
(2) In INPUT-FILES, why doesn't (info-index-type c) yield "lisp"?
Doesn't INFO-INDEX have an accessor INFO-INDEX-TYPE?


See answers below.


I may well be very confused about simple things. Thank you for your
patience, I appreciate it.

Robert Dodier

PS. info-index.lisp:
;; info-index.lisp -- ASDF component type for Maxima documentation 
index

;; copyright 2018 by Robert Dodier
;; I release this work under terms of the GNU General Public License

(require 'asdf)

(in-package :asdf)

(defclass info-index (cl-source-file)
  ((type :initform "lisp"))) ;; ISN'T THIS INHERITED FROM 
CL-SOURCE-FILE ??


Sure looks like it is.  This is in lisp-action.lisp in ASDF:
```
(defclass cl-source-file (source-file)
((type :initform "lisp"))
(:documentation "Component class for a Common Lisp source file 
(using type \"lisp\")"))

```


(defmethod perform ((o load-source-op) (c info-index))
  (call-next-method))

(defmethod perform ((o load-op) (c info-index))
  (call-next-method))


I don't see why the above two are necessary.



(defmethod input-files ((o compile-op) (c info-index))
  (let*
((foo (call-next-method))
 ;; WHY DOESN'T THE FOLLOWING LINE WORK ??
 ;; (bar (info-index-type c))


This is a CLOS class, not a struct, so you don't get an automatic 
`info-index-` prefixed accessor.  You need to use `(asdf:file-type c)`

 (bar "lisp")
 (baz (merge-pathnames (make-pathname :type bar) (first foo
(list baz)))

(defmethod output-files ((o compile-op) (c info-index))
  (call-next-method))

Again, why is this necessary?


(defun silly-copy (in-file out-file)
  (unless (equalp in-file out-file) ;; silently refuse to copy file to 
itself

(with-open-file (in in-file)
  (with-open-file (out out-file :direction :output :if-exists 
:supersede)

(do ((c (read-char in nil) (read-char in nil))) ((null c))
  (write-char c out))


Probably you need something like `uiop:pathname-equal` instead of 
`equalp` above.  And if there are symbolic links in the mix, you could 
have trouble.  Consider also `uiop:truename*`.  `uiop:copy-file` may be 
more efficient than your portable stream copy.



HTH

follow-up about handling .info file and index

[Robert Dodier]

Hi, I was asking recently about how to handle a .info file and its
index (CL) in ASDF.

What I finally settled on is this. When the operation is COMPILE-OP,
the .info file is copied to same location where the index fasl will go
(because the index has some code to read the .info and it assumes the
.info is in the same place as the fasl). In the .asd file, I say
(defsystem-depends-on "info-index") and (:info-index "foo-index") for
the component foo-index.lisp. This seems to be working as intended.

I've pasted the code I devised as a PS below. If anyone has any
comments about it, I would be interested to hear it.

I have a couple of specific questions about the code, which you can
see in the comments. (1) In the class definition for INFO-INDEX, it
seems to be necessary to intone (type :initform "lisp"). Is that
actually necessary? If so, why is that? I would have expected (on
little reason) that the parent class CL-SOURCE-FILE would supply that.
(2) In INPUT-FILES, why doesn't (info-index-type c) yield "lisp"?
Doesn't INFO-INDEX have an accessor INFO-INDEX-TYPE?

I may well be very confused about simple things. Thank you for your
patience, I appreciate it.

Robert Dodier

PS. info-index.lisp:
;; info-index.lisp -- ASDF component type for Maxima documentation index
;; copyright 2018 by Robert Dodier
;; I release this work under terms of the GNU General Public License

(require 'asdf)

(in-package :asdf)

(defclass info-index (cl-source-file)
  ((type :initform "lisp"))) ;; ISN'T THIS INHERITED FROM CL-SOURCE-FILE ??

(defmethod perform ((o load-source-op) (c info-index))
  (call-next-method))

(defmethod perform ((o load-op) (c info-index))
  (call-next-method))

(defmethod input-files ((o compile-op) (c info-index))
  (let*
((foo (call-next-method))
 ;; WHY DOESN'T THE FOLLOWING LINE WORK ??
 ;; (bar (info-index-type c))
 (bar "lisp")
 (baz (merge-pathnames (make-pathname :type bar) (first foo
(list baz)))

(defmethod output-files ((o compile-op) (c info-index))
  (call-next-method))

(defun silly-copy (in-file out-file)
  (unless (equalp in-file out-file) ;; silently refuse to copy file to itself
(with-open-file (in in-file)
  (with-open-file (out out-file :direction :output :if-exists :supersede)
(do ((c (read-char in nil) (read-char in nil))) ((null c))
  (write-char c out))

(defmethod perform ((o compile-op) (c info-index))
  (let*
((system-name (component-name (component-system c)))
 (info-name (make-pathname :name system-name :type "info"))
 (info-in-file (merge-pathnames info-name (first (input-files o c
 (info-out-file (merge-pathnames info-name (first (output-files o c)
(silly-copy info-in-file info-out-file)
(call-next-method)))