Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Dodier
Hi,
I am resuming my tinkering on the Maxima interface to ASDF. I have
bumped into a kind of minor problem for which I don't see a solution.

Maxima can read .info files for documentation items. An index file
(Lisp) which is created ahead of time (not run time) tells where the
items are in the .info. The .asd, index, and .info are all in the same
directory. Some code in the index file tells Maxima to read the .info
from the same place from which the index is loaded (via
*LOAD-PATHNAME*).

That works great when the asd is loaded via LOAD-SOURCE-OP since
*LOAD-PATHNAME* points to the directory where the .info file is. But
when the asd is loaded via LOAD-OP, it doesn't work, since
*LOAD-PATHNAME* points into the place that ASDF puts fasls.

I gather from the ASDF reference manual that there isn't any way to
load, without compiling, only the index file (I'm looking at "How do I
mark a source file to be loaded only and not compiled?"). So I'm
thinking about some other way to locate the .info file. Is there a
special variable or something which tells the path from which the .asd
is being loaded? I looked at the documentation and the list of
external symbols but couldn't puzzle out anything.

The more general idea is that the system contains a non-Lisp data
file. Is there a general approach to work with that?

Thanks for any light you can shed on this problem.

Robert Dodier



Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Lucien P
On Fri, Mar 16, 2018, 11:25 Robert Dodier  wrote:

> Hi,
> I am resuming my tinkering on the Maxima interface to ASDF. I have
> bumped into a kind of minor problem for which I don't see a solution.
>
> Maxima can read .info files for documentation items. An index file
> (Lisp) which is created ahead of time (not run time) tells where the
> items are in the .info. The .asd, index, and .info are all in the same
> directory. Some code in the index file tells Maxima to read the .info
> from the same place from which the index is loaded (via
> *LOAD-PATHNAME*).
>
> That works great when the asd is loaded via LOAD-SOURCE-OP since
> *LOAD-PATHNAME* points to the directory where the .info file is. But
> when the asd is loaded via LOAD-OP, it doesn't work, since
> *LOAD-PATHNAME* points into the place that ASDF puts fasls.
>
> I gather from the ASDF reference manual that there isn't any way to
> load, without compiling, only the index file (I'm looking at "How do I
> mark a source file to be loaded only and not compiled?"). So I'm
> thinking about some other way to locate the .info file. Is there a
> special variable or something which tells the path from which the .asd
> is being loaded? I looked at the documentation and the list of
> external symbols but couldn't puzzle out anything.
>
> The more general idea is that the system contains a non-Lisp data
> file. Is there a general approach to work with that?
>
> Thanks for any light you can shed on this problem.
>
> Robert Dodier
>

The way I've been doing this is with a :STATIC-FILE component (you must
include file extensions).

ASDF will put your static file into the FASL location without touching it
then you can find it to load using ASDF:FIND-COMPONENT. One of the
component's slots will have the pathname of the "compiled" file in it, you
don't have to use *LOAD-TRUENAME*.

>


Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Dodier
Thanks, Robert, that helps a lot. Using ASDF:SYSTEM-RELATIVE-PATHNAME
is a big step forward.

A follow-up question: Is there a way to know, within a file which is
being loaded, whether or not it's being loaded via ASDF? The index
file might be loaded via CL:LOAD also.

Just to clarify the situation a little, the .info is created from
.texi by makeinfo, and then it's parsed (by a Perl script -- OH MY
WHAT HAVE I DONE?? Ha ha, only serious) to get the offset and length
of each documentation item, which is stored in the index, which is
implemented as a Lisp program that reads the .info and builds suitable
hash tables, to be used by Maxima's documentation system at run time.
All that with makeinfo and the info parser happens long before ASDF
comes into the picture. The .info and index.lisp are packaged with the
rest of the code, and that's the package that ASDF sees.

I don't doubt that this systems has flaws, but at this point I am
taking it as a given; I'm trying to make progress on a lot of fronts,
and cleaning up this monstrosity is a lower priority item.

best,
Robert Dodier



Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Dodier
On Fri, Mar 16, 2018 at 11:05 AM, Lucien P  wrote:

> ASDF will put your static file into the FASL location without touching it
> then you can find it to load using ASDF:FIND-COMPONENT. One of the
> component's slots will have the pathname of the "compiled" file in it, you
> don't have to use *LOAD-TRUENAME*.

Hmm, examining the cache after running ASDF, I don't see the static
files there. But I think using the component slot to get the file
location is promising, I will give that a try.

best,
Robert Dodier



Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Goldman

On 16 Mar 2018, at 13:05, Lucien P wrote:

On Fri, Mar 16, 2018, 11:25 Robert Dodier  
wrote:



Hi,
I am resuming my tinkering on the Maxima interface to ASDF. I have
bumped into a kind of minor problem for which I don't see a solution.

Maxima can read .info files for documentation items. An index file
(Lisp) which is created ahead of time (not run time) tells where the
items are in the .info. The .asd, index, and .info are all in the 
same

directory. Some code in the index file tells Maxima to read the .info
from the same place from which the index is loaded (via
*LOAD-PATHNAME*).

That works great when the asd is loaded via LOAD-SOURCE-OP since
*LOAD-PATHNAME* points to the directory where the .info file is. But
when the asd is loaded via LOAD-OP, it doesn't work, since
*LOAD-PATHNAME* points into the place that ASDF puts fasls.

I gather from the ASDF reference manual that there isn't any way to
load, without compiling, only the index file (I'm looking at "How do 
I

mark a source file to be loaded only and not compiled?"). So I'm
thinking about some other way to locate the .info file. Is there a
special variable or something which tells the path from which the 
.asd

is being loaded? I looked at the documentation and the list of
external symbols but couldn't puzzle out anything.

The more general idea is that the system contains a non-Lisp data
file. Is there a general approach to work with that?

Thanks for any light you can shed on this problem.

Robert Dodier



The way I've been doing this is with a :STATIC-FILE component (you 
must

include file extensions).

ASDF will put your static file into the FASL location without touching 
it

then you can find it to load using ASDF:FIND-COMPONENT. One of the
component's slots will have the pathname of the "compiled" file in it, 
you

don't have to use *LOAD-TRUENAME*.






I don't believe a static file should be moved, and if not moved, you 
should be able to load it with something like this;


```
(asdf:system-relative-pathname "maxima" "subdir/my-static-file.blort")
```

To be honest, I don't fully understand the question.  How is the index 
file created, and by what process?  Is it created as side-effect of 
compilation, or is there some special compilation process that creates 
it?  If the latter, then that process could be brought under ASDF 
control, and the code used to create the index file could be configured, 
through the use of output-files configuration, to place the file 
wherever you want.


On the other hand, if the info files are static, and the indices are 
created in the FASL location, then you can just use 
`ASDF:SYSTEM-RELATIVE-PATHNAME` instead of `*load-pathname*` to find the 
file.


On the other other hand, you could use `*compile-file-truename*` instead 
of `*load-pathname*`, but that would have semantics that could vary 
(e.g., if you load some of this stuff interactively, instead of through 
ASDF), so that's not an ideal choice.

Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Goldman

On 16 Mar 2018, at 15:03, Robert Dodier wrote:


Thanks, Robert, that helps a lot. Using ASDF:SYSTEM-RELATIVE-PATHNAME
is a big step forward.

A follow-up question: Is there a way to know, within a file which is
being loaded, whether or not it's being loaded via ASDF? The index
file might be loaded via CL:LOAD also.

Just to clarify the situation a little, the .info is created from
.texi by makeinfo, and then it's parsed (by a Perl script -- OH MY
WHAT HAVE I DONE?? Ha ha, only serious) to get the offset and length
of each documentation item, which is stored in the index, which is
implemented as a Lisp program that reads the .info and builds suitable
hash tables, to be used by Maxima's documentation system at run time.
All that with makeinfo and the info parser happens long before ASDF
comes into the picture. The .info and index.lisp are packaged with the
rest of the code, and that's the package that ASDF sees.

I don't doubt that this systems has flaws, but at this point I am
taking it as a given; I'm trying to make progress on a lot of fronts,
and cleaning up this monstrosity is a lower priority item.

best,
Robert Dodier


Another trick you could try is to define a special class of files for 
the info files (probably just a subclass of `cl-source-file`).  Then you 
could tell ASDF not to move that class of files, by providing a method 
on `OUTPUT-FILES`.  Here's the relevant `DEFGENERIC`:

```
(defgeneric output-files (operation component)
(:documentation "Methods for this function return two values: a 
list of output files
corresponding to this action, and a boolean indicating if they have 
already been subjected

to relevant output translations and should not be further translated.

Methods on PERFORM *must* call this function to determine where their 
outputs are to be located.

They may rely on the order of the files to discriminate between outputs.
"))
```

You can return T for the second value in order to have 
output-translations *not* apply, IIRC.




Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Dodier
On Fri, Mar 16, 2018 at 1:59 PM, Robert Goldman  wrote:

> Another trick you could try is to define a special class of files for the
> info files (probably just a subclass of cl-source-file). Then you could tell
> ASDF not to move that class of files, by providing a method on OUTPUT-FILES.

> You can return T for the second value in order to have output-translations
> not apply, IIRC.

Actually, one way to achieve the effect that I want is to have the
info file copied into the place where the fasls end up (because then
*LOAD-PATHNAME* points to the place where the info file is found, just
as it is when the index.lisp is loaded via LOAD).

I suppose I could create a class that has the effect of copying the
info files without any other operation on them?

best,
Robert Dodier



Re: Finding out from where the .asd was loaded and/or referencing a non-executable data file

2018-03-16 Thread Robert Goldman

On 16 Mar 2018, at 17:52, Robert Dodier wrote:

On Fri, Mar 16, 2018 at 1:59 PM, Robert Goldman  
wrote:


Another trick you could try is to define a special class of files for 
the
info files (probably just a subclass of cl-source-file). Then you 
could tell
ASDF not to move that class of files, by providing a method on 
OUTPUT-FILES.


You can return T for the second value in order to have 
output-translations

not apply, IIRC.


Actually, one way to achieve the effect that I want is to have the
info file copied into the place where the fasls end up (because then
*LOAD-PATHNAME* points to the place where the info file is found, just
as it is when the index.lisp is loaded via LOAD).

I suppose I could create a class that has the effect of copying the
info files without any other operation on them?

best,
Robert Dodier


Yes, for example, you could override the `COMPILE-OP` for this class of 
files to just do the move, instead of being a no-op.


Now that I think of it, if these index files are lisp data (some kind of 
table, right?), couldn't you actually compile them?  If you can use 
`CL:LOAD` on them, couldn't you also use `COMPILE-FILE`?  In which case, 
I think you could just treat them like cl source files, and use `(or 
*compile-file-truename* *load-truename*)`.  I think that would give you 
the behavior you want.


Best,
r