I am pretty much certain that I now understand what happened to cause my asdf
problem. At the end of this I have a few final questions that maybe somebody
can answer definitively for me. If you are curious about the details, read on.
Here are some background facts:
What I was seeing is a failure of asdf to locate a new system in my
~/common-lisp directory as it should.
This problem only occurred in the CCL IDE, not when running CCL from the
command line.
There is an asdf internal variable called
asdf/source-registry:*source-registry* which contains a list of all the .asd
files it can find as a result of searching all of the directories where asdf
thinks they might be. In my case that was my ~/common-lisp/ directory. In asdf
itself.
That variable is only initialized when a file is searched for. That might
happen either via an explicit asdf call (e.g.(asdf::find-system <something>) or
indirectly via a quickload call such as (ql::quickload <something>).
Just loading asdf will not cause the *source-registry* to be initialized
However, it turns out that just loading quickload WILL cause the asdf
*source-registry* variable to be initialized.
Once that variable is initialized, asdf will never try to initialize it again
since that would waste a lot of time looking for .asd files all over again.
So knowing all that, here is what happened.
I installed and rebuilt the latest CCL a couple of months ago.
As part of building the CCL IDE after that, both asdf and quickload were loaded
into the CCL IDE image that was created (one question later is related to this)
As a normal result of loading quickload, the
asdf/source-registry:*source-registry* variable was initialized (another
question below is related to this)
When the CCL IDE image was built it included the value of that asdf
*source-registry* variable, so that when the IDE was launched that variable was
initialized.
Only recently did I add additional asdf-defined systems to the ~/common-lisp/
directory. But of course when asdf tried to find them it couldn’t because the
*source-registry* variable was effectively initialized when the IDE was built
(long before I added my new systems).
I can explicitly call an asdf function to reinitialize that variable and then
everything is found normally.
One question I have left is whether loading asdf and quickload into the CCL IDE
is something that is done explicitly by the process that is now used to build
the IDE or whether there is some option to not include them. Was there
something that I should have done or not done when building the IDE that would
have prevented this problem?
Had there been nothing in my ~/common-lisp directory to find, then the value of
that asdf variable would have been nil and that is enough to make asdf
reinitialize it. But assuring that each time I build seems like a pain.
My next question is why loading quicklisp causes it to initialize that
variable? Conceivably it needs to search for .asd files that might be in its
load environment and uses asdf to do so.
Next I’m wondering whether there is some change to asdf code that can be made
to reinitialize the *source-registry* every time the IDE is re-started.
As another possibility, perhaps the CCL code for dumping the IDE can be
modified in some way. In a private email Robert suggested:
"If you have time to look, take a peek at asdf/uiop/image.lisp and
asdf/bundle.lisp which should have code Fare wrote about dumping an image in a
way that it will restart happily.”
That seems like something that CCL maintainers might want to consider.
While waiting for fixes or other suggestions, I will put an explicit call to
reinitialize the *source-registry* in my ccl-ide-init.lisp file:
(asdf/source-registry::initialize-source-registry)
If anyone else has an environment similar to mine and rebuilds their own
version of the IDE, I would suggest they do the same.
