Re: How to use the current directory?
> I use a vanilla Quicklisp installation. For some local directory > trees I configure ASDF to prefer those trees over the standard > Quicklisp versions by placing text like the following in my > $HOME/.config/common-lisp/source-registry.conf file: > > (:source-registry > (:tree "/home/brown/toe/open-source/") > ;; For swank.asd > (:tree "/home/brown/local/software/source/slime/") > :inherit-configuration) +1 for this as the simplest path to the desired outcome. > ./configure > make > sudo make install > > and it works! I can put the clone anywhere I want. I can change it > anyway I want. The system doesn't fight me. that system is a struggle in so many other ways (dll hell; all the /usr/include stuff; etc...). and from all the various linux distros, precious few can achieve the equivalent of the CL ecosystem WRT reproducibility and fine-grained control over the versions of the dependencies (NixOS and Guix are the ones i'm aware of). the seeming simplicity of that ./configure dance has an enormous cost that only shows up when it doesn't work in apparent or in covertly surprising ways. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In his dealings with the world, the gentleman is not invariably for or against anything. He is on the side of what is moral.” — Confucius (551–479 BC)
Re: How to use the current directory?
> So my advice is: > > (push #P"/path/to/your/local/copy/" ql:*local-project-directories*) > (ql:quickload :your-system) alternatively, you can move or symlink your project into ~/quicklisp/local-projects/ -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “No people will tamely surrender their Liberties, nor can any be easily subdued, when knowledge is diffused and virtue is preserved. On the contrary, when people are universally ignorant, and debauched in their manners, they will sink under their own weight without the aid of foreign invaders.” — Samuel Adams (1722–1803)
Re: [RFC] subsystems vs. modules
using :if-feature is not deterministic (it depends on e.g. the load order of systems, unless fixed, see Robert's mail). using subsystems that explicitly record dependencies is deterministic. we once used asdf-system-connections, and the non-determinism that it introduced was a source of long hours of headaches... once we moved on to use subsystems and system names like :mylib+swank, it was all gone. - attila
Re: :initform vs. :default-initargs
i think i know why this is happening. i have attached a test that has the two versions that i have talked about. the slot override works, but the default-initarg doesn't. ASDF probably calls change-class, probably from reset-system-class, which invokes reinitialize-instance directly, and somewhere here the default initargs are ignored. but my CLOS is rusty. is this a user bug, or an ADSF bug? - attila test-system-default-initargs.script Description: Binary data
Re: :initform vs. :default-initargs
> Yes, this works as expected for me: > > (defclass foosys (asdf/system:system) > () > (:default-initargs :default-component-class 'foo)) > > (asdf/component:module-default-component-class (make-instance 'foosys)) => foo indeed. after Robert's mail i also tried this, and it worked, yet i'm seeing the error when i use :default-initargs, but not when i'm using a slot override. it's a more convoluted issue involving our entire codebase that i'll need to investigate. sorry for the noise! - attila
:initform vs. :default-initargs
dear list, i wanted to set the default-component-class of our own system subclass using :default-initargs, but it's ignored because the slot has a :initform (as opposed to setting the default value using a :default-initargs entry). in the current ASDF setup, i need to override the slot (or the accessor method), which involves messing with internal symbols. if ASDF used :default-initargs then i could just override that using a :default-initargs entry on my own subclass (IIUC). https://lisptips.com/post/11728375873/initform-and-default-initargs i'm sending it for your consideration, it's not a pressing issue on my side. - attila PGP: 5D5F 45C7 DFCD 0A39
Re: Rejiggering the branches
> > Nah, a tag is supposed to never change. The mechanism for a "tag that > changes" is called... a branch. the user story that i desire is a label that someone with the commit bit can move around freely, and then it gets automatically synchronized to everyone else who pulls/fetches the repo (i.e. *without* reading multiple blog posts and issuing multiple git commands). a release branch, especially with force-pushed commits, is rather far from that in terms of the user experience. but then maybe i should learn to love merge commits... now, how exactly git behaves is another story. it never seizes to surprise me. (i'm fine with calling the "stable" label as "release", or whatever else. the actual name wasn't my point.) - attila
Re: Rejiggering the branches
> > > Would the "stable" branch be any different from the "release" branch? > If it's actually a not-so-stable development branch for 3.3 while a > separate branch contains development for 3.4, then maybe indeed > calling branches v3.3 and v3.4 make more sense. > > +1 what i would do: - one branch that holds the bleeding edge. i'd call it main, just to go with the flow. - branches for ASDF versions (down to the desired resolution, probably major.minor), so that you can easily cherry pick or backport fixes into them. a new version-branch is forked off of main whenever a release happens. - optionally a stable *tag* as an indirection to the latest release. it communicates which specific git revision is it that the maintainer considers the stable state at any moment in time. it comes handy e.g. in CI scripts that want to check out the latest ASDF release, etc... note though that this last point requires force-pushing the stable tag, which i have done before, but i'm not completely sure it results in a slick workflow. the main question is whether or not a git fetch/pull silently and automatically updates the tag in the local repo. - attila
Re: only perfom when the output is missing
thanks for the ideas Robert! > I was wondering why it was important to avoid the build when the .h file > is newer than the .spec file. This shouldn't happen very often, should > it? Is it because: > the problem is much more down to earth: generating the spec file requires launching a libllvm based external tool called c2ffi. compiling and properly configuring this tool can sometimes consume a day, and the users of e.g. hu.dwim.sdl shouldn't ever need to do that. it's only needed when you want to capture newly added artifacts from the SDL.h files. one way to avoid this could be to not communicate the dependency to ASDF, and make the .h -> .spec generation manual (this used to be the case). for some reason that feels lame, but not the end of the world either. >1. > >The .h file is getting refreshed too often by something upstream of >ASDF? E.g., is make being too eager about rewriting the .h file? > > the problem is not with too often, but with anything greater than zero, e.g. a git checkout. - attila
Re: only perfom when the output is missing
FTR, i have managed to solve it with this: (defclass generate-lisp-op (selfward-operation) ((selfward-operation :initform '( ; we will specify it in our own COMPONENT-DEPENDS-ON (defmethod component-depends-on ((op generate-lisp-op) (c c2ffi-file)) `((load-op ,(find-system "cffi/c2ffi-generator")) ;; Regenerating the spec file is a lot of headache, so we ignore ;; the file modification times, and only communicate the ;; dependency to ASDF if the spec file is missing. ,@(let ((spec-file (input-file op c))) ; TODO is it legal to call ASDF:INPUT-FILES here? (when (or (not (probe-file spec-file)) (zerop (with-input-from-file (s spec-file) (file-length s `((generate-spec-op ,c ,@(call-next-method))) the commit: https://github.com/cffi/cffi/commit/46975c644aeb0a832cd170e953cc8ec3cbdc78df - attila PS: sorry for the double send, i have cleaned up my mail settings.
only perfom when the output is missing
dear hackers, i cannot convince ASDF to only perform my custom operation if the output file doesn't exist; i.e. regardless of the modification times. it's in cffi/c2ffi: foo.h -> foo.spec -> foo.lisp when the spec is there, regardless of the mod time, i'd like to skip the spec generation task. neither reading the manual, nor grepping around helped me to come up with the required magic. my custom operation-done-p isn't even called if the file's mod time instructs ASDF to run the operation. any hints are appreciated, including pointers to examples or tests. - attila PS: i sent the same mail from the wrong account, don't allow that through moderation please.
only perfom when output is missing
dear hackers, i cannot convince ASDF to only perform my custom operation if the output file doesn't exist; i.e. regardless of the modification times. it's in cffi/c2ffi: foo.h -> foo.spec -> foo.lisp when the spec is there, regardless of the mod time, i'd like to skip the spec generation task. neither reading the manual, nor grepping around helped me to come up with the required magic. my custom operation-done-p isn't even called if the file's mod time instructs ASDF to run the operation. any hints are appreciated, including pointers to examples or tests. - attila
Re: trouble with upgrade of asdf and versions
> Here is the problem, (and Quicklisp seems to be working OK) > > > CL-USER> (ql:quickload :quickproject) > ; Evaluation aborted on #. if you want to get help you'll need to copy-paste the actual error and the backtrace. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- No man is free who is not master of himself.
Re: Shims for old systems
> simple and sustainable is a key to survival. In asdf I see the > accumulation of complexity with no end. That is bad generally. old ASDF was broken in countless ways, and i've experienced the bugs in my everyday activities. the question is whether ASDF has any accidental complexity, i.e. complexity that is not inherent to the problem domain, and thus could be eliminated -- at least theoretically. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Where there are rulers, there are no rules, only interest-serving opinions enforced by violence. To want rulers is to beg for chaos.” — Brandon Roark
Re: Shims for old systems
> The shims idea is interesting, but isn't it a lot of trouble to make and > maintain a shim (which has to somehow indicate where to find the source files > for the library), compared to forking a system? +1 there's already the sharplispers group for taking over unmaintained systems (not to be confused with hostile systems): https://github.com/sharplispers -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Tact is a skill that can turn brutal honesty into just honesty. It's a skill that develops with practice, and one that's harder to use when emotions are running high. But you can't go towards someone with a verbal fist and expect a hug in return. When method matches intention, outcomes are much more peaceful.” — Doe Zantamata
Re: why does ASDF ask to please only define system/test?
sorry Stas, but i just had to dishonestly (?) cut your mail... :) with the hope that it will incite some reflection: > I specifically don't update cl-ppcre.asd [...] > > [...] and all the users can do is just to suck it up. FTR, there's an open PR with discussion: https://github.com/edicl/cl-ppcre/pull/30 i understand that "it was him who made the first change!", and "everything would still be fine if he doesn't make a change!"... but i'm not sure it's reasonable to expect this kind of conservatism in a collaborative space. old ASDF was a piece of software that wasn't designed for the tasks it is used for today (as in its API, let alone its implementation), and that was causing a lot of headache to "the users" you seem to be defending above. i don't have a strong opinion about this specific warning. to be honest, if i was the ASDF maintainer, it would be fine for me if the warning was off by default, and i would only turn it on in my own CI to send out the PR's and/or warnings to the relevant maintainers, and then let old and/or conservative libs continue to misbehave as they did with the old ASDF. but it's quite disheartening to see all the trash-talk against the refactored ASDF. i may be just one user, but the cleanup of ASDF helped me tremendously! maybe i was using more deps than usual (~100), and had to deal with live systems and whatnot that most CL users don't do...? but prior to the ASDF cleanup i had wasted countless hours hunting build-related bugs only to realize that there wasn't any bug (in my code), and by getting used to mindlessly rebuilding from scratch *any time* something strange happened. conservatism, and the general hostility, has always troubled me in the CL community, but i don't have a dog in this fight currently, or maybe even anymore, so i better cut the talk... good luck resolving whatever needs to be resolved for a better tomorrow! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The more artificial taboos and restrictions there are in the world, the more the people are impoverished…. The more that laws and regulations are given prominence, the more thieves and robbers there will be.” — Lao Tzu (sixth century BC)
Re: why does ASDF ask to please only define system/test?
> What great functionality are we getting for this? you can find and load a system without pre-processing every .asd file available. > and have worked perfectly well without following the new convention? was it, though? it took multiple seconds to read and process every .asd on my system before this change. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In a democracy, mass opinion creates power. Power diverts funds to the manufacturers of opinion, who manufacture more, etc. […] This feedback loop generates a playing field on which the most competitive ideas are not those which best correspond to reality, but those which produce the strongest feedback.” — Mencius Moldbug
Re: Problems with DEFSYSTEM-DEPENDS-ON
> A cheesy fix would simply be to wrap it in IGNORE-ERRORS. But it might cause > errors in its present form. i've learned, painfully, that indiscriminate ignore-errors will almost always bite you back (in the form of wasted debugging time), no matter how innocent they look. > This really is more a QL issue than an ASDF one (although it illustrates an > issue with wrapping errors, IMO). while this is true, the implementation of QL requires a reliable way to hook into the internal state of various versions of ASDF (namely, into the situation when ASDF is looking for a system, and not finding it will lead to an error without QL intervention). if i understand it correctly, this is the crux of this issue. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “A general State education is a mere contrivance for molding people to be exactly like one another; and as the mold in which it casts them is that which pleases the dominant power in the government […]; it establishes a despotism over the mind, leading by a natural tendency to one over the body.” — John Stuart Mill (1806–1873), 'On Liberty' (1869)
Re: Problems with DEFSYSTEM-DEPENDS-ON
> The usage of DEFSYSTEM-DEPENDS-ON to specify dependencies that will be > satisfied by QL:QUICKLOAD no longer seems to be working in asdf-3.3.1. FTR, here's the history of this issue: https://github.com/quicklisp/quicklisp-client/pull/122#issuecomment-160419822 https://github.com/quicklisp/quicklisp-client/issues/108 -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “When a country is well governed, poverty and a mean condition are things to be ashamed of. When a country is ill governed, riches and honor are things to be ashamed of.” — Confucius (551–479 BC), 'The Analects', Chapter VIII (邦有道貧且賤焉恥也，邦無道富且貴焉恥也。)
Re: "deprecated recursive use" warning
> I think the Right Thing is to realize that what ASDF does is not so much > transform files, as to maintain the consistency of the running lisp image. > Now, in no case does the running lisp image need either the JSON generation > system or the JSON to lisp translation system. All the running lisp image > needs to function correctly is an up-to-date lisp file produced by this > pipeline. So this is a very useful insight, thanks for formulating it explicitly! it's a good candidate for an introductory bird's eye view section of the manual. i already made the JSON generation a separate operation, and now i'm considering to do the same with the JSON -> lisp generation. in a way it makes sense: the lib author tested the code that he saw. if the lisp generation gets changed/fixed (by a new patch to CFFI), it may even break the rest of the lib that may be running with some old assumptions... so, maybe it's better to just make both of those as a separate process that needs to be initiated by the lib author explicitly, and then checked into the git history. > Once you think about what ASDF does and doesn't do, I think this makes > perfect sense. But, of course, it might be a big pain to do so. yeah, it feels like a lot of pain. it would be nice if there was a fork-like API in ASDF for implementing such exec'd compilation, but then i guess ASDF itself has no clue which /foo/bin/ directory has the underlying lisp binary, and/or which lisp to launch at all, let alone how... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Far from helping students to develop into mature, self-reliant, self-motivated individuals, schools seem to do everything they can to keep youngsters in a state of chronic, almost infantile, dependency. The pervasive atmosphere of distrust, together with rules covering the most minute aspects of existence, teach students every day that they are not people of worth, and certainly not individuals capable of regulating their own behavior.” — Alfie Kohn, 'Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes'
Re: Testing for ASDF 3.3.2 and beyond?
>> There is no batch loading, each test is run in a fresh CL process. >> > Interesting. The memory load might still be sensitive to compilation > order, if you reuse fasls shared with previous builds versus rebuild > new ones. it can help on SBCL to insert (sb-ext:gc :full t) to a few key places, like after operations that generate a lot of garbage (e.g. compilation). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Ultimately, man should not ask what the meaning of his life is, but rather must recognize that it is *he* who is asked. In a word, each man is questioned by life; and he can only answer to life by *answering for* his own life; to life he can only respond by being responsible.” — Viktor E. Frankl (1905–1997), 'Man's Search for Meaning' (1946)
Re: hu.dwim.zlib broke; broken operation-done-p
> If you generate the Lisp files, do you need to check in the .spec files? there's a strong argument for keeping the spec files: if the cffi/c2ffi generator itself gets a bug fix/upgrade, then the lisp files would need to be regenerated for those fixes/upgrades to take effect, and that's only possible if the spec files are available. which, BTW, currently happens automagically due to that unlucky component -> system dependency on the "cffi/c2ffi-generator" system. thanks for the review! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- The real voyage of discovery consists not in seeking new landscapes, but in looking with new eyes.
Re: hu.dwim.zlib broke; broken operation-done-p
> One solution is to create a new file with the correct timestamp, that > is either a copy of the existing spec file or a new generated one. ok, here's an idea that i'd be grateful if i could bounce off of you: what if i make the .spec generation a standalone operation that needs to be explicitly run by the library author? it's a bit more burden for the lib author because he needs to keep track of changes to the configuration and/or the .h file, but that would implicitly fix one of the bugs. in fact i have already pushed this change: https://github.com/cffi/cffi/commit/d59331b050fb31f3a3f9103f302890be176b77ce and shall i do the same with the next stage, namely the .lisp file generation? for some reason i think it's better to keep that automatic and integrated into the normal flow of the build process, but i'm not fully convinced about that either. if i did that, it would annull the last remaining issue, namely the current unlucky (load-op ,(find-system "cffi/c2ffi-generator")) dependency. but the price would be high: either users would need to regenerate .lisp files by hand each time they pull changes to the .spec file (this is unacceptable), or all libs would need to check in also the generated .lisp files into their repos (this is less of a constraint/burden, but still...). any thoughts? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Education is life. Life is education.
Re: hu.dwim.zlib broke; broken operation-done-p
> My theory is that that you're failing to re-generate the .spec when > it's present, even though your dependencies say it will be > re-generated. Therefore, ASDF decides that it's out-of-date and must > be re-generated again the next time over, etc. In other words, you lie > to ASDF, and ASDF punishes you right back. that was indeed a lurking bug. excellent remote debugging skills, thanks! :) i pushed the fix. now after i touch'd the appropriate spec files, it doesn't want to run c2ffi anymore, but it's still regenerating the lisp file unconditionally. i ran out of hacking steam for today. i'll look into it tomorrow, but further remote debugging magic is welcome of course... :) i'm kinda lost for now. i have no plan of action, so to say... :) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Liberty is a demand. Tyranny is submission.
hu.dwim.zlib broke; broken operation-done-p
if you issue the following (available in quicklisp): (asdf:load-system :hu.dwim.zlib) then for the first time it should generate a lisp file, which then gets compiled and loaded. issuing it for the second time shouldn't do anything, but since some revisions it regenerates the lisp file every time. the relevant code is somewhere around here: https://github.com/cffi/cffi/blob/master/src/c2ffi/asdf.lisp#L135 i tried to trace the INPUT-FILES, OUTPUT-FILES, PERFORM, OPERATION-DONE-P methods, but i don't see anything wrong. one unusual thing is that the output of GENERATE-LISP-OP goes into the src/ directory, not to the usual fasl output dir. same applies to GENERATE-SPEC-OP. any ideas how to debug this? or what to look at? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “For an instant I think I saw. I saw the loneliness of man as a gigantic wave which had been frozen in front of me, held back by the invisible wall of a metaphor.” — Carlos Castaneda (1925–1998), 'Journey to Ixtlan' (1972)
Re: "deprecated recursive use" warning
> I haven't used CFFI in a while. TL;DR: is this a sane fix? https://github.com/cffi/cffi/commit/4b9b06f15912e823581b1aeb8a0d5c2ef11f702d -- and here follows the elaborate email that led me to find the above solution: a bit of background: it's a subsystem of CFFI that generates the CFFI bindings from a json file, that is in turn autogenerated from the C source. so, 1) C -> json 2) json -> lisp (CFFI definitions) 3) asdf compiles/loads the generated lisp file 1) requires a heavyweight infrastructure (a binary run by run-program), so there's support to do that lazily, and just distribute the json files generated once by the lib author. 2) is relatively lightweight, but it still requires loading a broader scope of lisp dependencies, so there's support for the lib author to run the generation and distribute the generated lisp file. now, whether the generator code (extra dependencies) is needed is decided by whether or not the generated lisp file is up-to-date. unfortunately i cannot test it properly because of another bug/change that i'll report in a separate thread. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Enlightenment is a destructive process. It has nothing to do with becoming better or being happier. Enlightenment is the crumbling away of untruth. It's seeing through the facade of pretence. It's the complete eradication of everything we imagined to be true.” — Adyashanti (1962–)
"deprecated recursive use" warning
dear list,
shall i be concerned about this?
WARNING:
Deprecated recursive use of (ASDF/OPERATE:OPERATE
'ASDF/LISP-ACTION:LOAD-OP '("cffi/c2ffi-generator")) while
visiting (CFFI/C2FFI::GENERATE-LISP-OP "hu.dwim.zlib" "c2ffi-spec"
"zlib.h") - please use proper dependencies instead
it happens due to this in cffi/src/c2ffi/asdf.lisp:
(defmethod perform ((op generate-lisp-op) (c c2ffi-file))
...
(unless (component-loaded-p :cffi/c2ffi-generator)
(load-system :cffi/c2ffi-generator))
...
)
that system only needs to get loaded when the action is executed.
or is it safe to ignore that warning long term?
--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“If we have the truth, it cannot be harmed by investigation. If we
have not the truth, it ought to be harmed.”
— J. Reuben Clark (1871–1961)
Re: spurious reloads with systems not following the foo/bar naming convention?
> I believe this is the bug that was fixed in 3.3.1.3. FYI, there's no such tag pushed into the official repo. https://gitlab.common-lisp.net/asdf/asdf/tags using HEAD, the test case i sent doesn't fail anymore, but our own issue still prevails. it's a large environment with many systems, but the symptom looks the same, although for us due to cxml. i couldn't easily test with an older version of ASDF because my SBCL already ships with 3.3.0.1 (preventing an easy downgrade), but it used to work until recently. after i edit cxml.asd to follow the foo/bar convention, i see a few of these, that may or may not be the cause of the spurious reload that i still see: WARNING: Computing just-done stamp in plan NIL for action (ASDF/LISP-ACTION:COMPILE-OP "iolib/syscalls" "ffi-types"), but dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/syscalls" "ffi-types") wasn't done yet! WARNING: Computing just-done stamp in plan NIL for action (ASDF/LISP-ACTION:COMPILE-OP "iolib/os" "ffi-types"), but dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/os" "ffi-types") wasn't done yet! WARNING: Computing just-done stamp in plan NIL for action (ASDF/LISP-ACTION:COMPILE-OP "iolib/sockets" "grovel"), but dependency (ASDF/LISP-ACTION:PREPARE-OP "iolib/sockets" "grovel") wasn't done yet! please note that this is not a burning issue for us, it's just a friendly heads up with the hopes that it's useful. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “A true friendship involves two people who are both intensely focused on their own healing processes, on working on their own issues and maturing and evolving. Basically two people who are growing toward truth, and they both individually, in their own specific individual lives, have that as their purpose. And when they come together it becomes synergistic.” — Daniel Mackler
spurious reloads with systems not following the foo/bar naming convention?
dear list, i noticed it with CXML, but here's a smaller system to reproduce it with: git clone https://github.com/mmaul/cl-influxdb.git (ql:quickload :cl-influxdb) (asdf:load-system :cl-influxdb.examples) each time the examples are loaded the main systems gets reloaded. i see something interesting in the output that may be related: WARNING: Computing just-done stamp in plan NIL for action (ASDF/LISP-ACTION:PREPARE-OP "cl-influxdb" "cl-influxdb"), but dependency (ASDF/LISP-ACTION:LOAD-OP "cl-influxdb" "package") wasn't done yet! CL-USER> (asdf:asdf-version) "3.3.1.1" CL-USER> (lisp-implementation-version) "1.4.1.98.hu.dwim.3-a860f4780" -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “With or without religion, good people can behave well and bad people can do evil; but for good people to do evil — that takes religion.” — Steven Weinberg (1933–), 'A Designer Universe?'
Re: weirdness with compile-op and retry restart
> Would you perchance be using deferred-warnings? yes. i asked myself: what exactly (asdf:enable-deferred-warnings-check) does? i couldn't answer. then i didn't find anything in the manual, and a brief inspection of the code also didn't satisfy my curiosity... so, i guess i'll just turn it off then... :) thanks for the info, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “I'm not scared of the Maos and the Stalins and the Hitlers. I'm scared of the thousands of millions of people that hallucinate them to be "authority", and so do their bidding, and pay for their empires, and carry out their orders. I don't care if there's one looney with a stupid moustache. He's not a threat if the people do not believe in "authority".” — Larken Rose (1968–)
weirdness with compile-op and retry restart
dear devs, i have a system with a file that has a compilation error. i try to load it: (asdf:load-system :hu.dwim.web-server.application.test) it leads to an error with the following restart (notice its unusual wording, and that it's about the entire system, not just that one source file): 6: RETRY Retry completing compilation for #. the rest FTR: 5: ACCEPT Continue, treating completing compilation for # as having been successful. 4: RETRY Retry ASDF operation. 3: CLEAR-CONFIGURATION-AND-RETRY Retry ASDF operation after resetting the configuration. if i touch the file and select the restart, then cl:compile-file doesn't get called and no compilation seems to happen. there used to be a restart to recompile the single source file that has failed. i'm on ASDF 3.2.1 and the very latest SBCL (i'm testing it for the current freeze) note: these systems have some customizations from hu.dwim.asdf, but not much changed there recently. is this something obvious that i'm failing to see, or should i dig deeper? what else would help? it's not a show-stopper for me, so feel free to ignore this. i'm writing this up more as offering help than asking for help. thanks for your time, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Understand and abide by the following self-evident principle, and you will be an anarchist: If there is something that would be wrong for you to do yourself, then you shouldn't ask anyone else to do it either.” — Larken Rose (1968–)
Re: Plans for ASDF 3.3
> Also, I know you are interested in parallel variants of ASDF. It seems > like this would conflict with the desire to enforce deterministic > orderings, since if you have A must be before B, C, and D, but B, C, and > D are unordered in the system definition, determinism would lead you to > want to have a deterministic algorithm for ordering B, C, and D, but > parallelism would suggest you separately compile B, C, and D in > different images (each of which would have A loaded). while we are at it, i would welcome a change/feature where anything unspecified would be randomly ordered. so that issues come up while developing, not when the live system has a different filesystem order and boom, things just don't build... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Political speech and writing are largely the defense of the indefensible.” — George Orwell (1903–1950), 'Politics and the English Language' (1946)
Re: ASDF 3.2.1 released
> It is with great pleasure that we announce the release of ASDF 3.2.1. $ git checkout 3.2.1 error: pathspec '3.2.1' did not match any file(s) known to git. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If there is to be peace in the world, There must be peace in the nations. If there is to be peace in the nations, There must be peace in the cities. If there is to be peace in the cities, There must be peace between neighbors. If there is to be peace between neighbors, There must be peace in the home. If there is to be peace in the home, There must be peace in the heart.” — Lao Tzu (sixth century BC)
Re: ASDF release candidates
> That would have been worthy of an official bug report. wait, let me be more precise, because i may be wrong here: i vaguely remember that in the past i used to see redefinition warnings issued by sbcl regarding uiop, and it was in a context where it went against my model of reality, i.e. as far as i understood what was going on it shouldn't have happened. actual compilation may or may not have happened, i don't remember. i didn't file a bug because i couldn't easily reproduce it, and/or i wasn't even sure that whatever was happening was wrong. sorry for the fog, but it was some time ago, and it wasn't a crucial issue for me. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Anarchists did not try to carry out genocide against the Armenians in Turkey; they did not deliberately starve millions of Ukrainians; they did not create a system of death camps to kill Jews, gypsies, and Slavs in Europe; they did not fire-bomb scores of large German and Japanese cities and drop nuclear bombs on two of them; they did not carry out a ‘Great Leap Forward’ that killed scores of millions of Chinese; they did not attempt to kill everybody with any appreciable education in Cambodia; they did not launch one aggressive war after another; they did not implement trade sanctions that killed perhaps 500,000 Iraqi children. In debates between anarchists and statists, the burden of proof clearly should rest on those who place their trust in the state. Anarchy’s mayhem is wholly conjectural; the state’s mayhem is undeniably, factually horrendous.” — Robert Higgs
Re: ASDF release candidates
> Oh, you mean you had this bug before 3.3 already? That's a useful data > point! yes, i already used to see unnecessary rebuilding of uiop several months ago. getting rid of explicitly depending on uiop resolved this issue for me. i'm not sure whether explicitly depending on asdf also triggered this, but based on your advice that i read in some bug report somewhere, i also got rid of those dependencies. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The more retarded humanity is, the more it is in the hands of the politicians, in the hands of the priests, in the hands of all kinds of vested interests. Who is interested in transforming man? They want you to be completely blind and deaf. They want you to be just a robot: efficient, not creating any trouble – no strike, no protest, no revolution, no rebellion – just a robot who is always ready to say "Yes, sir."” — Osho (1931–1990), 'From Darkness to Light' (1985)
Re: ASDF release candidates
> I still haven't inspected the spurious rebuilds of uiop that happened > in some contexts. FWIW, i also see those rebuilds every now and then if i have explicit dependency on :uiop or :asdf (which i've gotten rid of, so i don't see this anymore). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Strong minds discuss ideas, average minds discuss events, weak minds discuss people.” — Socrates (c. 470–399 BC, tried and executed)
Re: ASDF release candidates
> Failures on new version only: > https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-62.html it's very interesting, babel on ccl fails with this: QUICKLISP-CLIENT:SYSTEM-NOT-FOUND : System "hu.dwim.stefil-api-impl" not found what is this -api-impl thing? rgrep "api" babel/ and hu.dwim.stefil/ gives nothing. rgrep -- "api-impl" quicklisp-* yields also nothing. where is this coming from? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Once the fabric of a just society is undone, it takes generations to weave it back together.” — Deepak Chopra
Re: ASDF 3.1.7.43 is 3.2 release candidate
> Sorry for misclicking on my phone. What I meant is that SBCL 1.1.10 is over > three years old. Should we really be supporting anything older than two or > three years? i just wanted to write that since then there were several changes to that infrastructure in SBCL, and it's most probably fixed long ago and safe to ignore. i got carried away looking into this, probably for no good reason. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Step into the fire of self-discovery. This fire will not burn you, it will only burn what you are not.” — Mooji (1954–)
Re: ASDF 3.1.7.43 is 3.2 release candidate
uicklisp $ rlwrap sbcl-1.1.10-x86-64-linux/run-sbcl.sh --no-userinit This is SBCL 1.1.10, an implementation of ANSI Common Lisp. * (load "~/quicklisp/setup.lisp") debugger invoked on a SB-INT:INVALID-ARRAY-INDEX-ERROR in thread #: Index 22 out of bounds for (AND (VECTOR T 10) (NOT SIMPLE-ARRAY)), should be nonnegative and <10. Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [RETRY] Retry EVAL of current toplevel form. 1: [CONTINUE ] Ignore error and continue loading file "/media/store/work/sbcl-1.1.10-x86-64-linux/contrib/sb-posix/sb-posix.asd". 2: [ABORT] Abort loading file "/media/store/work/sbcl-1.1.10-x86-64-linux/contrib/sb-posix/sb-posix.asd". 3: [RETRY] Retry compiling #. 4: [ACCEPT ] Continue, treating compiling # as having been successful. 5: Retry ASDF operation. 6: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration. 7: Retry EVAL of current toplevel form. 8: Ignore error and continue loading file "/home/alendvai/quicklisp/setup.lisp". 9: Abort loading file "/home/alendvai/quicklisp/setup.lisp". 10: Exit debugger, returning to top level. (SB-C::FIND-SOURCE-ROOT 22 #) 0] hth, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “All authority of any kind, especially in the field of thought and understanding, is the most destructive, evil thing. Leaders destroy the followers and followers destroy the leaders. You have to be your own teacher and your own disciple. You have to question everything that man has accepted as valuable, as necessary.” — Jiddu Krishnamurti (1895–1986), 'Freedom from the Known'
Re: Work-around built-in ~/common-lisp/ in search path?
> I could do that, but I that would also delete > > system-source-registry > system-source-registry-directory and > default-system-source-registry > > ... which I feared might cause me access to bundled SBCL libraries. i wanted to answer that you shouldn't be afraid, because it's implemented through an entry in asdf:*system-definition-search-functions*, but then i saw that it's not anymore, and i couldn't even find out in 5 minutes how ASDF finds SBCL contribs. nevertheless, i have this in my ~/.config/common-lisp/source-registry.conf: (:source-registry (:also-exclude "sbcl" "disabled-systems" "build") (:tree (:home "common-lisp/")) (:tree "/usr/local/share/common-lisp/source/") :ignore-inherited-configuration) and the SBCL contribs are found. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The greatest fear in the world is of the opinions of others. And the moment you are unafraid of the crowd you are no longer a sheep, you become a lion. A great roar arises in your heart, the roar of freedom.” — Osho (1931–1990)
Re: Version 3.1.7.35 has been pushed
another data point: it was caused by asdf-system-connections. i thought i got rid of that long ago, but it's included in ql, and it got installed on my system somehow. after (ql:uninstall :asdf-system-connections) my script continues, but then fails later on with iolib's IOLIB/GROVEL::PROCESS-OP. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Most people think they need a ruler. Perhaps we should give them a fake one that doesn't actually do anything, and then they won't think about it. It is sort of like giving an infant a pacifier.” — Perry Metzger
Re: Version 3.1.7.35 has been pushed
> Please try it out. Working towards a release of ASDF soon, so anything > you could do to help iron out bugs would be very much appreciated! i did a thoughtless git checkout and rebuild, and i got the attached backtrace. the issue is with the new ban on operation instances. note that it seems to happen during an upgrade of ASDF itself, which may or may not be relevant. maybe for a while it should only be a full warning? my script is basically a build script for an executable SBCL image that loads all dependencies of our projects except our projects themselves. the dependencies come from the a mixture of the latest of quicklisp and my workspace. hope this helps, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Nobody in the world, nobody in history, has ever gotten their freedom by appealing to the moral sense of the people who were oppressing them.” — Assata Shakur (1947–) ;;; loading the following systems into this image: (alexandria cffi uiop trivial-features babel cffi/c2ffi asdf cffi-libffi cffi-grovel cffi-toolchain cl-autowrap cl-json cl-ppcre cl-plus-c trivial-garbage projectured.sdl projectured.document parse-number projectured.editor s-xml projectured.projection drakma puri cl-base64 chunga trivial-gray-streams flexi-streams usocket sb-bsd-sockets cl+ssl sb-posix bordeaux-threads babel-streams cl-fad iolib iolib/base iolib/common-lisp split-sequence iolib/multiplex iolib/syscalls iolib/grovel iolib/asdf iolib/conf iolib/streams iolib/sockets idna swap-bytes iolib/pathnames iolib/os local-time rfc2109 rfc2388-binary contextl closer-mop lw-compat cl-graph metatilities-base cl-containers metabang-bind cl-l10n cl-l10n-cldr iterate cxml cxml-dom cxml-xml closure-common cxml-klacks cxml-test cl-typesetting cl-pdf zpb-ttf ironclad sb-rotate-byte nibbles anaphora sb-sprof cl-postgres+local-time cl-postgres md5 moptilities metacopy-with-contextl metacopy command-line-arguments cl-smtp sb-cover optima) To load "alexandria": Load 1 ASDF system: alexandria ; Loading "alexandria" debugger invoked on a ASDF/FIND-SYSTEM:FORMATTED-SYSTEM-DEFINITION-ERROR in thread #: OPERATION instances must only be created through MAKE-OPERATION. Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [RETRY] Retry ASDF operation. 1: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration. 2: [ABORT] Give up on "alexandria" 3: [RETRY] Retry EVAL of current toplevel form. 4: [CONTINUE ] Ignore error and continue loading file "/home/alendvai/common-lisp/hu.dwim.environment/bin/build-development-image.sh". 5: Abort loading file "/home/alendvai/common-lisp/hu.dwim.environment/bin/build-development-image.sh". 6: Ignore runtime option --eval "(with-open-file (s \"/home/alendvai/common-lisp/hu.dwim.environment/bin/build-development-image.sh\" :element-type 'character) (read-line s) (load s))". 7: Skip rest of --eval and --load options. 8: Skip to toplevel READ/EVAL/PRINT loop. 9: [EXIT ] Exit SBCL (calling #'EXIT, killing the process). (ASDF/FIND-SYSTEM:SYSDEF-ERROR "OPERATION instances must only be created through MAKE-OPERATION.") 0] back Backtrace for: # 0: (ASDF/FIND-SYSTEM:SYSDEF-ERROR "OPERATION instances must only be created through MAKE-OPERATION.") 1: ((:METHOD INITIALIZE-INSTANCE :BEFORE (ASDF/OPERATION:OPERATION)) #) [fast-method] 2: ((LAMBDA NIL :IN "/home/alendvai/common-lisp/hu.dwim.environment/bin/build-development-image.sh")) 3: (ASDF/INTERFACE::SYSTEM-LOADED-P CL-GRAPH-SYSTEM::CL-GRAPH) 4: ((FLET #:WRAPPER60 :IN ASDF/INTERFACE::REQUIRED-SYSTEMS-LOADED-P) CL-GRAPH-SYSTEM::CL-GRAPH) 5: (SB-KERNEL:%MAP-FOR-EFFECT-ARITY-1 # (CL-GRAPH-SYSTEM::CL-GRAPH CL-GRAPH-SYSTEM::CL-VARIATES)) 6: (ASDF/INTERFACE::REQUIRED-SYSTEMS-LOADED-P #) 7: ((LAMBDA (ASDF/INTERFACE::CONNECTION) :IN ASDF/INTERFACE::LOAD-CONNECTED-SYSTEMS) #) 8: (ASDF/INTERFACE:MAP-SYSTEM-CONNECTIONS #) 9: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) # # # # :VERBOSE NIL) 10: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) 11: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) # # :VERBOSE NIL) [fast-method] 12: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) # # ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) 13: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) 14: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) [fast-method] 15: (ASDF/OPERATE:LOAD-SYSTEM :ASDF :VERBOSE NIL) 16: (ASDF/UPGRADE:UPGRADE-ASDF) 17: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) 18: (ASDF/CACHE:CALL-WITH-ASDF-CACHE # :OVERRIDE NIL :KEY
Re: asdf version dependency
for some 0.02, this is my high-level perspective: the name of an entity/concept is some letters/sounds/notation that we humans use to point to it while communicating with each other. sometimes we use computers to achieve that, e.g. when packaging a library together with a .asd file. from this perspective, the question is: should names in the ASDF model be a simple string of characters, or should they be represented as a more formal encoding that includes the state of the entity being pointed to (i.e. the version information). IOW, for me the question boils down to this: shall ASDF implement version control features? and looking at the unfortunate fact that source code is still saved into flat text files today, and at the complexity/requiremenets of git & friends to deal with the state of these text files, my gut reaction is no, it should stay far away. professional integrators should use professional VCS systems to mark specific states of the entities they are integrating. i think anything less is just wasted efforts that grows complexity without solving the issue beyond the trivial cases. e.g. what about quicklisp, or integrators in general? shall they bundle different versions of the same system (they can't, because the lisp package namespace is shared)? or will they just resort to whichever tools they are using to resolve the integrator's headaches and just entirely ignore the new ASDF feature? FTR, for our libs i use darcs and overlapping tags to keep track of the collective state of the dependencies when packaging various different apps that use various different subsets of these dependencies. for foreign libs i also use whichever VCS is used to publish them. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Selfishness is not living as one wishes to live, it is asking others to live as one wishes to live.” — Oscar Wilde (1854–1900), 'The Soul of Man Under Socialism' (1895)
Re: newbie setup question
> * asdf:*central-registry* > > (#P"/home/LE/sbcl/" #P"/home/mordecai/quicklisp/quicklisp/") i'd just put my code under ~/quicklisp/local-projects/ and forget about ASDF configuration until it's again needed for something. you can also use symlinks. also, IIRC asdf:*central-registry* is obsoleted. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The first thing you have to do if you want to raise nice kids, is you have to talk to them like they are people instead of talking to them like they're property.” — Frank Zappa (1940–1993), 'The Howard Stern Show' (1987)
Re: Issues with new testing scripts
> This is the reason why I prefer 'git subtree' to 'git submodule': with oh, git... why didn't i expect to have more than one way to implement the same thing? as opposed to e.g. having a flag that switches between the two modes of the same abstraction, both documented at the same place... maybe? thanks for the wakeup call! :) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Unthinking respect for authority is the greatest enemy of truth.” — Albert Einstein (1879–1955)
Re: a FIND-SYSTEM inconsistency which may be a bug
> CLEANUP-SYSTEM-DEFINITION-SEARCH-FUNCTIONS attempts to remove > SYSDEF-PRELOADED-SYSTEM-SEARCH from the hooks, but it is never added. > and right under it, SEARCH-FOR-SYSTEM-DEFINITION also assumes that > it's supposed to be called explicitly after the hooks. i'm afraid i sent this mail too early. the comment above it says: "Remove known-incompatible sysdef functions from old versions of asdf." so, if this is indeed a cleanup needed for a proper upgrade, then please ignore my mail and sorry for the noise. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Justice is not concerned with the results of the various transactions, but only with whether the transactions themselves are fair.” — F.A. Hayek (1899–1992), 'Law, Legislation and Liberty', I.6.j
Re: register-immutable-system -> (setf system-mutable-p)
> the export entry in the package was not renamed, it still exports > REGISTER-IMMUTABLE-SYSTEM. damn, a dirty tree with my own unrecorded changes confused me. ignore this mail and sorry for the noise! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If a politician found he had cannibals among his constituents, he would promise them missionaries for dinner.” — H.L. Mencken (1880–1956)
register-immutable-system -> (setf system-mutable-p)
the export entry in the package was not renamed, it still exports REGISTER-IMMUTABLE-SYSTEM. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “You can get love only from a free agent whose uniqueness is respected by you, whose freedom is respected by you. It is out of the freedom of the other that this moment of love has happened. Don’t destroy it by trying to possess, by trying to hold, by creating a legal bondage, a marriage. Let the other be free, and remain free yourself. Don’t let anybody else possess you either. To possess or to be possessed, both are ugly. If you are possessed you lose your very soul.” — Osho (1931–1990), 'From Darkness to Light' (1985)
Re: trouble with :also-exclude
> For this to apply to your ~/common-lisp/ you'd have to list > ~/common-lisp/ explicitly, and then :ignore-inherited-configuration to > avoid the regular entry for ~/common-lisp/ and then explicitly include > any configuration you'd like to inherit anyway. ouch. is there any other way to tell :tree to ignore some branches? it would be useful both for optimization, and also for dealing with different versions of the same system. in my original config my hand-written scanner was e.g. ignoring _foo directories, so i could just rename dirs to use a different version. i'd like to use a default config with as little interference as possible, so i don't want to just shadow all the /etc/common-lisp/... configs. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The liberty of man consists solely in this, that he obeys the laws of nature because he has himself recognized them as such, and not because they have been imposed upon him externally by any foreign will whatsoever, human or divine, collective or individual.” — Mikhail Bakunin (1814–1876)
trouble with :also-exclude
i'm not sure what's the expected behavior. what i'm trying to achieve
is to exclude some dirs inside my ~/common-lisp/
$ cat ~/.config/common-lisp/source-registry.conf.d/00-excluded-systems.conf
(:also-exclude "sbcl" "disabled-systems")
$ rlwrap sbcl --no-userinit --no-sysinit
This is SBCL 1.2.4.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (require :asdf)
("ASDF" "asdf" "UIOP" "uiop")
;; force asdf upgrade
* (asdf:load-system :asdf)
T
* (gethash "cl-postgres+local-time" asdf::*source-registry*)
#P"/home/alendvai/common-lisp/disabled-systems/local-time.old-darcs/cl-postgres+local-time.asd"
T
* asdf::*source-registry-exclusions*
(".bzr" ".cdv" ".git" ".hg" ".pc" ".svn" "CVS" "RCS" "SCCS" "_darcs" "_sgbak"
"autom4te.cache" "cover_db" "_build" "debian")
* asdf/upgrade:*asdf-version*
"3.1.5.18"
--
it may or may not be normal that *source-registry-exclusions* doesn't
contain "disabled-systems".
(trace asdf/source-registry::process-source-registry-directive :print
asdf::*source-registry-exclusions*)
the output of the above^ trace suggests that it gets processed (and
recorded into a dynamically bound variable? which is then thrown
away?).
any hints?
--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The worst enemy of clear thinking is the propensity to hypostatize,
i.e. to ascribe substance or real existence to mental constructs or
concepts.”
— Ludwig von Mises (1881–1973), 'The Ultimate Foundations of Economic
Science' (1962)
Re: ASDF 3.1.5 test suite issues on ACL (8.2 and 10.0 GM)
>> * image-op and program-op couldn't have been fully portable without >> bundle-operations, and wouldn't have been attempted. > > I use program-op on a regular basis and it has replaced a very complex > scripting machinery which I gladly got rid of. +1 it eliminated an entire project for us (hu.dwim.build)! :) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is just as difficult and dangerous to try to free a people that wants to remain servile as it is to enslave a people that wants to remain free.” — Niccolò Machiavelli (1469–1527)
Re: Standalone application delivery
> is any of you interested in automating application delivery using ASDF? > Now that I fixed ASDF's bundle operations and CFFI, the missing piece > is the part where > the bundled object files are linked against the runtime to produce a > self-contained executable. that^ sounds as if it was possible to generate e.g. an SBCL executable core that also includes the CFFI groveler generated .a or .so files linked into it. is that possible? a cursory look suggests that it is, but not trivial at all: https://stackoverflow.com/questions/9449845/how-to-link-object-file-to-executable-compiled-binary -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Stupidity is an attitude.
Re: Standalone application delivery
> It's possible, and some people have done it for SBCL at Google using > bazel.io, though they never spent the time making it open source. I > could extract bits from it and reproduce the same thing with ASDF, but > I'd rather someone else do it, though I can explain what I see in > those files. does it happen a) from a final linking phase linking sbcl's .o files b) or using a vanilla sbcl exe (or executable core) and hacking the grovelled .o or .so files into it? > Or then again, the CL community could adopt bazel (not very likely). from a strictly engineering perspective, do you think using bazel to build cl code would be better than using ASDF? yes/no/hard-to-judge? to me it seems that it's an important ability to customize the build system from the .asd (because it's the same language), and that would be lost with bazel, no? feel free to ignore all of this, just random curiosity... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Politicians are successful criminals. Criminals are unsuccessful politicians. Criminals are poor, pitiable. They had tried but they failed. Politicians are of the same tribe, with only one difference: they have succeeded.” — Osho (1931–1990), 'From Darkness to Light' (1985)
Re: Almost there
Well, if we change the API to add a boolean slot to SYSTEM, we could this may be naive, but what i meant is a very simple change: introduce an exported SYSTEM-MUTABLE-P, together with a SETF version, that messes around with the current hashtable based implementation. IOW, it's pretty much just a rename of REGISTER-IMMUTABLE-SYSTEM - (SETF SYSTEM-MUTABLE-P), and a new SYSTEM-MUTABLE-P that queries the internal hashtable. the (setf ... false) version may be a not-yet-implemented error if there's too much state to mess around with before this release. moving to a slot based implementation is another story that can be delayed, and may not even be preferable (e.g. what if someone sometimes wants a set of systems to be immutable, and some other time not? although, this also applies to every slot of SYSTEM, so...). remove REGISTER-IMMUTABLE-SYSTEM, and make it the responsibility of the user to create the system, whether through register-preloaded-system or not. Actually, if the slot has an initarg keyword, then you can use register-preloaded-system directly to create an immutable system if not already present, or you'll have to use setf to make an otherwise existing one immutable. I'd rather you just release what we have for now, but if you want, I can make those changes before 3.1.5 (or after). another alternative is to just keep all these symbols private and delay their exporting. optionally multiple variants could be added but not exported, so that early adaptors can already use the to-be-published API by using ASDF:: prefix. probably it's only the two of us participating in this thread who are using this new feature. an untested sketch is attached that merely renames what's already there, and errors at (setf ... false). but to reiterate: i don't have hard feelings about this. i'm just trying to help avoiding the publishing of a confusing name that will be much harder to change down the road. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “All war is a symptom of man's failure as a thinking animal.” ― John Steinbeck (1902–1968) diff --git a/find-system.lisp b/find-system.lisp index 04b63f4..e40854e 100644 --- a/find-system.lisp +++ b/find-system.lisp @@ -129,20 +129,28 @@ downgrade, before you dump an image, use: but not loaded in memory :format-arguments (list name)) - (defun register-immutable-system (system-name key (version t)) -(let* ((system-name (coerce-name system-name)) - (registered-system (cdr (system-registered-p system-name))) - (default-version? (eql version t)) - (version (cond ((and default-version? registered-system) - (component-version registered-system)) - (default-version? nil) - (t version - (unless registered-system -(register-system (make-preloaded-system system-name (list :version version - (register-preloaded-system system-name :version version) - (unless *immutable-systems* -(setf *immutable-systems* (list-to-hash-set nil))) - (setf (gethash (coerce-name system-name) *immutable-systems*) t))) + (defun system-mutable-p (system-name) +(let ((system-name (coerce-name system-name))) + (or (sysdef-preloaded-system-search system-name) + (and *immutable-systems* + (gethash system-name *immutable-systems*) + + (defun (setf system-mutable-p) (new-value system-name key (version t)) +(if new-value +(let* ((system-name (coerce-name system-name)) + (registered-system (cdr (system-registered-p system-name))) + (default-version? (eql version t)) + (version (cond ((and default-version? registered-system) + (component-version registered-system)) + (default-version? nil) + (t version + (unless registered-system +(register-system (make-preloaded-system system-name (list :version version + (register-preloaded-system system-name :version version) + (unless *immutable-systems* +(setf *immutable-systems* (list-to-hash-set nil))) + (setf (gethash (coerce-name system-name) *immutable-systems*) t)) +(error Not yet implemented: (SETF SYSTEM-MUTABLE-P) with false new-value.))) (defun clear-system (system) Clear the entry for a SYSTEM in the database of systems previously loaded,
Re: Almost there
SYSTEM-MUTABLE-P take a real *SYSTEM* as argument, instead of having an odd API where only a system-NAME is acceptable. note: CL-USER (asdf:coerce-name (asdf:find-system :hu.dwim.def)) hu.dwim.def I think having a confusing name that we deprecate is better than taking the good name, and implementing something bad on it. This way we can think about a better API and put the better API on the better name. i thought (and stated previously, with a doubt) that RIS is something new, but now that i double checked i realized it's already there since 2014 aug, exported: https://github.com/fare/asdf/commit/1b38225b8cc5749fafac9f300ac469fd92beba86 it's a lost case then, it's already published, so there's no way other than the deprecation way. in that case it's not an urgent issue, just put it on the TODO. sorry for the noise! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Many people believe that evil is the presence of something. I think it’s the absence of something.” — Lisa Unger (1970–), 'Sliver of Truth'
Re: Almost there
maybe a simple defun SYSTEM-MUTABLE-P and a setf variant would be better? Maybe. I'll let Robert decide if he wants a way to make a system mutable no more. Up until now, the usage scenario was that systems would transition one way only from mutable to immutable, as you prepare an image for delivery. I'm reluctant to add more code unless there's a demonstrated need for the functionality. I'd remind everyone that ASDF has ballooned in size and for all our efforts that inevitably means it has collapsed in maintainability! If you'd like an extension to provide either introspection or mutabilification, please file an enhancement ticket. But unless you provide a use-case with it, it is likely to get short shrift. no, i'm fine with the functinality of REGISTER-IMMUTABLE-SYSTEM. what i wasn't fine with is an exported global holding a hashtable, and at that time i hadn't noticed RIS because i was expecting a different name. REGISTER-IMMUTABLE-SYSTEM doesn't register any systems, it rather marks a system, that is already known by asdf, immutable. it's a different story that the implementation registers it in a hashtable as opposed to e.g. setting a flag on the system object. so, before i knew about RIS, i proposed an API that i still think would be better, namely SYSTEM-MUTABLE-P. if it's still feasible i suggest to replace REGISTER-IMMUTABLE-SYSTEM with (SETF SYSTEM-MUTABLE-P) and stop exporting *IMMUTABLE-SYSTEMS*. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Until you make the unconscious conscious, it will direct your life and you will call it fate.” — Carl Jung (1875–1961)
Re: Almost there
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it. isn't that also the case for REGISTER-IMMUTABLE-SYSTEM? if that export sticks in the release then it'll be a headache down the road (assuming that it is indeed an unfortunate name and not just my lone opinion). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There is no such thing as the government. There are only a group of people who refer to themselves as the government and act in a governmental manner.” ― Murray N. Rothbard (1926–1995)
Re: program-op is nice!
I'm quite glad you, of all people, like it! what? am i that critical usually? :) 2- Use (defmethod output-files ((o program-op) (s (eql (find-system mysys (values ... t)) so it will be exactly where you want it. this one did the trick, thanks! hu.dwim.build is then pretty much obsoleted... great! -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In the electronics industry, patents are of no value whatsoever in spurring research and development.” — vice-president of Intel Corporation, Business Week, 11 May 1981.
program-op is nice!
Fare, your asdf/uiop image-dumping machinery is great! with only a few minutes of trying i've already made an exe for the dwim.hu service that seems to just work. i'm very pleasantly surprised, because it gets rid of the entire hu.dwim.build complexity, and is now even platform independent... thank you! :) FTR, the change is here: http://dwim.hu/darcsweb/darcsweb.cgi?r=HEAD%20hu.dwim.home;a=commitdiff;h=20150531001956-35491-e309a5382d80bfbe5426294d07fd4e891c40a8c6 it all seems to just work, but i have a hard time telling asdf where to put the resulting exe. the :build-pathname arg of defsystem errors with absolute paths (which sounds reasonable), but then it's merely the filename of the exe. my problem is that it gets saved into the fasl cache, which i cannot calculate from the shell script side of the build script to mv the file to its final place. is there some doc or test that i have missed? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “You think you know when you can learn; are more sure when you can write; even more when you can teach... but certain when you can program.” — Alan Perlis
Re: [Asdf-devel] request for test: experimental-submodules branch
Windows? Are .lnk files supported on all Windows implementations? Meh. FTR, windoze supports proper symlinks on NTFS since Vista. https://en.wikipedia.org/wiki/NTFS_symbolic_link -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The Army is a place where you get up early in the morning to be yelled at by people with short haircuts and tiny brains.” — Dave Barry ___ Asdf-devel mailing list Asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [Asdf-devel] syntax control
(labels(({(] rest [)(apply([ ])[))([(])(elt(]())]))(]()(
do-external-symbols(]`:cl)(push`,]`,))(sort `string`:key`string))(}(}
{)({`688({`875({`398()~{~A ~}~%(]()))}(+`,{(*)})))({`381)({`816`2/9)))
({`561()#'}`(874,948 7,6009 4862,370 10,12249)`(3,2 4,4 2,1 1,0)))
WTF!!
:D
--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Some people, when confronted with a Unix problem, think I know, I'll
use sed. Now they have two problems.”
— Jamie Zawinski
___
Asdf-devel mailing list
Asdf-devel@common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] Make the CL syntax predictable
We can provide new, improved functionality without breaking anything. And keep old systems working forever with new ASDF. forever is a very strong word Anton. e.g. if CL doesn't fade into the category of history in 10 years, meaning that a significant number of people will still use it to develop new software, then i'll be both surprised and disappointed. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Virtue is never left to stand alone. He who has it will have neighbors.” — Confucius (551–479 BC)
Re: [asdf-devel] Make the CL syntax predictable
I prefer the defaults of the CL implementation of my choice not to be overridden by some tool whose primary purpose is something unrelated to those defaults. That’s all. i fail to see how ASDF's primary purpose is unrelated to global state that affects the outcome of a build process. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “In order to rally people, governments need enemies. They want us to be afraid, to hate, so we will rally behind them. And if they do not have a real enemy, they will invent one in order to mobilize us.” ― Thich Nhat Hanh (1926–)
Re: [asdf-devel] Make the CL syntax predictable
You’re quoting me out of context. If deterministic by default had no cost associated with it, it would clearly be the desirable choice. But it has a cost associated with it, so this is less clear. am i right to assume here that the costs we are talking about is merely adding markers to those few already existing defsystem's that depend on leakage, or depend on non-portable (read: random) defaults? because that it has a cost associated with it above sounds like some serious pain... If I failed to convince you that this is the case, and that other people may have other preferences, then I apologize. reading from this side of the screen your preference seems to be to have change in what you like (new goodies coming from qicklisp), but not in what you don't like (the primary build infrastructure of the entire community maturing into something useful). -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Lisp has jokingly been called the most intelligent way to misuse a computer. I think that description is a great compliment because it transmits the full flavor of liberation: it has assisted a number of our most gifted fellow humans in thinking previously impossible thoughts.” — Edsger W. Dijkstra (1930–2002)
Re: [asdf-devel] Make the CL syntax predictable
I do have control: If femlisp or any other library makes a boneheaded decision that breaks my software, I can stop using it. yes, resolving that is trivial -- once you have identified the problem. regarding the recent discussions i'm generally baffled why it is at all a question whether to make a build software deterministic or not. in my view if there's anything in the global state that has an effect on the building of a software, anything, then it's a bug. and if this state can be set in a way that it leaks out and influences builds later in time, then it's an ugly bug feasting on programmer nerves and time. the only question to me is what should be the defaults, and how can the transition be made smoother with deprecation warnings and whatnot. and if there are some unmaintained, bitrotten libraries and some software depending on them, then they have all the freedom in the world not to upgrade ASDF, or to put on the maintainer hat and patch the bitrotten libs. by now the time on spent discussing this would have easily been enough to fix all of them twice over, and to add a section to the top of the manual, with bold, that lists the global state that ASDF guarantees and isolates. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Those who love peace must learn to organize as effectively as those who love war.” ― Martin Luther King, Jr. (1929–1968, assassinated)
Re: [asdf-devel] Make the CL syntax predictable
regarding the recent discussions i'm generally baffled why it is at all a question whether to make a build software deterministic or not. in my view if there's anything in the global state that has an effect on the building of a software, anything, then it's a bug. I think one question is whether it's worth taking a path to this goal that breaks programs that currently work. I don't think all such programs are bitrotted junk that can be fixed up in a day. those programs that currently work will keep on working if ASDF is not upgraded. no upgrade, no breakage. to me it's a very strange argument that the right way to avoid breaking unmaintained old code is by hindering change that makes new things better in the future. what's needed is people helping you Xach building, testing and maintaining quicklisp distros. i feel the frustration and i can understand it, becuase i remember how many times i backed out from a sprialing upgrade mess... and for you it's a regular TODO. at one point i tried to build a ql distro locally to add and test a new project, but i didn't even have the bird's eye view, so i gave up. if there was a blog post or something then people may join the effort... but that's off topic here. i just wanted to let you know that your work on ql is appreciated, and that some people sometimes even try to help out with it that you may harness. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The noble-minded are calm and steady. Little people are forever fussing and fretting.” — Confucius (551–479 BC), 'Analects of Confucius'
Re: [asdf-devel] Make the CL syntax predictable
and if this state can be set in a way that it leaks out and influences builds later in time, then it's an ugly bug feasting on programmer nerves and time. This would be true only if a software was equivalent to an ASDF system. That is not the case. I have seen many applications that are made up of a large number of ASDF systems, because ASDF does not necessarily provide nice structures for large applications. In such we also have libs with 10+ asdf systems. application builds, you WANT state to bleed from one ASDF system to another, because these ASDF systems are not freestanding entities. one can write a trivial loader file that calls ASDF with some extra keyword parameters to load such systems. Complaining that state leaks in and out of these systems is like complaining that state leaks into a closure. It's intended to leak, and that is not a bug. the vast majority of the asdf systems don't want leakage, and defaults should be covering the majority use-case (if/when it's possible without setting up landmines). reproducible builds are a valuable feature. if i wanted to cater for the use-case you described then i'd still make full isolation the default with features to let systems explicitly turn off specific isolations, or just simply let them on their own to subclass and customize ASDF the way we currently have to subclass and customize asdf to avoid the leakage from the hu.dwim.* universe (we use e.g. readtable customizations). and whoever is not ready to invest time in following the changes, should simply not upgrade, and then it's a non-issue. but this starts to feel like preaching and i don't really have that much stake in this anymore because we have already invested in covering our bases... so, these are just some 0.02... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The three most harmful addictions are heroin, carbohydrates, and a monthly salary.” — Nassim Taleb (1960–)
Re: [asdf-devel] Make the CL syntax predictable
That's why, for example, we have systems that bleed readtable across their boundaries: the systems really aren't stand alone entities, but limitations on ASDF expressiveness, and the desire to get more modularity than the (necessarily in-line) :MODULE will permit, causes a proliferation of systems that are NOT libraries, and have meaning only in context. I'm not going to move ASDF towards breaking such systems. Robert, this is how i see it reading this thread: - you know at least about one big and complex codebase with many asdf systems, where isolating *readtable* would break it and cause a lot of work/headache that is hard to debug, especially if the codebase doesn't compile free of warnings to begin with. this headache would probably be yours in case of this system. - to Fare (and to me, who have wasted precious hours due to this) readtable higiene clearly looks like a major improvement when looking at it objectively, so much so that somewhere down the road breaking systems that now break other systems (by modifying the global readtable) is acceptable. my impression is that you would also agree with this if the legacy was not pulling your back. *if* this is the case, then why not accommodate for such legacy systems/use-cases simply by e.g.: ;; force upgrade (asdf:load-system :asdf) ;; optionally, for extra safety (asdf:load-all-dependencies-of :my-legacy-system) (setf asdf:*readtable-higiene* nil) (asdf:load-system :my-legacy-system) and then we can have our cake and eat it too... (alternatively a new system class could be introduced and all legacy systems could specify a :class hu.dwim.asdf:hu.dwim.system, just like we need to do currently for self-defense purposes against e.g. readtable polluters and against we polluting the readtables.) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “I maintain that Truth is a pathless land, and you cannot approach it by any path whatsoever, by any religion, by any sect. [...] The moment you follow someone you cease to follow Truth.” — Jiddu Krishnamurti (1901–1986)
Re: [asdf-devel] Re: NAMED-READTABLES
I would like to forestall facile answers to the above claim, too. I don't just mean SLIME would have to handle IN-READTABLE. SLIME would have to be fixed, yes, but also ELI, Allegro Composer, Hemlock (or whatever the CCL IDE is), LispWorks whatchamacallit, etc., etc. True. However, lack of support there is not a regression. Things that used to work still will work. New things that may work will require additional support, but we can leave that up to those who will use those things. I suspect Attila and the dwim.hu guys have extensions like that already. it's a simple thing that can easily be adapted to named-readtables, and i'm willing to help doing that if things get moving on this front: http://dwim.hu/darcsweb/darcsweb.cgi?r=HEAD%20hu.dwim.def;a=headblob;f=/integration/swank.lisp it's loaded with hu.dwim.def+swank.asd we have a develop-op in hu.dwim.asdf that at the end looks for all the *+swank systems, and loads them if all of their dependencies are already loaded. (it also does a (declaim (optimize debug)), and various other changes with logging, etc... depending on what system is loaded with develop-op) ASDF should not demand a programming style that doesn't have full support for in-editor dynamic code writing and maintenance. ASDF doesn't demand anything; it just enables new extensions by providing hygiene. this reminds me of the general idea of backwards compatibility: people who want to live without this feature can chose not to upgrade, as always. or if they need the new stuff, then they can just patch asdf locally to disable the enforcement of this. or asdf could even accommodate this with a defsystem property that can opt a system out of this otherwise useful isolation. *or* this feature may not clearly be an improvement in general, but i'm not convinced about that for now. These extensions may in turn demand editor support, either way, if you use reader syntax in CL then you'll be neck deep anyways into duct taping things together, unless an infrastructure is set up that people agree to use. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Unthinking respect for authority is the greatest enemy of truth.” — Albert Einstein (1879–1955)
Re: [asdf-devel] Make the CL syntax predictable
The clean thing to do would be to use named-readtables and/or cl-syntax, and have each file evaluate (in-readtable :foo) or have a perform :around method or around-compile hook that does it for you. [...] Without ASDF, there's no way the libraries will be safe by default. well, there is, but it's not trivial, so most will not take the extra effort required: FTR, from hu.dwim.common we ended up shadowing cl:in-package in all our projects and added an internal mapping from package to readtable-setup-function (using a custom package definer that knows about readtables). our hu.dwim.common:in-package also sets the readtable and expects that our hu.dwim.asdf systems rebinds a copy of *readtabe* around compiling and loading. in a hu.dwim.common+swank integration system (or in one of those...) Slime is also notified about the readtables, so C-c C-c works as expected. i'd be happy if all this web of complexity could move out of our repos and into the environment, where it simultaneously becomes much simpler also. in short, i agree with Fare on this. But polluting arbitrary systems loaded after you is definitely a bad idea: yes, projects sideffecting *readtable* was annoying me quite a lot, and i've sent around my set of patches to a few projects that did that, including none other than iterate itself. based on a general-purpose each system has a set of private variables protocol. i don't have a strong opinion on this, but after some consideration we went with another protocol that is based on functions that modify a readtable that is given to them, which is much more flexible. not sure/don't remember whether this extra flexibility was actually needed, though. maybe it was needed for some combination of multiple syntaxes set up at the same time... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The Atheist does not say there is no god, but he says I do not know what you mean by god; I am without the idea of god; the word god is to me a sound conveying no clear or distinct affirmation. I do not deny god, because I cannot deny that of which I have no conception and the conception of which by its affirmer is so imperfect that he is unable to define it to me.” — Charles Bradlaugh (1833–1891), 'National Review' (1883–11–25)
Re: [asdf-devel] Alternate default lisp system location
Ok good. Then how about having ASDF loudly announce that it is registering the default directory (e.g. ~/cl/), as well as providing the hint about using a .conf file with the right syntax with :tree, if the user wants to exclude certain subdirectories. The goal here is to make the addition of the default registry directory as painless and surprise-free as possible for anyone who already happens to have stuff floating around in the new default directory. FTR, when i wrote my own version of :tree i also added a loud output that listed everything that got added to the registry, because previously i've wasted way too much time with loading the wrong systems, and with debugging build differences between dev environments and production deployments. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “It is truth that liberates, not your effort to be free.” — Jiddu Krishnamurti (1901–1986), 'The First and Last Freedom'
Re: [asdf-devel] Alternate default lisp system location
I *do* get the objection that Pascal had, and I absolutely agree that we should not choose a directory name that will collide with existing users' directories. when reading this i imagined my future child some decades down the road reading the ASDF manual (hopefull only for historical reasons while reifying CL semantics to have an environment where he can try his father's code)... and then pondering: what on earth made someone chose this weird ~/asdf-local-projects/ name instead of e.g. ~/common-lisp/ :) IOW, i'd like to draw more attentinon to the fact that what we are weighting here is a few active complaints today from experts (who are already aware of this issue reading the mailing list, even before the change itself)... against all the yet to be born/educated CL programmers until the time when ASDF is eventually obsoleted (or for that matter CL altogether), and all the aggregate time these future people will waste reading manuals and asking questions. i think a tendency towards not taking this into consideration has a lot to do why the CL community is not successful. the defaults of slime is another very good example of this, although it got much better since the times when i first climbed the CL/Emacs fence. also note: as Fare already pointed out, if e.g. XCVB gains more traction, then some/most systems will have two system definition files, but all sitting under the asdf specific ~/asdf-local-projects/ name. but hopefully by the time my children are old enough for programming computing systems will have finally obsoleted this silly idea that the base axioms for data storage is labeled binary numbers, and their labels organized into a tree... and with that rendering this whole question moot. hopefully... sorry for the sentiments. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Disregard for the preferences and interests of individuals alive today in order to pursue some distant social goal that their rulers have claimed is their duty to promote has been a common cause of misery for people throughout the ages.” — Isaiah Berlin (1909–1997)
Re: [asdf-devel] BUILD-OP
Usability demands an asdf:build function. The name may vary, but shorter is better. asdf:b or asdf:bnm (build 'n' munge?) might do, too. these sound awful to me, but then i'm against abbreviations in general and always have fuzzy completion on my TAB... a perfect candidate for this semantics would be 'operate', but sadly that's already taken. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- I wish there was more: Grace to accept with serenity the things that cannot be changed, Courage to change the things that should be changed, and Wisdom to distinguish the one from the other. — a secular version of The Serenity Prayer (http://youtu.be/zZvBknPV6hk?t=3h52m00s)
Re: [asdf-devel] Alternate default lisp system location
to expose so a :tree recursion there would be very annoying. And please don't tell me to :ignore-inherited-configuration, for what's the point of the default configuration if so many people either don't use it or have to explicitly ignore it ? well, for now it's two hard-core hackers, for whom it's probably marginal to get things back on track *if* it actually breaks something for them. but we have no idea about the number of newcomers who fire up an apt-get install sbcl for the first time in their lives, and just want to try a piece of code copied from the net... i still remember how goddamn high the fence was to first climb in to the CL+emacs garden. i'm with Robert on this. i'm not sure how the defaults would help on this, though. probably a very slick quicklisp experience has more value than any of these defaults can create. just some 2¢, -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind.” ― Dr. Seuss (1904–1991)
Re: [asdf-devel] Alternate default lisp system location
I think you will find that the configuration of the source search path is *very* complex, indeed too complex for a beginner. It's great for hackers with complex requirements, but there's too much to be read if you just want to make one or two simple systems. FTR, even i was mildly irritated by this when i first approached the new ASDF config infrastructure. but don't get me wrong, i have no ideas how to simplify it, and the docs are probably also much better these days... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- The essence of a civilized society: evidence trumps reason trumps emotions.
Re: [asdf-devel] Fixing asdf-sytem-connections
Actually, I would go so far as to discourage use of asdf-system-connections, and encourage only explicit dependency on connection systems. IIUC, the hu.dwim team changed their systems this way, at my suggestion. and everything became much much better. we just add a new .asd with the name system1+system2.asd and explicitly list whatever dependencies it has, and whatever extra files need to be loaded, usually from under the dir called integration/ in system1. chosing which system/repo/project to put the integration files in is subjective, but usually the choice gives itself. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “There is only one basic human right, the right to do as you damn well please. And with it comes the only basic human duty, the duty to take the consequences.” — P.J. O'Rourke (1947–)
Re: [asdf-devel] how to add one more directory to an initialized registry?
Sorry, why is it that you put in the line with *source-registry-parameter* instead of simply adding the new directory? reading your answer makes me think i was unclear, sorry! my situation is that i have a piece of code that collects directories and then initializes the registry by calling i-s-r. and then i have another piece of code later on where i would like to add one more directory to the registry. i couldn't find any other way to do that than the kludge in my previous mail. BTW, for exactly your use case -- making it easy for people to put configurations in a revision control system, and have an entire team get their configuration that way, we added :HERE to the DSL for initializing the source registry. thanks for the note! it will probably come handy when i get to port my registry initialization to the new way. but first i wanted to get an impression whether i want/should/need to do that. with your mail i'm leaning more towards a yes now, which renders the original question somewhat moot. [I feel compelled to disclose that because we have so much legacy code in that form, I still use the variable asdf:*central-registry* for my own configuration purposes, with an in-house library that essentially duplicates the function of Faré's directory traversal.] same here... :) -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Whenever you find yourself on the side of the majority, it is time to pause and reflect.” — Mark Twain (1835–1910)
Re: [asdf-devel] Pushed version 3.1.0.52 -- first version with checks for OPERATION subclasses -- please test!
For the record, I still think this intentional breakage is a bad idea. In the short term, it causes a known inconvenience FTR, i pulled asdf when i saw the request for testing, tried to build our stuff, saw that iolib failed (a dependency), and then i decided to git reset and stay there until others struggle through this, even though we also have some operation subclasses. there are situations when the least inconvenient and most productive way of breaking backwards compatibility is by starting with a clean slate. PS: I used to avoid committing spaces at end of line. I see that you have some. emacs has nice tools to show lose whitespace (see the varaible whitespace-style and there are some faces, too). it even has automatic cleanup, but that can be too intrusive when working on other people's code. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “If men are good, you don't need government; if men are evil or ambivalent, you don't dare have one.” — Robert LeFevre (1911–1986)
Re: [asdf-devel] Pushed version 3.1.0.52 -- first version with checks for OPERATION subclasses -- please test!
Yes, I turned these off in my editor. I ended up with too many distracting red blocks on my screen, and then too many spurious diffs when I would commit automatic whitespace cleanup. yep. i turned auto fixing off myself, and i have a very slight red tint on the whitespace mistakes. thanks for the mini-doc! i'm doing one more trial. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Study the past if you would define the future.” — Confucius (551–479 BC)
Re: [asdf-devel] Recompiling SBCL contribs
My point of view is that anything that makes system construction non-deterministic, conditional on the environment, sensitive to the order in which things are defined or otherwise surprising or hard to track is only going to make things harder in the end. my painful memories loudly add a big +1 for that. we moved to the foo+bar explicit dependency solution after we spent way too much time debugging stupid things that e.g. depended on inode order in the filesystem... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The individual has always had to struggle to keep from being overwhelmed by the tribe. If you try it, you will be lonely often, and sometimes frightened. But no price is too high to pay for the privilege of owning yourself.” — Friedrich Nietzsche (1844–1900)
[asdf-devel] deferred warnings doesn't tell which file
hi! so, i'm using deferred warnings, because warnings are useful! the only problem is that, in this specific case, i get the following after compilation has finished: ; compilation finished in 0:00:00.033 ; in: LAMBDA (SB-PCL::.METHOD-ARGS. SB-PCL::.NEXT-METHODS.) ; (SB-MOP:SLOT-VALUE-USING-CLASS # HU.DWIM.PEREC::INSTANCE ...) ; ; caught WARNING: ; undefined variable: HU.DWIM.PEREC::INSTANCE ; ; compilation unit finished ; Undefined variable: ; HU.DWIM.PEREC::INSTANCE ; caught 1 WARNING condition ; ; compilation unit aborted ; caught 4 fatal ERROR conditions and it doesn't even tell which file it happened with. (this error comes from the depths of sbcl internals around generic methods). how was it meant to work originally? is it assumed that the warning will always contain a name or something that helps identifying where it comes from? if so, then it's not always the case, and when warnings are not deferred at least the current toplevel form is visible in the output. note: this is not a showstopper just an inconvenience. i can identify the file by checking the *.sbcl-warnings files manually. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Here is the world. Beautiful and terrible things will happen. Don't be afraid.” — Frederick Buechner
Re: [asdf-devel] deferred warnings doesn't tell which file
If you look in uiop/lisp-build.lisp, functions reify-undefined-warning and unreify-undefined-warning, you'll see that we're trying to save and restore file context information; I don't know whether or not we're failing to save and restore it, or SBCL is failing to display it after we restore the data (you could try to disable the deferred warning mechanism, and see how it behaves — if it then displays file information, that's a UIOP failure; if it doesn't, probably an SBCL failure). there are more details in my mail to sbcl-devel, but this warning comes from a nested compilation initiated by some generic method internals. most probably sbcl fails to provide enough info, but here is the content of the .sbcl-warnings file for the initiated eyes: ((SB-C::*UNDEFINED-WARNINGS* (:VARIABLE HU.DWIM.PEREC::INSTANCE 1 (:ENCLOSING-SOURCE NIL :SOURCE NIL :ORIGINAL-SOURCE (SB-MOP:SLOT-VALUE-USING-CLASS # HU.DWIM.PEREC::INSTANCE ...) :CONTEXT ((LAMBDA (SB-PCL::.METHOD-ARGS. SB-PCL::.NEXT-METHODS.))) :FILE-NAME :LISP :FILE-POSITION 0 :ORIGINAL-SOURCE-PATH (1 2 3 2 4 2 0) to sum it up, it's a warning signaled by sbcl due to a bug, and doesn't contain source location. so, i think it's something we can safely ignore as a nasty corner case... -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- Pyramid schemes are illegal. Social Security is a pyramid scheme.
Re: [asdf-devel] Ordering of system :depends-on elements
ASDF's process for constructing a build plan from partial-order dependencies is (unless Faré changed something when I wasn't looking) non-deterministic. does it mean that it's actively randomized? i remember in our busier days (well before ASDF3) we used to have load order anomalies that seemed like they depended on filesystem order or something external to lisp. the symptom was that on my dev machine stuff compiled/loaded fine reproducibly, then when the recorded changes got pulled to the live server it suddenly failed to compile/load due to a dependency that was not explicitly added to the .asd file. i don't really remember if it was just partial reloading, or clean recompile/reload, but i think it was the former, because it was rather baffling and the former wouldn't be too baffling. if the ordering is actually not deterministic (as opposed to merely unspecified), then maybe it's a good idea to actively randomize lists before dependency constraints are applied? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Mathematics is as little a science as grammar is a language.” — Ernst Mayr
Re: [asdf-devel] adsf/bundle:monolithic-fasl sometimes puts dependent system first in list
Is it possible to get it to return a plain text source file instead of a fasl? yes. look at how asdf HEAD builds itself, especially monolithic-concatenate-source-op -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] An upgrade path for checking deferred-warnings
it is time that I declare this simultaneous change towards checking deferred warnings as a failure. i haven't experimented much with this infrastructure, so take it with a piece of salt... but maybe there's a middle ground, where the new warnings get printed but the load/compile is not forced to fail? or that's the case already? -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] standard io syntax woes
Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them this approach seems to be the most straightforward to me. especially since we have that great cl-test-grid running on multiple lisps, one of which is sbcl which will catch the wrongdoers. i'd also argue that it's also a potential time waster if asdf rebinds a copy and someone foolhardy tries to modify *readtable*, and will experience zero sideffects contrary to his expectations. he will waste his own time only, so it's already much better, but still not ideal. -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] help needed with asdf internals knowledge
I had code that looked just like that in cl-launch, now also copied to exscribe (see compile-and-load-file in exscribe.lisp). I suppose I should make it part of asdf/lisp-build even though asdf/defsystem itself won't use it. Is it OK for you to depend on a recent asdf-driver? yes, that would be perfect for me. (people should learn to NOT pull changes, or pull them together in waves) -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] load order of dependencies
There have been many attempts in the past to subvert ASDF into something that allows this kind of effects (see asdf-system-connections), but the semantics of such connections is not very clear (to me at least), and being familiar with ASDF itself, I don't see how such tricks can be made reliable at all. well, what i need is less than asdf-system-connections, because in my case the dependencies are explicit. i just want to have a word in the load order, also. a-s-c wants to jump in automagically and load new systems if some other systems are already loaded. we used it for a while but after countless headaches we switched to explicit systems called project1+project2+project3 that depended on the 3 projects and listed what else to load. before, or it's not, and might be loaded before or after and you can't retroactively unload and reload the system to have logging magically take effect. i'm programming for long enough to have learnt to make compromises, and i'd be happy to rmfasl and reload when i'm about to debug something. and re :system-depends-on, it bit me hard. took me hours of debugging why a file is being compiled/loaded twice while i was reinstalled http://dwim.hu and building the sbcl executable that runs on it (the symptom was a warning from defpackage about already exported symbols, it stopped the build). i was suspecting all kind of weird issues until i've unpulled this patch from hu.dwim.reiterate. for now, i'm ok. i'll just make sure i don't pull this patch to the live system... thanks for the help! -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] load order of dependencies
It might be an interesting test discipline (typically of interest only in combination with :FORCE t) to allow people to run ASDF operations with random tie-breaking. This seems like a poor debugging technique, though, since the number of such orderings will explode in systems with weak RECORDED dependencies, so stumbling on a missing dependency might well not happen quickly i think that exponential blowup of the problem space is not so bad, because the load order issues also materialize in big subspaces of it. but how about making then sorting the load order? (but then what about different lisps sorting unicode differently?) i've learned to live with this, but i thought i bring this up here, maybe someone has some good idea that could go into ASDF2. -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06 ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] [Sbcl-devel] Logical pathnames vs ASDF SBCL
The main reason why I'm having problems with ASDF 2.x is because it's alpha-quality software at best (you're still trying to figure out some very basic concepts, it seems to me), yet you don't even call it beta quality, but instead choose to call it 2.x, push it down everybody's throats, and turn a whole community into alpha testers who didn't volunteer to be guinea pigs in the first place. as people with positive opinion speak up much less, let me point out here that i'm a happy user of ASDF2 from its early days. it did require some work to follow it, but it made many things simpler and more reliable in our dev setup and production environment (the load order with underspecified dependencies using ASDF1 used to be influenced by the filesystem order, seemingly not even randomly, because errors consistently appeared when changes were pulled to another computer. rmfasl didn't influence it. a real pain in the ass when build fails on the production system after an upgrade...) any form of backwards incompatibility, and upgrading in general, is a pain, but we need to live with it because for now it's part of being a software developer (it's solvable on the language/vm/devtool level, but our ideas are mere vapour). i regularly unpull problematic patches of random libraries, and ASDF is just one of them. luckily ASDF2 can even properly upgrade itself now! to me it's certainly an improvement over ASDF1. thanks for all the work put into ASDF, both v1 and v2! -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06 BitCoin: 154uf86Vd9rpjMULd9CXa7nVwikknYZJiB ___ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] asdf verbosity
I don't like these kinds of user-visible changes, because if someone liked the old behavior, or was at least accustomed to it, they now have to do research and become involved in ASDF to learn how to get the previous behavior. i don't really have an opinion on the specific topic, and i don't mean to annoy... but what if someone *didn't* like the behavior? then they would have to do research and become involved in ASDF to learn how to get rid of the current behavior (notice the almost exact same wording of a counter argument). just pointing out that to me it seems like you're simply stating your personal preferences and/or dislike for change. it's a common human behavior and there's nothing wrong with that, especially in your case Zach: as author of quicklisp you obviously try to minimize headaches caused by changes to ASDF. but if this really is the case then we should be aware of that because it often hiders technical discussions and development of software systems (the closest example that comes to my mind is what is happening to Slime. or really, what is *not* happening to it...) the ultimate backwards compatibility is not having had even started the project in question (modulo my english grammar deficiencies). also worth noting that people much-much more often speak up of annoyance than being pleased by a change. /rant -- attila Notice your eroding (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06 BitCoin: 154uf86Vd9rpjMULd9CXa7nVwikknYZJiB ___ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
[asdf-devel] performance of find-system (on sbcl)
dear list, find-system is so slow that i took the time to look into it. this is what i've used to test: (sb-sprof:with-profiling () (trace sb-fasl::load-as-source) (time (labels ((recurse (system-name) (let ((system (asdf:find-system system-name nil))) (dolist (child (rest (first (asdf:component-depends-on 'asdf:compile-op system (recurse (asdf:find-system child nil)) (recurse (asdf:find-system :hu.dwim.walker nil) out of the box run: Evaluation took: 3.233 seconds of real time 2.35 seconds of total run time (2.12 user, 0.23 system) [ Run times consist of 0.410 seconds GC time, and 1.940 seconds non-GC time. ] 72.69% CPU 15,573 forms interpreted 2,688 lambdas converted 5,615,673,160 processor cycles 15 page faults 128,748,160 bytes consed when adding the following to asdf:find-system, just around the cl:load call: #+sbcl (sb-ext:*evaluator-mode* :interpret) Evaluation took: 1.506 seconds of real time 0.74 seconds of total run time (0.64 user, 0.10 system) [ Run times consist of 0.110 seconds GC time, and 0.630 seconds non-GC time. ] 49.14% CPU 16,582 forms interpreted 56 lambdas converted 2,764,841,332 processor cycles 2 page faults 58,959,808 bytes consed any thoughts on this? -- attila ___ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] how to query root of a system
(defun query-system-root (x) (asdf:component-pathname (asdf:find-system x))) or with a not too old ASDF: TEST (asdf:system-relative-pathname :hu.dwim.stefil ) #P/home/ati/workspace/hu.dwim.stefil/ -- attila ___ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Re: [asdf-devel] [RfC] default value for *central-registry*
or whatever, and have ASDF either recurse through these directories or go to the systems directory underneath? This would allow sharing the i would like to point out that the way asdf suggests the symlinking of systems as the default way is not too newbie friendly... a different setup would be much more slick for newcomers, where it has a list of dirs that it scans for *.asd's 1 level deep. this symlinking is a though-trap, from which it's hard to escape, but i still remember the point when i did my own scanner and happily forgot symlinking and systems dirs forever. /me ducks -- attila ___ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
