Tim, Thanks very much for the good humour and detailed reply.
On October 31, 2005 10:00 PM you wrote: > > Ah, language wars ... the only thing that out-emotions a > license war Damn! And I just finished saying I didn't want to start such a war... > > I suppose, having been cast as the bad guy in this morality > play, that I'm required to have the long monologue about my > evil ways (so the hero has time to sneak around behind me > and kill me) :-) So sit back, grab some coffee, and watch > the opinions fly... I wouldn't worry about some sneaky but merely human hero. It's the spirits of this play that haunt do most of the haunting. Is that a long echoing evil laugh I hear? Ah, it is still "All Hallow's Eve" here... :) http://en.wikipedia.org/wiki/All_Hallows_Eve What are the tools I need here? Hmmm... \cross and \dagger ? Trick or treat? > > POINT 1: Axiom needs to be future-proofed. > > Axiom needs to be torn down, re-architected, re-thought > and rewritten cleanly so it can be modularized, maintained, > and extended. Which implies a major rewrite, likely done > by me. > I am not convinced that this is necessary or even possible, although I do not doubt your knowledge or your ability to do this - given enough time. According to the history of Axiom that you have described in various previous emails, Axiom has already undergone several cycles of re-writing. I don't see any evidence that such re-writing has resulted in making Axiom any more "future-proof". On the other hand I don't think it is necessary, because Axiom is here now as open source. Axiom is certainly complex but there are already good examples of other large and complex open source programs that have managed to attract some very skilful and dedicated developers who are capable of dealing with this sort of complexity. The example of the Linux kernel comes to mind. Unfortunately, I think it is true that although several years have passed since Axiom's release as open source, there is still only one active developer with a thorough knowledge of Axiom - Tim Daly. In order to best "future proof" Axiom, rather than re-writing Axiom, I think the best strategy might simply to give up programming and learn something about genetics and cloning. ;) Failing that, there remains the task of documentation and education. I think you have been doing a great job at that and I urge you to continue. > > POINT 2: The rewriting will change existing code. > > If we look at the "start-up sequence" we see: > > restart (lisp) > init-memory-config (lisp) > initroot (lisp) > reroot (lisp) > spad (boot) > runspad (boot) > ncTopLevel (boot) > ncIntLoop (lisp) > intloop (boot) > SpadInterpretStream (boot) > intloopReadConsole (boot) > > and, if you follow the side-paths you'll see the constant > switching between boot and lisp at every level in nearly > every function. > > This inhibits "collapsing and rewriting" code which will > have to happen during the rewrite. Functions will either > have to be collapsed into lisp or boot. > As a long term strategy, I agree, however as I have stated I think boot is the better choice. > Historically this interleaving occurs because new function > was added/deleted/replaced. The "nc" prefix implies "New > Code", a newly rewritten parser that partially replaced the > previously newly rewritten parser. The "shoe" code, which is > the new, improved "boot" language exists in there. The "meta" > code used to exist but is gone. Addition of Fortran support > (for NAG), Foreign Language calls (for graphics and the > browser), Lisp vendor specific code, etc. required lisp. > Why was lisp required? Surely it is possible to do these things in BOOT. In fact, aren't most of the NAG extensions written in BOOT? E.g. ls src/interp/nag* > > POINT 3: Boot is a dead language. > > There are approximately 10 people still living who have coded > in boot and every one of them is doing something else with > their life. You have to be careful in boot. It is case sensitive > so APPEND is not the same as append. Wow, that comment certainly dates you! :) Almost all programming languages in the last 20 (30?) years have been case sensitive. Usually that is considered an advantage. > It is often not clear what a construct will translate to and > you end up reading the lisp to learn. A single, misplaced > space will change the meaning of the code. That is true of most programming languages. Understanding a "construct" by seeing how it "translates" is usually just a matter of lack of fluency in the language. This can be a result of inadequate documentation and/or experience. In a language that syntactically depends on indentation, of course misplaced spaces can change the meaning of the code. But this is not essentially different than misplaced commas or braces in other languages. > > POINT 4: Boot SERIOUSLY complicates the Axiom system. For > instance: > > 1) Bootstrapping boot > > The boot compiler is written in boot and needs to be compiled > using itself. If you have a running Axiom system this is not > a problem. However if you start from a clean lisp system it > is a big problem. I cheesed up a way to do this but it is > fragile and ugly. But given that we build from scratch there > is no other way. But that is exactly how the C programming language works. The Gnu C compiler is written in C and bootstrapped in a similar way. Also, the current Axiom build starts by first building GCL from the C sources. This is also a kind of bootstrap process. I don't see that the BOOT level adds much to this already existing complexity. > > 2) Makefiles come in multiple stages > > Boot forces Axiom to be built in stages. Thus Axiom cannot > use modern tools like ASDF. A pure common lisp Axiom interpreter > can be built and loaded directly into a lisp image, interpreted > or compiled. I thought there was already a version of interpsys called debugsys which provides an interpreted environment that is more friendly to debugging in lisp? Not being a lisp programmer, it is entirely unclear to me what advantages ASDF might have over more general tools like 'make' especially considering that the Axiom source files are actually literate programming documents (pamphlets), not lisp source code. > > Lisp, BOOTSYS, DEPSYS, INTERPSYS, AXIOMSYS. These can all be > collapsed if Boot disappears. > Although I still recall that this looked very complicated to me when I first started working with Axiom, now each of these stages in the current Axiom build process makes sense to me. Since Axiom is a large and complex system, building it a "layer" at a time seems quite logical. If these were all collapsed into one, then I worry that this would all start to look impossibly complicated to me again. > 3) lsp, lisp, clisp, ${LISP} > > All of these exist because boot exists. For example, .clisp > is a translated boot file whereas a lisp file is hand coded. > ${LISP} exists to smooth over this bump. I won't go into the > historical reasons why these came about but they exist as a > side-effect of boot. > Just another type of intermediate file, and Axiom already has many of these. > 4) The interpreter has a boot compiler built in > > There are complications to the interpreter to handle boot- > related development from the command line which no-one is > ever likely to do again. This code could disappear and, having > gone away, simplify the interpreter This still sounds a little vague, but in general if it was documented then perhaps someone who was working with BOOT would find this functionality useful and use it again. > > POINT 5: The boot language is undocumented > > It is likely to remain so as one of the authors is dead > and the other one might be. There are no primary sources of > documentation. That does present a problem. But since there is still a lot of boot code in Axiom, figuring out how boot works and documenting it does not seem to me to be any more difficult than figuring out how the boot code works and re-writing it in lisp. > > POINT 6: Boot does not give you the full power of common lisp. > > Boot cannot handle backquote, defstruct, and other primitives > not to mention CLOS, CLIM, etc. So as lisp evolves boot will > "further fork" away from the living edge of lisp ideas and > limit the axiom programmer. > This is true, but I think that perhaps by accident or by design it can be largely cast as an advantage. After all, the purpose of boot is to implement SPAD and the Axiom interpreter in a language that is higher level than common lisp - not to be a general purpose programming language. > Witness the data structures used in boot, mostly lists. > Frames are lists of lists, history is a circular list of > lists, etc. Where are the data structures? Who will redesign, > rewrite, and document boot extensions to fully cover common > lisp? This point was also recently raised by Juergen Weiss. I believe that his view is that it is possible to extend boot to include these data structures in boot syntax but that it might be preferable simply to use boot's built-in mechanism to "escape" to lisp where necessary to define these structures and then to call the data structure access methods in the usual way in boot. > > POINT 7: Boot loses program-data symmetry. > > This is a matter of religion but having been at this game for > 35 years I've seen a hundred languages built on lisp that try > to "eliminate the parens" and all will eventually die (including > Python). I don't see that this has to do with parens or not. Using parenthesis to denote nested lists is just one more syntax. > > The paren syntax is fundamental. Look at how Axiom really > works under the covers. It doesn't build boot functions, it > builds lisp functions. Why? Because you can cons up a lisp > function, compile it, and run it directly. Axiom code and > data intermingle in the algebra lisp files. The databases > have code and data in them. Data <==> Programs. Fundamental. I don't agree that lisp has anything special to offer here except perhaps by being the first language to formalism this relationship. Any higher level language in which functions are first order objects has this property, although usually not in the way it is implemented in lisp. This is especially true in functional languages like Haskell and Ocaml and probably also in Aldor. "cons'ing up a lisp function" is just a less structured way of doing the same thing. > > POINT 8: Tools don't support boot > > Emacs balances lisp parens. It lets me do lisp function > lookups. It understands the lisp syntax including escape > chars. SLIME lives in emacs and provides support. Code > walkers walk lisp code. Pretty printers understand lisp. > Debuggers understand lisp (I want to fix it in the language > I broke it in). ASDF can manipulate lisp. Programmers > speak lisp. Blank spaces don't break lisp. > > Boot is unsupported anywhere by anything. Kai Kaminiski has written a simple emacs mode for boot. See http://wiki.axiom-developer.org/AxiomInEmacs Boot is syntactically similar to Python so if there was a need for enhanced emacs support, then I expect a lot of this functionality could simply be "borrowed" from the Python extensions of emacs (and other syntax aware editors). I think it remains to be seen how some of these other lisp tools could be used with Axiom. I don't know many people who "speak lisp" except perhaps during severe rapid eye movement while asleep. ;) > > > POINT 9: Where are the programmers? > > Who will write in boot? I'm the only person likely to be > hacking the interpreter for the near future, mostly because > it is so big, ugly, unstructured, and undocumented. Even if > people code in boot to maintain the interpreter how often > will they do that? How will they maintain their skill at > writing boot? I've written many, many thousands of lines of > lisp and use it continuously. Who are the programmers who > will do that in boot? I agree! That is the #1 question. But besides Tim Daly, where are the other lisp programmers? How will they maintain their skills at using lisp in Axiom? Axiom is now open source and is at least potentially exposed to a much larger range of people than ever before. I believe that in the future there is a good chance that the people who may help to maintain and further develop Axiom will be those people who started by writing Axiom library code in SPAD. >From SPAD it is a natural step down the ladder to programming in boot. I expect that many of these people would never have had any previous exposure to lisp. So forcing them to program in lisp just to maintain Axiom would probably feel more like "falling off the ladder" than just stepping down. Of course this is currently mostly speculation (with the exception perhaps of myself and Martin Rubey). > > POINT 10: Boot was an experiment in surface syntax > > Historically Axiom was a platform for experimenting with > many research topics. One of these topics was a favourite > of Dick Jenks and Bill Burge, that of language design. > Scratchpad, now called Axiom, was changed many times to > introduce new ideas. This research is no longer the point > of Axiom and is better done elsewhere. We neither have the > expertise nor the research goal of surface language design. > Let Python do that. Or Spad/Aldor fans. I think the Python designers also (for the most part) consider their work done. I thought Spad/Aldor was an essential part of Axiom and necessarily closely related to the Axiom interpreter, so I am not sure what you mean by your last comment. > > POINT 11: Do language design where it counts. > > If we are going to devote research effort toward language > design lets do it at the Spad/Aldor level where it counts. > This is where the real language expertise should be applied. I agree. I don't see any serious need to re-design the boot language - simply to document it as it works now. > For instance, why do I have to write programs to express > functions? In Axiom you can already expression functions in other ways than imperative code. The Axiom interpreter, SPAD and Aldor have a (more or less) complete functional subset. > Why can't I write a domain using equations? Why doesn't > the Spad language support that? Could you elaborate on what you mean by "write a domain using equations"? Are you talking about an alternative to the way that Axiom implements "axioms" now simply by assertion? > > POINT 12: Future directions assume lisp > > Notice that one of the near future goals is to connect Axiom > and ACL2. ACL2 understands lisp-ish kind of languages, not > boot. Besides the fact that ACL2 is implemented in lisp, I don't see the connection between ACL2 and "lisp-ish" languages when it comes to proving program correctness. There are other tools similar to ACL2 such as Coq which is not written in lisp at all. Coq has already been used with Axiom in some research. > > > and now, on to the recent changes > > POINT 13: Don't break code until you understand it. > > I'm working hard to understand the code so the existing code > is a straightforward translation (well, mostly, with some > rewrites that I know are logically equivalent). > > Once I've completely figured out the various twists and turns, > uses and references, and other points I'll be recoding the > whole process. There is no reason (other than historical) why > there are a dozen entry points into the interpreter that set > weird, undocumented, stateful flags. All that cruft must go > away. I agree with all that, while de-emphasizing the need to remove all of the cruft right now. But I fail to see why recoding anything from boot into lisp is involved. > > So the code is ugly. At the moment. But it will get better, > cleaner, clearer, more robust, and easier to change. History > and Frames, for example will use structures that don't depend > on > (caar (cdddddr (cdddddr x))) > > It takes time. It takes work, probably 2-3 years worth of > work. I think you worry too much about how "ugly" to current code is. It works. That is what matters most to people who want to begin using Axiom now. I doubt that there is much you can do to the Axiom internal code in the way of re-writing it into lisp that would make it any more appealing to this putative new generation of Axiom users. Still I am hopeful that among the thousands of new people who have recently been exposed to Axiom, there will be some who will want to take up the challenge of dealing with what's on the inside. If/when they arrive, I think it is more important to have good documentation than it is to have pretty code. > > and now, on to the personal... > > POINT 14: I'm a lisp programmer > > (clearly) > I don't seen anything inherently evil in that. :) Cheers, Bill Page. _______________________________________________ Axiom-developer mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/axiom-developer
