On Monday 16 October 2006 20:05, Alejandro Jakubi wrote: > Tim, Alfredo, CY
> > One doesn't always want to achieve that. The first question to be asked > > should be "is this new behavior wrong, or was the old behavior wrong?" > > (In more subtle cases - obviously a crash is wrong.) It is possible the > > 2006 result was wrong. That's actually an objection I have heard in the > > past to > > Correction is one issue and reproducibility is another one. Fair enough. > By the way, one of my interests in looking at different CAS is checking > results in for errors... > > But the need of reproducibility is basic in science and the publication of > scientific results needs dating to set the record. By merging papers and > code, the need of reproducibility of code results is made explicit. There > are many possible sources of errors in a paper, and errors arising in bugs > of the CAS that could have been used for calculations is just one of them. > Whatever happens after publication, eg whether the bug is detected and > patched, the calculations are made again, a corrected version of the paper > is published, etc, is a different issue. Anybody should be able to able to > reproduce a given result as it was published, right or wrong. That's not possible in general - if nothing else, the hardware platforms on which software is run will not remain static. A result turned out on a PDP-8 might or might not be reproducible today - doing so reliably would take too much time and effort by way of either finding a working PDP-8 or building one. Emulators might work but introduce more correctness questions. That's why I think software needs to have formal statements of what is required from the underlying support systems, and the ability to test that those systems do in fact provide that. Give sufficient testing, it should be workable to have a running system virtually indefinitely. (In a sense that's what porting is, but rather than "does it run" the test could be "does EVERYTHING work" which is better for confidence.) Lisp helps in this by being virtually a "self contained" environment, and in gcl Paul Dietz's extensive ANSI test suite can be found. That provides a good benchmark for whether a lisp is performing as advertised. I understand what you are saying, but the nature of computers makes what you are asking for very difficult. The best we can hope to do is make computers like (say) different X-ray diffraction setups - a result should be reproducible from one model to another, and if something is not reproducible step 1 is to suspect the setup of the experiment, 2 the equipment, and 3 the result against which you are comparing your result. Being able to reproduce incorrect results is NOT possible in general, either in computers or elsewhere. > As I observe in Maple, along the late 12 years, there has been changes of > the most diverse nature in this system. They include, patched or obsoleted > libraries, changes in the language and in the format of the worksheet. Just > observe that several of the Maple entries in the Rosetta document are > currently (ie for Maple 10) obsolete. In cases that syntax is no longer > working. And it is just a few years since it was written! Right. I view this as a problem with how software is designed. Rather than "the best of all possible programs" we tend to shoot for "something that works." Experience is always the best teacher, so we learn from our mistakes. The benefit of open source is that those old versions are still available, and in theory they can be made to run. If you NEED to make it run to check an inconsistency between an old and new system then there is a problem (that shouldn't have to happen) but at least in theory it can be done (with a bit of effort.) Commercial software does not allow this possibility. No one, open source or closed, will maintain an old version with known problems indefinitely just for the sake of being able to reproduce an incorrect result - it is not a good use of limited time. The correct way is to identify if the old result or the new is correct, and if so why. Yet another reason I like the idea of automatically generating formal proofs. But if you REALLY need to reproduce the old result you can (in theory and in open source) put in the work to make it happen. > CAS, when successful, are long term projects. This means that different > generations of developers would work along the time, each one with its own > preferences. What is better, whether a development following rigid rules or > accommodating to circumstances, seems to be a matter of taste. The later > model can be easily observed in Maple that seems as an accumulation of > "geological strata" with commands working in very different ways, depending > on its era of development. To me the logical solution to that problem (insofar as it CAN be solved) is to make working with the CAS as much as possible like working with the mathematics itself. Sticking close to the mathematics should provide an "oracle" for what the right way to do things is. Of course this will only work up until a point, but you could compare it to the body of literature - you will probably notice strata of paper writing styles over the decades and centuries too. > Most open source projects are rather new. It will be interesting to see > whether they endure decades and how do they evolve. In particular whether a > Axiom 2036 will handle correctly the a document written with the syntax of > Axiom 2006. Unlikely. I would not regard Axiom 2006 as anything like a stable product. There is a great deal of change that we KNOW has to take place, and that removes any realistic claim we might have to being stable in my book. Maxima versions are also technically development versions. We have no formal language definition for SPAD, and Maxima has no formal language definition for it's language - expecting reproducibility is not realistic in such conditions. It will probably be CLOSE in many cases (Axiom and Maxima do get many things right, or they wouldn't exist in the first place) but a guarantee is something else entirely. > The evolution of the TeX system up to now, where almost any document > written in the past can be processed today as it was at its time, gives > hope that this example could be followed. TeX was and is an unusual case - almost all of its core logic was finished before it became a major player. LaTeX is a more realistic representation, and older LaTeX documents can occasionally present problems in my experience. TeX took a very long time to develop, but in the end it was finished. Most software is not finished or even close. I would like Axiom to be finished in terms of everything except its mathematical abilities (just as TeX is finished except in terms of addons like LaTeX and other convenience packages) but we aren't close yet. That level of polish almost never happens in software development. TeX is famous for a reason! > > general problem. Why shouldn't it be possible to do all of this work > > inside one larger, robust, and powerful framework? Then each new > > algorithm and tool would be immediately available for use in any new > > work. > > Agreed. The utopia of the universal system is very nice! Very practical too in that it would end the duplication of effort that must go into maintaining different systems. Who wants to see all those researchers out there handling the mundane, boring tasks of build systems, packaging, designing basic support environments and libraries for their calculations? They should be doing what only they can do, not project management! Ultimately this is all mathematics and we shouldn't be having to re-invent so many wheels. Of course I have the same feeling with graphical toolkits, and look where we stand there :-(. That's why it's important to study existing systems - how they do things and more importantly why they do them that way. > > Axiom's design gives me hope for this goal - it appears to be designed > > generally enough that it can scale. But there are many years of work > > ahead to make it a well documented and robust system. > > But note. The usefulness of a system may depend on factors rather > independent of the quality of design. For instance the size and diversity > of its community of users. Quite frequently I have found that the Maple > package that makes the job was contributed by a user... Yes, that's true. The only solution I can think of is to incorporate as many useful/important ideas as possible from those communities into a system that offers something so compelling it wouldn't be reasonable to ignore it. (Say, formal verification of the entire system and being able to have a full axiomatic proof of a result generated for you.) Right now, everybody builds in swamps because that's all there is. As you illustrated with your changing Maple versions, the ground can shift all over the place. Axiom should strive to be solid ground - a platform so compelling that the cost of NOT investigating and using it is too high to tolerate. It has a good start, but won't be there for many years. Eventually I think it could and should become the TeX of computer algebra. Cheers, CY _______________________________________________ Axiom-developer mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/axiom-developer
