On Dec 12, 2007 11:05 PM, Fernando Perez <[EMAIL PROTECTED]> wrote: > > (4) At some point though, I had to chose the numerical component for > > sage, since in > > fact my more recent research interests had broadened > > to more numerical things, and those of a lot of new Sage users (the > > number of Sage > > users increased by a huge amount in the last few months), were also > > fairly numerical. > > Given that I chose Python as the center of gravity for Sage, I had to > > find something > > that fit well with Python. The most natural choice is Scipy/Numpy, since it > > is a Python library supported by a successful company funding its > > development (http://www.enthought.com/), and having met the employees > > of that company I > > think it will stay successful. That said, I (and my student Josh > > Kantor) had substantial > > trouble -- which we eventually overcame through much hard work -- > > building Scipy from source on a wide range of platforms, so we tried very > > hard > > to make Octave serve as the numerical component of Sage. > > One thing that I'd really like to see is this info flowing back to the > numpy/scipy lists. If you guys had trouble with builds, I don't think > I ever saw a post from either of you on the lists. Nobody from numpy > can help you if we don't know you have difficulties. And if you fixed > anything in the build process, the right thing to do is to send those > patches back upstream, so the problem is really fixed at the source. > > One of Sage's huge strengths is the robustness of its build (through > lots of painful work, I know). But in areas where what you're doing > is getting an underlying package to build better, please send that > information back upstream. Otherwise you'll be forever re-fixing the > same problem over and over, and other users of the same package > outside of Sage will never benefit from the fixes. > > I do my best to keep an eye on things on sage-dev and help where > possible, but I do spend more time on the numpy lists and do miss > things. The right channel to discuss problems with numpy or scipy are > their respective mailing lists.
Josh Kantor -- correct me if I'm wrong since you did more work on this than I did, but just to clarify, that's not at all the sort of situation we were in -- the problem wasn't with numpy / scipy but with Fortran, interactions between Fortran and numpy/scipy, dependencies that numpy/scipy have, and that none of us really knew anything about the modern world and culture of Fortran at the outset, and the discussions with some Fortran experts (who are your friends, btw) though very helpful, didn't really take into consideration all the constraints we were working with at the time. For starters, by far the biggest hurdle was dealing with the FORTRAN dependency, i.e., how to make sure end users can build Sage without requiring them to install Fortran; installing Fortran on OS X is a confusing experience, to put it mildly; it can also be hard on Linux as well (e.g., imagine an old Linux box in a math department that runs Red Hat 3, doesn't have Fortran, all the sysadmins have quit, and you want to build Sage on it...) Add to that the fact that there are numerous possible "Fortran" implementations -- g95, gfortran, g77, f77, f2c + gcc, etc.; moreover, each has significant strengths and weaknesses, and *none* build from source as a normal C program using GCC (unless you build GCC itself from source as well). The point is just that there numerous choices, and this doesn't exactly make things easier. We could have just said "go install Fortran if you want to use Sage", and watched as Sage "Just Didn't Work", but that's not the Sage way to do things. Once we finally found a Fortran solution that satisfied our constraints, at least for now, we then had to figure out how to convince numpy/scipy to build using it (and not something else random on the user's system). This was also a lot of work (that Josh Kantor did), but it wasn't anything that needed to be contributed upstream to numpy -- it was just a matter of reading and understanding how the numpy distutils system works then applying it exactly as its intended to be applied. This took quite a long time to get all the bugs ironed out, since with different Fortran compilers, different libraries have to be linked in, etc. There were also some major surprises coming from incompatibilities between numpy and scipy versions at the time, which were of course documented online once we look. Finally, there were various library dependencies, e.g., a Fortran BLAS, Fortran LAPACK, etc., that scipy has, and that complicates things even further -- since we have to figure out how to build those. And it all has to be consistent. At this point, you're probably saying "just use Debian".... ;-) Unfortunately, that's not an option, given the goals of Sage, and that half of us use OS X, and fink isn't useful for this sort of stuff. Incidentally, the ATLAS build process has improved dramatically during the last 2 years, and the next release of Sage will include ATLAS. That said, evidently Josh Kantor and Michael Abshoff -- who is amazingly experienced at all things involving building Atlas or anything else -- still can't build ATLAS from source on OSX with sufficient support for what Sage requires (e.g., as a dynamic library with Fortran support). So all the problems mentioned above remain even after we ship ATLAS, until the problems mentioned in this paragraph are resolved. (OS X ships a version of ATLAS, which is of course not as good as the one we could build from source once we fix the issues.) So basically, at no point were there any real non-documented bugs or problems with numpy or scipy that we fixed. It was more that installing those libraries and all they depend on *from scratch* in the context of the constraints imposed by Sage had a steep learning curve. But at the end it wasn't that anything was broken in numpy/scipy, so there isn't anything to give back or upstream. I just hope that Sage somehow ends up increasing the number of people who are numpy/scipy users but don't have time for much of a learning curve (and who won't use Debian or the Enthought Python distribution for whatever reason). -- William -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---
