[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
Hi, This email is a followup about 1. maxima opens the root directory / and stats each file found there. Then it does the same thing for the /u (home) directory... Thanks to everybody that responded to my query below. The MAXIMA-SHAREDIR variable looks fine according to the output of maxima --directories. Off list, Andrej Vodopivec remarked It could be share-subdirs-list in init-cl.lisp. If that is true, then it should be easy to remove the call to share-subdirs-list..I tried doing what Andrej suggested, and it worked perfectly. Before doing that, Maxima would state about 4000 files (including all users' home directories) on startup, and afterwards it stat'd only about 70 files. The difference in performance on some NSF filesystems is huge -- a second versus potentially *minutes*.Even the difference on sage.math.washington.edu is quite noticeable (a very fast machine with a fast network). Looking at the output of makes this very clear: strace maxima --directories out 21; grep stat out|wc -l For now we'll be patching the Maxima in Sage so that share-subdirs-list (in init-cl.lisp) falls back to the old default behavior instead of the new behavior that was introduced in the recent rewrite of init-cl.lisp.I really hope whoever rewrote init-cl.lisp can think about the significant performance regression that was caused, and find a better solution. Thanks again for all the incredibly helpful feedback! -- William On Thu, Dec 10, 2009 at 10:58 PM, Robert Dodier robert.dod...@gmail.com wrote: On 12/10/09, William Stein wst...@gmail.com wrote: 1. maxima opens the root directory / and stats each file found there. Then it does the same thing for the /u (home) directory. The sys admin believes the bottleneck is the slow response of doing a stat on each NFS home directory, especially slowed down by the home directories that are remotely NFS mounted from across the country. My first guess is that the Maxima build was misconfigured and Maxima doesn't have the right value of *MAXIMA-SHAREDIR*. What does maxima --directories report? Maxima tries to construct the list of share directories at run time. I'm guessing it's looking at / in the mistaken belief that it's the top of the share directory. 2. The user reports that maxima reads the file local/lib/ecl-9.10.2/sysfun.lsp one character at a time doing 72K read() calls of one character each. The sys admin doubts that is anywhere near the problem of the NFS stat calls, but it does seem inefficient. Well, when I try it, I see sysfun.lsp being read in blocks of 8 K. Dunno why it would try to read it one byte at a time. Maybe ECL tried to determine a suitable block size and failed? Just a guess. FWIW Robert Dodier -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Please check if this computes seconds since Epoch
John Cremona wrote: I must admit I'm puzzled it needed the .sh on the end. Are you sure you had the execute permission on the version without the .sh ? Yes, definitely. (If not, even ./unixtime.sh would have failed.) Fair enough. I don't know much about dash, but I know if you proceed the name of a script by 'sh', the script will execute even if it has no execute permissions. bash-3.2$ ls -l testcc.sh -rw-r--r-- 1 drkirkby staff 4781 Dec 8 16:35 testcc.sh bash-3.2$ sh testcc.sh GCC bash-3.2$ Anyway, it does not matter when the code is added to determine seconds since the Epoch, since the top of $SAGE_ROOT/spkg/install has #!/usr/bin/env bash In this case, the code will be run with bash - irrespective of what shell you use. I'm still puzzled though I must admit, why it says No such file or directory, since the file clearly exists. I suppose I have never used the 'dash' shell, so don't know how it behaves. Nor me. But someone will try to build Sage with it one day! Very true. It's useful to get these things checked on many shells. John Thanks John. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Linking libraries - do we need a completely different approach ?
I've noticed on several occasions people reporting issues when Sage includes a library which the system has. This causes conflicts with the linker, and can result in messages which sometimes include: (possible multiple inclusion of the same file) Readline is one such example, but is by no means unique. This has code specific to Suse, as Sage's conflicts with the readline in Suse. #Readline code for Suse Linux ## # First we check for OpenSUSE 11.1 since bash is linked dynamically with a # readline that breaks when we build Sage's readline, so we work around this # for now. if [ -f /etc/SuSE-release ]; then if [ `grep 11.1 /etc/SuSE-release /dev/null; echo $?` -eq 0 ]; then echo OpenSUSE 11.1 detected if [ -d /usr/include/readline/ ]; then On OpenSolaris, gnutls does not build easily http://trac.sagemath.org/sage_trac/ticket/7387 due to what I believe are conflicts between libgrcypt and libgpg in Solaris, and those in Sage. The linker complains: ld: fatal: recording name conflict: file /export/home/drkirkby/sage-4.2/local/lib/libgpg-error.so' and file /usr/lib/libgpg-error.so' provide identical dependency names: libgpg-error.so.0 (possible multiple inclusion of the same file) I can get around that, by deleting the following 5 set of files. * SAGE_LOCAL/include/gcrypt-module.h * SAGE_LOCAL/include/gpg-error.h * SAGE_LOCAL/include/gcrypt.h * SAGE_LOCAL/lib/libgcrypt* * SAGE_LOCAL/lib/libgpg* I believe there are other examples too, where libraries in Sage are causing conflicts with libraries in the system. I asked on the gcc-help about how to resolve this. http://archives.free.net.ph/thread/20091215.122311.bf804bb1.en.html This is one reply: # where problems arise if a library included in Sage is installed on the system too. Of cause they do! It's simply wrong to use and install libraries that the system already supplies. A sane software package detects the libraries installed in the system and only compiles/installs those that are missing. If distributed in compiled form, the only sane way is to statically link in the libraries to avoid clashes with system supplied ones. Is there a way to specify that only the first copy of the library is linked, and not any other copies gcc might find? Readline is not the only library where issues have arisen (OpenSSL libraries on OpenSolaris is another example). AFAIK there is no way. Philipp ## Of course, that is not a lot of help if one needs a never version of a library in Sage, to what the system has, so I asked about that issue. This is the response. From gcc-help ### What could you do if the system supplies a library, but it's not the latest release, and you need the latest? Either explicitely link in the static version of said library or install the dynamic library somewhere else and link it in by passing the path to ld via -L and then adding said library to the users LD_LIBRARY_PATH. Philipp ### This got me thinking about an issue I once had with Mathematica using excessive CPU time, and how Casper Dik, a really hot Sun employee solved it for me. http://www.g8wrb.org/mathematica/ Essentially Sun had made a change in a library in the Solaris kernel, and change a minimum timeout permitted from 1 ms to 1 us. Wolfram Research had selected the minimum timeout (rather than 1 ms), and so this caused serious issues with Mathematica on slower computers, as something kept timing out, so basically computing something as simple as 1+1, returned 2 quickly, but pegged the CPU at 100% forever. Casper solved this for me, by posting a bit of C code which was a revised version of the code in the Solaris kernel. I compiled that, made a shared library, then used LD_PRELOAD_64 (or its 32-bit counterpart LD_PRELOAD), to load that library in preference to the one in the kernel. I can't help but feel there perhaps needs to be some changes in the way libraries are built in Sage, as hacks like copying system libraries over Sage libraries, or deleting Sage libraries, are really just hacks. It would be nice to find a way around this problem, so that there is never any conflict with libraries. I'm amazed the problems are not more frequent than what they are, which makes me think my understanding of the problem is not complete. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: tutorial on Python functional programming for mathematicians
On Dec 15, 4:34 am, mhampton hampto...@gmail.com wrote: Thanks for pointing that out. I was looking for svd, lower-case, and missed it. actually, me too. Is there a reason why it is uppercase? I'm for renaming it to lowercase and adding an uppercase synonym for backwards compatibility. h -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: tutorial on Python functional programming for mathematicians
Harald Schilly wrote: On Dec 15, 4:34 am, mhampton hampto...@gmail.com wrote: Thanks for pointing that out. I was looking for svd, lower-case, and missed it. actually, me too. Is there a reason why it is uppercase? I'm for renaming it to lowercase and adding an uppercase synonym for backwards compatibility. h It has caught me before too. I'd rather that it was lowercase. So...submit a patch! :) Thanks, Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
On Tue, Dec 15, 2009 at 4:41 AM, Raymond Toy toy.raym...@gmail.com wrote: William Stein wrote: Hi, This email is a followup about 1. maxima opens the root directory / and stats each file found there. Then it does the same thing for the /u (home) directory... This is not right. Maxima should not be stat'ing from /, but only from maxima-sharedir. Maxima is *definitely* stat'ing stuff all over the place. I've replicated this on several very different computers using various versions of Maxima. Have you tried? It's not difficult to observe. Thanks to everybody that responded to my query below. The MAXIMA-SHAREDIR variable looks fine according to the output of maxima --directories. Can you provide additional information like what Lisp is being used? We're using ecl. You can get the exact same setup by just installing sage-4.2.1. Then we can investigate why share-subdirs-list is not doing what it's supposed to do. Thanks! I run maxima across NFS, and I haven't noticed any slowdown (except that it takes a while to load up the maxima binary across the network). To physically notice it depends on the speed of your NFS. E.g., on one of my machines the difference in startup time is about 0.4 versus 0.3. This is hard to notice.It's best to worry more about the output of strace than the actual observed speed, because of disk caching. William -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Fwd: [cython-users] Scipy09 Proceedings
Hi folks, The SciPy 2009 proceedings [1] contain four papers that cite Sage. These are: (1) Stefan Behnel, Robert W. Bradshaw, and Dag Sverre Seljebotn. Cython Tutorial. In Gaël Varoquaux, Stéfan van der Walt, and K. Jarrod Millman (ed.). SciPy 2009: Proceedings of the 8th Python in Science Conference. pages 4--14, 2009. (2) Brian D'Urso. Multiprocess System for Virtual Instruments in Python. In Gaël Varoquaux, Stéfan van der Walt, and K. Jarrod Millman (ed.). SciPy 2009: Proceedings of the 8th Python in Science Conference. pages 76--80, 2009. (3) Dag Sverre Seljebotn. Fast Numerical Computations with Cython. In Gaël Varoquaux, Stéfan van der Walt, and K. Jarrod Millman (ed.). SciPy 2009: Proceedings of the 8th Python in Science Conference. pages 15--22, 2009. (4) Pavel Solin, Ondrej Certik, and Sameer Regmi. The FEMhub Project and Classroom Teaching of Numerical Methods. In Gaël Varoquaux, Stéfan van der Walt, and K. Jarrod Millman (ed.). SciPy 2009: Proceedings of the 8th Python in Science Conference. pages 58--61, 2009. The publications page [2] has been updated accordingly. [1] http://conference.scipy.org/proceedings/SciPy2009/ [2] http://www.sagemath.org/library-publications.html -- Regards Minh Van Nguyen -- Forwarded message -- From: Dag Sverre Seljebotn da...@student.matnat.uio.no Date: Tue, Dec 15, 2009 at 9:02 PM Subject: [cython-users] Scipy09 Proceedings To: cython-us...@googlegroups.com The SciPy 09 proceedings are out, with the final versions of two papers on Cython, Cython Tutorial and Fast numerical computations with Cython. See http://conference.scipy.org/proceedings/SciPy2009/ (near the top). If Cython plays an important and direct role in getting your scientific computations done, please consider citing the relevant papers. NOTE if you read the preprint of the numerical computation paper: I claimed in the preprint (but not the talk, luckily) that Cython was 50 times faster than NumPy, but that was a bug, it was only 10 times faster. Cython developers might also be interested in the Convert-XY paper on that page for an alternative approach for Python-C++ object conversion. If we decide to, say, auto-convert a Python dict to a std::map, then this is likely what we should go for. (However such conversion, where a copy is made, doesn't play well with pass-by-reference, so I'm not sure whether that should happen). Dag Sverre -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Linking libraries - do we need a completely different approach ?
Hi there, On Tue, Dec 15, 2009 at 01:44:50PM +, Dr. David Kirkby wrote: I've noticed on several occasions people reporting issues when Sage includes a library which the system has. This causes conflicts with the linker, and can result in messages which sometimes include: (possible multiple inclusion of the same file) Readline is one such example, but is by no means unique. This has code specific to Suse, as Sage's conflicts with the readline in Suse. #Readline code for Suse Linux ## # First we check for OpenSUSE 11.1 since bash is linked dynamically with a # readline that breaks when we build Sage's readline, so we work around this # for now. if [ -f /etc/SuSE-release ]; then if [ `grep 11.1 /etc/SuSE-release /dev/null; echo $?` -eq 0 ]; then echo OpenSUSE 11.1 detected if [ -d /usr/include/readline/ ]; then A quick remark: There were a report on sage-support of it being also broken for the new openSuSE 11.2 release (released november 11). I was supposed to look at this but in the end-of-the-year rush, I didn't find the time reinstalling my machine. Moreover, since I'm using my machine for 3 different teaching courses, I didn't want to take the risk to break all. I will do it during the vacations... We most probably want to update the test: if [ `grep 11.1 /etc/SuSE-release /dev/null; echo $?` -eq 0 ]; then To match release 11.2 as well. Cheers, Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Fwd: [cython-users] Scipy09 Proceedings
Hi there, On Wed, Dec 16, 2009 at 04:05:49AM +1100, Minh Nguyen wrote: Hi folks, The SciPy 2009 proceedings [1] contain four papers that cite Sage. These are: [2] http://www.sagemath.org/library-publications.html http://arxiv.org/abs/0912.2212 I don't know who's in charge but you can also add http://arxiv.org/abs/0912.2212 submitted in an extended abstract form to the conference FPSAC'2010. Full version of the paper should follow. Cheers, Florent -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Fwd: [cython-users] Scipy09 Proceedings
Hi Florent, On Wed, Dec 16, 2009 at 4:19 AM, Florent Hivert florent.hiv...@univ-rouen.fr wrote: SNIP I don't know who's in charge but you can also add You could email sage-devel or the website maintainers [1]. http://arxiv.org/abs/0912.2212 submitted in an extended abstract form to the conference FPSAC'2010. Full version of the paper should follow. Done. The Sage publications page [2] has been updated with your paper. Thank you for informing sage-devel about it. You can find your paper under the Preprints section of the publications page. [1] http://www.sagemath.org/contact.html [2] http://www.sagemath.org/library-publications.html -- Regards Minh Van Nguyen -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] input: and output: in docstrings
Hi, It has been brought to my attention that many docstrings in Sage do not explicitly and clearly list their input and output in INPUT: and OUTPUT: blocks. There are only 2711 OUTPUT blocks and 4371 INPUT blocks in sage-4.3: wst...@sage:~/build/sage-4.3.rc0$ ./sage -grep output: |wc -l 2711 wst...@sage:~/build/sage-4.3.rc0$ ./sage -grep input: |wc -l 4371 However, there are 23706 functions (80.7% of which have doctests). So only an average of just over 10% document their output and about 20% document their input. This is evidently very frustrating for some users of other Ma's (such as Magma), where the inputs and outputs of all functions are clearly documented. INPUT/OUTPUT blocks are required according to the developer's guide: http://sagemath.org/doc/developer/conventions.html#documentation-strings The point of this email for now is just to raise awareness (though there will be a bigger push later, including changing the coverage script to complain about missing INPUT/OUTPUT blocks, parameter mismatches, etc.). * If you're writing code, include INPUT/OUTPUT blocks for every single function. * If you're refereeing code, you should feel fully justified in bouncing all code that doesn't have INPUT/OUTPUT blocks in every function. William -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] graph theory tickets
Hello, The splitup of graph.py, the switch to c_graphs as default, and the resulting awesomeness are impending! I have just gone through all the graph theory tickets needing review this morning. There is one left which I don't feel qualified to review, namely the interactive graph editor #1321. Could someone more familiar with the notebook review this? If there are any other graph theory tickets out there, you should be motivated to get them reviewed and merged soon, as the above mentioned changes will be quite invasive to the graph theory corner of Sage, and the less open graph theory tickets with patches when we do this, the easier it will be on everyone. The current plan is to apply these changes (barring disaster, of course) at the end of the 4.3.1 release cycle. So, if there is something in graph theory you've been working on getting in, this week is the best time. Although I am still busily working on my thesis, I might be convinced to review a few things, at least during this transition :) -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
On Tue, Dec 15, 2009 at 9:15 AM, Raymond Toy toy.raym...@gmail.com wrote: William Stein wrote: On Tue, Dec 15, 2009 at 4:41 AM, Raymond Toy toy.raym...@gmail.com wrote: We're using ecl. You can get the exact same setup by just installing sage-4.2.1. Thanks, but no, I don't really want to install sage-4.2.1. I would be willing to give you an account on a (very nice) machine with 128GB RAM with sage pre-installed in your home directory, if that would be useful. You could later use said machine for maxima development and testing if you want. The NSF gave us some money for hardware for supporting projects in Sage such as Maxima. Then we can investigate why share-subdirs-list is not doing what it's supposed to do. Thanks! Can you do the following experiments for me? After starting maxima (with or without your fix to remove all the stat'ing of files), enter the following: :lisp *maxima-sharedir* That should produce a sensible value. :lisp (directory (pathname (concatenate 'string *maxima-sharedir* /))) This should produce a list of one item consisting of *maxima-sharedir* with a / appended. If that works, then just run :lisp (share-subdir-list) This should produce a list of directories starting from *maxima-sharedir*, not from root. Here's the output from the above: sage subshell$ strace maxima --directories out 21; grep stat out|wc -l 3088 /home/wstein/build/sage sage subshell$ /home/wstein/build/sage sage subshell$ maxima ;;; Loading #P/scratch/wstein/build/sage/local/lib/ecl/defsystem.fas ;;; Loading #P/scratch/wstein/build/sage/local/lib/ecl/cmp.fas ;;; Loading #P/scratch/wstein/build/sage/local/lib/ecl/sysfun.lsp Maxima 5.19.1 http://maxima.sourceforge.net Using Lisp ECL 9.8.4 Distributed under the GNU Public License. See the file COPYING. Dedicated to the memory of William Schelter. The function bug_report() provides bug reporting information. (%i1) :lisp *maxima-sharedir* /home/wstein/build/sage/local/share/maxima/5.19.1/share (%i1) :lisp (directory (pathname (concatenate 'string *maxima-sharedir* /))) (/scratch/wstein/build/sage/local/share/maxima/5.19.1/share/) (%i1) :lisp (share-subdir-list) Maxima encountered a Lisp error: The function SHARE-SUBDIR-LIST is undefined. Automatically continuing. To reenable the Lisp debugger set *debugger-hook* to nil. (%i1) I run maxima across NFS, and I haven't noticed any slowdown (except that it takes a while to load up the maxima binary across the network). To physically notice it depends on the speed of your NFS. E.g., on one of my machines the difference in startup time is about 0.4 versus 0.3. This is hard to notice. It's best to worry more about the output of strace than the actual observed speed, because of disk caching. Stracing my maxima (CVS) build on linux with ecl, I see that maxima (from my linux box to a mac via NFS over wifi) stats just the expected files. To me, this means the default maxima build is doing what it's supposed to be doing. Perhaps sage is changing something else, somehow. Ray -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
On Tue, Dec 15, 2009 at 11:37 AM, William Stein wst...@gmail.com wrote: (%i1) :lisp *maxima-sharedir* /home/wstein/build/sage/local/share/maxima/5.19.1/share (%i1) :lisp (directory (pathname (concatenate 'string *maxima-sharedir* /))) (/scratch/wstein/build/sage/local/share/maxima/5.19.1/share/) So far so good (%i1) :lisp (share-subdir-list) Maxima encountered a Lisp error: The function SHARE-SUBDIR-LIST is undefined. Oops, it should be (share-subdirs-list) . Robert Dodier -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
On Tue, Dec 15, 2009 at 10:41 AM, Robert Dodier robert.dod...@gmail.com wrote: On Tue, Dec 15, 2009 at 11:37 AM, William Stein wst...@gmail.com wrote: (%i1) :lisp *maxima-sharedir* /home/wstein/build/sage/local/share/maxima/5.19.1/share (%i1) :lisp (directory (pathname (concatenate 'string *maxima-sharedir* /))) (/scratch/wstein/build/sage/local/share/maxima/5.19.1/share/) So far so good (%i1) :lisp (share-subdir-list) Maxima encountered a Lisp error: The function SHARE-SUBDIR-LIST is undefined. Oops, it should be (share-subdirs-list) . and... utomatically continuing. To reenable the Lisp debugger set *debugger-hook* to nil. (%i1) :lisp (share-subdirs-list) (affine algebra algebra/charsets algebra/solver calculus colnew colnew/lisp combinatorics contrib contrib/altsimp contrib/amatrix contrib/bitwise contrib/boolsimp contrib/descriptive contrib/diffequations contrib/diffequations/tests contrib/distrib contrib/ezunits contrib/finance contrib/format contrib/fourier_elim contrib/fractals contrib/fresnel contrib/gentran contrib/gentran/man contrib/gentran/test contrib/gf contrib/graphs contrib/Grobner contrib/integration contrib/levin contrib/lurkmathml contrib/maximaMathML contrib/mcclim contrib/namespaces contrib/noninteractive contrib/nset contrib/numericalio contrib/pdiff contrib/prim contrib/rand contrib/sarag contrib/simplex contrib/simplex/Tests contrib/solve_rec contrib/state contrib/stats contrib/stringproc contrib/unit contrib/vector3d contrib/Zeilberger diffequations diff_form draw dynamics hypergeometric integequations integration lapack lapack/blas lapack/lapack lbfgs linearalgebra lisp-utils macro matrix minpack minpack/lisp misc numeric orthopoly physics simplification sym tensor trigonometry utils vector) -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: [Maxima] Maxima stats huge numbers of files at startup?
On Tue, Dec 15, 2009 at 11:42 AM, William Stein wst...@gmail.com wrote: (affine algebra algebra/charsets algebra/solver calculus colnew colnew/lisp : : : orthopoly physics simplification sym tensor trigonometry utils vector) Well, that looks right. I don't see any indication that Maxima was searching all of / . Do you see Maxima looking in all of / in the calls to stat? If not, what do you get for the above steps if you run it on a system in which Maxima searches all of / ?? FWIW Robert Dodier -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: input: and output: in docstrings
On Dec 15, 10:16 am, William Stein wst...@gmail.com wrote: Hi, It has been brought to my attention that many docstrings in Sage do not explicitly and clearly list their input and output in INPUT: and OUTPUT: blocks. There are only 2711 OUTPUT blocks and 4371 INPUT blocks in sage-4.3: wst...@sage:~/build/sage-4.3.rc0$ ./sage -grep output: |wc -l 2711 wst...@sage:~/build/sage-4.3.rc0$ ./sage -grep input: |wc -l 4371 However, there are 23706 functions (80.7% of which have doctests). So only an average of just over 10% document their output and about 20% document their input. This is evidently very frustrating for some users of other Ma's (such as Magma), where the inputs and outputs of all functions are clearly documented. (Some of these have no inputs except for self, right? So 10% is a bit low, because I don't think we need to list self as an input each time.) INPUT/OUTPUT blocks are required according to the developer's guide: http://sagemath.org/doc/developer/conventions.html#documentation-strings You can also use Sphinx/reST markup for this, described in the same place. Here is an example of how that looks: http://sagemath.org/doc/reference/sage/homology/ simplicial_complex.html#sage.homology.simplicial_complex.SimplicialComplex -- John -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: graph theory tickets
Hi Robert (and Nathann), I won't have time to do the fix Nathann C. recommended in http://trac.sagemath.org/sage_trac/ticket/6522 (which isn't so much a fix as using new functionality) for changing the copy to __copy__ in graphs until at least Thursday. But presumably it would be good to merge this now so that no one has to do it after the c_graph switch. Could you (or Nathann) do the not very difficult fix of using the functionality at http://trac.sagemath.org/sage_trac/ticket/7515 to define the deprecations? I would be happy to review it - with my current work setup, it is MUCH easier for me to review tickets than to make changes in patches. Alternately, Nathann could simply allow the patch to go through after all, though that does make http://trac.sagemath.org/sage_trac/ticket/7559 harder, it is true. Thanks for considering it, - kcrisman On Dec 15, 1:22 pm, Robert Miller rlmills...@gmail.com wrote: Hello, The splitup of graph.py, the switch to c_graphs as default, and the resulting awesomeness are impending! I have just gone through all the graph theory tickets needing review this morning. There is one left which I don't feel qualified to review, namely the interactive graph editor #1321. Could someone more familiar with the notebook review this? If there are any other graph theory tickets out there, you should be motivated to get them reviewed and merged soon, as the above mentioned changes will be quite invasive to the graph theory corner of Sage, and the less open graph theory tickets with patches when we do this, the easier it will be on everyone. The current plan is to apply these changes (barring disaster, of course) at the end of the 4.3.1 release cycle. So, if there is something in graph theory you've been working on getting in, this week is the best time. Although I am still busily working on my thesis, I might be convinced to review a few things, at least during this transition :) -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Browser math rendering
Guys, have you heard about this? http://www.mathjax.com Ram. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Browser math rendering
cool-RR wrote: Guys, have you heard about this? http://www.mathjax.com Yes, isn't it exciting?! I can't wait for it. See this post from this morning: http://groups.google.com/group/sage-devel/browse_thread/thread/a8518afd8922ddc0 Thanks, Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Linking libraries - do we need a completely different approach ?
Hmm, I don't quite get the point of your message. Installing the dynamic library somewhere else and link it in by ... adding said library to the users LD_LIBRARY_PATH ... is pretty much exactly the way things are designed and supposed to work (and actually do work to a large extent) in Sage. I don't think LD_PRELOAD in its various incarnations is portable enough for the Sage purposes (I haven't checked though). The problems always arise with spkgs whose build systems are complex and involved (and trying to be smart ...), so it is hard to tell them that e.g. binaries they produce, that depend on certain libraries, should look up these libraries dynamically at runtime in LD_LIBRARY_PATH (and additionally in DYLD_LIBRARY_PATH on Mac OS X 10.4 and 10.5). And nowhere else! Sometimes this could be solved only by some dirty tricks like referencing explicitly libfoo.1.2.3.so instead of libfoo.so (the libpng case on Mac OS X, as far as I remember). This approach comes wih a certain cost --- in order for things to work together, you are more or less forced to have the sane environment you get by typing ./sage -sh, or else to work completely outside of it, but you can hardly mix. (There was a recent thread I don't remember exactly on calling exterior functionality from inside a Sage session, where you had to use some raw calling mechanism, forgetting these environment settings.) However, given that one might want (or need ...) to be able to work in two or more possibly incompatible Sage environments (e.g. different Sage versions rsp. installations) at the same time without installing and re-installing, this is rather a feature than a bug. In a nutshell: there has been put much effort in the dynamic libraries/ LD_LIBRARY_PATH approach already. But more hard work awaits us, e.g. w.r.t. readline and gnutls. There have been discussions of distributing bash itself within Sage, because bash links in readline dynamically (which is the heart of the problem on Suse) AFAIR. The same with gcc, there already exist some versions of a gcc.spkg. From a distance, Sage would profit from having some kind of inventory of all the dynamic libraries its parts depend on, and from a flexible mechanism to switch between using either versions of these installed already somewhere in the system, or else using a version (to be) installed under SAGE_LOCAL/lib/. The debianization has paved some way in this direction, but currently, I don't think we have the manpower to stem the inevitable complexity of introducing such an infrastrucure, let alone only maintaining it. OTOH, there are already now various things which could be bundled together in some kind of Sage Registry, e.g. the sage-location and sage-flags text files, the spkg/installed/ directory content, ... Cheers, Georg -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Browser math rendering
Great. Sorry for double-posting, apparently the search feature is broken in Google Groups. Also, I dream that it will be possible to also *type* math like that in the browser, and have it become pretty in real-time. But this is impressive right now as it is. Ram. On Dec 16, 12:42 am, Jason Grout jason-s...@creativetrax.com wrote: cool-RR wrote: Guys, have you heard about this? http://www.mathjax.com Yes, isn't it exciting?! I can't wait for it. See this post from this morning:http://groups.google.com/group/sage-devel/browse_thread/thread/a8518a... Thanks, Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Browser math rendering
cool-RR wrote: Great. Sorry for double-posting, apparently the search feature is broken in Google Groups. Also, I dream that it will be possible to also *type* math like that in the browser, and have it become pretty in real-time. But this is impressive right now as it is. Like this? http://www.math.union.edu/~dpvc/talks/2006-12-08.IMA/editor.html Davide has expressed interest in building a similar editor on top of the MathJax codebase, so hopefully this is on the horizon! Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: Linking libraries - do we need a completely different approach ?
Georg S. Weber wrote: Hmm, I don't quite get the point of your message. Sorry, I was trying to get some discussion, and hopefully understanding, of why these issues arise, with a hope to trying to fix them. It's clear the present situation has 'issues' and to my knowledge, nobody has a foolproof way of building Sage without having to rely on 'hacks' on some operating systems. Installing the dynamic library somewhere else and link it in by ... adding said library to the users LD_LIBRARY_PATH ... is pretty much exactly the way things are designed and supposed to work (and actually do work to a large extent) in Sage. I don't think LD_PRELOAD in its various incarnations is portable enough for the Sage purposes (I haven't checked though). LD_LIBRARY_PATH tells the program where to look for libraries it can't find. I'm not aware of how it is supposed to behave if there are system libraries and libraries of the same name in LD_LIBRARY_PATH. Can we be sure that system libraries will be used in preference to those in LD_LIBRARY_PATH, or can we be sure those in LD_LIBRARY_PATH will be used in preference to those in the system? It seems to me, if the library is on the system in somewhere like /usr/lib, then it will be loaded from there, despite the fact LD_LIBRARY_PATH might specify a location where another copy resides. For Solaris at least, LD_PRELOAD can be used to force something to be loaded in preference to what exists on the system. I've no idea how portable that is. I'm not suggesting it is the way forward. The problems always arise with spkgs whose build systems are complex which is not unusual. and involved (and trying to be smart ...), so it is hard to tell them that e.g. binaries they produce, that depend on certain libraries, should look up these libraries dynamically at runtime in LD_LIBRARY_PATH (and additionally in DYLD_LIBRARY_PATH on Mac OS X 10.4 and 10.5). And nowhere else! Sometimes this could be solved only by some dirty tricks like referencing explicitly libfoo.1.2.3.so instead of libfoo.so (the libpng case on Mac OS X, as far as I remember). But from what I understand, setting LD_LIBRARY_PATH does *not* guarantee that a program will first look in the directories specified in LD_LIBRARY_PATH, then look in system directories like /usr/lib. This approach comes wih a certain cost --- in order for things to work together, you are more or less forced to have the sane environment you get by typing ./sage -sh, or else to work completely outside of it, but you can hardly mix. (There was a recent thread I don't remember exactly on calling exterior functionality from inside a Sage session, where you had to use some raw calling mechanism, forgetting these environment settings.) However, given that one might want (or need ...) to be able to work in two or more possibly incompatible Sage environments (e.g. different Sage versions rsp. installations) at the same time without installing and re-installing, this is rather a feature than a bug. In a nutshell: there has been put much effort in the dynamic libraries/ LD_LIBRARY_PATH approach already. But more hard work awaits us, e.g. w.r.t. readline and gnutls. There have been discussions of distributing bash itself within Sage, because bash links in readline dynamically (which is the heart of the problem on Suse) AFAIR. The same with gcc, there already exist some versions of a gcc.spkg. I can't help but feel distributing bash is just a hack. It's not really solving the underlying problem. We might resolve the issues on Suse with readline by distributing bash, but it's anyones guess what will be the next problem, on the next platform. Given that some versions of gcc are far more stable than others, and that not all gcc's are built with Fortran support, I can see a more convincing argument for distributing gcc. But for very different reasons, which have nothing to do with libraries. From a distance, Sage would profit from having some kind of inventory of all the dynamic libraries its parts depend on, and from a flexible mechanism to switch between using either versions of these installed already somewhere in the system, or else using a version (to be) installed under SAGE_LOCAL/lib/. The debianization has paved some way in this direction, but currently, I don't think we have the manpower to stem the inevitable complexity of introducing such an infrastrucure, let alone only maintaining it. I can't see that being very practical if a program is going to support multiple operating systems. I was wondering if something could be done with version numbers of libraries, or perhaps all libraries are built with the name 'sage' at the front, so there is libsagereadline for example. But I suspect that is the name of the functions in the library which are important, not the library name, so that would probably not work. Dave -- To post to this group, send an email to
Re: [sage-devel] Linking libraries - do we need a completely different approach ?
Florent Hivert wrote: Hi there, On Tue, Dec 15, 2009 at 01:44:50PM +, Dr. David Kirkby wrote: I've noticed on several occasions people reporting issues when Sage includes a library which the system has. This causes conflicts with the linker, and can result in messages which sometimes include: (possible multiple inclusion of the same file) Readline is one such example, but is by no means unique. This has code specific to Suse, as Sage's conflicts with the readline in Suse. #Readline code for Suse Linux ## # First we check for OpenSUSE 11.1 since bash is linked dynamically with a # readline that breaks when we build Sage's readline, so we work around this # for now. if [ -f /etc/SuSE-release ]; then if [ `grep 11.1 /etc/SuSE-release /dev/null; echo $?` -eq 0 ]; then echo OpenSUSE 11.1 detected if [ -d /usr/include/readline/ ]; then A quick remark: There were a report on sage-support of it being also broken for the new openSuSE 11.2 release (released november 11). I was supposed to look at this but in the end-of-the-year rush, I didn't find the time reinstalling my machine. Moreover, since I'm using my machine for 3 different teaching courses, I didn't want to take the risk to break all. I will do it during the vacations... We most probably want to update the test: if [ `grep 11.1 /etc/SuSE-release /dev/null; echo $?` -eq 0 ]; then To match release 11.2 as well. Cheers, Florent This is the problem with hacks like this. We don't really solve the issue. It comes back to haunt you with the next operating system, or the next release of an old operating system. Dave -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] Re: Linking libraries - do we need a completely different approach ?
Dr. David Kirkby wrote: Georg S. Weber wrote: Hmm, I don't quite get the point of your message. Sorry, I was trying to get some discussion, and hopefully understanding, of why these issues arise, with a hope to trying to fix them. It's clear the present situation has 'issues' and to my knowledge, nobody has a foolproof way of building Sage without having to rely on 'hacks' on some operating systems. Installing the dynamic library somewhere else and link it in by ... adding said library to the users LD_LIBRARY_PATH ... is pretty much exactly the way things are designed and supposed to work (and actually do work to a large extent) in Sage. I don't think LD_PRELOAD in its various incarnations is portable enough for the Sage purposes (I haven't checked though). LD_LIBRARY_PATH tells the program where to look for libraries it can't find. I'm not aware of how it is supposed to behave if there are system libraries and libraries of the same name in LD_LIBRARY_PATH. Can we be sure that system libraries will be used in preference to those in LD_LIBRARY_PATH, or can we be sure those in LD_LIBRARY_PATH will be used in preference to those in the system? It seems to me, if the library is on the system in somewhere like /usr/lib, then it will be loaded from there, despite the fact LD_LIBRARY_PATH might specify a location where another copy resides. For Solaris at least, LD_PRELOAD can be used to force something to be loaded in preference to what exists on the system. I've no idea how portable that is. I'm not suggesting it is the way forward. While not a guarantee, here's what I get from the manpage of ld.so on Ubuntu (x86_64): The necessary shared libraries needed by the program are searched for in the following order * Using the environment variable LD_LIBRARY_PATH (LD_AOUT_LIBRARY_PATH for a.out programs). Except if the exe‐ cutable is a setuid/setgid binary, in which case it is ignored. * From the cache file /etc/ld.so.cache which contains a compiled list of candidate libraries previously found in the augmented library path. * In the default path /lib, and then /usr/lib. So it looks like, for Linux, at least, setting LD_LIBRARY_PATH is sufficient. Also, there's this: LD_PRELOAD A whitespace-separated list of additional, user-specified, ELF shared libraries to be loaded before all others. This can be used to selectively override functions in other shared libraries. For setuid/setgid ELF binaries, only libraries in the standard search directories that are also setgid will be loaded. I don't know very much about these issues, but the above might help, at least for (Ubuntu) linux. Jason -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] Re: input: and output: in docstrings
On Wed, Dec 16, 2009 at 2:46 AM, John H Palmieri jhpalmier...@gmail.com wrote: You can also use Sphinx/reST markup for this, described in the same place. Here is an example of how that looks: http://sagemath.org/doc/reference/sage/homology/ simplicial_complex.html#sage.homology.simplicial_complex.SimplicialComplex I also like the Sphinx markup since it adds some semantics to the markup instead of just using a plain list. It makes it easier to extract this information for other purposes in the future if we need it. The only reason everything wasn't switched to it before was the the previous input and output blocks were varied enough that it wasn't really feasible to map the old format to this one. --Mike -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
