On Sun, Feb 9, 2025 at 3:06 AM Sébastien Labbé <[email protected]> wrote: > > With Ubuntu 22.04, compiling 10.6.beta5 worked. Thank you. > > Running doctests with optional and external packages that I care about yields > a lot of failures. A summary of them is pasted below. It seems that the big > amount of developers we gain by going to github or the extraordinary > automatic machines did not replaced humans taking care of these failures.
(I started to write a short answer, but it quickly become a long one. Sorry.) As humans engaged in pushing through their grand visions rather than fixing bugs, and automatic machines they added haven't really worked well with testing optional packages, the problems we had only got worse. The failures are mostly due to the legacy code noone active in the project understands well enough, or pehaps not at all: * ginac/pynac (the fork diverged from Ginac quite far many years ago, it's an old C++ code, so no outside help, although Ginac project is still alive; the original developer of pynac fork has long left the project completely) * libsingular interface (libsingular is undocumented, and while developers at Kaiserslautern are responding to our questions, their primary interests are integration into the Oscar project; the interface developer has his interests shifted to other projects, and he's a very busy academic now, managing big grants) * (lib)maxima interface is faithfully reproducing Maxima bugs, old and new, not something upstream is fixing fast enough. * libgap interface has some bugs, which, again, are partly on us, partly on GAP. As GAP's primary current goal is integration into Oscar, they are stretched very thin. The primary developers of the interface are either not active anymore, or too busy. (the gappy effort, of spinning the libgap python interface into a separate project, has stalled) * givaro/linbox/fflas-ffpack collection has not seen a release in about 4 or 5 years. Given that C++ compilers evolve quite fast, this is a growing probem. * cypari/cypari2 interface to pari/gp does not have the original developers working on them any more, or not much, while there are unresolved memory management problems. All these interfaces have a problem which comes from the C/C++ compilers optimising more and more. They are all creating handles for externally managed in C/C++ code objects, and the handles have to stay in sync with these external objects in systems which is some cases have their own built-in GCs. One has to orchestrate the working together of the latter GCs with Python's own GC. It's particularly tricky when an external system has to recover from an error, e.g. an error in a function call. A scary bug we saw with libgap not long ago is of this sort - in calls to libgap an optimising compiler allocates these handles on registers, which are invalidated immediately after the call, producing a crash (on Python 3.12 - earlier it would be "just" a memory leak). The high level decision making is almost paralysed in the project. There are divergent opinions on how it has to go forward, and the wishful thinking that it can go on as it is has (or had in the recent past) too much weight to even allow to agree on changing the outdated motto "creating a FOSS alternative to big Ms". The effort to create a standing council/technical committee has stalled - the project has no real leadership of any form, no decisions In my opinion we only have a fighting chance to survive, in a long run, if we stop grandstanding, humble down to being a good, normal member of the Python universe, shed as many vendored dependencies as possible (in particular almost all the vendored Python packages), simplify the build system to a sane level, and concentrate on fixing the above grave issues. And still it's only a fighting chance, as the amount of work to be done is huge, this kind of work is tricky software engineering work, and financing it is hard. > > I am going to give some efforts to treat these failures during Sage Days 128 > that we organize in Le Teich [https://wiki.sagemath.org/days128]. At least > try to split the issues and create relevant tickets. > > Do you know if I can still use my old meta-ticket to centralize the > organization of those failing doctests? > meta-ticket: failing internet/optional doctests #25536: > https://github.com/sagemath/sage/issues/25536 yes, sure, #25536 is a normal, open, GitHub issue, it points to the correct GitHub issues/PRs, etc. You can do as you please with it. Dima > > Cheers, > > Sébastien > > *** > > Running doctests with ID 2025-02-08-19-46-27-f417099d. > Git branch: develop > Git ref: 10.6.beta3-426-gdc99dc846c5-dirty > Running with SAGE_LOCAL='/home/slabbe/GitBox/sage/local' and > SAGE_VENV='/home/slabbe/GitBox/sage/local/var/lib/sage/venv-python3.10' > Using > --optional=build,ccache,debian,dot2tex,external,glucose,kissat,notedown,pandoc_attributes,pip,rst2ipynb,sage,sage_spkg > Features to be detected: > 4ti2,SAGE_SRC,benzene,bliss,buckygen,conway_polynomials,coxeter3,cplex,csdp,cvxopt,cvxopt,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_cubic_hecke,database_ellcurves,database_graphs,database_jones_numfield,database_knotinfo,dot2tex,dvipng,dvips,ecm,ffmpeg,fpylll,fricas,gap_package_atlasrep,gap_package_design,gap_package_grape,gap_package_guava,gap_package_hap,gap_package_polenta,gap_package_polycyclic,gap_package_qpa,gap_package_quagroup,gfan,giac,glucose,graphviz,gurobi,imagemagick,info,internet,ipython,jmol,jupymake,jupyter_sphinx,kenzo,kissat,latex,latex_package_tkz_graph,latte_int,lrcalc_python,lrslib,lualatex,macaulay2,magma,maple,mathematica,mathics,matlab,matroid_database,mcqd,meataxe,mpmath,msolve,nauty,networkx,numpy,octave,palp,pandoc,pdf2svg,pdflatex,pdftocairo,pexpect,phitigra,pillow,plantri,polytopes_db,polytopes_db_4d,pplpy,primecountpy,ptyprocess,pycosat,pycryptosat,pynormaliz,pyparsing,python_igraph,requests,rpy2,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.braiding,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.giac,sage.libs.homfly,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.sat,sage.schemes,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,scilab,scipy,singular,sirocco,sloane_database,sphinx,symengine_py,sympy,tdlib,threejs,topcom,xelatex > > [...] > > ---------------------------------------------------------------------- > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/doc/en/developer/coding_basics.rst # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/calculus/calculus.py # 6 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/calculus/desolvers.py # 13 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/combinat/designs/ext_rep.py # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/combinat/posets/posets.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/combinat/quickref.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/combinat/species/library.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/databases/findstat.py # 3 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/databases/oeis.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/error.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/gamma.py # 4 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/generalized.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/jacobi.py # 4 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/log.py # 3 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/orthogonal_polys.py # 6 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/other.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/functions/special.py # 10 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/graphs/generators/smallgraphs.py # 8 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/graphs/strongly_regular_db.pyx # 4 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/interfaces/fricas.py # 172 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/matrix/matrix1.pyx # 4 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 src/sage/misc/latex.py > # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/misc/latex_standalone.py # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/misc/package.py # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/repl/interpreter.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/complex_double.pyx # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/finite_rings/integer_mod.pyx # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/finite_rings/integer_mod_ring.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/infinity.py # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/integer_ring.pyx # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/lazy_series.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/rational_field.py # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/real_double.pyx # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/rings/real_mpfr.pyx # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/symbolic/expression.pyx # 2 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/symbolic/expression_conversions.py # 20 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/symbolic/integration/external.py # 14 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/symbolic/integration/integral.py # 3 doctests failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/symbolic/ring.pyx # 1 doctest failed > sage -t --long --warn-long 30.0 > --random-seed=294214240793050914365381090068790310159 > src/sage/tests/cmdline.py # 2 doctests failed > ---------------------------------------------------------------------- > Total time for all tests: 3506.3 seconds > cpu time: 24639.0 seconds > cumulative wall time: 25302.0 seconds > Features detected for doctesting: > 4ti2,conway_polynomials,cvxopt,database_cremona_mini_ellcurve,database_ellcurves,database_graphs,dot2tex,dvipng,dvips,ffmpeg,fpylll,fricas,gap_package_polycyclic,gfan,giac,graphviz,imagemagick,info,internet,jupyter_sphinx,latex,latex_package_tkz_graph,lrcalc_python,lrslib,lualatex,mpmath,nauty,networkx,numpy,palp,pandoc,pdf2svg,pdflatex,pdftocairo,pexpect,pillow,polytopes_db,pplpy,primecountpy,ptyprocess,pyparsing,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.braiding,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.giac,sage.libs.homfly,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.schemes,sage.symbolic,sagemath_doc_html,scipy,singular,sphinx,sympy,threejs,xelatex > > -- > You received this message because you are subscribed to the Google Groups > "sage-release" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/sage-release/f2f47268-f709-4892-8126-843e4858c5bbn%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "sage-release" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/sage-release/CAAWYfq3YWY-FG-UssWs6R%2BWD%3DYWBET8LeFYYeNu6XV95EDUSGQ%40mail.gmail.com.
