#13731: Fix libsingular memory management
--------------------------------------------------------------+-------------
Reporter: nbruin | Owner:
rlm
Type: defect | Status:
needs_review
Priority: major | Milestone:
sage-5.6
Component: memleak | Resolution:
Keywords: | Work issues:
Report Upstream: Fixed upstream, in a later stable release. | Reviewers:
Authors: Nils Bruin, Simon King | Merged in:
Dependencies: | Stopgaps:
--------------------------------------------------------------+-------------
Changes (by SimonKing):
* status: new => needs_review
Old description:
> We have several indications that interaction with libsingular's memory
> management is tricky. Debugging is hard, because libsingular uses omalloc
> for its memory management and that hides all allocation/deallocation from
> conventional memory checking tools. Singular's allocation library is
> relatively pluggable, though, and (at a speed penalty) one can mostly
> switch it to using the system malloc for everything.
>
> For that purpose, one can use one of the following:
>
> * [http://sage.math.washington.edu/home/nbruin/singular-3-1-5.malloc-
> linux.spkg] (for linux). It replaces omalloc by the system malloc and
> tries to keep the changes small. It does build a libsingular library, but
> the Singular executable does not work.
> *
> [http://sage.math.washington.edu/home/nbruin/singular-3-1-5.malloc.spkg]
> (for OSX). Same as the previous item.
> *
> [http://sage.math.washington.edu/home/SimonKing/SAGE/spkgs/singular-3-1-5.p2.spkg]
> (both linux and OSX).
> * It contains [attachment:singular_15435.patch] and [attachment:
> singular_part_of_changeset_baadc0f7.patch], which backport upstream
> fixes.
> * It usually builds with omalloc. If `export SINGULAR_XALLOC=yes`, then
> it builds with xalloc, which is a compatibility layer for omalloc on top
> of malloc. See the discussion on
> [https://groups.google.com/forum/?hl=en&fromgroups=#!topic/libsingular-
> devel/AZI-6eZAgus libsingular-devel]
> * It builds both a working libsingular library and a Singular
> executable.
>
> With the [http://sage.math.washington.edu/home/nbruin/singular-3-1-5
> .malloc-linux.spkg linux spkg], one obtains:
> {{{
> $ export MALLOC_CHECK_ 3
> $ sage
> ...
> sage: A.<x,y> = FreeAlgebra(QQ, 2)
> sage: P.<x,y> = A.g_algebra({y*x:-x*y})
> sage: x*y
> *** glibc detected *** /usr/local/sage/5.5b1/local/bin/python: free():
> }}}
> gdb backtrace:
> {{{
> #0 0x00000031cfe36285 in raise () from /lib64/libc.so.6
> #1 0x00000031cfe37b9b in abort () from /lib64/libc.so.6
> #2 0x00000031cfe7774e in __libc_message () from /lib64/libc.so.6
> #3 0x00000031cfe7da76 in malloc_printerr () from /lib64/libc.so.6
> #4 0x00007fffd82d5c34 in gnc_mm_Mult_nn (F0=<optimized out>,
> G0=<optimized out>, r=0x2faa720) at gring.cc:507
> #5 0x00007fffd82d6abc in gnc_p_Mult_mm_Common (p=0x326e8c0,
> m=<optimized out>, side=0, r=0x2faa720) at gring.cc:424
> #6 0x00007fffd76c7860 in nc_mm_Mult_pp (p=0x342aaf0, r=0x2faa720,
> m=0x2fb6530) at /usr/local/sage/5.5b1/local/include/singular/gring.h:
> 171
> #7 pp_Mult_qq (r=0x2faa720, q=0x342aaf0, p=0x2fb6530) at /usr/local/
> sage/5.5b1/local/include/singular/pInline2.h:667
> #8 __pyx_f_4sage_4libs_8singular_10polynomial_singular_polynomial_mul
> (__pyx_v_ret=0x7fffffffb3c8, __pyx_v_p=0x2fb6530, __pyx_v_q=0x342aaf0,
> __pyx_v_r=0x2faa720)
> at sage/libs/singular/polynomial.cpp:3895
> #9 0x00007fffd7b096f8 in
> __pyx_f_4sage_5rings_10polynomial_6plural_19NCPolynomial_plural__mul_
> (__pyx_v_left=0x7fffbe6bd758, __pyx_v_right=0x7fffbe6bd878,
> __pyx_skip_dispatch=<optimized out>)
> at sage/rings/polynomial/plural.cpp:12060
> }}}
> running it in valgrind gets you complaints about the same locations.
> Meanwhile two of the underlying problems where found and fixed upstream.
> [http://sage.math.washington.edu/home/SimonKing/SAGE/spkgs/singular-3-1-5.p2.spkg]
> contains backports of these fixes.
New description:
We created a new patchlevel for the singular-3-1-5 spkg. It contains
backports of some upstream fixes of several memory corruptions.
If one does `export SINGULAR_XALLOC=yes` before installing the spkg,
Singular's usual memory manager "omalloc" will be replaced by "xalloc".
This is a compatibility layer on top of malloc. The resulting Singular
executable and library will be slower, but allows the use of some debug
tools.
Here is the background.
We have several indications that interaction with libsingular's memory
management is tricky. Debugging is hard, because libsingular uses omalloc
for its memory management and that hides all allocation/deallocation from
conventional memory checking tools. Singular's allocation library is
relatively pluggable, though, and (at a speed penalty) one can mostly
switch it to using the system malloc for everything.
Preliminary packages for linux and for osx using malloc are found here:
* [http://sage.math.washington.edu/home/nbruin/singular-3-1-5.malloc-
linux.spkg] (for linux). It replaces omalloc by the system malloc and
tries to keep the changes small. It does build a libsingular library, but
the Singular executable does not work.
* [http://sage.math.washington.edu/home/nbruin/singular-3-1-5.malloc.spkg]
(for OSX). Same as the previous item.
With the help from
[https://groups.google.com/forum/?hl=en&fromgroups=#!topic/libsingular-
devel/AZI-6eZAgus libsingular-devel], we managed to use xalloc instead of
malloc. We now obtain Singular executable and library, and can still use
debugging tools.
'''__Install__'''
*
[http://sage.math.washington.edu/home/SimonKing/SAGE/spkgs/singular-3-1-5.p2.spkg]
(both linux and OSX).
The spkg contains [attachment:singular_15435.patch] and [attachment:
singular_part_of_changeset_baadc0f7.patch], which backport upstream fixes.
It usually builds with omalloc. If `export SINGULAR_XALLOC=yes`, then it
builds with xalloc, which is a compatibility layer for omalloc on top of
malloc.
'''__Bugs fixed__'''
With the preliminary
[http://sage.math.washington.edu/home/nbruin/singular-3-1-5.malloc-
linux.spkg linux spkg], one obtains:
{{{
$ export MALLOC_CHECK_ 3
$ sage
...
sage: A.<x,y> = FreeAlgebra(QQ, 2)
sage: P.<x,y> = A.g_algebra({y*x:-x*y})
sage: x*y
*** glibc detected *** /usr/local/sage/5.5b1/local/bin/python: free():
}}}
gdb backtrace:
{{{
#0 0x00000031cfe36285 in raise () from /lib64/libc.so.6
#1 0x00000031cfe37b9b in abort () from /lib64/libc.so.6
#2 0x00000031cfe7774e in __libc_message () from /lib64/libc.so.6
#3 0x00000031cfe7da76 in malloc_printerr () from /lib64/libc.so.6
#4 0x00007fffd82d5c34 in gnc_mm_Mult_nn (F0=<optimized out>,
G0=<optimized out>, r=0x2faa720) at gring.cc:507
#5 0x00007fffd82d6abc in gnc_p_Mult_mm_Common (p=0x326e8c0,
m=<optimized out>, side=0, r=0x2faa720) at gring.cc:424
#6 0x00007fffd76c7860 in nc_mm_Mult_pp (p=0x342aaf0, r=0x2faa720,
m=0x2fb6530) at /usr/local/sage/5.5b1/local/include/singular/gring.h:
171
#7 pp_Mult_qq (r=0x2faa720, q=0x342aaf0, p=0x2fb6530) at /usr/local/
sage/5.5b1/local/include/singular/pInline2.h:667
#8 __pyx_f_4sage_4libs_8singular_10polynomial_singular_polynomial_mul
(__pyx_v_ret=0x7fffffffb3c8, __pyx_v_p=0x2fb6530, __pyx_v_q=0x342aaf0,
__pyx_v_r=0x2faa720)
at sage/libs/singular/polynomial.cpp:3895
#9 0x00007fffd7b096f8 in
__pyx_f_4sage_5rings_10polynomial_6plural_19NCPolynomial_plural__mul_
(__pyx_v_left=0x7fffbe6bd758, __pyx_v_right=0x7fffbe6bd878,
__pyx_skip_dispatch=<optimized out>)
at sage/rings/polynomial/plural.cpp:12060
}}}
running it in valgrind gets you complaints about the same locations.
On OSX, `MALLOC_CHECK_` is not available. But gmalloc does detect the
crash.
The new spkg fixes that problem
'''__Testing__'''
Meanwhile two of the underlying problems where found and fixed upstream.
[http://sage.math.washington.edu/home/SimonKing/SAGE/spkgs/singular-3-1-5.p2.spkg]
contains backports of these fixes. With the exeption of one unrelated
problem in sage/graphs/, the whole Sage test suite passes on Linux with
`export MALLOC_CHECK_=yes`.
--
Comment:
I have produced a new patchlevel for the singular spkg. The optional patch
is now smaller, since I simply erase the content of the omalloc/ folder,
so that the removed files are not part of the patch.
Needs review!
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/13731#comment:117>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
