Re: [sage-devel] Unreasonably slow piecewise function

[Nils Bruin]

On Wednesday 1 May 2024 at 16:45:36 UTC-7 Kwankyu Lee wrote:


I wonder if they, maintainers of maxima, would regard this as a bug... 


I'm pretty sure the piecewise functions are NOT borrowed from maxima. It 
probably gets called because there are some inequalities concerning the 
symbolic ring. So the performance observed here is not reduced to something 
that could be reported as a bug to maxima. I'd expect that performance can 
be significantly improved by optimizing the sage code.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/2331637e-f9f3-4ee7-8697-2e55b1358ea3n%40googlegroups.com.

Re: [sage-devel] Unreasonably slow piecewise function

[Kwankyu Lee]

On Thursday, May 2, 2024 at 12:37:44 AM UTC+9 Nils Bruin wrote:

Working on *why* it might be so slow a bit:

%prun for i in range(100r): f(0.1)
 798103 function calls (791903 primitive calls) in 1.327 seconds


Impressive. Thanks.
 

so, most stuff is happening in maxima and in _subs_, and somehow random 
elements of an intervalfield are needed ...
I don't think the design was primarily focused on numerical evaluation 
speed but rather on having symbolic piecewise functions.


I wonder if they, maintainers of maxima, would regard this as a bug... 

 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/4cc8f012-c573-4e4e-917d-afb231fbdc8dn%40googlegroups.com.

[sage-devel] (Re-)building an inclusive SageMath community. III: Our relations to the projects that Sage depends on

[Matthias Koeppe]

Previous posts in the series: 
https://groups.google.com/g/sage-devel/c/OeN8o14s6Jc/m/ChnpijP3AgAJ, 
https://groups.google.com/g/sage-devel/c/xBzaINHWwUQ/m/Tq17YRqOAAAJ

As we all know, SageMath makes use of hundreds of *"upstream" projects: 
third-party, separately maintained packages* written either in 
Python/Cython or in other languages (C, C++, Common Lisp, Fortran, and the 
domain-specific languages of systems such as GAP, Singular, Maxima, ...).

The role of SageMath, although it does have a role as a software 
distribution, is in a clear contrast to that of general software 
distributions such as Ubuntu or conda-forge: It's probably rare for users 
to say "*I computed this Gröbner basis using Ubuntu Linux*" or "a strong 
generating set for this matrix group was computed using conda-forge". But 
many users say such things all the time about using SageMath.

Of course this is because of the added value of SageMath over the 
collection of its dependencies:
1. Abstraction and unification of the interface to multiple upstream 
dependencies, and integration.
2. The algorithms, structures, and applications implemented in the Sage 
library itself.

I posit that there is an *intrinsic conflict between 
abstraction/unification/integration and attribution for the upstream 
projects: *Regardless of intent and purpose, a real side effect 
of abstraction/unification/integration is that the use of the upstream 
project is obscured to some degree.

It is natural if individuals who contribute to the upstream projects (or 
have contributed to them in the past) are concerned or unhappy about such 
effects. And it is understandable if they perceive that the SageMath 
project is using their work, consuming attention/visibility/attribution, 
but not "giving back" sufficiently. Contributors are entitled to taking 
pride in their workpersonship, in the success of the project that they have 
been contributing to, the brand that they have created, etc. Even if some 
of us may be wary of possible toxic gradations such as tribalism, it is 
clear that *attention and attribution are important, positive, and 
legitimate motivating factors for open source contributors* in general, and 
moreover attribution via academic citations may indirectly translate into 
individuals' careers and success in obtaining funding. 

The 2018 sage-devel thread "Suggestion for the SageMath website" 
(https://groups.google.com/g/sage-devel/c/H8FcZD90O0Y/m/VRIRzj1sBAAJ) 
focused on *getting upstream projects credited on our website. *Although 
the suggestions there, to randomly rotate the names of external 
dependencies that are listed on the main page so they all get equal 
exposure, or scrolling lists, were not implemented, we have come a long way 
since then regarding better attribution for upstream projects.
William's message in that thread, 
https://groups.google.com/g/sage-devel/c/H8FcZD90O0Y/m/4ke5ekyVAgAJ, 
suggested that "*linking to dependencies should be done much more, but in a 
way that provides clear value to users*:
- being able to better know what is in Sage,
- being able to read the original upstreams docs and source code more 
easily,
- knowing which upstreams devs to contact for *support*, to ask 
for features, to contribute work, and to thank,
- being able to properly acknowledge what they are using."

Regarding William's first point, *being able to better know what is in 
Sage:* The main page of http://sagemath.org now links to our reference 
manual with a list of dependencies that is always up to date because it is 
automatically generated from the source code. And in the most current 
version, this long list is broken into categories such as "Mathematics" for 
better navigability: 
https://deploy-livedoc--sagemath.netlify.app/html/en/reference/spkg/
For each dependency, we have a page with various information, including a 
short description, installation instructions and a link to the upstream 
project. 

Regarding the second point, Simon King's suggestion in the thread, to 
"[...] on that list have a link to the doc for each package that provides 
docs" 
(https://groups.google.com/g/sage-devel/c/H8FcZD90O0Y/m/YEp6aO8cAQAJ), has 
not been implemented and would still be a valuable improvement. But we have 
just made progress in a similar way by *facilitating Sphinx hyperlinks to 
specific pages of package documentation*, see 
https://github.com/sagemath/sage/wiki/Sage-10.4-Release-Tour#linking-to-external-package-documentation
Also TB's suggestion to use "the Sphinx extensions viewcode and linkcode 
[...] add links next to functions and classes with the corresponding source 
code" (https://groups.google.com/g/sage-devel/c/H8FcZD90O0Y/m/RisE0AajAgAJ) 
was implemented in this development cycle, 
see 
https://github.com/sagemath/sage/wiki/Sage-10.4-Release-Tour#links-to-source-code

*Giving attribution to the projects that supported a particular computation 
is a hard problem that cannot be fully automated.* I don't know how widely 

Re: [sage-devel] Re: Urgent and important: Please vote on disputed PR #36964 (next step of the modularization project)

[Matthias Koeppe]

Hi Sage developers, 

Since I posted my request to urgently vote on the modularization PRs, the 
big revert (https://github.com/sagemath/sage/pull/37796) was merged into 
Sage 10.4.beta4.
The modularization PRs have now been re-created (thanks, Julian, for your 
help with this). 

*I'm now asking you to vote on the new PRs; it's important – participation 
matters!*
- https://github.com/sagemath/sage/pull/37900 (*Restructure sage.*.all for 
modularization, replace relative by absolute imports*). (As I explained, the 
PR is "mostly harmless": There are no user-visible changes; it's just a 
bunch of imports that are moved around. It includes no policy change of any 
kind; it only executes a design that was previously reviewed and carefully 
documented in separate PRs. Nothing permanent or irreversible is done here. 
The new files provide the top-level namespaces needed for doctesting 
modularized installations of Sage.)
- https://github.com/sagemath/sage/pull/37901 (*Add # sage_setup: 
distribution directives to all files, remove remaining # coding: utf-8*).

I'm responding to a few messages that were posted here in this sage-devel 
thread in the meantime and I did not have a chance to respond to earlier.

On Thursday, April 25, 2024 at 6:28:48 AM UTC-7 *Dima Pasechnik* wrote:

> On Wednesday, April 24, 2024 at 10:14:09 PM UTC-5 Matthias Koeppe wrote: 
> Yes, native Windows would clearly be a very important target. 

Essential components of sagelib such as GAP, Singular, don't run on native 
Windows (on Cygwin, yes [...]) and I don't think anyone is keen on doing 
the hard work to port it. This puts native Windows support into the area of 
wishful thinking.


Yes, porting software to new platforms is hard (thanks all for the detailed 
and entertaining discussion regarding GAP). 
But Dima's message is ignoring the very point of why we are talking about 
porting to new platforms in this thread: 
*The modularization project enables us to port those parts of Sage to new 
platforms for which there is interest to port*, without being held back by 
those parts and libraries for which porting is too hard or in which there 
is no interest.


On Thursday, April 25, 2024 at 5:00:33 PM UTC-7 *TB* wrote:

On 25/04/2024 15:28, Nathan Dunfield wrote:

In another direction: I have started a port of Sage to pyodide, the 
distribution of Python for WebAssembly (WASM), which makes Python runnable 
directly in the browser. I can already run and test the modularized 
distributions **sagemath-objects**, **sagemath-categories** there.


It would be amazing if a decent portion of Sage could be run in the browser 
this way, e.g. to have the occasional HW assignment that needs Sage without 
the overhead of using something like CoCalc. 

Although SageMathCell  does not run 
locally, it does run in the browser. There are examples of Sage exercises 
in this book  and more on the about 
page  of SageMathCell. 
Having a completely offline version of parts of Sage that can run in the 
browser with WASM will be wonderful indeed.

Yes, *pyodide will enable running portions of Sage completely offline, i.e. 
in serverless mode.* There is currently a lot of momentum in the scientific 
computing community for developing such deployments, see for example the 
post https://blog.pyodide.org/posts/marimo/ on the port of the (very 
impressive!) *reactive Python notebook* *marimo* to pyodide.


On Thursday, April 25, 2024 at 5:45:33 AM UTC-7 *Nathan Dunfield* wrote:

Another example is large-scale pure math computation on clusters.  Because 
of Sage's size and the nature of distributive file systems, the time to 
startup Sage can be 30 seconds or more, which complicates things if you 
want to do 100,000 calculations that are only 10 seconds each.  I was 
recently at a workshop on computational topology, and several researchers 
there regarded using Sage in this context as a non-starter, in one case 
they were completely changing their approach to avoid using Sage. 


Indeed, starting up sage (when installed from a shared volume) not only 
takes long, but it also incurs a huge load on the cluster's network from 
network file system operations for loading all the Python modules. This can 
degrade other jobs' performance. (My student's jobs using Sage on our HPC 
cluster were occasionally flagged by the admins for this.)
 

On Thursday, April 25, 2024 at 12:17:31 AM UTC-7 *Martin R* wrote:

On Thursday 25 April 2024 at 05:13:37 UTC+2 Matthias Koeppe wrote:

On Wednesday, April 24, 2024 at 1:07:44 AM UTC-7 Martin R wrote:

You mentioned several times, that discoverability is an important aspect.  
Do you have any evidence to support that?


I mentioned "discoverability" in the context of how I have *named* the 
distributions.


Sorry that my question was not clear enough.  Do you have evidence, that 
this naming enhances 

Re: [sage-devel] Unreasonably slow piecewise function

[Nils Bruin]

Working on *why* it might be so slow a bit:

%prun for i in range(100r): f(0.1)
 798103 function calls (791903 primitive calls) in 1.327 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 80000.5560.0000.6280.000 maxima_lib.py:438(_eval_line)
  1000.2990.0031.3180.013 piecewise.py:188(_subs_)
160000.0590.0000.0670.000 {method 'random_element' of 
'sage.rings.real_mpfi.RealIntervalField_class' objects}
 17000.0540.0000.0600.000 
maxima_lib.py:298(max_to_string)
   2322000.0320.0000.0480.000 calculus.py:2204(_is_function)
  9000.0280.0000.0500.000 calculus.py:2319()
 27000.0240.0000.0240.000 {built-in method 
sage.libs.ecl.ecl_eval}
  9000.0210.0000.0470.000 calculus.py:2317()
   2737000.0180.0000.0180.000 {built-in method 
builtins.isinstance}
  9000.0150.0000.0330.000 {method 'parse_sequence' of 
'sage.misc.parser.Parser' objects}

so, most stuff is happening in maxima and in _subs_, and somehow random 
elements of an intervalfield are needed ...
I don't think the design was primarily focused on numerical evaluation 
speed but rather on having symbolic piecewise functions.
On Wednesday 1 May 2024 at 06:38:36 UTC-7 David Joyner wrote:

> For another data point, on an ubuntu laptop:
>
> sage: time [f(0.1*i) for i in range(1,10)]
> CPU times: user 136 ms, sys: 0 ns, total: 136 ms
> Wall time: 99.1 ms
> [1, 1, 1, 1, 1, 1, 1, 1, 1]
> sage: time f(0.1)
> CPU times: user 13.6 ms, sys: 0 ns, total: 13.6 ms
> Wall time: 13.5 ms
> 1
> sage: version()
> 'SageMath version 10.3.rc1, Release Date: 2024-02-29'
>
> On Wed, May 1, 2024 at 5:21 AM Kwankyu Lee  wrote:
> >
> > Hi,
> >
> > I get
> >
> > sage: f = piecewise([((0,1),1)])
> > sage: time f(0.1)
> > CPU times: user 135 ms, sys: 4.23 ms, total: 140 ms
> > Wall time: 146 ms
> > 1
> > sage: time f(0.2)
> > CPU times: user 133 ms, sys: 3.56 ms, total: 136 ms
> > Wall time: 137 ms
> > 1
> >
> > This is painfully slow. Is this normal?
> >
> > --
> > You received this message because you are subscribed to the Google 
> Groups "sage-devel" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to sage-devel+...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sage-devel/af2161ba-1ee9-4636-9cea-98ea6031796bn%40googlegroups.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/e01afcef-943f-4f58-8978-882f807f763en%40googlegroups.com.

Re: [sage-devel] Unreasonably slow piecewise function

[David Joyner]

For another data point, on an ubuntu laptop:

sage: time [f(0.1*i) for i in range(1,10)]
CPU times: user 136 ms, sys: 0 ns, total: 136 ms
Wall time: 99.1 ms
[1, 1, 1, 1, 1, 1, 1, 1, 1]
sage: time f(0.1)
CPU times: user 13.6 ms, sys: 0 ns, total: 13.6 ms
Wall time: 13.5 ms
1
sage: version()
'SageMath version 10.3.rc1, Release Date: 2024-02-29'

On Wed, May 1, 2024 at 5:21 AM Kwankyu Lee  wrote:
>
> Hi,
>
> I get
>
> sage: f = piecewise([((0,1),1)])
> sage: time f(0.1)
> CPU times: user 135 ms, sys: 4.23 ms, total: 140 ms
> Wall time: 146 ms
> 1
> sage: time f(0.2)
> CPU times: user 133 ms, sys: 3.56 ms, total: 136 ms
> Wall time: 137 ms
> 1
>
> This is painfully slow. Is this normal?
>
> --
> You received this message because you are subscribed to the Google Groups 
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sage-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sage-devel/af2161ba-1ee9-4636-9cea-98ea6031796bn%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/CAEQuuAWBAs2ouVu0iRhj7_FjROEfMLuaj7OQLeu-qmQC%2BOkgRw%40mail.gmail.com.

[sage-devel] Re: testing optional packages

[Dima Pasechnik]

As I already told Marc, one should use "pytest" rather than "sage -t" - for 
obvious reasons: Sage has very own testing system,
and one should not expect it to be able to test non-Sage code. 

On Monday, April 29, 2024 at 1:08:01 PM UTC+1 Marc Culler wrote:

> I don't know what the expectations are for testing optional packages, but 
> in 10.4.beta4 when I run:
>
> ./sage -t venv/lib/python3.11/site-packages/symengine
>
> I get the exception:
>
> NameError: name 'test_sage_conversions' is not defined
>
> - Marc
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/2b7b37d7-df3c-4288-91f6-0f055c5d3b25n%40googlegroups.com.

[sage-devel] Unreasonably slow piecewise function

[Kwankyu Lee]

Hi,

I get

sage: f = piecewise([((0,1),1)])
sage: time f(0.1)
CPU times: user 135 ms, sys: 4.23 ms, total: 140 ms
Wall time: 146 ms
1
sage: time f(0.2)
CPU times: user 133 ms, sys: 3.56 ms, total: 136 ms
Wall time: 137 ms
1

This is painfully slow. Is this normal?

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/af2161ba-1ee9-4636-9cea-98ea6031796bn%40googlegroups.com.