#9631: Remerge #9501 after resolving NFS and/or doctest problems with @fork
----------------------------+-----------------------------------------------
Reporter: mpatel | Owner: jason
Type: defect | Status: positive_review
Priority: major | Milestone: sage-4.7.2
Component: misc | Keywords:
Work_issues: | Upstream: N/A
Reviewer: Volker Braun | Author: William Stein, Mitesh Patel
Merged: | Dependencies:
----------------------------+-----------------------------------------------
Comment(by leif):
Replying to [comment:8 was]:
> Hi Leif, I just attached trac_9631-fork_decorator.4.patch which
polished the docstrings fixing some typos. If there are any remaining
typos, please point them out explicitly.
{{{
#!diff
--- trac_9631-fork_decorator.4.patch.orig 2011-08-15
05:19:25.000000000 +0200
+++ trac_9631-fork_decorator.4.patch 2011-08-15 06:10:11.000000000
+0200
@@ -17,7 +17,7 @@
+++ b/sage/parallel/decorate.py
@@ -15,19 +15,17 @@
r"""
- Convert a to a pair (args, kwds) using some rules:
+ Convert ``a`` to a pair ``(args, kwds)`` using some rules:
- * if already of that form, leave that way.
- * if a is a tuple make (a,{})
@@ -44,7 +44,7 @@
@@ -53,9 +51,14 @@
class Parallel:
r"""
- Create parallel decorated function.
+ Create ``parallel``-decorated function.
-
"""
def __init__(self, p_iter = 'fork', ncpus=None, **kwds):
@@ -56,7 +56,7 @@
+ """
# The default p_iter is currently the reference implementation.
# This may change.
- self.p_iter = None
+ self.p_iter = None # ??? = p_iter, which defaults to 'fork', not
the sequ. ref. impl.
@@ -81,19 +84,16 @@
def __call__(self, f):
@@ -67,8 +67,8 @@
- in possibly random order. Here x is replaced by its
+ Create a function that wraps ``f`` and that when called with a
+ list of inputs returns an iterator over pairs ``(x, f(x))`` in
-+ possibly random order. Here ``x`` is replaced by its
- normalized form (args, kwds) using normalize_inputs.
++ possibly random order. Here ``x`` is replaced by its
+ normalized form ``(args, kwds)`` using ``normalize_inputs()``.
INPUT:
-
@@ -102,7 +102,7 @@
+ The parallel subprocesses will not have access to data
+ created in pexpect interfaces. This behavior with respect to
+ pexpect interfaces is very important to keep in mind when
-+ setting up certain computations. It's the one big limitation
++ setting up certain computations. It's the one big limitation
+ of this decorator.
+
INPUT:
@@ -114,24 +114,24 @@
+ - ``fork`` -- (default) use a new forked process
for each input
+ - ``multiprocessing`` -- use multiprocessing library
+ - ``reference`` -- use a fake serial reference
implementation
- - ``ncpus`` -- integer, number of cpus
- - ``timeout`` -- number of seconds until task is killed (only
supported by 'fork')
+ - ``ncpus`` -- integer, maximal number of subprocesses to use at
the same time
+ - ``timeout`` -- number of seconds until a subprocess is killed
(only supported by ``fork``; zero means not at all)
+ .. warning::
+
-+ If you use anything but 'fork' above, then a whole new
++ If you use anything but ``fork`` above, then a whole new
+ subprocess is spawned, so none of your local state (variables,
+ certain functions, etc.) is available.
+
EXAMPLES:
- We create a simple decoration for a simple function. The number
+ We create a simple decoration for a simple function. The number
@@ -148,7 +166,6 @@
sage: @parallel(2)
... def f(n): return n*n
-
- We create a decorator that uses 3 processes, and times out
+ We create a decorator that uses three subprocesses, and times out
individual processes after 10 seconds::
@@ -174,3 +191,152 @@
@@ -144,7 +144,7 @@
+
+###################################################################
+# The @fork decorator -- evaluate a function with no side effects
-+# in memory, so the only side effects are on disk.
++# in memory, so the only side effects (if any) are on disk.
+#
+# We have both a function and a class below, so that the decorator
+# can be used with or without options:
@@ -158,13 +158,13 @@
+
+class Fork:
+ """
-+ A fork decorator class.
++ A ``fork`` decorator class.
+ """
+ def __init__(self, timeout=0, verbose=False):
+ """
+ INPUT:
-+ - ``timeout`` -- (default: 0) kills subrocess after this many
-+ seconds, or if timeout=0, do not kill the subprocess.
++ - ``timeout`` -- (default: 0) kill each subprocess after it has
run this many
++ seconds (wall time), or if ``timeout`` is zero, do not kill
any subprocesses.
+ - ``verbose`` -- (default: ``False``) whether to print
+ anything about what the decorator does (e.g., killing
+ subprocesses)
@@ -182,9 +182,11 @@
+ def __call__(self, f):
+ """
+ INPUT:
+
+ - ``f`` -- a function
+
+ OUTPUT:
+
+ - A decorated function.
+
+ EXAMPLES::
@@ -206,30 +208,30 @@
+ """
+ Decorate a function so that when called it runs in a forked
+ subprocess. This means that it won't have any in-memory
-+ side-effects on the parent Sage process. The pexpect interfaces
++ side effects on the parent Sage process. The pexpect interfaces
+ are all reset.
+
+ INPUT:
+ - ``f`` -- a function
-+ - ``timeout`` -- (default: 0) if positive, kills subrocess after
-+ this many seconds
++ - ``timeout`` -- (default: 0) if positive, kill subprocess after
++ this many seconds (wall time)
+ - ``verbose`` -- (default: ``False``) whether to print anything
-+ about what the decorator does (e.g., killing subprocesses)
++ about what the decorator does (e.g., killing the subprocess)
+
+ .. warning::
+
-+ The forked subprocesses will not have access to data created
++ The forked subprocess will not have access to data created
+ in pexpect interfaces. This behavior with respect to pexpect
+ interfaces is very important to keep in mind when setting up
-+ certain computations. It's the one big limitation of this
++ certain computations. It's the one big limitation of this
+ decorator.
+
+ EXAMPLES:
+
-+ We create a function and run it with the fork decorator. Note
++ We create a function and run it with the ``fork`` decorator. Note
+ that it does not have a side effect. Despite trying to change
-+ the global variable "a" below in g, the variable a does not get
-+ changed.::
++ the global variable ``a`` below in ``g``, the variable ``a`` does
not get
++ changed::
+
+ sage: a = 5
+ sage: @fork
@@ -242,7 +244,7 @@
+ sage: a
+ 5
+
-+ We use fork to make sure that the function terminates after 1
++ We use ``fork`` to make sure that the function terminates after one
+ second, no matter what::
+
+ sage: @fork(timeout=1, verbose=True)
@@ -253,7 +255,7 @@
+ Killing subprocess ... with input ((10000000,), {'m': 5}) which
took too long
+ 'NO DATA (timed out)'
+
-+ We illustrate that pexpect interface state is not affected by
++ We illustrate that the state of the pexpect interface is not altered
by
+ forked functions (they get their own new pexpect interfaces!)::
+
+ sage: gp.eval('a = 5')
@@ -305,14 +307,14 @@
- - ``timeout`` -- (float) time in seconds until a subprocess
is automatically killed
- - ``verbose`` -- whether to print anything about what the
iterator does (e.g., killing subprocesses)
+ - ``ncpus`` -- the maximal number of simultaneous
-+ processes to spawn
-+ - ``timeout`` -- (float, default: 0) time in seconds until
++ subprocesses to spawn
++ - ``timeout`` -- (float, default: 0) wall time in seconds
until
+ a subprocess is automatically killed
+ - ``verbose`` -- (default: False) whether to print
+ anything about what the iterator does (e.g., killing
+ subprocesses)
+ - ``reset_interfaces`` -- (default: True) whether to reset
-+ all expect interfaces
++ all pexpect interfaces
EXAMPLES::
@@ -326,7 +328,7 @@
"""
@@ -206,7 +213,8 @@
- # The expect interfaces (and objects defined in them) are
+ # The pexpect interfaces (and objects defined in them) are
# not valid.
- sage.interfaces.quit.invalidate_all()
+ if self.reset_interfaces:
}}}
Explicit, but perhaps a bit inconvenient. Didn't apply it, only looked at
the patch.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/9631#comment:9>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
