#10004: The gap instances in a parallelised function do not always have distinct
_local_tmpfile
--------------------------+-------------------------------------------------
Reporter: SimonKing | Owner: was
Type: defect | Status: needs_review
Priority: blocker | Milestone: sage-4.6
Component: interfaces | Keywords: _local_tmpfile, parallel
Author: Simon King | Upstream: N/A
Reviewer: | Merged:
Work_issues: |
--------------------------+-------------------------------------------------
Old description:
> Using the {...@parallel}}} decorator on a function, the Gap instances in
> the different branches of the function should have distinct files for
> I/O.
>
> Indeed, under normal circumstances, they are distinct:
> {{{
> sage: @parallel
> ....: def f(n):
> ....: return gap._local_tmpfile()
> ....:
> sage: L = [t[1] for t in f(range(5))]
> sage: len(set(L)) # this is how it should be
> 5
> }}}
>
> However, it suffices to call {{{gap._local_tmpfile()}}} in the main sage
> branch in order to break diversity:
> {{{
> sage: print gap._local_tmpfile()
> /home/king/.sage//temp/gauss/16041//interface//tmp16041
> sage: L = [t[1] for t in f(range(5))]
> sage: len(set(L)) # this is how it must not be
> 1
> }}}
>
> The consequences are obvious: Having different processes use the same
> file for passing command lines must end in a disaster.
>
> I hope it is OK to mark this as blocker. After all, parallelisation and
> interfaces are both important.
New description:
Using the {...@parallel}}} decorator on a function, the Gap instances in
the different branches of the function should have distinct files for I/O.
Indeed, under normal circumstances, they are distinct:
{{{
sage: @parallel
....: def f(n):
....: return gap._local_tmpfile()
....:
sage: L = [t[1] for t in f(range(5))]
sage: len(set(L)) # this is how it should be
5
}}}
However, it suffices to call {{{gap._local_tmpfile()}}} in the main sage
branch in order to break diversity:
{{{
sage: print gap._local_tmpfile()
/home/king/.sage//temp/gauss/16041//interface//tmp16041
sage: L = [t[1] for t in f(range(5))]
sage: len(set(L)) # this is how it must not be
1
}}}
The reason for this behaviour lies in the {{{quit()}}} method of Expect
interfaces: If {{{_local_tmpfile()}}} is called then the resulting name is
stored in {{{__local_tmpfile}}}. The {...@parallel}}} decorator needs the
interfaces in a clean state, so, it calls {{{quit()}}}. However, quit does
not unset {{{__local_tmpfile}}}, and thus all interface instances in the
different branches of the parallel computation will use the same
previously stored file name.
The consequences are obvious: Having different processes use the same file
for passing command lines must end in a disaster.
I hope it is OK to mark this as blocker. After all, parallelisation and
interfaces are both important.
--
Comment(by SimonKing):
Replying to [comment:8 mpatel]:
> ... work in `Expect.quit`? Then you could simplify the `try` block in
`Expect._local_tmpfile`. It seems there's a tradeoff.
That was my question: Is there a tradeoff? So, now that you taught me how
to deal with the double underscore attribute, I'll try to change my patch
accordingly.
> Could you explain where `Expect.quit` is called (indirectly) in your
example in the description?
OK, done. {{{quit}}} is called in {...@parallel}}}, because that decorator
needs to wipe all interfaces -- but it was not rigorous enough, as it did
not unset the name of the temporary file.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/10004#comment:9>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
