#9739: Handle duplicate file basenames when testing multiple files in parallel
-----------------------------------------------+----------------------------
Reporter: mpatel | Owner: mvngu
Type: defect | Status: needs_review
Priority: blocker | Milestone: sage-4.7.2
Component: doctest | Keywords: doctest
scripts
Work_issues: | Upstream: N/A
Reviewer: Robert Bradshaw, Leif Leonhardy | Author: Mitesh Patel,
John Palmieri
Merged: | Dependencies:
-----------------------------------------------+----------------------------
Old description:
> When we test
>
> `/path/to/foo.py`,
>
> `sage-doctest` writes
>
> `SAGE_TESTDIR/.doctest_foo.py`,
>
> runs the new file through `python`, and deletes it. This can cause
> collisions when we test in parallel multiple files with the same
> basename, e.g., `__init__`, `all`, `misc`, `conf`, `constructor`,
> `morphism`, `index`, `tests`, `homset`, `element`, `twist`, `tutorial`,
> `sagetex`, `crystals`, `cartesian_product`, `template`, `ring`, etc.
>
> There's a similar problem with testing non-library files, which `sage-
> doctest` first effectively copies to `SAGE_TESTDIR`.
>
> See [http://groups.google.com/group/sage-
> devel/browse_thread/thread/0239f712a39fce4a/367bfc0d83c0e9b8#367bfc0d83c0e9b8
> sage-devel] for background.
>
> This ticket may help with some of the doctesting problems discussed on
> the Sage mailing lists. Related tickets: #9224, #9225, #9449.
>
> ----
>
> There's now a meta-ticket for all doctest-related issues / tickets,
> #11337.
>
> ----
>
> I don't know much about this, so don't feel able to work on it.
>
> But for those that do, perhaps a sub-optimal solution is a better
> temporary measure than a complete industrial strength bullet-proof
> solution.
>
> Normally I don't like "quick fixes" that don't do the job properly, but
> in this case it seems if something is not done, then the problem will
> remain open forever, as nothing has actually been done on it for 7
> months.
>
> Sometimes I feel that trac tickets get held up for unnecessary long
> periods due to a reviewer being over pedantic and wanting things outside
> the scope of the original ticket, which would be better put on another
> ticket.
>
> (Dave)
>
> -------
>
> Apply only [attachment:trac_9739.patch].
New description:
When we test
`/path/to/foo.py`,
`sage-doctest` writes
`SAGE_TESTDIR/.doctest_foo.py`,
runs the new file through `python`, and deletes it. This can cause
collisions when we test in parallel multiple files with the same basename,
e.g., `__init__`, `all`, `misc`, `conf`, `constructor`, `morphism`,
`index`, `tests`, `homset`, `element`, `twist`, `tutorial`, `sagetex`,
`crystals`, `cartesian_product`, `template`, `ring`, etc.
There's a similar problem with testing non-library files, which `sage-
doctest` first effectively copies to `SAGE_TESTDIR`.
See [http://groups.google.com/group/sage-
devel/browse_thread/thread/0239f712a39fce4a/367bfc0d83c0e9b8#367bfc0d83c0e9b8
sage-devel] for background.
This ticket may help with some of the doctesting problems discussed on the
Sage mailing lists. Related tickets: #9224, #9225, #9449.
----
There's now a meta-ticket for all doctest-related issues / tickets,
#11337.
----
I don't know much about this, so don't feel able to work on it.
But for those that do, perhaps a sub-optimal solution is a better
temporary measure than a complete industrial strength bullet-proof
solution.
Normally I don't like "quick fixes" that don't do the job properly, but in
this case it seems if something is not done, then the problem will remain
open forever, as nothing has actually been done on it for 7 months.
Sometimes I feel that trac tickets get held up for unnecessary long
periods due to a reviewer being over pedantic and wanting things outside
the scope of the original ticket, which would be better put on another
ticket.
(Dave)
-------
Apply
- [attachment:trac_9739.v2.patch] to the scripts repository
- [attachment:trac_9739-graphviz.patch] to the Sage library
--
Comment(by jhpalmieri):
Here are some new patches. First, [attachment:trac_9739.v2.patch] and a
"delta" patch [attachment:trac_9739-delta1to2.patch]: the differences from
the previous version are
- It changes how filenames are mangled: it uses "NAME_PID.py" instead of
"doctest_PID_NAME.py" or whatever what there before.
- I've decided that I agree with you: we shouldn't print the messages
about removing the doctesting directory every time. I've changed it so it
only prints them if "-verbose" is one of the command-line options. (This
is overusing the "-verbose" option, I suppose, so we could instead could
completely disable printing just by setting the variable "verbose" in
sage-ptest to be False always.) If the directory is not deleted at the
end, then a message is printed regardless of verbosity settings.
Along these lines, we have [attachment:trac_9739-graphviz.patch], a patch
for the main Sage library, which writes a temporary file to SAGE_TMP
rather than to SAGE_TESTDIR, so that the doctesting directory is indeed
empty after doctesting the Sage library.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/9739#comment:75>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
