#20729: doctest runner breaks loading modules from packages that use relative
imports
---------------------------------+----------------------------
Reporter: embray | Owner:
Type: defect | Status: new
Priority: minor | Milestone: sage-7.3
Component: doctest framework | Keywords:
Merged in: | Authors:
Reviewers: | Report Upstream: N/A
Work issues: | Branch:
Commit: | Dependencies:
Stopgaps: |
---------------------------------+----------------------------
I encountered this issue when trying to run the doctests in a new sub-
package I added to `sage_setup`, in which I used relative imports between
the submodules. Trying to run the doctests on these modules blows up
with: `ValueError: Attempted relative import in non-package`
In this particular case I was able to work around this by fiddling with
`FileDocTestSource.in_lib` to include all of `sage_setup` (previously it
only contained `sage_setup.docbuild`--I can't tell you how much hair I
pulled out trying to figure out why the relative imports in
`sage_setup.docbuild` were not a problem :)
But in general this won't do if someone wants to test their own package
using Sage's doctest runner, if it happens to use relative imports.
The cause of this error is an implementation detail of how relative
imports work in Python. They are ''only'' meant to work in modules that
belong to a package. This means the module must have either a correct
`__package__` or a `__path__` attribute as it would if it were imported
through the normal import system.
In the doctest runner, `FileDocTestSource.create_doctests` calls
`sage.repr.load.load` on the module being tested if it is not `in_lib`,
which in turn compiles and execs that module in a namespace with `__name__
= '__main__'`. This won't do when trying to exec a module that belongs to
a package if it uses relative imports.
And even if it doesn't use relative imports, but uses absolute imports, it
may not import other modules in the same package from the correct path.
This statement deserves further explanation: Say I have a package:
{{{
foo/
__init__.py
a.py
b.py
}}}
and `__init__.py` contains:
{{{
#!python
import foo.a
import foo.b
}}}
these are perfectly good absolute imports. Let's also say I have the
`foo` package under `mysrc/foo`. If I then run `./sage -t
mysrc/foo/__init__.py` it will `load()` `__init__.py` as a stand-alone
module. The imports in it ''might'' work if I happen to have `mysrc/` in
my `sys.path`, or if I have installed some version of the `foo` package
into my site-packages. The latter case is especially bad because it means
I'm testing `mysrc/foo/__init__.py` in my source tree, but it's importing
`foo.a` and `foo.b` from an installed version.
The best workaround, though annoying to do, is when `exec`'ing a module
that belongs to a package one needs to "set up" the environment it execs
in as though it were imported as part of that package, through the normal
import system. This can mean several things. Depending on what happens
in the package's `__init__.py` (and the `__init__.py` of any super-
packages if it is a sub-package) it will mostly likely be necessary to
import those packages for the module to run properly at all.
So the thing to do is walk the entire package hierarchy for the module by
looking up the directory tree for `__init__.py`s (this may fail to find
PEP-420 namespace packages, but that's probably not a problem) and import
each package in the hierarchy (making sure to add the package's path to
`sys.path`).
--
Ticket URL: <http://trac.sagemath.org/ticket/20729>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/sage-trac.
For more options, visit https://groups.google.com/d/optout.
