Am 19.07.2013 01:25, schrieb Aaron Meurer:
3. Use relative imports.
Actually, now that we have dropped Python 2.5 support,
Oh did we? I missed the final decision then.
The final decision was made a while ago to keep support until 0.7.3,
but that's been released since last week. All remaining support was
removed in https://github.com/sympy/sympy/pull/2273, except for some
the things mentioned in
https://github.com/sympy/sympy/pull/2273#issuecomment-20853941
(relative imports being one of those things).
Ah I see I have pounded on wide-open doors with the proposal to do
relative imports :-)
I thought that most C uses could be solved by moving imports inside
functions.
Those with real cycles: yes.
I'd prefer to use normal importing where an inside-function import isn't
needed (see below).
You could also use bottom imports (moving the imports to the bottom of
the file), but frankly, I'd rather have imports inside functions,
unless the function is called enough that it's a performance hit.
What problems would that solve?
Specifying the superclass of a class is going to break with that: The
superclass won't be imported at that moment.
So I don't think that everything can be moved to the end, and we'd be
having two import lists, one at the top and one at the bottom. That's a
definitive disadvantage.
... hm well, okay, what we *can* profile is whether this has an impact on a
function that does very little other than the import.
If the impact is negligible even then, that would give us more design
alternatives.
Still, I'm of a static school (not the best mindset for Python, I know), and
I prefer to be able to see the dependencies at a glance in the header.
Importing can have a performance hit. See for example
https://github.com/sympy/sympy/commit/769dccca81006fe6e0a3f71085241b3f85e6ff99.
Sounds like in-function imports do have a noticeable impact.
This benchmark shows a potential impact of at least 600% slowdown via a
local import, possibly more if the optimized function actually reached
the import statement in a substantial fraction of cases, or if the
function is doing a lot of stuff.
Sheesh, that doesn't look good. We'd really have to determine for each
function how often it is really called in practice, which is going to
slow the whole activity to a crawl (and also bind a lot of attention in
the future).
With that background, I'd want to avoid in-function imports as far as
possible.
Maybe the classes should be initialized in an on-demand fashion.
Possibly like this:
- Have a getter for each class attribute (or other attribute) where
initialization is expensive or risky.
- The getter is supposed to run only once, namely the first time the
attribute is accessed.
- The getter is specified using a lambda. The getter runs the lambda and
assigns whatever result the lambda emits to the raw variable.
- The getter deletes itself, exposing the raw variable to any future access.
- As the final polish, the getter returns the lambda's result that got
assigned to the member variable.
I'm not sure what the best way to write such a getter would be; it could
be a raw function as in
_a = lazyInit('_a', lambda: ...)
or maybe a decorator could be made to work:
@lazyInit
_a = lambda: ...
or
@lazyInit
def _a():
...
The main challenge is whether the decorator knows the name of the
attribute/function it is being applied to; I dimly remember that that's
the case without further ado in decorators, so the last approach looks
best to me.
... oh. stackoverflow for the win.
See
http://stackoverflow.com/questions/3237678/how-to-create-decorator-for-lazy-initialization-of-a-property
, the code is already available in precanned form. @read_only_lazyprop
seems to do all that we need.
You're right that it only matters for functions that are called a lot,
though (but _sympify is such a function).
Oh yes, that thing is pervasive.
I guess the module-level imports that replace C. should be done on the
core functions first, so that any cycles that we might still have to
solve with in-function imports would happen elsewhere.
I hope to get rid of all cycles via lazy initialization though.
No, it doesn't matter if it's slow when the class is used. The class
attribute is rarely called. It's only called if those particular
functions are used, and in that case, it is nothing compared to the
rest of the algorithm.
Unless that function is being called in a tight loop.
Or doing a recursion with many levels and a high multiplicity.
Or if user code call into that function for a gazillion times, possibly
indirectly so the user doesn't even notice.
I find the prospect of in-function imports quite scary, and would prefer
to keep it at arm's length as a tool of last resort.
> On the other hand, you can really feel even the
smallest things at import time, because that multiplies up over
various libraries to make loading things like pylab slower
I noticed that all import statements are run twice.
Well, maybe not all, just those that I added logging code to; it was
just half a dozen cases where I did that. Still, the outputs would
always be twice when run from bin/test.
I haven't investigated this effect at all, but if this affects not just
testing but also normal use, that could be a contributing factor to
start-up times.
I'll probably take a look as soon as I get around to doing more Python
work. Might take a while though, so feel free to investigate yourself at
your leisure :-)
> (and even
without considering performance, running things like sqrt(pi) at
import time make debugging the core a huge pain, because if sqrt(pi)
raises an exception, then you can't even import sympy.
Oh. How would calculating a constant like sqrt(pi) ever raise an exception?
> This actually,
not performance, was the true motivation behind that commit).
I can believe that :-)
Seems like that simple C. replacement mission is growing as we get along.
Things on my list right now are:
- Find out why that import seems to run twice
- Agreeing on and implementing a @lazyProperty decorator
- Decorating all properties (and possibly other stuff) with
@lazyProperty (includes performance testing to see what parts of Sympy
initialization really take so long)
- Replace all imports with relative imports
- Possibly replace all imports with imports from the declaring module
- Get rid of all those C. references... the smallest part of it all
I'm looking forward to do that all, so no trouble here - just a heads-up
that it's going to take longer than initially expected.
--
You received this message because you are subscribed to the Google Groups
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sympy.
For more options, visit https://groups.google.com/groups/opt_out.
