On Thu, Jan 16, 2014 at 12:12 PM, Michel Pelletier < [email protected]> wrote:
> Hmm, I don't have any strong feelings either way. All python imports > seems slow to me on almost all useful libraries, I use a lot of numpy, > scipy, pandas, and scikit-learn for my job and I'm used to multi-second > startup times. Even ipython takes a second or two on my new macbook. For > long running jobs and apps it doesn't seem to be an issue. > > If you're running very short scripts that need to fire and exit quickly, > then I can see that being a problem, but 0.5 isn't a huge win to me. But > like I said, I don't have strong feelings, and I think you could count > pyczmq's users on one hand at this point. Anyone else have an opinion? > > Also, this might be some slowness related to using dlopen instead of > verify in cffi. verify is *much* slower the first time due to compilation, > but then is much faster for subsequent imports. Maybe investigating that > approach first would be fruitful. > If you use cffi.verify, you can also do the compilation at build time, as recommended by cffi<http://cffi.readthedocs.org/en/latest/index.html?highlight=get_extension#distributing-modules-using-cffi>. PyZMQ does this<https://github.com/zeromq/pyzmq/blob/f58bb80b3b7583fb45824d0a1037128f7f29d44a/setup.py#L1003> for its CFFI backend on PyPy, and it seems to work reasonably well. -MinRK > > -Michel > > > On Thu, Jan 16, 2014 at 9:53 AM, Greg Ward <[email protected]> wrote: > >> Speaking of pyczmq, it is *very* slow to import: >> >> $ time python -c 'import pyczmq' >> python -c 'import pyczmq' 1.11s user 0.05s system 100% cpu 1.152 total >> >> Turns out this is because pyczmq/__init__.py imports every submodule, >> as well as some identifiers in a couple of submodules. I'm pretty sure >> this is considered an anti-pattern. (I dimly recall seeing Guido van >> Rossum disparage the practice on python-dev many years ago.) >> >> I've come to agree with him, if only on performance grounds. E.g. >> here's the best case: >> >> $ echo -n > pyczmq/__init__.py >> $ time python -c 'import pyczmq' >> python -c 'import pyczmq' 0.03s user 0.01s system 99% cpu 0.039 total >> >> A more realistic test: >> >> $ time python -c 'from pyczmq import zmq, zctx, zsocket' >> python -c 'from pyczmq import zmq, zctx, zsocket' 0.46s user 0.05s >> system 100% cpu 0.508 total >> >> Or, for all the submodules needed to implement Pieter's "ironhouse" >> example in Python: >> >> $ time python -c 'from pyczmq import zmq, zctx, zsocket, zcert, zauth, >> zstr' >> python -c 'from pyczmq import zmq, zctx, zsocket, zcert, zauth, zstr' >> 0.61s user 0.03s system 100% cpu 0.640 total >> >> That's not great, but 0.6 sec is still better than 1.1 sec. >> >> Is pyczmq young enough that we can consider breaking backwards >> compatibility like this? Anyone currently doing >> >> import pyczmq >> >> ctx = pyczmq.zctx.new() >> sock = pyczmq.zsocket.new(...) >> >> would have to change their imports to: >> >> import pyczmq.zctx >> import pyczmq.zsocket >> >> If that's OK, I'll work up a patch and send it. >> >> Greg >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
