2011/10/7 Vitja Makarov <[email protected]>: > 2011/10/7 mark florisson <[email protected]>: >> On 6 October 2011 21:56, Vitja Makarov <[email protected]> wrote: >>> 2011/10/6 mark florisson <[email protected]>: >>>> On 6 October 2011 07:46, Stefan Behnel <[email protected]> wrote: >>>>> mark florisson, 05.10.2011 15:53: >>>>>> >>>>>> On 5 October 2011 08:16, Stefan Behnel wrote: >>>>>>> >>>>>>> mark florisson, 04.10.2011 23:19: >>>>>>>> >>>>>>>> Another issue is that Cython compile time is increasing with the >>>>>>>> addition of control flow and cython utilities. If you use fused types >>>>>>>> you're also going to combinatorially add more compile time. >>>>>>> >>>>>>> I don't see that locally - a compiled Cython is hugely fast for me. In >>>>>>> comparison, the C compiler literally takes ages to compile the result. >>>>>>> An >>>>>>> external shared library may or may not help with both - in particular, >>>>>>> it >>>>>>> is >>>>>>> not clear to me what makes the C compiler slow. If the compile time is >>>>>>> dominated by the number of inlined functions (which is not unlikely), a >>>>>>> shared library + header file will not make a difference. >>>>>> >>>>>> Have you tried with the memoryviews merged? >>>>> >>>>> No. I didn't expect the difference to be quite that large. >>>>> >>>>> >>>>>> e.g. if I have this code: >>>>>> >>>>>> from libc.stdlib cimport malloc >>>>>> cdef int[:] slice =<int[:10]> <int *> malloc(sizeof(int) * 10) >>>>>> >>>>>> [0] [14:45] ~ ➤ time cython test.pyx >>>>>> cython test.pyx 2.61s user 0.08s system 99% cpu 2.695 total >>>>>> [0] [14:45] ~ ➤ time zsh compile >>>>>> zsh compile 1.88s user 0.06s system 99% cpu 1.946 total >>>>>> >>>>>> where 'compile' is the script that invoked the same gcc command >>>>>> distutils uses. As you can see it took more than 2.5 seconds to >>>>>> compile this code (simply because the memoryview utilities get >>>>>> included). >>>>> >>>>> Ok, that hints at serious performance problems. Could you profile it to >>>>> see >>>>> where the issues are? Is it more that the code is loaded from an external >>>>> file? Or the fact that more utility code is parsed than necessary? >>>> >>>> I haven't profiled it yet (I'll do that), but I'm fairly sure it's the >>>> parsing of Cython utility files (not the loading). Maybe Tempita also >>>> adds to the overhead, I'll find out. >>>> >>> >>> Compiling this regex gives 5ms instead of 10ms on my machine >>> >>> https://github.com/cython/cython/blob/master/Cython/Compiler/Code.py#L85 >>> >>> And on your example gives 3% speedup >>> >> >> Sorry, which code gets you 10ms? Also, is this about loading + regex >> matching, or just about compiling the pattern? >> > > I've added decorator for load_utilities_from_file that prints time for > current call and total sum for this function and total gives 10ms. > > Btw that's not that much. > >
Here is small comparison on compiling urllib.py with cython: ((e8527c5...)) vitja@mchome:~/work/cython-vitek-git/zzz$ time python ../cython.py urllib.py real 0m1.699s user 0m1.650s sys 0m0.040s (master) vitja@mchome:~/work/cython-vitek-git/zzz$ time python ../cython.py urllib.py real 0m2.830s user 0m2.790s sys 0m0.030s It's about 1.5 times slower. -- vitja. _______________________________________________ cython-devel mailing list [email protected] http://mail.python.org/mailman/listinfo/cython-devel
