Hello Sébastien,
I worked a bit on the pypy implementation of ctypes today, and now it is
possible to pass ctypes arrays as parameters without leaving the fast path.
The benchmark is still slower on PyPy, but not as much as before. Here are the
results on my machine for running arp.py:
cpython 2.7.2: 224 ms
pypy-c 1.8: 1059 ms
pypy-c trunk: 696 ms
so, we are about 50% faster than before, although still slower than CPython.
However, 224 ms is definitely too short for the JIT to warm up, so it would be
nice if you could rerun the benchmark with a larger capture file.
It's not necessary to retranslate pypy: you can just download pypy 1.8 and put
the binary inside an updated copy of the mercurial repo (the relevant checkin
is 6566e81c76a8).
thank you!
ciao,
Anto
On 02/13/2012 01:33 PM, Sébastien Volle wrote:
Hi all,
My team is working on a project of fast packet sniffers and I'm comparing
performance between different languages.
So, we came up with a simple ARP sniffer that I ported to Python using ctypes.
During my investigations, I turned out that using ctypes, PyPy 1.8 is
4x slower than CPython 2.6.5.
After looking at the PyPy buglist, it's seems there are couple open issues
about ctypes so I figured I would ask you guys first before filing a new bug.
I'm pretty new to ctypes and pypy so I'm not sure I understand what's going.
My program seems to spend a lot of time in ctypes/function.py:_convert_args
though, has the following profile trace demonstrates:
$ pypy -m cProfile -s time arp.py
Packet buffer now at 0x9CCECB2
Capture started
elapsed time : 3983.84ms
Total packets : 35571
packets/s : 8928.81
7839198 function calls (7838340 primitive calls) in 4.105 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
69876 0.546 0.000 1.584 0.000 function.py:480(_convert_args)
214696 0.256 0.000 0.429 0.000 structure.py:236(_subarray)
1052195 0.206 0.000 0.206 0.000 {isinstance}
632437 0.192 0.000 0.192 0.000 {method 'append' of 'list'
objects}
175326 0.187 0.000 0.407 0.000 function.py:350(_call_funcptr)
1 0.173 0.173 4.105 4.105 arp.py:1(<module>)
209628 0.158 0.000 0.587 0.000 primitive.py:272(from_param)
214696 0.149 0.000 0.963 0.000 structure.py:90(__get__)
71144 0.143 0.000 0.198 0.000 structure.py:216(__new__)
106713 0.130 0.000 0.208 0.000 array.py:70(_CData_output)
105450 0.124 0.000 2.281 0.000 function.py:689(__call__)
69876 0.123 0.000 1.943 0.000 function.py:278(__call__)
321412 0.102 0.000 0.102 0.000 {method 'fromaddress' of
'Array' objects}
209628 0.088 0.000 0.811 0.000 function.py:437(_conv_param)
179125 0.083 0.000 0.083 0.000 {method 'fieldaddress' of
'StructureInstance' objects}
69883 0.080 0.000 0.122 0.000 primitive.py:308(__init__)
71142 0.076 0.000 0.320 0.000 structure.py:174(from_address)
105450 0.075 0.000 0.145 0.000 function.py:593(_build_result)
139755 0.072 0.000 0.120 0.000
primitive.py:64(generic_xxx_p_from_param)
107983 0.070 0.000 0.125 0.000 basics.py:60(_CData_output)
209828 0.062 0.000 0.062 0.000 {method 'get' of 'dict' objects}
107986 0.055 0.000 0.055 0.000 {method '__new__' of
'_ctypes.primitive.SimpleType' objects}
71142 0.052 0.000 0.372 0.000 pointer.py:77(getcontents)
35578 0.052 0.000 0.125 0.000 pointer.py:62(__init__)
35576 0.050 0.000 0.062 0.000 pointer.py:83(setcontents)
106713 0.047 0.000 0.047 0.000 {method '__new__' of
'_ctypes.array.ArrayMeta' objects}
139750 0.043 0.000 0.181 0.000
primitive.py:84(from_param_void_p)
209625 0.043 0.000 0.691 0.000 basics.py:50(get_ffi_param)
283592/283435 0.041 0.000 0.041 0.000 {len}
71144 0.040 0.000 0.040 0.000 {method '__new__' of
'_ctypes.structure.StructOrUnionMeta' objects}
105450 0.039 0.000 0.039 0.000 {method 'free_temp_buffers' of
'_ffi.FuncPtr' objects}
176683 0.037 0.000 0.037 0.000 {hasattr}
35571 0.037 0.000 0.423 0.000 pcap.py:89(next)
Anyway, I attached my full project to this mail, including a sample pcap
capture file. It requires libpcap to be installed on your system. Run arp.py
to run the sniffer on the included demo.pcap capture file.
I realize I should try and narrow down the issue some more, but I can't really
afford to spend too much time on this right now so hopefully, this will be a
least a bit helpful.
Thanks!
Regards,
Sébastien
_______________________________________________
pypy-dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________
pypy-dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-dev
