Hi,
I've been trying some things over the last couple days...
The pypy problem was indeed due to some external modules. Debug-pox.py does
not provide much helpfull information. Can I suggest adding a
traceback.print_exc() when import fails ( +- line 80 in boot.py [after:
print("Module not found:", base_name)]). In my case it really showed which
import failed.
For profiling I tried yappi (https://code.google.com/p/yappi/). Not as
handy as cProfile with RunSnakeRun, but it works with the threading model
and provides #calls, total time,... per function. It requires some changes
in the code, but it's possible to create some wrappers and load it as a POX
module. Let me know if you're interested in the code :)
And than the disconnecting links issue in discovery. First to be sure it's
completely clear: it's only discovery that fires timeout event. The
connections to the switches don't timeout.
I have indeed some heavier processing code. The first bottleneck (search in
a list) is resolved (I'm using another algorithm now). Performance got a
lot better.
The second issue is the parsing of flow stats. When I'm getting a
flow_stat_reply from a switch I'm parsing the statistics for each flow
contained in the reply. It goes for up to about 300 flows, but from that
point links start disconnecting again. I tried to split the calculation
into different parts (don't process them in one loop, but fire an event for
each flow which will process only that flow). So far this had no measurable
impact. I'm guessing these events are processed right afterwards which
basically goes back to the "one big for loop" scenario. Can this be the
case? Pypy offers an improvement of going up to about 550 flows, but then
the same issues arise again.
Further, I was looking at the recoco and revent libraries. What I'd like to
do is submit the "processing events" with a lower priority, so the packet
in events are processed first. I guess this can resolve the problem?
Are there some features in recoco or revent which could help in
implementing this? When I print the length of the schedule queue (cycle
function in recoco), not all fired events seem to be scheduled as a
separate tasks. Where is the processing queue for the events situated?
And finally I noticed strongly varying performance (100 flows more or less
in the reply before it starts crashing) with exactly the same traffic
patterns. Has this something to do with the random generator in the recoco
scheduler cycle() function?
Thanks!
Tim
2013/3/19 Murphy McCauley <[email protected]>
> On Mar 19, 2013, at 12:02 PM, Tmusic wrote:
> > I'm trying to run with pypy but it can't import some of my modules (all
> in the /pox directory). For example myrouter.mylinkmonitor does not work,
> but my myrouter.mypackethandler does.
> >
> > python pox.py <modules> works like a charm
> > ./pox.py <modules> cannot find all modules
> > <path>/pypy pox.py <modules> has the same problem as the latter
> >
> > It's probably something silly, but I can't find it...
>
> This is probably because you're importing modules that aren't installed in
> PyPy. If they're C modules, you may be out of luck. If they're Python
> modules, you just need to adjust your path or install them in PyPy. For
> more info, try "pypy/bin/pypy debug-pox.py ..." or something along those
> lines and see if it's more informative.
>
> > Concerning the number of packet_in events. It's about 10 per second (+
> about 10 stats_requests and replies, which are logged by a separate module).
>
> Are your event handlers doing much work (and stalling the coop thread)?
>
> -- Murphy
