Hi FUJITA: I'm using heavily customized ofagent which is totally different from the origin one.
My agent calls remove flow inside packet_in handler which causes deadlock. the packet_in handler is blocked by remove flow waiting for the barrier reply, but the receive queue is flooded with packet_in message and blocks the receive green thread. As a result, no msg will be processed, and the receive green thread blocks forever On Tue, Feb 10, 2015 at 10:14 PM, FUJITA Tomonori <fujita.tomon...@lab.ntt.co.jp> wrote: > On Mon, 9 Feb 2015 16:24:41 +0800 > Zang MingJie <zealot0...@gmail.com> wrote: > >> Hi all: >> >> I'm developing a neutron agent based on ryu controller. >> >> I found ryu controller randomly hang when doing stress test, below is >> the stacktrace of hanged greenthread: > > You use ofa_neutron_agent? What kinda of the stress test? > > >> ------ Green Thread ------ >> >> /usr/lib/python2.7/dist-packages/eventlet/greenthread.py:194 in main >> `result = function(*args, **kwargs)` >> >> /usr/lib/python2.7/dist-packages/ryu/lib/hub.py:52 in _launch >> `func(*args, **kwargs)` >> >> /usr/lib/python2.7/dist-packages/ryu/base/app_manager.py:274 in _event_loop >> `handler(ev)` >> >> /usr/lib/python2.7/dist-packages/neutron/plugins/ofagent/agent/ofa_neutron_agent.py:713 >> in _packet_in_handler >> `self.agent._of_packet_in(ev)` >> >> /usr/lib/python2.7/dist-packages/neutron/common/log.py:34 in wrapper >> `return method(*args, **kwargs)` >> >> /usr/lib/python2.7/dist-packages/neutron/plugins/ofagent/agent/ofa_neutron_agent.py:1028 >> in _of_packet_in >> `self.remove_flows(ofpp.OFPMatch(eth_src=p_ether.dst))` >> >> /usr/lib/python2.7/dist-packages/neutron/plugins/ofagent/agent/ofa_neutron_agent.py:883 >> in remove_flows >> `ryu_api.send_msg(app=self.ryuapp, msg=flow)` >> >> /usr/lib/python2.7/dist-packages/ryu/app/ofctl/api.py:65 in send_msg >> `reply_multi=reply_multi))()` >> >> /usr/lib/python2.7/dist-packages/ryu/base/app_manager.py:265 in send_request >> `return req.reply_q.get()` >> >> /usr/lib/python2.7/dist-packages/eventlet/queue.py:298 in get >> `return waiter.wait()` >> >> /usr/lib/python2.7/dist-packages/eventlet/queue.py:129 in wait >> `return get_hub().switch()` >> >> /usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py:187 in switch >> `return self.greenlet.switch()` >> >> >> the green thread will hang for days never recover. I have several >> green thread sending openflow commands, I don't know if it is causing >> problems >> >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming. The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. Take a >> look and join the conversation now. http://goparallel.sourceforge.net/ >> _______________________________________________ >> Ryu-devel mailing list >> Ryu-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/ryu-devel ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Ryu-devel mailing list Ryu-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ryu-devel
