Re: [Ryu-devel] Have Ryu application function as client instead of server?

Iwase Yusuke Tue, 10 Oct 2017 23:03:08 -0700

Hi Matt,


On 2017年10月11日 02:02, Matthew Moskowitz wrote:

Hi Iwase,
I think my controller was receiving OFPFeaturesRequest due to a problem in my setup, once I got thesystem working it was no longer receiving this message.
As far as the server_loop, I think it is desirable to be able to run ryu only as client withouthaving a server running as well. So perhaps there should be a flag to turn off the server?


Could you let me know more details about why server_loop should not be started 
with client_loop?
If running server_loop and client_loop simultaneously is dangerous, a new flag 
should not be
required, I guess.
server_loop or client_loop should suppressed "implicitly" or with warning 
messages.


By the way, if server_loop suppressed as following, will ryu-manager quit 
immediately?

             addr = tuple(_split_addr(address))
              self.spawn_client_loop(addr)

+ # Don't spawn a server loop is Ryu has been run as a client

 +        if self._clients != {}:
 +            return
          self.server_loop(self.ofp_tcp_listen_port,
                           self.ofp_ssl_listen_port)


The reason of this behavior is that ryu-manager is waiting for the exit of 
server_loop.
If we don't start server_loop, we need to block the exit in 
OpenFlowController.__call__() method.
Then, How about the following?
I've also stop spawning server_loop in __init__() method.


$ git diff
diff --git a/ryu/controller/controller.py b/ryu/controller/controller.py
index 2de14a7..28ca21f 100644
--- a/ryu/controller/controller.py
+++ b/ryu/controller/controller.py
@@ -121,24 +121,15 @@ def _split_addr(addr):
 class OpenFlowController(object):
     def __init__(self):
         super(OpenFlowController, self).__init__()
-        if not CONF.ofp_tcp_listen_port and not CONF.ofp_ssl_listen_port:
-            self.ofp_tcp_listen_port = ofproto_common.OFP_TCP_PORT
-            self.ofp_ssl_listen_port = ofproto_common.OFP_SSL_PORT
-            # For the backward compatibility, we spawn a server loop
-            # listening on the old OpenFlow listen port 6633.
-            hub.spawn(self.server_loop,
-                      ofproto_common.OFP_TCP_PORT_OLD,
-                      ofproto_common.OFP_SSL_PORT_OLD)
-        else:
-            self.ofp_tcp_listen_port = CONF.ofp_tcp_listen_port
-            self.ofp_ssl_listen_port = CONF.ofp_ssl_listen_port
-
         # Example:
         # self._clients = {
         #     ('127.0.0.1', 6653): <instance of StreamClient>,
         # }
         self._clients = {}

+        # Event to stop client/server loops
+        self._event = hub.Event()
+
     # entry point
     def __call__(self):
         # LOG.debug('call')
@@ -146,8 +137,25 @@ class OpenFlowController(object):
             addr = tuple(_split_addr(address))
             self.spawn_client_loop(addr)

-        self.server_loop(self.ofp_tcp_listen_port,
-                         self.ofp_ssl_listen_port)
+        # If starting client mode, avoid spawning server loop
+        if self._clients:
+            self._event.wait()
+            return
+
+        if not CONF.ofp_tcp_listen_port and not CONF.ofp_ssl_listen_port:
+            ofp_tcp_listen_port = ofproto_common.OFP_TCP_PORT
+            ofp_ssl_listen_port = ofproto_common.OFP_SSL_PORT
+            # For the backward compatibility, we spawn a server loop
+            # listening on the old OpenFlow listen port 6633.
+            hub.spawn(self.server_loop,
+                      ofproto_common.OFP_TCP_PORT_OLD,
+                      ofproto_common.OFP_SSL_PORT_OLD)
+        else:
+            ofp_tcp_listen_port = CONF.ofp_tcp_listen_port
+            ofp_ssl_listen_port = CONF.ofp_ssl_listen_port
+
+        hub.spawn(self.server_loop, ofp_tcp_listen_port, ofp_ssl_listen_port)
+        self._event.wait()

     def spawn_client_loop(self, addr, interval=None):
         interval = interval or CONF.ofp_switch_connect_interval
@@ -186,6 +194,11 @@ class OpenFlowController(object):
         # LOG.debug('loop')
         server.serve_forever()

+    def stop(self):
+        for client in self._clients.values():
+            client.stop()
+        self._event.set()
+

 def _deactivate(method):
     def deactivate(self):
diff --git a/ryu/controller/ofp_handler.py b/ryu/controller/ofp_handler.py
index 4f439c2..c62e6a8 100644
--- a/ryu/controller/ofp_handler.py
+++ b/ryu/controller/ofp_handler.py
@@ -59,6 +59,10 @@ class OFPHandler(ryu.base.app_manager.RyuApp):
         self.controller = OpenFlowController()
         return hub.spawn(self.controller)

+    def stop(self):
+        self.controller.stop()
+        super(OFPHandler, self).stop()
+
     def _hello_failed(self, datapath, error_desc):
         self.logger.error(error_desc)
         error_msg = datapath.ofproto_parser.OFPErrorMsg(datapath)


Thanks,
Iwase


Thanks,
Matt

On Mon, Oct 9, 2017 at 8:16 PM, Iwase Yusuke <iwase.yusu...@gmail.com<mailto:iwase.yusu...@gmail.com>> wrote:


    Hi Matt,

    Sorry for the delay.


    Currently, OFPFeaturesRequest's parser is not registered because 
OFPFeaturesRequest message should
    be sent from controller to switch and not be received by controller.
    Ryu is a controller implementation, generally needs not to receive (and 
parse) OFPFeaturesRequest
    message, I guess.

    To register the parser of OFPFeaturesRequest looks good to me, but why your 
Ryu needs to receive
    OFPFeaturesRequest message?


    By the way, your patch stops spawning 'server_loop' if any client exists, 
but for example, Ryu can
    can be server against switch A and can be client against switch B 
simultaneously, I guess.
    I apologize if I probably misled you with my patch comment, I want to avoid 
having Ryu act as
    server and client simultaneously against the "same" switch.
    e.g.)
       OK:  switchA(client)---(server)Ryu(client)---(server)switchB
       NG:  switchA(client AND server)---(server AND client)Ryu


    Thanks,
    Iwase

    On 2017年10月10日 08:37, Matthew Moskowitz wrote:

        Hello,

        I have gotten the application to successfully function as a client. 
I've attached a patch
        with a couple of small changes I made to your branch, Iwase. Also, here 
is a link to my fork
        of the repo with those changes. One of the changes is to not allow Ryu 
to run as client and
        server simultaneously. I am not sure if this is desirable or not so 
please advise.

        
https://github.com/MMoskowitz9/ryu/tree/controller-Support_proactive_connection
        
<https://github.com/MMoskowitz9/ryu/tree/controller-Support_proactive_connection>

        Thanks,
        Matt

        On Mon, Oct 9, 2017 at 9:58 AM, Matthew Moskowitz <m...@bestateless.com
        <mailto:m...@bestateless.com> <mailto:m...@bestateless.com 
<mailto:m...@bestateless.com>>>
        wrote:

             Fortunately I have an update on the issue from above. The class
        OFPFeaturesRequest(MsgBase):
             appears to be missing the @_register_parser decorator. I can push 
a patch for this but
        I will
             continue testing the code today.

             On Sun, Oct 8, 2017 at 3:50 PM, Matthew Moskowitz 
<m...@bestateless.com
        <mailto:m...@bestateless.com>
             <mailto:m...@bestateless.com <mailto:m...@bestateless.com>>> wrote:

                 Hi Iwase,

                 I am currently trying to figure out an issue in the new code. 
When I use the
                 --ofp-switch-address-list flag, it goes into the 
ofproto_v1_5_parser.py file
        shortly after
                 launch. The msg_parser function receives a message with 
msg_type of 5, however as
        you can
                 see below message type 5 is not loaded into the contents of 
the _MSG_PARSERS dictionary
                 (contents of dict printed below). I am not sure if this is an 
issue with my
        environment or
                 the new code.



                 {0: <bound method OFPHello.parser of <class
        'ryu.ofproto.ofproto_v1_5_parser.OFPHello'>>,
                   1: <bound method OFPErrorMsg.parser of <class
        'ryu.ofproto.ofproto_v1_5_parser.OFPErrorMsg'>>,
                   2: <bound method OFPEchoRequest.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPEchoRequest'>>,
                   3: <bound method OFPEchoReply.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPEchoReply'>>,
                   4: <bound method OFPExperimenter.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPExperimenter'>>,
                   6: <bound method OFPSwitchFeatures.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPSwitchFeatures'>>,
                   8: <bound method OFPGetConfigReply.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPGetConfigReply'>>,
                   10: <bound method OFPPacketIn.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPPacketIn'>>,
                   11: <bound method OFPFlowRemoved.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPFlowRemoved'>>,
                   12: <bound method OFPPortStatus.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPPortStatus'>>,
                   14: <bound method OFPFlowMod.parser of <class
        'ryu.ofproto.ofproto_v1_5_parser.OFPFlowMod'>>,
                   15: <bound method OFPGroupMod.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPGroupMod'>>,
                   19: <bound method OFPMultipartReply.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPMultipartReply'>>,
                   21: <bound method MsgBase.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPBarrierReply'>>,
                   25: <bound method OFPRoleReply.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPRoleReply'>>,
                   27: <bound method OFPGetAsyncReply.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPGetAsyncReply'>>,
                   29: <bound method OFPMeterMod.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPMeterMod'>>,
                   30: <bound method OFPRoleStatus.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPRoleStatus'>>,
                   31: <bound method OFPTableStatus.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPTableStatus'>>,
                   32: <bound method OFPRequestForward.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPRequestForward'>>,
                   33: <bound method OFPBundleCtrlMsg.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPBundleCtrlMsg'>>,
                   35: <bound method OFPControllerStatus.parser of <class
                 'ryu.ofproto.ofproto_v1_5_parser.OFPControllerStatus'>>}


                 On Fri, Oct 6, 2017 at 9:32 AM, Matthew Moskowitz 
<m...@bestateless.com
        <mailto:m...@bestateless.com>
                 <mailto:m...@bestateless.com <mailto:m...@bestateless.com>>> 
wrote:

                     Hi Iwase,

                     Great, I will test this out today.

                     On Thu, Oct 5, 2017 at 7:43 PM, Iwase Yusuke 
<iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>
                     <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>> wrote:

                         Hi Matt,

                         I've posted patches to support this feature. Could you 
try it?
                            [Ryu-devel] [PATCH 1/3] controller: Support 
proactive connection

                         Note: Currently, Ryu does not aware of whether 
connections is reactive or
        proactive,
                         so it is not recommended to use them simultaneously 
against the same switch.
                         The connection might be duplicated.

                         Thanks,
                         Iwase


                         On 2017年10月06日 07:24, Matthew Moskowitz wrote:

                             Awesome! I look forward to trying it out.

                             On Thu, Oct 5, 2017 at 12:34 AM, Iwase Yusuke 
<iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>>> wrote:

                                  Hi Matt,

                                  Currently, I'm testing this feature on my 
branch.
                                  (Still this have some buggy behaviors 
though...)
        
https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection
        
<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection>

<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection

        
<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection>>

<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection

        
<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection>

<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection

        
<https://github.com/iwaseyusuke/ryu/tree/controller-Support_proactive_connection>>>

                                  But, the amount of task should not differ so 
much from this
        branch, I guess.

                                  Thanks,
                                  Iwase


                                  On 2017年10月05日 15:16, Matthew Moskowitz wrote:

                                      Thank you for the advice on the failover 
Iwase.

                                      As far as having Ryu behave as the 
client, do you have any
        insight into
                             how large of a task
                                      this would be?

                                      Matt

                                      On Tue, Oct 3, 2017 at 8:58 PM, Iwase 
Yusuke
        <iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>
                                      <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com> <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>>>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>>
                                      <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com> <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>>>>> wrote:

                                           Hi Matt,

                                           Thank you for sharing use-cases.

                                           For the latter case, I guess it 
should take some lags to
        change
                             controllers addresses
                                      on switches
                                           to do efficiently failover.
                                           Might have still some lags, but how 
about using virtual
        IPs on
                             controller side (like
                                      clustering
                                           controller)?
                                           I should be smaller lags than 
changing addresses on
        switch side.

                                           Thanks,
                                           Iwase


                                           On 2017年10月04日 10:56, Matthew 
Moskowitz wrote:

                                               Thanks Iwase and Carlos for your 
thoughts,

                                               2 cases that this feature is 
interesting to me for
        would be:

                                               Integrating Ryu with other 
software that is already
        running a
                             different
                                               webserver/asynchronous library. 
Nicer in this case
        not to have
                             to deal with 2 different
                                               webservers/coroutine types.

                                               Efficient failover if one Ryu 
application should fail
        for any
                             reason, the new one can
                                               instantly connect to the switch. 
Your suggestion to
        use the
                             ovsdb library could
                                      help with
                                               this one.

                                               Thanks,
                                               Matt

                                               On Tue, Oct 3, 2017 at 7:20 PM, 
Iwase Yusuke
                             <iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>>
                                      <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com> <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>>>
                                               <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>>>
                                      <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com> <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>>>

                                               <mailto:iwase.yusu...@gmail.com
        <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>
        <mailto:iwase.yusu...@gmail.com <mailto:iwase.yusu...@gmail.com>
                             <mailto:iwase.yusu...@gmail.com 
<mailto:iwase.yusu...@gmail.com>>>>>>
        wrote:

                                                    Hi Carlos,

                                                    Thank you for your reply!

                                                    As you said, the OpenFlow 
Spec does not restrict
        Switches
                             must initiate
                                      connection to
                                               Controllers.
                                                    But initiating connection 
by Controllers seems
        to be the
                             "optional" feature.
                                                    ===
                                                    Quote: OpenFlow Spec 1.3.5 
"6.3.3 Connection Setup"
                                                    ===
                                                    Optionally, the switch may 
allow the controller to
                             initiate the connection. In
                                      this
                                               case, the
                                                    switch should
                                                    accept incoming standard 
TLS or TCP connections
        from the
                             controller, using
                                      either a
                                               user-specified
                                                    transport port or the 
default OpenFlow transport
        port
                             6653. Connections
                                      initiated by
                                               the switch and
                                                    the controller behave the 
same once the transport
                             connection is established.
                                                    ===

                                                    If OVS does not know on 
where Ryu is listening
        when OVS
                             starting up, how about
                                      using
                                               OVSDB library
                                                    to tell controller's 
IP:Port to OVS?
                                                    If OVS listening on OVSDB 
connection,
        "set-controller"
                             API can tell its own
                                      address to OVS.
        
https://github.com/osrg/ryu/blob/6e69e9b8a6ebd2bb06aa864cc76ef0b79d1ac9c3/ryu/lib/ovs/bridge.py#L128
        
<https://github.com/osrg/ryu/blob/6e69e9b8a6ebd2bb06aa864cc76ef0b79d1ac9c3/ryu/lib/ovs/bridge.py#L128>