Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Jamo, As per the log -> Connection closed by device, Device:/127.0.0.1:49130, NodeId:null, I am seeing there are some tcp connections being established and ended from the localhost ip to the localhost of controller node. So if there is a tcpdump available for loopback ip, we can try to find which application/process trying to establish the tcp connection to 6653 port. I have checked below tcp dumps but I am not seeing any loopback traces. Regards, Somashekhar -Original Message- From: Jamo Luhrsen [mailto:jluhr...@gmail.com] Sent: Tuesday, November 13, 2018 11:45 PM To: SOMASHEKHAR MANOHARA JAVALAGI Cc: D Arunprakash ; openflowplugin-dev ; odl netvirt dev Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections Somashekhar, are the pcaps we already take not good enough? I'm not sure if you need something extra, or if you missed the links to them from below. here they are again: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/control_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_2/ Thanks, JamO On 11/12/18 8:54 PM, SOMASHEKHAR MANOHARA JAVALAGI wrote: > Hi Jamo, > > I am able to see from the logs that some tcp connection is getting > established on port 6653 and as no openflow handshaking is not happening, it > is disconnecting. The same thing is repeating for every 2 seconds. Can you > please capture the wireshark traces on controller node for loopback messages. > > Regards, > Somashekhar > > -Original Message- > From: D Arunprakash > Sent: Tuesday, November 06, 2018 8:47 AM > To: Jamo Luhrsen ; openflowplugin-dev > ; odl netvirt dev > > Subject: RE: [netvirt-dev] problems with openflowplugin/ovs > connections > > We will look into this and log a jira and add the relevant links. > > Regards, > Arun > > -Original Message- > From: Jamo Luhrsen [mailto:jluhr...@gmail.com] > Sent: Tuesday, November 06, 2018 5:56 AM > To: D Arunprakash ; openflowplugin-dev > ; odl netvirt dev > > Subject: Re: [netvirt-dev] problems with openflowplugin/ovs > connections > > Hi Arun, > > any update on this one? do we need a jira to track it? These logs are coming > a lot and makes it very cumbersome to look at. Not to mention, they may be > indicating some problem internally. > > JamO > > On 10/29/18 4:29 PM, Jamo Luhrsen wrote: >> Hi Arun, >> >> sorry for the late response. We don't have packet captures on the odl >> nodes, but we do have them on the ovs nodes (2 computes and 1 control). >> >> they are done on a per suite basis, but I would just look at the >> captures during the elan suite since I know for sure that we are >> seeing all these messages when that suite is running. >> >> just look here: >> >> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-cs >> i >> t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntr >> a >> ck-oxygen/86/control_1/ >> >> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-cs >> i >> t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntr >> a >> ck-oxygen/86/compute_1/ >> >> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-cs >> i >> t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntr >> a >> ck-oxygen/86/compute_2/ >> >> >> in those folders you'll find a file like: >> >> tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz >> >> >> Thanks, >> JamO >> >> >> >> On 10/17/18 10:22 PM, D Arunprakash wrote: >>> Hi Jamo, >>> Do we have packet capture at odl node level, there seems to be connection >>> established on 6653 port in a repeated manner. >>> >>> Regards, >>> Arun >>> >>> -Original Message- >>> From: netvirt-dev-boun...@lists.opendaylight.org >>> [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of >>> Jamo Luhrsen >>> Sent: Wednesday, October 17, 2018 11:06 PM >>> To: openflowplugin-dev ; >>> odl netvirt dev >>> Subject: [netvirt-dev] problems with openflowplugin/ovs connections >>> >>> Hi OFP, >>> >>> we are debugging different failures in netvirt CSIT and
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
On 11/13/18 10:39 AM, Sai Sindhur Malleni wrote:
On Tue, Nov 13, 2018 at 1:32 PM Victor Pickard mailto:vpick...@redhat.com>> wrote:
On Mon, Nov 5, 2018 at 10:17 PM D Arunprakash mailto:d.arunprak...@ericsson.com>> wrote:
We will look into this and log a jira and add the relevant links.
Hi,
Was there a JIRA opened for these openflowplugin logs? We are seeing this
consistently in d/s oxygen cluster
deployments.
I think Arun was opening a jira, but I can't find it. Arun, do we need
one.
It's very bad actually. Here is a karaf log from a typical 3node netvirt
csit job:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-oxygen/132/odl_2/odl2_karaf.log.gz
70k of the 100k lines in that log file are these:
2018-11-12T06:08:19,108 | INFO | epollEventLoopGroup-9-1 | AbstractConnectionAdapter| 391 -
org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024
2018-11-12T06:08:19,199 | INFO | epollEventLoopGroup-9-3 | SystemNotificationsListenerImpl | 382 -
org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:41588,
NodeId:null
I did spin up a local ODL cluster in docker containers and connected
OVS to it from my host system. I could not reproduce it though. There
must be something else happening in our openstack deployments that
creates this situation.
JamO
D/S BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1649510
2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 |
SystemNotificationsListenerImpl | 382 -
org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection
closed by device, Device:/127.0.0.1:52546
<http://127.0.0.1:52546>,
NodeId:null
Thanks,
Vic
Regards,
Arun
-Original Message-
From: Jamo Luhrsen [mailto:jluhr...@gmail.com
<mailto:jluhr...@gmail.com>]
Sent: Tuesday, November 06, 2018 5:56 AM
To: D Arunprakash mailto:d.arunprak...@ericsson.com>>; openflowplugin-dev
mailto:openflowplugin-dev@lists.opendaylight.org>>; odl netvirt dev
mailto:netvirt-...@lists.opendaylight.org>>
Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections
Hi Arun,
any update on this one? do we need a jira to track it? These logs are
coming a lot and makes it very cumbersome
to look at. Not to mention, they may be indicating some problem
internally.
JamO
On 10/29/18 4:29 PM, Jamo Luhrsen wrote:
> Hi Arun,
>
> sorry for the late response. We don't have packet captures on the odl
> nodes, but we do have them on the ovs nodes (2 computes and 1
control).
>
> they are done on a per suite basis, but I would just look at the
> captures during the elan suite since I know for sure that we are
> seeing all these messages when that suite is running.
>
> just look here:
>
>
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi
>
t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra
> ck-oxygen/86/control_1/
>
>
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi
>
t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra
> ck-oxygen/86/compute_1/
>
>
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi
>
t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra
> ck-oxygen/86/compute_2/
>
>
> in those folders you'll find a file like:
>
> tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz
>
>
> Thanks,
> JamO
>
>
>
> On 10/17/18 10:22 PM, D Arunprakash wrote:
>> Hi Jamo,
>> Do we have packet capture at odl node level, there seems to be
connection established on 6653 port in a
repeated manner.
>>
>> Regards,
>> Arun
>>
>> -Original Message-
>> From: netvirt-dev-boun...@lists.opendaylight.org
<mailto:netvirt-dev-boun...@lists.opendaylight.org>
>> [mailto:netvirt-dev-boun...@lists.opendaylight.org
<mailto:netvirt-dev-boun...@lists.opendaylight.org>] On
Behalf Of Jamo
>> Luhrsen
>> Sent: Wednesday, October 17, 2018 11:06 PM
>> To: openflowplugin-dev mailto:openflowplugin-dev@lists.opendaylight.org>>;
>> odl
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
On Tue, Nov 13, 2018 at 1:32 PM Victor Pickard wrote: > > > On Mon, Nov 5, 2018 at 10:17 PM D Arunprakash > wrote: > >> We will look into this and log a jira and add the relevant links. >> > > Hi, > Was there a JIRA opened for these openflowplugin logs? We are seeing this > consistently in d/s oxygen cluster deployments. > D/S BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1649510 > > 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | > SystemNotificationsListenerImpl | 382 - > org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection > closed by device, Device:/127.0.0.1:52546, > NodeId:null > > > Thanks, > Vic > > >> >> Regards, >> Arun >> >> -Original Message- >> From: Jamo Luhrsen [mailto:jluhr...@gmail.com] >> Sent: Tuesday, November 06, 2018 5:56 AM >> To: D Arunprakash ; openflowplugin-dev < >> openflowplugin-dev@lists.opendaylight.org>; odl netvirt dev < >> netvirt-...@lists.opendaylight.org> >> Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections >> >> Hi Arun, >> >> any update on this one? do we need a jira to track it? These logs are >> coming a lot and makes it very cumbersome to look at. Not to mention, they >> may be indicating some problem internally. >> >> JamO >> >> On 10/29/18 4:29 PM, Jamo Luhrsen wrote: >> > Hi Arun, >> > >> > sorry for the late response. We don't have packet captures on the odl >> > nodes, but we do have them on the ovs nodes (2 computes and 1 control). >> > >> > they are done on a per suite basis, but I would just look at the >> > captures during the elan suite since I know for sure that we are >> > seeing all these messages when that suite is running. >> > >> > just look here: >> > >> > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi >> > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra >> > ck-oxygen/86/control_1/ >> > >> > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi >> > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra >> > ck-oxygen/86/compute_1/ >> > >> > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi >> > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra >> > ck-oxygen/86/compute_2/ >> > >> > >> > in those folders you'll find a file like: >> > >> > tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz >> > >> > >> > Thanks, >> > JamO >> > >> > >> > >> > On 10/17/18 10:22 PM, D Arunprakash wrote: >> >> Hi Jamo, >> >> Do we have packet capture at odl node level, there seems to be >> connection established on 6653 port in a repeated manner. >> >> >> >> Regards, >> >> Arun >> >> >> >> -Original Message- >> >> From: netvirt-dev-boun...@lists.opendaylight.org >> >> [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo >> >> Luhrsen >> >> Sent: Wednesday, October 17, 2018 11:06 PM >> >> To: openflowplugin-dev ; >> >> odl netvirt dev >> >> Subject: [netvirt-dev] problems with openflowplugin/ovs connections >> >> >> >> Hi OFP, >> >> >> >> we are debugging different failures in netvirt CSIT and it's starting >> >> to feel like we have some problems coming from OFP. To start, could >> someone take a look at this one: >> >> >> >> https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1 >> >> 039?filter=allopenissues >> >> >> >> I think this might only be happening in our clustering jobs, but it >> >> seems like the high level problem is that we can't set the connected >> node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. >> >> >> >> >> >> Also, we see a ton of logs like the below. what is this 127.0.0.1 >> >> nodeid:null thing? Since this is in the karaf.log, it seems that >> 127.0.0.1 would be on the host running ODL and that's not a node that we >> are running ovs (that I know of): >> >> >> >> >> >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | >> >> AbstractConnectionAdapter| 391 - >> >> org.opendaylight.openflowplug
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
On Mon, Nov 5, 2018 at 10:17 PM D Arunprakash wrote: > We will look into this and log a jira and add the relevant links. > Hi, Was there a JIRA opened for these openflowplugin logs? We are seeing this consistently in d/s oxygen cluster deployments. 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null Thanks, Vic > > Regards, > Arun > > -Original Message- > From: Jamo Luhrsen [mailto:jluhr...@gmail.com] > Sent: Tuesday, November 06, 2018 5:56 AM > To: D Arunprakash ; openflowplugin-dev < > openflowplugin-dev@lists.opendaylight.org>; odl netvirt dev < > netvirt-...@lists.opendaylight.org> > Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections > > Hi Arun, > > any update on this one? do we need a jira to track it? These logs are > coming a lot and makes it very cumbersome to look at. Not to mention, they > may be indicating some problem internally. > > JamO > > On 10/29/18 4:29 PM, Jamo Luhrsen wrote: > > Hi Arun, > > > > sorry for the late response. We don't have packet captures on the odl > > nodes, but we do have them on the ovs nodes (2 computes and 1 control). > > > > they are done on a per suite basis, but I would just look at the > > captures during the elan suite since I know for sure that we are > > seeing all these messages when that suite is running. > > > > just look here: > > > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > > ck-oxygen/86/control_1/ > > > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > > ck-oxygen/86/compute_1/ > > > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > > ck-oxygen/86/compute_2/ > > > > > > in those folders you'll find a file like: > > > > tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz > > > > > > Thanks, > > JamO > > > > > > > > On 10/17/18 10:22 PM, D Arunprakash wrote: > >> Hi Jamo, > >> Do we have packet capture at odl node level, there seems to be > connection established on 6653 port in a repeated manner. > >> > >> Regards, > >> Arun > >> > >> -Original Message- > >> From: netvirt-dev-boun...@lists.opendaylight.org > >> [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo > >> Luhrsen > >> Sent: Wednesday, October 17, 2018 11:06 PM > >> To: openflowplugin-dev ; > >> odl netvirt dev > >> Subject: [netvirt-dev] problems with openflowplugin/ovs connections > >> > >> Hi OFP, > >> > >> we are debugging different failures in netvirt CSIT and it's starting > >> to feel like we have some problems coming from OFP. To start, could > someone take a look at this one: > >> > >> https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1 > >> 039?filter=allopenissues > >> > >> I think this might only be happening in our clustering jobs, but it > >> seems like the high level problem is that we can't set the connected > node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. > >> > >> > >> Also, we see a ton of logs like the below. what is this 127.0.0.1 > >> nodeid:null thing? Since this is in the karaf.log, it seems that > 127.0.0.1 would be on the host running ODL and that's not a node that we > are running ovs (that I know of): > >> > >> > >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | > >> AbstractConnectionAdapter| 391 - > >> org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - > >> 0.6.4 | The channel outbound queue size:1024 > >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | > >> SystemNotificationsListenerImpl | 382 - > >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: > >> Connection closed by device, Device:/127.0.0.1:52538, NodeId:null > >> 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | > >> SystemNotificationsListenerImpl | 382 - > >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: > >> Connection closed by device,
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Somashekhar, are the pcaps we already take not good enough? I'm not sure if you need something extra, or if you missed the links to them from below. here they are again: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/control_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_2/ Thanks, JamO On 11/12/18 8:54 PM, SOMASHEKHAR MANOHARA JAVALAGI wrote: Hi Jamo, I am able to see from the logs that some tcp connection is getting established on port 6653 and as no openflow handshaking is not happening, it is disconnecting. The same thing is repeating for every 2 seconds. Can you please capture the wireshark traces on controller node for loopback messages. Regards, Somashekhar -Original Message- From: D Arunprakash Sent: Tuesday, November 06, 2018 8:47 AM To: Jamo Luhrsen ; openflowplugin-dev ; odl netvirt dev Subject: RE: [netvirt-dev] problems with openflowplugin/ovs connections We will look into this and log a jira and add the relevant links. Regards, Arun -Original Message- From: Jamo Luhrsen [mailto:jluhr...@gmail.com] Sent: Tuesday, November 06, 2018 5:56 AM To: D Arunprakash ; openflowplugin-dev ; odl netvirt dev Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections Hi Arun, any update on this one? do we need a jira to track it? These logs are coming a lot and makes it very cumbersome to look at. Not to mention, they may be indicating some problem internally. JamO On 10/29/18 4:29 PM, Jamo Luhrsen wrote: Hi Arun, sorry for the late response. We don't have packet captures on the odl nodes, but we do have them on the ovs nodes (2 computes and 1 control). they are done on a per suite basis, but I would just look at the captures during the elan suite since I know for sure that we are seeing all these messages when that suite is running. just look here: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra ck-oxygen/86/control_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra ck-oxygen/86/compute_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra ck-oxygen/86/compute_2/ in those folders you'll find a file like: tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz Thanks, JamO On 10/17/18 10:22 PM, D Arunprakash wrote: Hi Jamo, Do we have packet capture at odl node level, there seems to be connection established on 6653 port in a repeated manner. Regards, Arun -Original Message- From: netvirt-dev-boun...@lists.opendaylight.org [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo Luhrsen Sent: Wednesday, October 17, 2018 11:06 PM To: openflowplugin-dev ; odl netvirt dev Subject: [netvirt-dev] problems with openflowplugin/ovs connections Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1 039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Jamo, I am able to see from the logs that some tcp connection is getting established on port 6653 and as no openflow handshaking is not happening, it is disconnecting. The same thing is repeating for every 2 seconds. Can you please capture the wireshark traces on controller node for loopback messages. Regards, Somashekhar -Original Message- From: D Arunprakash Sent: Tuesday, November 06, 2018 8:47 AM To: Jamo Luhrsen ; openflowplugin-dev ; odl netvirt dev Subject: RE: [netvirt-dev] problems with openflowplugin/ovs connections We will look into this and log a jira and add the relevant links. Regards, Arun -Original Message- From: Jamo Luhrsen [mailto:jluhr...@gmail.com] Sent: Tuesday, November 06, 2018 5:56 AM To: D Arunprakash ; openflowplugin-dev ; odl netvirt dev Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections Hi Arun, any update on this one? do we need a jira to track it? These logs are coming a lot and makes it very cumbersome to look at. Not to mention, they may be indicating some problem internally. JamO On 10/29/18 4:29 PM, Jamo Luhrsen wrote: > Hi Arun, > > sorry for the late response. We don't have packet captures on the odl > nodes, but we do have them on the ovs nodes (2 computes and 1 control). > > they are done on a per suite basis, but I would just look at the > captures during the elan suite since I know for sure that we are > seeing all these messages when that suite is running. > > just look here: > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/control_1/ > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/compute_1/ > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/compute_2/ > > > in those folders you'll find a file like: > > tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz > > > Thanks, > JamO > > > > On 10/17/18 10:22 PM, D Arunprakash wrote: >> Hi Jamo, >> Do we have packet capture at odl node level, there seems to be connection >> established on 6653 port in a repeated manner. >> >> Regards, >> Arun >> >> -Original Message- >> From: netvirt-dev-boun...@lists.opendaylight.org >> [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo >> Luhrsen >> Sent: Wednesday, October 17, 2018 11:06 PM >> To: openflowplugin-dev ; >> odl netvirt dev >> Subject: [netvirt-dev] problems with openflowplugin/ovs connections >> >> Hi OFP, >> >> we are debugging different failures in netvirt CSIT and it's starting >> to feel like we have some problems coming from OFP. To start, could someone >> take a look at this one: >> >> https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1 >> 039?filter=allopenissues >> >> I think this might only be happening in our clustering jobs, but it >> seems like the high level problem is that we can't set the connected node to >> SLAVE and cancel the ovs connect (after 20s). Then it just repeats. >> >> >> Also, we see a ton of logs like the below. what is this 127.0.0.1 >> nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 >> would be on the host running ODL and that's not a node that we are running >> ovs (that I know of): >> >> >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | >> AbstractConnectionAdapter | 391 - >> org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - >> 0.6.4 | The channel outbound queue size:1024 >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:52538, NodeId:null >> 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:49130, NodeId:null >> 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:49126, NodeId:null >> 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | >> SystemNotificationsL
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
We will look into this and log a jira and add the relevant links. Regards, Arun -Original Message- From: Jamo Luhrsen [mailto:jluhr...@gmail.com] Sent: Tuesday, November 06, 2018 5:56 AM To: D Arunprakash ; openflowplugin-dev ; odl netvirt dev Subject: Re: [netvirt-dev] problems with openflowplugin/ovs connections Hi Arun, any update on this one? do we need a jira to track it? These logs are coming a lot and makes it very cumbersome to look at. Not to mention, they may be indicating some problem internally. JamO On 10/29/18 4:29 PM, Jamo Luhrsen wrote: > Hi Arun, > > sorry for the late response. We don't have packet captures on the odl > nodes, but we do have them on the ovs nodes (2 computes and 1 control). > > they are done on a per suite basis, but I would just look at the > captures during the elan suite since I know for sure that we are > seeing all these messages when that suite is running. > > just look here: > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/control_1/ > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/compute_1/ > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csi > t-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntra > ck-oxygen/86/compute_2/ > > > in those folders you'll find a file like: > > tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz > > > Thanks, > JamO > > > > On 10/17/18 10:22 PM, D Arunprakash wrote: >> Hi Jamo, >> Do we have packet capture at odl node level, there seems to be connection >> established on 6653 port in a repeated manner. >> >> Regards, >> Arun >> >> -Original Message- >> From: netvirt-dev-boun...@lists.opendaylight.org >> [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo >> Luhrsen >> Sent: Wednesday, October 17, 2018 11:06 PM >> To: openflowplugin-dev ; >> odl netvirt dev >> Subject: [netvirt-dev] problems with openflowplugin/ovs connections >> >> Hi OFP, >> >> we are debugging different failures in netvirt CSIT and it's starting >> to feel like we have some problems coming from OFP. To start, could someone >> take a look at this one: >> >> https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1 >> 039?filter=allopenissues >> >> I think this might only be happening in our clustering jobs, but it >> seems like the high level problem is that we can't set the connected node to >> SLAVE and cancel the ovs connect (after 20s). Then it just repeats. >> >> >> Also, we see a ton of logs like the below. what is this 127.0.0.1 >> nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 >> would be on the host running ODL and that's not a node that we are running >> ovs (that I know of): >> >> >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | >> AbstractConnectionAdapter | 391 - >> org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - >> 0.6.4 | The channel outbound queue size:1024 >> 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:52538, NodeId:null >> 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:49130, NodeId:null >> 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:49126, NodeId:null >> 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:52546, NodeId:null >> 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | >> SystemNotificationsListenerImpl | 382 - >> org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: >> Connection closed by device, Device:/127.0.0.1:52542, NodeId:null >> 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | >> AbstractConnectionAdapter
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Arun, any update on this one? do we need a jira to track it? These logs are coming a lot and makes it very cumbersome to look at. Not to mention, they may be indicating some problem internally. JamO On 10/29/18 4:29 PM, Jamo Luhrsen wrote: Hi Arun, sorry for the late response. We don't have packet captures on the odl nodes, but we do have them on the ovs nodes (2 computes and 1 control). they are done on a per suite basis, but I would just look at the captures during the elan suite since I know for sure that we are seeing all these messages when that suite is running. just look here: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/control_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_2/ in those folders you'll find a file like: tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz Thanks, JamO On 10/17/18 10:22 PM, D Arunprakash wrote: Hi Jamo, Do we have packet capture at odl node level, there seems to be connection established on 6653 port in a repeated manner. Regards, Arun -Original Message- From: netvirt-dev-boun...@lists.opendaylight.org [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo Luhrsen Sent: Wednesday, October 17, 2018 11:06 PM To: openflowplugin-dev ; odl netvirt dev Subject: [netvirt-dev] problems with openflowplugin/ovs connections Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52542, NodeId:null 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-9-10 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-11-8 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,914 | INFO | epollEventLoopGroup-9-11 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-9 | AbstractConnectionAdapter | 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-10 |
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Arun, sorry for the late response. We don't have packet captures on the odl nodes, but we do have them on the ovs nodes (2 computes and 1 control). they are done on a per suite basis, but I would just look at the captures during the elan suite since I know for sure that we are seeing all these messages when that suite is running. just look here: https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/control_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_1/ https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-3node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-snat-conntrack-oxygen/86/compute_2/ in those folders you'll find a file like: tcpdump_port_6653__09_elan__10.30.170.134.pcap.xz Thanks, JamO On 10/17/18 10:22 PM, D Arunprakash wrote: Hi Jamo, Do we have packet capture at odl node level, there seems to be connection established on 6653 port in a repeated manner. Regards, Arun -Original Message- From: netvirt-dev-boun...@lists.opendaylight.org [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo Luhrsen Sent: Wednesday, October 17, 2018 11:06 PM To: openflowplugin-dev ; odl netvirt dev Subject: [netvirt-dev] problems with openflowplugin/ovs connections Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52542, NodeId:null 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-9-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-11-8 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,914 | INFO | epollEventLoopGroup-9-11 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 Thanks, JamO ___ netvirt-dev mailing list
Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Jamo, Do we have packet capture at odl node level, there seems to be connection established on 6653 port in a repeated manner. Regards, Arun -Original Message- From: netvirt-dev-boun...@lists.opendaylight.org [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo Luhrsen Sent: Wednesday, October 17, 2018 11:06 PM To: openflowplugin-dev ; odl netvirt dev Subject: [netvirt-dev] problems with openflowplugin/ovs connections Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52542, NodeId:null 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-9-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-11-8 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,914 | INFO | epollEventLoopGroup-9-11 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 Thanks, JamO ___ netvirt-dev mailing list netvirt-...@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/netvirt-dev ___ openflowplugin-dev mailing list openflowplugin-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
