Re: [openflowplugin-dev] [netvirt-dev] problems with openflowplugin/ovs connections
Hi Jamo, Do we have packet capture at odl node level, there seems to be connection established on 6653 port in a repeated manner. Regards, Arun -Original Message- From: netvirt-dev-boun...@lists.opendaylight.org [mailto:netvirt-dev-boun...@lists.opendaylight.org] On Behalf Of Jamo Luhrsen Sent: Wednesday, October 17, 2018 11:06 PM To: openflowplugin-dev ; odl netvirt dev Subject: [netvirt-dev] problems with openflowplugin/ovs connections Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52542, NodeId:null 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-9-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-11-8 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,914 | INFO | epollEventLoopGroup-9-11 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 Thanks, JamO ___ netvirt-dev mailing list netvirt-...@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/netvirt-dev ___ openflowplugin-dev mailing list openflowplugin-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
[openflowplugin-dev] problems with openflowplugin/ovs connections
Hi OFP, we are debugging different failures in netvirt CSIT and it's starting to feel like we have some problems coming from OFP. To start, could someone take a look at this one: https://jira.opendaylight.org/projects/OPNFLWPLUG/issues/OPNFLWPLUG-1039?filter=allopenissues I think this might only be happening in our clustering jobs, but it seems like the high level problem is that we can't set the connected node to SLAVE and cancel the ovs connect (after 20s). Then it just repeats. Also, we see a ton of logs like the below. what is this 127.0.0.1 nodeid:null thing? Since this is in the karaf.log, it seems that 127.0.0.1 would be on the host running ODL and that's not a node that we are running ovs (that I know of): 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-11-7 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:38,904 | INFO | epollEventLoopGroup-9-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52538, NodeId:null 2018-10-07T21:54:38,905 | INFO | epollEventLoopGroup-11-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49130, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-11-6 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:49126, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-8 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52546, NodeId:null 2018-10-07T21:54:38,906 | INFO | epollEventLoopGroup-9-7 | SystemNotificationsListenerImpl | 382 - org.opendaylight.openflowplugin.impl - 0.6.4 | ConnectionEvent: Connection closed by device, Device:/127.0.0.1:52542, NodeId:null 2018-10-07T21:54:40,912 | INFO | epollEventLoopGroup-9-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-9-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,913 | INFO | epollEventLoopGroup-11-8 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,914 | INFO | epollEventLoopGroup-9-11 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-9 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 2018-10-07T21:54:40,915 | INFO | epollEventLoopGroup-11-10 | AbstractConnectionAdapter| 391 - org.opendaylight.openflowplugin.openflowjava.openflow-protocol-impl - 0.6.4 | The channel outbound queue size:1024 Thanks, JamO ___ openflowplugin-dev mailing list openflowplugin-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
Re: [openflowplugin-dev] [release] Autorelease oxygen failed to build openflow-protocol-it from openflowplugin
Make sense. Thanks. On Wed, Oct 17, 2018 at 9:00 AM Anil Vishnoi wrote: > Last patch merged to openflowplugin stable/oxygen branch was on October > 4th, so no recent change. It looks like test is trying to start a client to > test the tls connection and timed out. I never seen this test failing > before. I will wait for tomorrow's autorelease and see if this happens > again. If it happens again, we will investigate the issue. > > Thanks > Anil > > On Tue, Oct 16, 2018 at 10:41 PM Ariel Adam wrote: > >> Anil, did you guys change anything? >> We get the following timeout exception: >> >> --- >> T E S T S >> --- >> Running org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest >> Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.949 sec >> <<< FAILURE! - in >> org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest >> testTlsHandshake(org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest) >> Time elapsed: 3.793 sec <<< ERROR! >> java.util.concurrent.TimeoutException: Waited 2000 milliseconds for >> com.google.common.util.concurrent.SettableFuture@242ff747[status=PENDING] >> at >> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:470) >> at >> com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:92) >> at >> org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest.createAndStartClient(IntegrationTest.java:255) >> at >> org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest.testTlsHandshake(IntegrationTest.java:134) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) >> at >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) >> at >> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) >> at >> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) >> at >> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) >> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) >> at >> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) >> at org.junit.runners.ParentRunner.run(ParentRunner.java:309) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) >> at >> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) >> at >> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) >> at >> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) >> at >> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) >> >> >> Results : >> >> Tests in error: >> IntegrationTest.testTlsHandshake:134->createAndStartClient:255 » Timeout >> Waite... >> >> Tests run: 6, Failures: 0, Errors: 1, Skipped: 0 >> >> >> Thanks. >> >> On Wed, Oct 17, 2018 at 6:57 AM Jenkins < >> jenkins-dontre...@opendaylight.org> wrote: >> >>> Attention openflowplugin-devs, >>> >>> Autorelease oxygen failed to build openflow-protocol-it from >>> openflowplugin in build >>> 455. Attached is a snippet of the error message related to the >>> failure that we were able to automatically parse as well as console >>> logs. >>> >>> >>> Console Logs: >>> >>> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-oxygen/455 >>> >>> Jenkins Build: >>> >>> https://jenkins.opendaylight.org/releng/job/autorelease-release-oxygen/455/ >>> >>> Please review and provide an ETA on when a fix will be available. >>> >>> Thanks, >>> ODL releng/autorelease team >>> >>> ___ >>> release mailing list >>> rele...@lists.opendaylight.or
Re: [openflowplugin-dev] [release] Autorelease oxygen failed to build openflow-protocol-it from openflowplugin
Anil, did you guys change anything? We get the following timeout exception: --- T E S T S --- Running org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.949 sec <<< FAILURE! - in org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest testTlsHandshake(org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest) Time elapsed: 3.793 sec <<< ERROR! java.util.concurrent.TimeoutException: Waited 2000 milliseconds for com.google.common.util.concurrent.SettableFuture@242ff747[status=PENDING] at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:470) at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:92) at org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest.createAndStartClient(IntegrationTest.java:255) at org.opendaylight.openflowjava.protocol.it.integration.IntegrationTest.testTlsHandshake(IntegrationTest.java:134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) Results : Tests in error: IntegrationTest.testTlsHandshake:134->createAndStartClient:255 » Timeout Waite... Tests run: 6, Failures: 0, Errors: 1, Skipped: 0 Thanks. On Wed, Oct 17, 2018 at 6:57 AM Jenkins wrote: > Attention openflowplugin-devs, > > Autorelease oxygen failed to build openflow-protocol-it from > openflowplugin in build > 455. Attached is a snippet of the error message related to the > failure that we were able to automatically parse as well as console logs. > > > Console Logs: > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-oxygen/455 > > Jenkins Build: > https://jenkins.opendaylight.org/releng/job/autorelease-release-oxygen/455/ > > Please review and provide an ETA on when a fix will be available. > > Thanks, > ODL releng/autorelease team > > ___ > release mailing list > rele...@lists.opendaylight.org > https://lists.opendaylight.org/mailman/listinfo/release > ___ openflowplugin-dev mailing list openflowplugin-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
