Re: [openflowplugin-dev] Test feedback

[Tomáš Slušný]

Thank you for explanation Luis. I missed step 3 when I was going through test 
report. At least now I know where to search for issue and after this last one 
will be fixed, single layer model implementation will be finally finished, 
because other than this it is not causing any more regressions both with flag 
turned off and on, at least CSIT tests are not reporting any more regressions.

Regards,
Tomas

Od: Luis Gomez [ece...@gmail.com]
Odoslané: 16. februára 2017 18:51
Komu: Tomáš Slušný
Kópia: openflowplugin-dev
Predmet: Re: [openflowplugin-dev] Test feedback

Hi Tomas,

The reconciliation test does the following:

0) Enable stale-flow entry feature (otherwise step 5 will not work)
1) add 2000 groups + 1000 flows (+10 table-miss)
2) Disconnect network
3) Remove 20 groups + 10 flows
4) Reconnect network
5) Check groups and flows are deleted from switch (total: 1980 groups + 1000 
flows)

So what happens with your patch is that the flows (10) are being removed but 
the groups (20) are not, so somehow your patch creates regression on the 
stale-flow entry feature.

BR/Luis

> On Feb 16, 2017, at 6:57 AM, Tomáš Slušný  wrote:
>
> Hi again Luis,
>
> I managed to get single layer serialization to pretty stable state, but I 
> have 1 quick question, and hopefully you will know the answer.
>
> According to test result from flow-services CSIT here: 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/512/archives/log.html.gz
> my patch is causing 2 regressions, and that is not correct count of groups 
> after mininet and controller restart. If I understand this right, in that 
> test, we are first sending
> group 1 100 times to each of 10 switches, and then group 2 100 times to each 
> of 10 switches, so that in total makes 2000 groups that should be present. 
> And here is my question.
> The CSIT tests is checking that 1980 groups should be present, and it was 
> passing before, but on my changes, I have all 2000 groups present. So, is 
> there reason, why we are
> expecting there to have 20 groups missing?
>
> Regards.
> Tomas
>
> 
> Od: Tomáš Slušný [tomas.slu...@pantheon.tech]
> Odoslané: 12. februára 2017 10:39
> Komu: Luis Gomez
> Kópia: openflowplugin-dev
> Predmet: Re: [openflowplugin-dev] Test feedback
>
> Thank you for testing it Luis.
>
> 1) I sort of expected this, because it was a lot of code and there was a lot 
> possibilities for mistakes. I need to double check all serializers and
>deserializers again.
> 2) Thank you for pointing this out, I missed the config parameter that tells 
> to skip collecting table features when rewriting device initialization
>logic.
> 
> Od: Luis Gomez [ece...@gmail.com]
> Odoslané: 11. februára 2017 2:48
> Komu: Tomáš Slušný
> Kópia: openflowplugin-dev
> Predmet: Re: [openflowplugin-dev] Test feedback
>
> Tomas,
>
> 1) New path enabled looks far from ready, I see too many regressions: lack of 
> switch info in inventory, flows not being pushed, stats not reported, among 
> other things.
>
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/494/archives/log.html.gz
>
> 2) With path disabled I see you are bringing all the table features back so 
> its takes very long to update inventory, topology and register the connection:
>
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/492/archives/log.html.gz#s1-s1-s1
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/480/archives/log.html.gz#s1-s1
>
> Please let me know if I can help with anything else.
>
> On Feb 9, 2017, at 11:35 AM, Tomáš Slušný 
> > wrote:
>
> Hi Luis,
>
> this is gerrit link for model cleanup (serialization of openflowplugin models 
> directly) https://git.opendaylight.org/gerrit/#/c/51304/.
> It needs to be tested with new path disabled for regressions, and also with 
> new path enabled.
> New path can be enabled/disabled with use-single-layer serialization flag in 
> openflow-plugin-provider-config YANG:
> https://git.opendaylight.org/gerrit/#/c/50819/8/openflowplugin-api/src/main/yang/openflow-provider-config.yang
>
> Regards,
> Tomas Slusny
>
> 
> Od: Luis Gomez [ece...@gmail.com]
> Odoslané: 9. februára 2017 19:28
> Komu: openflowplugin-dev
> Predmet: [openflowplugin-dev] Test feedback
>
> Hi all,
>
> Infrastructure seems to be recovered from last days issues, please send me 
> the gerrit # that require full system test verification.
>
> FYI I already tried the switch connection patch [1] and got some failures in 
> the entity owner test [2] I have to investigate further and come back to you.
>
> Also as I commented this morning, lately I see a very 

Re: [openflowplugin-dev] opendaylight [ Lithuim ] crashes due to "java.lang.OutOfMemoryError: unable to create new native thread"

[Muthukumaran K]

Hi Santosh,

Couple of questions  -

a)  Does crash happen instantaneously – I mean within shorter intervals or 
degradation is progressive over time ?

b) since you emulate the flaps, is it possible to control the frequency of 
flapping with your emulation ?

If (a) holds true or (b) is possible in your emulation, trend of threads 
created can be captured using tools like VisualVM or even from JConsole to find 
if there is a progressively increasing thread-creation happening for this 
scenario

Btw, I assume you are using only single ODL controller and not a cluster – 
right ?

Regards
Muthu





From: openflowplugin-dev-boun...@lists.opendaylight.org 
[mailto:openflowplugin-dev-boun...@lists.opendaylight.org] On Behalf Of Santosh 
Singh
Sent: Friday, February 17, 2017 1:08 PM
To: Anil Vishnoi 
Cc: openflowplugin-dev@lists.opendaylight.org
Subject: Re: [openflowplugin-dev] opendaylight [ Lithuim ] crashes due to 
"java.lang.OutOfMemoryError: unable to create new native thread"

Hi Anil ,

Thanks for your response ..

We are running with following memory option ,  I think which is sufficient  for 
ODL instance having 150 OF connection.

-Xms128M -Xmx31393m -XX:MaxPermSize=15696m

Any thoughts on this ??

We would be trying to recreate this issue in order to get heap dump 

Thanks
Santosh

On Fri, Feb 17, 2017 at 12:53 PM, Anil Vishnoi 
> wrote:
Hi Santosh,

Looks like your controller crashed while spawning a new native JVM thread, 
because your JVM Is out of native heap space.

Can you increase your native heap space and see if you still hit the issue (it 
might take longer to recreate the issue). Meanwhile if you have the heapdump, 
please upload the heapdump, that will help in analyzing the possible cause.

Thanks
Anil

On Thu, Feb 16, 2017 at 11:05 PM, Santosh Singh 
> wrote:
Hello openflowplugin developers ,

I have been using lithium release of opendaylight.  We are seeing ODL crashes 
with error mentioned in the subject line , when we test the scenario of 
frequent  connection  flap .

If this issue  has been already addressed as part of the latest release , could 
anyone point to the  corresponding bug.

I have pasted the complete stack trace at the below of this mail..

Thanks
Santosh



2017-02-12 22:19:15,360 | ERROR | lt-dispatcher-27 | ActorSystemImpl | 156 - 
com.typesafe.akka.slf4j - 2.3.10 | Uncaught error from thread 
[opendaylight-cluster-data-akka.actor.default-dispatcher-4] shutting down JVM 
since 'akka.jvm-exit-on-fatal-error' is enabled
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)[:1.7.0_95]
at java.lang.Thread.start(Thread.java:714)[:1.7.0_95]
at 
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)[:1.7.0_95]
at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)[:1.7.0_95]
at 
java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:628)[:1.7.0_95]
at 
com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:550)[51:com.google.guava:18.0.0]
at 
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)[:1.7.0_95]
at 
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)[51:com.google.guava:18.0.0]
at 
org.opendaylight.openflowplugin.impl.services.SalRoleServiceImpl.setRole(SalRoleServiceImpl.java:109)
at 
org.opendaylight.openflowplugin.impl.role.RoleContextImpl.onRoleChanged(RoleContextImpl.java:110)
at 
org.opendaylight.openflowplugin.impl.role.OpenflowOwnershipListener.ownershipChanged(OpenflowOwnershipListener.java:62)
at 
org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerActor.onEntityOwnershipChanged(EntityOwnershipListenerActor.java:44)[170:org.opendaylight.controller.sal-distributed-datastore:1.2.4.SNAPSHOT]
at 
org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerActor.handleReceive(EntityOwnershipListenerActor.java:36)[170:org.opendaylight.controller.sal-distributed-datastore:1.2.4.SNAPSHOT]
at 
org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:34)[162:org.opendaylight.controller.sal-clustering-commons:1.2.4.SNAPSHOT]
at 
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)[155:com.typesafe.akka.actor:2.3.10]
at 
akka.actor.Actor$class.aroundReceive(Actor.scala:467)[155:com.typesafe.akka.actor:2.3.10]

___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org

Re: [openflowplugin-dev] opendaylight [ Lithuim ] crashes due to "java.lang.OutOfMemoryError: unable to create new native thread"

[Santosh Singh]

Hi Anil ,

Thanks for your response ..

We are running with following memory option ,  I think which is sufficient
for ODL instance having 150 OF connection.

-Xms128M -Xmx31393m -XX:MaxPermSize=15696m

Any thoughts on this ??

We would be trying to recreate this issue in order to get heap dump 

Thanks
Santosh

On Fri, Feb 17, 2017 at 12:53 PM, Anil Vishnoi 
wrote:

> Hi Santosh,
>
> Looks like your controller crashed while spawning a new native JVM thread,
> because your JVM Is out of native heap space.
>
> Can you increase your native heap space and see if you still hit the issue
> (it might take longer to recreate the issue). Meanwhile if you have the
> heapdump, please upload the heapdump, that will help in analyzing the
> possible cause.
>
> Thanks
> Anil
>
> On Thu, Feb 16, 2017 at 11:05 PM, Santosh Singh  > wrote:
>
>> Hello openflowplugin developers ,
>>
>> I have been using lithium release of opendaylight.  We are seeing ODL
>> crashes with error mentioned in the subject line , when we test the
>> scenario of frequent  connection  flap .
>>
>> If this issue  has been already addressed as part of the latest release ,
>> could anyone point to the  corresponding bug.
>>
>> I have pasted the complete stack trace at the below of this mail..
>>
>> Thanks
>> Santosh
>>
>>
>>
>> 2017-02-12 22:19:15,360 | ERROR | lt-dispatcher-27 | ActorSystemImpl |
>> 156 - com.typesafe.akka.slf4j - 2.3.10 | Uncaught error from thread
>> [opendaylight-cluster-data-akka.actor.default-dispatcher-4] shutting
>> down JVM since 'akka.jvm-exit-on-fatal-error' is enabled
>> java.lang.OutOfMemoryError: unable to create new native thread
>> at java.lang.Thread.start0(Native Method)[:1.7.0_95]
>> at java.lang.Thread.start(Thread.java:714)[:1.7.0_95]
>> at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPool
>> Executor.java:949)[:1.7.0_95]
>> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolEx
>> ecutor.java:1360)[:1.7.0_95]
>> at java.util.concurrent.Executors$DelegatedExecutorService.exec
>> ute(Executors.java:628)[:1.7.0_95]
>> at com.google.common.util.concurrent.MoreExecutors$ListeningDec
>> orator.execute(MoreExecutors.java:550)[51:com.google.guava:18.0.0]
>> at java.util.concurrent.AbstractExecutorService.submit(Abstract
>> ExecutorService.java:132)[:1.7.0_95]
>> at com.google.common.util.concurrent.AbstractListeningExecutorS
>> ervice.submit(AbstractListeningExecutorService.java:58)[51:
>> com.google.guava:18.0.0]
>> at org.opendaylight.openflowplugin.impl.services.SalRoleService
>> Impl.setRole(SalRoleServiceImpl.java:109)
>> at org.opendaylight.openflowplugin.impl.role.RoleContextImpl.
>> onRoleChanged(RoleContextImpl.java:110)
>> at org.opendaylight.openflowplugin.impl.role.OpenflowOwnershipL
>> istener.ownershipChanged(OpenflowOwnershipListener.java:62)
>> at org.opendaylight.controller.cluster.datastore.entityownershi
>> p.EntityOwnershipListenerActor.onEntityOwnershipChanged(Enti
>> tyOwnershipListenerActor.java:44)[170:org.opendaylight.contr
>> oller.sal-distributed-datastore:1.2.4.SNAPSHOT]
>> at org.opendaylight.controller.cluster.datastore.entityownershi
>> p.EntityOwnershipListenerActor.handleReceive(EntityOwnership
>> ListenerActor.java:36)[170:org.opendaylight.controller.
>> sal-distributed-datastore:1.2.4.SNAPSHOT]
>> at org.opendaylight.controller.cluster.common.actor.AbstractUnt
>> ypedActor.onReceive(AbstractUntypedActor.java:34)[162:org.
>> opendaylight.controller.sal-clustering-commons:1.2.4.SNAPSHOT]
>> at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(Untyp
>> edActor.scala:167)[155:com.typesafe.akka.actor:2.3.10]
>> at akka.actor.Actor$class.aroundReceive(Actor.scala:467)[155:
>> com.typesafe.akka.actor:2.3.10]
>>
>> ___
>> openflowplugin-dev mailing list
>> openflowplugin-dev@lists.opendaylight.org
>> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
>>
>>
>
>
> --
> Thanks
> Anil
>
___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

Re: [openflowplugin-dev] opendaylight [ Lithuim ] crashes due to "java.lang.OutOfMemoryError: unable to create new native thread"

[Anil Vishnoi]

Hi Santosh,

Looks like your controller crashed while spawning a new native JVM thread,
because your JVM Is out of native heap space.

Can you increase your native heap space and see if you still hit the issue
(it might take longer to recreate the issue). Meanwhile if you have the
heapdump, please upload the heapdump, that will help in analyzing the
possible cause.

Thanks
Anil

On Thu, Feb 16, 2017 at 11:05 PM, Santosh Singh 
wrote:

> Hello openflowplugin developers ,
>
> I have been using lithium release of opendaylight.  We are seeing ODL
> crashes with error mentioned in the subject line , when we test the
> scenario of frequent  connection  flap .
>
> If this issue  has been already addressed as part of the latest release ,
> could anyone point to the  corresponding bug.
>
> I have pasted the complete stack trace at the below of this mail..
>
> Thanks
> Santosh
>
>
>
> 2017-02-12 22:19:15,360 | ERROR | lt-dispatcher-27 | ActorSystemImpl | 156
> - com.typesafe.akka.slf4j - 2.3.10 | Uncaught error from thread
> [opendaylight-cluster-data-akka.actor.default-dispatcher-4] shutting down
> JVM since 'akka.jvm-exit-on-fatal-error' is enabled
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)[:1.7.0_95]
> at java.lang.Thread.start(Thread.java:714)[:1.7.0_95]
> at java.util.concurrent.ThreadPoolExecutor.addWorker(
> ThreadPoolExecutor.java:949)[:1.7.0_95]
> at java.util.concurrent.ThreadPoolExecutor.execute(
> ThreadPoolExecutor.java:1360)[:1.7.0_95]
> at java.util.concurrent.Executors$DelegatedExecutorService.
> execute(Executors.java:628)[:1.7.0_95]
> at com.google.common.util.concurrent.MoreExecutors$
> ListeningDecorator.execute(MoreExecutors.java:550)[51:
> com.google.guava:18.0.0]
> at java.util.concurrent.AbstractExecutorService.submit(
> AbstractExecutorService.java:132)[:1.7.0_95]
> at com.google.common.util.concurrent.AbstractListeningExecutorServi
> ce.submit(AbstractListeningExecutorService.java:58)[51:com.google.
> guava:18.0.0]
> at org.opendaylight.openflowplugin.impl.services.
> SalRoleServiceImpl.setRole(SalRoleServiceImpl.java:109)
> at org.opendaylight.openflowplugin.impl.role.
> RoleContextImpl.onRoleChanged(RoleContextImpl.java:110)
> at org.opendaylight.openflowplugin.impl.role.OpenflowOwnershipListener.
> ownershipChanged(OpenflowOwnershipListener.java:62)
> at org.opendaylight.controller.cluster.datastore.entityownership.
> EntityOwnershipListenerActor.onEntityOwnershipChanged(
> EntityOwnershipListenerActor.java:44)[170:org.opendaylight.
> controller.sal-distributed-datastore:1.2.4.SNAPSHOT]
> at org.opendaylight.controller.cluster.datastore.entityownership.
> EntityOwnershipListenerActor.handleReceive(EntityOwnershipListenerActor.
> java:36)[170:org.opendaylight.controller.sal-distributed-
> datastore:1.2.4.SNAPSHOT]
> at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.
> onReceive(AbstractUntypedActor.java:34)[162:org.opendaylight.
> controller.sal-clustering-commons:1.2.4.SNAPSHOT]
> at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(
> UntypedActor.scala:167)[155:com.typesafe.akka.actor:2.3.10]
> at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
> [155:com.typesafe.akka.actor:2.3.10]
>
> ___
> openflowplugin-dev mailing list
> openflowplugin-dev@lists.opendaylight.org
> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
>
>


-- 
Thanks
Anil
___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

[openflowplugin-dev] opendaylight [ Lithuim ] crashes due to "java.lang.OutOfMemoryError: unable to create new native thread"

[Santosh Singh]

Hello openflowplugin developers ,

I have been using lithium release of opendaylight.  We are seeing ODL
crashes with error mentioned in the subject line , when we test the
scenario of frequent  connection  flap .

If this issue  has been already addressed as part of the latest release ,
could anyone point to the  corresponding bug.

I have pasted the complete stack trace at the below of this mail..

Thanks
Santosh



2017-02-12 22:19:15,360 | ERROR | lt-dispatcher-27 | ActorSystemImpl | 156
- com.typesafe.akka.slf4j - 2.3.10 | Uncaught error from thread
[opendaylight-cluster-data-akka.actor.default-dispatcher-4] shutting down
JVM since 'akka.jvm-exit-on-fatal-error' is enabled
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)[:1.7.0_95]
at java.lang.Thread.start(Thread.java:714)[:1.7.0_95]
at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
[:1.7.0_95]
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
[:1.7.0_95]
at
java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:628)
[:1.7.0_95]
at
com.google.common.util.concurrent.MoreExecutors$ListeningDecorator.execute(MoreExecutors.java:550)
[51:com.google.guava:18.0.0]
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
[:1.7.0_95]
at
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)
[51:com.google.guava:18.0.0]
at
org.opendaylight.openflowplugin.impl.services.SalRoleServiceImpl.setRole(SalRoleServiceImpl.java:109)
at
org.opendaylight.openflowplugin.impl.role.RoleContextImpl.onRoleChanged(RoleContextImpl.java:110)
at
org.opendaylight.openflowplugin.impl.role.OpenflowOwnershipListener.ownershipChanged(OpenflowOwnershipListener.java:62)
at
org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerActor.onEntityOwnershipChanged(EntityOwnershipListenerActor.java:44)
[170:org.opendaylight.controller.sal-distributed-datastore:1.2.4.SNAPSHOT]
at
org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerActor.handleReceive(EntityOwnershipListenerActor.java:36)
[170:org.opendaylight.controller.sal-distributed-datastore:1.2.4.SNAPSHOT]
at
org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:34)
[162:org.opendaylight.controller.sal-clustering-commons:1.2.4.SNAPSHOT]
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
[155:com.typesafe.akka.actor:2.3.10]
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
[155:com.typesafe.akka.actor:2.3.10]
___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

[openflowplugin-dev] OpenFlow Plugin M4 Status

[Abhijit Kumbhare]

OpenFlow Plugin

1. Please provide updates on any previously-incomplete items from prior
milestone readouts. N/A


2. Has your project achieved API freeze such that all externally accessible
Stable or Provisional APIs will not be modified after now? Yes

https://git.opendaylight.org/gerrit/gitweb?p=openflowplugin.git;a=tree;f=
openflowplugin-api/src/main;hb=HEAD


3. Do you have content in your project documentation? Yes

*  word count: User guide: ~5000, developer guide: ~800

*  (For each document, link to the file in gerrit)

https://git.opendaylight.org/gerrit/gitweb?p=docs.git;a=
tree;f=manuals/developer-guide/src/main/asciidoc/openflowplugin;h=
c5100a7985545df76d19caf868a9597427b0152d;hb=refs/heads/master

https://git.opendaylight.org/gerrit/gitweb?p=docs.git;a=
tree;f=manuals/user-guide/src/main/asciidoc/openflowplugin;h=
a44c3c03babc92c6cf1668442e4b6fcf06b7cf6d;hb=refs/heads/master


4. Has your project met the requirements to be included in Maven Central
[2]? (Yes/No)

Yes



5. Were project-specific deliverables planned for this milestone delivered
successfully? Yes (caveat: documentation cleanup not done due to change in
employment of a key member)



6. Have you started automated system testing for your top-level features.
(Yes/No)

Yes

*  (If yes, link to test report)

Link: https://git.opendaylight.org/gerrit/gitweb?p=integration/test.git;
a=tree;f=csit/suites/openflowplugin;h=aecbb801bcb364a5b9f3d8e0624231
b2d835a572;hb=HEAD

*  (If no, explain why)



7. Does your project use any ports, including for testing? (Yes/No)

Yes

*  (If yes, list of ports used)

6633, 6653

*  (If yes, have you updated the wiki [3] with all ports used? Yes/No)

Yes



8. Does your project build successful in Autorelease?

Yes (with occassional failures)
___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

Re: [openflowplugin-dev] Test feedback

[Luis Gomez]

Hi Tomas,

The reconciliation test does the following:

0) Enable stale-flow entry feature (otherwise step 5 will not work) 
1) add 2000 groups + 1000 flows (+10 table-miss)
2) Disconnect network
3) Remove 20 groups + 10 flows
4) Reconnect network
5) Check groups and flows are deleted from switch (total: 1980 groups + 1000 
flows)

So what happens with your patch is that the flows (10) are being removed but 
the groups (20) are not, so somehow your patch creates regression on the 
stale-flow entry feature.

BR/Luis

> On Feb 16, 2017, at 6:57 AM, Tomáš Slušný  wrote:
> 
> Hi again Luis,
> 
> I managed to get single layer serialization to pretty stable state, but I 
> have 1 quick question, and hopefully you will know the answer.
> 
> According to test result from flow-services CSIT here: 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/512/archives/log.html.gz
> my patch is causing 2 regressions, and that is not correct count of groups 
> after mininet and controller restart. If I understand this right, in that 
> test, we are first sending
> group 1 100 times to each of 10 switches, and then group 2 100 times to each 
> of 10 switches, so that in total makes 2000 groups that should be present. 
> And here is my question.
> The CSIT tests is checking that 1980 groups should be present, and it was 
> passing before, but on my changes, I have all 2000 groups present. So, is 
> there reason, why we are
> expecting there to have 20 groups missing?
> 
> Regards.
> Tomas
> 
> 
> Od: Tomáš Slušný [tomas.slu...@pantheon.tech]
> Odoslané: 12. februára 2017 10:39
> Komu: Luis Gomez
> Kópia: openflowplugin-dev
> Predmet: Re: [openflowplugin-dev] Test feedback
> 
> Thank you for testing it Luis.
> 
> 1) I sort of expected this, because it was a lot of code and there was a lot 
> possibilities for mistakes. I need to double check all serializers and
>deserializers again.
> 2) Thank you for pointing this out, I missed the config parameter that tells 
> to skip collecting table features when rewriting device initialization
>logic.
> 
> Od: Luis Gomez [ece...@gmail.com]
> Odoslané: 11. februára 2017 2:48
> Komu: Tomáš Slušný
> Kópia: openflowplugin-dev
> Predmet: Re: [openflowplugin-dev] Test feedback
> 
> Tomas,
> 
> 1) New path enabled looks far from ready, I see too many regressions: lack of 
> switch info in inventory, flows not being pushed, stats not reported, among 
> other things.
> 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/494/archives/log.html.gz
> 
> 2) With path disabled I see you are bringing all the table features back so 
> its takes very long to update inventory, topology and register the connection:
> 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/492/archives/log.html.gz#s1-s1-s1
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/480/archives/log.html.gz#s1-s1
> 
> Please let me know if I can help with anything else.
> 
> On Feb 9, 2017, at 11:35 AM, Tomáš Slušný 
> > wrote:
> 
> Hi Luis,
> 
> this is gerrit link for model cleanup (serialization of openflowplugin models 
> directly) https://git.opendaylight.org/gerrit/#/c/51304/.
> It needs to be tested with new path disabled for regressions, and also with 
> new path enabled.
> New path can be enabled/disabled with use-single-layer serialization flag in 
> openflow-plugin-provider-config YANG:
> https://git.opendaylight.org/gerrit/#/c/50819/8/openflowplugin-api/src/main/yang/openflow-provider-config.yang
> 
> Regards,
> Tomas Slusny
> 
> 
> Od: Luis Gomez [ece...@gmail.com]
> Odoslané: 9. februára 2017 19:28
> Komu: openflowplugin-dev
> Predmet: [openflowplugin-dev] Test feedback
> 
> Hi all,
> 
> Infrastructure seems to be recovered from last days issues, please send me 
> the gerrit # that require full system test verification.
> 
> FYI I already tried the switch connection patch [1] and got some failures in 
> the entity owner test [2] I have to investigate further and come back to you.
> 
> Also as I commented this morning, lately I see a very often issue of switch 
> not getting table miss flow when it connects to 3 cluster nodes [3]. I will 
> try to reproduce issue and open bug.
> 
> BR/Luis
> 
> [1] https://git.opendaylight.org/gerrit/#/c/51500
> [2] 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/474/archives/log.html.gz
> [3] 
> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/469/archives/log.html.gz
> 
> ___
> openflowplugin-dev mailing list
> 

Re: [openflowplugin-dev] Test feedback

[Tomáš Slušný]

Hi again Luis,

I managed to get single layer serialization to pretty stable state, but I have 
1 quick question, and hopefully you will know the answer.

According to test result from flow-services CSIT here: 
https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/512/archives/log.html.gz
my patch is causing 2 regressions, and that is not correct count of groups 
after mininet and controller restart. If I understand this right, in that test, 
we are first sending
group 1 100 times to each of 10 switches, and then group 2 100 times to each of 
10 switches, so that in total makes 2000 groups that should be present. And 
here is my question.
The CSIT tests is checking that 1980 groups should be present, and it was 
passing before, but on my changes, I have all 2000 groups present. So, is there 
reason, why we are
expecting there to have 20 groups missing?

Regards.
Tomas


Od: Tomáš Slušný [tomas.slu...@pantheon.tech]
Odoslané: 12. februára 2017 10:39
Komu: Luis Gomez
Kópia: openflowplugin-dev
Predmet: Re: [openflowplugin-dev] Test feedback

Thank you for testing it Luis.

1) I sort of expected this, because it was a lot of code and there was a lot 
possibilities for mistakes. I need to double check all serializers and
deserializers again.
2) Thank you for pointing this out, I missed the config parameter that tells to 
skip collecting table features when rewriting device initialization
logic.

Od: Luis Gomez [ece...@gmail.com]
Odoslané: 11. februára 2017 2:48
Komu: Tomáš Slušný
Kópia: openflowplugin-dev
Predmet: Re: [openflowplugin-dev] Test feedback

Tomas,

1) New path enabled looks far from ready, I see too many regressions: lack of 
switch info in inventory, flows not being pushed, stats not reported, among 
other things.

https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/494/archives/log.html.gz

2) With path disabled I see you are bringing all the table features back so its 
takes very long to update inventory, topology and register the connection:

https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-1node-flow-services-only-carbon/492/archives/log.html.gz#s1-s1-s1
https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/480/archives/log.html.gz#s1-s1

Please let me know if I can help with anything else.

On Feb 9, 2017, at 11:35 AM, Tomáš Slušný 
> wrote:

Hi Luis,

this is gerrit link for model cleanup (serialization of openflowplugin models 
directly) https://git.opendaylight.org/gerrit/#/c/51304/.
It needs to be tested with new path disabled for regressions, and also with new 
path enabled.
New path can be enabled/disabled with use-single-layer serialization flag in 
openflow-plugin-provider-config YANG:
https://git.opendaylight.org/gerrit/#/c/50819/8/openflowplugin-api/src/main/yang/openflow-provider-config.yang

Regards,
Tomas Slusny


Od: Luis Gomez [ece...@gmail.com]
Odoslané: 9. februára 2017 19:28
Komu: openflowplugin-dev
Predmet: [openflowplugin-dev] Test feedback

Hi all,

Infrastructure seems to be recovered from last days issues, please send me the 
gerrit # that require full system test verification.

FYI I already tried the switch connection patch [1] and got some failures in 
the entity owner test [2] I have to investigate further and come back to you.

Also as I commented this morning, lately I see a very often issue of switch not 
getting table miss flow when it connects to 3 cluster nodes [3]. I will try to 
reproduce issue and open bug.

BR/Luis

[1] https://git.opendaylight.org/gerrit/#/c/51500
[2] 
https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/474/archives/log.html.gz
[3] 
https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3node-clustering-only-carbon/469/archives/log.html.gz

___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

TomášSlušný
Software Developer

Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R centrum / Janka Kráľa 9 /  974 01 Banská Bystrica / Slovakia
+421 911 083 902 / tomas.slu...@pantheon.tech
reception: +421 2 206 65 114 / www.pantheon.tech

[logo]




TomášSlušný
Software Developer

Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R centrum / Janka Kráľa 9 /  974 01 Banská Bystrica / Slovakia
+421 911 083 902 / tomas.slu...@pantheon.tech
reception: +421 2 206 65 114 / www.pantheon.tech

[logo]


___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

Re: [openflowplugin-dev] [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind the value

[Michal Rehak -X (mirehak - PANTHEON TECHNOLOGIES at Cisco)]

Hi Muthukrishnan,

there is no golden standard behind this value. Setting it to 100ms or even 
lower value will result into low latency system but this also decreases 
throughput of system.


If you have few switches and little no big requirement on flows per second 
throughput then 100ms might suit better to your setup. On the other hand if 
there are hundreds of devices and throughput of thousands of flows per second 
required then 500ms or even higher will improve overall performance.


Also there is a complementary parameter - amount of messages in outbound queue 
without barrier inside. The more messages kept in queue the higher throughput 
you shall get. But there is memory cost and also by extending size of those 
queues (per device) there comes overload risk of md-sal notification queue.


True - this value might be configurable. But that would solve the situation 
only for static requirements.



Regards,

Michal




From: openflowjava-dev-boun...@lists.opendaylight.org 
 on behalf of Muthukrishnan 
Thangasamy 
Sent: Wednesday, February 15, 2017 08:13
To: openflowplugin-dev@lists.opendaylight.org
Cc: openflowjava-...@lists.opendaylight.org
Subject: [openflowjava-dev] Barrier Message Timeout - 500ms - Rational behind 
the value



Dear Team ,


I have been working in Lithium and Boron Stable Version .


Lets take Boron , In openflowplugin.cfg default timer for barrier message is 
500ms is there any

rational behind the same ? why not 100ms ?


If I push via SAL-FLOW RPC , total time taken for flow push = Time taken for 
flow push + barrier delay  = ~ 30 ms + 520 ms (barrier ).


If I change the value from 500ms to 100 ms I am getting drastic performance 
improvement in flow push processing . Can anyone tell why we have gone for 
500ms as default timer value ?

How we have arrived this golden value of 500ms ?



Reference.


File : openflowplugin.cfg

"#Timeout interval in milliseconds between each barrier message.

#Default value is set to 500 milliseconds
barrier-interval-timeout-limit=500"

Regards
Muthukrishnan
9952012433

___
openflowplugin-dev mailing list
openflowplugin-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev

[openflowplugin-dev] 回复： Singleton Clustering issue

[guo]

Hi Anil,


Why is it happening in Issue 1?
"and then you disconnect the device from second controller and reconnect it, 
ownership goes to third controller"


I found that when disconnect the device from the second controller, the device 
data in data store will be deleted. So the FRM will deregister the service 
instance on the third controller, so the ownership goes to the first controller.


guo


-- 原始邮件 --
发件人: "Anil Vishnoi";;
发送时间: 2017年2月16日(星期四) 凌晨4:32
收件人: "Jozef Bacigál"; 
抄送: 
"openflowplugin-dev@lists.opendaylight.org";
 
主题: Re: [openflowplugin-dev] Singleton Clustering issue



Hi Jozef,


I think this does not solve the issue, it actually will make sure that node 
deleted first and then added after that, so that user can see the node. But 
this delete and add, will create two data change notification for the 
application and will give a impression that device was disconnected and 
connected back, which is not really a case. I think the ideal solution as you 
mentioned is if clustering service provide a notification saying the device has 
no owner, so that it can clean-up. I think we should raise a bug to the 
clustering team to provide this kind of API, so that we can use this to give a 
proper solution.


On Tue, Feb 14, 2017 at 12:54 AM, Jozef Bacigál  
wrote:
   
HI Anil, guys
 
 
 
I am facing the same issue you are mentioned in Issue 2 with my single layer 
implementation. The plugin is not able to know if there is another controller 
connected  to the switch so the only one and not good, even slow solution 
is/were (I am using right now) that if we lose mastership we are deleting node 
from DS and HOPE that is sooner than new master will write new node into DS. 
The best solution were to have the information  if this was the last master in 
cluster for the switch. And then and only then delete the node from DS. What I 
am trying right know to hold status before the node is deleted from DS and then 
send the ImmediateFuture back to mdsal singleton, so the new master  can be 
elected.
 
 
 
Anyway it is very bad implementation FOR plugin from singleton service.
 
 
 
Jozef
 
 
 
From: Anil Vishnoi [mailto:vishnoia...@gmail.com] 
 Sent: Tuesday, February 14, 2017 4:37 AM
 To: Jozef Bacigál ; Abhijit Kumbhare 
; Tomáš Slušný ; Shuva Jyoti 
Kar ; Luis Gomez ; Muthukumaran 
K 
 Cc: openflowplugin-dev@lists.opendaylight.org
 Subject: Singleton Clustering issue
 
 
   
Hi Jozef/Tomas/Luis,
 
  
 
 
  
I was investigating Bug 7736 and came across few issue in our clustering 
implementation and  also some limitation with singleton clustering as well.
 
  
 
 
  
Issue 1 : Registering application on data change notification.
 
  
In the current implementation, when plugin receives the connection from device, 
it register itself as a service instance to clustering singleton service. After 
registering with clustering service, it receives the notification to initialize 
 the instance. It then try to set the master role to the device and then write 
the device data to the data store. Forwarding-Rule-Manager then listen on the 
data store notification and whenever it see that node is added to the data 
store, it registers itself  as a service instance for that node. Given that we 
are using ClusteredDataTreeChangeListener, all the FRM instances get the node 
added notification from data store and all the cluster nodes end up registering 
themselves as a service instance on the same service  identifier. So even if 
device is connected to only one controller FRM register itself on all the three 
nodes, that's not correct behavior. So this bug can cause a issue where 
openflowplugin cluster will be almost unusable. We have seen an issue where if 
you  connect the device to two controllers and disconnect the device from first 
controller and connect it back, ownership goes to second controller where 
device is also connected, and then you disconnect the device from second 
controller and reconnect it, ownership  goes to third controller, but given 
that now ownership for that service identity is with controller 3, even if 
device connect back to controller1/2, those controller don't push the master 
role down. And this scenario can occur trigger the moment your device  
disconnect from any of the controller.
 
  
 
 
  
Now problem is that for applications there is no way to find out if the device 
is connected to it's host controller instance (until and unless we write some 
hardcoded controller number/name in the data store for each device where it's 
connected).  The only way i can see is through the yang notification, where 
plugin can send the nodeAdded/nodeRemoved notification and 

[openflowplugin-dev] 回复： Singleton Clustering issue

[guo]

Hi Anil,


Why is it happening in Issue 1?
"and then you disconnect the device from second controller and reconnect it, 
ownership goes to third controller"


I found that when disconnect the device from the second controller, the device 
data in data store will be deleted. So the FRM will deregister the service 
instance on the third controller, so the ownership goes to the second 
controller.


guo


-- 原始邮件 --
发件人: "Anil Vishnoi";;
发送时间: 2017年2月16日(星期四) 凌晨4:32
收件人: "Jozef Bacigál"; 
抄送: 
"openflowplugin-dev@lists.opendaylight.org";
 
主题: Re: [openflowplugin-dev] Singleton Clustering issue



Hi Jozef,


I think this does not solve the issue, it actually will make sure that node 
deleted first and then added after that, so that user can see the node. But 
this delete and add, will create two data change notification for the 
application and will give a impression that device was disconnected and 
connected back, which is not really a case. I think the ideal solution as you 
mentioned is if clustering service provide a notification saying the device has 
no owner, so that it can clean-up. I think we should raise a bug to the 
clustering team to provide this kind of API, so that we can use this to give a 
proper solution.


On Tue, Feb 14, 2017 at 12:54 AM, Jozef Bacigál  
wrote:
   
HI Anil, guys
 
 
 
I am facing the same issue you are mentioned in Issue 2 with my single layer 
implementation. The plugin is not able to know if there is another controller 
connected  to the switch so the only one and not good, even slow solution 
is/were (I am using right now) that if we lose mastership we are deleting node 
from DS and HOPE that is sooner than new master will write new node into DS. 
The best solution were to have the information  if this was the last master in 
cluster for the switch. And then and only then delete the node from DS. What I 
am trying right know to hold status before the node is deleted from DS and then 
send the ImmediateFuture back to mdsal singleton, so the new master  can be 
elected.
 
 
 
Anyway it is very bad implementation FOR plugin from singleton service.
 
 
 
Jozef
 
 
 
From: Anil Vishnoi [mailto:vishnoia...@gmail.com] 
 Sent: Tuesday, February 14, 2017 4:37 AM
 To: Jozef Bacigál ; Abhijit Kumbhare 
; Tomáš Slušný ; Shuva Jyoti 
Kar ; Luis Gomez ; Muthukumaran 
K 
 Cc: openflowplugin-dev@lists.opendaylight.org
 Subject: Singleton Clustering issue
 
 
   
Hi Jozef/Tomas/Luis,
 
  
 
 
  
I was investigating Bug 7736 and came across few issue in our clustering 
implementation and  also some limitation with singleton clustering as well.
 
  
 
 
  
Issue 1 : Registering application on data change notification.
 
  
In the current implementation, when plugin receives the connection from device, 
it register itself as a service instance to clustering singleton service. After 
registering with clustering service, it receives the notification to initialize 
 the instance. It then try to set the master role to the device and then write 
the device data to the data store. Forwarding-Rule-Manager then listen on the 
data store notification and whenever it see that node is added to the data 
store, it registers itself  as a service instance for that node. Given that we 
are using ClusteredDataTreeChangeListener, all the FRM instances get the node 
added notification from data store and all the cluster nodes end up registering 
themselves as a service instance on the same service  identifier. So even if 
device is connected to only one controller FRM register itself on all the three 
nodes, that's not correct behavior. So this bug can cause a issue where 
openflowplugin cluster will be almost unusable. We have seen an issue where if 
you  connect the device to two controllers and disconnect the device from first 
controller and connect it back, ownership goes to second controller where 
device is also connected, and then you disconnect the device from second 
controller and reconnect it, ownership  goes to third controller, but given 
that now ownership for that service identity is with controller 3, even if 
device connect back to controller1/2, those controller don't push the master 
role down. And this scenario can occur trigger the moment your device  
disconnect from any of the controller.
 
  
 
 
  
Now problem is that for applications there is no way to find out if the device 
is connected to it's host controller instance (until and unless we write some 
hardcoded controller number/name in the data store for each device where it's 
connected).  The only way i can see is through the yang notification, where 
plugin can send the nodeAdded/nodeRemoved notification and