Yes Anil, therfore we came to suspicion that jenkins FRS test uses OVS 2.5.
It would be easier for us to fix 8 failures or so if they are not mixed up with
118/120 failures caused by BADMATCH error.
(2.0 and also 2.3 I think, can deal with it by replacing values in address
according to mask)
Anyway, example of flow from ODL_TEST_PROJECT/csit/variables/xmls/f18.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<flow xmlns="urn:opendaylight:flow:inventory">
<strict>false</strict>
<flow-name>FooXf18</flow-name>
<id>141</id>
<cookie_mask>255</cookie_mask>
<cookie>18</cookie>
<table_id>2</table_id>
<priority>18</priority>
<installHw>false</installHw>
<instructions>
<instruction>
<order>0</order>
<apply-actions>
<action>
<order>0</order>
<dec-nw-ttl/>
</action>
</apply-actions>
</instruction>
</instructions>
<match>
<ethernet-match>
<ethernet-type>
<type>34525</type>
</ethernet-type>
</ethernet-match>
<ipv6-source>fe80::2acf:e9ff:fe21:6431/128</ipv6-source>
<ipv6-destination>aabb:1234:2acf:e9ff::fe21:6431/64</ipv6-destination>
</match>
</flow>
________________________________
From: Anil Vishnoi <[email protected]>
Sent: Wednesday, October 5, 2016 8:10 AM
To: Luis Gomez
Cc: [email protected];
[email protected]; Miroslav Macko
Subject: Re: [integration-dev] [openflowplugin-dev] Failing robot tests
openflowplugin-csit-1node-flow-services
OVS 2.5 throws "Device reported error type BADMATCH code BADWILDCARDS" when
you are passing the wrong ip/mac value with the mask. Like if you pass
192.168.12.11/24, OVS should throw the above error message, but if you using
192.168.12.0/24, it should be fine. can you please paste the flow that is
failing, that probably will give more insight.
Thanks
Anil
On Tue, Oct 4, 2016 at 9:41 PM, Luis Gomez
<[email protected]<mailto:[email protected]>> wrote:
Yeah, the +8 failures in FRS are due to some failure to reconcile oper flow
with config flow (alien flow ID), I was able to reproduce the issue locally but
in my case it fails in not exactly same flows. What is interesting is that if I
try the flows that fail in the suite manually one by one, they show fine in
operational (no alien ID) which makes me think there is some sort of race
condition or interference when we install many flows very fast like this test
does.
On Oct 4, 2016, at 5:00 AM, Miroslav Macko
<[email protected]<mailto:[email protected]>> wrote:
Hi guys,
Luis. We agree with you.
1) Current issues in FRS flow service suite using OVS 2.0 are due to FRS being
slower than FRM.
Yes the FRS is slower then FRM.
2) Many flow templates we use in OVS 2.0 do not work in recent versions like
OVS 2.5. We tried to repair this is in the past but we never got the
resources/time to do it, I hope this release we can do it :)
Great.
But what we wanted to point out is:
- that both, FRM and also FRS is working with OVS vesrsion 2.0.
- and both FRM and FRS is failing, with the same errors, with OVS version 2.3+.
I would like to emphasize, that also FRM is failing.
So this is the part what we would like to solve. And that is why we taught, it
is running on different OVS versions.
Results form local tests:
FRM with OVS version 2.0 - 1 fail
FRM with OVS version 2.4 - 112 fail - logs in the attachment
Flows_Additional_TCs-frm-OVS24
FRS with OVS version 2.0 - 8 fail - logs in the attachment
Flows_Additional_TCs-frs-OVS20
FRS with OVS version 2.4 - 120 fail
@Jamo thank you for checking the virtual machine for both jobs.
@Luis is there something about the jobs that would make ovs have a different
version please?
If it is running with the same OVS version, than it is really strange.
Thanks a lot.
Miro
________________________________
Od: Luis Gomez <[email protected]<mailto:[email protected]>>
Odoslané: 4. októbra 2016 11:30
Komu: Andrej Leitner
Kópia: Jamo Luhrsen; Miroslav Macko;
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Predmet: Re: [integration-dev] [openflowplugin-dev] Failing robot tests
openflowplugin-csit-1node-flow-services
I think we are talking about 2 different things:
1) Current issues in FRS flow service suite using OVS 2.0 are due to FRS being
slower than FRM.
2) Many flow templates we use in OVS 2.0 do not work in recent versions like
OVS 2.5. We tried to repair this is in the past but we never got the
resources/time to do it, I hope this release we can do it :)
BR/Luis
On Oct 4, 2016, at 2:16 AM, Andrej Leitner
<[email protected]<mailto:[email protected]>> wrote:
Hi guys,
we know that FRS is currently slower than FRM, but a week ago we refactored
with Miroslav the robot test locally to sleep more than 1 or 3 seconds
just because we wanted to eliminate the failures caused by perfromance issues.
We tried to run tests for FRM and FRS with OVS versions 2.0/2.3/2.4/2.5
and we see both are failing on same flows with particular ovs versions -
example from todays testing:
flow: odl/test/csit/variables/xmls/f18.xml
log:set DEBUG org.opendaylight.openflowplugin.impl.services.SalFlowServiceImpl
feature:install odl-openflowplugin-flow-services-rest
OVS 2.0
2016-10-04 10:46:58,499 | DEBUG | entLoopGroup-7-2 | SalFlowServiceImpl
| 209 - org.opendaylight.openflowplugin.impl - 0.4.0.SNAPSHOT | Flow add
with id=141 finished without error
OVS 2.5
2016-10-04 10:50:17,148 | DEBUG | entLoopGroup-7-4 | SalFlowServiceImpl
| 209 - org.opendaylight.openflowplugin.impl - 0.4.0.SNAPSHOT | Flow add
failed for flow=AddFlowInput [_cookie=FlowCookie [_value=18],
_cookieMask=FlowCookie [_value=255], _flowName=FooXf18, _flowRef=FlowRef
[_value=KeyedInstanceIdentifier{targetType=interface
...
_strict=false, augmentation=[]], errors=Device reported error type BADMATCH
code BADWILDCARDS
feature:install odl-restconf; feature:install
odl-openflowplugin-app-config-pusher; feature:install
odl-openflowplugin-app-topology; feature:install
odl-openflowplugin-app-forwardingrules-sync;
OVS 2.0
2016-10-04 10:55:02,697 | DEBUG | entLoopGroup-9-5 | SalFlowServiceImpl
| 281 - org.opendaylight.openflowplugin.impl - 0.4.0.SNAPSHOT | Flow add
with id=141 finished without error
OVS 2.5
2016-10-04 10:54:08,438 | DEBUG | entLoopGroup-9-3 | SalFlowServiceImpl
| 281 - org.opendaylight.openflowplugin.impl - 0.4.0.SNAPSHOT | Flow add
failed for flow=AddFlowInput [_cookie=FlowCookie [_value=18],
_cookieMask=FlowCookie [_value=255], _flowName=FooXf18, _flowRef=FlowRef
[_value=KeyedInstanceIdentifier{targetType=interface
...
_priority=18, _tableId=2, _installHw=false, _strict=false, augmentation=[]],
errors=Device reported error type BADMATCH code BADWILDCARDS
Are you sure the problem is not in different ovs versions?
________________________________
From: Luis Gomez <[email protected]<mailto:[email protected]>>
Sent: Monday, October 3, 2016 10:55 PM
To: Jamo Luhrsen; Miroslav Macko
Cc:
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Re: [integration-dev] [openflowplugin-dev] Failing robot tests
openflowplugin-csit-1node-flow-services
So looking at the FRS test in detail, the reason for the multiple failures is
not the OVS version but the FRS feature requiring more time to install flows +
update flow stats in oper DS.
In general it is bad practice to use sleep in tests but in this case it was
useful to flag the slowness of FRS, in particular:
- Flows_OF13 suite is failing because we give 1 sec between programming the
flow and see the flow in OVS, with FRS we need more time and that is why it is
fails.
- Stats_Manager_extended suite is failing because we give 3 secs between
programming flows and see them in operational DS, with FRS we need more time
and thats is why it fails.
- The Groups_Meters_OF13 is also failing because CPqD switch cannot connect to
controller, this could be a CPqD issue, with the old FRM is working but it
takes long time to connect.
Finally you can observe the differences in performance in these jobs:
https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-periodic-scale-stats-collection-daily-frs-only-boron/plot/
https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-periodic-scale-stats-collection-daily-only-boron/plot/
The plot titles are not very good but I can explain:
- Config Performance measures time for updating all flow stats in oper DS after
adding and after deleting.
- Performance rate measures rate and time for getting flows in config DS.
In both cases the FRM is superior than FRS.
BR/Luis
On Oct 3, 2016, at 9:20 AM, Jamo Luhrsen
<[email protected]<mailto:[email protected]>> wrote:
Miro,
both of the jobs you pointed to are using the same VM to run OVS.
"ubuntu-trusty-mininet-2c-2g"
is there something about the jobs that would make ovs have a different version?
As for using ovs 2.5, I recall hearing the plan would be to get all of the
openflowplugin csit
running 2.5+. I think it will create some issues that will need to be debugged
and fixed in our
CSIT.
JamO
On 10/03/2016 06:33 AM, Miroslav Macko wrote:
?Hi guys,
Could you please check this two robot tests [1] and [2]?
[1] is not failing
[2] is failing - some flows are rejected by device ("match inconsistent").
It looks like that [1] is running on the OVS version 2.0 and [2] is probably
running on some higher version of the OVS.
When we tried to use OVS 2.4 for [1], it was also failing with "match
inconsistent".
When we tried to use OVS 2.0 for [2], it is also not failing.
?
We have discussed it with Luis already, and he would like to use OVS version
2.5.0 for all tests.
So it will be needed to update failing flows xml input files
(/test/csit/variables/xmls).
Thank you.
Miro
[1]
https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-flow-services-only-carbon/
[pastedImage.png]
[2]
https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-flow-services-frs-only-carbon/
[pastedImage.png]
MiroslavMacko
Software Developer
Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R&D centrum / Janka Krála 9 / 974 01 Banská Bystrica / Slovakia
/ [email protected]<mailto:[email protected]>
reception: +421 2 206 65 114 / www.pantheon.sk<http://www.pantheon.sk/>
logo
_______________________________________________
openflowplugin-dev mailing list
[email protected]<mailto:[email protected]>
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
_______________________________________________
integration-dev mailing list
[email protected]<mailto:[email protected]>
https://lists.opendaylight.org/mailman/listinfo/integration-dev
AndrejLeitner
Software Developer
Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R&D centrum / Janka Krála 9 / 974 01 Banská Bystrica / Slovakia
/ [email protected]<mailto:[email protected]>
reception: +421 2 206 65 114 / www.pantheon.sk<http://www.pantheon.sk/>
[logo]
MiroslavMacko
Software Developer
Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R&D centrum / Janka Krála 9 / 974 01 Banská Bystrica / Slovakia
/ [email protected]<mailto:[email protected]>
reception: +421 2 206 65 114 / www.pantheon.sk<http://www.pantheon.sk/>
[logo]
<Flows_Additional_TCs-frm-OVS24.zip><Flows_Additional_TCs-frs-OVS20.zip>
_______________________________________________
openflowplugin-dev mailing list
[email protected]<mailto:[email protected]>
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
--
Thanks
Anil
AndrejLeitner
Software Developer
Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R&D centrum / Janka Krála 9 / 974 01 Banská Bystrica / Slovakia
/ [email protected]
reception: +421 2 206 65 114 / www.pantheon.sk
[logo]
_______________________________________________
openflowplugin-dev mailing list
[email protected]
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
