On 6/8/23 07:57, Han Zhou wrote:
> On Wed, Jun 7, 2023 at 2:05 PM Numan Siddique <[email protected]> wrote:
>>
>> On Fri, Jun 2, 2023 at 12:12 AM Han Zhou <[email protected]> wrote:
>>>
>>> Maintain a set of lsp names that are referenced in the column
>>> "ip_port_mappings" of load_balancer records (used for health check
>>> purpose), which will be used for lsp incremental processing.
>>>
>>> Signed-off-by: Han Zhou <[email protected]>
>>> Reviewed-by: Ales Musil <[email protected]>
>>
>> Thanks for adding the lsp I-P in northd.
>>
>> LGTM.
>>
>> For this patch and for the rest of the patches (P7-P14)
>>
>> Acked-by: Numan Siddique <[email protected]>
>>
> Thanks Numan. I applied all P6-P14 to main.
>
> Han
>
Hi Han,
Sorry, I still didn't get a chance to review this series thoroughly,
I'll do that in-tree. However, I see some failures on one of my test
machines when running tests with latest main (this series applied).
1144: LSP incremental processing -- ovn-northd -- parallelization=yes FAILED
(ovn-northd.at:9487)
746: propagate Port_Binding.up to NB and OVS -- ovn-northd --
parallelization=yes -- ovn_monitor_all=yes FAILED (ovs-macros.at:258)
748: propagate Port_Binding.up to NB and OVS -- ovn-northd --
parallelization=no -- ovn_monitor_all=yes FAILED (ovs-macros.at:258)
747: propagate Port_Binding.up to NB and OVS -- ovn-northd --
parallelization=yes -- ovn_monitor_all=no FAILED (ovs-macros.at:258)
749: propagate Port_Binding.up to NB and OVS -- ovn-northd --
parallelization=no -- ovn_monitor_all=no FAILED (ovs-macros.at:258)
I didn't look into details yet but here's the testsuite log for
746 and 1144 in case you want to check it out already. Bisecting
this it seems to point to:
b41d74fcb382 ("northd: Ignore "up" column change if ignore_lsp_down is true.")
Note: these tests pass fine on my laptop and in GitHub CI..
Regards,
Dumitru
746. ovn.at:30779: testing propagate Port_Binding.up to NB and OVS --
ovn-northd -- parallelization=yes -- ovn_monitor_all=yes ...
creating ovn-sb database
ovsdb-server -vjsonrpc
--remote=punix:/root/ovn/tests/testsuite.dir/0746/ovn-sb/ovn-sb.sock
--remote=db:OVN_Southbound,SB_Global,connections
--private-key=/root/ovn/tests/testpki-test-privkey.pem
--certificate=/root/ovn/tests/testpki-test-cert.pem
--ca-cert=/root/ovn/tests/testpki-cacert.pem
/root/ovn/tests/testsuite.dir/0746/ovn-sb/ovn-sb.db -vconsole:off --detach
--no-chdir --pidfile --log-file
creating ovn-nb database
ovsdb-server -vjsonrpc
--remote=punix:/root/ovn/tests/testsuite.dir/0746/ovn-nb/ovn-nb.sock
/root/ovn/tests/testsuite.dir/0746/ovn-nb/ovn-nb.db -vconsole:off --detach
--no-chdir --pidfile --log-file
starting northd
ovn-northd --n-threads=4 -vjsonrpc
--ovnnb-db=unix:/root/ovn/tests/testsuite.dir/0746/ovn-nb/ovn-nb.sock
--ovnsb-db=unix:/root/ovn/tests/testsuite.dir/0746/ovn-sb/ovn-sb.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
2023-06-08T09:50:57Z|00001|ovn_northd|INFO|Using 4 threads
starting northd-backup
ovn-northd --n-threads=4 -vjsonrpc
--ovnnb-db=unix:/root/ovn/tests/testsuite.dir/0746/ovn-nb/ovn-nb.sock
--ovnsb-db=unix:/root/ovn/tests/testsuite.dir/0746/ovn-sb/ovn-sb.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
2023-06-08T09:50:57Z|00001|ovn_northd|INFO|Using 4 threads
0ab11919-51d7-4258-b00b-865e8c497a00
ovn-macros.at:234: waiting until TCP_PORT=`sed -n 's/.*0:.*: listening on port
\([0-9]*\)$/\1/p' "$d/ovn-sb/ovsdb-server.log"` && test X != X"$TCP_PORT"...
ovn-macros.at:234: wait succeeded immediately
adding simulator 'main'
ovsdb-server --remote=punix:/root/ovn/tests/testsuite.dir/0746/main/db.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
ovs-vswitchd --enable-dummy=system -vvconn -vofproto_dpif -vunixctl
-vconsole:off --detach --no-chdir --pidfile --log-file
adding simulator 'hv1'
ovsdb-server --remote=punix:/root/ovn/tests/testsuite.dir/0746/hv1/db.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
ovs-vswitchd --enable-dummy=system -vvconn -vofproto_dpif -vunixctl
-vconsole:off --detach --no-chdir --pidfile --log-file
ovn-controller --enable-dummy-vif-plug -vconsole:off --detach --no-chdir
--pidfile --log-file
ovn-nbctl ls-add ls
./ovn-macros.at:419: "$@"
## ----------------------------- ##
## add OVS port for existing LSP ##
## ----------------------------- ##
ovn-nbctl lsp-add ls lsp1
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=hv sync
./ovn-macros.at:419: "$@"
Checking values in sb Port_Binding with logical_port=lsp1 against false...
found false
ovs-vsctl add-port br-int lsp1 -- set Interface lsp1 external-ids:iface-id=lsp1
./ovn-macros.at:419: "$@"
Waiting until up in sb Port_Binding with logical_port=lsp1 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
Waiting until up in nb Logical_Switch_Port with name=lsp1 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
ovn.at:30779: waiting until test `ovs-vsctl get Interface lsp1
external_ids:ovn-installed` = '"true"'...
ovn.at:30779: wait succeeded immediately
## ----------------------------- ##
## add LSP for existing OVS port ##
## ----------------------------- ##
ovs-vsctl add-port br-int lsp2 -- set Interface lsp2 external-ids:iface-id=lsp2
./ovn-macros.at:419: "$@"
ovn-nbctl lsp-add ls lsp2
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=hv sync
./ovn-macros.at:419: "$@"
Checking values in sb Port_Binding with logical_port=lsp2 against true... found
true
Waiting until up in nb Logical_Switch_Port with name=lsp2 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
ovn.at:30779: waiting until test `ovs-vsctl get Interface lsp2
external_ids:ovn-installed` = '"true"'...
ovn.at:30779: wait succeeded immediately
## -------------------------------------------------------------- ##
## ovn-controller should not reset Port_Binding.up without northd ##
## -------------------------------------------------------------- ##
ovn-sbctl clear Port_Binding lsp1 up
./ovn-macros.at:419: "$@"
ovn-sbctl clear Port_Binding lsp1 chassis
./ovn-macros.at:419: "$@"
Waiting until chassis in sb Port_Binding with logical_port=lsp1 is
3e5b1360-b992-48c3-874c-1e3203a2e6d3...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
Checking values in sb Port_Binding with logical_port=lsp1 against ... found
Waiting until up in sb Port_Binding with logical_port=lsp1 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
Waiting until up in nb Logical_Switch_Port with name=lsp1 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
## ------------------------------------------------------- ##
## ovn-controller should reset Port_Binding.up - from NULL ##
## ------------------------------------------------------- ##
ovn-sbctl clear Port_Binding lsp1 up
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=sb sync
./ovn-macros.at:419: "$@"
Waiting until up in nb Logical_Switch_Port with name=lsp1 is false...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait succeeded immediately
Waiting until up in sb Port_Binding with logical_port=lsp1 is true...
ovn-macros.at:524: waiting until
found=$(ovn-${db}ctl --bare --columns $column find $table $a $b $c $d $e)
found=$(for d in $found; do echo $d; done | sort)
test "$expected" = "$found"
...
ovn-macros.at:524: wait failed after 30 seconds
up in sb table Port_Binding has value , from the following rows:
_uuid : 54ce6370-38a3-452d-b2cb-7cb9bbb90e67
additional_chassis : []
additional_encap : []
chassis : 3e5b1360-b992-48c3-874c-1e3203a2e6d3
datapath : 0d18aa35-a297-4b3c-869d-1f71de7378cd
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : lsp2
mac : []
mirror_rules : []
nat_addresses : []
options : {}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 2
type : ""
up : true
virtual_parent : []
_uuid : b778b79c-1d82-478b-b93d-f0567e99ac41
additional_chassis : []
additional_encap : []
chassis : 3e5b1360-b992-48c3-874c-1e3203a2e6d3
datapath : 0d18aa35-a297-4b3c-869d-1f71de7378cd
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : lsp1
mac : []
mirror_rules : []
nat_addresses : []
options : {}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 1
type : ""
up : []
virtual_parent : []
./ovs-macros.at:258: hard failure
746. ovn.at:30779: 746. propagate Port_Binding.up to NB and OVS -- ovn-northd
-- parallelization=yes -- ovn_monitor_all=yes (ovn.at:30779): FAILED
(ovs-macros.at:258)
---
1144. ovn-northd.at:9487: testing LSP incremental processing -- ovn-northd --
parallelization=yes ...
creating ovn-sb database
ovsdb-server -vjsonrpc
--remote=punix:/root/ovn/tests/testsuite.dir/1144/ovn-sb/ovn-sb.sock
--remote=db:OVN_Southbound,SB_Global,connections
--private-key=/root/ovn/tests/testpki-test-privkey.pem
--certificate=/root/ovn/tests/testpki-test-cert.pem
--ca-cert=/root/ovn/tests/testpki-cacert.pem
/root/ovn/tests/testsuite.dir/1144/ovn-sb/ovn-sb.db -vconsole:off --detach
--no-chdir --pidfile --log-file
creating ovn-nb database
ovsdb-server -vjsonrpc
--remote=punix:/root/ovn/tests/testsuite.dir/1144/ovn-nb/ovn-nb.sock
/root/ovn/tests/testsuite.dir/1144/ovn-nb/ovn-nb.db -vconsole:off --detach
--no-chdir --pidfile --log-file
starting northd
ovn-northd --n-threads=4 -vjsonrpc
--ovnnb-db=unix:/root/ovn/tests/testsuite.dir/1144/ovn-nb/ovn-nb.sock
--ovnsb-db=unix:/root/ovn/tests/testsuite.dir/1144/ovn-sb/ovn-sb.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
2023-06-08T09:56:03Z|00001|ovn_northd|INFO|Using 4 threads
starting northd-backup
ovn-northd --n-threads=4 -vjsonrpc
--ovnnb-db=unix:/root/ovn/tests/testsuite.dir/1144/ovn-nb/ovn-nb.sock
--ovnsb-db=unix:/root/ovn/tests/testsuite.dir/1144/ovn-sb/ovn-sb.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
2023-06-08T09:56:03Z|00001|ovn_northd|INFO|Using 4 threads
b07d4382-d342-47b7-81b8-46fb188dc614
ovn-macros.at:234: waiting until TCP_PORT=`sed -n 's/.*0:.*: listening on port
\([0-9]*\)$/\1/p' "$d/ovn-sb/ovsdb-server.log"` && test X != X"$TCP_PORT"...
ovn-macros.at:234: wait succeeded immediately
adding simulator 'main'
ovsdb-server --remote=punix:/root/ovn/tests/testsuite.dir/1144/main/db.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
ovs-vswitchd --enable-dummy=system -vvconn -vofproto_dpif -vunixctl
-vconsole:off --detach --no-chdir --pidfile --log-file
adding simulator 'hv1'
ovsdb-server --remote=punix:/root/ovn/tests/testsuite.dir/1144/hv1/db.sock
-vconsole:off --detach --no-chdir --pidfile --log-file
ovs-vswitchd --enable-dummy=system -vvconn -vofproto_dpif -vunixctl
-vconsole:off --detach --no-chdir --pidfile --log-file
ovn-controller --enable-dummy-vif-plug -vconsole:off --detach --no-chdir
--pidfile --log-file
ovn-nbctl --wait=hv ls-add ls0
./ovn-macros.at:419: "$@"
as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=hv lsp-add ls0 lsp0-0 -- lsp-set-addresses lsp0-0 unknown
./ovn-macros.at:419: "$@"
ovn-northd.at:9487: waiting until test `as northd ovn-appctl -t ovn-northd
inc-engine/show-stats northd recompute` = 5...
ovn-northd.at:9487: wait succeeded immediately
ovn-northd.at:9487: waiting until test `as northd ovn-appctl -t ovn-northd
inc-engine/show-stats lflow recompute` = 5...
ovn-northd.at:9487: wait succeeded immediately
as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=hv lsp-add ls0 lsp0-1 -- lsp-set-addresses lsp0-1
aa:aa:aa:00:00:01 192.168.0.11
./ovn-macros.at:419: "$@"
./ovn-northd.at:9487: as northd ovn-appctl -t ovn-northd inc-engine/show-stats
northd recompute
./ovn-northd.at:9487: as northd ovn-appctl -t ovn-northd inc-engine/show-stats
lflow recompute
as northd ovn-appctl -t ovn-northd inc-engine/clear-stats
./ovn-macros.at:419: "$@"
ovn-nbctl --wait=hv lsp-add ls0 lsp0-2 -- lsp-set-addresses lsp0-2
aa:aa:aa:00:00:02 192.168.0.12
./ovn-macros.at:419: "$@"
./ovn-northd.at:9487: as northd ovn-appctl -t ovn-northd inc-engine/show-stats
northd recompute
--- - 2023-06-08 05:56:06.104378452 -0400
+++ /root/ovn/tests/testsuite.dir/at-groups/1144/stdout 2023-06-08
05:56:06.100714181 -0400
@@ -1,2 +1,2 @@
-0
+1
1144. ovn-northd.at:9487: 1144. LSP incremental processing -- ovn-northd --
parallelization=yes (ovn-northd.at:9487): FAILED (ovn-northd.at:9487)
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
