Hi, Terry!
In continuation to our yesterday’s conversation [0], we were able to reproduce
the issue with KeyError. We found that the problem is not connected with
ovsdb-server load but it appears when the ovsdb-server schema is converted
online (it even doesn’t matter whether the real ovs schema is changed) while
the active connection persists.
Please use next commands to reproduce it:
# in terminal 1
ovsdb-tool create ./ovs.db /usr/share/ovn/ovn-nb.ovsschema
ovsdb-server --remote punix://$(pwd)/ovs.sock <punix://$(pwd)/ovs.sock>
$(pwd)/ovs.db -vconsole:dbg
# in terminal 2. run python shell
python3
# setup connection
import ovsdbapp.schema.ovn_northbound.impl_idl as nb_idl
from ovsdbapp.backend.ovs_idl import connection
remote = "unix:///<path to ovs.sock>"
def get_idl():
"""Connection getter."""
idl = connection.OvsdbIdl.from_server(remote, "OVN_Northbound",
leader_only=False)
return nb_idl.OvnNbApiIdlImpl(connection.Connection(idl, 100))
idl = get_idl()
# in terminal 1
ovsdb-client convert unix:$(pwd)/ovs.sock /usr/share/ovn/ovn-nb.ovsschema
# in terminal 2 python shell:
idl.ls_add("test").execute()
We get following traceback:
Traceback (most recent call last):
File
"/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line
131, in run
txn.results.put(txn.do_commit())
File
"/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 143, in do_commit
self.post_commit(txn)
File
"/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 73, in post_commit
command.post_commit(txn)
File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
line 94, in post_commit
row = self.api.tables[self.table_name].rows[real_uuid]
File "/usr/lib64/python3.6/collections/__init__.py", line 991, in __getitem__
raise KeyError(key)
KeyError: UUID('0256afa4-6dd0-4c2c-b6a2-686a360ab331')
In ovsdb-server debug logs we see that update2 or update3 messages are not sent
from server in response to client’s transaction, just reply with result UUID:
2022-11-24T17:42:36Z|00306|poll_loop|DBG|wakeup due to [POLLIN] on fd 18
(///root/ovsdb-problem/ovs.sock<->) at lib/stream-fd.c:157
2022-11-24T17:42:36Z|00307|jsonrpc|DBG|unix#5: received request,
method="transact",
params=["OVN_Northbound",{"uuid-name":"row03ef28d6_93f1_43bc_b07a_eae58d4bd1c5","table":"Logical_Switch","op":"insert","row":{"name”:"test"}}],
id=5
2022-11-24T17:42:36Z|00308|jsonrpc|DBG|unix#5: send reply,
result=[{"uuid":["uuid","4eb7c407-beec-46ca-b816-19f942e57721"]}], id=5
We checked same with ovn-nbctl running in daemon mode and found that the
problem is not reproduced (ovsdb-server after database conversion sends out
update3 message to ovn-nbctl daemon process in response to transaction, for
example ovs-appctl -t <nbctl daemon socket> run ls-add test-ls):
2022-11-24T17:54:51Z|00623|jsonrpc|DBG|unix#7: received request,
method="transact",
params=["OVN_Northbound",{"uuid-name":"rowcdb152ce_a9af_4761_b965_708ad300fcb7","table":"Logical_Switch","op":"insert","row":{"name":"test-ls"}},{"comment":"ovn-nbctl:
run ls-add test-ls","op":"comment"}], id=5
2022-11-24T17:54:51Z|00624|jsonrpc|DBG|unix#7: send notification,
method="update3",
params=[["monid","OVN_Northbound"],"00000000-0000-0000-0000-000000000000",{"Logical_Switch":{"0b147f2c-248d-496a-b718-a5328d3c2995":{"insert":{"name":"test-ls"}}}}]
2022-11-24T17:54:51Z|00625|jsonrpc|DBG|unix#7: send reply,
result=[{"uuid":["uuid","0b147f2c-248d-496a-b718-a5328d3c2995"]},{}], id=5
So it seems that the problem is in python-ovs, not in ovsdb-server.
We tested with ovsdb-server running version 2.17.3 and python-ovs 2.13.5 and
also python-ovs 2.17.3, the behaviour is the same.
Do you have any ideas what can be a reason for such behaviour?
0:
https://review.opendev.org/c/openstack/ovsdbapp/+/865454/comments/674c57e6_3849591b
<https://review.opendev.org/c/openstack/ovsdbapp/+/865454/comments/674c57e6_3849591b>
Regards, Anton.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
