Benchmark script for test plan
** Description changed:
+ [ Impact ]
+
+ The python3-openvswitch package ships the OVS Python bindings without the
+ compiled JSON C extension (ovs._json). The extension fails to build during
+ package compilation and the failure is silently ignored (the build log shows
+ "WARNING: The C extension could not be compiled, speedups are not enabled"),
so
+ all JSON parsing falls back to the pure-Python parser.
+
+ Because every OVSDB transaction must be parsed, JSON parsing is a major
+ performance bottleneck in the IDL. Under load, particularly in large clouds
with
+ many ports/networks, the pure-Python parser consumes large amounts of CPU in
+ neutron-server workers and causes API requests to hang or time out while ovsdb
+ transactions are being processed.
+
+ The extension fails to build for two reasons: python/setup.py in OVS 2.17
+ has no mechanism to receive include paths and linker flags for the C
+ extension, and the package does not build with -fPIC, which is required to
+ link the static libopenvswitch into the shared extension module.
+
+ The fix backports two upstream OVS commits that enable python/setup.py to
+ honor the enable_shared/extra_cflags/extra_libs environment variables, and
+ updates d/rules to build with -fPIC and pass the static linkage flags at
+ Python install time. python3-openvswitch becomes an arch:any package as it
+ now ships a compiled module. The upstream commits first appeared in OVS 3.0
[0]
+ and 3.1 [1] respectively, and the packaging changes mirror what Noble's
+ openvswitch already does.
+
+ [ Test Plan ]
+
+ ** Setup OpenStack Environment **
+
+ 1. Deploy a test cloud using stsstack-bundles:
+ $ cd stsstack-bundles/openstack
+ $ ./generate-bundle.sh --series jammy --release yoga --name
openvswitch-test --run
+ # wait until all units are active/idle (some will be blocked/waiting)
+ $ ./tools/vault-unseal-and-authorise.sh
+ # wait until all units are active/idle
+ $ ./configure
+
+ ** Verify the Bug **
+
+ 2. SSH into the neutron-api unit:
+ $ juju ssh neutron-api/0
+
+ 3. Confirm the C extension is not shipped:
+ $ dpkg -L python3-openvswitch | grep _json
+ (empty)
+
+ 4. Confirm the pure-Python parser is active:
+ $ python3 -c "import ovs._json, ovs.json; print('Active Parser Engine:',
ovs.json.PARSER)"
+
+ Observed result:
+ ModuleNotFoundError: No module named 'ovs._json'
+
+ 5. Confirm neutron-server workers are using the pure-Python parser:
+ $ for p in $(pgrep -f neutron-server); do sudo grep -q ovs/_json
/proc/$p/maps && echo "$p: C" || echo "$p: python"; done
+
+ Observed result:
+ "python" printed for all workers
+
+ 6. Run the JSON parsing benchmark (see attached benchmark.py, which parses
+ a synthetic OVSDB payload 10 times):
+ $ python3 benchmark.py
+
+ Observed result:
+ === Benchmark Results ===
+ Host Default Engine: Pure-Python (fallback)
+ Total Elapsed Time: 13.8047 s
+ Parsing Throughput: 1.50 MB/s
+
+ ** Verify the Fix **
+
+ 7. SSH into the neutron-api unit and install the updated openvswitch packages:
+ $ juju ssh neutron-api/0
+ $ sudo apt update && sudo apt upgrade -y openvswitch-common
python3-openvswitch
+
+ 8. Restart neutron-server
+ $ sudo systemctl restart neutron-server
+
+ 9. Confirm the C extension is shipped:
+ $ dpkg -L python3-openvswitch | grep _json
+
+ Observed result:
+ /usr/lib/python3/dist-packages/ovs/_json.cpython-310-x86_64-linux-gnu.so
+
+ 10. Confirm the C parser is active:
+ $ python3 -c "import ovs._json, ovs.json; print('Active Parser Engine:',
ovs.json.PARSER)"
+
+ Observed result:
+ Active Parser Engine: C
+
+ 11. Confirm workers load the C extension:
+ $ for p in $(pgrep -f neutron-server); do sudo grep -q ovs/_json
/proc/$p/maps && echo "$p: C" || echo "$p: python"; done
+
+ Observed result:
+ Should show "C" for all workers.
+
+ 12. Re-run the benchmark:
+ $ python3 benchmark.py
+
+ Observed result:
+ === Benchmark Results ===
+ Host Default Engine: C (ovs._json)
+ Total Elapsed Time: 0.6149 s
+ Parsing Throughput: 33.78 MB/s
+
+ [ Where problems could occur ]
+
+ - The extension is statically linked against libopenvswitch, so no new
+ runtime library dependencies are introduced; a linkage mistake would
+ show up as an import failure of ovs._json, caught by step 10.
+
+ - The build now uses -fPIC for the whole package. This is the default for
+ shared objects already and has no functional effect on the produced C
+ code, but it does rebuild all objects; a toolchain issue would manifest
+ at build time, not runtime.
+
+ - python3-openvswitch changes from Architecture: all to linux-any. Reverse
+ dependencies were switched from ${source:Version} to ${binary:Version}; any
+ mistake there would surface as unresolvable dependencies at install time.
+
+ - If the extension fails to build in a future rebuild, the upstream fallback
+ logic silently drops it again, reverting to today's behavior (pure-Python
+ parser) rather than failing the build.
+
+ [ Other Info ]
+
+ The two setup.py commits are upstream: 671f93f ("python: Allow building json C
+ extension with static OVS library.") in OVS 3.0 [0], and 791c472 ("python: Add
+ ability to pass extra libs and cflags for C extension.") in OVS 3.1 [1].
+
+ The d/rules and d/control changes mirror three Debian packaging commits:
+ "d/rules: Build the Python JSON C extension" [2], "d/control: Make
+ python3-openvswitch a binary package, drop python3-six dependency" [3], and
+ "d/rules: Add comment about the explicit -fPIC for this package" [4]. These
+ first appeared in Debian's openvswitch 3.0.1-1.
+
+ Ubuntu releases Noble onward and the OVN/OVS versions shipped in corresponding
+ UCA pockets already ship the compiled extension, so only Jammy needs this fix.
+ Upstream OVS 2.17 is EOL, so the setup.py changes cannot be backported there.
+
+ A test package was built and is available in the following PPA:
+ https://launchpad.net/~munirsid/+archive/ubuntu/sf427278-test
+
+ [0] -
https://github.com/openvswitch/ovs/commit/671f93fe42d3c15aa0bf883bef0f05716bc2b18b
+ [1] -
https://github.com/openvswitch/ovs/commit/791c47273983b830dbc840ae7bf18a23bc9e72e4
+ [2] -
https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/45734ca10e2ac97fe2646588596933adb00f81ba
+ [3] -
https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/22f00e3fcfece1753d4cde17c9ae0b9394a4a966
+ [4] -
https://salsa.debian.org/openstack-team/third-party/openvswitch/-/commit/b120fc6a2caad12a7225c7600172f90102c0e414
+
+ Original Description: Find the original description of the bug below:
+
The C extension wrapping OVS's own json parser is 60x faster than the
pure-python version currently being used. Profiling shows JSON parsing
to be the biggest bottleneck using python-openvswitch IDL library.
setuptools can build the extension as long as the python-dev packages
are installed and openvswitch-dev is installed. It should be available
in OVS 2.6+
** Attachment added: "benchmark.py"
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1672522/+attachment/6008439/+files/benchmark.py
** Also affects: openvswitch (Ubuntu Stonking)
Importance: Medium
Status: Fix Released
** Also affects: openvswitch (Ubuntu Noble)
Importance: Undecided
Status: New
** Also affects: openvswitch (Ubuntu Jammy)
Importance: Undecided
Status: New
** Also affects: openvswitch (Ubuntu Resolute)
Importance: Undecided
Status: New
** Changed in: openvswitch (Ubuntu Resolute)
Status: New => Fix Released
** Changed in: openvswitch (Ubuntu Noble)
Status: New => Fix Released
** Changed in: openvswitch (Ubuntu Jammy)
Status: New => In Progress
** Changed in: openvswitch (Ubuntu Jammy)
Assignee: (unassigned) => Munir Siddiqui (munirsid)
** Also affects: cloud-archive
Importance: Undecided
Status: New
** Also affects: cloud-archive/epoxy
Importance: Undecided
Status: New
** Also affects: cloud-archive/zed
Importance: Undecided
Status: New
** Also affects: cloud-archive/ovn-22.03
Importance: Undecided
Status: New
** Also affects: cloud-archive/bobcat
Importance: Undecided
Status: New
** Also affects: cloud-archive/dalmatian
Importance: Undecided
Status: New
** Also affects: cloud-archive/antelope
Importance: Undecided
Status: New
** Also affects: cloud-archive/caracal
Importance: Undecided
Status: New
** Also affects: cloud-archive/flamingo
Importance: Undecided
Status: New
** Also affects: cloud-archive/yoga
Importance: Undecided
Status: New
** Also affects: cloud-archive/gazpacho
Importance: Undecided
Status: New
** Changed in: cloud-archive
Status: New => Fix Released
** Changed in: cloud-archive/zed
Status: New => Fix Released
** Changed in: cloud-archive/yoga
Status: New => In Progress
** Changed in: cloud-archive/ovn-22.03
Status: New => In Progress
** Changed in: cloud-archive/gazpacho
Status: New => Fix Released
** Changed in: cloud-archive/flamingo
Status: New => Fix Released
** Changed in: cloud-archive/epoxy
Status: New => Fix Released
** Changed in: cloud-archive/dalmatian
Status: New => Fix Released
** Changed in: cloud-archive/caracal
Status: New => Fix Released
** Changed in: cloud-archive/bobcat
Status: New => Fix Released
** Changed in: cloud-archive/antelope
Status: New => Fix Released
** Changed in: cloud-archive/ovn-22.03
Assignee: (unassigned) => Munir Siddiqui (munirsid)
** Changed in: cloud-archive/yoga
Assignee: (unassigned) => Munir Siddiqui (munirsid)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672522
Title:
The python-openvswitch package should build the json C extension
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1672522/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
