+ Frode On Mon, Mar 27, 2023 at 09:42:59PM +0200, Ilya Maximets wrote: > Currently, database schema conversion in case of clustered database > produces a transaction record with both new schema and converted > database data. So, the sequence of events is following: > > 1. Get the new schema. > 2. Convert the database to a new schema. > 3. Translate the newly converted database into JSON. > 4. Write the schema + data JSON to the storage. > 5. Destroy converted version of a database. > 6. Read schema + data JSON from the storage and parse. > 7. Create a new database from a parsed database data. > 8. Replace current database with the new one. > > Most of these steps are very computationally expensive. Also, > conversion to/from JSON is much more expensive than direct database > conversion with ovsdb_convert() that can make use of shallow data > copies. > > Instead of doing all that, let's make use of previously introduced > ability to not write the converted data into the storage. The process > will look like this then: > > 1. Get the new schema. > 2. Convert the database to a new schema > (to verify that it is possible). > 3. Write the schema to the storage. > 4. Destroy converted version of a database. > 5. Read the new schema from the storage and parse. > 6. Convert the database to a new schema. > 7. Replace current database with the new one. > > Most of the operations here are performed on the small schema object, > instead of the actual database data. Two remaining data operations > (actual conversion) are noticeably faster than conversion to/from > JSON due to reference counting and shallow data copies. > > Steps 4-6 can be optimized later to not convert twice on the > process that initiates the conversion. > > The change results in following performance improvements in conversion > of OVN_Southbound database schema from version 20.23.0 to 20.27.0 > (measured on a single-server RAFT cluster with no clients): > > | Before | After > +---------+-------------------+---------+------------------ > DB size | Total | Max poll interval | Total | Max poll interval > --------+---------+-------------------+---------+------------------ > 542 MB | 47 sec. | 26 sec. | 15 sec. | 10 sec. > 225 MB | 19 sec. | 10 sec. | 6 sec. | 4.5 sec. > > 542 MB database had 19.5 M atoms, 225 MB database had 7.5 M atoms. > > Overall performance improvement is about 3x. > > Also, note that before this change database conversion basically > doubles the database file on disk. Now it only writes a small > schema JSON. > > Since the change requires backward-incompatible database file format > changes, documentation is updated on how to perform an upgrade. > Handled the same way as we did for the previous incompatible format > change in 2.15 (column diffs). > > Reported-at: > https://mail.openvswitch.org/pipermail/ovs-discuss/2022-December/052140.html > Signed-off-by: Ilya Maximets <[email protected]>
Reviewed-by: Simon Horman <[email protected]> _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
