Also added a NEWS entry.
Signed-off-by: Ilya Maximets <[email protected]>
---
Documentation/automake.mk | 1 +
Documentation/topics/index.rst | 1 +
Documentation/topics/record-replay.rst | 138 +++++++++++++++++++++++++
NEWS | 4 +
4 files changed, 144 insertions(+)
create mode 100644 Documentation/topics/record-replay.rst
diff --git a/Documentation/automake.mk b/Documentation/automake.mk
index ea3475f35..0fd7fdf7e 100644
--- a/Documentation/automake.mk
+++ b/Documentation/automake.mk
@@ -54,6 +54,7 @@ DOC_SOURCE = \
Documentation/topics/ovs-extensions.rst \
Documentation/topics/ovsdb-replication.rst \
Documentation/topics/porting.rst \
+ Documentation/topics/record-replay.rst \
Documentation/topics/tracing.rst \
Documentation/topics/userspace-tso.rst \
Documentation/topics/windows.rst \
diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst
index 08af3a24d..0036567eb 100644
--- a/Documentation/topics/index.rst
+++ b/Documentation/topics/index.rst
@@ -48,6 +48,7 @@ OVS
dpdk/index
windows
language-bindings
+ record-replay
testing
tracing
userspace-tso
diff --git a/Documentation/topics/record-replay.rst
b/Documentation/topics/record-replay.rst
new file mode 100644
index 000000000..5fd672f1a
--- /dev/null
+++ b/Documentation/topics/record-replay.rst
@@ -0,0 +1,138 @@
+..
+ Copyright 2021, Red Hat, Inc.
+
+ Licensed under the Apache License, Version 2.0 (the "License"); you may
+ not use this file except in compliance with the License. You may obtain
+ a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ License for the specific language governing permissions and limitations
+ under the License.
+
+ Convention for heading levels in Open vSwitch documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ~~~~~~~ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
+============================
+Debugging with Record/Replay
+============================
+
+The ``ovs-replay`` library provides a set of internal functions for recording
+certain events for later replay. This library is integrated into the
+``stream`` and some other modules to record all incoming data across all
+streams (ssl, tcp, unixctl) of applications baed on Open vSwitch libraries
+and play these streams later for debugging or performance testing purposes.
+
+Support for this feature is currently integrated into the ``ovsdb-server`` and
+``ovsdb-client`` applications. As a result, this allows to record lifecycle
+of the ``ovsdb-server`` process in large OVN deployments. Later, by using only
+the recorded data, the user can replay transactions and connections that
+occurred in a large deployment on their local PC. At the same time it is
+possible to tweak various log levels, run a process under debugger or tracer,
+measure performance with ``perf``, and so on.
+
+ .. note::
+
+ The current version of record/replay engine does not work correctly with
+ internal time-based events that leats to communications with other
+ processes. For this reason it can not be used with clustered databases
+ (RAFT implementation is heavily time dependent).
+ In addition, recording automatically disables inactivity probes on
+ JSONRPC connections and updates for the Manager status in a _Server
+ database.
+
+High-level feature overview was presented on Open vSwitch and OVN 2020 Fall
+Conference: `Debugging OVSDB with stream record/replay`__
+
+__
https://www.openvswitch.org/support/ovscon2020/slides/Debugging-OVSDB-with-stream-record_replay.pdf
+
+Recording ovsdb-server events
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To start recording events for the ``ovsdb-server`` process, there is a special
+command line argument ``--replay-record``. Before starting the database
+server, make sure that you have a copy of a database file, so you can use it
+for replay later. Here are the general steps to take:
+
+1. Create a directory where the replay files will be stored::
+
+ $ mkdir replay-dir
+ $ REPLAY_DIR=$(pwd)/replay-dir
+
+2. Copy the current database file for later use::
+
+ $ cp my_database $REPLAY_DIR/
+
+3. Run ``ovsdb-server`` with recording enabled::
+
+ $ ovsdb-server --replay-record=$REPLAY_DIR <other arguments> my_database
+
+4. Work with the database as usual.
+
+5. Stop the ``ovsdb-server`` process at the end (it is important to send an
+ ``exit`` command so that during replay the process will exit in the end
+ too)::
+
+ $ ovs-appctl -t ovsdb-server exit
+
+After that ``$REPLAY_DIR`` should contain replay files with recorded data.
+
+Replay of recorded session
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+During replay, the ``ovsdb-server`` will receive all the same connections,
+transactions and commands as it had at the time of recording, but it will not
+create any actual network/socket connections and will not communicate with
+any other process. Everything will be read from the replay files.
+
+Since there is no need to wait for IPC, all events will be received one by one
+without any delays, so the application will process them as quickly as
+possible. This can be used as a performance test where the user can measure how
+quickly the ``ovsdb-server`` can handle some workload recorded in a real
+deployment.
+
+The command line argument to start a replay session is ``--replay``. The steps
+will look like this:
+
+1. Restore the database file from a previous copy::
+
+ $ cp $REPLAY_DIR/my_database my_database
+
+2. Start ``ovsdb-server`` with the same set of arguments as in the recording
+ stage, except for ``--replay-record``::
+
+ $ ovsdb-server --replay=$REPLAY_DIR <other arguments> my_database
+
+3. The process should exit in the end when the ``exit`` command is replayed.
+
+On step 2 it is possible to add extra logging arguments to debug some recorded
+issue, or run the process under debugger. It's also possible to replay with
+a different version of ``ovsdb-server`` binary as long as this does not affect
+the data that goes in and out of the process, e.g. pure performance
+optimizations.
+
+~~~~~~~~~~~
+Limitations
+~~~~~~~~~~~
+
+The record/replay engine has the following limitations:
+
+1. Record/Replay of clustered databases is not supported.
+
+2. Inactivity probes on JSONRPC connections are suppressed.
+
+3. Manager status updates suppressed in ``ovsdb-server``.
+
+To remove above limitations, it is necessary to implement correct handling of
+internally generated time-based events. (possibly by recording of time and
+subsequent time warping).
diff --git a/NEWS b/NEWS
index 95cf922aa..28b558162 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,9 @@
Post-v2.15.0
---------------------
+ - OVSDB:
+ * New command line options --replay-record/--replay for ovsdb-server
+ and ovsdb-client to record and replay all the incoming transactions,
+ monitors, etc. More datails in Documentation/topics/record-replay.rst.
- In ovs-vsctl and vtep-ctl, the "find" command now accept new
operators {in} and {not-in}.
- Userspace datapath:
--
2.26.2
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
