Re: [PATCH v4 09/10] tests/qtest: make more migration pre-copy scenarios run non-live

2023-06-01 Thread Juan Quintela
Daniel P. Berrangé  wrote:
> There are 27 pre-copy live migration scenarios being tested. In all of
> these we force non-convergence and run for one iteration, then let it
> converge and wait for completion during the second (or following)
> iterations. At 3 mbps bandwidth limit the first iteration takes a very
> long time (~30 seconds).
>
> While it is important to test the migration passes and convergence
> logic, it is overkill to do this for all 27 pre-copy scenarios. The
> TLS migration scenarios in particular are merely exercising different
> code paths during connection establishment.
>
> To optimize time taken, switch most of the test scenarios to run
> non-live (ie guest CPUs paused) with no bandwidth limits. This gives
> a massive speed up for most of the test scenarios.
>
> For test coverage the following scenarios are unchanged
>
>  * Precopy with UNIX sockets
>  * Precopy with UNIX sockets and dirty ring tracking
>  * Precopy with XBZRLE
>  * Precopy with UNIX compress
>  * Precopy with UNIX compress (nowait)
>  * Precopy with multifd
>
> On a test machine this reduces execution time from 13 minutes to
> 8 minutes.
>
> Tested-by: Thomas Huth 
> Signed-off-by: Daniel P. Berrangé 

Reviewed-by: Juan Quintela 




[PATCH v4 09/10] tests/qtest: make more migration pre-copy scenarios run non-live

2023-06-01 Thread Daniel P . Berrangé
There are 27 pre-copy live migration scenarios being tested. In all of
these we force non-convergence and run for one iteration, then let it
converge and wait for completion during the second (or following)
iterations. At 3 mbps bandwidth limit the first iteration takes a very
long time (~30 seconds).

While it is important to test the migration passes and convergence
logic, it is overkill to do this for all 27 pre-copy scenarios. The
TLS migration scenarios in particular are merely exercising different
code paths during connection establishment.

To optimize time taken, switch most of the test scenarios to run
non-live (ie guest CPUs paused) with no bandwidth limits. This gives
a massive speed up for most of the test scenarios.

For test coverage the following scenarios are unchanged

 * Precopy with UNIX sockets
 * Precopy with UNIX sockets and dirty ring tracking
 * Precopy with XBZRLE
 * Precopy with UNIX compress
 * Precopy with UNIX compress (nowait)
 * Precopy with multifd

On a test machine this reduces execution time from 13 minutes to
8 minutes.

Tested-by: Thomas Huth 
Signed-off-by: Daniel P. Berrangé 
---
 tests/qtest/migration-test.c | 81 +---
 1 file changed, 66 insertions(+), 15 deletions(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 23fb61506c..0b9d045152 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -577,9 +577,12 @@ typedef struct {
 MIG_TEST_FAIL_DEST_QUIT_ERR,
 } result;
 
-/* Optional: set number of migration passes to wait for */
+/* Optional: set number of migration passes to wait for, if live==true */
 unsigned int iterations;
 
+/* Optional: whether the guest CPUs should be running during migration */
+bool live;
+
 /* Postcopy specific fields */
 void *postcopy_data;
 bool postcopy_preempt;
@@ -1385,8 +1388,6 @@ static void test_precopy_common(MigrateCommon *args)
 return;
 }
 
-migrate_ensure_non_converge(from);
-
 if (args->start_hook) {
 data_hook = args->start_hook(from, to);
 }
@@ -1396,6 +1397,31 @@ static void test_precopy_common(MigrateCommon *args)
 wait_for_serial("src_serial");
 }
 
+if (args->live) {
+/*
+ * Testing live migration, we want to ensure that some
+ * memory is re-dirtied after being transferred, so that
+ * we exercise logic for dirty page handling. We achieve
+ * this with a ridiculosly low bandwidth that guarantees
+ * non-convergance.
+ */
+migrate_ensure_non_converge(from);
+} else {
+/*
+ * Testing non-live migration, we allow it to run at
+ * full speed to ensure short test case duration.
+ * For tests expected to fail, we don't need to
+ * change anything.
+ */
+if (args->result == MIG_TEST_SUCCEED) {
+qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}");
+if (!got_src_stop) {
+qtest_qmp_eventwait(from, "STOP");
+}
+migrate_ensure_converge(from);
+}
+}
+
 if (!args->connect_uri) {
 g_autofree char *local_connect_uri =
 migrate_get_socket_address(to, "socket-address");
@@ -1413,25 +1439,41 @@ static void test_precopy_common(MigrateCommon *args)
 qtest_set_expected_status(to, EXIT_FAILURE);
 }
 } else {
-if (args->iterations) {
-while (args->iterations--) {
+if (args->live) {
+if (args->iterations) {
+while (args->iterations--) {
+wait_for_migration_pass(from);
+}
+} else {
 wait_for_migration_pass(from);
 }
-} else {
-wait_for_migration_pass(from);
-}
 
-migrate_ensure_converge(from);
+migrate_ensure_converge(from);
 
-/* We do this first, as it has a timeout to stop us
- * hanging forever if migration didn't converge */
-wait_for_migration_complete(from);
+/*
+ * We do this first, as it has a timeout to stop us
+ * hanging forever if migration didn't converge
+ */
+wait_for_migration_complete(from);
 
-if (!got_src_stop) {
-qtest_qmp_eventwait(from, "STOP");
+if (!got_src_stop) {
+qtest_qmp_eventwait(from, "STOP");
+}
+} else {
+wait_for_migration_complete(from);
+/*
+ * Must wait for dst to finish reading all incoming
+ * data on the socket before issuing 'cont' otherwise
+ * it'll be ignored
+ */
+wait_for_migration_complete(to);
+
+qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}");
 }
 
-qtest_qmp_eventwait(to, "RESUME");
+if (!got_dst_resume) {
+