[PATCH 07/16] util/dsa: Implement zero page checking in DSA task.

2023-10-25 Thread Hao Xiang
at least two individual tasks. There are times we need to submit a single task and hence a single DSA task submission is also required. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang --- include/qemu/dsa.h | 16 +++ util/dsa.c | 252 + 2

[PATCH 12/16] migration/multifd: Enable set normal page ratio test hook in multifd.

2023-10-25 Thread Hao Xiang
Test hook is disabled by default. To set it, a normal page ratio between 0 and 100 are valid. If the ratio is set to 50, it means at least 50% of all pages are sent as normal pages. Signed-off-by: Hao Xiang --- include/qemu/dsa.h | 7 ++- migration/migration-hmp-cmds.c | 7

[PATCH 11/16] migration/multifd: Add test hook to set normal page ratio.

2023-10-25 Thread Hao Xiang
to be sent as a normal page. This is useful for live migration performance analysis and optimization. Signed-off-by: Hao Xiang --- migration/options.c | 31 +++ migration/options.h | 1 + qapi/migration.json | 17 ++--- 3 files changed, 46 insertions(+), 3

[PATCH 01/16] Cherry pick a set of patches that enables multifd zero page feature.

2023-10-25 Thread Hao Xiang
Juan Quintela had a patchset enabling zero page checking in multifd threads. https://lore.kernel.org/all/20220802063907.18882-13-quint...@redhat.com/ Some of the changes in that patchset has already made to upstream but a few are still being reviewed. This patch contains the changes from the

[PATCH 03/16] util/dsa: Add dependency idxd.

2023-10-25 Thread Hao Xiang
Idxd is the device driver for DSA (Intel Data Streaming Accelerator). The driver is fully functioning since Linux kernel 5.19. This change adds the driver's header file used for userspace development. Signed-off-by: Hao Xiang --- linux-headers/linux/idxd.h | 356

[PATCH 16/16] migration/multifd: Add integration tests for multifd with Intel DSA offloading.

2023-10-25 Thread Hao Xiang
* Add test case to start and complete multifd live migration with DSA offloading enabled. * Add test case to start and cancel multifd live migration with DSA offloading enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 66

[PATCH 14/16] migration/multifd: Enable set packet size migration option.

2023-10-25 Thread Hao Xiang
size is useful for performance tuning. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 7 +++ migration/multifd-zlib.c | 4 ++-- migration/multifd-zstd.c | 4 ++-- migration/multifd.c| 6 -- migration/multifd.h| 3 --- 5 files changed, 15

[PATCH 15/16] util/dsa: Add unit test coverage for Intel DSA task submission and completion.

2023-10-25 Thread Hao Xiang
* Test DSA start and stop path. * Test DSA configure and cleanup path. * Test DSA task submission and completion path. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- tests/unit/meson.build | 6 + tests/unit/test-dsa.c | 448 + 2 files changed

[PATCH 02/16] meson: Introduce new instruction set enqcmd to the build system.

2023-10-25 Thread Hao Xiang
Enable instruction set enqcmd in build. Signed-off-by: Hao Xiang --- meson.build | 2 ++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 7 insertions(+) diff --git a/meson.build b/meson.build index bd65a111aa..6ea859829c 100644

[PATCH 00/16] Use Intel DSA accelerator to offload zero page checking in multifd live migration.

2023-10-25 Thread Hao Xiang
using DSA offloading. * Patchset Apply this patchset on top of commit 800485762e6564e04e2ab315132d477069562d91 Hao Xiang (16): Cherry pick a set of patches that enables multifd zero page feature. meson: Introduce new instruction set enqcmd to the build system. util/dsa: Add depende

[PATCH 06/16] util/dsa: Implement DSA task asynchronous completion thread model.

2023-10-25 Thread Hao Xiang
. Signed-off-by: Hao Xiang --- util/dsa.c | 243 - 1 file changed, 242 insertions(+), 1 deletion(-) diff --git a/util/dsa.c b/util/dsa.c index f82282ce99..0e68013ffb 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -44,6 +44,7 @@ #define DSA_WQ_SIZE

[PATCH 09/16] migration/multifd: Add new migration option for multifd DSA offloading.

2023-10-25 Thread Hao Xiang
Intel DSA offloading is an optional feature that turns on if proper hardware and software stack is available. To turn on DSA offloading in multifd live migration: multifd-dsa-accel="[dsa_dev_path1] ] [dsa_dev_path2] ... [dsa_dev_pathX]" This feature is turned off by default. Signed-o

[PATCH 13/16] migration/multifd: Add migration option set packet size.

2023-10-25 Thread Hao Xiang
The current multifd packet size is 128 * 4kb. This change adds an option to set the packet size. Both sender and receiver needs to set the same packet size for things to work. Signed-off-by: Hao Xiang --- migration/options.c | 34 ++ migration/options.h | 1

[PATCH 05/16] util/dsa: Implement DSA task enqueue and dequeue.

2023-10-25 Thread Hao Xiang
* Use a safe thread queue for DSA task enqueue/dequeue. * Implement DSA task submission. * Implement DSA batch task submission. Signed-off-by: Hao Xiang --- include/qemu/dsa.h | 35 util/dsa.c | 196 + 2 files changed, 231 insertions

[PATCH 10/16] migration/multifd: Enable DSA offloading in multifd sender path.

2023-10-25 Thread Hao Xiang
. The sender thread submits a batch of pages to DSA hardware and waits for the DSA completion thread to signal for work completion. Signed-off-by: Hao Xiang --- migration/multifd.c | 101 +--- migration/multifd.h | 3 ++ 2 files changed, 89 insertions(+), 15

[PATCH 04/16] util/dsa: Implement DSA device start and stop logic.

2023-10-25 Thread Hao Xiang
* DSA device open and close. * DSA group contains multiple DSA devices. * DSA group configure/start/stop/clean. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang --- include/qemu/dsa.h | 49 +++ util/dsa.c | 338 + util/meson.build

[PATCH 08/16] util/dsa: Implement DSA task asynchronous submission and wait for completion.

2023-10-25 Thread Hao Xiang
* Add a DSA task completion callback. * DSA completion thread will call the tasks's completion callback on every task/batch task completion. * DSA submission path to wait for completion. * Implement CPU fallback if DSA is not able to complete the task. Signed-off-by: Hao Xiang Signed-off

Re: [External] Re: [PATCH 01/16] Cherry pick a set of patches that enables multifd zero page feature.

2023-10-27 Thread Hao Xiang
On Fri, Oct 27, 2023 at 5:30 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > Juan Quintela had a patchset enabling zero page checking in multifd > > threads. > > > > https://lore.kernel.org/all/20220802063907.18882-13-quint...@redhat.com/ > > Hmm, risk

Re: [External] Re: [PATCH 01/16] Cherry pick a set of patches that enables multifd zero page feature.

2023-10-27 Thread Hao Xiang
On Fri, Oct 27, 2023 at 6:22 AM Peter Maydell wrote: > > On Fri, 27 Oct 2023 at 13:32, Fabiano Rosas wrote: > > > > Hao Xiang writes: > > > > > Juan Quintela had a patchset enabling zero page checking in multifd > > > threads. > > > > &

Re: [External] Re: [PATCH 00/16] Use Intel DSA accelerator to offload zero page checking in multifd live migration.

2023-10-30 Thread Hao Xiang
On Mon, Oct 30, 2023 at 8:26 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > * Background: > > I posted an RFC about DSA offloading in QEMU: > > https://patchew.org/QEMU/20230529182001.2232069-1-hao.xi...@bytedance.com/ > > > > This patchset imp

Re: [External] Re: [PATCH 01/16] Cherry pick a set of patches that enables multifd zero page feature.

2023-11-06 Thread Hao Xiang
On Mon, Oct 30, 2023 at 6:58 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > On Fri, Oct 27, 2023 at 5:30 AM Fabiano Rosas wrote: > >> > >> Hao Xiang writes: > >> > >> > Juan Quintela had a patchset enabling zero page c

Re: [External] Re: [PATCH 09/16] migration/multifd: Add new migration option for multifd DSA offloading.

2023-11-06 Thread Hao Xiang
On Mon, Oct 30, 2023 at 7:41 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > Intel DSA offloading is an optional feature that turns on if > > proper hardware and software stack is available. To turn on > > DSA offloading in multifd live migration: > > >

Integrating with new Intel DSA hardware offloading

2023-01-18 Thread Hao Xiang .
Hi everyone, I am looking into using the new Data Streaming Accelerator (DSA) hardware acceleration feature built into the Intel Sapphire Rapids system. One of the scenarios I am investigating is the QEMU live migration feature. I noticed that during live migration, there is pretty busy work

Re: [PATCH 0/4] Add Intel Data Streaming Accelerator offloading

2023-05-29 Thread Hao Xiang
Hi all, this is meant to be an RFC. Sorry I didn't put that in the email subject correctly. From: "Hao Xiang" Date: Mon, May 29, 2023, 11:20 Subject: [PATCH 0/4] Add Intel Data Streaming Accelerator offloading To: , , Cc: "Hao Xiang" * Idea: Intel Data Streaming Accelerato

[PATCH 3/4] Implement zero page checking using DSA.

2023-05-29 Thread Hao Xiang
1. Adds a memory comparison function by submitting the work to the idxd driver. 2. Add interface to set bufferiszero accel function to DSA offloading. 3. Fallback to use CPU accel function if DSA offloading fails due to page fault. Signed-off-by: Hao Xiang --- include/qemu/cutils.h | 6

[PATCH 1/4] Introduce new instruction set enqcmd/mmovdir64b to the build system.

2023-05-29 Thread Hao Xiang
1. Enable instruction set enqcmd in build. 2. Enable instruction set movdir64b in build. Signed-off-by: Hao Xiang --- meson.build | 3 +++ meson_options.txt | 4 scripts/meson-buildoptions.sh | 6 ++ 3 files changed, 13 insertions(+) diff --git

[PATCH 4/4] Add QEMU command line argument to enable DSA offloading.

2023-05-29 Thread Hao Xiang
This change adds a new argument --dsa-accelerate to qemu. Signed-off-by: Hao Xiang --- qemu-options.hx | 10 ++ softmmu/runstate.c | 4 softmmu/vl.c | 22 ++ storage-daemon/qemu-storage-daemon.c

[PATCH 2/4] Add dependency idxd.

2023-05-29 Thread Hao Xiang
Idxd is the device driver for DSA (Intel Data Streaming Accelerator). The driver is fully functioning since Linux kernel 5.19. This change adds the driver's header file used for userspace development. Signed-off-by: Hao Xiang --- linux-headers/linux/idxd.h | 356

[PATCH 0/4] Add Intel Data Streaming Accelerator offloading

2023-05-29 Thread Hao Xiang
e interrupt for true async completion. 4. The DSA device can also offload other operations. * memcpy * xbzrle encoding/decoding * crc32 base-commit: ac84b57b4d74606f7f83667a0606deef32b2049d Hao Xiang (4): Introduce new instruction set enqcmd/mmovdir64b to the build system. Add dependency id

Configure no longer works after pulling in the latest QEMU commits

2023-05-18 Thread Hao Xiang
Hi, After pulling in the QEMU latest commits, I can no longer run the "configure" command. Below is the error message I am seeing. I believe this is related to this change "configure: create a python venv unconditionally". I am running on Debian GNU/Linux 11 (bullseye) and 5.15 kernel version.

Re: [External] Re: [PATCH 3/4] Implement zero page checking using DSA.

2023-05-30 Thread Hao Xiang
On Tue, May 30, 2023 at 2:26 AM Duan, Zhenzhong wrote: > > Hi Hao, > > > +memset(, 0, sizeof(descriptor)); > > + > > +descriptor.opcode = DSA_OPCODE_COMPARE; > > What about using DSA_OPCODE_COMPVAL with zero? > DSA_OPCODE_COMPARE consumes double bandwidth by reading > from both src and

Re: [External] Re: Configure no longer works after pulling in the latest QEMU commits

2023-05-19 Thread Hao Xiang
Hi Paolo, I have applied this patch locally and I am able to run "configure" now. From: "Paolo Bonzini" Date: Fri, May 19, 2023, 1:31 AM Subject: [External] Re: Configure no longer works after pulling in the latest QEMU commits To: "Hao Xiang", "qemu-devel@nongn

Re: [External] Re: [PATCH v3 01/20] multifd: Add capability to enable/disable zero_page

2024-01-23 Thread Hao Xiang
On Tue, Jan 23, 2024 at 7:11 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > On Sun, Jan 14, 2024 at 10:02 PM Shivam Kumar > > wrote: > >> > >> > >> > >> > On 04-Jan-2024, at 6:14 AM, Hao Xiang wrote: > >> > > &g

Re: [External] Re: [PATCH v3 03/20] multifd: Zero pages transmission

2024-01-22 Thread Hao Xiang
On Sun, Jan 14, 2024 at 11:01 PM Shivam Kumar wrote: > > > > > On 04-Jan-2024, at 6:14 AM, Hao Xiang wrote: > > > > From: Juan Quintela > > > > This implements the zero page dection and handling. > > > > Signed-off-by:

Re: [External] Re: [PATCH v3 01/20] multifd: Add capability to enable/disable zero_page

2024-01-22 Thread Hao Xiang
On Sun, Jan 14, 2024 at 10:02 PM Shivam Kumar wrote: > > > > > On 04-Jan-2024, at 6:14 AM, Hao Xiang wrote: > > > > From: Juan Quintela > > > > We have to enable it by default until we introduce the new code. > > > > Signed-off-by:

Re: [External] Re: [PATCH v2 04/20] So we use multifd to transmit zero pages.

2024-01-22 Thread Hao Xiang
On Thu, Nov 16, 2023 at 7:14 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > From: Juan Quintela > > > > Signed-off-by: Juan Quintela > > Reviewed-by: Leonardo Bras > > --- > > migration/multifd.c | 7 --- > > migration/options

Re: [External] Re: [PATCH v3 13/20] migration/multifd: Prepare to introduce DSA acceleration on the multifd path.

2024-01-22 Thread Hao Xiang
On Sun, Jan 14, 2024 at 10:46 PM Shivam Kumar wrote: > > > > > On 04-Jan-2024, at 6:14 AM, Hao Xiang wrote: > > > > 1. Refactor multifd_send_thread function. > > 2. Implement buffer_is_zero_use_cpu to handle CPU based zero page > > checking. &g

Re: [External] Re: [PATCH 3/6] migration/multifd: Support for zero pages transmission in multifd format.

2024-02-08 Thread Hao Xiang
On Tue, Feb 6, 2024 at 8:25 PM Peter Xu wrote: > > On Tue, Feb 06, 2024 at 11:19:05PM +0000, Hao Xiang wrote: > > diff --git a/migration/multifd.c b/migration/multifd.c > > index 25cbc6dc6b..a20d0ed10e 100644 > > --- a/migration/multifd.c > > +++ b/migration/

Re: [External] [PATCH v2 05/23] migration/multifd: Drop MultiFDSendParams.normal[] array

2024-02-08 Thread Hao Xiang
On Fri, Feb 2, 2024 at 2:30 AM wrote: > > From: Peter Xu > > This array is redundant when p->pages exists. Now we extended the life of > p->pages to the whole period where pending_job is set, it should be safe to > always use p->pages->offset[] rather than p->normal[]. Drop the array. > >

Re: [External] [PATCH v2 05/23] migration/multifd: Drop MultiFDSendParams.normal[] array

2024-02-13 Thread Hao Xiang
On Fri, Feb 9, 2024 at 4:20 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > On Fri, Feb 2, 2024 at 2:30 AM wrote: > >> > >> From: Peter Xu > >> > >> This array is redundant when p->pages exists. Now we extended the life of >

[PATCH v2 2/7] migration/multifd: Support for zero pages transmission in multifd format.

2024-02-16 Thread Hao Xiang
This change adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang --- migration/multifd.c| 43 ++ migration/multifd.h| 21 - migration/ram.c

[PATCH v2 0/7] Introduce multifd zero page checking.

2024-02-16 Thread Hao Xiang
+ |multifd-2-threads | 4643 | ++ |multifd-4-threads | 4143 | ++ Apply this patchset on top of commit 5767815218efd3cbfd409505ed824d5f356044ae Hao Xiang (7): migration/multifd: Add new migration option zero-page-detectio

[PATCH v2 4/7] migration/multifd: Enable zero page checking from multifd threads.

2024-02-16 Thread Hao Xiang
main thread. Signed-off-by: Hao Xiang --- migration/multifd.c | 1 + migration/options.c | 2 +- migration/ram.c | 53 ++--- 3 files changed, 42 insertions(+), 14 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fbb40ea10b

[PATCH v2 6/7] migration/multifd: Add zero pages and zero bytes counter to migration status interface.

2024-02-16 Thread Hao Xiang
This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 migration/migration.c | 2 ++ qapi/migration.json | 15 ++- tests/migration

[PATCH v2 1/7] migration/multifd: Add new migration option zero-page-detection.

2024-02-16 Thread Hao Xiang
This new parameter controls where the zero page checking is running. 1. If this parameter is set to 'legacy', zero page checking is done in the migration main thread. 2. If this parameter is set to 'none', zero page checking is disabled. Signed-off-by: Hao Xiang --- hw/core/qdev-properties

[PATCH v2 5/7] migration/multifd: Add new migration test cases for legacy zero page checking.

2024-02-16 Thread Hao Xiang
migration main thread. Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 52 1 file changed, 52 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 8a5bb1752e..c27083110a 100644 --- a/tests/qtest/migration-te

[PATCH v2 3/7] migration/multifd: Zero page transmission on the multifd thread.

2024-02-16 Thread Hao Xiang
off-by: Hao Xiang --- migration/meson.build | 1 + migration/multifd-zero-page.c | 59 +++ migration/multifd-zlib.c | 26 --- migration/multifd-zstd.c | 25 --- migration/multifd.c |

[PATCH v2 7/7] Update maintainer contact for migration multifd zero page checking acceleration.

2024-02-16 Thread Hao Xiang
Add myself to maintain multifd zero page checking acceleration function. Signed-off-by: Hao Xiang --- MAINTAINERS | 5 + 1 file changed, 5 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a24c2b51b6..3ca407cb58 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3403,6 +3403,11 @@ F

[PATCH 6/6] migration/multifd: Add a new migration test case for legacy zero page checking.

2024-02-06 Thread Hao Xiang
thread. Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 26 ++ 1 file changed, 26 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 7675519cfa..2c13df04c3 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest

[PATCH 3/6] migration/multifd: Support for zero pages transmission in multifd format.

2024-02-06 Thread Hao Xiang
This change adds zero page counters and updates multifd send/receive tracing format to track the newly added counters. Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 4 migration/multifd.c| 43 ++ migration/multifd.h

[PATCH 4/6] migration/multifd: Zero page transmission on the multifd thread.

2024-02-06 Thread Hao Xiang
This implements the zero page detection and handling on the multifd threads. Signed-off-by: Hao Xiang --- migration/multifd.c | 62 + migration/multifd.h | 5 2 files changed, 62 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b

[PATCH 1/6] migration/multifd: Add new migration option multifd-zero-page.

2024-02-06 Thread Hao Xiang
This new parameter controls where the zero page checking is running. If this parameter is set to true, zero page checking is done in the multifd sender threads. If this parameter is set to false, zero page checking is done in the migration main thread. Signed-off-by: Hao Xiang --- migration

[PATCH 0/6] Introduce multifd zero page checking.

2024-02-06 Thread Hao Xiang
| 21.53GB/s | +--+ |multifd-4-threads | 4143 | 24.13GB/s | +--+ Apply this patchset on top of commit 39a6e4f87e7b75a45b08d6dc8b8b7c2954c87440 Hao Xiang (6): migration/multifd: Add new migration

[PATCH 2/6] migration/multifd: Add zero pages and zero bytes counter to migration status interface.

2024-02-06 Thread Hao Xiang
This change extends the MigrationStatus interface to track zero pages and zero bytes counter. Signed-off-by: Hao Xiang --- qapi/migration.json | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index ff033a0344

[PATCH 5/6] migration/multifd: Enable zero page checking from multifd threads.

2024-02-06 Thread Hao Xiang
main thread. Signed-off-by: Hao Xiang --- migration/multifd.c | 3 ++- migration/ram.c | 49 - 2 files changed, 42 insertions(+), 10 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index c031f947c7..c6833ccb07 100644

Re: [External] Re: Re: [PATCH 2/6] migration/multifd: Add zero pages and zero bytes counter to migration status interface.

2024-02-07 Thread Hao Xiang
On Wed, Feb 7, 2024 at 12:41 AM Jiri Denemark wrote: > > On Wed, Feb 07, 2024 at 12:37:15 +0800, Peter Xu wrote: > > On Wed, Feb 07, 2024 at 12:13:10PM +0800, Peter Xu wrote: > > > On Tue, Feb 06, 2024 at 11:19:04PM +, Hao Xiang wrote: > > > > This change exte

Re: [External] Re: [PATCH 0/6] Introduce multifd zero page checking.

2024-02-07 Thread Hao Xiang
On Tue, Feb 6, 2024 at 7:39 PM Peter Xu wrote: > > On Tue, Feb 06, 2024 at 11:19:02PM +0000, Hao Xiang wrote: > > This patchset is based on Juan Quintela's old series here > > https://lore.kernel.org/all/20220802063907.18882-1-quint...@redhat.com/ > > > > In th

Re: [External] Re: [PATCH 1/6] migration/multifd: Add new migration option multifd-zero-page.

2024-02-07 Thread Hao Xiang
On Tue, Feb 6, 2024 at 7:45 PM Peter Xu wrote: > > On Tue, Feb 06, 2024 at 11:19:03PM +0000, Hao Xiang wrote: > > diff --git a/qapi/migration.json b/qapi/migration.json > > index 819708321d..ff033a0344 100644 > > --- a/qapi/migration.json > > +++ b/qapi/migrat

Re: [External] RE: Regarding to the recent Intel IAA/DSA/QAT support on migration

2024-02-07 Thread Hao Xiang
riginal Message- > > From: Peter Xu > > Sent: Wednesday, February 7, 2024 4:10 PM > > To: Bryan Zhang ; Hao Xiang > > ; Liu, Yuan1 > > Cc: Fabiano Rosas ; QEMU Devel Mailing List > de...@nongnu.org> > > Subject: Regarding to the recent Intel IAA/DSA/QAT

Re: [External] Re: [PATCH v2 09/20] util/dsa: Implement DSA task asynchronous completion thread model.

2023-12-18 Thread Hao Xiang
On Sun, Dec 17, 2023 at 7:11 PM Wang, Lei wrote: > > On 11/14/2023 13:40, Hao Xiang wrote:> * Create a dedicated thread for DSA > task > completion. > > * DSA completion thread runs a loop and poll for completed tasks. > > * Start and stop DSA completion thread d

Re: [External] Re: [PATCH v2 12/20] migration/multifd: Add new migration option for multifd DSA offloading.

2023-12-18 Thread Hao Xiang
On Mon, Dec 11, 2023 at 11:44 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > Intel DSA offloading is an optional feature that turns on if > > proper hardware and software stack is available. To turn on > > DSA offloading in multifd live migration: > > >

Re: [External] Re: [PATCH v2 09/20] util/dsa: Implement DSA task asynchronous completion thread model.

2023-12-18 Thread Hao Xiang
On Mon, Dec 18, 2023 at 5:34 PM Wang, Lei wrote: > > On 12/19/2023 2:57, Hao Xiang wrote:> On Sun, Dec 17, 2023 at 7:11 PM Wang, > Lei > wrote: > >> > >> On 11/14/2023 13:40, Hao Xiang wrote:> * Create a dedicated thread for DSA > >> task > &

Re: [External] Re: [PATCH v2 07/20] util/dsa: Implement DSA device start and stop logic.

2023-12-18 Thread Hao Xiang
On Mon, Dec 11, 2023 at 1:28 PM Fabiano Rosas wrote: > > Hao Xiang writes: > > > * DSA device open and close. > > * DSA group contains multiple DSA devices. > > * DSA group configure/start/stop/clean. > > > > Signed-off-by: Hao Xiang > > Signed-o

Re: [External] Re: [PATCH v2 05/20] meson: Introduce new instruction set enqcmd to the build system.

2023-12-15 Thread Hao Xiang
On Mon, Dec 11, 2023 at 7:41 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > Enable instruction set enqcmd in build. > > > > Signed-off-by: Hao Xiang > > --- > > meson.build | 2 ++ > > meson_options.txt | 2 ++ >

Re: [External] Re: [PATCH v2 08/20] util/dsa: Implement DSA task enqueue and dequeue.

2023-12-26 Thread Hao Xiang
On Tue, Dec 12, 2023 at 8:10 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > * Use a safe thread queue for DSA task enqueue/dequeue. > > * Implement DSA task submission. > > * Implement DSA batch task submission. > > > > Signed-off-by: Hao Xiang

Re: [External] Re: [PATCH v2 07/20] util/dsa: Implement DSA device start and stop logic.

2023-12-26 Thread Hao Xiang
On Tue, Dec 19, 2023 at 5:19 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > >> > >> > +} > >> > + > >> > +void dsa_start(void) {} > >> > + > >> > +void dsa_stop(void) {} > >> > + > >&

Re: [External] Re: [PATCH v2 11/20] util/dsa: Implement DSA task asynchronous submission and wait for completion.

2023-12-26 Thread Hao Xiang
On Wed, Dec 13, 2023 at 6:01 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > * Add a DSA task completion callback. > > * DSA completion thread will call the tasks's completion callback > > on every task/batch task completion. > > * DSA submission path to wai

Re: [External] Re: [PATCH v2 18/20] migration/multifd: Enable set packet size migration option.

2024-01-03 Thread Hao Xiang
On Wed, Dec 13, 2023 at 9:33 AM Fabiano Rosas wrote: > > Hao Xiang writes: > > > During live migration, if the latency between sender and receiver > > is high but bandwidth is high (a long and fat pipe), using a bigger > > packet size can help reduce migration tot

[PATCH v3 03/20] multifd: Zero pages transmission

2024-01-03 Thread Hao Xiang
From: Juan Quintela This implements the zero page dection and handling. Signed-off-by: Juan Quintela --- migration/multifd.c | 41 +++-- migration/multifd.h | 5 + 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/migration/multifd.c

[PATCH v3 12/20] migration/multifd: Add new migration option for multifd DSA offloading.

2024-01-03 Thread Hao Xiang
Intel DSA offloading is an optional feature that turns on if proper hardware and software stack is available. To turn on DSA offloading in multifd live migration: multifd-dsa-accel="[dsa_dev_path1] [dsa_dev_path2] ... [dsa_dev_pathX]" This feature is turned off by default. Signed-o

[PATCH v3 13/20] migration/multifd: Prepare to introduce DSA acceleration on the multifd path.

2024-01-03 Thread Hao Xiang
1. Refactor multifd_send_thread function. 2. Implement buffer_is_zero_use_cpu to handle CPU based zero page checking. 3. Introduce the batch task structure in MultiFDSendParams. Signed-off-by: Hao Xiang --- include/qemu/dsa.h | 43 +++-- migration/multifd.c | 77

[PATCH v3 08/20] util/dsa: Implement DSA task enqueue and dequeue.

2024-01-03 Thread Hao Xiang
* Use a safe thread queue for DSA task enqueue/dequeue. * Implement DSA task submission. * Implement DSA batch task submission. Signed-off-by: Hao Xiang --- include/qemu/dsa.h | 28 +++ util/dsa.c | 201 + 2 files changed, 229 insertions

[PATCH v3 05/20] meson: Introduce new instruction set enqcmd to the build system.

2024-01-03 Thread Hao Xiang
Enable instruction set enqcmd in build. Signed-off-by: Hao Xiang --- meson.build | 14 ++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 19 insertions(+) diff --git a/meson.build b/meson.build index 6c77d9687d

[PATCH v3 04/20] So we use multifd to transmit zero pages.

2024-01-03 Thread Hao Xiang
From: Juan Quintela Signed-off-by: Juan Quintela Reviewed-by: Leonardo Bras --- migration/multifd.c | 7 --- migration/options.c | 17 + migration/ram.c | 45 ++--- qapi/migration.json | 1 - 4 files changed, 51 insertions(+),

[PATCH v3 19/20] util/dsa: Add unit test coverage for Intel DSA task submission and completion.

2024-01-03 Thread Hao Xiang
* Test DSA start and stop path. * Test DSA configure and cleanup path. * Test DSA task submission and completion path. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- tests/unit/meson.build | 6 + tests/unit/test-dsa.c | 475 + 2 files changed

[PATCH v3 01/20] multifd: Add capability to enable/disable zero_page

2024-01-03 Thread Hao Xiang
From: Juan Quintela We have to enable it by default until we introduce the new code. Signed-off-by: Juan Quintela --- migration/options.c | 15 +++ migration/options.h | 1 + qapi/migration.json | 8 +++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git

[PATCH v3 18/20] migration/multifd: Enable set packet size migration option.

2024-01-03 Thread Hao Xiang
the packet size is useful for performance tuning. Set the option: migrate_set_parameter multifd-packet-size 4190208 Signed-off-by: Hao Xiang --- migration/migration-hmp-cmds.c | 7 +++ migration/multifd-zlib.c | 6 -- migration/multifd-zstd.c | 6 -- migration/multifd.c

[PATCH v3 11/20] util/dsa: Implement DSA task asynchronous submission and wait for completion.

2024-01-03 Thread Hao Xiang
* Add a DSA task completion callback. * DSA completion thread will call the tasks's completion callback on every task/batch task completion. * DSA submission path to wait for completion. * Implement CPU fallback if DSA is not able to complete the task. Signed-off-by: Hao Xiang Signed-off

[PATCH v3 00/20] Use Intel DSA accelerator to offload zero page checking in multifd live migration.

2024-01-03 Thread Hao Xiang
Apply this patchset on top of commit 7425b6277f12e82952cede1f531bfc689bf77fb1 Hao Xiang (16): meson: Introduce new instruction set enqcmd to the build system. util/dsa: Add dependency idxd. util/dsa: Implement DSA device start and stop logic. util/dsa: Implement DSA task enqueue and dequeue.

[PATCH v3 10/20] util/dsa: Implement zero page checking in DSA task.

2024-01-03 Thread Hao Xiang
at least two individual tasks. There are times we need to submit a single task and hence a single DSA task submission is also required. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang --- include/qemu/dsa.h | 18 util/dsa.c | 247 + 2

[PATCH v3 14/20] migration/multifd: Enable DSA offloading in multifd sender path.

2024-01-03 Thread Hao Xiang
. The sender thread submits a batch of pages to DSA hardware and waits for the DSA completion thread to signal for work completion. Signed-off-by: Hao Xiang --- migration/multifd.c | 51 - 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/migration

[PATCH v3 20/20] migration/multifd: Add integration tests for multifd with Intel DSA offloading.

2024-01-03 Thread Hao Xiang
* Add test case to start and complete multifd live migration with DSA offloading enabled. * Add test case to start and cancel multifd live migration with DSA offloading enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang --- tests/qtest/migration-test.c | 77

[PATCH v3 15/20] migration/multifd: Add test hook to set normal page ratio.

2024-01-03 Thread Hao Xiang
to be sent as a normal page. This is useful for live migration performance analysis and optimization. Signed-off-by: Hao Xiang --- migration/options.c | 32 migration/options.h | 1 + qapi/migration.json | 18 +++--- 3 files changed, 48 insertions(+), 3

[PATCH v3 06/20] util/dsa: Add dependency idxd.

2024-01-03 Thread Hao Xiang
Idxd is the device driver for DSA (Intel Data Streaming Accelerator). The driver is fully functioning since Linux kernel 5.19. This change adds the driver's header file used for userspace development. Signed-off-by: Hao Xiang --- linux-headers/linux/idxd.h | 356

[PATCH v3 02/20] multifd: Support for zero pages transmission

2024-01-03 Thread Hao Xiang
From: Juan Quintela This patch adds counters and similar. Logic will be added on the following patch. Signed-off-by: Juan Quintela --- migration/multifd.c| 37 ++--- migration/multifd.h| 17 - migration/trace-events | 8 3

[PATCH v3 09/20] util/dsa: Implement DSA task asynchronous completion thread model.

2024-01-03 Thread Hao Xiang
. Signed-off-by: Hao Xiang --- include/qemu/dsa.h | 1 + util/dsa.c | 274 - 2 files changed, 274 insertions(+), 1 deletion(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 37cae8d9d2..2513192a2b 100644 --- a/include/qemu/dsa.h +++ b

[PATCH v3 17/20] migration/multifd: Add migration option set packet size.

2024-01-03 Thread Hao Xiang
The current multifd packet size is 128 * 4kb. This change adds an option to set the packet size. Both sender and receiver needs to set the same packet size for things to work. Signed-off-by: Hao Xiang --- migration/options.c | 36 migration/options.h | 1

[PATCH v3 16/20] migration/multifd: Enable set normal page ratio test hook in multifd.

2024-01-03 Thread Hao Xiang
Test hook is disabled by default. To set it, a normal page ratio between 0 and 100 are valid. If the ratio is set to 50, it means at least 50% of all pages are sent as normal pages. Set the option: migrate_set_parameter multifd-normal-page-ratio 60 Signed-off-by: Hao Xiang --- include/qemu

[PATCH v3 07/20] util/dsa: Implement DSA device start and stop logic.

2024-01-03 Thread Hao Xiang
* DSA device open and close. * DSA group contains multiple DSA devices. * DSA group configure/start/stop/clean. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang --- include/qemu/dsa.h | 72 +++ util/dsa.c | 316 + util/meson.build

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-08 Thread Hao Xiang
On Mon, Jan 8, 2024 at 9:15 AM Gregory Price wrote: > > On Fri, Jan 05, 2024 at 09:59:19PM -0800, Hao Xiang wrote: > > On Wed, Jan 3, 2024 at 1:56 PM Gregory Price > > wrote: > > > > > > For a variety of performance reasons, this will not work the way you &g

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-08 Thread Hao Xiang
On Mon, Jan 8, 2024 at 2:47 PM Hao Xiang wrote: > > On Mon, Jan 8, 2024 at 9:15 AM Gregory Price > wrote: > > > > On Fri, Jan 05, 2024 at 09:59:19PM -0800, Hao Xiang wrote: > > > On Wed, Jan 3, 2024 at 1:56 PM Gregory Price > > > wrote: > > >

Re: [External] Re: [PATCH 3/5] migration: Introduce unimplemented 'qatzip' compression method

2024-01-10 Thread Hao Xiang
On Mon, Jan 8, 2024 at 6:26 PM Liu, Yuan1 wrote: > > > -Original Message- > > From: Fabiano Rosas > > Sent: Tuesday, January 9, 2024 4:28 AM > > To: Liu, Yuan1 ; Hao Xiang > > Cc: Bryan Zhang ; qemu-devel@nongnu.org; > > marcandre.lur.

Re: [External] Re: [PATCH v3 01/20] multifd: Add capability to enable/disable zero_page

2024-01-10 Thread Hao Xiang
On Mon, Jan 8, 2024 at 12:39 PM Fabiano Rosas wrote: > > Hao Xiang writes: > > > From: Juan Quintela > > > > We have to enable it by default until we introduce the new code. > > > > Signed-off-by: Juan Quintela > > --- > > migration/options.c

Re: [External] Re: [PATCH 3/5] migration: Introduce unimplemented 'qatzip' compression method

2024-01-10 Thread Hao Xiang
On Mon, Jan 8, 2024 at 12:28 PM Fabiano Rosas wrote: > > "Liu, Yuan1" writes: > > >> -Original Message- > >> From: Hao Xiang > >> Sent: Saturday, January 6, 2024 7:53 AM > >> To: Fabiano Rosas > >> Cc: Bryan Zhan

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-09 Thread Hao Xiang
On Mon, Jan 8, 2024 at 5:13 PM Gregory Price wrote: > > On Mon, Jan 08, 2024 at 05:05:38PM -0800, Hao Xiang wrote: > > On Mon, Jan 8, 2024 at 2:47 PM Hao Xiang wrote: > > > > > > On Mon, Jan 8, 2024 at 9:15 AM Gregory Price > > > wrote: > > > &g

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-09 Thread Hao Xiang
On Tue, Jan 9, 2024 at 11:58 AM Gregory Price wrote: > > On Tue, Jan 09, 2024 at 11:33:04AM -0800, Hao Xiang wrote: > > On Mon, Jan 8, 2024 at 5:13 PM Gregory Price > > wrote: > > > > Sounds like the technical details are explained on the other thread. > &g

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-09 Thread Hao Xiang
On Tue, Jan 9, 2024 at 2:13 PM Gregory Price wrote: > > On Tue, Jan 09, 2024 at 01:27:28PM -0800, Hao Xiang wrote: > > On Tue, Jan 9, 2024 at 11:58 AM Gregory Price > > wrote: > > > > > > If you drop this line: > > > > > > -numa no

Re: [External] Re: [PATCH 3/5] migration: Introduce unimplemented 'qatzip' compression method

2024-01-05 Thread Hao Xiang
methods effectively unimplemented. This is in preparation of a > > subsequent commit that will implement actually using QAT for compression > > and decompression. > > > > Signed-off-by: Bryan Zhang > > Signed-off-by: Hao Xiang > > --- > > hw/core/q

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-05 Thread Hao Xiang
On Tue, Jan 2, 2024 at 5:04 AM David Hildenbrand wrote: > > On 01.01.24 08:53, Ho-Ren (Jack) Chuang wrote: > > Introduce a new configuration option 'host-mem-type=' in the > > '-object memory-backend-ram', allowing users to specify > > from which type of memory to allocate. > > > > Users can

Re: [External] Re: [QEMU-devel][RFC PATCH 1/1] backends/hostmem: qapi/qom: Add an ObjectOption for memory-backend-* called HostMemType and its arg 'cxlram'

2024-01-05 Thread Hao Xiang
On Wed, Jan 3, 2024 at 1:56 PM Gregory Price wrote: > > On Sun, Dec 31, 2023 at 11:53:15PM -0800, Ho-Ren (Jack) Chuang wrote: > > Introduce a new configuration option 'host-mem-type=' in the > > '-object memory-backend-ram', allowing users to specify > > from which type of memory to allocate. > >

  1   2   3   >