This patch defines the structure for IAA jobs related to data
compression and decompression, as well as the initialization and
deinitialization processes for IAA.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/iaa-ram-compress.c | 152
for data (de)compression
using IAA during the live migration process.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 9 -
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 13 insertions(+), 1 deletion(-)
diff
Refactor legacy RAM compression functions to support both IAA
compression and CPU compression.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/migration.c| 6 +--
migration/ram-compress.c | 81
migration/ram-compress.h | 10
compress on. If the migration compression
capability is enabled and the IAA compression parameter is set, IAA will
be used instead of CPU for data (de)compression.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/migration-hmp-cmds.c | 8
migration/options.c| 20
Implement the functions of IAA for data compression and decompression.
The implementation uses non-blocking job submission and polling to check
the job completion status to reduce IAA's overhead in the live migration
process.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/iaa
, resulting in performance gains.
The implementation of the IAA (de)compression code is based on Intel Query
Processing Library (QPL), an open-source software project designed for
IAA high-level software programming.
Best regards,
Yuan Liu
Yuan Liu (5):
configure: add qpl meson option
qapi
the Zlib compression algorithm. QPL can automatically choose
software or hardware acceleration based on the platform.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/meson.build | 1 +
migration/multifd-qpl.c | 326
2 files changed, 327
, the acceleration function can be disabled.
Similarly, users can explicitly specify a specific accelerator name, such
as multifd-compression-accel=qpl.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c| 11 +++
include/hw/qdev-properties-system.h | 4
in the multifd
compression accelerator
- fixed the issue that QPL was compiled into the migration
module by default
Yuan Liu (4):
migration: Introduce multifd-compression-accel parameter
multifd: Implement multifd compression accelerator
configure: add qpl option
multifd: Introduce
when starting multifd live migration, if the compression method is
enabled, compression method can be accelerated using accelerators.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd.c | 38 --
migration/multifd.h | 8
2 files
during the live migration.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 7 +++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 12 insertions(+)
diff --git a/meson.build b/meson.build
index 259dc5f308
, the acceleration function can be disabled.
Similarly, users can explicitly specify a specific accelerator name, such
as multifd-compression-accel=qpl.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c| 11 ++
include/hw/qdev-properties-system.h | 4
when starting multifd live migration, if the compression method is
enabled, compression method can be accelerated using accelerators.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
---
migration/multifd.c | 40 ++--
migration
during the live migration.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 18 ++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 3 +++
3 files changed, 23 insertions(+)
diff --git a/meson.build b/meson.build
index
in multifd.c file
Yuan Liu (4):
migration: Introduce multifd-compression-accel parameter
multifd: Implement multifd compression accelerator
configure: add qpl option
multifd: Introduce QPL compression accelerator
hw/core/qdev-properties-system.c| 11 +
include/hw/qdev-properties
the Zlib compression algorithm. QPL can automatically choose
software or hardware acceleration based on the platform.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/meson.build | 1 +
migration/multifd-qpl.c | 323
2 files changed, 324
.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
hw/core/qdev-properties-system.c | 2 +-
migration/meson.build| 1 +
migration/multifd-qpl.c | 20
migration/multifd.h | 1 +
qapi/migration.json | 7 ++-
5 files changed
-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 284 +++-
1 file changed, 280 insertions(+), 4 deletions(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 89fa51091a..9a1fddbdd0 100644
--- a/migration/multifd-qpl.c
+++ b
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 8
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 262 +++
2 files changed, 263 insertions(+)
create mode
-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 5d6d8cd634..0f75ed7c49 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest
is not
available, the qpl compression will fallback to the software path.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 272 +++-
1 file changed, 271 insertions(+), 1 deletion(-)
diff --git a/migration/multifd-qpl.c b/migration
add qpl to compression method test for multifd migration
the migration with qpl compression needs to access IAA hardware
resource, please run "check-qtest" with sudo or root permission,
otherwise migration test will fail
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
t
compression initialization will fail.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 243 +++-
1 file changed, 242 insertions(+), 1 deletion(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 056a68a060
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 231 +++
2 files changed, 232 insertions(+)
create mode
method for migration.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
The qpl only supports one compression level, there is no qpl
compression level parameter added, users do not need to specify
the qpl compression level.
Signed-off-by: Yuan Liu
algorithm
and use Intel In-Memory Analytics Accelerator(IAA) hardware for
compression and decompression acceleration.
Please refer to the following for more information about QPL
https://intel.github.io/qpl/documentation/introduction_docs/introduction.html
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration
each qpl job is used to (de)compress a normal page and it can
be processed independently by the IAA hardware. All qpl jobs
are submitted to the hardware at once, and wait for all jobs
completion.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 229
el.org/all/CAAYibXib+TWnJpV22E=adncdBmwXJRqgRjJXK7X71J=bdfa...@mail.gmail.com/
2.
https://lore.kernel.org/all/ph7pr11mb594123f7eefebfce219af100a3...@ph7pr11mb5941.namprd11.prod.outlook.com/
Yuan Liu (1):
migration/multifd: solve zero page causing multiple page faults
migration/multifd-zero-
it is
unlikely that the zero page will be migrated multiple times.
Signed-off-by: Yuan Liu
---
migration/multifd-zero-page.c | 4 +++-
migration/multifd-zlib.c | 1 +
migration/multifd-zstd.c | 1 +
migration/multifd.c | 1 +
migration/ram.c | 4
migration/ram.h
add QPL compression method introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 231 +++
2 files changed, 232 insertions(+)
create mode 100644 docs/devel/migration/qpl
algorithm
and use Intel In-Memory Analytics Accelerator(IAA) hardware for
compression and decompression acceleration.
Please refer to the following for more information about QPL
https://intel.github.io/qpl/documentation/introduction_docs/introduction.html
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai
method for migration.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
The qpl only supports one compression level, there is no qpl
compression level parameter added, users do not need to specify
the qpl compression level.
Signed-off-by: Yuan Liu
add qpl to compression method test for multifd migration
the migration with qpl compression needs to access IAA hardware
resource, please run "check-qtest" with sudo or root permission,
otherwise migration test will fail
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
t
add zlib and zstd compression levels in multifd parameter
testing and application and add compression level tests
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reported-by: Xiaohui Li
---
migration/options.c | 12
tests/qtest/migration-test.c | 16
2
this data
block. For no compression, each IOV is used to send a page, so the
number of IOVs required is the same as the number of pages.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-zlib.c | 18 +-
migration/multifd-zstd.c | 18 +-
migration
each qpl job is used to (de)compress a normal page and it can
be processed independently by the IAA hardware. All qpl jobs
are submitted to the hardware at once, and wait for all jobs
completion.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 219
compression initialization will fail.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd-qpl.c | 128 ++--
1 file changed, 122 insertions(+), 6 deletions(-)
diff --git a/migration/multifd-qpl.c b/migration/multifd-qpl.c
index 6b94e732ac
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
meson.build | 8
compression and decompression CPU overhead.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
There is no qpl compression level parameter added since it only supports
level one, users do not need to specify the qpl compression level.
Signed-off-by: Yuan Liu
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 262 +++
2 files changed, 263 insertions(+)
create mode
during initialization, a software job is allocated to each channel
for software path fallabck when the IAA hardware is unavailable or
the hardware job submission fails. If the IAA hardware is available,
multiple hardware jobs are allocated for batch processing.
Signed-off-by: Yuan Liu
Reviewed
-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Peter Xu
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b7e3406471..ef0c3f5e28 100644
--- a/tests/qtest/migration-test.c
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed
QPL compression and decompression will use IAA hardware first.
If IAA hardware is not available, it will automatically fall
back to QPL software path, if the software job also fails,
the uncompressed page is sent directly.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
---
migration/multifd
add Intel Query Processing Library (QPL) compression method
introduction
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
Acked-by: Peter Xu
---
docs/devel/migration/features.rst| 1 +
docs/devel/migration/qpl-compression.rst | 260
compression and decompression CPU overhead.
How to enable qpl compression during migration:
migrate_set_parameter multifd-compression qpl
There is no qpl compression level parameter added since it only supports
level one, users do not need to specify the qpl compression level.
Signed-off-by: Yuan Liu
In-Memory Analytics Accelerator(IAA) hardware for compression
and decompression acceleration.
For more live migration with IAA, please refer to the document
docs/devel/migration/qpl-compression.rst
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Fabiano Rosas
---
meson.build
-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed-by: Peter Xu
Reviewed-by: Fabiano Rosas
---
tests/qtest/migration-test.c | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index b7e3406471..ef0c3f5e28 100644
Different compression methods may require different numbers of IOVs.
Based on streaming compression of zlib and zstd, all pages will be
compressed to a data block, so two IOVs are needed for packet header
and compressed data block.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
Reviewed
during initialization, a software job is allocated to each channel
for software path fallabck when the IAA hardware is unavailable or
the hardware job submission fails. If the IAA hardware is available,
multiple hardware jobs are allocated for batch processing.
Signed-off-by: Yuan Liu
Reviewed
. No fallback, always wait for work queues to become available
total time: 18381 ms
downtime: 25 ms
throughput: 13698.65 mbps
pages-per-second: 859607
If both the hardware and software paths fail, the uncompressed page is
sent directly.
Signed-off-by: Yuan Liu
Reviewed-by: Nanhai Zou
54 matches
Mail list logo
