[PATCH] MAINTAINERS: Remove myself as reviewer from Live Migration

2023-12-21 Thread Leonardo Bras
I am currently focusing in kernel development, so I will probably not be of much help in reviewing general Live Migration changes. For above reason I am removing my Reviewer status from Migration and RDMA Migration. Signed-off-by: Leonardo Bras --- MAINTAINERS | 2 -- 1 file changed, 2

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-10 Thread Leonardo Bras Soares Passos
On Mon, Jul 10, 2023 at 3:16 PM Michael S. Tsirkin wrote: > > On Mon, Jul 10, 2023 at 02:49:05PM -0300, Leonardo Bras Soares Passos wrote: > > On Thu, Jul 6, 2023 at 5:00 PM Michael S. Tsirkin wrote: > > > > > > On Thu, Jul 06, 2023 at 03:02:07PM -0400, Peter Xu

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-10 Thread Leonardo Bras Soares Passos
n Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos > > > > wrote: > > > > > > I asked the same question, and I still keep confused: whether > > > > > > there's a > > > > > > first bad commit? Starting from whe

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 3:24 PM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos wrote: > > > I asked the same question, and I still keep confused: whether there's a > > > first bad commit? Starting from when it fails?

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 3:24 PM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos wrote: > > > I asked the same question, and I still keep confused: whether there's a > > > first bad commit? Starting from when it fails?

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 11:35 AM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 01:55:47AM -0300, Leonardo Bras wrote: > > When trying to migrate a machine type pc-q35-6.0 or lower, with this > > cmdline options, > > > > -device > > driver=pcie-root-port,port=1

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 4:37 AM Juan Quintela wrote: > > Leonardo Bras wrote: > > When trying to migrate a machine type pc-q35-6.0 or lower, with this > > cmdline options, > > > > -device > > driver=pcie-root-port,port=18,chassis=19,id=pcie-root-port18,bus

[PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-05 Thread Leonardo Bras
So, clear PCI_EXP_SLTSTA_PDS bit on cmask, so the fake incompatibility on get_pci_config_device() does not abort the migration. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215819 Signed-off-by: Leonardo Bras --- hw/pci/pcie.c | 4 1 file changed, 4 insertions(+) diff --git a/hw

Re: [PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-07-05 Thread Leonardo Bras Soares Passos
On Wed, Jul 5, 2023 at 3:40 AM Leonardo Bras Soares Passos wrote: > > On Tue, Jul 4, 2023 at 3:43 AM Michael S. Tsirkin wrote: > > > > On Tue, Jul 04, 2023 at 03:20:36AM -0300, Leonardo Brás wrote: > > > Hello Peter and Michael, I have a few updates on this: > >

Re: [PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-07-05 Thread Leonardo Bras Soares Passos
> > > > > > Thanks for figuring this out. Let me copy a few more potential > > > > > > > > reviewers > > > > > > > > from commit 17858a1695 ("hw/acpi/ich9: Set ACPI PCI hot-plug as > > > > > > >

[PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-06-29 Thread Leonardo Bras
he fake incompatibility on get_pci_config_device() does not abort the migration. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215819 Signed-off-by: Leonardo Bras --- hw/pci/pcie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index b8c24cf45f..2def176

Re: [PATCH v2 16/16] migration/multifd: Compute transferred bytes correctly

2023-05-26 Thread Leonardo Bras Soares Passos
> > Humm, those are atomic ops, right? > > > > You think we could have 'multifd_bytes' and 'transferred' in the same > > cacheline, > > to avoid 2 cacheline bounces? > > Don't matter on next series. > > mig_stats.transferred is dropped. > > And transferred becomes: > > qemu_file_transferred + multifd_bytes + rdma_bytes. > > So everytime that we do a write, we only update one counter. That's even better :) Thanks! > > > Well, it's unrelated to this patchset, so: > > > > Reviewed-by: Leonardo Bras >

Re: [PATCH v2 15/16] migration/rdma: Simplify the function that saves a page

2023-05-26 Thread Leonardo Bras Soares Passos
* the current chunk. So, we have to call qemu_update_position() > >> - * later on when the actual write occurs. > >> - */ > >> -if (bytes_sent) { > >> -*bytes_sent = 1; > >> -} > >> - > >> /* > >> * Drain the Completion Queue if possible, but do not block, > >> * just poll. > > > > Oh, so this one complements 13/16. > > Since it doesn't do imaginary transfers anymore, there is no need to use > > bytes_sent pointer to keep track of them anymore. > > > > Other than the pages_sent above that I couldn't understand: > > Reviewed-by: Leonardo Bras > > Dropping that bit. > > Thanks. >

Re: [PATCH v2 09/16] migration: We don't need the field rate_limit_used anymore

2023-05-26 Thread Leonardo Bras Soares Passos
gt; Already in tree. > > See explanation for the split on previous patch. > > The other reason for the split is that this part of the patch is trivial > to review O:-) :-) > > Later, Juan. > > > > > Reviewed-by: Leonardo Bras >

Re: [PATCH v2 08/16] migration: Use migration_transferred_bytes() to calculate rate_limit

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:17 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:57 +0200, Juan Quintela wrote: > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Cédric Le Goater > >> --- > >> migration/migration-stats.h | 8 +++- > >>

Re: [PATCH v2 04/16] qemu-file: Account for rate_limit usage on qemu_fflush()

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:09 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> That is the moment we know we have transferred something. > >> > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Cédric Le Goater > >> --- > >>

Re: [PATCH v2 03/16] migration: Move setup_time to mig_stats

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:07 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> It is a time that needs to be cleaned each time cancel migration. > >> Once there create migration_time_since() to calculate how time since a > >> time

Re: [PATCH v2 02/16] migration: Correct transferred bytes value

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:04 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> We forget several places to add to trasferred amount of data. With > >> this fixes I get: > >> > >>qemu_file_transferred() + multifd_bytes ==

Re: [PATCH v1 1/1] hw/pci: Disable PCI_ERR_UNCOR_MASK register for machine type < 8.0

2023-05-03 Thread Leonardo Bras Soares Passos
On Wed, May 3, 2023 at 6:49 AM Jonathan Cameron wrote: > > On Tue, 2 May 2023 21:27:02 -0300 > Leonardo Bras wrote: > > > Since it's implementation on v8.0.0-rc0, having the PCI_ERR_UNCOR_MASK > > set for machine types < 8.0 will cause migration to fail if the target

[PATCH v1 1/1] hw/pci: Disable PCI_ERR_UNCOR_MASK register for machine type < 8.0

2023-05-02 Thread Leonardo Bras
2. Fixes: 010746ae1d ("hw/pci/aer: Implement PCI_ERR_UNCOR_MASK register") Suggested-by: Michael S. Tsirkin Signed-off-by: Leonardo Bras --- include/hw/pci/pci.h | 2 ++ hw/core/machine.c| 1 + hw/pci/pci.c | 2 ++ hw/pci/pcie_aer.c| 11 +++ 4 files changed,

[PATCH v2 1/1] migration: Disable postcopy + multifd migration

2023-04-24 Thread Leonardo Bras
there is not enough testing/support nor any reported users for this scenario, we should disable this combination before it may cause any problems for users. Suggested-by: Dr. David Alan Gilbert Signed-off-by: Leonardo Bras Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- Changes since RFC

Re: [PATCH] MAINTAINERS: Add Leonardo and Peter as reviewers

2023-04-24 Thread Leonardo Bras Soares Passos
y they got CC'd on every migration patch. > > > > Signed-off-by: Juan Quintela > > Acked-by: Peter Xu > > -- > Peter Xu > Acked-by: Leonardo Bras Thanks, Leonardo Bras

[RFC PATCH v1 1/1] migration: Disable postcopy + multifd migration

2023-03-27 Thread Leonardo Bras
there is not enough testing/support nor any reported users for this scenario, we should disable this combination before it may cause any problems for users. Suggested-by: Dr. David Alan Gilbert Signed-off-by: Leonardo Bras --- migration/migration.c | 5 + 1 file changed, 5 insertions(+) diff --git

[PATCH v1 2/4] migration/multifd: Remove unnecessary assignment on multifd_load_cleanup()

2023-02-09 Thread Leonardo Bras
Before assigning "p->quit = true" for every multifd channel, multifd_load_cleanup() will call multifd_recv_terminate_threads() which already does the same assignment, while protected by a mutex. So there is no point doing the same assignment again. Signed-off-by: Leonardo Bras -

[PATCH v1 3/4] migration/multifd: Join all multifd threads in order to avoid leaks

2023-02-09 Thread Leonardo Bras
migration is usually just ran at most a few times, and after it succeeds there is not much to be done before exiting the process. Yet still, it should not hurt performance to join all of them. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 ++- 1 file changed, 2 insertions(+), 1

[PATCH v1 1/4] migration/multifd: Change multifd_load_cleanup() signature and usage

2023-02-09 Thread Leonardo Bras
to decide on setting autostart = false, which will never happen. In order to simplify the codebase, change multifd_load_cleanup() signature to 'void multifd_load_cleanup(void)', and for every usage remove error handling or decision made based on return value != 0. Signed-off-by: Leonardo Bras ---

[PATCH v1 4/4] migration/multifd: Move load_cleanup inside incoming_state_destroy

2023-02-09 Thread Leonardo Bras
ure nothing else is received before dirty_bitmap_mig_before_vm_start(). Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/migration.c | 4 +++- migration/multifd.c | 7 +++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/multifd.h b/migrat

[PATCH v1 1/4] migration/multifd: Change multifd_load_cleanup() signature and usage

2023-02-09 Thread Leonardo Bras
to decide on setting autostart = false, which will never happen. In order to simplify the codebase, change multifd_load_cleanup() signature to 'void multifd_load_cleanup(void)', and for every usage remove error handling or decision made based on return value != 0. Signed-off-by: Leonardo Bras ---

[PATCH v1 4/4] migration/multifd: Move load_cleanup inside incoming_state_destroy

2023-02-09 Thread Leonardo Bras
ure nothing else is received before dirty_bitmap_mig_before_vm_start(). Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/migration.c | 4 +++- migration/multifd.c | 7 +++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/multifd.h b/migrat

[PATCH v1 3/4] migration/multifd: Join all multifd threads in order to avoid leaks

2023-02-09 Thread Leonardo Bras
migration is usually just ran at most a few times, and after it succeeds there is not much to be done before exiting the process. Yet still, it should not hurt performance to join all of them. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 ++- 1 file changed, 2 insertions(+), 1

[PATCH v1 2/4] migration/multifd: Remove unnecessary assignment on multifd_load_cleanup()

2023-02-09 Thread Leonardo Bras
Before assigning "p->quit = true" for every multifd channel, multifd_load_cleanup() will call multifd_recv_terminate_threads() which already does the same assignment, while protected by a mutex. So there is no point doing the same assignment again. Signed-off-by: Leonardo Bras -

Re: [PATCH] migration: Show downtime during postcopy phase

2022-12-20 Thread Leonardo Bras Soares Passos
} else { > info->has_total_time = true; > info->total_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - > s->start_time; > +} > + > +if (migrate_show_downtime(s)) { > +info->has_downtime = true; > + info->downtime = s->downtime; > +} else { > info->has_expected_downtime = true; > info->expected_downtime = s->expected_downtime; > } > -- > 2.37.3 > FWIW: Reviewed-by: Leonardo Bras

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-29 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Nov 24, 2022 at 1:04 PM Peter Xu wrote: > > On Wed, Nov 09, 2022 at 02:56:29AM -0300, Leonardo Bras wrote: > > diff --git a/migration/savevm.c b/migration/savevm.c > > index a0cdb714f7..250caff7f4 100644 > > --- a/migration/savevm.c >

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-14 Thread Leonardo Bras Soares Passos
On Thu, Nov 10, 2022 at 10:48 AM Juan Quintela wrote: > > Leonardo Bras wrote: > D> When multifd and postcopy-ram capabilities are enabled, if a > > migrate-start-postcopy is attempted, the migration will finish sending the > > memory pages and then crash with the follo

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-09 Thread Leonardo Bras Soares Passos
On Wed, Nov 9, 2022 at 10:31 AM Dr. David Alan Gilbert wrote: > > * Leonardo Bras (leob...@redhat.com) wrote: > > When multifd and postcopy-ram capabilities are enabled, if a > > migrate-start-postcopy is attempted, the migration will finish sending the > > mem

[PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-08 Thread Leonardo Bras
ore MIGRATION_YANK_INSTANCE is unregistered. Fixes: b5eea99ec2 ("migration: Add yank feature") Reported-by: Li Xiaohui Signed-off-by: Leonardo Bras --- migration/migration.h | 1 + migration/migration.c | 18 +- migration/savevm.c| 2 ++ 3 files changed, 16 insertions(+), 5 delet

[RFC PATCH 0/4] MultiFD zero-copy improvements

2022-10-24 Thread Leonardo Bras
while the most recent ones are ongoing, instead of waiting for everything to finish before sending more. It all works fine in my tests, but maybe I missed some cornercase. Please provide any feedback you find fit. Thank you all! Best regards, Leo Leonardo Bras (4): migration/multifd/zero-copy

[RFC PATCH 1/4] migration/multifd/zero-copy: Create helper function for flushing

2022-10-24 Thread Leonardo Bras
Move flushing code from multifd_send_sync_main() to a new helper, and call it in multifd_send_sync_main(). Signed-off-by: Leonardo Bras --- migration/multifd.c | 30 +++--- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/migration/multifd.c b/migration

[RFC PATCH 4/4] migration/multifd/zero-copy: Flush only the LRU half of the header array

2022-10-24 Thread Leonardo Bras
consuming less cpu time. Signed-off-by: Leonardo Bras --- migration/multifd.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index c5d1f911a4..fe9df460f6 100644 --- a/migration/multifd.c +++ b/migration/multifd.c

[RFC PATCH 3/4] QIOChannel: Add max_pending parameter to qio_channel_flush()

2022-10-24 Thread Leonardo Bras
number of pending writes remaining before the function returns. Also, implement this change in qio_channel_socket_flush(). Change current calls of qio_channel_flush() so (max_pending == 0), and the flush-all behavior is maintained. Signed-off-by: Leonardo Bras --- include/io/channel.h | 7

[RFC PATCH 2/4] migration/multifd/zero-copy: Merge header & pages send in a single write

2022-10-24 Thread Leonardo Bras
element of the array will be used as a header. Suggested-by: Juan Quintela Signed-off-by: Leonardo Bras --- migration/multifd.h | 5 - migration/multifd.c | 52 - 2 files changed, 32 insertions(+), 25 deletions(-) diff --git a/migration/multifd.h b

Re: [PATCH v7 07/12] multifd: Prepare to send a packet without the mutex held

2022-08-20 Thread Leonardo Bras Soares Passos
On Fri, Aug 19, 2022 at 8:32 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Tue, 2022-08-02 at 08:39 +0200, Juan Quintela wrote: > >> We do the send_prepare() and the fill of the head packet without the > >> mutex held. It will help a lot for compression and later in the > >> series

Re: [PATCH v7 06/12] multifd: Make flags field thread local

2022-08-20 Thread Leonardo Bras Soares Passos
to keep the sync info, instead of the previous > > flags > > local var, and thus it can set p->flags = 0 earlier. Seems to not change any > > behavior AFAICS. > > The protection of the global flags was being wrong. That is the reason > that I decided to change it to the sync_needed. > > The problem was that at some point we were still sending a packet (that > shouldn't have the SYNC flag enabled), but we received a > multifd_main_sync() and it got enabled anyways. The easier way that I > found te fix it was this way. > > Problem was difficult to detect, that is the reason that I change it > this way. Oh, I see. > > >> -if (flags & MULTIFD_FLAG_SYNC) { > >> +if (sync_needed) { > >> qemu_sem_post(_recv_state->sem_sync); > >> qemu_sem_wait(>sem_sync); > >> } > > > > Ok, IIUC this part should have the same behavior as before, but using a bool > > instead of an u32. > > I changed it to make sure that we only checked the flags at the > beggining of the function, with the lock taken. Thanks for sharing! Best regards, Leo > > > > > FWIW: > > Reviewed-by: Leonardo Bras > > Thanks, Juan. >

Re: [PATCH v7 05/12] migration: Make ram_save_target_page() a pointer

2022-08-20 Thread Leonardo Bras Soares Passos
On Fri, Aug 19, 2022 at 6:52 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Tue, 2022-08-02 at 08:39 +0200, Juan Quintela wrote: > >> We are going to create a new function for multifd latest in the series. > >> > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Dr. David Alan Gilbert

Re: [RFC PATCH 2/2] kvm/kvm-all.c: listener should delay kvm_vm_ioctl to the commit phase

2022-08-18 Thread Leonardo Bras Soares Passos
On Thu, Aug 18, 2022 at 5:05 PM Peter Xu wrote: > > On Tue, Aug 16, 2022 at 06:12:50AM -0400, Emanuele Giuseppe Esposito wrote: > > +static void kvm_memory_region_node_add(KVMMemoryListener *kml, > > + struct kvm_userspace_memory_region > > *mem) > > +{ > >

[PATCH v1 1/1] QIOChannelSocket: Add support for MSG_ZEROCOPY + IPV6

2022-08-04 Thread Leonardo Bras
c58ffc29 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index e6e7a12ae1..4f6bb6fe37 1

Re: [PATCH v1 1/1] migration: add remaining params->has_* = true in migration_instance_init()

2022-07-26 Thread Leonardo Bras Soares Passos
n Mon, Jul 25, 2022 at 10:02 PM Leonardo Bras wrote: > > Some of params->has_* = true are missing in migration_instance_init, this > causes migrate_params_check() to skip some tests, allowing some > unsupported scenarios. > > Fix this by adding all missing params->has_* = true

[PATCH v1 1/1] migration: add remaining params->has_* = true in migration_instance_init()

2022-07-25 Thread Leonardo Bras
Some of params->has_* = true are missing in migration_instance_init, this causes migrate_params_check() to skip some tests, allowing some unsupported scenarios. Fix this by adding all missing params->has_* = true in migration_instance_init(). Signed-off-by: Leonardo Bras --- mig

[PATCH v1 1/1] migration: Avoid false-positive on non-supported scenarios for zero-copy-send

2022-07-19 Thread Leonardo Bras
s_check(). 3) XBZRLE is also a compression capability, so it makes sense to also add it to the list of capabilities which are not supported with zero-copy-send. Fixes: 1abaec9a1b2c ("migration: Change zero_copy_send from migration parameter to migration capability") Signed-off-

Re: [PATCH v4 3/3] migration/multifd: Report to user when zerocopy not working

2022-07-12 Thread Leonardo Bras Soares Passos
On Tue, Jul 12, 2022 at 7:42 PM Peter Xu wrote: > > On Mon, Jul 11, 2022 at 06:11:13PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using > >

[PATCH v4 3/3] migration/multifd: Report to user when zerocopy not working

2022-07-11 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment dirty_sync_missed_zero_copy migration stat to let the user know about it. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- migration/ram.h

[PATCH v4 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-11 Thread Leonardo Bras
Signed-off-by: Leonardo Bras Acked-by: Markus Armbruster Reviewed-by: Daniel P. Berrangé --- qapi/migration.json | 7 ++- migration/migration.c | 2 ++ monitor/hmp-cmds.c| 5 + 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi

[PATCH v4 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-11 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Acked-by: Daniel P. Berrangé Reviewed-by: Juan Quintela --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(

[PATCH v4 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-11 Thread Leonardo Bras
release number changed from 7.2 to 7.1 - migration stat renamed from zero-copy-copied to dirty-sync-missed-zero-copy - Updated documentation to make it more user-friendly Changes since v1: - Idea of using a warning replaced by using a migration stat counter Leonardo Bras (3

Re: [PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-11 Thread Leonardo Bras Soares Passos
On Thu, Jul 7, 2022 at 7:18 PM Peter Xu wrote: > > On Thu, Jul 07, 2022 at 06:14:17PM -0300, Leonardo Brás wrote: > > Having 'if(queued == sent)' will cause us to falsely return '1' in two buggy > > cases, while 'if queued == 0) will either skip early or go into 'infinite' > > loop. > > I'm not

Re: [PATCH v3 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:56 PM Peter Xu wrote: > > On Mon, Jul 04, 2022 at 05:23:15PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on u

Re: [PATCH v3 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:54 PM Peter Xu wrote: > > On Mon, Jul 04, 2022 at 05:23:14PM -0300, Leonardo Bras wrote: > > Signed-off-by: Leonardo Bras > > --- > > qapi/migration.json | 7 ++- > > migration/migration.c | 2 ++ > > monitor

Re: [PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:47 PM Peter Xu wrote: > > Hi, Leo, > > On Mon, Jul 04, 2022 at 05:23:13PM -0300, Leonardo Bras wrote: > > If flush is called when no buffer was sent with MSG_ZEROCOPY, it currently > > returns 1. This return code should be used only

[PATCH v3 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-04 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment dirty_sync_missed_zero_copy migration stat to let the user know about it. Signed-off-by: Leonardo Bras --- migration/ram.h | 2 ++ migration/multifd.c | 2

[PATCH v3 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-04 Thread Leonardo Bras
Signed-off-by: Leonardo Bras --- qapi/migration.json | 7 ++- migration/migration.c | 2 ++ monitor/hmp-cmds.c| 4 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index 7102e474a6..fed08b9b88 100644 --- a/qapi/migration.json

[PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-04 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v3 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-04 Thread Leonardo Bras
user-friendly Changes since v1: - Idea of using a warning replaced by using a migration stat counter Leonardo Bras (3): QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent Add dirty-sync-missed-zero-copy migration stat migration/multifd: Warn user when zerocopy

[PATCH v2 2/3] Add zero-copy-copied migration stat

2022-07-01 Thread Leonardo Bras
Signed-off-by: Leonardo Bras --- qapi/migration.json | 5 - migration/migration.c | 1 + monitor/hmp-cmds.c| 4 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index 7102e474a6..925f009868 100644 --- a/qapi/migration.json

[PATCH v2 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-01 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment zero_copy_copied migration stat to let the user know about it. Signed-off-by: Leonardo Bras --- migration/ram.h | 2 ++ migration/multifd.c | 2 ++ migration

[PATCH v2 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-01 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v2 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-01 Thread Leonardo Bras
to keep track of [*]. Honestly I would like some help with this naming, which I don't think is quite good, but I could also not think on anything better. The third patch keeps track of how many zero-copy flushes retured 1 [*] Leonardo Bras (3): QIOChannelSocket: Fix zero-copy flush returning

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-28 Thread Leonardo Bras Soares Passos
On Tue, Jun 28, 2022 at 10:52 AM Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berra...@redhat.com) wrote: > > On Tue, Jun 28, 2022 at 09:32:04AM -0300, Leonardo Bras Soares Passos wrote: > > > On Tue, Jun 28, 2022 at 4:53 AM Daniel P. Berrangé > > > wrot

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-28 Thread Leonardo Bras Soares Passos
On Tue, Jun 28, 2022 at 4:53 AM Daniel P. Berrangé wrote: > > On Mon, Jun 27, 2022 at 10:09:09PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using &g

[PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-27 Thread Leonardo Bras
dirty-bitmap iterations were not able to use zero-copy send. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/multifd.c b/migration/multifd.c index 684c014c86..9c62aec84e 100644 --- a/migration/multifd.c +++ b/migration

[PATCH v1 0/2] Zero copy improvements (QIOChannel + multifd)

2022-06-27 Thread Leonardo Bras
the frequency of multifd syncs, or else it could potentially send 20 warnings per second. ... Leonardo Bras (2): QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent migration/multifd: Warn user when

[PATCH v1 1/2] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-06-27 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v4 4/4] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-19 Thread Leonardo Bras
how recently the change got merged, it was decided that it's still time to make it right, and convert zero_copy_send into a Migration capability. Signed-off-by: Leonardo Bras --- qapi/migration.json | 33 --- migration/migration.c | 52

[PATCH v4 2/4] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-19 Thread Leonardo Bras
a corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) d

Re: [PATCH v4 4/4] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-19 Thread Leonardo Bras Soares Passos
CC: Jiri Denemark On Mon, Jun 20, 2022 at 2:40 AM Leonardo Bras wrote: > > When originally implemented, zero_copy_send was designed as a Migration > paramenter. > > But taking into account how is that supposed to work, and how > the difference between a capability and a parame

[PATCH v4 0/4] MSG_ZEROCOPY fixes & improvements

2022-06-19 Thread Leonardo Bras
ch 4 is about making zero-copy-send a migration capability, instead of a migration parameter. Which actually makes more sense and helps the implementation of the libvirt code. Leonardo Bras (4): QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability QIOChannelSocket: Fix

[PATCH v4 3/4] migration: zero-copy flush only at the end of bitmap scanning

2022-06-19 Thread Leonardo Bras
. Change so it flushes only after a whole scanning of the dirty bimap, so it never sends a newer version of a page before an older one, while avoiding unnecessary overhead. Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/multifd.c | 58

[PATCH v4 1/4] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-19 Thread Leonardo Bras
-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- io/channel-socket.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index dc9c165de1..dac9e60c20 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -578,11

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-14 Thread Leonardo Bras Soares Passos
On Tue, Jun 14, 2022 at 5:36 AM Daniel P. Berrangé wrote: > > On Mon, Jun 13, 2022 at 06:21:18PM -0300, Leonardo Bras Soares Passos wrote: > > On Fri, Jun 10, 2022 at 5:25 AM Daniel P. Berrangé > > wrote: > > > > > > > [...] > > > > &g

[PATCH v3 2/2] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-13 Thread Leonardo Bras
a corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-socket.

[PATCH v3 1/2] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-13 Thread Leonardo Bras
-by: Leonardo Bras --- io/channel-socket.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index dc9c165de1..cdce7b0b45 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -578,11 +578,13 @@ static ssize_t

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-13 Thread Leonardo Bras Soares Passos
On Mon, Jun 13, 2022 at 7:53 PM Peter Xu wrote: > > On Mon, Jun 13, 2022 at 05:58:44PM -0300, Leonardo Bras Soares Passos wrote: > > Hello Peter, > > > > On Wed, Jun 8, 2022 at 5:23 PM Peter Xu wrote: > > [...] > > > > In a previous iteration of

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-13 Thread Leonardo Bras Soares Passos
On Fri, Jun 10, 2022 at 5:25 AM Daniel P. Berrangé wrote: > [...] > Ok, so if it is checked earlier then we merely need an assert. > > if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { > #ifdef QEMU_MSG_ZEROCOPY > sflags = MSG_ZEROCOPY; > zero_copy_enabled = true; > #else

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-13 Thread Leonardo Bras Soares Passos
Hello Peter, On Wed, Jun 8, 2022 at 5:23 PM Peter Xu wrote: [...] > > In a previous iteration of the patchset, it was made clear that it's > > desirable to detect when the kernel falls back to copying mechanism, > > so the user of 'QIOChannelSocket' can switch to copying and avoid the > >

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-09 Thread Leonardo Bras Soares Passos
Hello Daniel, On Thu, Jun 9, 2022 at 5:10 AM Daniel P. Berrangé wrote: > > On Wed, Jun 08, 2022 at 06:04:02PM -0300, Leonardo Bras wrote: > > During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were > > introduced, particularly at qio_channel_socket_writev(). &g

[PATCH v2 2/2] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras
a corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-socket.

[PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-08 Thread Leonardo Bras
During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev(). Rewrite some of those changes so it's easier to read. ... Signed-off-by: Leonardo Bras --- io

Re: [PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras Soares Passos
On Wed, Jun 8, 2022 at 5:55 PM Peter Xu wrote: > > On Wed, Jun 08, 2022 at 04:26:10PM -0400, Peter Xu wrote: > > On Wed, Jun 08, 2022 at 03:18:09PM -0300, Leonardo Bras wrote: > > > Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial > > > part

Re: [PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras Soares Passos
Hello Daniel, On Wed, Jun 8, 2022 at 3:46 PM Daniel P. Berrangé wrote: > > On Wed, Jun 08, 2022 at 03:18:09PM -0300, Leonardo Bras wrote: > > Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial > > part of the flushing mechanism got missing: incrementin

[PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras
in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --gi

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-08 Thread Leonardo Bras Soares Passos
On Wed, Jun 8, 2022 at 8:41 AM Peter Xu wrote: > > On Wed, Jun 08, 2022 at 02:37:28AM -0300, Leonardo Bras Soares Passos wrote: > > (1) is not an option, as the interface currently uses ret=1 to make > > sure MSG_ZEROCOPY is getting used, > > I added that so the user of qi

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Wed, Jun 1, 2022 at 10:58 AM Peter Xu wrote: > [...] > > Hi, Leonardo. I'm also paying attention to the application of MSG_ZEROCOPY > > in live migration recently. I noticed that you defined a member > > `zero_copy_queued` in the struct QIOChannelSocket, but I can't find out > >

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-07 Thread Leonardo Bras Soares Passos
Hello 徐闯, Thanks for reviewing! On Wed, Jun 1, 2022 at 6:37 AM 徐闯 wrote: [...] > Hi, Leonardo. I'm also paying attention to the application of > MSG_ZEROCOPY in live migration recently. I noticed that you defined a > member `zero_copy_queued` in the struct QIOChannelSocket, but I can't > find

[PATCH v13 8/8] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)

2022-05-13 Thread Leonardo Bras
tion with zero-copy enabled, so disabling the feature should be necessary for low-privileged users trying to perform multifd migrations. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- migration/multifd.h | 2 ++ migration/migration.c

[PATCH v13 6/8] multifd: multifd_send_sync_main now returns negative on error

2022-05-13 Thread Leonardo Bras
is important to next patch on multifd zero copy implementation, to make it sure an error in zero-copy flush does not go unnoticed. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu --- migration/multifd.h | 2 +- migration/multifd.c | 10 ++ migration/ram.c

[PATCH v13 4/8] migration: Add zero-copy-send parameter for QMP/HMP for Linux

2022-05-13 Thread Leonardo Bras
. On non-Linux builds this parameter is compiled-out. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Acked-by: Markus Armbruster --- qapi/migration.json | 24 migration/migration.h | 5 + migration

[PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-05-13 Thread Leonardo Bras
it can still be accessible to less privileged users. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé Reviewed-by: Juan Quintela --- include/io/channel-socket.h | 2 + io/channel-socket.c | 116 ++-- 2 files changed, 114

[PATCH v13 7/8] multifd: Send header packet without flags if zero-copy-send is enabled

2022-05-13 Thread Leonardo Bras
happening before. This only changes how a migration with zero-copy-send=true works, not changing any current behavior for migrations with zero-copy-send=false. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- migration/multifd.c | 22 +++-

[PATCH v13 2/8] QIOChannel: Add flags on io_writev and introduce io_flush callback

2022-05-13 Thread Leonardo Bras
. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Reviewed-by: Juan Quintela --- include/io/channel.h| 38 +- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buffer.c

[PATCH v13 1/8] meson.build: Fix docker-test-build@alpine when including linux/errqueue.h

2022-05-13 Thread Leonardo Bras
=== As above error message suggests, 'struct __kernel_timespec' was already defined by liburing/compat.h. Fix alpine CI by adding test to disable liburing in configure step if a redefinition happens between linux/errqueue.h and liburing/compat.h. Signed-off-by: Leonardo Bras --- meson.build | 11

[PATCH v13 5/8] migration: Add migrate_use_tls() helper

2022-05-13 Thread Leonardo Bras
A lot of places check parameters.tls_creds in order to evaluate if TLS is in use, and sometimes call migrate_get_current() just for that test. Add new helper function migrate_use_tls() in order to simplify testing for TLS usage. Signed-off-by: Leonardo Bras Reviewed-by: Juan Quintela Reviewed

  1   2   3   4   >