[PATCH v4 0/5] virtiofsd xattr name mappings

2020-10-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is the 4th cut of an xattr name mapping option for virtiofsd. It allows the user of virtiofsd to define a fairly flexible mapping from the view of the xattr names the host fs has and the ones that the guest sees. The hope is this allows things like: a)

[PATCH] arm/trace: Fix hex printing

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use of 0x%d - make up our mind as 0x%x Signed-off-by: Dr. David Alan Gilbert --- hw/arm/trace-events | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/arm/trace-events b/hw/arm/trace-events index c8a4d80f6b..a335ee891d 100644 ---

[PATCH v3 3/5] tools/virtiofsd: xattr name mappings: Map server xattr names

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names coming from the server, i.e. the host filesystem; currently this is only from listxattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 89 1 file changed, 89 insertions(+) diff

[PATCH v3 4/5] tools/virtiofsd: xattr name mapping examples

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a few examples of xattrmaps to the documentation. Signed-off-by: Dr. David Alan Gilbert --- docs/tools/virtiofsd.rst | 50 1 file changed, 50 insertions(+) diff --git a/docs/tools/virtiofsd.rst

[PATCH v3 5/5] tools/virtiofsd: xattr name mappings: Simple 'map'

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The mapping rule system implemented in the last few patches is extremely flexible, but not easy to use. Add a simple 'map' type as a sprinkling of sugar to make it easy. e.g. -o xattrmap=":map::user.virtiofs.:" would be sufficient to prefix all xattr's or

[PATCH v3 2/5] tools/virtiofsd: xattr name mappings: Map client xattr names

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names originating at the client; from get/set/remove xattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 101 ++- 1 file changed, 98 insertions(+), 3 deletions(-) diff --git

[PATCH v3 1/5] tools/virtiofsd: xattr name mappings: Add option

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add an option to define mappings of xattr names so that the client and server filesystems see different views. This can be used to have different SELinux mappings as seen by the guest, to run the virtiofsd with less privileges (e.g. in a case where it can't set

[PATCH v3 0/5] virtiofsd xattr name mappings

2020-10-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a 3rd cut of an xattr name mapping option for virtiofsd. It allows the user of virtiofsd to define a fairly flexible mapping from the view of the xattr names the host fs has and the ones that the guest sees. The hope is this allows things like: a)

[PULL 4/6] virtiofsd: avoid /proc/self/fd tempdir

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: Stefan Hajnoczi In order to prevent /proc/self/fd escapes a temporary directory is created where /proc/self/fd is bind-mounted. This doesn't work on read-only file systems. Avoid the temporary directory by bind-mounting /proc/self/fd over /proc. This does not affect other processes since

[PULL 2/6] tools/virtiofsd: add support for --socket-group

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: Alex Bennée If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée

[PULL 5/6] migration/dirtyrate: record start_time and calc_time while at the measuring state

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Querying could include both the start-time and the calc-time while at the measuring state, allowing a caller to determine when they should expect to come back looking for a result. Signed-off-by: Chuan Zheng Message-Id:

[PULL 3/6] virtiofsd: Call qemu_init_exec_dir

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée Signed-off-by: Dr. David Alan Gilbert Message-Id: <20201002124015.44820-1-dgilb...@redhat.com> Tested-by:

[PULL 6/6] migration/dirtyrate: present dirty rate only when querying the rate has completed

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Make dirty_rate field optional, present dirty rate only when querying the rate has completed. The qmp results is shown as follow: @unstarted: {"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"} @measuring:

[PULL 0/6] migration queue

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 2387df497b4b4bcf754eb7398edca82889e2ef54: Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into staging (2020-10-12 11:29:42 +0100) are available in the Git repository at:

[PULL 1/6] virtiofsd: Silence gcc warning

2020-10-12 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert Message-Id:

[PULL 10/10] migration/dirtyrate: present dirty rate only when querying the rate has completed

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Make dirty_rate field optional, present dirty rate only when querying the rate has completed. The qmp results is shown as follow: @unstarted: {"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"} @measuring:

[PULL 06/10] migration: Introduce migrate_send_rp_message_req_pages()

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Peter Xu This is another layer wrapper for sending a page request to the source VM. The new migrate_send_rp_message_req_pages() will be used elsewhere in coming patches. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-3-pet...@redhat.com>

[PULL 08/10] migration: Sync requested pages after postcopy recovery

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Peter Xu We synchronize the requested pages right after a postcopy recovery happens. This helps to synchronize the prioritized pages on source so that the faulted threads can be served faster. Reported-by: Xiaohui Li Signed-off-by: Peter Xu Message-Id:

[PULL 03/10] virtiofsd: Call qemu_init_exec_dir

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée Signed-off-by: Dr. David Alan Gilbert Message-Id: <20201002124015.44820-1-dgilb...@redhat.com> Tested-by:

[PULL 04/10] virtiofsd: avoid /proc/self/fd tempdir

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Stefan Hajnoczi In order to prevent /proc/self/fd escapes a temporary directory is created where /proc/self/fd is bind-mounted. This doesn't work on read-only file systems. Avoid the temporary directory by bind-mounting /proc/self/fd over /proc. This does not affect other processes since

[PULL 09/10] migration/dirtyrate: record start_time and calc_time while at the measuring state

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Querying could include both the start-time and the calc-time while at the measuring state, allowing a caller to determine when they should expect to come back looking for a result. Signed-off-by: Chuan Zheng Message-Id:

[PULL 02/10] tools/virtiofsd: add support for --socket-group

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Alex Bennée If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée

[PULL 00/10] migration queue

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit e64cf4d569f6461d6b9072e00d6e78d0ab8bd4a7: Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20201008' into staging (2020-10-08 17:18:46 +0100) are available in the Git repository at: git://github.com/dagrh/qemu.git

[PULL 07/10] migration: Maintain postcopy faulted addresses

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Peter Xu Maintain a list of faulted addresses on the destination host for which we're waiting on. This is implemented using a GTree rather than a real list to make sure even there're plenty of vCPUs/threads that are faulting, the lookup will still be fast with O(log(N)) (because we'll do

[PULL 05/10] migration: Pass incoming state into qemu_ufd_copy_ioctl()

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: Peter Xu It'll be used in follow up patches to access more fields out of it. Meanwhile fetch the userfaultfd inside the function. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-2-pet...@redhat.com> Signed-off-by: Dr. David Alan Gilbert

[PULL 01/10] virtiofsd: Silence gcc warning

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert Message-Id:

[PATCH] mingw: Fix builds on f33 mingw

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Fedora 33's mingw moans about: In file included from ../tests/test-bitmap.c:12: /home/dgilbert/git/migpull/include/qemu/osdep.h:76: error: "__USE_MINGW_ANSI_STDIO" redefined [-Werror] 76 | #define __USE_MINGW_ANSI_STDIO 1 | the fix is to make sure

[PATCH] tests/migration: Allow longer timeouts

2020-10-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In travis, with gcov and gprof we're seeing timeouts; hopefully fix this by increasing the test timeouts a bit, but for xbzrle ensure it really does get a couple of cycles through to test the cache. I think the problem in travis is we have about 2 host CPU

[PULL 06/10] migration: Introduce migrate_send_rp_message_req_pages()

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Peter Xu This is another layer wrapper for sending a page request to the source VM. The new migrate_send_rp_message_req_pages() will be used elsewhere in coming patches. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-3-pet...@redhat.com>

[PULL 07/10] migration: Maintain postcopy faulted addresses

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Peter Xu Maintain a list of faulted addresses on the destination host for which we're waiting on. This is implemented using a GTree rather than a real list to make sure even there're plenty of vCPUs/threads that are faulting, the lookup will still be fast with O(log(N)) (because we'll do

[PULL 04/10] virtiofsd: avoid /proc/self/fd tempdir

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Stefan Hajnoczi In order to prevent /proc/self/fd escapes a temporary directory is created where /proc/self/fd is bind-mounted. This doesn't work on read-only file systems. Avoid the temporary directory by bind-mounting /proc/self/fd over /proc. This does not affect other processes since

[PULL 03/10] virtiofsd: Call qemu_init_exec_dir

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée Signed-off-by: Dr. David Alan Gilbert Message-Id: <20201002124015.44820-1-dgilb...@redhat.com> Tested-by:

[PULL 08/10] migration: Sync requested pages after postcopy recovery

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Peter Xu We synchronize the requested pages right after a postcopy recovery happens. This helps to synchronize the prioritized pages on source so that the faulted threads can be served faster. Reported-by: Xiaohui Li Signed-off-by: Peter Xu Message-Id:

[PULL 01/10] virtiofsd: Silence gcc warning

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert Message-Id:

[PULL 09/10] migration/dirtyrate: record start_time and calc_time while at the measuring state

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Querying could include both the start-time and the calc-time while at the measuring state, allowing a caller to determine when they should expect to come back looking for a result. Signed-off-by: Chuan Zheng Message-Id:

[PULL 00/10] migration queue

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit f2687fdb7571a444b5af3509574b659d35ddd601: Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2020-10-06 15:04:10 +0100) are available in the Git repository at:

[PULL 02/10] tools/virtiofsd: add support for --socket-group

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Alex Bennée If you like running QEMU as a normal user (very common for TCG runs) but you have to run virtiofsd as a root user you run into connection problems. Adding support for an optional --socket-group allows the users to keep using the command line. Signed-off-by: Alex Bennée

[PULL 05/10] migration: Pass incoming state into qemu_ufd_copy_ioctl()

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Peter Xu It'll be used in follow up patches to access more fields out of it. Meanwhile fetch the userfaultfd inside the function. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Message-Id: <20201002175336.30858-2-pet...@redhat.com> Signed-off-by: Dr. David Alan Gilbert

[PULL 10/10] migration/dirtyrate: present dirty rate only when querying the rate has completed

2020-10-07 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Make dirty_rate field optional, present dirty rate only when querying the rate has completed. The qmp results is shown as follow: @unstarted: {"return":{"status":"unstarted","start-time":0,"calc-time":0},"id":"libvirt-12"} @measuring:

[PATCH] virtiofsd: Call qemu_init_exec_dir

2020-10-02 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Since fcb4f59c879 qemu_get_local_state_pathname relies on the init_exec_dir, and virtiofsd asserts because we never set it. Set it. Reported-by: Alex Bennée Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 2 ++ 1 file changed, 2

[PULL 1/1] Revert "migration: Properly destroy variables on incoming side"

2020-09-28 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This reverts commit c02039a6f3730ddcf683a0ba9a175688c6db71a0. This is breaking test 068 that does a loadvm twice. Signed-off-by: Dr. David Alan Gilbert --- migration/migration.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git

[PULL 0/1] migration queue

2020-09-28 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 8d16e72f2d4df2c9e631393adf1669a1da7efe8a: Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200925a' into staging (2020-09-25 14:46:18 +0100) are available in the Git repository at:

[PULL 26/26] virtiofsd: Add -o allow_direct_io|no_allow_direct_io options

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Jiachen Zhang Due to the commit 65da4539803373ec4eec97ffc49ee90083e56efd, the O_DIRECT open flag of guest applications will be discarded by virtiofsd. While this behavior makes it consistent with the virtio-9p scheme when guest applications use direct I/O, we no longer have any chance to

[PULL 23/26] monitor: Use LOCK_GUARD macros

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use the lock guard macros in monitor/misc.c - saves a lot of unlocks in error paths, and the occasional goto. Signed-off-by: Dr. David Alan Gilbert Message-Id: <20200922095741.101911-1-dgilb...@redhat.com> Reviewed-by: Markus Armbruster Reviewed-by: Philippe

[PULL 24/26] virtiofsd: document cache=auto default

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: "Harry G. Coin" The virtiofsd --help output documents the cache=auto default value but the man page does not. Fix this. Signed-off-by: Harry G. Coin Signed-off-by: Stefan Hajnoczi Message-Id: <20200916112250.760245-1-stefa...@redhat.com> Reviewed-by: Dr. David Alan Gilbert

[PULL 22/26] migration/tls: add trace points for multifd-tls

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng add trace points for multifd-tls for debug. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-7-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/multifd.c| 10

[PULL 25/26] virtiofsd: Used glib "shared" thread pool

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Vivek Goyal glib offers thread pools and it seems to support "exclusive" and "shared" thread pools. https://developer.gnome.org/glib/stable/glib-Thread-Pools.html#g-thread-pool-new Currently we use "exlusive" thread pools but its performance seems to be poor. I tried using "shared"

[PULL 19/26] migration/tls: add tls_hostname into MultiFDSendParams

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Since multifd creation is async with migration_channel_connect, we should pass the hostname from MigrationState to MultiFDSendParams. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Message-Id: <1600139042-104593-4-git-send-email-zhengch...@huawei.com> Reviewed-by: Daniel

[PULL 21/26] migration/tls: add support for multifd tls-handshake

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Similar like migration main thread, we need to do handshake for each multifd thread. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-6-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan

[PULL 12/26] migration/dirtyrate: Implement calculate_dirtyrate() function

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Implement calculate_dirtyrate() function. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-11-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert ---

[PULL 17/26] migration/tls: save hostname into MigrationState

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng hostname is need in multifd-tls, save hostname into MigrationState. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Message-Id: <1600139042-104593-2-git-send-email-zhengch...@huawei.com> Reviewed-by: Daniel P. Berrangé Signed-off-by: Dr. David Alan Gilbert ---

[PULL 20/26] migration/tls: extract cleanup function for common-use

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng multifd channel cleanup is need if multifd handshake failed, let's extract it. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-5-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert ---

[PULL 18/26] migration/tls: extract migration_tls_client_create for common-use

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng migration_tls_client_create will be used in multifd-tls, let's extract it. Signed-off-by: Chuan Zheng Signed-off-by: Yan Jin Reviewed-by: Daniel P. Berrangé Message-Id: <1600139042-104593-3-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert ---

[PULL 16/26] migration: increase max-bandwidth to 128 MiB/s (1 Gib/s)

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Laurent Vivier max-bandwidth is set by default to 32 MiB/s (256 Mib/s) since 2008 (5bb7910af031c). Most of the CPUs can dirty memory faster than that now, and this is clearly a problem with POWER where the page size is 64 kiB and not 4 KiB. Signed-off-by: Laurent Vivier Message-Id:

[PULL 10/26] migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng In order to sample real RAM, skip ramblock with size below MIN_RAMBLOCK_SIZE which is set as 128M. Signed-off-by: Chuan Zheng Reviewed-by: David Edmondson Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id:

[PULL 13/26] migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be called Signed-off-by: Chuan Zheng Message-Id: <1600237327-33618-12-git-send-email-zhengch...@huawei.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert atomic function

[PULL 15/26] migration: Truncate state file in xen-save-devices-state

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Dov Murik When running the xen-save-devices-state QMP command, if the filename already exists it will be truncated before dumping the devices' state into it. Signed-off-by: Dov Murik Message-Id: <20200921094830.114028-1-dovmu...@linux.vnet.ibm.com> Reviewed-by: Dr. David Alan Gilbert

[PULL 07/26] migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng RAMBLOCK_FOREACH_MIGRATABLE is need in dirtyrate measure, move the existing definition up into migration/ram.h Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id:

[PULL 14/26] migration/dirtyrate: Add trace_calls to make it easier to debug

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Add trace_calls to make it easier to debug Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Message-Id: <1600237327-33618-13-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert --- migration/dirtyrate.c |

[PULL 08/26] migration/dirtyrate: Record hash results for each sampled page

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Record hash results for each sampled page, crc32 is taken to calculate hash results for each sampled length in TARGET_PAGE_SIZE. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id:

[PULL 05/26] migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Add RamblockDirtyInfo to store sampled page info of each ramblock. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id: <1600237327-33618-4-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr.

[PULL 11/26] migration/dirtyrate: Implement set_sample_page_period() and is_sample_period_valid()

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Implement is_sample_period_valid() to check if the sample period is vaild and do set_sample_page_period() to sleep specific time between sample actions. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang

[PULL 03/26] migration/dirtyrate: setup up query-dirtyrate framwork

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Add get_dirtyrate_thread() functions to setup query-dirtyrate framework. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: David Edmondson Reviewed-by: Li Qiang Message-Id:

[PULL 06/26] migration/dirtyrate: Add dirtyrate statistics series functions

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Add dirtyrate statistics functions to record/update dirtyrate info. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-5-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert ---

[PULL 02/26] migration: Rework migrate_send_rp_req_pages() function

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Peter Xu We duplicated the logic of maintaining the last_rb variable at both callers of this function. Pass *rb pointer into the function so that we can avoid duplicating the logic. Also, when we have the rb pointer, it's also easier to remove the original 2nd & 4th parameters, because

[PULL 09/26] migration/dirtyrate: Compare page hash results for recorded sampled page

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng Compare page hash results for recorded sampled page. Signed-off-by: Chuan Zheng Signed-off-by: YanYing Zhuang Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-8-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan

[PULL 04/26] migration/dirtyrate: add DirtyRateStatus to denote calculation status

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Chuan Zheng add DirtyRateStatus to denote calculating status. Signed-off-by: Chuan Zheng Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Li Qiang Message-Id: <1600237327-33618-3-git-send-email-zhengch...@huawei.com> Signed-off-by: Dr. David Alan Gilbert atomic name fixup ---

[PULL 00/26] migration and friends queue

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 8c1c07929feae876202ba26f07a540c5115c18cd: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2020-09-24 18:48:45 +0100) are available in the Git repository at:

[PULL 01/26] migration: Properly destroy variables on incoming side

2020-09-25 Thread Dr. David Alan Gilbert (git)
From: Peter Xu In migration_incoming_state_destroy(), we've got a few variables that aren't destroyed properly, namely: main_thread_load_event postcopy_pause_sem_dst postcopy_pause_sem_fault rp_mutex Destroy them properly. Signed-off-by: Peter Xu Message-Id:

[PATCH v2] monitor: Use LOCK_GUARD macros

2020-09-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use the lock guard macros in monitor/misc.c - saves a lot of unlocks in error paths, and the occasional goto. Signed-off-by: Dr. David Alan Gilbert -- v2: The file changed quite a bit since my original posting, so reworked. --- monitor/misc.c | 44

[PATCH] monitor: Use LOCK_GUARD macros

2020-09-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use the lock guard macros in monitor/misc.c - saves a bunch of goto's, and a temporary variable, but mostly because I prefer not having to release them in error paths. Signed-off-by: Dr. David Alan Gilbert --- monitor/misc.c | 62

[PULL 8/8] virtiofsd: probe unshare(CLONE_FS) and print an error

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Stefan Hajnoczi An assertion failure is raised during request processing if unshare(CLONE_FS) fails. Implement a probe at startup so the problem can be detected right away. Unfortunately Docker/Moby does not include unshare in the seccomp.json list unless CAP_SYS_ADMIN is given. Other

[PULL 7/8] virtiofsd: drop CAP_DAC_READ_SEARCH

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Stefan Hajnoczi virtiofsd does not need CAP_DAC_READ_SEARCH because it already has the more powerful CAP_DAC_OVERRIDE. Drop it from the list of capabilities. This is important because container runtimes may not include CAP_DAC_READ_SEARCH by default. This patch allows virtiofsd to reduce

[PULL 6/8] virtiofsd: Remove "norace" from cmdline help and docs

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Sergio Lopez Commit 93bb3d8d4cda ("virtiofsd: remove symlink fallbacks") removed the implementation of the "norace" option, so remove it from the cmdline help and the documentation too. Signed-off-by: Sergio Lopez Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefano Garzarella

[PULL 5/8] virtiofsd: Disable remote posix locks by default

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Vivek Goyal Right now we enable remote posix locks by default. That means when guest does a posix lock it sends request to server (virtiofsd). But currently we only support non-blocking posix lock and return -EOPNOTSUPP for blocking version. This means that existing applications which are

[PULL 2/8] migration: add vsock as data channel support

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: "Longpeng(Mike)" The vsock channel is more widely use in some new features, for example, the Nitro/Enclave. It can also be used as the migration channel. Signed-off-by: Longpeng(Mike) Message-Id: <20200806074030.174-3-longpe...@huawei.com> Reviewed-by: Dr. David Alan Gilbert

[PULL 3/8] migration: improve error reporting of block driver state name

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé With blockdev, a BlockDriverState may not have a device name, so using a node name is required as an alternative. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Message-Id: <20200827111606.1408275-2-berra...@redhat.com> Signed-off-by: Dr. David

[PULL 4/8] migration: tls: fix memory leak in migration_tls_get_creds

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: Zhenyu Ye Currently migration_tls_get_creds() adds the reference of creds but there was no place to unref it. So the OBJECT(creds) will never be freed and result in memory leak. The leak stack: Direct leak of 104 byte(s) in 1 object(s) allocated from: #0 0xa88bd20b in

[PULL 1/8] migration: unify the framework of socket-type channel

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: "Longpeng(Mike)" Currently, the only difference of tcp channel and unix channel in migration/socket.c is the way to build SocketAddress, but socket_parse() can handle these two types, so use it to instead of tcp_build_address() and unix_build_address(). The socket-type channel can be

[PULL 0/8] migration and virtiofsd queue

2020-08-28 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 3e39dac0354c39b4b647940e42360c6b1f3edc02: Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2020-08-28 11:05:08 +0100) are available in the Git repository at:

[PATCH v2 5/6] tools/virtiofsd: xattr name mappings: Map server xattr names

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names coming from the server, i.e. the host filesystem; currently this is only from listxattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 88 1 file changed, 88 insertions(+) diff

[PATCH v2 3/6] tools/virtiofsd: xattr name mappings: Add option

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add an option to define mappings of xattr names so that the client and server filesystems see different views. This can be used to have different SELinux mappings as seen by the guest, to run the virtiofsd with less privileges (e.g. in a case where it can't set

[PATCH v2 4/6] tools/virtiofsd: xattr name mappings: Map client xattr names

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names originating at the client; from get/set/remove xattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 102 ++- 1 file changed, 99 insertions(+), 3 deletions(-) diff --git

[PATCH v2 1/6] virtiofsd: Silence gcc warning

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc worries fd might be used unset, in reality it's always set if fi is set, and only used if fi is set so it's safe. Initialise it to -1 just to keep gcc happy for now. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 2 +- 1 file

[PATCH v2 6/6] tools/virtiofsd: xattr name mapping examples

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a few examples of xattrmaps to the documentation. Signed-off-by: Dr. David Alan Gilbert --- docs/tools/virtiofsd.rst | 49 1 file changed, 49 insertions(+) diff --git a/docs/tools/virtiofsd.rst

[PATCH v2 0/6] virtiofsd xattr name mappings

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This is a second cut of a xattr name mapping option for virtiofsd. It allows the user of virtiofsd to define a fairly flexible mapping from the view of the xattr names the host fs has and the ones that the guest sees. The hope is this allows things like:

[PATCH v2 2/6] virtiofsd: Add printf checking to fuse_log

2020-08-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Use qemu's GCC_FMT_ATTR to add printf style checking to fuse_log. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/fuse_log.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/virtiofsd/fuse_log.h b/tools/virtiofsd/fuse_log.h index

[PATCH] qemu-io-cmds: Simplify help_oneline

2020-08-24 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" help_oneline is declared and starts as: static void help_oneline(const char *cmd, const cmdinfo_t *ct) { if (cmd) { printf("%s ", cmd); } else { printf("%s ", ct->name); if (ct->altname) { printf("(or %s)

[PATCH 2/3] tools/virtiofsd: xattr name mappings: Map client xattr names

2020-08-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names originating at the client; from get/set/remove xattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 101 ++- 1 file changed, 99 insertions(+), 2 deletions(-) diff --git

[PATCH 3/3] tools/virtiofsd: xattr name mappings: Map host xattr names

2020-08-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Map xattr names coming from the host, i.e. listxattr. Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c | 88 1 file changed, 88 insertions(+) diff --git a/tools/virtiofsd/passthrough_ll.c

[PATCH 0/3] virtiofsd xattr name mappings

2020-08-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is a first cut of a xattr name mapping option for virtiofsd. It allows the user of virtiofsd to define a fairly flexible mapping from the view of the xattr names the host fs has and the ones that the guest sees. The hope is this allows things like:

[PATCH 1/3] tools/virtiofsd: xattr name mappings: Add option

2020-08-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add an option to define mappings of xattr names so that the client and host filesystems see different views. This can be used to have different SELinux mappings as seen by the guest, to run the virtiofsd with less privileges (e.g. in a case where it can't set

[PATCH] kvm: kvm_init_vcpu take Error pointer

2020-07-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Clean up the error handling in kvm_init_vcpu so we can see what went wrong more easily. Make it take an Error ** and fill it out with what failed, including the cpu id, so you can tell if it only fails at a given ID. Replace the remaining DPRINTF by a trace.

[PULL 5/5] migration: Count new_dirty instead of real_dirty

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: Keqian Zhu real_dirty_pages becomes equal to total ram size after dirty log sync in ram_init_bitmaps, the reason is that the bitmap of ramblock is initialized to be all set, so old path counts them as "real dirty" at beginning. This causes wrong dirty rate and false positive throttling.

[PULL 4/5] migration: postcopy take proper error return

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This function returns a boolean success and we're returning -1; lets just use the 'out' error path. Signed-off-by: Dr. David Alan Gilbert Fixes: 58b7c17e226 ("Disable mlock around incoming postcopy") Buglink: https://bugs.launchpad.net/qemu/+bug/1885720

[PULL 3/5] virtiofsd: Allow addition or removal of capabilities

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Allow capabilities to be added or removed from the allowed set for the daemon; e.g. default: CapPrm: 88df CapEff: 88df -o modcaps=+sys_admin CapPrm: 882000df CapEff: 882000df -o modcaps=+sys_admin:-chown CapPrm:

[PULL 0/5] migration queue

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 6651620b92bc08cde07cb500e9a43dba7bd9b2b7: Merge remote-tracking branch 'remotes/kraxel/tags/seabios-20200702-pull-request' into staging (2020-07-03 09:55:35 +0100) are available in the Git repository at:

[PULL 2/5] virtiofsd: Check capability calls

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Check the capability calls worked. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Stefan Hajnoczi Acked-by: Vivek Goyal Message-Id: <20200629115420.98443-3-dgilb...@redhat.com> Signed-off-by: Dr. David Alan Gilbert --- tools/virtiofsd/passthrough_ll.c |

[PULL 1/5] virtiofsd: Terminate capability list

2020-07-03 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" capng_updatev is a varargs function that needs a -1 to terminate it, but it was missing. In practice what seems to have been happening is that it's added the capabilities we asked for, then runs into junk on the stack, so if we're unlucky it might be adding some

[PATCH] migration: postcopy take proper error return

2020-07-01 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" This function returns a boolean success and we're returning -1; lets just use the 'out' error path. Signed-off-by: Dr. David Alan Gilbert Fixes: 58b7c17e226 ("Disable mlock around incoming postcopy") Buglink: https://bugs.launchpad.net/qemu/+bug/1885720 ---

<    4   5   6   7   8   9   10   11   12   13   >