Re: [Qemu-devel] [PATCH v7 2/2] Add QTest testcase for the Intel Hexadecimal Object File Loader.

[Su Hang]

Great! Thanks for your suggestion!

Best,
SU Hang

"Steffen Görtz" wrote:
> On 10.05.2018 09:18, Su Hang wrote:
> 
> Hi,
> this will be my first comment on devel as part of my GSoC participation
> this year.
> 
> > +
> > +QTestState *s = qtest_startf(
> > +"-M versatilepb -m 128M -nographic -kernel 
> > ../tests/hex-loader-check-data/test.hex");
> > +
> 
> The test binary "text.hex" is not being copied to out-of-tree builds at
> the moment and the test run will therefore fail.
> The acpi tests also use a fixture (acpi-test-data) which is being copied
> to the out-of-tree build in ./configure:L7181 .
> This might also be an option for your test.

[Qemu-devel] [Bug 1435101] Re: Windows, QEMU 2.2.50 fails to boot XP CD

[Launchpad Bug Tracker]

[Expired for QEMU because there has been no activity for 60 days.]

** Changed in: qemu
   Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1435101

Title:
  Windows, QEMU 2.2.50 fails to boot XP CD

Status in QEMU:
  Expired

Bug description:
  Running XP Pro SP3 host 32bit.  When I launch qemu booting from CD, it fails 
to complete load, getting stuck at "Setup is starting Windows". It does not 
proceed past. I tried to disable floppy but still no go. Download older version 
1.5.1-win32, 0.9.1, same problem. 
  qemu-system-i386w.exe -cdrom "d:\XP.ISO" -hda d:\xp.img -boot d 
  with -global isa-fdc.driveA=c or -no-fd-bootchk switches but no go. I see 
others have run into this problem as well but no real solutions. Some folks 
says turning off floppy works and I tried.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1435101/+subscriptions

Re: [Qemu-devel] [PATCH] iothread: fix epollfd leak in the process of delIOThread

[Peter Xu]

On Wed, May 16, 2018 at 09:38:31AM +0800, Fam Zheng wrote:
> On Tue, 05/15 20:00, Jie Wang wrote:
> > When we call addIOThread, the epollfd created in aio_context_setup,
> > but not close it in the process of delIOThread, so the epollfd will leak.
> > 
> > Signed-off-by: Jie Wang 
> > ---
> >  iothread.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/iothread.c b/iothread.c
> > index aff1281..23ac0a8 100644
> > --- a/iothread.c
> > +++ b/iothread.c
> > @@ -128,6 +128,7 @@ static void iothread_instance_finalize(Object *obj)
> >   * GSources first before destroying any GMainContext.
> >   */
> >  if (iothread->ctx) {
> > +close(iothread->ctx->epollfd);
> >  aio_context_unref(iothread->ctx);
> >  iothread->ctx = NULL;
> >  }
> > -- 
> > 1.8.3.1
> > 
> 
> Please add an aio_context_destroy() function in aio-posix.c and call it from
> aio_context_finalize(). IOThread code should not touch AioContext internals.

I believe Fam means aio_ctx_finalize().

> Also please remember to wrap the close() code in CONFIG_EPOLL_CREATE1.  An 
> empty
> function may need to be added to aio-win32.c, too.

Agreed.

-- 
Peter Xu

Re: [Qemu-devel] [PATCH RFC] qemu-error: introduce error_report_once

[Peter Xu]

On Tue, May 15, 2018 at 05:56:34PM +0200, Markus Armbruster wrote:

[...]

> I see.
> 
> Add a contract comment (suggest to start with the one next to
> error_report()), expand the tabs, replace the reserved identifiers
> (caught by patchew; you can use foo_ instead of __foo), throw in at
> least one use to serve as a showcase (ideally one demonstrating the
> difference to trace points), and consider working some of your
> additional rationale into your commit message.

Will do.  Thanks Markus.

-- 
Peter Xu

Re: [Qemu-devel] [PATCH RFC] qemu-error: introduce error_report_once

[Peter Xu]

On Tue, May 15, 2018 at 10:29:39AM -0500, Eric Blake wrote:
> On 05/15/2018 04:13 AM, Peter Xu wrote:
> > I stole the printk_once() macro.
> > 
> > I always wanted to be able to print some error directly if there is a
> > buffer to dump, however we can't use error_report() really quite often
> > when there can be any DDOS attack.  To avoid that, we can introduce a
> > print-once function for it.
> > 
> > CC: Markus Armbruster 
> > Signed-off-by: Peter Xu 
> > ---
> > We can for sure introduce similar functions for the rest of the
> > error_*() functions, it's just an idea to see whether we'd like it
> > in general.
> > ---
> >   include/qemu/error-report.h | 12 
> >   1 file changed, 12 insertions(+)
> > 
> > diff --git a/include/qemu/error-report.h b/include/qemu/error-report.h
> > index e1c8ae1a52..efebb80e2c 100644
> > --- a/include/qemu/error-report.h
> > +++ b/include/qemu/error-report.h
> > @@ -44,6 +44,18 @@ void error_report(const char *fmt, ...) GCC_FMT_ATTR(1, 
> > 2);
> >   void warn_report(const char *fmt, ...) GCC_FMT_ATTR(1, 2);
> >   void info_report(const char *fmt, ...) GCC_FMT_ATTR(1, 2);
> > +#define error_report_once(fmt, ...) \
> > +({  \
> > +static bool __print_once;   \
> 
> Double-underscore names are reserved for the compiler's use, not ours.
> Better would be naming this:
> 
> static bool print_once_;
> 
> with a trailing underscore, or at most a single leading underscore.
> 
> > +bool __ret_print_once = !__print_once;  \
> 
> Same comment for this variable.

Sure!

(I am wondering why Linux is always using that way to name lots of
 variables, and I'm surprised that I got 385350 after I run this under
 the Linux repo: 'git grep "__[a-z][a-z]" | wc -l', even considering
 some false positives)

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH qemu v2] slirp/debug: Print IP addresses in human readable form

[Alexey Kardashevskiy]

On 16/5/18 12:30 am, Eric Blake wrote:
> On 05/14/2018 02:00 AM, Alexey Kardashevskiy wrote:
>> On 13/3/18 6:44 pm, Samuel Thibault wrote:
>>> Alexey Kardashevskiy, on mar. 13 mars 2018 15:49:44 +1100, wrote:
 Signed-off-by: Alexey Kardashevskiy 
>>>
>>> Applied to my tree, thanks!
>>
>>
>> And what is your tree, is this something to be merged sometime later
>> somewhere? :)
> 
> Per MAINTAINERS:
> 
> SLIRP
> M: Samuel Thibault 
> M: Jan Kiszka 
> S: Maintained
> F: slirp/
> F: net/slirp.c
> F: include/net/slirp.h
> T: git git://git.kiszka.org/qemu.git queues/slirp
> 
> which shows Jan's staging tree, but not Samuel's.
> 
> It's not a requirement for a maintainer to have a public-facing staging
> tree, but many of them do, as it gives you a chance to test that what will
> later be in a pull request matches what you expect.
> 
> At any rate,
> https://wiki.qemu.org/Contribute/SubmitAPatch#Is_my_patch_in.3F mentions
> that maintainers will batch together various related patches and send a
> pull request for inclusion in the main repository, thus even when a patch
> has been reviewed and staged, it may be another week or two before it lands
> in mainline.  Yes, it can feel slow, but in general it works out for the
> best (as we have more chances to flag potential problems before they affect
> everyone by being in mainline).


If it was 2 weeks or even 6 - I would not bother but when it is 2 months,
chances are it is forgotten/lost somewhere, hence my ping. I am not
complaining at all, just pinging :)



-- 
Alexey

Re: [Qemu-devel] [PATCH v6 0/4] slirp: Add query-usernet QMP command

[Fam Zheng]

On Fri, 05/04 15:42, Fam Zheng wrote:
> v6: Rename QAPI enum: TCPS -> UsernetTcpState. [Daniel]

Ping?

Fam

Re: [Qemu-devel] [PATCH v3 0/6] Extend vhost-user to support registering external host notifiers

[Tiwei Bie]

On Wed, May 16, 2018 at 04:41:48AM +0300, Michael S. Tsirkin wrote:
> On Thu, Apr 12, 2018 at 11:12:26PM +0800, Tiwei Bie wrote:
> > The original subject is: Extend vhost-user to support VFIO based 
> > accelerators
> > 
> > Update notes
> > 
> > 
> > Now, this patch set just focuses on adding the support for
> > registering memory region based host notifiers. With this
> > support, guest driver in the VM will be able to notify the
> > hardware device at the vhost backend directly.
> > 
> > It's one of the most important things in vDPA -- the host
> > notification offload. Because, normally, the hardware device
> > heavily depends on the notifications. Without this support,
> > there will be a lot of VM-Exit happen due to the notifications
> > from guest driver (it will drop the VM performance) and a
> > lot of CPU resources wasted to do the notification relay
> > (it will make the hardware offload less attractive, because
> > one important goal of hardware offload is to free the CPU
> > resources).
> > 
> > More backgrounds of this patch set can be found from the
> > cover letter of the previous versions:
> > 
> > RFC: http://lists.nongnu.org/archive/html/qemu-devel/2017-12/msg04844.html
> > v1:  http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg06028.html
> > v2:  http://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg05009.html
> 
> So I don't think there are any outstanding issues here.
> Did I miss anything?

For the host notifiers support in vhost-user, it's
everything here!

The support in libvhost-user and vhost-user-bridge
is being done in another patchset. And we are also
working on the vIOMMU support. If we really see any
new issues in the future, I'll fix them in small
patches actively.

Thank you very much!

Best regards,
Tiwei Bie

> 
> > v2 -> v3:
> > - A better implementation of the shared vhost-user state (MST);
> > - Use bus callback to add/delete subregions for notification (MST);
> > - Refine APIs' names which add/delete subregions for notification (MST);
> > - Refine the doc of the new vhost-user types and messages (MST);
> > - Separate host notification offload from the guest notification offload 
> > (MST);
> > - Drop the guest notification offload support from this patch;
> > - Add memory filter for vhost backend to filter the sections they can 
> > handle;
> > 
> > v1 -> v2:
> > - Add some explanations about why extend vhost-user in commit log (Paolo);
> > - Bug fix in slave_read() according to Stefan's fix in DPDK;
> > - Remove IOMMU feature check and related commit log;
> > - Some minor refinements;
> > - Rebase to the latest QEMU;
> > 
> > RFC -> v1:
> > - Add some details about how vDPA works in cover letter (Alexey)
> > - Add some details about the OVS offload use-case in cover letter (Jason)
> > - Move PCI specific stuffs out of vhost-user (Jason)
> > - Handle the virtual IOMMU case (Jason)
> > - Move VFIO group management code into vfio/common.c (Alex)
> > - Various refinements;
> > (approximately sorted by comment posting time)
> > 
> > Tiwei Bie (6):
> >   vhost-user: add Net prefix to internal state structure
> >   vhost-user: introduce shared vhost-user state
> >   vhost-user: support receiving file descriptors in slave_read
> >   virtio: support setting memory region based host notifier
> >   vhost: allow backends to filter memory sections
> >   vhost-user: support registering external host notifiers
> > 
> >  backends/cryptodev-vhost-user.c |  20 +++-
> >  docs/interop/vhost-user.txt |  33 ++
> >  hw/block/vhost-user-blk.c   |  22 +++-
> >  hw/scsi/vhost-user-scsi.c   |  20 +++-
> >  hw/virtio/Makefile.objs |   2 +-
> >  hw/virtio/vhost-stub.c  |  10 ++
> >  hw/virtio/vhost-user.c  | 206 
> > ++--
> >  hw/virtio/vhost.c   |   9 +-
> >  hw/virtio/virtio-pci.c  |  22 
> >  hw/virtio/virtio.c  |  13 +++
> >  include/hw/virtio/vhost-backend.h   |   4 +
> >  include/hw/virtio/vhost-user-blk.h  |   2 +
> >  include/hw/virtio/vhost-user-scsi.h |   2 +
> >  include/hw/virtio/vhost-user.h  |  28 +
> >  include/hw/virtio/virtio-bus.h  |   2 +
> >  include/hw/virtio/virtio.h  |   2 +
> >  net/vhost-user.c|  78 +-
> >  17 files changed, 433 insertions(+), 42 deletions(-)
> >  create mode 100644 include/hw/virtio/vhost-user.h
> > 
> > -- 
> > 2.11.0

Re: [Qemu-devel] [PATCH v3 0/6] Extend vhost-user to support registering external host notifiers

[Michael S. Tsirkin]

On Thu, Apr 12, 2018 at 11:12:26PM +0800, Tiwei Bie wrote:
> The original subject is: Extend vhost-user to support VFIO based accelerators
> 
> Update notes
> 
> 
> Now, this patch set just focuses on adding the support for
> registering memory region based host notifiers. With this
> support, guest driver in the VM will be able to notify the
> hardware device at the vhost backend directly.
> 
> It's one of the most important things in vDPA -- the host
> notification offload. Because, normally, the hardware device
> heavily depends on the notifications. Without this support,
> there will be a lot of VM-Exit happen due to the notifications
> from guest driver (it will drop the VM performance) and a
> lot of CPU resources wasted to do the notification relay
> (it will make the hardware offload less attractive, because
> one important goal of hardware offload is to free the CPU
> resources).
> 
> More backgrounds of this patch set can be found from the
> cover letter of the previous versions:
> 
> RFC: http://lists.nongnu.org/archive/html/qemu-devel/2017-12/msg04844.html
> v1:  http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg06028.html
> v2:  http://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg05009.html

So I don't think there are any outstanding issues here.
Did I miss anything?

> v2 -> v3:
> - A better implementation of the shared vhost-user state (MST);
> - Use bus callback to add/delete subregions for notification (MST);
> - Refine APIs' names which add/delete subregions for notification (MST);
> - Refine the doc of the new vhost-user types and messages (MST);
> - Separate host notification offload from the guest notification offload 
> (MST);
> - Drop the guest notification offload support from this patch;
> - Add memory filter for vhost backend to filter the sections they can handle;
> 
> v1 -> v2:
> - Add some explanations about why extend vhost-user in commit log (Paolo);
> - Bug fix in slave_read() according to Stefan's fix in DPDK;
> - Remove IOMMU feature check and related commit log;
> - Some minor refinements;
> - Rebase to the latest QEMU;
> 
> RFC -> v1:
> - Add some details about how vDPA works in cover letter (Alexey)
> - Add some details about the OVS offload use-case in cover letter (Jason)
> - Move PCI specific stuffs out of vhost-user (Jason)
> - Handle the virtual IOMMU case (Jason)
> - Move VFIO group management code into vfio/common.c (Alex)
> - Various refinements;
> (approximately sorted by comment posting time)
> 
> Tiwei Bie (6):
>   vhost-user: add Net prefix to internal state structure
>   vhost-user: introduce shared vhost-user state
>   vhost-user: support receiving file descriptors in slave_read
>   virtio: support setting memory region based host notifier
>   vhost: allow backends to filter memory sections
>   vhost-user: support registering external host notifiers
> 
>  backends/cryptodev-vhost-user.c |  20 +++-
>  docs/interop/vhost-user.txt |  33 ++
>  hw/block/vhost-user-blk.c   |  22 +++-
>  hw/scsi/vhost-user-scsi.c   |  20 +++-
>  hw/virtio/Makefile.objs |   2 +-
>  hw/virtio/vhost-stub.c  |  10 ++
>  hw/virtio/vhost-user.c  | 206 
> ++--
>  hw/virtio/vhost.c   |   9 +-
>  hw/virtio/virtio-pci.c  |  22 
>  hw/virtio/virtio.c  |  13 +++
>  include/hw/virtio/vhost-backend.h   |   4 +
>  include/hw/virtio/vhost-user-blk.h  |   2 +
>  include/hw/virtio/vhost-user-scsi.h |   2 +
>  include/hw/virtio/vhost-user.h  |  28 +
>  include/hw/virtio/virtio-bus.h  |   2 +
>  include/hw/virtio/virtio.h  |   2 +
>  net/vhost-user.c|  78 +-
>  17 files changed, 433 insertions(+), 42 deletions(-)
>  create mode 100644 include/hw/virtio/vhost-user.h
> 
> -- 
> 2.11.0

Re: [Qemu-devel] [PATCH] iothread: fix epollfd leak in the process of delIOThread

[Fam Zheng]

On Tue, 05/15 20:00, Jie Wang wrote:
> When we call addIOThread, the epollfd created in aio_context_setup,
> but not close it in the process of delIOThread, so the epollfd will leak.
> 
> Signed-off-by: Jie Wang 
> ---
>  iothread.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/iothread.c b/iothread.c
> index aff1281..23ac0a8 100644
> --- a/iothread.c
> +++ b/iothread.c
> @@ -128,6 +128,7 @@ static void iothread_instance_finalize(Object *obj)
>   * GSources first before destroying any GMainContext.
>   */
>  if (iothread->ctx) {
> +close(iothread->ctx->epollfd);
>  aio_context_unref(iothread->ctx);
>  iothread->ctx = NULL;
>  }
> -- 
> 1.8.3.1
> 

Please add an aio_context_destroy() function in aio-posix.c and call it from
aio_context_finalize(). IOThread code should not touch AioContext internals.
Also please remember to wrap the close() code in CONFIG_EPOLL_CREATE1.  An empty
function may need to be added to aio-win32.c, too.

Fam

Re: [Qemu-devel] [PATCH v2 1/1] sandbox: disable -sandbox if CONFIG_SECCOMP undefined

[Yi Min Zhao]

在 2018/5/15 下午11:25, Eric Blake 写道:

On 05/15/2018 06:33 AM, Yi Min Zhao wrote:

If CONFIG_SECCOMP is undefined, the option 'elevateprivileges' remains
compiled. This would make libvirt set the corresponding capability and
then trigger the guest startup fails. So this patch excludes the code


s/trigger the guest startup fails/trigger failure during guest startup/


regarding seccomp staff if CONFIG_SECCOMP is undefined.


s/staff/command line options/



Signed-off-by: Yi Min Zhao 
---
  vl.c | 13 -
  1 file changed, 8 insertions(+), 5 deletions(-)



A maintainer can touch up the commit message, so:
Reviewed-by: Eric Blake 


Thanks for your comments! Have updated commit msg.

[Qemu-devel] [PULL 40/40] Migration+TLS: Fix crash due to double cleanup

[Juan Quintela]

From: "Dr. David Alan Gilbert" 

During a TLS connect we see:
  migration_channel_connect calls
  migration_tls_channel_connect
  (calls after TLS setup)
  migration_channel_connect

My previous error handling fix made migration_channel_connect
call migrate_fd_connect in all cases; unfortunately the above
means it gets called twice and crashes doing double cleanup.

Fixes: 688a3dcba98

Reported-by: Peter Krempa 
Signed-off-by: Dr. David Alan Gilbert 
Reviewed-by: Daniel P. Berrangé 
Message-Id: <20180430185943.35714-1-dgilb...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/channel.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/migration/channel.c b/migration/channel.c
index 716192bf75..33e0e9b82f 100644
--- a/migration/channel.c
+++ b/migration/channel.c
@@ -71,6 +71,15 @@ void migration_channel_connect(MigrationState *s,
 !object_dynamic_cast(OBJECT(ioc),
  TYPE_QIO_CHANNEL_TLS)) {
 migration_tls_channel_connect(s, ioc, hostname, );
+
+if (!error) {
+/* tls_channel_connect will call back to this
+ * function after the TLS handshake,
+ * so we mustn't call migrate_fd_connect until then
+ */
+
+return;
+}
 } else {
 QEMUFile *f = qemu_fopen_channel_output(ioc);
 
-- 
2.17.0

[Qemu-devel] [PULL 38/40] migration: update index field when delete or qsort RDMALocalBlock

[Juan Quintela]

From: Lidong Chen 

rdma_delete_block function deletes RDMALocalBlock base on index field,
but not update the index field. So when next time invoke rdma_delete_block,
it will not work correctly.

If start and cancel migration repeatedly, some RDMALocalBlock not invoke
ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than
max locked memory limitation, ibv_reg_mr will failed, and migration can not
start successfully again.

Signed-off-by: Lidong Chen 
Reviewed-by: Dr. David Alan Gilbert 
Message-Id: <1525618499-1560-1-git-send-email-lidongc...@tencent.com>
Signed-off-by: Juan Quintela 

Signed-off-by: Lidong Chen 
---
 migration/rdma.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/migration/rdma.c b/migration/rdma.c
index da474fc19f..7d233b0820 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -708,6 +708,9 @@ static int rdma_delete_block(RDMAContext *rdma, 
RDMALocalBlock *block)
 memcpy(local->block + block->index, old + (block->index + 1),
 sizeof(RDMALocalBlock) *
 (local->nb_blocks - (block->index + 1)));
+for (x = block->index; x < local->nb_blocks - 1; x++) {
+local->block[x].index--;
+}
 }
 } else {
 assert(block == local->block);
@@ -3246,6 +3249,10 @@ static int qemu_rdma_registration_handle(QEMUFile *f, 
void *opaque)
 qsort(rdma->local_ram_blocks.block,
   rdma->local_ram_blocks.nb_blocks,
   sizeof(RDMALocalBlock), dest_ram_sort_func);
+for (i = 0; i < local->nb_blocks; i++) {
+local->block[i].index = i;
+}
+
 if (rdma->pin_all) {
 ret = qemu_rdma_reg_whole_ram_blocks(rdma);
 if (ret) {
-- 
2.17.0

[Qemu-devel] [PULL 32/40] qmp/migration: new command migrate-recover

[Juan Quintela]

From: Peter Xu 

The first allow-oob=true command.  It's used on destination side when
the postcopy migration is paused and ready for a recovery.  After
execution, a new migration channel will be established for postcopy to
continue.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-21-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
s/2.12/2.13/
---
 migration/migration.c | 24 
 migration/migration.h |  1 +
 migration/savevm.c|  3 +++
 qapi/migration.json   | 20 
 4 files changed, 48 insertions(+)

diff --git a/migration/migration.c b/migration/migration.c
index c1832b0263..1beb5e07fb 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1475,6 +1475,30 @@ void qmp_migrate_incoming(const char *uri, Error **errp)
 once = false;
 }
 
+void qmp_migrate_recover(const char *uri, Error **errp)
+{
+MigrationIncomingState *mis = migration_incoming_get_current();
+
+if (mis->state != MIGRATION_STATUS_POSTCOPY_PAUSED) {
+error_setg(errp, "Migrate recover can only be run "
+   "when postcopy is paused.");
+return;
+}
+
+if (atomic_cmpxchg(>postcopy_recover_triggered,
+   false, true) == true) {
+error_setg(errp, "Migrate recovery is triggered already");
+return;
+}
+
+/*
+ * Note that this call will never start a real migration; it will
+ * only re-setup the migration stream and poke existing migration
+ * to continue using that newly established channel.
+ */
+qemu_start_incoming_migration(uri, errp);
+}
+
 bool migration_is_blocked(Error **errp)
 {
 if (qemu_savevm_state_blocked(errp)) {
diff --git a/migration/migration.h b/migration/migration.h
index b4438ccb65..f83f1064b5 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -77,6 +77,7 @@ struct MigrationIncomingState {
 struct PostcopyBlocktimeContext *blocktime_ctx;
 
 /* notify PAUSED postcopy incoming migrations to try to continue */
+bool postcopy_recover_triggered;
 QemuSemaphore postcopy_pause_sem_dst;
 QemuSemaphore postcopy_pause_sem_fault;
 };
diff --git a/migration/savevm.c b/migration/savevm.c
index 8e7653badc..4251125831 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2190,6 +2190,9 @@ static bool 
postcopy_pause_incoming(MigrationIncomingState *mis)
 {
 trace_postcopy_pause_incoming();
 
+/* Clear the triggered bit to allow one recovery */
+mis->postcopy_recover_triggered = false;
+
 migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
   MIGRATION_STATUS_POSTCOPY_PAUSED);
 
diff --git a/qapi/migration.json b/qapi/migration.json
index 4e8e61ceef..da351d7421 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1191,3 +1191,23 @@
 # Since: 2.9
 ##
 { 'command': 'xen-colo-do-checkpoint' }
+
+##
+# @migrate-recover:
+#
+# Provide a recovery migration stream URI.
+#
+# @uri: the URI to be used for the recovery of migration stream.
+#
+# Returns: nothing.
+#
+# Example:
+#
+# -> { "execute": "migrate-recover",
+#  "arguments": { "uri": "tcp:192.168.1.200:12345" } }
+# <- { "return": {} }
+#
+# Since: 2.13
+##
+{ 'command': 'migrate-recover', 'data': { 'uri': 'str' },
+  'allow-oob': true }
-- 
2.17.0

[Qemu-devel] [Bug 1771238] Re: Not able to passthrough > 32 PCIe devices to a KVM Guest

[David Coronel]

@Khaled El Mously: It's more a feature request.

** No longer affects: qemu

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1771238

Title:
  Not able to passthrough > 32 PCIe devices to a KVM Guest

Status in qemu package in Ubuntu:
  New

Bug description:
  Using an Ubuntu Server 16.04-based host with KVM hypervisor installed,
  we are unable to launch a vanilla Ubuntu Server 16.04.4 guest with >=
  32 PCIe devices. It is 100% reproducible. Using fewer PCIe devices
  works fine. We are using the vanilla kvm and qemu packages from the
  Canonical repos.

  The ultimate goal is to create a KVM Guest wherein I can passthrough
  44 PCI devices.

  When a KVM Guest launches, it also has some internal PCIe devices
  including host bridge, USB, IDE (for virtual disk), and virtual nic
  etc.

  Script used to launch all devices looks like this:

  #!/bin/bash 
  NAME=16gpuvm 

  sudo qemu-img create -f qcow2 /home/lab/kvm/images/${NAME}.img 40G && 
  sudo virt-install \ 
  --name ${NAME} \ 
  --ram 716800 \ 
  --vcpus 88 \ 
  --disk path=/home/lab/kvm/images/${NAME}.img,format=qcow2 \ 
  --network bridge=virbr0 \ 
  --graphics none \ 
  --host-device 34:00.0 \ 
  --host-device 36:00.0 \ 
  --host-device 39:00.0 \ 
  --host-device 3b:00.0 \ 
  --host-device 57:00.0 \ 
  --host-device 59:00.0 \ 
  --host-device 5c:00.0 \ 
  --host-device 5e:00.0 \ 
  --host-device 61:00.0 \ 
  --host-device 62:00.0 \ 
  --host-device 63:00.0 \ 
  --host-device 65:00.0 \ 
  --host-device 66:00.0 \ 
  --host-device 67:00.0 \ 
  --host-device 35:00.0 \ 
  --host-device 3a:00.0 \ 
  --host-device 58:00.0 \ 
  --host-device 5d:00.0 \ 
  --host-device 2e:00.0 \ 
  --host-device 2f:00.0 \ 
  --host-device 51:00.0 \ 
  --host-device 52:00.0 \ 
  --host-device b7:00.0 \ 
  --host-device b9:00.0 \ 
  --host-device bc:00.0 \ 
  --host-device be:00.0 \ 
  --host-device e0:00.0 \ 
  --host-device e2:00.0 \ 
  --host-device e5:00.0 \ 
  --host-device e7:00.0 \ 
  --host-device c1:00.0 \ 
  --host-device c2:00.0 \ 
  --host-device c3:00.0 \ 
  --host-device c5:00.0 \ 
  --host-device c6:00.0 \ 
  --host-device c7:00.0 \ 
  --host-device b8:00.0 \ 
  --host-device bd:00.0 \ 
  --host-device e1:00.0 \ 
  --host-device e6:00.0 \ 
  --host-device b1:00.0 \ 
  --host-device b2:00.0 \ 
  --host-device da:00.0 \ 
  --host-device db:00.0 \ 
  --console pty,target_type=serial \ 
  --location http://ftp.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64 \ 
  --initrd-inject=/home/lab/kvm/images/preseed.cfg \ 
  --extra-args=" 
  console=ttyS0,115200 
  locale=en_US 
  console-keymaps-at/keymap=us 
  console-setup/ask_detect=false 
  console-setup/layoutcode=us 
  keyboard-configuration/layout=USA 
  keyboard-configuration/variant=USA 
  hostname=${NAME} 
  file=file:/preseed.cfg 
  "

  Passing > 32 device causes this issue:  32nd device hits a DPC error
  and the host/HV crashes:

  
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.977496] dpc :5b:10.0:pcie210: 
DPC containment event, status:0x0009 source:0x
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.977500] dpc :5b:10.0:pcie210: 
DPC unmasked uncorrectable error detected, remove downstream devices
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.994326] vfio-pci :5e:00.0: 
Refused to change power state, currently in D3
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.994427] iommu: Removing device 
:5e:00.0 from group 92

  
  From syslog (attached)

  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194358] dpc :bb:04.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194387] dpc :bb:10.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194413] dpc :d9:00.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194439] dpc :d9:01.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194472] dpc :d9:02.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194499] dpc :d9:03.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194526] dpc :d9:04.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194553] dpc :d9:0c.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 

[Qemu-devel] [PULL 24/40] migration: new message MIG_RP_MSG_RECV_BITMAP

[Juan Quintela]

From: Peter Xu 

Introducing new return path message MIG_RP_MSG_RECV_BITMAP to send
received bitmap of ramblock back to source.

This is the reply message of MIG_CMD_RECV_BITMAP, it contains not only
the header (including the ramblock name), and it was appended with the
whole ramblock received bitmap on the destination side.

When the source receives such a reply message (MIG_RP_MSG_RECV_BITMAP),
it parses it, convert it to the dirty bitmap by inverting the bits.

One thing to mention is that, when we send the recv bitmap, we are doing
these things in extra:

- converting the bitmap to little endian, to support when hosts are
  using different endianess on src/dst.

- do proper alignment for 8 bytes, to support when hosts are using
  different word size (32/64 bits) on src/dst.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-13-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  |  68 +++
 migration/migration.h  |   2 +
 migration/ram.c| 144 +
 migration/ram.h|   3 +
 migration/savevm.c |   2 +-
 migration/trace-events |   3 +
 6 files changed, 221 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index ec3bc9ae20..7c5e20b3f6 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -95,6 +95,7 @@ enum mig_rp_message_type {
 
 MIG_RP_MSG_REQ_PAGES_ID, /* data (start: be64, len: be32, id: string) */
 MIG_RP_MSG_REQ_PAGES,/* data (start: be64, len: be32) */
+MIG_RP_MSG_RECV_BITMAP,  /* send recved_bitmap back to source */
 
 MIG_RP_MSG_MAX
 };
@@ -524,6 +525,45 @@ void migrate_send_rp_pong(MigrationIncomingState *mis,
 migrate_send_rp_message(mis, MIG_RP_MSG_PONG, sizeof(buf), );
 }
 
+void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis,
+ char *block_name)
+{
+char buf[512];
+int len;
+int64_t res;
+
+/*
+ * First, we send the header part. It contains only the len of
+ * idstr, and the idstr itself.
+ */
+len = strlen(block_name);
+buf[0] = len;
+memcpy(buf + 1, block_name, len);
+
+if (mis->state != MIGRATION_STATUS_POSTCOPY_RECOVER) {
+error_report("%s: MSG_RP_RECV_BITMAP only used for recovery",
+ __func__);
+return;
+}
+
+migrate_send_rp_message(mis, MIG_RP_MSG_RECV_BITMAP, len + 1, buf);
+
+/*
+ * Next, we dump the received bitmap to the stream.
+ *
+ * TODO: currently we are safe since we are the only one that is
+ * using the to_src_file handle (fault thread is still paused),
+ * and it's ok even not taking the mutex. However the best way is
+ * to take the lock before sending the message header, and release
+ * the lock after sending the bitmap.
+ */
+qemu_mutex_lock(>rp_mutex);
+res = ramblock_recv_bitmap_send(mis->to_src_file, block_name);
+qemu_mutex_unlock(>rp_mutex);
+
+trace_migrate_send_rp_recv_bitmap(block_name, res);
+}
+
 MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp)
 {
 MigrationCapabilityStatusList *head = NULL;
@@ -1802,6 +1842,7 @@ static struct rp_cmd_args {
 [MIG_RP_MSG_PONG]   = { .len =  4, .name = "PONG" },
 [MIG_RP_MSG_REQ_PAGES]  = { .len = 12, .name = "REQ_PAGES" },
 [MIG_RP_MSG_REQ_PAGES_ID]   = { .len = -1, .name = "REQ_PAGES_ID" },
+[MIG_RP_MSG_RECV_BITMAP]= { .len = -1, .name = "RECV_BITMAP" },
 [MIG_RP_MSG_MAX]= { .len = -1, .name = "MAX" },
 };
 
@@ -1846,6 +1887,19 @@ static bool 
postcopy_pause_return_path_thread(MigrationState *s)
 return true;
 }
 
+static int migrate_handle_rp_recv_bitmap(MigrationState *s, char *block_name)
+{
+RAMBlock *block = qemu_ram_block_by_name(block_name);
+
+if (!block) {
+error_report("%s: invalid block name '%s'", __func__, block_name);
+return -EINVAL;
+}
+
+/* Fetch the received bitmap and refresh the dirty bitmap */
+return ram_dirty_bitmap_reload(s, block);
+}
+
 /*
  * Handles messages sent on the return path towards the source VM
  *
@@ -1951,6 +2005,20 @@ retry:
 migrate_handle_rp_req_pages(ms, (char *)[13], start, len);
 break;
 
+case MIG_RP_MSG_RECV_BITMAP:
+if (header_len < 1) {
+error_report("%s: missing block name", __func__);
+mark_source_rp_bad(ms);
+goto out;
+}
+/* Format: len (1B) + idstr (<255B). This ends the idstr. */
+buf[buf[0] + 1] = '\0';
+if (migrate_handle_rp_recv_bitmap(ms, (char *)(buf + 1))) {
+mark_source_rp_bad(ms);
+goto out;
+}
+break;
+
 default:
 break;
 }
diff --git a/migration/migration.h 

[Qemu-devel] [PULL 25/40] migration: new cmd MIG_CMD_POSTCOPY_RESUME

[Juan Quintela]

From: Peter Xu 

Introducing this new command to be sent when the source VM is ready to
resume the paused migration.  What the destination does here is
basically release the fault thread to continue service page faults.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-14-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/savevm.c | 35 +++
 migration/savevm.h |  1 +
 migration/trace-events |  2 ++
 3 files changed, 38 insertions(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index 7176b350d5..a7e793eef7 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -80,6 +80,7 @@ enum qemu_vm_cmd {
 MIG_CMD_POSTCOPY_RAM_DISCARD,  /* A list of pages to discard that
   were previously sent during
   precopy but are dirty. */
+MIG_CMD_POSTCOPY_RESUME,   /* resume postcopy on dest */
 MIG_CMD_PACKAGED,  /* Send a wrapped stream within this stream */
 MIG_CMD_RECV_BITMAP,   /* Request for recved bitmap on dst */
 MIG_CMD_MAX
@@ -98,6 +99,7 @@ static struct mig_cmd_args {
 [MIG_CMD_POSTCOPY_RUN] = { .len =  0, .name = "POSTCOPY_RUN" },
 [MIG_CMD_POSTCOPY_RAM_DISCARD] = {
.len = -1, .name = "POSTCOPY_RAM_DISCARD" },
+[MIG_CMD_POSTCOPY_RESUME]  = { .len =  0, .name = "POSTCOPY_RESUME" },
 [MIG_CMD_PACKAGED] = { .len =  4, .name = "PACKAGED" },
 [MIG_CMD_RECV_BITMAP]  = { .len = -1, .name = "RECV_BITMAP" },
 [MIG_CMD_MAX]  = { .len = -1, .name = "MAX" },
@@ -958,6 +960,12 @@ void qemu_savevm_send_postcopy_run(QEMUFile *f)
 qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RUN, 0, NULL);
 }
 
+void qemu_savevm_send_postcopy_resume(QEMUFile *f)
+{
+trace_savevm_send_postcopy_resume();
+qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RESUME, 0, NULL);
+}
+
 void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name)
 {
 size_t len;
@@ -1768,6 +1776,30 @@ static int 
loadvm_postcopy_handle_run(MigrationIncomingState *mis)
 return LOADVM_QUIT;
 }
 
+static int loadvm_postcopy_handle_resume(MigrationIncomingState *mis)
+{
+if (mis->state != MIGRATION_STATUS_POSTCOPY_RECOVER) {
+error_report("%s: illegal resume received", __func__);
+/* Don't fail the load, only for this. */
+return 0;
+}
+
+/*
+ * This means source VM is ready to resume the postcopy migration.
+ * It's time to switch state and release the fault thread to
+ * continue service page faults.
+ */
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_RECOVER,
+  MIGRATION_STATUS_POSTCOPY_ACTIVE);
+qemu_sem_post(>postcopy_pause_sem_fault);
+
+trace_loadvm_postcopy_handle_resume();
+
+/* TODO: Tell source that "we are ready" */
+
+return 0;
+}
+
 /**
  * Immediately following this command is a blob of data containing an embedded
  * chunk of migration stream; read it and load it.
@@ -1933,6 +1965,9 @@ static int loadvm_process_command(QEMUFile *f)
 case MIG_CMD_POSTCOPY_RAM_DISCARD:
 return loadvm_postcopy_ram_handle_discard(mis, len);
 
+case MIG_CMD_POSTCOPY_RESUME:
+return loadvm_postcopy_handle_resume(mis);
+
 case MIG_CMD_RECV_BITMAP:
 return loadvm_handle_recv_bitmap(mis, len);
 }
diff --git a/migration/savevm.h b/migration/savevm.h
index b8cee00d41..b24ff073ad 100644
--- a/migration/savevm.h
+++ b/migration/savevm.h
@@ -47,6 +47,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t 
*buf, size_t len);
 void qemu_savevm_send_postcopy_advise(QEMUFile *f);
 void qemu_savevm_send_postcopy_listen(QEMUFile *f);
 void qemu_savevm_send_postcopy_run(QEMUFile *f);
+void qemu_savevm_send_postcopy_resume(QEMUFile *f);
 void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name);
 
 void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
diff --git a/migration/trace-events b/migration/trace-events
index 72e57089f3..29cc10f299 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -18,6 +18,7 @@ loadvm_postcopy_handle_listen(void) ""
 loadvm_postcopy_handle_run(void) ""
 loadvm_postcopy_handle_run_cpu_sync(void) ""
 loadvm_postcopy_handle_run_vmstart(void) ""
+loadvm_postcopy_handle_resume(void) ""
 loadvm_postcopy_ram_handle_discard(void) ""
 loadvm_postcopy_ram_handle_discard_end(void) ""
 loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len) 
"%s: %ud"
@@ -35,6 +36,7 @@ savevm_send_open_return_path(void) ""
 savevm_send_ping(uint32_t val) "0x%x"
 savevm_send_postcopy_listen(void) ""
 savevm_send_postcopy_run(void) ""
+savevm_send_postcopy_resume(void) ""
 savevm_send_recv_bitmap(char *name) "%s"
 savevm_state_setup(void) ""
 savevm_state_header(void) ""
-- 
2.17.0

Re: [Qemu-devel] [PATCH] WHPX Add signature CPUID

[Alessandro Pilotti]

Hi Paolo,

The main reason for different signatures is to allow guest workloads to be 
aware of the differences between the two platforms (eg VirtIO vs VMBus).

Thanks,

Alessandro

> On 15 May 2018, at 16:44, Paolo Bonzini  wrote:
> 
>> On 15/05/2018 13:37, petrutlucia...@gmail.com wrote:
>> From: Lucian Petrut 
>> 
>> Adds the CPUID trap for CPUID 0x4000, sending the WHPX signature
>> to the guest upon request. This is consistent with other QEMU
>> accelerators (KVM).
>> 
>> Signed-off-by: Alessandro Pilotti 
>> Signed-off-by: Justin Terry (VM) 
>> Signed-off-by: Lucian Petrut 
> 
> Is it worth defining a different signature?  Can WHPX implement part of
> the Hyper-V spec, and if so would it be better to return the Hv
> signature ("Hv#1") instead?
> 
> Thanks,
> 
> Paolo
> 
>> ---
>> As opposed to the previous patch, this one will set the result of
>> this specific CPUID leaf when initializing the accelerator so that
>> we avoid a vcpu exit.
>> 
>> target/i386/whpx-all.c | 23 +++
>> 1 file changed, 23 insertions(+)
>> 
>> diff --git a/target/i386/whpx-all.c b/target/i386/whpx-all.c
>> index 5843517..c8310de 100644
>> --- a/target/i386/whpx-all.c
>> +++ b/target/i386/whpx-all.c
>> @@ -29,6 +29,8 @@
>> #include 
>> #include 
>> 
>> +#define WHPX_CPUID_SIGNATURE 0x4000
>> +
>> struct whpx_state {
>> uint64_t mem_quota;
>> WHV_PARTITION_HANDLE partition;
>> @@ -1342,6 +1344,27 @@ static int whpx_accel_init(MachineState *ms)
>>  cpuidExitList,
>>  RTL_NUMBER_OF(cpuidExitList) * 
>> sizeof(UINT32));
>> 
>> +UINT32 signature[3] = {0};
>> +memcpy(signature, "WHPXWHPXWHPX", 12);
>> +
>> +WHV_X64_CPUID_RESULT cpuidResultList[1] = {0};
>> +cpuidResultList[0].Function = WHPX_CPUID_SIGNATURE;
>> +cpuidResultList[0].Eax = 0;
>> +cpuidResultList[0].Ebx = signature[0];
>> +cpuidResultList[0].Ecx = signature[1];
>> +cpuidResultList[0].Edx = signature[2];
>> +hr = WHvSetPartitionProperty(whpx->partition,
>> + WHvPartitionPropertyCodeCpuidResultList,
>> + cpuidResultList,
>> + RTL_NUMBER_OF(cpuidResultList) *
>> +sizeof(WHV_X64_CPUID_RESULT));
>> +if (FAILED(hr)) {
>> +error_report("WHPX: Failed to set partition CpuidResultList 
>> hr=%08lx",
>> + hr);
>> +ret = -EINVAL;
>> +goto error;
>> +}
>> +
>> if (FAILED(hr)) {
>> error_report("WHPX: Failed to set partition CpuidExitList hr=%08lx",
>>  hr);
>> 
> 

[Qemu-devel] [PULL 37/40] migration: update docs

[Juan Quintela]

From: "Dr. David Alan Gilbert" 

Update the migration docs:

Among other changes:
  * Added a general list of advice for device authors
  * Reordered the section on conditional state (subsections etc)
into the order we prefer.
  * Add a note about firmware

Signed-off-by: Dr. David Alan Gilbert 
Reviewed-by: Peter Xu 
Reviewed-by: Balamuruhan S 
Reviewed-by: Juan Quintela 
Message-Id: <20180503191059.19576-1-dgilb...@redhat.com>
Signed-off-by: Juan Quintela 
---
 docs/devel/migration.rst | 532 +++
 1 file changed, 376 insertions(+), 156 deletions(-)

diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
index 9342a8af06..40f136f6be 100644
--- a/docs/devel/migration.rst
+++ b/docs/devel/migration.rst
@@ -28,11 +28,11 @@ the guest to be stopped.  Typically the time that the guest 
is
 unresponsive during live migration is the low hundred of milliseconds
 (notice that this depends on a lot of things).
 
-Types of migration
-==
+Transports
+==
 
-Now that we have talked about live migration, there are several ways
-to do migration:
+The migration stream is normally just a byte stream that can be passed
+over any transport.
 
 - tcp migration: do the migration using tcp sockets
 - unix migration: do the migration using unix sockets
@@ -40,16 +40,16 @@ to do migration:
 - fd migration: do the migration using an file descriptor that is
   passed to QEMU.  QEMU doesn't care how this file descriptor is opened.
 
-All these four migration protocols use the same infrastructure to
+In addition, support is included for migration using RDMA, which
+transports the page data using ``RDMA``, where the hardware takes care of
+transporting the pages, and the load on the CPU is much lower.  While the
+internals of RDMA migration are a bit different, this isn't really visible
+outside the RAM migration code.
+
+All these migration protocols use the same infrastructure to
 save/restore state devices.  This infrastructure is shared with the
 savevm/loadvm functionality.
 
-State Live Migration
-
-
-This is used for RAM and block devices.  It is not yet ported to vmstate.
-
-
 Common infrastructure
 =
 
@@ -57,60 +57,75 @@ The files, sockets or fd's that carry the migration stream 
are abstracted by
 the  ``QEMUFile`` type (see `migration/qemu-file.h`).  In most cases this
 is connected to a subtype of ``QIOChannel`` (see `io/`).
 
+
 Saving the state of one device
 ==
 
-The state of a device is saved using intermediate buffers.  There are
-some helper functions to assist this saving.
-
-There is a new concept that we have to explain here: device state
-version.  When we migrate a device, we save/load the state as a series
-of fields.  Some times, due to bugs or new functionality, we need to
-change the state to store more/different information.  We use the
-version to identify each time that we do a change.  Each version is
-associated with a series of fields saved.  The `save_state` always saves
-the state as the newer version.  But `load_state` sometimes is able to
-load state from an older version.
-
-Legacy way
---
-
-This way is going to disappear as soon as all current users are ported to 
VMSTATE.
-
-Each device has to register two functions, one to save the state and
-another to load the state back.
-
-.. code:: c
-
-  int register_savevm(DeviceState *dev,
-  const char *idstr,
-  int instance_id,
-  int version_id,
-  SaveStateHandler *save_state,
-  LoadStateHandler *load_state,
-  void *opaque);
-
-  typedef void SaveStateHandler(QEMUFile *f, void *opaque);
-  typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);
-
-The important functions for the device state format are the `save_state`
-and `load_state`.  Notice that `load_state` receives a version_id
-parameter to know what state format is receiving.  `save_state` doesn't
-have a version_id parameter because it always uses the latest version.
+For most devices, the state is saved in a single call to the migration
+infrastructure; these are *non-iterative* devices.  The data for these
+devices is sent at the end of precopy migration, when the CPUs are paused.
+There are also *iterative* devices, which contain a very large amount of
+data (e.g. RAM or large tables).  See the iterative device section below.
+
+General advice for device developers
+
+
+- The migration state saved should reflect the device being modelled rather
+  than the way your implementation works.  That way if you change the 
implementation
+  later the migration stream will stay compatible.  That model may include
+  internal state that's not 

[Qemu-devel] [PULL 39/40] migration: Textual fixups for blocktime

[Juan Quintela]

From: "Dr. David Alan Gilbert" 

Blank lines and comments as suggested by Eric.

Signed-off-by: Dr. David Alan Gilbert 
Reviewed-by: Juan Quintela 
Reviewed-by: Eric Blake 
Message-Id: <20180427111502.9822-1-dgilb...@redhat.com>
Signed-off-by: Juan Quintela 
---
 qapi/migration.json | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/qapi/migration.json b/qapi/migration.json
index 23ba85ed3a..3ec418dabf 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -162,11 +162,13 @@
 #  error strings. (Since 2.7)
 #
 # @postcopy-blocktime: total time when all vCPU were blocked during postcopy
-#   live migration (Since 2.13)
+#   live migration. This is only present when the postcopy-blocktime
+#   migration capability is enabled. (Since 2.13)
 #
-# @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU (Since 
2.13)
+# @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU.  This is
+#   only present when the postcopy-blocktime migration capability
+#   is enabled. (Since 2.13)
 #
-
 #
 # Since: 0.14.0
 ##
@@ -368,7 +370,6 @@
 #
 # @x-multifd: Use more than one fd for migration (since 2.11)
 #
-#
 # @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps.
 # (since 2.12)
 #
-- 
2.17.0

[Qemu-devel] [PULL 23/40] migration: new cmd MIG_CMD_RECV_BITMAP

[Juan Quintela]

From: Peter Xu 

Add a new vm command MIG_CMD_RECV_BITMAP to request received bitmap for
one ramblock.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-12-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/savevm.c | 61 ++
 migration/savevm.h |  1 +
 migration/trace-events |  2 ++
 3 files changed, 64 insertions(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index 6ee69d8283..9f4a95d411 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -81,6 +81,7 @@ enum qemu_vm_cmd {
   were previously sent during
   precopy but are dirty. */
 MIG_CMD_PACKAGED,  /* Send a wrapped stream within this stream */
+MIG_CMD_RECV_BITMAP,   /* Request for recved bitmap on dst */
 MIG_CMD_MAX
 };
 
@@ -98,6 +99,7 @@ static struct mig_cmd_args {
 [MIG_CMD_POSTCOPY_RAM_DISCARD] = {
.len = -1, .name = "POSTCOPY_RAM_DISCARD" },
 [MIG_CMD_PACKAGED] = { .len =  4, .name = "PACKAGED" },
+[MIG_CMD_RECV_BITMAP]  = { .len = -1, .name = "RECV_BITMAP" },
 [MIG_CMD_MAX]  = { .len = -1, .name = "MAX" },
 };
 
@@ -956,6 +958,19 @@ void qemu_savevm_send_postcopy_run(QEMUFile *f)
 qemu_savevm_command_send(f, MIG_CMD_POSTCOPY_RUN, 0, NULL);
 }
 
+void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name)
+{
+size_t len;
+char buf[256];
+
+trace_savevm_send_recv_bitmap(block_name);
+
+buf[0] = len = strlen(block_name);
+memcpy(buf + 1, block_name, len);
+
+qemu_savevm_command_send(f, MIG_CMD_RECV_BITMAP, len + 1, (uint8_t *)buf);
+}
+
 bool qemu_savevm_state_blocked(Error **errp)
 {
 SaveStateEntry *se;
@@ -1801,6 +1816,49 @@ static int 
loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 return ret;
 }
 
+/*
+ * Handle request that source requests for recved_bitmap on
+ * destination. Payload format:
+ *
+ * len (1 byte) + ramblock_name (<255 bytes)
+ */
+static int loadvm_handle_recv_bitmap(MigrationIncomingState *mis,
+ uint16_t len)
+{
+QEMUFile *file = mis->from_src_file;
+RAMBlock *rb;
+char block_name[256];
+size_t cnt;
+
+cnt = qemu_get_counted_string(file, block_name);
+if (!cnt) {
+error_report("%s: failed to read block name", __func__);
+return -EINVAL;
+}
+
+/* Validate before using the data */
+if (qemu_file_get_error(file)) {
+return qemu_file_get_error(file);
+}
+
+if (len != cnt + 1) {
+error_report("%s: invalid payload length (%d)", __func__, len);
+return -EINVAL;
+}
+
+rb = qemu_ram_block_by_name(block_name);
+if (!rb) {
+error_report("%s: block '%s' not found", __func__, block_name);
+return -EINVAL;
+}
+
+/* TODO: send the bitmap back to source */
+
+trace_loadvm_handle_recv_bitmap(block_name);
+
+return 0;
+}
+
 /*
  * Process an incoming 'QEMU_VM_COMMAND'
  * 0   just a normal return
@@ -1874,6 +1932,9 @@ static int loadvm_process_command(QEMUFile *f)
 
 case MIG_CMD_POSTCOPY_RAM_DISCARD:
 return loadvm_postcopy_ram_handle_discard(mis, len);
+
+case MIG_CMD_RECV_BITMAP:
+return loadvm_handle_recv_bitmap(mis, len);
 }
 
 return 0;
diff --git a/migration/savevm.h b/migration/savevm.h
index cf4f0d37ca..b8cee00d41 100644
--- a/migration/savevm.h
+++ b/migration/savevm.h
@@ -47,6 +47,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t 
*buf, size_t len);
 void qemu_savevm_send_postcopy_advise(QEMUFile *f);
 void qemu_savevm_send_postcopy_listen(QEMUFile *f);
 void qemu_savevm_send_postcopy_run(QEMUFile *f);
+void qemu_savevm_send_recv_bitmap(QEMUFile *f, char *block_name);
 
 void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
uint16_t len,
diff --git a/migration/trace-events b/migration/trace-events
index 7f836499d1..5bee6d525a 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -12,6 +12,7 @@ loadvm_state_cleanup(void) ""
 loadvm_handle_cmd_packaged(unsigned int length) "%u"
 loadvm_handle_cmd_packaged_main(int ret) "%d"
 loadvm_handle_cmd_packaged_received(int ret) "%d"
+loadvm_handle_recv_bitmap(char *s) "%s"
 loadvm_postcopy_handle_advise(void) ""
 loadvm_postcopy_handle_listen(void) ""
 loadvm_postcopy_handle_run(void) ""
@@ -34,6 +35,7 @@ savevm_send_open_return_path(void) ""
 savevm_send_ping(uint32_t val) "0x%x"
 savevm_send_postcopy_listen(void) ""
 savevm_send_postcopy_run(void) ""
+savevm_send_recv_bitmap(char *name) "%s"
 savevm_state_setup(void) ""
 savevm_state_header(void) ""
 savevm_state_iterate(void) ""
-- 
2.17.0

[Qemu-devel] [PULL 34/40] migration: introduce lock for to_dst_file

[Juan Quintela]

From: Peter Xu 

Let's introduce a lock for that QEMUFile since we are going to operate
on it in multiple threads.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-23-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/channel.c   |  3 ++-
 migration/migration.c | 22 +++---
 migration/migration.h |  6 ++
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/migration/channel.c b/migration/channel.c
index c5eaf0fa0e..716192bf75 100644
--- a/migration/channel.c
+++ b/migration/channel.c
@@ -74,8 +74,9 @@ void migration_channel_connect(MigrationState *s,
 } else {
 QEMUFile *f = qemu_fopen_channel_output(ioc);
 
+qemu_mutex_lock(>qemu_file_lock);
 s->to_dst_file = f;
-
+qemu_mutex_unlock(>qemu_file_lock);
 }
 }
 migrate_fd_connect(s, error);
diff --git a/migration/migration.c b/migration/migration.c
index 1beb5e07fb..3deded90e5 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1234,6 +1234,7 @@ static void migrate_fd_cleanup(void *opaque)
 
 if (s->to_dst_file) {
 Error *local_err = NULL;
+QEMUFile *tmp;
 
 trace_migrate_fd_cleanup();
 qemu_mutex_unlock_iothread();
@@ -1246,8 +1247,15 @@ static void migrate_fd_cleanup(void *opaque)
 if (multifd_save_cleanup(_err) != 0) {
 error_report_err(local_err);
 }
-qemu_fclose(s->to_dst_file);
+qemu_mutex_lock(>qemu_file_lock);
+tmp = s->to_dst_file;
 s->to_dst_file = NULL;
+qemu_mutex_unlock(>qemu_file_lock);
+/*
+ * Close the file handle without the lock to make sure the
+ * critical section won't block for long.
+ */
+qemu_fclose(tmp);
 }
 
 assert((s->state != MIGRATION_STATUS_ACTIVE) &&
@@ -2531,14 +2539,20 @@ static MigThrError postcopy_pause(MigrationState *s)
 assert(s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE);
 
 while (true) {
+QEMUFile *file;
+
 migrate_set_state(>state, s->state,
   MIGRATION_STATUS_POSTCOPY_PAUSED);
 
 /* Current channel is possibly broken. Release it. */
 assert(s->to_dst_file);
-qemu_file_shutdown(s->to_dst_file);
-qemu_fclose(s->to_dst_file);
+qemu_mutex_lock(>qemu_file_lock);
+file = s->to_dst_file;
 s->to_dst_file = NULL;
+qemu_mutex_unlock(>qemu_file_lock);
+
+qemu_file_shutdown(file);
+qemu_fclose(file);
 
 error_report("Detected IO failure for postcopy. "
  "Migration paused.");
@@ -3007,6 +3021,7 @@ static void migration_instance_finalize(Object *obj)
 MigrationParameters *params = >parameters;
 
 qemu_mutex_destroy(>error_mutex);
+qemu_mutex_destroy(>qemu_file_lock);
 g_free(params->tls_hostname);
 g_free(params->tls_creds);
 qemu_sem_destroy(>pause_sem);
@@ -3046,6 +3061,7 @@ static void migration_instance_init(Object *obj)
 qemu_sem_init(>postcopy_pause_sem, 0);
 qemu_sem_init(>postcopy_pause_rp_sem, 0);
 qemu_sem_init(>rp_state.rp_sem, 0);
+qemu_mutex_init(>qemu_file_lock);
 }
 
 /*
diff --git a/migration/migration.h b/migration/migration.h
index f83f1064b5..8f0c82159b 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -114,6 +114,12 @@ struct MigrationState
 QemuThread thread;
 QEMUBH *cleanup_bh;
 QEMUFile *to_dst_file;
+/*
+ * Protects to_dst_file pointer.  We need to make sure we won't
+ * yield or hang during the critical section, since this lock will
+ * be used in OOB command handler.
+ */
+QemuMutex qemu_file_lock;
 
 /* bytes already send at the beggining of current interation */
 uint64_t iteration_initial_bytes;
-- 
2.17.0

[Qemu-devel] [PULL 21/40] migration: new state "postcopy-recover"

[Juan Quintela]

From: Peter Xu 

Introducing new migration state "postcopy-recover". If a migration
procedure is paused and the connection is rebuilt afterward
successfully, we'll switch the source VM state from "postcopy-paused" to
the new state "postcopy-recover", then we'll do the resume logic in the
migration thread (along with the return path thread).

This patch only do the state switch on source side. Another following up
patch will handle the state switching on destination side using the same
status bit.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-10-pet...@redhat.com>
Signed-off-by: Juan Quintela 

---
s/2.11/2.13/
---
 migration/migration.c | 78 ---
 qapi/migration.json   |  4 ++-
 2 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 038fae93ab..4ab637a1fe 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -577,6 +577,7 @@ static bool migration_is_setup_or_active(int state)
 case MIGRATION_STATUS_ACTIVE:
 case MIGRATION_STATUS_POSTCOPY_ACTIVE:
 case MIGRATION_STATUS_POSTCOPY_PAUSED:
+case MIGRATION_STATUS_POSTCOPY_RECOVER:
 case MIGRATION_STATUS_SETUP:
 case MIGRATION_STATUS_PRE_SWITCHOVER:
 case MIGRATION_STATUS_DEVICE:
@@ -658,6 +659,7 @@ static void fill_source_migration_info(MigrationInfo *info)
 case MIGRATION_STATUS_PRE_SWITCHOVER:
 case MIGRATION_STATUS_DEVICE:
 case MIGRATION_STATUS_POSTCOPY_PAUSED:
+case MIGRATION_STATUS_POSTCOPY_RECOVER:
  /* TODO add some postcopy stats */
 info->has_status = true;
 info->has_total_time = true;
@@ -2318,6 +2320,13 @@ typedef enum MigThrError {
 MIG_THR_ERR_FATAL = 2,
 } MigThrError;
 
+/* Return zero if success, or <0 for error */
+static int postcopy_do_resume(MigrationState *s)
+{
+/* TODO: do the resume logic */
+return 0;
+}
+
 /*
  * We don't return until we are in a safe state to continue current
  * postcopy migration.  Returns MIG_THR_ERR_RECOVERED if recovered, or
@@ -2326,29 +2335,55 @@ typedef enum MigThrError {
 static MigThrError postcopy_pause(MigrationState *s)
 {
 assert(s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE);
-migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
-  MIGRATION_STATUS_POSTCOPY_PAUSED);
 
-/* Current channel is possibly broken. Release it. */
-assert(s->to_dst_file);
-qemu_file_shutdown(s->to_dst_file);
-qemu_fclose(s->to_dst_file);
-s->to_dst_file = NULL;
+while (true) {
+migrate_set_state(>state, s->state,
+  MIGRATION_STATUS_POSTCOPY_PAUSED);
 
-error_report("Detected IO failure for postcopy. "
- "Migration paused.");
+/* Current channel is possibly broken. Release it. */
+assert(s->to_dst_file);
+qemu_file_shutdown(s->to_dst_file);
+qemu_fclose(s->to_dst_file);
+s->to_dst_file = NULL;
 
-/*
- * We wait until things fixed up. Then someone will setup the
- * status back for us.
- */
-while (s->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
-qemu_sem_wait(>postcopy_pause_sem);
+error_report("Detected IO failure for postcopy. "
+ "Migration paused.");
+
+/*
+ * We wait until things fixed up. Then someone will setup the
+ * status back for us.
+ */
+while (s->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
+qemu_sem_wait(>postcopy_pause_sem);
+}
+
+if (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER) {
+/* Woken up by a recover procedure. Give it a shot */
+
+/*
+ * Firstly, let's wake up the return path now, with a new
+ * return path channel.
+ */
+qemu_sem_post(>postcopy_pause_rp_sem);
+
+/* Do the resume logic */
+if (postcopy_do_resume(s) == 0) {
+/* Let's continue! */
+trace_postcopy_pause_continued();
+return MIG_THR_ERR_RECOVERED;
+} else {
+/*
+ * Something wrong happened during the recovery, let's
+ * pause again. Pause is always better than throwing
+ * data away.
+ */
+continue;
+}
+} else {
+/* This is not right... Time to quit. */
+return MIG_THR_ERR_FATAL;
+}
 }
-
-trace_postcopy_pause_continued();
-
-return MIG_THR_ERR_RECOVERED;
 }
 
 static MigThrError migration_detect_error(MigrationState *s)
@@ -2668,7 +2703,10 @@ void migrate_fd_connect(MigrationState *s, Error 
*error_in)
 }
 
 if (resume) {
-/* TODO: do the resume logic */
+/* Wakeup the main migration thread to do the recovery */
+  

[Qemu-devel] [PULL 36/40] migration/hmp: add migrate_pause command

[Juan Quintela]

From: Peter Xu 

Wrapper for QMP command "migrate-pause".

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-25-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 hmp-commands.hx | 14 ++
 hmp.c   |  9 +
 hmp.h   |  1 +
 3 files changed, 24 insertions(+)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 2fa387d341..0734fea931 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -971,6 +971,20 @@ STEXI
 @item migrate_recover @var{uri}
 @findex migrate_recover
 Continue a paused incoming postcopy migration using the @var{uri}.
+ETEXI
+
+{
+.name   = "migrate_pause",
+.args_type  = "",
+.params = "",
+.help   = "Pause an ongoing migration (postcopy-only)",
+.cmd= hmp_migrate_pause,
+},
+
+STEXI
+@item migrate_pause
+@findex migrate_pause
+Pause an ongoing migration.  Currently it only supports postcopy.
 ETEXI
 
 {
diff --git a/hmp.c b/hmp.c
index 2a36c1cf8d..ef93f4878b 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1527,6 +1527,15 @@ void hmp_migrate_recover(Monitor *mon, const QDict 
*qdict)
 hmp_handle_error(mon, );
 }
 
+void hmp_migrate_pause(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+
+qmp_migrate_pause();
+
+hmp_handle_error(mon, );
+}
+
 /* Kept for backwards compatibility */
 void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict)
 {
diff --git a/hmp.h b/hmp.h
index b6b56c8161..20f27439d3 100644
--- a/hmp.h
+++ b/hmp.h
@@ -69,6 +69,7 @@ void hmp_migrate_cancel(Monitor *mon, const QDict *qdict);
 void hmp_migrate_continue(Monitor *mon, const QDict *qdict);
 void hmp_migrate_incoming(Monitor *mon, const QDict *qdict);
 void hmp_migrate_recover(Monitor *mon, const QDict *qdict);
+void hmp_migrate_pause(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_speed(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict);
-- 
2.17.0

[Qemu-devel] [PULL 33/40] hmp/migration: add migrate_recover command

[Juan Quintela]

From: Peter Xu 

Sister command to migrate-recover in QMP.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-22-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 hmp-commands.hx | 13 +
 hmp.c   | 10 ++
 hmp.h   |  1 +
 3 files changed, 24 insertions(+)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index a7051ee391..2fa387d341 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -957,7 +957,20 @@ STEXI
 @findex migrate_incoming
 Continue an incoming migration using the @var{uri} (that has the same syntax
 as the -incoming option).
+ETEXI
 
+{
+.name   = "migrate_recover",
+.args_type  = "uri:s",
+.params = "uri",
+.help   = "Continue a paused incoming postcopy migration",
+.cmd= hmp_migrate_recover,
+},
+
+STEXI
+@item migrate_recover @var{uri}
+@findex migrate_recover
+Continue a paused incoming postcopy migration using the @var{uri}.
 ETEXI
 
 {
diff --git a/hmp.c b/hmp.c
index a7aa878788..2a36c1cf8d 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1517,6 +1517,16 @@ void hmp_migrate_incoming(Monitor *mon, const QDict 
*qdict)
 hmp_handle_error(mon, );
 }
 
+void hmp_migrate_recover(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+const char *uri = qdict_get_str(qdict, "uri");
+
+qmp_migrate_recover(uri, );
+
+hmp_handle_error(mon, );
+}
+
 /* Kept for backwards compatibility */
 void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict)
 {
diff --git a/hmp.h b/hmp.h
index 4e2ec375b0..b6b56c8161 100644
--- a/hmp.h
+++ b/hmp.h
@@ -68,6 +68,7 @@ void hmp_info_snapshots(Monitor *mon, const QDict *qdict);
 void hmp_migrate_cancel(Monitor *mon, const QDict *qdict);
 void hmp_migrate_continue(Monitor *mon, const QDict *qdict);
 void hmp_migrate_incoming(Monitor *mon, const QDict *qdict);
+void hmp_migrate_recover(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_downtime(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_speed(Monitor *mon, const QDict *qdict);
 void hmp_migrate_set_capability(Monitor *mon, const QDict *qdict);
-- 
2.17.0

[Qemu-devel] [PULL 35/40] migration/qmp: add command migrate-pause

[Juan Quintela]

From: Peter Xu 

It pauses an ongoing migration.  Currently it only supports postcopy.
Note that this command will work on either side of the migration.
Basically when we trigger this on one side, it'll interrupt the other
side as well since the other side will get notified on the disconnect
event.

However, it's still possible that the other side is not notified, for
example, when the network is totally broken, or due to some firewall
configuration changes.  In that case, we will also need to run the same
command on the other side so both sides will go into the paused state.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-24-pet...@redhat.com>
Signed-off-by: Juan Quintela 

---
s/2.12/2.13/
---
 migration/migration.c | 29 +
 qapi/migration.json   | 16 
 2 files changed, 45 insertions(+)

diff --git a/migration/migration.c b/migration/migration.c
index 3deded90e5..05aec2c905 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1507,6 +1507,35 @@ void qmp_migrate_recover(const char *uri, Error **errp)
 qemu_start_incoming_migration(uri, errp);
 }
 
+void qmp_migrate_pause(Error **errp)
+{
+MigrationState *ms = migrate_get_current();
+MigrationIncomingState *mis = migration_incoming_get_current();
+int ret;
+
+if (ms->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+/* Source side, during postcopy */
+qemu_mutex_lock(>qemu_file_lock);
+ret = qemu_file_shutdown(ms->to_dst_file);
+qemu_mutex_unlock(>qemu_file_lock);
+if (ret) {
+error_setg(errp, "Failed to pause source migration");
+}
+return;
+}
+
+if (mis->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ret = qemu_file_shutdown(mis->from_src_file);
+if (ret) {
+error_setg(errp, "Failed to pause destination migration");
+}
+return;
+}
+
+error_setg(errp, "migrate-pause is currently only supported "
+   "during postcopy-active state");
+}
+
 bool migration_is_blocked(Error **errp)
 {
 if (qemu_savevm_state_blocked(errp)) {
diff --git a/qapi/migration.json b/qapi/migration.json
index da351d7421..23ba85ed3a 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1211,3 +1211,19 @@
 ##
 { 'command': 'migrate-recover', 'data': { 'uri': 'str' },
   'allow-oob': true }
+
+##
+# @migrate-pause:
+#
+# Pause a migration.  Currently it only supports postcopy.
+#
+# Returns: nothing.
+#
+# Example:
+#
+# -> { "execute": "migrate-pause" }
+# <- { "return": {} }
+#
+# Since: 2.13
+##
+{ 'command': 'migrate-pause', 'allow-oob': true }
-- 
2.17.0

[Qemu-devel] [PULL 19/40] qmp: hmp: add migrate "resume" option

[Juan Quintela]

From: Peter Xu 

It will be used when we want to resume one paused migration.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-8-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
s/2.12/2.13/
---
 hmp-commands.hx   | 7 ---
 hmp.c | 4 +++-
 migration/migration.c | 2 +-
 qapi/migration.json   | 5 -
 4 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 227f7eee88..a7051ee391 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -897,13 +897,14 @@ ETEXI
 
 {
 .name   = "migrate",
-.args_type  = "detach:-d,blk:-b,inc:-i,uri:s",
-.params = "[-d] [-b] [-i] uri",
+.args_type  = "detach:-d,blk:-b,inc:-i,resume:-r,uri:s",
+.params = "[-d] [-b] [-i] [-r] uri",
 .help   = "migrate to URI (using -d to not wait for completion)"
  "\n\t\t\t -b for migration without shared storage with"
  " full copy of disk\n\t\t\t -i for migration without "
  "shared storage with incremental copy of disk "
- "(base image shared between src and destination)",
+ "(base image shared between src and destination)"
+  "\n\t\t\t -r to resume a paused migration",
 .cmd= hmp_migrate,
 },
 
diff --git a/hmp.c b/hmp.c
index bdb340605c..a7aa878788 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1929,10 +1929,12 @@ void hmp_migrate(Monitor *mon, const QDict *qdict)
 bool detach = qdict_get_try_bool(qdict, "detach", false);
 bool blk = qdict_get_try_bool(qdict, "blk", false);
 bool inc = qdict_get_try_bool(qdict, "inc", false);
+bool resume = qdict_get_try_bool(qdict, "resume", false);
 const char *uri = qdict_get_str(qdict, "uri");
 Error *err = NULL;
 
-qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, );
+qmp_migrate(uri, !!blk, blk, !!inc, inc,
+false, false, true, resume, );
 if (err) {
 hmp_handle_error(mon, );
 return;
diff --git a/migration/migration.c b/migration/migration.c
index 3d3b0a5b4a..19f07fb64f 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1413,7 +1413,7 @@ bool migration_is_blocked(Error **errp)
 
 void qmp_migrate(const char *uri, bool has_blk, bool blk,
  bool has_inc, bool inc, bool has_detach, bool detach,
- Error **errp)
+ bool has_resume, bool resume, Error **errp)
 {
 Error *local_err = NULL;
 MigrationState *s = migrate_get_current();
diff --git a/qapi/migration.json b/qapi/migration.json
index 244334e9f4..b8ca60ac43 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1031,6 +1031,8 @@
 # @detach: this argument exists only for compatibility reasons and
 #  is ignored by QEMU
 #
+# @resume: resume one paused migration, default "off". (since 2.13)
+#
 # Returns: nothing on success
 #
 # Since: 0.14.0
@@ -1052,7 +1054,8 @@
 #
 ##
 { 'command': 'migrate',
-  'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', '*detach': 'bool' } }
+  'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool',
+   '*detach': 'bool', '*resume': 'bool' } }
 
 ##
 # @migrate-incoming:
-- 
2.17.0

[Qemu-devel] [PULL 31/40] migration: init dst in migration_object_init too

[Juan Quintela]

From: Peter Xu 

Though we may not need it, now we init both the src/dst migration
objects in migration_object_init() so that even incoming migration
object would be thread safe (it was not).

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-20-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c | 37 +++--
 1 file changed, 19 insertions(+), 18 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 240960d951..c1832b0263 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -106,6 +106,7 @@ enum mig_rp_message_type {
dynamic creation of migration */
 
 static MigrationState *current_migration;
+static MigrationIncomingState *current_incoming;
 
 static bool migration_object_check(MigrationState *ms, Error **errp);
 static int migration_maybe_pause(MigrationState *s,
@@ -121,6 +122,22 @@ void migration_object_init(void)
 assert(!current_migration);
 current_migration = MIGRATION_OBJ(object_new(TYPE_MIGRATION));
 
+/*
+ * Init the migrate incoming object as well no matter whether
+ * we'll use it or not.
+ */
+assert(!current_incoming);
+current_incoming = g_new0(MigrationIncomingState, 1);
+current_incoming->state = MIGRATION_STATUS_NONE;
+current_incoming->postcopy_remote_fds =
+g_array_new(FALSE, TRUE, sizeof(struct PostCopyFD));
+qemu_mutex_init(_incoming->rp_mutex);
+qemu_event_init(_incoming->main_thread_load_event, false);
+qemu_sem_init(_incoming->postcopy_pause_sem_dst, 0);
+qemu_sem_init(_incoming->postcopy_pause_sem_fault, 0);
+
+init_dirty_bitmap_incoming_migration();
+
 if (!migration_object_check(current_migration, )) {
 error_report_err(err);
 exit(1);
@@ -151,24 +168,8 @@ MigrationState *migrate_get_current(void)
 
 MigrationIncomingState *migration_incoming_get_current(void)
 {
-static bool once;
-static MigrationIncomingState mis_current;
-
-if (!once) {
-mis_current.state = MIGRATION_STATUS_NONE;
-memset(_current, 0, sizeof(MigrationIncomingState));
-mis_current.postcopy_remote_fds = g_array_new(FALSE, TRUE,
-   sizeof(struct PostCopyFD));
-qemu_mutex_init(_current.rp_mutex);
-qemu_event_init(_current.main_thread_load_event, false);
-qemu_sem_init(_current.postcopy_pause_sem_dst, 0);
-qemu_sem_init(_current.postcopy_pause_sem_fault, 0);
-
-init_dirty_bitmap_incoming_migration();
-
-once = true;
-}
-return _current;
+assert(current_incoming);
+return current_incoming;
 }
 
 void migration_incoming_state_destroy(void)
-- 
2.17.0

[Qemu-devel] [PULL 30/40] migration: final handshake for the resume

[Juan Quintela]

From: Peter Xu 

Finish the last step to do the final handshake for the recovery.

First source sends one MIG_CMD_RESUME to dst, telling that source is
ready to resume.

Then, dest replies with MIG_RP_MSG_RESUME_ACK to source, telling that
dest is ready to resume (after switch to postcopy-active state).

When source received the RESUME_ACK, it switches its state to
postcopy-active, and finally the recovery is completed.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-19-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c | 28 
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 19ef8b05b1..240960d951 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1924,7 +1924,8 @@ static int migrate_handle_rp_resume_ack(MigrationState 
*s, uint32_t value)
 migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_RECOVER,
   MIGRATION_STATUS_POSTCOPY_ACTIVE);
 
-/* TODO: notify send thread that time to continue send pages */
+/* Notify send thread that time to continue send pages */
+qemu_sem_post(>rp_state.rp_sem);
 
 return 0;
 }
@@ -2451,6 +2452,21 @@ typedef enum MigThrError {
 MIG_THR_ERR_FATAL = 2,
 } MigThrError;
 
+static int postcopy_resume_handshake(MigrationState *s)
+{
+qemu_savevm_send_postcopy_resume(s->to_dst_file);
+
+while (s->state == MIGRATION_STATUS_POSTCOPY_RECOVER) {
+qemu_sem_wait(>rp_state.rp_sem);
+}
+
+if (s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+return 0;
+}
+
+return -1;
+}
+
 /* Return zero if success, or <0 for error */
 static int postcopy_do_resume(MigrationState *s)
 {
@@ -2468,10 +2484,14 @@ static int postcopy_do_resume(MigrationState *s)
 }
 
 /*
- * TODO: handshake with dest using MIG_CMD_RESUME,
- * MIG_RP_MSG_RESUME_ACK, then switch source state to
- * "postcopy-active"
+ * Last handshake with destination on the resume (destination will
+ * switch to postcopy-active afterwards)
  */
+ret = postcopy_resume_handshake(s);
+if (ret) {
+error_report("%s: handshake failed: %d", __func__, ret);
+return ret;
+}
 
 return 0;
 }
-- 
2.17.0

[Qemu-devel] [PULL 18/40] migration: allow fault thread to pause

[Juan Quintela]

From: Peter Xu 

Allows the fault thread to stop handling page faults temporarily. When
network failure happened (and if we expect a recovery afterwards), we
should not allow the fault thread to continue sending things to source,
instead, it should halt for a while until the connection is rebuilt.

When the dest main thread noticed the failure, it kicks the fault thread
to switch to pause state.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-7-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c|  1 +
 migration/migration.h|  1 +
 migration/postcopy-ram.c | 54 +---
 migration/savevm.c   |  3 +++
 migration/trace-events   |  2 ++
 5 files changed, 57 insertions(+), 4 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index f9c58d2511..3d3b0a5b4a 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -160,6 +160,7 @@ MigrationIncomingState *migration_incoming_get_current(void)
 qemu_mutex_init(_current.rp_mutex);
 qemu_event_init(_current.main_thread_load_event, false);
 qemu_sem_init(_current.postcopy_pause_sem_dst, 0);
+qemu_sem_init(_current.postcopy_pause_sem_fault, 0);
 
 init_dirty_bitmap_incoming_migration();
 
diff --git a/migration/migration.h b/migration/migration.h
index 32fd50e9be..4ea5949104 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -76,6 +76,7 @@ struct MigrationIncomingState {
 
 /* notify PAUSED postcopy incoming migrations to try to continue */
 QemuSemaphore postcopy_pause_sem_dst;
+QemuSemaphore postcopy_pause_sem_fault;
 };
 
 MigrationIncomingState *migration_incoming_get_current(void);
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 8ceeaa2a93..658b750a8e 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -830,6 +830,17 @@ static void mark_postcopy_blocktime_end(uintptr_t addr)
   affected_cpu);
 }
 
+static bool postcopy_pause_fault_thread(MigrationIncomingState *mis)
+{
+trace_postcopy_pause_fault_thread();
+
+qemu_sem_wait(>postcopy_pause_sem_fault);
+
+trace_postcopy_pause_fault_thread_continued();
+
+return true;
+}
+
 /*
  * Handle faults detected by the USERFAULT markings
  */
@@ -880,6 +891,22 @@ static void *postcopy_ram_fault_thread(void *opaque)
 break;
 }
 
+if (!mis->to_src_file) {
+/*
+ * Possibly someone tells us that the return path is
+ * broken already using the event. We should hold until
+ * the channel is rebuilt.
+ */
+if (postcopy_pause_fault_thread(mis)) {
+mis->last_rb = NULL;
+/* Continue to read the userfaultfd */
+} else {
+error_report("%s: paused but don't allow to continue",
+ __func__);
+break;
+}
+}
+
 if (pfd[1].revents) {
 uint64_t tmp64 = 0;
 
@@ -942,18 +969,37 @@ static void *postcopy_ram_fault_thread(void *opaque)
 (uintptr_t)(msg.arg.pagefault.address),
 msg.arg.pagefault.feat.ptid, rb);
 
+retry:
 /*
  * Send the request to the source - we want to request one
  * of our host page sizes (which is >= TPS)
  */
 if (rb != mis->last_rb) {
 mis->last_rb = rb;
-migrate_send_rp_req_pages(mis, qemu_ram_get_idstr(rb),
- rb_offset, qemu_ram_pagesize(rb));
+ret = migrate_send_rp_req_pages(mis,
+qemu_ram_get_idstr(rb),
+rb_offset,
+qemu_ram_pagesize(rb));
 } else {
 /* Save some space */
-migrate_send_rp_req_pages(mis, NULL,
- rb_offset, qemu_ram_pagesize(rb));
+ret = migrate_send_rp_req_pages(mis,
+NULL,
+rb_offset,
+qemu_ram_pagesize(rb));
+}
+
+if (ret) {
+/* May be network failure, try to wait for recovery */
+if (ret == -EIO && postcopy_pause_fault_thread(mis)) {
+/* We got reconnected somehow, try to continue */
+mis->last_rb = NULL;
+goto retry;
+} else {
+/* This is a unavoidable fault */
+error_report("%s: migrate_send_rp_req_pages() get %d",
+   

[Qemu-devel] [PULL 07/40] migration: Be sure all recv channels are created

[Juan Quintela]

We need them before we start migration.

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 
---
 migration/migration.c |  6 +-
 migration/ram.c   | 11 +++
 migration/ram.h   |  1 +
 3 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 4a7959c111..8e5b421b97 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -462,7 +462,11 @@ void migration_ioc_process_incoming(QIOChannel *ioc)
  */
 bool migration_has_all_channels(void)
 {
-return true;
+bool all_channels;
+
+all_channels = multifd_recv_all_channels_created();
+
+return all_channels;
 }
 
 /*
diff --git a/migration/ram.c b/migration/ram.c
index 1aa661e6ab..d5335c10b6 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -671,6 +671,17 @@ int multifd_load_setup(void)
 return 0;
 }
 
+bool multifd_recv_all_channels_created(void)
+{
+int thread_count = migrate_multifd_channels();
+
+if (!migrate_use_multifd()) {
+return true;
+}
+
+return thread_count == atomic_read(_recv_state->count);
+}
+
 void multifd_recv_new_channel(QIOChannel *ioc)
 {
 /* nothing to do yet */
diff --git a/migration/ram.h b/migration/ram.h
index 06dbddc2a2..3f4b7daee8 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -45,6 +45,7 @@ int multifd_save_setup(void);
 int multifd_save_cleanup(Error **errp);
 int multifd_load_setup(void);
 int multifd_load_cleanup(Error **errp);
+bool multifd_recv_all_channels_created(void);
 void multifd_recv_new_channel(QIOChannel *ioc);
 
 uint64_t ram_pagesize_summary(void);
-- 
2.17.0

[Qemu-devel] [PULL 27/40] migration: introduce SaveVMHandlers.resume_prepare

[Juan Quintela]

From: Peter Xu 

This is hook function to be called when a postcopy migration wants to
resume from a failure. For each module, it should provide its own
recovery logic before we switch to the postcopy-active state.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-16-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 include/migration/register.h |  2 ++
 migration/migration.c| 20 +++-
 migration/savevm.c   | 25 +
 migration/savevm.h   |  1 +
 migration/trace-events   |  1 +
 5 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/include/migration/register.h b/include/migration/register.h
index f6f12f9b1a..d287f4c317 100644
--- a/include/migration/register.h
+++ b/include/migration/register.h
@@ -64,6 +64,8 @@ typedef struct SaveVMHandlers {
 LoadStateHandler *load_state;
 int (*load_setup)(QEMUFile *f, void *opaque);
 int (*load_cleanup)(void *opaque);
+/* Called when postcopy migration wants to resume from failure */
+int (*resume_prepare)(MigrationState *s, void *opaque);
 } SaveVMHandlers;
 
 int register_savevm_live(DeviceState *dev,
diff --git a/migration/migration.c b/migration/migration.c
index 4f2c6d22d1..b0217c4823 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2454,7 +2454,25 @@ typedef enum MigThrError {
 /* Return zero if success, or <0 for error */
 static int postcopy_do_resume(MigrationState *s)
 {
-/* TODO: do the resume logic */
+int ret;
+
+/*
+ * Call all the resume_prepare() hooks, so that modules can be
+ * ready for the migration resume.
+ */
+ret = qemu_savevm_state_resume_prepare(s);
+if (ret) {
+error_report("%s: resume_prepare() failure detected: %d",
+ __func__, ret);
+return ret;
+}
+
+/*
+ * TODO: handshake with dest using MIG_CMD_RESUME,
+ * MIG_RP_MSG_RESUME_ACK, then switch source state to
+ * "postcopy-active"
+ */
+
 return 0;
 }
 
diff --git a/migration/savevm.c b/migration/savevm.c
index 6a2d77cbf3..8e7653badc 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1031,6 +1031,31 @@ void qemu_savevm_state_setup(QEMUFile *f)
 }
 }
 
+int qemu_savevm_state_resume_prepare(MigrationState *s)
+{
+SaveStateEntry *se;
+int ret;
+
+trace_savevm_state_resume_prepare();
+
+QTAILQ_FOREACH(se, _state.handlers, entry) {
+if (!se->ops || !se->ops->resume_prepare) {
+continue;
+}
+if (se->ops && se->ops->is_active) {
+if (!se->ops->is_active(se->opaque)) {
+continue;
+}
+}
+ret = se->ops->resume_prepare(s, se->opaque);
+if (ret < 0) {
+return ret;
+}
+}
+
+return 0;
+}
+
 /*
  * this function has three return values:
  *   negative: there was one error, and we have -errno.
diff --git a/migration/savevm.h b/migration/savevm.h
index b24ff073ad..a5e65b8ae3 100644
--- a/migration/savevm.h
+++ b/migration/savevm.h
@@ -31,6 +31,7 @@
 
 bool qemu_savevm_state_blocked(Error **errp);
 void qemu_savevm_state_setup(QEMUFile *f);
+int qemu_savevm_state_resume_prepare(MigrationState *s);
 void qemu_savevm_state_header(QEMUFile *f);
 int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy);
 void qemu_savevm_state_cleanup(void);
diff --git a/migration/trace-events b/migration/trace-events
index 40a7217829..be36fbccfe 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -39,6 +39,7 @@ savevm_send_postcopy_run(void) ""
 savevm_send_postcopy_resume(void) ""
 savevm_send_recv_bitmap(char *name) "%s"
 savevm_state_setup(void) ""
+savevm_state_resume_prepare(void) ""
 savevm_state_header(void) ""
 savevm_state_iterate(void) ""
 savevm_state_cleanup(void) ""
-- 
2.17.0

[Qemu-devel] [PULL 22/40] migration: wakeup dst ram-load-thread for recover

[Juan Quintela]

From: Peter Xu 

On the destination side, we cannot wake up all the threads when we got
reconnected. The first thing to do is to wake up the main load thread,
so that we can continue to receive valid messages from source again and
reply when needed.

At this point, we switch the destination VM state from postcopy-paused
back to postcopy-recover.

Now we are finally ready to do the resume logic.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-11-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c | 30 --
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 4ab637a1fe..ec3bc9ae20 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -440,8 +440,34 @@ void migration_incoming_process(void)
 
 void migration_fd_process_incoming(QEMUFile *f)
 {
-migration_incoming_setup(f);
-migration_incoming_process();
+MigrationIncomingState *mis = migration_incoming_get_current();
+
+if (mis->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
+/* Resumed from a paused postcopy migration */
+
+mis->from_src_file = f;
+/* Postcopy has standalone thread to do vm load */
+qemu_file_set_blocking(f, true);
+
+/* Re-configure the return path */
+mis->to_src_file = qemu_file_get_return_path(f);
+
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_PAUSED,
+  MIGRATION_STATUS_POSTCOPY_RECOVER);
+
+/*
+ * Here, we only wake up the main loading thread (while the
+ * fault thread will still be waiting), so that we can receive
+ * commands from source now, and answer it if needed. The
+ * fault thread will be woken up afterwards until we are sure
+ * that source is ready to reply to page requests.
+ */
+qemu_sem_post(>postcopy_pause_sem_dst);
+} else {
+/* New incoming migration */
+migration_incoming_setup(f);
+migration_incoming_process();
+}
 }
 
 void migration_ioc_process_incoming(QIOChannel *ioc)
-- 
2.17.0

[Qemu-devel] [PULL 29/40] migration: setup ramstate for resume

[Juan Quintela]

From: Peter Xu 

After we updated the dirty bitmaps of ramblocks, we also need to update
the critical fields in RAMState to make sure it is ready for a resume.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-18-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/ram.c| 45 +-
 migration/trace-events |  1 +
 2 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index b16eabcfb9..5bcbf7a9f9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2491,6 +2491,41 @@ static int ram_init_all(RAMState **rsp)
 return 0;
 }
 
+static void ram_state_resume_prepare(RAMState *rs, QEMUFile *out)
+{
+RAMBlock *block;
+uint64_t pages = 0;
+
+/*
+ * Postcopy is not using xbzrle/compression, so no need for that.
+ * Also, since source are already halted, we don't need to care
+ * about dirty page logging as well.
+ */
+
+RAMBLOCK_FOREACH(block) {
+pages += bitmap_count_one(block->bmap,
+  block->used_length >> TARGET_PAGE_BITS);
+}
+
+/* This may not be aligned with current bitmaps. Recalculate. */
+rs->migration_dirty_pages = pages;
+
+rs->last_seen_block = NULL;
+rs->last_sent_block = NULL;
+rs->last_page = 0;
+rs->last_version = ram_list.version;
+/*
+ * Disable the bulk stage, otherwise we'll resend the whole RAM no
+ * matter what we have sent.
+ */
+rs->ram_bulk_stage = false;
+
+/* Update RAMState cache of output QEMUFile */
+rs->f = out;
+
+trace_ram_state_resume_prepare(pages);
+}
+
 /*
  * Each of ram_save_setup, ram_save_iterate and ram_save_complete has
  * long-running RCU critical section.  When rcu-reclaims in the code
@@ -3486,8 +3521,16 @@ out:
 static int ram_resume_prepare(MigrationState *s, void *opaque)
 {
 RAMState *rs = *(RAMState **)opaque;
+int ret;
 
-return ram_dirty_bitmap_sync_all(s, rs);
+ret = ram_dirty_bitmap_sync_all(s, rs);
+if (ret) {
+return ret;
+}
+
+ram_state_resume_prepare(rs, s->to_dst_file);
+
+return 0;
 }
 
 static SaveVMHandlers savevm_ram_handlers = {
diff --git a/migration/trace-events b/migration/trace-events
index 53243e17ec..3c798ddd11 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -88,6 +88,7 @@ ram_dirty_bitmap_reload_complete(char *str) "%s"
 ram_dirty_bitmap_sync_start(void) ""
 ram_dirty_bitmap_sync_wait(void) ""
 ram_dirty_bitmap_sync_complete(void) ""
+ram_state_resume_prepare(uint64_t v) "%" PRId64
 
 # migration/migration.c
 await_return_path_close_on_source_close(void) ""
-- 
2.17.0

[Qemu-devel] [PULL 15/40] migration: implement "postcopy-pause" src logic

[Juan Quintela]

From: Peter Xu 

Now when network down for postcopy, the source side will not fail the
migration. Instead we convert the status into this new paused state, and
we will try to wait for a rescue in the future.

If a recovery is detected, migration_thread() will reset its local
variables to prepare for that.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-4-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  | 99 +++---
 migration/migration.h  |  3 ++
 migration/trace-events |  1 +
 3 files changed, 97 insertions(+), 6 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 02ebd6c9d1..8392cf467d 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2245,6 +2245,80 @@ bool migrate_colo_enabled(void)
 return s->enabled_capabilities[MIGRATION_CAPABILITY_X_COLO];
 }
 
+typedef enum MigThrError {
+/* No error detected */
+MIG_THR_ERR_NONE = 0,
+/* Detected error, but resumed successfully */
+MIG_THR_ERR_RECOVERED = 1,
+/* Detected fatal error, need to exit */
+MIG_THR_ERR_FATAL = 2,
+} MigThrError;
+
+/*
+ * We don't return until we are in a safe state to continue current
+ * postcopy migration.  Returns MIG_THR_ERR_RECOVERED if recovered, or
+ * MIG_THR_ERR_FATAL if unrecovery failure happened.
+ */
+static MigThrError postcopy_pause(MigrationState *s)
+{
+assert(s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE);
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+  MIGRATION_STATUS_POSTCOPY_PAUSED);
+
+/* Current channel is possibly broken. Release it. */
+assert(s->to_dst_file);
+qemu_file_shutdown(s->to_dst_file);
+qemu_fclose(s->to_dst_file);
+s->to_dst_file = NULL;
+
+error_report("Detected IO failure for postcopy. "
+ "Migration paused.");
+
+/*
+ * We wait until things fixed up. Then someone will setup the
+ * status back for us.
+ */
+while (s->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
+qemu_sem_wait(>postcopy_pause_sem);
+}
+
+trace_postcopy_pause_continued();
+
+return MIG_THR_ERR_RECOVERED;
+}
+
+static MigThrError migration_detect_error(MigrationState *s)
+{
+int ret;
+
+/* Try to detect any file errors */
+ret = qemu_file_get_error(s->to_dst_file);
+
+if (!ret) {
+/* Everything is fine */
+return MIG_THR_ERR_NONE;
+}
+
+if (s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE && ret == -EIO) {
+/*
+ * For postcopy, we allow the network to be down for a
+ * while. After that, it can be continued by a
+ * recovery phase.
+ */
+return postcopy_pause(s);
+} else {
+/*
+ * For precopy (or postcopy with error outside IO), we fail
+ * with no time.
+ */
+migrate_set_state(>state, s->state, MIGRATION_STATUS_FAILED);
+trace_migration_thread_file_err();
+
+/* Time to stop the migration, now. */
+return MIG_THR_ERR_FATAL;
+}
+}
+
 static void migration_calculate_complete(MigrationState *s)
 {
 uint64_t bytes = qemu_ftell(s->to_dst_file);
@@ -2401,6 +2475,7 @@ static void *migration_thread(void *opaque)
 {
 MigrationState *s = opaque;
 int64_t setup_start = qemu_clock_get_ms(QEMU_CLOCK_HOST);
+MigThrError thr_error;
 
 rcu_register_thread();
 
@@ -2450,13 +2525,22 @@ static void *migration_thread(void *opaque)
 }
 }
 
-if (qemu_file_get_error(s->to_dst_file)) {
-if (migration_is_setup_or_active(s->state)) {
-migrate_set_state(>state, s->state,
-  MIGRATION_STATUS_FAILED);
-}
-trace_migration_thread_file_err();
+/*
+ * Try to detect any kind of failures, and see whether we
+ * should stop the migration now.
+ */
+thr_error = migration_detect_error(s);
+if (thr_error == MIG_THR_ERR_FATAL) {
+/* Stop migration */
 break;
+} else if (thr_error == MIG_THR_ERR_RECOVERED) {
+/*
+ * Just recovered from a e.g. network failure, reset all
+ * the local variables. This is important to avoid
+ * breaking transferred_bytes and bandwidth calculation
+ */
+s->iteration_start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
+s->iteration_initial_bytes = 0;
 }
 
 current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
@@ -2614,6 +2698,7 @@ static void migration_instance_finalize(Object *obj)
 g_free(params->tls_hostname);
 g_free(params->tls_creds);
 qemu_sem_destroy(>pause_sem);
+qemu_sem_destroy(>postcopy_pause_sem);
 error_free(ms->error);
 }
 
@@ -2643,6 +2728,8 @@ static void 

[Qemu-devel] [PULL 26/40] migration: new message MIG_RP_MSG_RESUME_ACK

[Juan Quintela]

From: Peter Xu 

Creating new message to reply for MIG_CMD_POSTCOPY_RESUME. One uint32_t
is used as payload to let the source know whether destination is ready
to continue the migration.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-15-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  | 37 +
 migration/migration.h  |  3 +++
 migration/savevm.c |  3 ++-
 migration/trace-events |  1 +
 4 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 7c5e20b3f6..4f2c6d22d1 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -96,6 +96,7 @@ enum mig_rp_message_type {
 MIG_RP_MSG_REQ_PAGES_ID, /* data (start: be64, len: be32, id: string) */
 MIG_RP_MSG_REQ_PAGES,/* data (start: be64, len: be32) */
 MIG_RP_MSG_RECV_BITMAP,  /* send recved_bitmap back to source */
+MIG_RP_MSG_RESUME_ACK,   /* tell source that we are ready to resume */
 
 MIG_RP_MSG_MAX
 };
@@ -564,6 +565,14 @@ void migrate_send_rp_recv_bitmap(MigrationIncomingState 
*mis,
 trace_migrate_send_rp_recv_bitmap(block_name, res);
 }
 
+void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t value)
+{
+uint32_t buf;
+
+buf = cpu_to_be32(value);
+migrate_send_rp_message(mis, MIG_RP_MSG_RESUME_ACK, sizeof(buf), );
+}
+
 MigrationCapabilityStatusList *qmp_query_migrate_capabilities(Error **errp)
 {
 MigrationCapabilityStatusList *head = NULL;
@@ -1843,6 +1852,7 @@ static struct rp_cmd_args {
 [MIG_RP_MSG_REQ_PAGES]  = { .len = 12, .name = "REQ_PAGES" },
 [MIG_RP_MSG_REQ_PAGES_ID]   = { .len = -1, .name = "REQ_PAGES_ID" },
 [MIG_RP_MSG_RECV_BITMAP]= { .len = -1, .name = "RECV_BITMAP" },
+[MIG_RP_MSG_RESUME_ACK] = { .len =  4, .name = "RESUME_ACK" },
 [MIG_RP_MSG_MAX]= { .len = -1, .name = "MAX" },
 };
 
@@ -1900,6 +1910,25 @@ static int migrate_handle_rp_recv_bitmap(MigrationState 
*s, char *block_name)
 return ram_dirty_bitmap_reload(s, block);
 }
 
+static int migrate_handle_rp_resume_ack(MigrationState *s, uint32_t value)
+{
+trace_source_return_path_thread_resume_ack(value);
+
+if (value != MIGRATION_RESUME_ACK_VALUE) {
+error_report("%s: illegal resume_ack value %"PRIu32,
+ __func__, value);
+return -1;
+}
+
+/* Now both sides are active. */
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_RECOVER,
+  MIGRATION_STATUS_POSTCOPY_ACTIVE);
+
+/* TODO: notify send thread that time to continue send pages */
+
+return 0;
+}
+
 /*
  * Handles messages sent on the return path towards the source VM
  *
@@ -2019,6 +2048,14 @@ retry:
 }
 break;
 
+case MIG_RP_MSG_RESUME_ACK:
+tmp32 = ldl_be_p(buf);
+if (migrate_handle_rp_resume_ack(ms, tmp32)) {
+mark_source_rp_bad(ms);
+goto out;
+}
+break;
+
 default:
 break;
 }
diff --git a/migration/migration.h b/migration/migration.h
index 2321ea37b3..556964d9d9 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -24,6 +24,8 @@
 
 struct PostcopyBlocktimeContext;
 
+#define  MIGRATION_RESUME_ACK_VALUE  (1)
+
 /* State for the incoming migration */
 struct MigrationIncomingState {
 QEMUFile *from_src_file;
@@ -262,6 +264,7 @@ int migrate_send_rp_req_pages(MigrationIncomingState *mis, 
const char* rbname,
   ram_addr_t start, size_t len);
 void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis,
  char *block_name);
+void migrate_send_rp_resume_ack(MigrationIncomingState *mis, uint32_t value);
 
 void dirty_bitmap_mig_before_vm_start(void);
 void init_dirty_bitmap_incoming_migration(void);
diff --git a/migration/savevm.c b/migration/savevm.c
index a7e793eef7..6a2d77cbf3 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1795,7 +1795,8 @@ static int 
loadvm_postcopy_handle_resume(MigrationIncomingState *mis)
 
 trace_loadvm_postcopy_handle_resume();
 
-/* TODO: Tell source that "we are ready" */
+/* Tell source that "we are ready" */
+migrate_send_rp_resume_ack(mis, MIGRATION_RESUME_ACK_VALUE);
 
 return 0;
 }
diff --git a/migration/trace-events b/migration/trace-events
index 29cc10f299..40a7217829 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -120,6 +120,7 @@ source_return_path_thread_entry(void) ""
 source_return_path_thread_loop_top(void) ""
 source_return_path_thread_pong(uint32_t val) "0x%x"
 source_return_path_thread_shut(uint32_t val) "0x%x"
+source_return_path_thread_resume_ack(uint32_t v) "%"PRIu32
 migrate_global_state_post_load(const char *state) "loaded state: %s"
 migrate_global_state_pre_save(const 

[Qemu-devel] [PULL 28/40] migration: synchronize dirty bitmap for resume

[Juan Quintela]

From: Peter Xu 

This patch implements the first part of core RAM resume logic for
postcopy. ram_resume_prepare() is provided for the work.

When the migration is interrupted by network failure, the dirty bitmap
on the source side will be meaningless, because even the dirty bit is
cleared, it is still possible that the sent page was lost along the way
to destination. Here instead of continue the migration with the old
dirty bitmap on source, we ask the destination side to send back its
received bitmap, then invert it to be our initial dirty bitmap.

The source side send thread will issue the MIG_CMD_RECV_BITMAP requests,
once per ramblock, to ask for the received bitmap. On destination side,
MIG_RP_MSG_RECV_BITMAP will be issued, along with the requested bitmap.
Data will be received on the return-path thread of source, and the main
migration thread will be notified when all the ramblock bitmaps are
synchronized.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-17-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  |  2 ++
 migration/migration.h  |  1 +
 migration/ram.c| 47 ++
 migration/trace-events |  4 
 4 files changed, 54 insertions(+)

diff --git a/migration/migration.c b/migration/migration.c
index b0217c4823..19ef8b05b1 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2967,6 +2967,7 @@ static void migration_instance_finalize(Object *obj)
 qemu_sem_destroy(>pause_sem);
 qemu_sem_destroy(>postcopy_pause_sem);
 qemu_sem_destroy(>postcopy_pause_rp_sem);
+qemu_sem_destroy(>rp_state.rp_sem);
 error_free(ms->error);
 }
 
@@ -2999,6 +3000,7 @@ static void migration_instance_init(Object *obj)
 
 qemu_sem_init(>postcopy_pause_sem, 0);
 qemu_sem_init(>postcopy_pause_rp_sem, 0);
+qemu_sem_init(>rp_state.rp_sem, 0);
 }
 
 /*
diff --git a/migration/migration.h b/migration/migration.h
index 556964d9d9..b4438ccb65 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -135,6 +135,7 @@ struct MigrationState
 QEMUFile *from_dst_file;
 QemuThreadrp_thread;
 bool  error;
+QemuSemaphore rp_sem;
 } rp_state;
 
 double mbps;
diff --git a/migration/ram.c b/migration/ram.c
index 5542843adc..b16eabcfb9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -54,6 +54,7 @@
 #include "migration/block.h"
 #include "sysemu/sysemu.h"
 #include "qemu/uuid.h"
+#include "savevm.h"
 
 /***/
 /* ram save/restore */
@@ -3364,6 +3365,38 @@ static bool ram_has_postcopy(void *opaque)
 return migrate_postcopy_ram();
 }
 
+/* Sync all the dirty bitmap with destination VM.  */
+static int ram_dirty_bitmap_sync_all(MigrationState *s, RAMState *rs)
+{
+RAMBlock *block;
+QEMUFile *file = s->to_dst_file;
+int ramblock_count = 0;
+
+trace_ram_dirty_bitmap_sync_start();
+
+RAMBLOCK_FOREACH(block) {
+qemu_savevm_send_recv_bitmap(file, block->idstr);
+trace_ram_dirty_bitmap_request(block->idstr);
+ramblock_count++;
+}
+
+trace_ram_dirty_bitmap_sync_wait();
+
+/* Wait until all the ramblocks' dirty bitmap synced */
+while (ramblock_count--) {
+qemu_sem_wait(>rp_state.rp_sem);
+}
+
+trace_ram_dirty_bitmap_sync_complete();
+
+return 0;
+}
+
+static void ram_dirty_bitmap_reload_notify(MigrationState *s)
+{
+qemu_sem_post(>rp_state.rp_sem);
+}
+
 /*
  * Read the received bitmap, revert it as the initial dirty bitmap.
  * This is only used when the postcopy migration is paused but wants
@@ -3438,12 +3471,25 @@ int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock 
*block)
 
 trace_ram_dirty_bitmap_reload_complete(block->idstr);
 
+/*
+ * We succeeded to sync bitmap for current ramblock. If this is
+ * the last one to sync, we need to notify the main send thread.
+ */
+ram_dirty_bitmap_reload_notify(s);
+
 ret = 0;
 out:
 free(le_bitmap);
 return ret;
 }
 
+static int ram_resume_prepare(MigrationState *s, void *opaque)
+{
+RAMState *rs = *(RAMState **)opaque;
+
+return ram_dirty_bitmap_sync_all(s, rs);
+}
+
 static SaveVMHandlers savevm_ram_handlers = {
 .save_setup = ram_save_setup,
 .save_live_iterate = ram_save_iterate,
@@ -3455,6 +3501,7 @@ static SaveVMHandlers savevm_ram_handlers = {
 .save_cleanup = ram_save_cleanup,
 .load_setup = ram_load_setup,
 .load_cleanup = ram_load_cleanup,
+.resume_prepare = ram_resume_prepare,
 };
 
 void ram_mig_init(void)
diff --git a/migration/trace-events b/migration/trace-events
index be36fbccfe..53243e17ec 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -82,8 +82,12 @@ ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 
" %x"
 

[Qemu-devel] [PULL 20/40] migration: rebuild channel on source

[Juan Quintela]

From: Peter Xu 

This patch detects the "resume" flag of migration command, rebuild the
channels only if the flag is set.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-9-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c | 91 +--
 1 file changed, 70 insertions(+), 21 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 19f07fb64f..038fae93ab 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1411,49 +1411,75 @@ bool migration_is_blocked(Error **errp)
 return false;
 }
 
-void qmp_migrate(const char *uri, bool has_blk, bool blk,
- bool has_inc, bool inc, bool has_detach, bool detach,
- bool has_resume, bool resume, Error **errp)
+/* Returns true if continue to migrate, or false if error detected */
+static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc,
+bool resume, Error **errp)
 {
 Error *local_err = NULL;
-MigrationState *s = migrate_get_current();
-const char *p;
+
+if (resume) {
+if (s->state != MIGRATION_STATUS_POSTCOPY_PAUSED) {
+error_setg(errp, "Cannot resume if there is no "
+   "paused migration");
+return false;
+}
+/* This is a resume, skip init status */
+return true;
+}
 
 if (migration_is_setup_or_active(s->state) ||
 s->state == MIGRATION_STATUS_CANCELLING ||
 s->state == MIGRATION_STATUS_COLO) {
 error_setg(errp, QERR_MIGRATION_ACTIVE);
-return;
+return false;
 }
+
 if (runstate_check(RUN_STATE_INMIGRATE)) {
 error_setg(errp, "Guest is waiting for an incoming migration");
-return;
+return false;
 }
 
 if (migration_is_blocked(errp)) {
-return;
+return false;
 }
 
-if ((has_blk && blk) || (has_inc && inc)) {
+if (blk || blk_inc) {
 if (migrate_use_block() || migrate_use_block_incremental()) {
 error_setg(errp, "Command options are incompatible with "
"current migration capabilities");
-return;
+return false;
 }
 migrate_set_block_enabled(true, _err);
 if (local_err) {
 error_propagate(errp, local_err);
-return;
+return false;
 }
 s->must_remove_block_options = true;
 }
 
-if (has_inc && inc) {
+if (blk_inc) {
 migrate_set_block_incremental(s, true);
 }
 
 migrate_init(s);
 
+return true;
+}
+
+void qmp_migrate(const char *uri, bool has_blk, bool blk,
+ bool has_inc, bool inc, bool has_detach, bool detach,
+ bool has_resume, bool resume, Error **errp)
+{
+Error *local_err = NULL;
+MigrationState *s = migrate_get_current();
+const char *p;
+
+if (!migrate_prepare(s, has_blk && blk, has_inc && inc,
+ has_resume && resume, errp)) {
+/* Error detected, put into errp */
+return;
+}
+
 if (strstart(uri, "tcp:", )) {
 tcp_start_outgoing_migration(s, p, _err);
 #ifdef CONFIG_RDMA
@@ -1928,7 +1954,8 @@ out:
 return NULL;
 }
 
-static int open_return_path_on_source(MigrationState *ms)
+static int open_return_path_on_source(MigrationState *ms,
+  bool create_thread)
 {
 
 ms->rp_state.from_dst_file = qemu_file_get_return_path(ms->to_dst_file);
@@ -1937,6 +1964,12 @@ static int open_return_path_on_source(MigrationState *ms)
 }
 
 trace_open_return_path_on_source();
+
+if (!create_thread) {
+/* We're done */
+return 0;
+}
+
 qemu_thread_create(>rp_state.rp_thread, "return path",
source_return_path_thread, ms, QEMU_THREAD_JOINABLE);
 
@@ -2593,6 +2626,9 @@ static void *migration_thread(void *opaque)
 
 void migrate_fd_connect(MigrationState *s, Error *error_in)
 {
+int64_t rate_limit;
+bool resume = s->state == MIGRATION_STATUS_POSTCOPY_PAUSED;
+
 s->expected_downtime = s->parameters.downtime_limit;
 s->cleanup_bh = qemu_bh_new(migrate_fd_cleanup, s);
 if (error_in) {
@@ -2601,12 +2637,21 @@ void migrate_fd_connect(MigrationState *s, Error 
*error_in)
 return;
 }
 
+if (resume) {
+/* This is a resumed migration */
+rate_limit = INT64_MAX;
+} else {
+/* This is a fresh new migration */
+rate_limit = s->parameters.max_bandwidth / XFER_LIMIT_RATIO;
+s->expected_downtime = s->parameters.downtime_limit;
+s->cleanup_bh = qemu_bh_new(migrate_fd_cleanup, s);
+
+/* Notify before starting migration thread */
+notifier_list_notify(_state_notifiers, s);
+}
+
+qemu_file_set_rate_limit(s->to_dst_file, rate_limit);
 

[Qemu-devel] [PULL 14/40] migration: new postcopy-pause state

[Juan Quintela]

From: Peter Xu 

Introducing a new state "postcopy-paused", which can be used when the
postcopy migration is paused. It is targeted for postcopy network
failure recovery.

Reviewed-by: Dr. David Alan Gilbert 
Reviewed-by: Juan Quintela 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-3-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c | 2 ++
 qapi/migration.json   | 5 -
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 61c4ee7850..02ebd6c9d1 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -574,6 +574,7 @@ static bool migration_is_setup_or_active(int state)
 switch (state) {
 case MIGRATION_STATUS_ACTIVE:
 case MIGRATION_STATUS_POSTCOPY_ACTIVE:
+case MIGRATION_STATUS_POSTCOPY_PAUSED:
 case MIGRATION_STATUS_SETUP:
 case MIGRATION_STATUS_PRE_SWITCHOVER:
 case MIGRATION_STATUS_DEVICE:
@@ -654,6 +655,7 @@ static void fill_source_migration_info(MigrationInfo *info)
 case MIGRATION_STATUS_POSTCOPY_ACTIVE:
 case MIGRATION_STATUS_PRE_SWITCHOVER:
 case MIGRATION_STATUS_DEVICE:
+case MIGRATION_STATUS_POSTCOPY_PAUSED:
  /* TODO add some postcopy stats */
 info->has_status = true;
 info->has_total_time = true;
diff --git a/qapi/migration.json b/qapi/migration.json
index f3974c6807..244334e9f4 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -89,6 +89,8 @@
 #
 # @postcopy-active: like active, but now in postcopy mode. (since 2.5)
 #
+# @postcopy-paused: during postcopy but paused. (since 2.13)
+#
 # @completed: migration is finished.
 #
 # @failed: some error occurred during migration process.
@@ -106,7 +108,8 @@
 ##
 { 'enum': 'MigrationStatus',
   'data': [ 'none', 'setup', 'cancelling', 'cancelled',
-'active', 'postcopy-active', 'completed', 'failed', 'colo',
+'active', 'postcopy-active', 'postcopy-paused',
+'completed', 'failed', 'colo',
 'pre-switchover', 'device' ] }
 
 ##
-- 
2.17.0

[Qemu-devel] [PULL 17/40] migration: allow src return path to pause

[Juan Quintela]

From: Peter Xu 

Let the thread pause for network issues.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-6-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  | 35 +--
 migration/migration.h  |  1 +
 migration/trace-events |  2 ++
 3 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 4a7c02fd3c..f9c58d2511 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1779,6 +1779,18 @@ static void migrate_handle_rp_req_pages(MigrationState 
*ms, const char* rbname,
 }
 }
 
+/* Return true to retry, false to quit */
+static bool postcopy_pause_return_path_thread(MigrationState *s)
+{
+trace_postcopy_pause_return_path();
+
+qemu_sem_wait(>postcopy_pause_rp_sem);
+
+trace_postcopy_pause_return_path_continued();
+
+return true;
+}
+
 /*
  * Handles messages sent on the return path towards the source VM
  *
@@ -1795,6 +1807,8 @@ static void *source_return_path_thread(void *opaque)
 int res;
 
 trace_source_return_path_thread_entry();
+
+retry:
 while (!ms->rp_state.error && !qemu_file_get_error(rp) &&
migration_is_setup_or_active(ms->state)) {
 trace_source_return_path_thread_loop_top();
@@ -1886,13 +1900,28 @@ static void *source_return_path_thread(void *opaque)
 break;
 }
 }
-if (qemu_file_get_error(rp)) {
+
+out:
+res = qemu_file_get_error(rp);
+if (res) {
+if (res == -EIO) {
+/*
+ * Maybe there is something we can do: it looks like a
+ * network down issue, and we pause for a recovery.
+ */
+if (postcopy_pause_return_path_thread(ms)) {
+/* Reload rp, reset the rest */
+rp = ms->rp_state.from_dst_file;
+ms->rp_state.error = false;
+goto retry;
+}
+}
+
 trace_source_return_path_thread_bad_end();
 mark_source_rp_bad(ms);
 }
 
 trace_source_return_path_thread_end();
-out:
 ms->rp_state.from_dst_file = NULL;
 qemu_fclose(rp);
 return NULL;
@@ -2700,6 +2729,7 @@ static void migration_instance_finalize(Object *obj)
 g_free(params->tls_creds);
 qemu_sem_destroy(>pause_sem);
 qemu_sem_destroy(>postcopy_pause_sem);
+qemu_sem_destroy(>postcopy_pause_rp_sem);
 error_free(ms->error);
 }
 
@@ -2731,6 +2761,7 @@ static void migration_instance_init(Object *obj)
 params->has_xbzrle_cache_size = true;
 
 qemu_sem_init(>postcopy_pause_sem, 0);
+qemu_sem_init(>postcopy_pause_rp_sem, 0);
 }
 
 /*
diff --git a/migration/migration.h b/migration/migration.h
index 0ccdcb8ffe..32fd50e9be 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -200,6 +200,7 @@ struct MigrationState
 
 /* Needed by postcopy-pause state */
 QemuSemaphore postcopy_pause_sem;
+QemuSemaphore postcopy_pause_rp_sem;
 };
 
 void migrate_set_state(int *state, int old_state, int new_state);
diff --git a/migration/trace-events b/migration/trace-events
index e23ec019be..cd971bf9fe 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -99,6 +99,8 @@ migration_thread_setup_complete(void) ""
 open_return_path_on_source(void) ""
 open_return_path_on_source_continue(void) ""
 postcopy_start(void) ""
+postcopy_pause_return_path(void) ""
+postcopy_pause_return_path_continued(void) ""
 postcopy_pause_continued(void) ""
 postcopy_pause_incoming(void) ""
 postcopy_pause_incoming_continued(void) ""
-- 
2.17.0

[Qemu-devel] [PULL 04/40] migration: Set error state in case of error

[Juan Quintela]

Signed-off-by: Juan Quintela 
---
 migration/ram.c | 26 --
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index da0b567003..4d8be30676 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -448,10 +448,22 @@ struct {
 int count;
 } *multifd_send_state;
 
-static void terminate_multifd_send_threads(Error *errp)
+static void terminate_multifd_send_threads(Error *err)
 {
 int i;
 
+if (err) {
+MigrationState *s = migrate_get_current();
+migrate_set_error(s, err);
+if (s->state == MIGRATION_STATUS_SETUP ||
+s->state == MIGRATION_STATUS_PRE_SWITCHOVER ||
+s->state == MIGRATION_STATUS_DEVICE ||
+s->state == MIGRATION_STATUS_ACTIVE) {
+migrate_set_state(>state, s->state,
+  MIGRATION_STATUS_FAILED);
+}
+}
+
 for (i = 0; i < multifd_send_state->count; i++) {
 MultiFDSendParams *p = _send_state->params[i];
 
@@ -548,10 +560,20 @@ struct {
 int count;
 } *multifd_recv_state;
 
-static void terminate_multifd_recv_threads(Error *errp)
+static void terminate_multifd_recv_threads(Error *err)
 {
 int i;
 
+if (err) {
+MigrationState *s = migrate_get_current();
+migrate_set_error(s, err);
+if (s->state == MIGRATION_STATUS_SETUP ||
+s->state == MIGRATION_STATUS_ACTIVE) {
+migrate_set_state(>state, s->state,
+  MIGRATION_STATUS_FAILED);
+}
+}
+
 for (i = 0; i < multifd_recv_state->count; i++) {
 MultiFDRecvParams *p = _recv_state->params[i];
 
-- 
2.17.0

[Qemu-devel] [PULL 16/40] migration: allow dst vm pause on postcopy

[Juan Quintela]

From: Peter Xu 

When there is IO error on the incoming channel (e.g., network down),
instead of bailing out immediately, we allow the dst vm to switch to the
new POSTCOPY_PAUSE state. Currently it is still simple - it waits the
new semaphore, until someone poke it for another attempt.

One note is that here on ram loading thread we cannot detect the
POSTCOPY_ACTIVE state, but we need to detect the more specific
POSTCOPY_INCOMING_RUNNING state, to make sure we have already loaded all
the device states.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-5-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/migration.c  |  1 +
 migration/migration.h  |  3 ++
 migration/savevm.c | 63 --
 migration/trace-events |  2 ++
 4 files changed, 67 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 8392cf467d..4a7c02fd3c 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -159,6 +159,7 @@ MigrationIncomingState *migration_incoming_get_current(void)
sizeof(struct PostCopyFD));
 qemu_mutex_init(_current.rp_mutex);
 qemu_event_init(_current.main_thread_load_event, false);
+qemu_sem_init(_current.postcopy_pause_sem_dst, 0);
 
 init_dirty_bitmap_incoming_migration();
 
diff --git a/migration/migration.h b/migration/migration.h
index 60283c39b2..0ccdcb8ffe 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -73,6 +73,9 @@ struct MigrationIncomingState {
  * live migration, to calculate vCPU block time
  * */
 struct PostcopyBlocktimeContext *blocktime_ctx;
+
+/* notify PAUSED postcopy incoming migrations to try to continue */
+QemuSemaphore postcopy_pause_sem_dst;
 };
 
 MigrationIncomingState *migration_incoming_get_current(void);
diff --git a/migration/savevm.c b/migration/savevm.c
index e2be02afe4..8ad99b1eaa 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1564,8 +1564,8 @@ static int 
loadvm_postcopy_ram_handle_discard(MigrationIncomingState *mis,
  */
 static void *postcopy_ram_listen_thread(void *opaque)
 {
-QEMUFile *f = opaque;
 MigrationIncomingState *mis = migration_incoming_get_current();
+QEMUFile *f = mis->from_src_file;
 int load_res;
 
 migrate_set_state(>state, MIGRATION_STATUS_ACTIVE,
@@ -1579,6 +1579,14 @@ static void *postcopy_ram_listen_thread(void *opaque)
  */
 qemu_file_set_blocking(f, true);
 load_res = qemu_loadvm_state_main(f, mis);
+
+/*
+ * This is tricky, but, mis->from_src_file can change after it
+ * returns, when postcopy recovery happened. In the future, we may
+ * want a wrapper for the QEMUFile handle.
+ */
+f = mis->from_src_file;
+
 /* And non-blocking again so we don't block in any cleanup */
 qemu_file_set_blocking(f, false);
 
@@ -1668,7 +1676,7 @@ static int 
loadvm_postcopy_handle_listen(MigrationIncomingState *mis)
 /* Start up the listening thread and wait for it to signal ready */
 qemu_sem_init(>listen_thread_sem, 0);
 qemu_thread_create(>listen_thread, "postcopy/listen",
-   postcopy_ram_listen_thread, mis->from_src_file,
+   postcopy_ram_listen_thread, NULL,
QEMU_THREAD_DETACHED);
 qemu_sem_wait(>listen_thread_sem);
 qemu_sem_destroy(>listen_thread_sem);
@@ -2055,11 +2063,44 @@ void qemu_loadvm_state_cleanup(void)
 }
 }
 
+/* Return true if we should continue the migration, or false. */
+static bool postcopy_pause_incoming(MigrationIncomingState *mis)
+{
+trace_postcopy_pause_incoming();
+
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+  MIGRATION_STATUS_POSTCOPY_PAUSED);
+
+assert(mis->from_src_file);
+qemu_file_shutdown(mis->from_src_file);
+qemu_fclose(mis->from_src_file);
+mis->from_src_file = NULL;
+
+assert(mis->to_src_file);
+qemu_file_shutdown(mis->to_src_file);
+qemu_mutex_lock(>rp_mutex);
+qemu_fclose(mis->to_src_file);
+mis->to_src_file = NULL;
+qemu_mutex_unlock(>rp_mutex);
+
+error_report("Detected IO failure for postcopy. "
+ "Migration paused.");
+
+while (mis->state == MIGRATION_STATUS_POSTCOPY_PAUSED) {
+qemu_sem_wait(>postcopy_pause_sem_dst);
+}
+
+trace_postcopy_pause_incoming_continued();
+
+return true;
+}
+
 static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis)
 {
 uint8_t section_type;
 int ret = 0;
 
+retry:
 while (true) {
 section_type = qemu_get_byte(f);
 
@@ -2104,6 +2145,24 @@ static int qemu_loadvm_state_main(QEMUFile *f, 
MigrationIncomingState *mis)
 out:
 if (ret < 0) {
 qemu_file_set_error(f, ret);
+
+/*
+ * Detect whether it is:
+  

[Qemu-devel] [PULL 10/40] migration: Delay start of migration main routines

[Juan Quintela]

We need to make sure that we have started all the multifd threads.

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 
---
 migration/migration.c | 4 ++--
 migration/migration.h | 1 +
 migration/ram.c   | 3 +++
 migration/socket.c| 4 
 4 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 8e5b421b97..61c4ee7850 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -430,7 +430,7 @@ static void migration_incoming_setup(QEMUFile *f)
 qemu_file_set_blocking(f, false);
 }
 
-static void migration_incoming_process(void)
+void migration_incoming_process(void)
 {
 Coroutine *co = qemu_coroutine_create(process_incoming_migration_co, NULL);
 qemu_coroutine_enter(co);
@@ -448,7 +448,7 @@ void migration_ioc_process_incoming(QIOChannel *ioc)
 
 if (!mis->from_src_file) {
 QEMUFile *f = qemu_fopen_channel_input(ioc);
-migration_fd_process_incoming(f);
+migration_incoming_setup(f);
 return;
 }
 multifd_recv_new_channel(ioc);
diff --git a/migration/migration.h b/migration/migration.h
index 7c69598c54..26e5951d56 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -200,6 +200,7 @@ void migrate_set_state(int *state, int old_state, int 
new_state);
 
 void migration_fd_process_incoming(QEMUFile *f);
 void migration_ioc_process_incoming(QIOChannel *ioc);
+void migration_incoming_process(void);
 
 bool  migration_has_all_channels(void);
 
diff --git a/migration/ram.c b/migration/ram.c
index 87434d3fce..f7e8615e15 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -717,6 +717,9 @@ void multifd_recv_new_channel(QIOChannel *ioc)
 qemu_thread_create(>thread, p->name, multifd_recv_thread, p,
QEMU_THREAD_JOINABLE);
 atomic_inc(_recv_state->count);
+if (multifd_recv_state->count == migrate_multifd_channels()) {
+migration_incoming_process();
+}
 }
 
 /**
diff --git a/migration/socket.c b/migration/socket.c
index e09fd1aae5..7a5eb562b8 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -170,6 +170,10 @@ static void 
socket_accept_incoming_migration(QIONetListener *listener,
 qio_net_listener_disconnect(listener);
 
 object_unref(OBJECT(listener));
+
+if (!migrate_use_multifd()) {
+migration_incoming_process();
+}
 }
 }
 
-- 
2.17.0

[Qemu-devel] [PULL 13/40] migration: let incoming side use thread context

[Juan Quintela]

From: Peter Xu 

The old incoming migration is running in main thread and default
gcontext.  With the new qio_channel_add_watch_full() we can now let it
run in the thread's own gcontext (if there is one).

Currently this patch does nothing alone.  But when any of the incoming
migration is run in another iothread (e.g., the upcoming migrate-recover
command), this patch will bind the incoming logic to the iothread
instead of the main thread (which may already get page faulted and
hanged).

RDMA is not considered for now since it's not even using the QIO watch
framework at all.

Reviewed-by: Dr. David Alan Gilbert 
Reviewed-by: Juan Quintela 
Signed-off-by: Peter Xu 
Message-Id: <20180502104740.12123-2-pet...@redhat.com>
Signed-off-by: Juan Quintela 
---
 migration/exec.c   | 9 -
 migration/fd.c | 9 -
 migration/socket.c | 7 ---
 3 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/migration/exec.c b/migration/exec.c
index 0bc5a427dd..9d0f82f1f0 100644
--- a/migration/exec.c
+++ b/migration/exec.c
@@ -65,9 +65,8 @@ void exec_start_incoming_migration(const char *command, Error 
**errp)
 }
 
 qio_channel_set_name(ioc, "migration-exec-incoming");
-qio_channel_add_watch(ioc,
-  G_IO_IN,
-  exec_accept_incoming_migration,
-  NULL,
-  NULL);
+qio_channel_add_watch_full(ioc, G_IO_IN,
+   exec_accept_incoming_migration,
+   NULL, NULL,
+   g_main_context_get_thread_default());
 }
diff --git a/migration/fd.c b/migration/fd.c
index cd06182d1e..9a380bbbc4 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -66,9 +66,8 @@ void fd_start_incoming_migration(const char *infd, Error 
**errp)
 }
 
 qio_channel_set_name(QIO_CHANNEL(ioc), "migration-fd-incoming");
-qio_channel_add_watch(ioc,
-  G_IO_IN,
-  fd_accept_incoming_migration,
-  NULL,
-  NULL);
+qio_channel_add_watch_full(ioc, G_IO_IN,
+   fd_accept_incoming_migration,
+   NULL, NULL,
+   g_main_context_get_thread_default());
 }
diff --git a/migration/socket.c b/migration/socket.c
index 7a5eb562b8..3456eb76e9 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -190,9 +190,10 @@ static void socket_start_incoming_migration(SocketAddress 
*saddr,
 return;
 }
 
-qio_net_listener_set_client_func(listener,
- socket_accept_incoming_migration,
- NULL, NULL);
+qio_net_listener_set_client_func_full(listener,
+  socket_accept_incoming_migration,
+  NULL, NULL,
+  g_main_context_get_thread_default());
 }
 
 void tcp_start_incoming_migration(const char *host_port, Error **errp)
-- 
2.17.0

[Qemu-devel] [PULL 12/40] migration: Define MultifdRecvParams sooner

[Juan Quintela]

Once there, we don't need the struct names anywhere, just the
typedefs.  And now also document all fields.

Signed-off-by: Juan Quintela 
Reviewed-by: Dr. David Alan Gilbert 
---
 migration/ram.c | 46 +++---
 1 file changed, 31 insertions(+), 15 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index f46a373074..cb14399ef9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -445,17 +445,45 @@ typedef struct {
 uint8_t id;
 } __attribute__((packed)) MultiFDInit_t;
 
-struct MultiFDSendParams {
+typedef struct {
+/* this fields are not changed once the thread is created */
+/* channel number */
 uint8_t id;
+/* channel thread name */
 char *name;
+/* channel thread id */
 QemuThread thread;
+/* communication channel */
 QIOChannel *c;
+/* sem where to wait for more work */
 QemuSemaphore sem;
+/* this mutex protects the following parameters */
 QemuMutex mutex;
+/* is this channel thread running */
 bool running;
+/* should this thread finish */
 bool quit;
-};
-typedef struct MultiFDSendParams MultiFDSendParams;
+}  MultiFDSendParams;
+
+typedef struct {
+/* this fields are not changed once the thread is created */
+/* channel number */
+uint8_t id;
+/* channel thread name */
+char *name;
+/* channel thread id */
+QemuThread thread;
+/* communication channel */
+QIOChannel *c;
+/* sem where to wait for more work */
+QemuSemaphore sem;
+/* this mutex protects the following parameters */
+QemuMutex mutex;
+/* is this channel thread running */
+bool running;
+/* should this thread finish */
+bool quit;
+} MultiFDRecvParams;
 
 static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
 {
@@ -657,18 +685,6 @@ int multifd_save_setup(void)
 return 0;
 }
 
-struct MultiFDRecvParams {
-uint8_t id;
-char *name;
-QemuThread thread;
-QIOChannel *c;
-QemuSemaphore sem;
-QemuMutex mutex;
-bool running;
-bool quit;
-};
-typedef struct MultiFDRecvParams MultiFDRecvParams;
-
 struct {
 MultiFDRecvParams *params;
 /* number of created threads */
-- 
2.17.0

[Qemu-devel] [PULL 03/40] tests: Migration ppc now inlines its program

[Juan Quintela]

No need to write it to a file.  Just need a proper firmware O:-)

Signed-off-by: Juan Quintela 
Reviewed-by: Laurent Vivier 
Reviewed-by: Thomas Huth 
---
 tests/migration-test.c | 41 +
 1 file changed, 5 insertions(+), 36 deletions(-)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index 2b9c0ce91e..3a85446f95 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -19,9 +19,6 @@
 #include "qemu/sockets.h"
 #include "chardev/char.h"
 #include "sysemu/sysemu.h"
-#include "hw/nvram/chrp_nvram.h"
-
-#define MIN_NVRAM_SIZE 8192 /* from spapr_nvram.c */
 
 const unsigned start_address = 1024 * 1024;
 const unsigned end_address = 100 * 1024 * 1024;
@@ -92,36 +89,6 @@ static void init_bootfile_x86(const char *bootpath)
 fclose(bootfile);
 }
 
-static void init_bootfile_ppc(const char *bootpath)
-{
-FILE *bootfile;
-char buf[MIN_NVRAM_SIZE];
-ChrpNvramPartHdr *header = (ChrpNvramPartHdr *)buf;
-
-memset(buf, 0, MIN_NVRAM_SIZE);
-
-/* Create a "common" partition in nvram to store boot-command property */
-
-header->signature = CHRP_NVPART_SYSTEM;
-memcpy(header->name, "common", 6);
-chrp_nvram_finish_partition(header, MIN_NVRAM_SIZE);
-
-/* FW_MAX_SIZE is 4MB, but slof.bin is only 900KB,
- * so let's modify memory between 1MB and 100MB
- * to do like PC bootsector
- */
-
-sprintf(buf + 16,
-"boot-command=hex .\" _\" begin %x %x do i c@ 1 + i c! 1000 +loop "
-".\" B\" 0 until", end_address, start_address);
-
-/* Write partition to the NVRAM file */
-
-bootfile = fopen(bootpath, "wb");
-g_assert_cmpint(fwrite(buf, MIN_NVRAM_SIZE, 1, bootfile), ==, 1);
-fclose(bootfile);
-}
-
 /*
  * Wait for some output in the serial output file,
  * we get an 'A' followed by an endless string of 'B's
@@ -422,12 +389,14 @@ static void test_migrate_start(QTestState **from, 
QTestState **to,
 if (access("/sys/module/kvm_hv", F_OK)) {
 accel = "tcg";
 }
-init_bootfile_ppc(bootpath);
 cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
   " -name source,debug-threads=on"
   " -serial file:%s/src_serial"
-  " -drive file=%s,if=pflash,format=raw",
-  accel, tmpfs, bootpath);
+  " -prom-env '"
+  "boot-command=hex .\" _\" begin %x %x "
+  "do i c@ 1 + i c! 1000 +loop .\" B\" 0 "
+  "until'",  accel, tmpfs, end_address,
+  start_address);
 cmd_dst = g_strdup_printf("-machine accel=%s -m 256M"
   " -name target,debug-threads=on"
   " -serial file:%s/dest_serial"
-- 
2.17.0

[Qemu-devel] [PULL 08/40] migration: Export functions to create send channels

[Juan Quintela]

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 
---
 migration/socket.c | 28 +++-
 migration/socket.h |  7 +++
 2 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/migration/socket.c b/migration/socket.c
index 122d8ccfbe..e09fd1aae5 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -28,6 +28,28 @@
 #include "trace.h"
 
 
+struct SocketOutgoingArgs {
+SocketAddress *saddr;
+} outgoing_args;
+
+void socket_send_channel_create(QIOTaskFunc f, void *data)
+{
+QIOChannelSocket *sioc = qio_channel_socket_new();
+qio_channel_socket_connect_async(sioc, outgoing_args.saddr,
+ f, data, NULL, NULL);
+}
+
+int socket_send_channel_destroy(QIOChannel *send)
+{
+/* Remove channel */
+object_unref(OBJECT(send));
+if (outgoing_args.saddr) {
+qapi_free_SocketAddress(outgoing_args.saddr);
+outgoing_args.saddr = NULL;
+}
+return 0;
+}
+
 static SocketAddress *tcp_build_address(const char *host_port, Error **errp)
 {
 SocketAddress *saddr;
@@ -95,6 +117,11 @@ static void socket_start_outgoing_migration(MigrationState 
*s,
 struct SocketConnectData *data = g_new0(struct SocketConnectData, 1);
 
 data->s = s;
+
+/* in case previous migration leaked it */
+qapi_free_SocketAddress(outgoing_args.saddr);
+outgoing_args.saddr = saddr;
+
 if (saddr->type == SOCKET_ADDRESS_TYPE_INET) {
 data->hostname = g_strdup(saddr->u.inet.host);
 }
@@ -106,7 +133,6 @@ static void socket_start_outgoing_migration(MigrationState 
*s,
  data,
  socket_connect_data_free,
  NULL);
-qapi_free_SocketAddress(saddr);
 }
 
 void tcp_start_outgoing_migration(MigrationState *s,
diff --git a/migration/socket.h b/migration/socket.h
index 6b91e9db38..528c3b0202 100644
--- a/migration/socket.h
+++ b/migration/socket.h
@@ -16,6 +16,13 @@
 
 #ifndef QEMU_MIGRATION_SOCKET_H
 #define QEMU_MIGRATION_SOCKET_H
+
+#include "io/channel.h"
+#include "io/task.h"
+
+void socket_send_channel_create(QIOTaskFunc f, void *data);
+int socket_send_channel_destroy(QIOChannel *send);
+
 void tcp_start_incoming_migration(const char *host_port, Error **errp);
 
 void tcp_start_outgoing_migration(MigrationState *s, const char *host_port,
-- 
2.17.0

[Qemu-devel] [PULL 11/40] migration: Transmit initial package through the multifd channels

[Juan Quintela]

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 

--

Be network agnostic.
Add error checking for all values.
---
 migration/ram.c | 104 +---
 1 file changed, 99 insertions(+), 5 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index f7e8615e15..f46a373074 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -52,6 +52,8 @@
 #include "qemu/rcu_queue.h"
 #include "migration/colo.h"
 #include "migration/block.h"
+#include "sysemu/sysemu.h"
+#include "qemu/uuid.h"
 
 /***/
 /* ram save/restore */
@@ -433,6 +435,16 @@ exit:
 
 /* Multiple fd's */
 
+#define MULTIFD_MAGIC 0x11223344U
+#define MULTIFD_VERSION 1
+
+typedef struct {
+uint32_t magic;
+uint32_t version;
+unsigned char uuid[16]; /* QemuUUID */
+uint8_t id;
+} __attribute__((packed)) MultiFDInit_t;
+
 struct MultiFDSendParams {
 uint8_t id;
 char *name;
@@ -445,6 +457,68 @@ struct MultiFDSendParams {
 };
 typedef struct MultiFDSendParams MultiFDSendParams;
 
+static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
+{
+MultiFDInit_t msg;
+int ret;
+
+msg.magic = cpu_to_be32(MULTIFD_MAGIC);
+msg.version = cpu_to_be32(MULTIFD_VERSION);
+msg.id = p->id;
+memcpy(msg.uuid, _uuid.data, sizeof(msg.uuid));
+
+ret = qio_channel_write_all(p->c, (char *), sizeof(msg), errp);
+if (ret != 0) {
+return -1;
+}
+return 0;
+}
+
+static int multifd_recv_initial_packet(QIOChannel *c, Error **errp)
+{
+MultiFDInit_t msg;
+int ret;
+
+ret = qio_channel_read_all(c, (char *), sizeof(msg), errp);
+if (ret != 0) {
+return -1;
+}
+
+be32_to_cpus();
+be32_to_cpus();
+
+if (msg.magic != MULTIFD_MAGIC) {
+error_setg(errp, "multifd: received packet magic %x "
+   "expected %x", msg.magic, MULTIFD_MAGIC);
+return -1;
+}
+
+if (msg.version != MULTIFD_VERSION) {
+error_setg(errp, "multifd: received packet version %d "
+   "expected %d", msg.version, MULTIFD_VERSION);
+return -1;
+}
+
+if (memcmp(msg.uuid, _uuid, sizeof(qemu_uuid))) {
+char *uuid = qemu_uuid_unparse_strdup(_uuid);
+char *msg_uuid = qemu_uuid_unparse_strdup((const QemuUUID *)msg.uuid);
+
+error_setg(errp, "multifd: received uuid '%s' and expected "
+   "uuid '%s' for channel %hhd", msg_uuid, uuid, msg.id);
+g_free(uuid);
+g_free(msg_uuid);
+return -1;
+}
+
+if (msg.id > migrate_multifd_channels()) {
+error_setg(errp, "multifd: received channel version %d "
+   "expected %d", msg.version, MULTIFD_VERSION);
+return -1;
+}
+
+return msg.id;
+}
+
 struct {
 MultiFDSendParams *params;
 /* number of created threads */
@@ -509,6 +583,11 @@ int multifd_save_cleanup(Error **errp)
 static void *multifd_send_thread(void *opaque)
 {
 MultiFDSendParams *p = opaque;
+Error *local_err = NULL;
+
+if (multifd_send_initial_packet(p, _err) < 0) {
+goto out;
+}
 
 while (true) {
 qemu_mutex_lock(>mutex);
@@ -520,6 +599,11 @@ static void *multifd_send_thread(void *opaque)
 qemu_sem_wait(>sem);
 }
 
+out:
+if (local_err) {
+multifd_send_terminate_threads(local_err);
+}
+
 qemu_mutex_lock(>mutex);
 p->running = false;
 qemu_mutex_unlock(>mutex);
@@ -704,12 +788,22 @@ bool multifd_recv_all_channels_created(void)
 void multifd_recv_new_channel(QIOChannel *ioc)
 {
 MultiFDRecvParams *p;
-/* we need to invent channels id's until we transmit */
-/* we will remove this on a later patch */
-static int i;
+Error *local_err = NULL;
+int id;
 
-p = _recv_state->params[i];
-i++;
+id = multifd_recv_initial_packet(ioc, _err);
+if (id < 0) {
+multifd_recv_terminate_threads(local_err);
+return;
+}
+
+p = _recv_state->params[id];
+if (p->c != NULL) {
+error_setg(_err, "multifd: received id '%d' already setup'",
+   id);
+multifd_recv_terminate_threads(local_err);
+return;
+}
 p->c = ioc;
 object_ref(OBJECT(ioc));
 
-- 
2.17.0

[Qemu-devel] [PULL 06/40] migration: terminate_* can be called for other threads

[Juan Quintela]

Once there, make  count field to always be accessed with atomic
operations.  To make blocking operations, we need to know that the
thread is running, so create a bool to indicate that.

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 

--

Once here, s/terminate_multifd_*-threads/multifd_*_terminate_threads/
This is consistente with every other function
---
 migration/ram.c | 44 ++--
 1 file changed, 30 insertions(+), 14 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 826172c342..1aa661e6ab 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -439,6 +439,7 @@ struct MultiFDSendParams {
 QemuThread thread;
 QemuSemaphore sem;
 QemuMutex mutex;
+bool running;
 bool quit;
 };
 typedef struct MultiFDSendParams MultiFDSendParams;
@@ -449,7 +450,7 @@ struct {
 int count;
 } *multifd_send_state;
 
-static void terminate_multifd_send_threads(Error *err)
+static void multifd_send_terminate_threads(Error *err)
 {
 int i;
 
@@ -465,7 +466,7 @@ static void terminate_multifd_send_threads(Error *err)
 }
 }
 
-for (i = 0; i < multifd_send_state->count; i++) {
+for (i = 0; i < migrate_multifd_channels(); i++) {
 MultiFDSendParams *p = _send_state->params[i];
 
 qemu_mutex_lock(>mutex);
@@ -483,11 +484,13 @@ int multifd_save_cleanup(Error **errp)
 if (!migrate_use_multifd()) {
 return 0;
 }
-terminate_multifd_send_threads(NULL);
-for (i = 0; i < multifd_send_state->count; i++) {
+multifd_send_terminate_threads(NULL);
+for (i = 0; i < migrate_multifd_channels(); i++) {
 MultiFDSendParams *p = _send_state->params[i];
 
-qemu_thread_join(>thread);
+if (p->running) {
+qemu_thread_join(>thread);
+}
 qemu_mutex_destroy(>mutex);
 qemu_sem_destroy(>sem);
 g_free(p->name);
@@ -514,6 +517,10 @@ static void *multifd_send_thread(void *opaque)
 qemu_sem_wait(>sem);
 }
 
+qemu_mutex_lock(>mutex);
+p->running = false;
+qemu_mutex_unlock(>mutex);
+
 return NULL;
 }
 
@@ -528,7 +535,7 @@ int multifd_save_setup(void)
 thread_count = migrate_multifd_channels();
 multifd_send_state = g_malloc0(sizeof(*multifd_send_state));
 multifd_send_state->params = g_new0(MultiFDSendParams, thread_count);
-multifd_send_state->count = 0;
+atomic_set(_send_state->count, 0);
 for (i = 0; i < thread_count; i++) {
 MultiFDSendParams *p = _send_state->params[i];
 
@@ -537,10 +544,11 @@ int multifd_save_setup(void)
 p->quit = false;
 p->id = i;
 p->name = g_strdup_printf("multifdsend_%d", i);
+p->running = true;
 qemu_thread_create(>thread, p->name, multifd_send_thread, p,
QEMU_THREAD_JOINABLE);
 
-multifd_send_state->count++;
+atomic_inc(_send_state->count);
 }
 return 0;
 }
@@ -551,6 +559,7 @@ struct MultiFDRecvParams {
 QemuThread thread;
 QemuSemaphore sem;
 QemuMutex mutex;
+bool running;
 bool quit;
 };
 typedef struct MultiFDRecvParams MultiFDRecvParams;
@@ -561,7 +570,7 @@ struct {
 int count;
 } *multifd_recv_state;
 
-static void terminate_multifd_recv_threads(Error *err)
+static void multifd_recv_terminate_threads(Error *err)
 {
 int i;
 
@@ -575,7 +584,7 @@ static void terminate_multifd_recv_threads(Error *err)
 }
 }
 
-for (i = 0; i < multifd_recv_state->count; i++) {
+for (i = 0; i < migrate_multifd_channels(); i++) {
 MultiFDRecvParams *p = _recv_state->params[i];
 
 qemu_mutex_lock(>mutex);
@@ -593,11 +602,13 @@ int multifd_load_cleanup(Error **errp)
 if (!migrate_use_multifd()) {
 return 0;
 }
-terminate_multifd_recv_threads(NULL);
-for (i = 0; i < multifd_recv_state->count; i++) {
+multifd_recv_terminate_threads(NULL);
+for (i = 0; i < migrate_multifd_channels(); i++) {
 MultiFDRecvParams *p = _recv_state->params[i];
 
-qemu_thread_join(>thread);
+if (p->running) {
+qemu_thread_join(>thread);
+}
 qemu_mutex_destroy(>mutex);
 qemu_sem_destroy(>sem);
 g_free(p->name);
@@ -625,6 +636,10 @@ static void *multifd_recv_thread(void *opaque)
 qemu_sem_wait(>sem);
 }
 
+qemu_mutex_lock(>mutex);
+p->running = false;
+qemu_mutex_unlock(>mutex);
+
 return NULL;
 }
 
@@ -639,7 +654,7 @@ int multifd_load_setup(void)
 thread_count = migrate_multifd_channels();
 multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state));
 multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count);
-multifd_recv_state->count = 0;
+atomic_set(_recv_state->count, 0);
 for (i = 0; i < thread_count; i++) {
 MultiFDRecvParams *p = _recv_state->params[i];
 
@@ -648,9 +663,10 @@ int multifd_load_setup(void)
 p->quit = 

[Qemu-devel] [PULL 00/40] Migration PULL requset (take 2)

[Juan Quintela]

Hi

- dropped xbzrle test (have to check with slower systems)
- fixed s/2.12/2.13/ detected by eric

Please Apply.

The following changes since commit c416eecea5f3aea863ab8fda5a36a24157b8f704:

  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging 
(2018-05-15 17:02:00 +0100)

are available in the Git repository at:

  git://github.com/juanquintela/qemu.git tags/migration/20180515

for you to fetch changes up to 8b7bf2badac25c0a52aff1b181ad75fdb304dd0c:

  Migration+TLS: Fix crash due to double cleanup (2018-05-15 22:13:08 +0200)


migration/next for 20180515


Dr. David Alan Gilbert (3):
  migration: update docs
  migration: Textual fixups for blocktime
  Migration+TLS: Fix crash due to double cleanup

Juan Quintela (11):
  tests: Add migration precopy test
  tests: Migration ppc now inlines its program
  migration: Set error state in case of error
  migration: Introduce multifd_recv_new_channel()
  migration: terminate_* can be called for other threads
  migration: Be sure all recv channels are created
  migration: Export functions to create send channels
  migration: Create multifd channels
  migration: Delay start of migration main routines
  migration: Transmit initial package through the multifd channels
  migration: Define MultifdRecvParams sooner

Lidong Chen (1):
  migration: update index field when delete or qsort RDMALocalBlock

Peter Xu (24):
  migration: let incoming side use thread context
  migration: new postcopy-pause state
  migration: implement "postcopy-pause" src logic
  migration: allow dst vm pause on postcopy
  migration: allow src return path to pause
  migration: allow fault thread to pause
  qmp: hmp: add migrate "resume" option
  migration: rebuild channel on source
  migration: new state "postcopy-recover"
  migration: wakeup dst ram-load-thread for recover
  migration: new cmd MIG_CMD_RECV_BITMAP
  migration: new message MIG_RP_MSG_RECV_BITMAP
  migration: new cmd MIG_CMD_POSTCOPY_RESUME
  migration: new message MIG_RP_MSG_RESUME_ACK
  migration: introduce SaveVMHandlers.resume_prepare
  migration: synchronize dirty bitmap for resume
  migration: setup ramstate for resume
  migration: final handshake for the resume
  migration: init dst in migration_object_init too
  qmp/migration: new command migrate-recover
  hmp/migration: add migrate_recover command
  migration: introduce lock for to_dst_file
  migration/qmp: add command migrate-pause
  migration/hmp: add migrate_pause command

Xiao Guangrong (1):
  migration: fix saving normal page even if it's been compressed

 docs/devel/migration.rst | 532 
 hmp-commands.hx  |  34 ++-
 hmp.c|  23 +-
 hmp.h|   2 +
 include/migration/register.h |   2 +
 migration/channel.c  |  12 +-
 migration/exec.c |   9 +-
 migration/fd.c   |   9 +-
 migration/migration.c| 559 +++
 migration/migration.h|  22 ++
 migration/postcopy-ram.c |  54 -
 migration/ram.c  | 500 +++---
 migration/ram.h  |   6 +
 migration/rdma.c |   7 +
 migration/savevm.c   | 191 ++-
 migration/savevm.h   |   3 +
 migration/socket.c   |  39 ++-
 migration/socket.h   |   7 +
 migration/trace-events   |  21 ++
 qapi/migration.json  |  57 -
 tests/migration-test.c   |  85 ---
 21 files changed, 1864 insertions(+), 310 deletions(-)

[Qemu-devel] [PULL 09/40] migration: Create multifd channels

[Juan Quintela]

In both sides.  We still don't transmit anything through them.

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 
---
 migration/ram.c | 52 +++--
 1 file changed, 42 insertions(+), 10 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index d5335c10b6..87434d3fce 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -437,6 +437,7 @@ struct MultiFDSendParams {
 uint8_t id;
 char *name;
 QemuThread thread;
+QIOChannel *c;
 QemuSemaphore sem;
 QemuMutex mutex;
 bool running;
@@ -491,6 +492,8 @@ int multifd_save_cleanup(Error **errp)
 if (p->running) {
 qemu_thread_join(>thread);
 }
+socket_send_channel_destroy(p->c);
+p->c = NULL;
 qemu_mutex_destroy(>mutex);
 qemu_sem_destroy(>sem);
 g_free(p->name);
@@ -524,6 +527,27 @@ static void *multifd_send_thread(void *opaque)
 return NULL;
 }
 
+static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque)
+{
+MultiFDSendParams *p = opaque;
+QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));
+Error *local_err = NULL;
+
+if (qio_task_propagate_error(task, _err)) {
+if (multifd_save_cleanup(_err) != 0) {
+migrate_set_error(migrate_get_current(), local_err);
+}
+} else {
+p->c = QIO_CHANNEL(sioc);
+qio_channel_set_delay(p->c, false);
+p->running = true;
+qemu_thread_create(>thread, p->name, multifd_send_thread, p,
+   QEMU_THREAD_JOINABLE);
+
+atomic_inc(_send_state->count);
+}
+}
+
 int multifd_save_setup(void)
 {
 int thread_count;
@@ -544,11 +568,7 @@ int multifd_save_setup(void)
 p->quit = false;
 p->id = i;
 p->name = g_strdup_printf("multifdsend_%d", i);
-p->running = true;
-qemu_thread_create(>thread, p->name, multifd_send_thread, p,
-   QEMU_THREAD_JOINABLE);
-
-atomic_inc(_send_state->count);
+socket_send_channel_create(multifd_new_send_channel_async, p);
 }
 return 0;
 }
@@ -557,6 +577,7 @@ struct MultiFDRecvParams {
 uint8_t id;
 char *name;
 QemuThread thread;
+QIOChannel *c;
 QemuSemaphore sem;
 QemuMutex mutex;
 bool running;
@@ -609,6 +630,8 @@ int multifd_load_cleanup(Error **errp)
 if (p->running) {
 qemu_thread_join(>thread);
 }
+object_unref(OBJECT(p->c));
+p->c = NULL;
 qemu_mutex_destroy(>mutex);
 qemu_sem_destroy(>sem);
 g_free(p->name);
@@ -663,10 +686,6 @@ int multifd_load_setup(void)
 p->quit = false;
 p->id = i;
 p->name = g_strdup_printf("multifdrecv_%d", i);
-p->running = true;
-qemu_thread_create(>thread, p->name, multifd_recv_thread, p,
-   QEMU_THREAD_JOINABLE);
-atomic_inc(_recv_state->count);
 }
 return 0;
 }
@@ -684,7 +703,20 @@ bool multifd_recv_all_channels_created(void)
 
 void multifd_recv_new_channel(QIOChannel *ioc)
 {
-/* nothing to do yet */
+MultiFDRecvParams *p;
+/* we need to invent channels id's until we transmit */
+/* we will remove this on a later patch */
+static int i;
+
+p = _recv_state->params[i];
+i++;
+p->c = ioc;
+object_ref(OBJECT(ioc));
+
+p->running = true;
+qemu_thread_create(>thread, p->name, multifd_recv_thread, p,
+   QEMU_THREAD_JOINABLE);
+atomic_inc(_recv_state->count);
 }
 
 /**
-- 
2.17.0

[Qemu-devel] [PULL 02/40] tests: Add migration precopy test

[Juan Quintela]

Signed-off-by: Juan Quintela 
Reviewed-by: Dr. David Alan Gilbert 
Reviewed-by: Peter Xu 
---
 tests/migration-test.c | 44 --
 1 file changed, 42 insertions(+), 2 deletions(-)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index b99661b773..2b9c0ce91e 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -536,7 +536,7 @@ static void test_deprecated(void)
 qtest_quit(from);
 }
 
-static void test_migrate(void)
+static void test_postcopy(void)
 {
 char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
 QTestState *from, *to;
@@ -611,6 +611,45 @@ static void test_baddest(void)
 test_migrate_end(from, to, false);
 }
 
+static void test_precopy_unix(void)
+{
+char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+QTestState *from, *to;
+
+test_migrate_start(, , uri, false);
+
+/* We want to pick a speed slow enough that the test completes
+ * quickly, but that it doesn't complete precopy even on a slow
+ * machine, so also set the downtime.
+ */
+/* 1 ms should make it not converge*/
+migrate_set_parameter(from, "downtime-limit", "1");
+/* 1GB/s */
+migrate_set_parameter(from, "max-bandwidth", "10");
+
+/* Wait for the first serial output from the source */
+wait_for_serial("src_serial");
+
+migrate(from, uri);
+
+wait_for_migration_pass(from);
+
+/* 300 ms should converge */
+migrate_set_parameter(from, "downtime-limit", "300");
+
+if (!got_stop) {
+qtest_qmp_eventwait(from, "STOP");
+}
+
+qtest_qmp_eventwait(to, "RESUME");
+
+wait_for_serial("dest_serial");
+wait_for_migration_complete(from);
+
+test_migrate_end(from, to, true);
+g_free(uri);
+}
+
 int main(int argc, char **argv)
 {
 char template[] = "/tmp/migration-test-XX";
@@ -630,9 +669,10 @@ int main(int argc, char **argv)
 
 module_call_init(MODULE_INIT_QOM);
 
-qtest_add_func("/migration/postcopy/unix", test_migrate);
+qtest_add_func("/migration/postcopy/unix", test_postcopy);
 qtest_add_func("/migration/deprecated", test_deprecated);
 qtest_add_func("/migration/bad_dest", test_baddest);
+qtest_add_func("/migration/precopy/unix", test_precopy_unix);
 
 ret = g_test_run();
 
-- 
2.17.0

[Qemu-devel] [PULL 05/40] migration: Introduce multifd_recv_new_channel()

[Juan Quintela]

Signed-off-by: Juan Quintela 
Reviewed-by: Daniel P. Berrangé 
---
 migration/migration.c | 3 ++-
 migration/ram.c   | 6 ++
 migration/ram.h   | 2 ++
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 35f2781b03..4a7959c111 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -449,8 +449,9 @@ void migration_ioc_process_incoming(QIOChannel *ioc)
 if (!mis->from_src_file) {
 QEMUFile *f = qemu_fopen_channel_input(ioc);
 migration_fd_process_incoming(f);
+return;
 }
-/* We still only have a single channel.  Nothing to do here yet */
+multifd_recv_new_channel(ioc);
 }
 
 /**
diff --git a/migration/ram.c b/migration/ram.c
index 4d8be30676..826172c342 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -36,6 +36,7 @@
 #include "xbzrle.h"
 #include "ram.h"
 #include "migration.h"
+#include "socket.h"
 #include "migration/register.h"
 #include "migration/misc.h"
 #include "qemu-file.h"
@@ -654,6 +655,11 @@ int multifd_load_setup(void)
 return 0;
 }
 
+void multifd_recv_new_channel(QIOChannel *ioc)
+{
+/* nothing to do yet */
+}
+
 /**
  * save_page_header: write page header to wire
  *
diff --git a/migration/ram.h b/migration/ram.h
index 5030be110a..06dbddc2a2 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -32,6 +32,7 @@
 #include "qemu-common.h"
 #include "qapi/qapi-types-migration.h"
 #include "exec/cpu-common.h"
+#include "io/channel.h"
 
 extern MigrationStats ram_counters;
 extern XBZRLECacheStats xbzrle_counters;
@@ -44,6 +45,7 @@ int multifd_save_setup(void);
 int multifd_save_cleanup(Error **errp);
 int multifd_load_setup(void);
 int multifd_load_cleanup(Error **errp);
+void multifd_recv_new_channel(QIOChannel *ioc);
 
 uint64_t ram_pagesize_summary(void);
 int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t len);
-- 
2.17.0

[Qemu-devel] [PULL 01/40] migration: fix saving normal page even if it's been compressed

[Juan Quintela]

From: Xiao Guangrong 

Fix the bug introduced by da3f56cb2e767016 (migration: remove
ram_save_compressed_page()), It should be 'return' rather than
'res'

Sorry for this stupid mistake :(

Signed-off-by: Xiao Guangrong 
Message-Id: <20180428081045.8878-1-xiaoguangr...@tencent.com>
Signed-off-by: Juan Quintela 
---
 migration/ram.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index 912810c18e..da0b567003 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1490,7 +1490,7 @@ static int ram_save_target_page(RAMState *rs, 
PageSearchStatus *pss,
  * CPU resource.
  */
 if (block == rs->last_sent_block && save_page_use_compression(rs)) {
-res = compress_page_with_multi_thread(rs, block, offset);
+return compress_page_with_multi_thread(rs, block, offset);
 }
 
 return ram_save_page(rs, pss, last_stage);
-- 
2.17.0

Re: [Qemu-devel] [PATCH 3/3] nvdimm: platform capabilities command line option

[Ross Zwisler]

On Thu, May 10, 2018 at 03:28:48PM +0200, Igor Mammedov wrote:
> On Fri, 27 Apr 2018 15:53:14 -0600
> Ross Zwisler  wrote:
> 
> > Add a device command line option to allow the user to control the Platform
> > Capabilities Structure in the virtualized NFIT.
> > 
> > Signed-off-by: Ross Zwisler 
> > ---
> >  docs/nvdimm.txt | 22 ++
> >  hw/acpi/nvdimm.c| 29 +
> >  hw/mem/nvdimm.c | 28 
> >  include/hw/mem/nvdimm.h |  6 ++
> >  4 files changed, 81 insertions(+), 4 deletions(-)
> > 
> > diff --git a/docs/nvdimm.txt b/docs/nvdimm.txt
> > index e903d8bb09..13a2c15b70 100644
> > --- a/docs/nvdimm.txt
> > +++ b/docs/nvdimm.txt
> > @@ -153,3 +153,25 @@ guest NVDIMM region mapping structure.  This unarmed 
> > flag indicates
> >  guest software that this vNVDIMM device contains a region that cannot
> >  accept persistent writes. In result, for example, the guest Linux
> >  NVDIMM driver, marks such vNVDIMM device as read-only.
> > +
> > +Platform Capabilities
> > +-
> > +
> > +ACPI 6.2 Errata A added support for a new Platform Capabilities Structure
> > +which allows the platform to communicate what features it supports related 
> > to
> > +NVDIMM data durability.  Users can provide a capabilities value to a guest 
> > via
> > +the optional "cap" device command line option:
> > +
> > + -device nvdimm,id=nvdimm1,memdev=mem1,cap=3
> > +
> > +As of ACPI 6.2 Errata A, the following values are valid for the bottom two
> > +bits:
> > +
> > +2 - Memory Controller Flush to NVDIMM Durability on Power Loss Capable.
> > +3 - CPU Cache Flush to NVDIMM Durability on Power Loss Capable.
> > +
> > +For a complete list of the flags available please consult the ACPI spec.
> > +
> > +These platform capabilities apply to the entire virtual platform, so it is
> > +recommended that only one "cap" device command option be given per virtual
> > +machine.  This value will apply to all NVDIMMs in the virtual platform.
> This looks like it should be machine property instead of per device one,
> you can get rid of static variable and mismatch check and a weird nvdimm CLI
> option that implies that the option is per device.

Yep, that's much better.  I have this implemented and ready to go.

> Also an extra patch to for make check that will test setting 'cap'
> would be nice (an extra testcase in tests/bios-tables-test.c)

Hmm...I've been looking at this, and it doesn't look like there is any
verification around a lot of the ACPI tables (NFIT, SRAT, etc).

I've verified my patch by interacting with a guest with various settings - is
this good enough, or do you really want me to test the value (which I think
would just be "do I get out what I put in at the command line") via the unit
test infrastructure?

Thank you for the review.

[Qemu-devel] [Bug 1771238] Re: Not able to passthrough > 32 PCIe devices to a KVM Guest

[Khaled El Mously]

@David Coronel: It's not clear to me - is this a regression?

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1771238

Title:
  Not able to passthrough > 32 PCIe devices to a KVM Guest

Status in QEMU:
  New
Status in qemu package in Ubuntu:
  New

Bug description:
  Using an Ubuntu Server 16.04-based host with KVM hypervisor installed,
  we are unable to launch a vanilla Ubuntu Server 16.04.4 guest with >=
  32 PCIe devices. It is 100% reproducible. Using fewer PCIe devices
  works fine. We are using the vanilla kvm and qemu packages from the
  Canonical repos.

  The ultimate goal is to create a KVM Guest wherein I can passthrough
  44 PCI devices.

  When a KVM Guest launches, it also has some internal PCIe devices
  including host bridge, USB, IDE (for virtual disk), and virtual nic
  etc.

  Script used to launch all devices looks like this:

  #!/bin/bash 
  NAME=16gpuvm 

  sudo qemu-img create -f qcow2 /home/lab/kvm/images/${NAME}.img 40G && 
  sudo virt-install \ 
  --name ${NAME} \ 
  --ram 716800 \ 
  --vcpus 88 \ 
  --disk path=/home/lab/kvm/images/${NAME}.img,format=qcow2 \ 
  --network bridge=virbr0 \ 
  --graphics none \ 
  --host-device 34:00.0 \ 
  --host-device 36:00.0 \ 
  --host-device 39:00.0 \ 
  --host-device 3b:00.0 \ 
  --host-device 57:00.0 \ 
  --host-device 59:00.0 \ 
  --host-device 5c:00.0 \ 
  --host-device 5e:00.0 \ 
  --host-device 61:00.0 \ 
  --host-device 62:00.0 \ 
  --host-device 63:00.0 \ 
  --host-device 65:00.0 \ 
  --host-device 66:00.0 \ 
  --host-device 67:00.0 \ 
  --host-device 35:00.0 \ 
  --host-device 3a:00.0 \ 
  --host-device 58:00.0 \ 
  --host-device 5d:00.0 \ 
  --host-device 2e:00.0 \ 
  --host-device 2f:00.0 \ 
  --host-device 51:00.0 \ 
  --host-device 52:00.0 \ 
  --host-device b7:00.0 \ 
  --host-device b9:00.0 \ 
  --host-device bc:00.0 \ 
  --host-device be:00.0 \ 
  --host-device e0:00.0 \ 
  --host-device e2:00.0 \ 
  --host-device e5:00.0 \ 
  --host-device e7:00.0 \ 
  --host-device c1:00.0 \ 
  --host-device c2:00.0 \ 
  --host-device c3:00.0 \ 
  --host-device c5:00.0 \ 
  --host-device c6:00.0 \ 
  --host-device c7:00.0 \ 
  --host-device b8:00.0 \ 
  --host-device bd:00.0 \ 
  --host-device e1:00.0 \ 
  --host-device e6:00.0 \ 
  --host-device b1:00.0 \ 
  --host-device b2:00.0 \ 
  --host-device da:00.0 \ 
  --host-device db:00.0 \ 
  --console pty,target_type=serial \ 
  --location http://ftp.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64 \ 
  --initrd-inject=/home/lab/kvm/images/preseed.cfg \ 
  --extra-args=" 
  console=ttyS0,115200 
  locale=en_US 
  console-keymaps-at/keymap=us 
  console-setup/ask_detect=false 
  console-setup/layoutcode=us 
  keyboard-configuration/layout=USA 
  keyboard-configuration/variant=USA 
  hostname=${NAME} 
  file=file:/preseed.cfg 
  "

  Passing > 32 device causes this issue:  32nd device hits a DPC error
  and the host/HV crashes:

  
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.977496] dpc :5b:10.0:pcie210: 
DPC containment event, status:0x0009 source:0x
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.977500] dpc :5b:10.0:pcie210: 
DPC unmasked uncorrectable error detected, remove downstream devices
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.994326] vfio-pci :5e:00.0: 
Refused to change power state, currently in D3
  Apr 25 22:34:35 xpl-evt-16 kernel: [18125.994427] iommu: Removing device 
:5e:00.0 from group 92

  
  From syslog (attached)

  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194358] dpc :bb:04.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194387] dpc :bb:10.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194413] dpc :d9:00.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194439] dpc :d9:01.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194472] dpc :d9:02.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194499] dpc :d9:03.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194526] dpc :d9:04.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP PIO Log 0, DL_ActiveErr+
  Apr 25 22:37:13 xpl-evt-16 kernel: [2.194553] dpc :d9:0c.0:pcie210: 
DPC error containment capabilities: Int Msg #3, RPExt- PoisonedTLP+ SwTrigger+ 
RP 

[Qemu-devel] [PATCH v6 24/28] fpu/softfloat: Pass FloatClass to pickNaN

[Richard Henderson]

For each operand, pass a single enumeration instead of a pair of booleans.
The commit also merges multiple different ifdef-selected implementations
of pickNaN into a single function whose body is ifdef-selected.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 168 ++---
 fpu/softfloat.c|   3 +-
 2 files changed, 82 insertions(+), 89 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index d1e06da75b..2695183188 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -501,10 +501,10 @@ static float32 commonNaNToFloat32(commonNaNT a, 
float_status *status)
 | tie-break rule.
 **/
 
-#if defined(TARGET_ARM)
-static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
+static int pickNaN(FloatClass a_cls, FloatClass b_cls,
flag aIsLargerSignificand)
 {
+#if defined(TARGET_ARM) || defined(TARGET_MIPS) || defined(TARGET_HPPA)
 /* ARM mandated NaN propagation rules (see FPProcessNaNs()), take
  * the first of:
  *  1. A if it is signaling
@@ -513,20 +513,6 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
  *  4. B (quiet)
  * A signaling NaN is always quietened before returning it.
  */
-if (aIsSNaN) {
-return 0;
-} else if (bIsSNaN) {
-return 1;
-} else if (aIsQNaN) {
-return 0;
-} else {
-return 1;
-}
-}
-#elif defined(TARGET_MIPS) || defined(TARGET_HPPA)
-static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
-flag aIsLargerSignificand)
-{
 /* According to MIPS specifications, if one of the two operands is
  * a sNaN, a new qNaN has to be generated. This is done in
  * floatXX_silence_nan(). For qNaN inputs the specifications
@@ -540,35 +526,21 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
  *  4. B (quiet)
  * A signaling NaN is always silenced before returning it.
  */
-if (aIsSNaN) {
+if (is_snan(a_cls)) {
 return 0;
-} else if (bIsSNaN) {
+} else if (is_snan(b_cls)) {
 return 1;
-} else if (aIsQNaN) {
+} else if (is_qnan(a_cls)) {
 return 0;
 } else {
 return 1;
 }
-}
-#elif defined(TARGET_PPC) || defined(TARGET_XTENSA)
-static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
-   flag aIsLargerSignificand)
-{
+#elif defined(TARGET_PPC) || defined(TARGET_XTENSA) || defined(TARGET_M68K)
 /* PowerPC propagation rules:
  *  1. A if it sNaN or qNaN
  *  2. B if it sNaN or qNaN
  * A signaling NaN is always silenced before returning it.
  */
-if (aIsSNaN || aIsQNaN) {
-return 0;
-} else {
-return 1;
-}
-}
-#elif defined(TARGET_M68K)
-static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
-   flag aIsLargerSignificand)
-{
 /* M68000 FAMILY PROGRAMMER'S REFERENCE MANUAL
  * 3.4 FLOATING-POINT INSTRUCTION DETAILS
  * If either operand, but not both operands, of an operation is a
@@ -583,16 +555,12 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
  * a nonsignaling NaN. The operation then continues as described in the
  * preceding paragraph for nonsignaling NaNs.
  */
-if (aIsQNaN || aIsSNaN) { /* a is the destination operand */
-return 0; /* return the destination operand */
+if (is_nan(a_cls)) {
+return 0;
 } else {
-return 1; /* return b */
+return 1;
 }
-}
 #else
-static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
-flag aIsLargerSignificand)
-{
 /* This implements x87 NaN propagation rules:
  * SNaN + QNaN => return the QNaN
  * two SNaNs => return the one with the larger significand, silenced
@@ -603,13 +571,13 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
  * If we get down to comparing significands and they are the same,
  * return the NaN with the positive sign bit (if any).
  */
-if (aIsSNaN) {
-if (bIsSNaN) {
+if (is_snan(a_cls)) {
+if (is_snan(b_cls)) {
 return aIsLargerSignificand ? 0 : 1;
 }
-return bIsQNaN ? 1 : 0;
-} else if (aIsQNaN) {
-if (bIsSNaN || !bIsQNaN) {
+return is_qnan(b_cls) ? 1 : 0;
+} else if (is_qnan(a_cls)) {
+if (is_snan(b_cls) || !is_qnan(b_cls)) {
 return 0;
 } else {
 return aIsLargerSignificand ? 0 : 1;
@@ -617,8 +585,8 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
 } else {
 return 1;
 }
-}
 #endif
+}
 
 

[Qemu-devel] [PATCH v6 19/28] target/s390x: Remove floatX_maybe_silence_nan from conversions

[Richard Henderson]

This is now handled properly by the generic softfloat code.

Cc: Alexander Graf 
Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 target/s390x/fpu_helper.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/target/s390x/fpu_helper.c b/target/s390x/fpu_helper.c
index 43f8bf1c94..5c5b451b3b 100644
--- a/target/s390x/fpu_helper.c
+++ b/target/s390x/fpu_helper.c
@@ -269,7 +269,7 @@ uint64_t HELPER(ldeb)(CPUS390XState *env, uint64_t f2)
 {
 float64 ret = float32_to_float64(f2, >fpu_status);
 handle_exceptions(env, GETPC());
-return float64_maybe_silence_nan(ret, >fpu_status);
+return ret;
 }
 
 /* convert 128-bit float to 64-bit float */
@@ -277,7 +277,7 @@ uint64_t HELPER(ldxb)(CPUS390XState *env, uint64_t ah, 
uint64_t al)
 {
 float64 ret = float128_to_float64(make_float128(ah, al), >fpu_status);
 handle_exceptions(env, GETPC());
-return float64_maybe_silence_nan(ret, >fpu_status);
+return ret;
 }
 
 /* convert 64-bit float to 128-bit float */
@@ -285,7 +285,7 @@ uint64_t HELPER(lxdb)(CPUS390XState *env, uint64_t f2)
 {
 float128 ret = float64_to_float128(f2, >fpu_status);
 handle_exceptions(env, GETPC());
-return RET128(float128_maybe_silence_nan(ret, >fpu_status));
+return RET128(ret);
 }
 
 /* convert 32-bit float to 128-bit float */
@@ -293,7 +293,7 @@ uint64_t HELPER(lxeb)(CPUS390XState *env, uint64_t f2)
 {
 float128 ret = float32_to_float128(f2, >fpu_status);
 handle_exceptions(env, GETPC());
-return RET128(float128_maybe_silence_nan(ret, >fpu_status));
+return RET128(ret);
 }
 
 /* convert 64-bit float to 32-bit float */
@@ -301,7 +301,7 @@ uint64_t HELPER(ledb)(CPUS390XState *env, uint64_t f2)
 {
 float32 ret = float64_to_float32(f2, >fpu_status);
 handle_exceptions(env, GETPC());
-return float32_maybe_silence_nan(ret, >fpu_status);
+return ret;
 }
 
 /* convert 128-bit float to 32-bit float */
@@ -309,7 +309,7 @@ uint64_t HELPER(lexb)(CPUS390XState *env, uint64_t ah, 
uint64_t al)
 {
 float32 ret = float128_to_float32(make_float128(ah, al), >fpu_status);
 handle_exceptions(env, GETPC());
-return float32_maybe_silence_nan(ret, >fpu_status);
+return ret;
 }
 
 /* 32-bit FP compare */
-- 
2.17.0

[Qemu-devel] [PATCH v6 23/28] fpu/softfloat: Make is_nan et al available to softfloat-specialize.h

[Richard Henderson]

We will need these helpers within softfloat-specialize.h, so move
the definitions above the include.  After specialization, they will
not always be used so mark them to avoid the Werror.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat.c | 30 --
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 55e6701f26..ea252e0c84 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -181,6 +181,22 @@ typedef enum __attribute__ ((__packed__)) {
 float_class_snan,
 } FloatClass;
 
+/* Simple helpers for checking if, or what kind of, NaN we have */
+static inline __attribute__((unused)) bool is_nan(FloatClass c)
+{
+return unlikely(c >= float_class_qnan);
+}
+
+static inline __attribute__((unused)) bool is_snan(FloatClass c)
+{
+return c == float_class_snan;
+}
+
+static inline __attribute__((unused)) bool is_qnan(FloatClass c)
+{
+return c == float_class_qnan;
+}
+
 /*
  * Structure holding all of the decomposed parts of a float. The
  * exponent is unbiased and the fraction is normalized. All
@@ -536,20 +552,6 @@ static float64 float64_round_pack_canonical(FloatParts p, 
float_status *s)
 return float64_pack_raw(round_canonical(p, s, _params));
 }
 
-/* Simple helpers for checking if what NaN we have */
-static bool is_nan(FloatClass c)
-{
-return unlikely(c >= float_class_qnan);
-}
-static bool is_snan(FloatClass c)
-{
-return c == float_class_snan;
-}
-static bool is_qnan(FloatClass c)
-{
-return c == float_class_qnan;
-}
-
 static FloatParts return_nan(FloatParts a, float_status *s)
 {
 switch (a.cls) {
-- 
2.17.0

[Qemu-devel] [PATCH v6 12/28] fpu/softfloat: re-factor float to float conversions

[Richard Henderson]

From: Alex Bennée 

This allows us to delete a lot of additional boilerplate
code which is no longer needed.

Reviewed-by: Peter Maydell 
Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 

---
v2
  - pass FloatFmt to float_to_float instead of sizes
  - split AHP handling to another patch
  - use rth's suggested re-packing (+ setting .exp)
v3
  - also rm extractFloat16Sign
v4
  - update for canonical nan handling
v5
  - merge arm alt fp16 support into this patch
---
 fpu/softfloat-specialize.h |  40 ---
 include/fpu/softfloat.h|   8 +-
 fpu/softfloat.c| 488 +
 3 files changed, 122 insertions(+), 414 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 571d1df378..995a0132c6 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -377,46 +377,6 @@ float16 float16_maybe_silence_nan(float16 a, float_status 
*status)
 return a;
 }
 
-/*
-| Returns the result of converting the half-precision floating-point NaN
-| `a' to the canonical NaN format.  If `a' is a signaling NaN, the invalid
-| exception is raised.
-**/
-
-static commonNaNT float16ToCommonNaN(float16 a, float_status *status)
-{
-commonNaNT z;
-
-if (float16_is_signaling_nan(a, status)) {
-float_raise(float_flag_invalid, status);
-}
-z.sign = float16_val(a) >> 15;
-z.low = 0;
-z.high = ((uint64_t) float16_val(a)) << 54;
-return z;
-}
-
-/*
-| Returns the result of converting the canonical NaN `a' to the half-
-| precision floating-point format.
-**/
-
-static float16 commonNaNToFloat16(commonNaNT a, float_status *status)
-{
-uint16_t mantissa = a.high >> 54;
-
-if (status->default_nan_mode) {
-return float16_default_nan(status);
-}
-
-if (mantissa) {
-return make_float16(uint16_t) a.sign) << 15)
- | (0x1F << 10) | mantissa));
-} else {
-return float16_default_nan(status);
-}
-}
-
 /*
 | Returns 1 if the single-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 43962dc3f5..a6860e858d 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -211,10 +211,10 @@ float128 uint64_to_float128(uint64_t, float_status 
*status);
 /*
 | Software half-precision conversion routines.
 **/
-float16 float32_to_float16(float32, flag, float_status *status);
-float32 float16_to_float32(float16, flag, float_status *status);
-float16 float64_to_float16(float64 a, flag ieee, float_status *status);
-float64 float16_to_float64(float16 a, flag ieee, float_status *status);
+float16 float32_to_float16(float32, bool ieee, float_status *status);
+float32 float16_to_float32(float16, bool ieee, float_status *status);
+float16 float64_to_float16(float64 a, bool ieee, float_status *status);
+float64 float16_to_float64(float16 a, bool ieee, float_status *status);
 int16_t float16_to_int16(float16, float_status *status);
 uint16_t float16_to_uint16(float16 a, float_status *status);
 int16_t float16_to_int16_round_to_zero(float16, float_status *status);
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 64e1ad4f98..55e6701f26 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -113,15 +113,6 @@ static inline int extractFloat16Exp(float16 a)
 return (float16_val(a) >> 10) & 0x1f;
 }
 
-/*
-| Returns the sign bit of the single-precision floating-point value `a'.
-**/
-
-static inline flag extractFloat16Sign(float16 a)
-{
-return float16_val(a)>>15;
-}
-
 /*
 | Returns the fraction bits of the single-precision floating-point value `a'.
 **/
@@ -254,6 +245,11 @@ static const FloatFmt float16_params = {
 FLOAT_PARAMS(5, 10)
 };
 
+static const FloatFmt float16_params_ahp = {
+FLOAT_PARAMS(5, 10),
+.arm_althp = true
+};
+
 static const FloatFmt float32_params = {
 FLOAT_PARAMS(8, 23)
 };
@@ -497,14 +493,27 @@ static FloatParts round_canonical(FloatParts p, 
float_status *s,
 return p;
 }
 
+/* Explicit 

[Qemu-devel] [PATCH v6 17/28] target/mips: Remove floatX_maybe_silence_nan from conversions

[Richard Henderson]

This is now handled properly by the generic softfloat code.

Cc: Aurelien Jarno 
Cc: Yongbok Kim 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/mips/msa_helper.c | 4 
 target/mips/op_helper.c  | 2 --
 2 files changed, 6 deletions(-)

diff --git a/target/mips/msa_helper.c b/target/mips/msa_helper.c
index 8fb7a369ca..c74e3cdc65 100644
--- a/target/mips/msa_helper.c
+++ b/target/mips/msa_helper.c
@@ -1615,7 +1615,6 @@ static inline float16 float16_from_float32(int32_t a, 
flag ieee,
   float16 f_val;
 
   f_val = float32_to_float16((float32)a, ieee, status);
-  f_val = float16_maybe_silence_nan(f_val, status);
 
   return a < 0 ? (f_val | (1 << 15)) : f_val;
 }
@@ -1625,7 +1624,6 @@ static inline float32 float32_from_float64(int64_t a, 
float_status *status)
   float32 f_val;
 
   f_val = float64_to_float32((float64)a, status);
-  f_val = float32_maybe_silence_nan(f_val, status);
 
   return a < 0 ? (f_val | (1 << 31)) : f_val;
 }
@@ -1636,7 +1634,6 @@ static inline float32 float32_from_float16(int16_t a, 
flag ieee,
   float32 f_val;
 
   f_val = float16_to_float32((float16)a, ieee, status);
-  f_val = float32_maybe_silence_nan(f_val, status);
 
   return a < 0 ? (f_val | (1 << 31)) : f_val;
 }
@@ -1646,7 +1643,6 @@ static inline float64 float64_from_float32(int32_t a, 
float_status *status)
   float64 f_val;
 
   f_val = float32_to_float64((float64)a, status);
-  f_val = float64_maybe_silence_nan(f_val, status);
 
   return a < 0 ? (f_val | (1ULL << 63)) : f_val;
 }
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index 798cdad030..9025f42366 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -2700,7 +2700,6 @@ uint64_t helper_float_cvtd_s(CPUMIPSState *env, uint32_t 
fst0)
 uint64_t fdt2;
 
 fdt2 = float32_to_float64(fst0, >active_fpu.fp_status);
-fdt2 = float64_maybe_silence_nan(fdt2, >active_fpu.fp_status);
 update_fcr31(env, GETPC());
 return fdt2;
 }
@@ -2790,7 +2789,6 @@ uint32_t helper_float_cvts_d(CPUMIPSState *env, uint64_t 
fdt0)
 uint32_t fst2;
 
 fst2 = float64_to_float32(fdt0, >active_fpu.fp_status);
-fst2 = float32_maybe_silence_nan(fst2, >active_fpu.fp_status);
 update_fcr31(env, GETPC());
 return fst2;
 }
-- 
2.17.0

[Qemu-devel] [PATCH v6 13/28] target/arm: Use floatX_silence_nan when we have already checked for SNaN

[Richard Henderson]

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-a64.c |  6 +++---
 target/arm/helper.c | 12 ++--
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c
index 4f8034c513..6f0eb83661 100644
--- a/target/arm/helper-a64.c
+++ b/target/arm/helper-a64.c
@@ -376,7 +376,7 @@ float16 HELPER(frecpx_f16)(float16 a, void *fpstp)
 float16 nan = a;
 if (float16_is_signaling_nan(a, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float16_maybe_silence_nan(a, fpst);
+nan = float16_silence_nan(a, fpst);
 }
 if (fpst->default_nan_mode) {
 nan = float16_default_nan(fpst);
@@ -405,7 +405,7 @@ float32 HELPER(frecpx_f32)(float32 a, void *fpstp)
 float32 nan = a;
 if (float32_is_signaling_nan(a, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float32_maybe_silence_nan(a, fpst);
+nan = float32_silence_nan(a, fpst);
 }
 if (fpst->default_nan_mode) {
 nan = float32_default_nan(fpst);
@@ -434,7 +434,7 @@ float64 HELPER(frecpx_f64)(float64 a, void *fpstp)
 float64 nan = a;
 if (float64_is_signaling_nan(a, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float64_maybe_silence_nan(a, fpst);
+nan = float64_silence_nan(a, fpst);
 }
 if (fpst->default_nan_mode) {
 nan = float64_default_nan(fpst);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 238a3ceba8..e05c7230d4 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11731,7 +11731,7 @@ float16 HELPER(recpe_f16)(float16 input, void *fpstp)
 float16 nan = f16;
 if (float16_is_signaling_nan(f16, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float16_maybe_silence_nan(f16, fpst);
+nan = float16_silence_nan(f16, fpst);
 }
 if (fpst->default_nan_mode) {
 nan =  float16_default_nan(fpst);
@@ -11779,7 +11779,7 @@ float32 HELPER(recpe_f32)(float32 input, void *fpstp)
 float32 nan = f32;
 if (float32_is_signaling_nan(f32, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float32_maybe_silence_nan(f32, fpst);
+nan = float32_silence_nan(f32, fpst);
 }
 if (fpst->default_nan_mode) {
 nan =  float32_default_nan(fpst);
@@ -11827,7 +11827,7 @@ float64 HELPER(recpe_f64)(float64 input, void *fpstp)
 float64 nan = f64;
 if (float64_is_signaling_nan(f64, fpst)) {
 float_raise(float_flag_invalid, fpst);
-nan = float64_maybe_silence_nan(f64, fpst);
+nan = float64_silence_nan(f64, fpst);
 }
 if (fpst->default_nan_mode) {
 nan =  float64_default_nan(fpst);
@@ -11926,7 +11926,7 @@ float16 HELPER(rsqrte_f16)(float16 input, void *fpstp)
 float16 nan = f16;
 if (float16_is_signaling_nan(f16, s)) {
 float_raise(float_flag_invalid, s);
-nan = float16_maybe_silence_nan(f16, s);
+nan = float16_silence_nan(f16, s);
 }
 if (s->default_nan_mode) {
 nan =  float16_default_nan(s);
@@ -11970,7 +11970,7 @@ float32 HELPER(rsqrte_f32)(float32 input, void *fpstp)
 float32 nan = f32;
 if (float32_is_signaling_nan(f32, s)) {
 float_raise(float_flag_invalid, s);
-nan = float32_maybe_silence_nan(f32, s);
+nan = float32_silence_nan(f32, s);
 }
 if (s->default_nan_mode) {
 nan =  float32_default_nan(s);
@@ -12013,7 +12013,7 @@ float64 HELPER(rsqrte_f64)(float64 input, void *fpstp)
 float64 nan = f64;
 if (float64_is_signaling_nan(f64, s)) {
 float_raise(float_flag_invalid, s);
-nan = float64_maybe_silence_nan(f64, s);
+nan = float64_silence_nan(f64, s);
 }
 if (s->default_nan_mode) {
 nan =  float64_default_nan(s);
-- 
2.17.0

[Qemu-devel] [PATCH v6 16/28] target/m68k: Use floatX_silence_nan when we have already checked for SNaN

[Richard Henderson]

Reviewed-by: Peter Maydell 
Reviewed-by: Laurent Vivier 
Signed-off-by: Richard Henderson 
---
 target/m68k/softfloat.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/m68k/softfloat.c b/target/m68k/softfloat.c
index d093997219..b45a5e8690 100644
--- a/target/m68k/softfloat.c
+++ b/target/m68k/softfloat.c
@@ -31,13 +31,14 @@ static floatx80 propagateFloatx80NaNOneArg(floatx80 a, 
float_status *status)
 {
 if (floatx80_is_signaling_nan(a, status)) {
 float_raise(float_flag_invalid, status);
+a = floatx80_silence_nan(a, status);
 }
 
 if (status->default_nan_mode) {
 return floatx80_default_nan(status);
 }
 
-return floatx80_maybe_silence_nan(a, status);
+return a;
 }
 
 /*
-- 
2.17.0

[Qemu-devel] [PATCH v6 11/28] fpu/softfloat: Partial support for ARM Alternative half-precision

[Richard Henderson]

From: Alex Bennée 

For float16 ARM supports an alternative half-precision format which
sacrifices the ability to represent NaN/Inf in return for a higher
dynamic range.  The new FloatFmt flag, arm_althp, is then used to
modify the behaviour of canonicalize and round_canonical with respect
to representation and exception raising.

Usage of this new flag waits until we re-factor float-to-float conversions.

Reviewed-by: Peter Maydell 
Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 

---
v3
  - squash NaN to 0 if destination is AHP F16
v4
  - handle inf -> ahp max in float_to_float not round_canonical
  - assert no nan and inf for ahp in round_canonical
  - check ahp before snan in float_to_float
v5
  - split out canonicalize and round_canonical changes from the rest
---
 fpu/softfloat.c | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 41253c6749..64e1ad4f98 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -220,8 +220,10 @@ typedef struct {
  *   frac_shift: shift to normalise the fraction with DECOMPOSED_BINARY_POINT
  * The following are computed based the size of fraction
  *   frac_lsb: least significant bit of fraction
- *   fram_lsbm1: the bit bellow the least significant bit (for rounding)
+ *   frac_lsbm1: the bit below the least significant bit (for rounding)
  *   round_mask/roundeven_mask: masks used for rounding
+ * The following optional modifiers are available:
+ *   arm_althp: handle ARM Alternative Half Precision
  */
 typedef struct {
 int exp_size;
@@ -233,6 +235,7 @@ typedef struct {
 uint64_t frac_lsbm1;
 uint64_t round_mask;
 uint64_t roundeven_mask;
+bool arm_althp;
 } FloatFmt;
 
 /* Expand fields based on the size of exponent and fraction */
@@ -324,7 +327,7 @@ static inline float64 float64_pack_raw(FloatParts p)
 static FloatParts canonicalize(FloatParts part, const FloatFmt *parm,
float_status *status)
 {
-if (part.exp == parm->exp_max) {
+if (part.exp == parm->exp_max && !parm->arm_althp) {
 if (part.frac == 0) {
 part.cls = float_class_inf;
 } else {
@@ -413,7 +416,15 @@ static FloatParts round_canonical(FloatParts p, 
float_status *s,
 }
 frac >>= frac_shift;
 
-if (unlikely(exp >= exp_max)) {
+if (parm->arm_althp) {
+/* ARM Alt HP eschews Inf and NaN for a wider exponent.  */
+if (unlikely(exp > exp_max)) {
+/* Overflow.  Return the maximum normal.  */
+flags = float_flag_invalid;
+exp = exp_max;
+frac = -1;
+}
+} else if (unlikely(exp >= exp_max)) {
 flags |= float_flag_overflow | float_flag_inexact;
 if (overflow_norm) {
 exp = exp_max - 1;
@@ -464,12 +475,14 @@ static FloatParts round_canonical(FloatParts p, 
float_status *s,
 
 case float_class_inf:
 do_inf:
+assert(!parm->arm_althp);
 exp = exp_max;
 frac = 0;
 break;
 
 case float_class_qnan:
 case float_class_snan:
+assert(!parm->arm_althp);
 exp = exp_max;
 frac >>= parm->frac_shift;
 break;
-- 
2.17.0

[Qemu-devel] [PATCH v6 22/28] fpu/softfloat: Specialize on snan_bit_is_one

[Richard Henderson]

Only MIPS requires snan_bit_is_one to be variable.  While we are
specializing softfloat behaviour, allow other targets to eliminate
this runtime check.

Cc: Aurelien Jarno 
Cc: Yongbok Kim 
Cc: David Gibson 
Cc: Alexander Graf 
Cc: Guan Xuetao 
Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 

---
v5
  - do not remove the set_snan_bit_is_one function
  - tidy the language in the snan_bit_is_one block comment
---
 fpu/softfloat-specialize.h| 68 ++-
 include/fpu/softfloat-types.h |  1 +
 target/hppa/cpu.c |  1 -
 target/ppc/fpu_helper.c   |  1 -
 target/sh4/cpu.c  |  1 -
 target/unicore32/cpu.c|  2 --
 6 files changed, 44 insertions(+), 30 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index d7033b7757..d1e06da75b 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -79,13 +79,31 @@ this code that are retained.
  * version 2 or later. See the COPYING file in the top-level directory.
  */
 
-#if defined(TARGET_XTENSA)
 /* Define for architectures which deviate from IEEE in not supporting
  * signaling NaNs (so all NaNs are treated as quiet).
  */
+#if defined(TARGET_XTENSA)
 #define NO_SIGNALING_NANS 1
 #endif
 
+/* Define how the architecture discriminates signaling NaNs.
+ * This done with the most significant bit of the fraction.
+ * In IEEE 754-1985 this was implementation defined, but in IEEE 754-2008
+ * the msb must be zero.  MIPS is (so far) unique in supporting both the
+ * 2008 revision and backward compatibility with their original choice.
+ * Thus for MIPS we must make the choice at runtime.
+ */
+static inline flag snan_bit_is_one(float_status *status)
+{
+#if defined(TARGET_MIPS)
+return status->snan_bit_is_one;
+#elif defined(TARGET_HPPA) || defined(TARGET_UNICORE32) || defined(TARGET_SH4)
+return 1;
+#else
+return 0;
+#endif
+}
+
 /*
 | For the deconstructed floating-point with fraction FRAC, return true
 | if the fraction represents a signalling NaN; otherwise false.
@@ -97,7 +115,7 @@ static bool parts_is_snan_frac(uint64_t frac, float_status 
*status)
 return false;
 #else
 flag msb = extract64(frac, DECOMPOSED_BINARY_POINT - 1, 1);
-return msb == status->snan_bit_is_one;
+return msb == snan_bit_is_one(status);
 #endif
 }
 
@@ -118,7 +136,7 @@ static FloatParts parts_default_nan(float_status *status)
 #elif defined(TARGET_HPPA)
 frac = 1ULL << (DECOMPOSED_BINARY_POINT - 2);
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 frac = (1ULL << (DECOMPOSED_BINARY_POINT - 1)) - 1;
 } else {
 #if defined(TARGET_MIPS)
@@ -151,7 +169,7 @@ static FloatParts parts_silence_nan(FloatParts a, 
float_status *status)
 a.frac &= ~(1ULL << (DECOMPOSED_BINARY_POINT - 1));
 a.frac |= 1ULL << (DECOMPOSED_BINARY_POINT - 2);
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 return parts_default_nan(status);
 } else {
 a.frac |= 1ULL << (DECOMPOSED_BINARY_POINT - 1);
@@ -169,7 +187,7 @@ float16 float16_default_nan(float_status *status)
 #if defined(TARGET_ARM)
 return const_float16(0x7E00);
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 return const_float16(0x7DFF);
 } else {
 #if defined(TARGET_MIPS)
@@ -195,7 +213,7 @@ float32 float32_default_nan(float_status *status)
 #elif defined(TARGET_HPPA)
 return const_float32(0x7FA0);
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 return const_float32(0x7FBF);
 } else {
 #if defined(TARGET_MIPS)
@@ -220,7 +238,7 @@ float64 float64_default_nan(float_status *status)
 #elif defined(TARGET_HPPA)
 return const_float64(LIT64(0x7FF4));
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 return const_float64(LIT64(0x7FF7));
 } else {
 #if defined(TARGET_MIPS)
@@ -242,7 +260,7 @@ floatx80 floatx80_default_nan(float_status *status)
 r.low = LIT64(0x);
 r.high = 0x7FFF;
 #else
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 r.low = LIT64(0xBFFF);
 r.high = 0x7FFF;
 } else {
@@ -274,7 +292,7 @@ float128 float128_default_nan(float_status *status)
 {
 float128 r;
 
-if (status->snan_bit_is_one) {
+if (snan_bit_is_one(status)) {
 r.low = LIT64(0x);
 r.high = LIT64(0x7FFF7FFF);
 } else {
@@ -319,7 +337,7 @@ int float16_is_quiet_nan(float16 a_, float_status *status)
 return 

[Qemu-devel] [PATCH v6 28/28] fpu/softfloat: Define floatN_silence_nan in terms of parts_silence_nan

[Richard Henderson]

Isolate the target-specific choice to 3 functions instead of 6.

The code in floatx80_default_nan tried to be over-general.  There are
only two targets that support this format: x86 and m68k.  Thus there
is no point in inventing a mechanism for snan_bit_is_one.

Move routines that no longer have ifdefs out of softfloat-specialize.h.

Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 81 ++
 fpu/softfloat.c| 31 +++
 2 files changed, 35 insertions(+), 77 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index ec4fb6ba8b..16c0bcb6fa 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -278,24 +278,6 @@ int float16_is_signaling_nan(float16 a_, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN from a signalling NaN for the half-precision
-| floating point value `a'.
-**/
-
-float16 float16_silence_nan(float16 a, float_status *status)
-{
-#ifdef NO_SIGNALING_NANS
-g_assert_not_reached();
-#else
-if (snan_bit_is_one(status)) {
-return float16_default_nan(status);
-} else {
-return a | (1 << 9);
-}
-#endif
-}
-
 /*
 | Returns 1 if the single-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
@@ -334,30 +316,6 @@ int float32_is_signaling_nan(float32 a_, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN from a signalling NaN for the single-precision
-| floating point value `a'.
-**/
-
-float32 float32_silence_nan(float32 a, float_status *status)
-{
-#ifdef NO_SIGNALING_NANS
-g_assert_not_reached();
-#else
-if (snan_bit_is_one(status)) {
-# ifdef TARGET_HPPA
-a &= ~0x0040;
-a |=  0x0020;
-return a;
-# else
-return float32_default_nan(status);
-# endif
-} else {
-return a | (1 << 22);
-}
-#endif
-}
-
 /*
 | Returns the result of converting the single-precision floating-point NaN
 | `a' to the canonical NaN format.  If `a' is a signaling NaN, the invalid
@@ -706,31 +664,6 @@ int float64_is_signaling_nan(float64 a_, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN from a signalling NaN for the double-precision
-| floating point value `a'.
-**/
-
-float64 float64_silence_nan(float64 a, float_status *status)
-{
-#ifdef NO_SIGNALING_NANS
-g_assert_not_reached();
-#else
-if (snan_bit_is_one(status)) {
-# ifdef TARGET_HPPA
-a &= ~0x0008ULL;
-a |=  0x0004ULL;
-return a;
-# else
-return float64_default_nan(status);
-# endif
-} else {
-return a | LIT64(0x0008);
-}
-#endif
-}
-
-
 /*
 | Returns the result of converting the double-precision floating-point NaN
 | `a' to the canonical NaN format.  If `a' is a signaling NaN, the invalid
@@ -886,16 +819,10 @@ int floatx80_is_signaling_nan(floatx80 a, float_status 
*status)
 
 floatx80 floatx80_silence_nan(floatx80 a, float_status *status)
 {
-#ifdef NO_SIGNALING_NANS
-g_assert_not_reached();
-#else
-if (snan_bit_is_one(status)) {
-return floatx80_default_nan(status);
-} else {
-a.low |= LIT64(0xC000);
-return a;
-}
-#endif
+/* None of the targets that have snan_bit_is_one use floatx80.  */
+assert(!snan_bit_is_one(status));
+a.low |= LIT64(0xC000);
+return a;
 }
 
 /*
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index c8b33e35f4..8cd2400081 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -2134,6 +2134,37 @@ float128 float128_default_nan(float_status *status)
 return r;
 }
 
+/*
+| Returns a quiet NaN from a signalling NaN for the floating point value `a'.
+**/
+
+float16 float16_silence_nan(float16 a, float_status *status)
+{
+FloatParts p = float16_unpack_raw(a);
+p.frac <<= float16_params.frac_shift;
+p = parts_silence_nan(p, status);
+p.frac >>= float16_params.frac_shift;
+return float16_pack_raw(p);
+}
+
+float32 float32_silence_nan(float32 a, float_status *status)
+{

[Qemu-devel] [PATCH v6 15/28] target/hppa: Remove floatX_maybe_silence_nan from conversions

[Richard Henderson]

This is now handled properly by the generic softfloat code.

Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/hppa/op_helper.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/target/hppa/op_helper.c b/target/hppa/op_helper.c
index a3af62daf7..912e8d5be4 100644
--- a/target/hppa/op_helper.c
+++ b/target/hppa/op_helper.c
@@ -341,7 +341,6 @@ float64 HELPER(fdiv_d)(CPUHPPAState *env, float64 a, 
float64 b)
 float64 HELPER(fcnv_s_d)(CPUHPPAState *env, float32 arg)
 {
 float64 ret = float32_to_float64(arg, >fp_status);
-ret = float64_maybe_silence_nan(ret, >fp_status);
 update_fr0_op(env, GETPC());
 return ret;
 }
@@ -349,7 +348,6 @@ float64 HELPER(fcnv_s_d)(CPUHPPAState *env, float32 arg)
 float32 HELPER(fcnv_d_s)(CPUHPPAState *env, float64 arg)
 {
 float32 ret = float64_to_float32(arg, >fp_status);
-ret = float32_maybe_silence_nan(ret, >fp_status);
 update_fr0_op(env, GETPC());
 return ret;
 }
-- 
2.17.0

[Qemu-devel] [PATCH v6 21/28] fpu/softfloat: Remove floatX_maybe_silence_nan

[Richard Henderson]

These functions are now unused.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 63 --
 include/fpu/softfloat.h|  5 ---
 2 files changed, 68 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 4fa068a5dc..d7033b7757 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -364,19 +364,6 @@ float16 float16_silence_nan(float16 a, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN if the half-precision floating point value `a' is a
-| signaling NaN; otherwise returns `a'.
-**/
-
-float16 float16_maybe_silence_nan(float16 a, float_status *status)
-{
-if (float16_is_signaling_nan(a, status)) {
-return float16_silence_nan(a, status);
-}
-return a;
-}
-
 /*
 | Returns 1 if the single-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
@@ -438,18 +425,6 @@ float32 float32_silence_nan(float32 a, float_status 
*status)
 }
 #endif
 }
-/*
-| Returns a quiet NaN if the single-precision floating point value `a' is a
-| signaling NaN; otherwise returns `a'.
-**/
-
-float32 float32_maybe_silence_nan(float32 a, float_status *status)
-{
-if (float32_is_signaling_nan(a, status)) {
-return float32_silence_nan(a, status);
-}
-return a;
-}
 
 /*
 | Returns the result of converting the single-precision floating-point NaN
@@ -864,18 +839,6 @@ float64 float64_silence_nan(float64 a, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN if the double-precision floating point value `a' is a
-| signaling NaN; otherwise returns `a'.
-**/
-
-float64 float64_maybe_silence_nan(float64 a, float_status *status)
-{
-if (float64_is_signaling_nan(a, status)) {
-return float64_silence_nan(a, status);
-}
-return a;
-}
 
 /*
 | Returns the result of converting the double-precision floating-point NaN
@@ -1037,19 +1000,6 @@ floatx80 floatx80_silence_nan(floatx80 a, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN if the extended double-precision floating point value
-| `a' is a signaling NaN; otherwise returns `a'.
-**/
-
-floatx80 floatx80_maybe_silence_nan(floatx80 a, float_status *status)
-{
-if (floatx80_is_signaling_nan(a, status)) {
-return floatx80_silence_nan(a, status);
-}
-return a;
-}
-
 /*
 | Returns the result of converting the extended double-precision floating-
 | point NaN `a' to the canonical NaN format.  If `a' is a signaling NaN, the
@@ -1204,19 +1154,6 @@ float128 float128_silence_nan(float128 a, float_status 
*status)
 #endif
 }
 
-/*
-| Returns a quiet NaN if the quadruple-precision floating point value `a' is
-| a signaling NaN; otherwise returns `a'.
-**/
-
-float128 float128_maybe_silence_nan(float128 a, float_status *status)
-{
-if (float128_is_signaling_nan(a, status)) {
-return float128_silence_nan(a, status);
-}
-return a;
-}
-
 /*
 | Returns the result of converting the quadruple-precision floating-point NaN
 | `a' to the canonical NaN format.  If `a' is a signaling NaN, the invalid
diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index a6860e858d..69f4dbc4db 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -258,7 +258,6 @@ int float16_compare_quiet(float16, float16, float_status 
*status);
 int float16_is_quiet_nan(float16, float_status *status);
 int float16_is_signaling_nan(float16, float_status *status);
 float16 float16_silence_nan(float16, float_status *status);
-float16 float16_maybe_silence_nan(float16, float_status *status);
 
 static inline int float16_is_any_nan(float16 a)
 {
@@ -370,7 +369,6 @@ float32 

[Qemu-devel] [PATCH v6 03/28] fpu/softfloat: Split floatXX_silence_nan from floatXX_maybe_silence_nan

[Richard Henderson]

The new function assumes that the input is an SNaN and
does not double-check.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 174 +
 include/fpu/softfloat.h|   5 ++
 2 files changed, 123 insertions(+), 56 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 58b05718c8..4fc9ea4ac0 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -271,22 +271,35 @@ int float16_is_signaling_nan(float16 a_, float_status 
*status)
 #endif
 }
 
+/*
+| Returns a quiet NaN from a signalling NaN for the half-precision
+| floating point value `a'.
+**/
+
+float16 float16_silence_nan(float16 a, float_status *status)
+{
+#ifdef NO_SIGNALING_NANS
+g_assert_not_reached();
+#else
+if (status->snan_bit_is_one) {
+return float16_default_nan(status);
+} else {
+return a | (1 << 9);
+}
+#endif
+}
+
 /*
 | Returns a quiet NaN if the half-precision floating point value `a' is a
 | signaling NaN; otherwise returns `a'.
 **/
-float16 float16_maybe_silence_nan(float16 a_, float_status *status)
+
+float16 float16_maybe_silence_nan(float16 a, float_status *status)
 {
-if (float16_is_signaling_nan(a_, status)) {
-if (status->snan_bit_is_one) {
-return float16_default_nan(status);
-} else {
-uint16_t a = float16_val(a_);
-a |= (1 << 9);
-return make_float16(a);
-}
+if (float16_is_signaling_nan(a, status)) {
+return float16_silence_nan(a, status);
 }
-return a_;
+return a;
 }
 
 /*
@@ -367,30 +380,40 @@ int float32_is_signaling_nan(float32 a_, float_status 
*status)
 #endif
 }
 
+/*
+| Returns a quiet NaN from a signalling NaN for the single-precision
+| floating point value `a'.
+**/
+
+float32 float32_silence_nan(float32 a, float_status *status)
+{
+#ifdef NO_SIGNALING_NANS
+g_assert_not_reached();
+#else
+if (status->snan_bit_is_one) {
+# ifdef TARGET_HPPA
+a &= ~0x0040;
+a |=  0x0020;
+return a;
+# else
+return float32_default_nan(status);
+# endif
+} else {
+return a | (1 << 22);
+}
+#endif
+}
 /*
 | Returns a quiet NaN if the single-precision floating point value `a' is a
 | signaling NaN; otherwise returns `a'.
 **/
 
-float32 float32_maybe_silence_nan(float32 a_, float_status *status)
+float32 float32_maybe_silence_nan(float32 a, float_status *status)
 {
-if (float32_is_signaling_nan(a_, status)) {
-if (status->snan_bit_is_one) {
-#ifdef TARGET_HPPA
-uint32_t a = float32_val(a_);
-a &= ~0x0040;
-a |=  0x0020;
-return make_float32(a);
-#else
-return float32_default_nan(status);
-#endif
-} else {
-uint32_t a = float32_val(a_);
-a |= (1 << 22);
-return make_float32(a);
-}
+if (float32_is_signaling_nan(a, status)) {
+return float32_silence_nan(a, status);
 }
-return a_;
+return a;
 }
 
 /*
@@ -776,30 +799,41 @@ int float64_is_signaling_nan(float64 a_, float_status 
*status)
 #endif
 }
 
+/*
+| Returns a quiet NaN from a signalling NaN for the double-precision
+| floating point value `a'.
+**/
+
+float64 float64_silence_nan(float64 a, float_status *status)
+{
+#ifdef NO_SIGNALING_NANS
+g_assert_not_reached();
+#else
+if (status->snan_bit_is_one) {
+# ifdef TARGET_HPPA
+a &= ~0x0008ULL;
+a |=  0x0004ULL;
+return a;
+# else
+return float64_default_nan(status);
+# endif
+} else {
+return a | LIT64(0x0008);
+}
+#endif
+}
+
 /*
 | Returns a quiet NaN if the double-precision floating point value `a' is a
 | signaling NaN; otherwise returns `a'.
 

[Qemu-devel] [PATCH v6 27/28] fpu/softfloat: Clean up parts_default_nan

[Richard Henderson]

Reduce the number of ifdefs.  Correct the result for OpenRISC
and TriCore (although TriCore fixed in target-specific code).

Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 21 ++---
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 9d562ed504..ec4fb6ba8b 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -129,22 +129,29 @@ static FloatParts parts_default_nan(float_status *status)
 uint64_t frac;
 
 #if defined(TARGET_SPARC) || defined(TARGET_M68K)
+/* !snan_bit_is_one, set all bits */
 frac = (1ULL << DECOMPOSED_BINARY_POINT) - 1;
-#elif defined(TARGET_PPC) || defined(TARGET_ARM) || defined(TARGET_ALPHA) || \
-  defined(TARGET_S390X) || defined(TARGET_RISCV)
+#elif defined(TARGET_I386) || defined(TARGET_X86_64) \
+|| defined(TARGET_MICROBLAZE)
+/* !snan_bit_is_one, set sign and msb */
 frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
+sign = 1;
 #elif defined(TARGET_HPPA)
+/* snan_bit_is_one, set msb-1.  */
 frac = 1ULL << (DECOMPOSED_BINARY_POINT - 2);
 #else
+/* This case is true for Alpha, ARM, MIPS, OpenRISC, PPC, RISC-V,
+ * S390, SH4, TriCore, and Xtensa.  I cannot find documentation
+ * for Unicore32; the choice from the original commit is unchanged.
+ * Our other supported targets, CRIS, LM32, Moxie, Nios2, and Tile,
+ * do not have floating-point.
+ */
 if (snan_bit_is_one(status)) {
+/* set all bits other than msb */
 frac = (1ULL << (DECOMPOSED_BINARY_POINT - 1)) - 1;
 } else {
-#if defined(TARGET_MIPS)
+/* set msb */
 frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
-#else
-frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
-sign = 1;
-#endif
 }
 #endif
 
-- 
2.17.0

[Qemu-devel] [PATCH v6 14/28] target/arm: Remove floatX_maybe_silence_nan from conversions

[Richard Henderson]

This is now handled properly by the generic softfloat code.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-a64.c |  1 -
 target/arm/helper.c | 12 ++--
 2 files changed, 2 insertions(+), 11 deletions(-)

diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c
index 6f0eb83661..f92bdea732 100644
--- a/target/arm/helper-a64.c
+++ b/target/arm/helper-a64.c
@@ -466,7 +466,6 @@ float32 HELPER(fcvtx_f64_to_f32)(float64 a, CPUARMState 
*env)
 set_float_rounding_mode(float_round_to_zero, );
 set_float_exception_flags(0, );
 r = float64_to_float32(a, );
-r = float32_maybe_silence_nan(r, );
 exflags = get_float_exception_flags();
 if (exflags & float_flag_inexact) {
 r = make_float32(float32_val(r) | 1);
diff --git a/target/arm/helper.c b/target/arm/helper.c
index e05c7230d4..db8bbe52a6 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11348,20 +11348,12 @@ FLOAT_CONVS(ui, d, 64, u)
 /* floating point conversion */
 float64 VFP_HELPER(fcvtd, s)(float32 x, CPUARMState *env)
 {
-float64 r = float32_to_float64(x, >vfp.fp_status);
-/* ARM requires that S<->D conversion of any kind of NaN generates
- * a quiet NaN by forcing the most significant frac bit to 1.
- */
-return float64_maybe_silence_nan(r, >vfp.fp_status);
+return float32_to_float64(x, >vfp.fp_status);
 }
 
 float32 VFP_HELPER(fcvts, d)(float64 x, CPUARMState *env)
 {
-float32 r =  float64_to_float32(x, >vfp.fp_status);
-/* ARM requires that S<->D conversion of any kind of NaN generates
- * a quiet NaN by forcing the most significant frac bit to 1.
- */
-return float32_maybe_silence_nan(r, >vfp.fp_status);
+return float64_to_float32(x, >vfp.fp_status);
 }
 
 /* VFP3 fixed point conversion.  */
-- 
2.17.0

[Qemu-devel] [PATCH v6 20/28] fpu/softfloat: Use float*_silence_nan in propagateFloat*NaN

[Richard Henderson]

We have already checked the arguments for SNaN;
we don't need to do it again.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 44 +-
 1 file changed, 34 insertions(+), 10 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 995a0132c6..4fa068a5dc 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -498,7 +498,7 @@ static float32 commonNaNToFloat32(commonNaNT a, 
float_status *status)
 | The routine is passed various bits of information about the
 | two NaNs and should return 0 to select NaN a and 1 for NaN b.
 | Note that signalling NaNs are always squashed to quiet NaNs
-| by the caller, by calling floatXX_maybe_silence_nan() before
+| by the caller, by calling floatXX_silence_nan() before
 | returning them.
 |
 | aIsLargerSignificand is only valid if both a and b are NaNs
@@ -536,7 +536,7 @@ static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
 {
 /* According to MIPS specifications, if one of the two operands is
  * a sNaN, a new qNaN has to be generated. This is done in
- * floatXX_maybe_silence_nan(). For qNaN inputs the specifications
+ * floatXX_silence_nan(). For qNaN inputs the specifications
  * says: "When possible, this QNaN result is one of the operand QNaN
  * values." In practice it seems that most implementations choose
  * the first operand if both operands are qNaN. In short this gives
@@ -788,9 +788,15 @@ static float32 propagateFloat32NaN(float32 a, float32 b, 
float_status *status)
 
 if (pickNaN(aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
 aIsLargerSignificand)) {
-return float32_maybe_silence_nan(b, status);
+if (bIsSignalingNaN) {
+return float32_silence_nan(b, status);
+}
+return b;
 } else {
-return float32_maybe_silence_nan(a, status);
+if (aIsSignalingNaN) {
+return float32_silence_nan(a, status);
+}
+return a;
 }
 }
 
@@ -950,9 +956,15 @@ static float64 propagateFloat64NaN(float64 a, float64 b, 
float_status *status)
 
 if (pickNaN(aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
 aIsLargerSignificand)) {
-return float64_maybe_silence_nan(b, status);
+if (bIsSignalingNaN) {
+return float64_silence_nan(b, status);
+}
+return b;
 } else {
-return float64_maybe_silence_nan(a, status);
+if (aIsSignalingNaN) {
+return float64_silence_nan(a, status);
+}
+return a;
 }
 }
 
@@ -1121,9 +1133,15 @@ floatx80 propagateFloatx80NaN(floatx80 a, floatx80 b, 
float_status *status)
 
 if (pickNaN(aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
 aIsLargerSignificand)) {
-return floatx80_maybe_silence_nan(b, status);
+if (bIsSignalingNaN) {
+return floatx80_silence_nan(b, status);
+}
+return b;
 } else {
-return floatx80_maybe_silence_nan(a, status);
+if (aIsSignalingNaN) {
+return floatx80_silence_nan(a, status);
+}
+return a;
 }
 }
 
@@ -1270,8 +1288,14 @@ static float128 propagateFloat128NaN(float128 a, 
float128 b,
 
 if (pickNaN(aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
 aIsLargerSignificand)) {
-return float128_maybe_silence_nan(b, status);
+if (bIsSignalingNaN) {
+return float128_silence_nan(b, status);
+}
+return b;
 } else {
-return float128_maybe_silence_nan(a, status);
+if (aIsSignalingNaN) {
+return float128_silence_nan(a, status);
+}
+return a;
 }
 }
-- 
2.17.0

[Qemu-devel] [PATCH v6 05/28] fpu/softfloat: Canonicalize NaN fraction

[Richard Henderson]

Shift the NaN fraction to a canonical position, much like we
do for the fraction of normal numbers.  This will facilitate
manipulation of NaNs within the shared code paths.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 0d17027379..607c4a78d5 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -330,10 +330,11 @@ static FloatParts canonicalize(FloatParts part, const 
FloatFmt *parm,
 if (part.frac == 0) {
 part.cls = float_class_inf;
 } else {
+part.frac <<= parm->frac_shift;
 #ifdef NO_SIGNALING_NANS
 part.cls = float_class_qnan;
 #else
-int64_t msb = part.frac << (parm->frac_shift + 2);
+int64_t msb = part.frac << 2;
 if ((msb < 0) == status->snan_bit_is_one) {
 part.cls = float_class_snan;
 } else {
@@ -480,6 +481,7 @@ static FloatParts round_canonical(FloatParts p, 
float_status *s,
 case float_class_qnan:
 case float_class_snan:
 exp = exp_max;
+frac >>= parm->frac_shift;
 break;
 
 default:
@@ -503,6 +505,7 @@ static float16 float16_round_pack_canonical(FloatParts p, 
float_status *s)
 case float_class_dnan:
 return float16_default_nan(s);
 case float_class_msnan:
+p.frac >>= float16_params.frac_shift;
 return float16_maybe_silence_nan(float16_pack_raw(p), s);
 default:
 p = round_canonical(p, s, _params);
@@ -521,6 +524,7 @@ static float32 float32_round_pack_canonical(FloatParts p, 
float_status *s)
 case float_class_dnan:
 return float32_default_nan(s);
 case float_class_msnan:
+p.frac >>= float32_params.frac_shift;
 return float32_maybe_silence_nan(float32_pack_raw(p), s);
 default:
 p = round_canonical(p, s, _params);
@@ -539,6 +543,7 @@ static float64 float64_round_pack_canonical(FloatParts p, 
float_status *s)
 case float_class_dnan:
 return float64_default_nan(s);
 case float_class_msnan:
+p.frac >>= float64_params.frac_shift;
 return float64_maybe_silence_nan(float64_pack_raw(p), s);
 default:
 p = round_canonical(p, s, _params);
-- 
2.17.0

[Qemu-devel] [PATCH v6 26/28] fpu/softfloat: Define floatN_default_nan in terms of parts_default_nan

[Richard Henderson]

Isolate the target-specific choice to 2 functions instead of 6.

The code in float16_default_nan was only correct for ARM, MIPS, and X86.
Though float16 support is rare among our targets.

The code in float128_default_nan was arguably wrong for Sparc.  While
QEMU supports the Sparc 128-bit insns, no real cpu enables it.

The code in floatx80_default_nan tried to be over-general.  There are
only two targets that support this format: x86 and m68k.  Thus there
is no point in inventing a value for snan_bit_is_one.

Move routines that no longer have ifdefs out of softfloat-specialize.h.

Signed-off-by: Richard Henderson 

---
v6
  - shift the nan fraction into place before raw packing
---
 fpu/softfloat-specialize.h | 105 +++--
 fpu/softfloat.c|  41 +++
 2 files changed, 47 insertions(+), 99 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 0399dfe011..9d562ed504 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -179,94 +179,22 @@ static FloatParts parts_silence_nan(FloatParts a, 
float_status *status)
 return a;
 }
 
-/*
-| The pattern for a default generated half-precision NaN.
-**/
-float16 float16_default_nan(float_status *status)
-{
-#if defined(TARGET_ARM)
-return const_float16(0x7E00);
-#else
-if (snan_bit_is_one(status)) {
-return const_float16(0x7DFF);
-} else {
-#if defined(TARGET_MIPS)
-return const_float16(0x7E00);
-#else
-return const_float16(0xFE00);
-#endif
-}
-#endif
-}
-
-/*
-| The pattern for a default generated single-precision NaN.
-**/
-float32 float32_default_nan(float_status *status)
-{
-#if defined(TARGET_SPARC) || defined(TARGET_M68K)
-return const_float32(0x7FFF);
-#elif defined(TARGET_PPC) || defined(TARGET_ARM) || defined(TARGET_ALPHA) || \
-  defined(TARGET_XTENSA) || defined(TARGET_S390X) || \
-  defined(TARGET_TRICORE) || defined(TARGET_RISCV)
-return const_float32(0x7FC0);
-#elif defined(TARGET_HPPA)
-return const_float32(0x7FA0);
-#else
-if (snan_bit_is_one(status)) {
-return const_float32(0x7FBF);
-} else {
-#if defined(TARGET_MIPS)
-return const_float32(0x7FC0);
-#else
-return const_float32(0xFFC0);
-#endif
-}
-#endif
-}
-
-/*
-| The pattern for a default generated double-precision NaN.
-**/
-float64 float64_default_nan(float_status *status)
-{
-#if defined(TARGET_SPARC) || defined(TARGET_M68K)
-return const_float64(LIT64(0x7FFF));
-#elif defined(TARGET_PPC) || defined(TARGET_ARM) || defined(TARGET_ALPHA) || \
-  defined(TARGET_S390X) || defined(TARGET_RISCV)
-return const_float64(LIT64(0x7FF8));
-#elif defined(TARGET_HPPA)
-return const_float64(LIT64(0x7FF4));
-#else
-if (snan_bit_is_one(status)) {
-return const_float64(LIT64(0x7FF7));
-} else {
-#if defined(TARGET_MIPS)
-return const_float64(LIT64(0x7FF8));
-#else
-return const_float64(LIT64(0xFFF8));
-#endif
-}
-#endif
-}
-
 /*
 | The pattern for a default generated extended double-precision NaN.
 **/
 floatx80 floatx80_default_nan(float_status *status)
 {
 floatx80 r;
+
+/* None of the targets that have snan_bit_is_one use floatx80.  */
+assert(!snan_bit_is_one(status));
 #if defined(TARGET_M68K)
 r.low = LIT64(0x);
 r.high = 0x7FFF;
 #else
-if (snan_bit_is_one(status)) {
-r.low = LIT64(0xBFFF);
-r.high = 0x7FFF;
-} else {
-r.low = LIT64(0xC000);
-r.high = 0x;
-}
+/* X86 */
+r.low = LIT64(0xC000);
+r.high = 0x;
 #endif
 return r;
 }
@@ -285,27 +213,6 @@ floatx80 floatx80_default_nan(float_status *status)
 const floatx80 floatx80_infinity
 = make_floatx80_init(floatx80_infinity_high, floatx80_infinity_low);
 
-/*
-| The pattern for a default generated quadruple-precision NaN.
-**/
-float128 float128_default_nan(float_status *status)
-{
-float128 r;
-
-if (snan_bit_is_one(status)) {
-r.low = LIT64(0x);
-

[Qemu-devel] [PATCH v6 18/28] target/riscv: Remove floatX_maybe_silence_nan from conversions

[Richard Henderson]

This is now handled properly by the generic softfloat code.

Cc: Palmer Dabbelt 
Cc: Sagar Karandikar 
Cc: Bastian Koppelmann 
Reviewed-by: Michael Clark 
Signed-off-by: Richard Henderson 
---
 target/riscv/fpu_helper.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/target/riscv/fpu_helper.c b/target/riscv/fpu_helper.c
index abbadead5c..fdb87d8d82 100644
--- a/target/riscv/fpu_helper.c
+++ b/target/riscv/fpu_helper.c
@@ -279,14 +279,12 @@ uint64_t helper_fmax_d(CPURISCVState *env, uint64_t frs1, 
uint64_t frs2)
 
 uint64_t helper_fcvt_s_d(CPURISCVState *env, uint64_t rs1)
 {
-rs1 = float64_to_float32(rs1, >fp_status);
-return float32_maybe_silence_nan(rs1, >fp_status);
+return float64_to_float32(rs1, >fp_status);
 }
 
 uint64_t helper_fcvt_d_s(CPURISCVState *env, uint64_t rs1)
 {
-rs1 = float32_to_float64(rs1, >fp_status);
-return float64_maybe_silence_nan(rs1, >fp_status);
+return float32_to_float64(rs1, >fp_status);
 }
 
 uint64_t helper_fsqrt_d(CPURISCVState *env, uint64_t frs1)
-- 
2.17.0

[Qemu-devel] [PATCH v6 09/28] target/arm: convert conversion helpers to fpst/ahp_flag

[Richard Henderson]

From: Alex Bennée 

Instead of passing env and leaving it up to the helper to get the
right fpstatus we pass it explicitly. There was already a get_fpstatus
helper for neon for the 32 bit code. We also add an get_ahp_flag() for
passing the state of the alternative FP16 format flag. This leaves
scope for later tracking the AHP state in translation flags.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 

---
v4
  - remove neon_fcvt_*; they are now identical to vfp_fcvt_*.
  - add flags to vfp_fcvt_* helper decls.
  - add some missing tcg_temp_free_*.
v5
  - always use get_fpstatus_ptr(false) for FZ, since FZ16 is
supposed to be supressed.
v6
  - really always use get_fpstatus_ptr(false).
---
 target/arm/helper.h| 10 +++---
 target/arm/translate.h | 12 +++
 target/arm/helper.c| 56 +
 target/arm/translate-a64.c | 37 +++
 target/arm/translate.c | 74 +-
 5 files changed, 112 insertions(+), 77 deletions(-)

diff --git a/target/arm/helper.h b/target/arm/helper.h
index ce89968b2d..047f3bc1ca 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -187,12 +187,10 @@ DEF_HELPER_3(vfp_uqtoh, f16, i64, i32, ptr)
 DEF_HELPER_FLAGS_2(set_rmode, TCG_CALL_NO_RWG, i32, i32, ptr)
 DEF_HELPER_FLAGS_2(set_neon_rmode, TCG_CALL_NO_RWG, i32, i32, env)
 
-DEF_HELPER_2(vfp_fcvt_f16_to_f32, f32, i32, env)
-DEF_HELPER_2(vfp_fcvt_f32_to_f16, i32, f32, env)
-DEF_HELPER_2(neon_fcvt_f16_to_f32, f32, i32, env)
-DEF_HELPER_2(neon_fcvt_f32_to_f16, i32, f32, env)
-DEF_HELPER_FLAGS_2(vfp_fcvt_f16_to_f64, TCG_CALL_NO_RWG, f64, i32, env)
-DEF_HELPER_FLAGS_2(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, i32, f64, env)
+DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f32, TCG_CALL_NO_RWG, f32, f16, ptr, i32)
+DEF_HELPER_FLAGS_3(vfp_fcvt_f32_to_f16, TCG_CALL_NO_RWG, f16, f32, ptr, i32)
+DEF_HELPER_FLAGS_3(vfp_fcvt_f16_to_f64, TCG_CALL_NO_RWG, f64, f16, ptr, i32)
+DEF_HELPER_FLAGS_3(vfp_fcvt_f64_to_f16, TCG_CALL_NO_RWG, f16, f64, ptr, i32)
 
 DEF_HELPER_4(vfp_muladdd, f64, f64, f64, f64, ptr)
 DEF_HELPER_4(vfp_muladds, f32, f32, f32, f32, ptr)
diff --git a/target/arm/translate.h b/target/arm/translate.h
index 37a1bba056..45f04244be 100644
--- a/target/arm/translate.h
+++ b/target/arm/translate.h
@@ -177,4 +177,16 @@ void arm_free_cc(DisasCompare *cmp);
 void arm_jump_cc(DisasCompare *cmp, TCGLabel *label);
 void arm_gen_test_cc(int cc, TCGLabel *label);
 
+/* Return state of Alternate Half-precision flag, caller frees result */
+static inline TCGv_i32 get_ahp_flag(void)
+{
+TCGv_i32 ret = tcg_temp_new_i32();
+
+tcg_gen_ld_i32(ret, cpu_env,
+   offsetof(CPUARMState, vfp.xregs[ARM_VFP_FPSCR]));
+tcg_gen_extract_i32(ret, ret, 26, 1);
+
+return ret;
+}
+
 #endif /* TARGET_ARM_TRANSLATE_H */
diff --git a/target/arm/helper.c b/target/arm/helper.c
index c6fd7f9479..1762042fc7 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11540,64 +11540,24 @@ uint32_t HELPER(set_neon_rmode)(uint32_t rmode, 
CPUARMState *env)
 }
 
 /* Half precision conversions.  */
-static float32 do_fcvt_f16_to_f32(uint32_t a, CPUARMState *env, float_status 
*s)
+float32 HELPER(vfp_fcvt_f16_to_f32)(float16 a, void *fpstp, uint32_t ahp_mode)
 {
-int ieee = (env->vfp.xregs[ARM_VFP_FPSCR] & (1 << 26)) == 0;
-float32 r = float16_to_float32(make_float16(a), ieee, s);
-if (ieee) {
-return float32_maybe_silence_nan(r, s);
-}
-return r;
+return float16_to_float32(a, !ahp_mode, fpstp);
 }
 
-static uint32_t do_fcvt_f32_to_f16(float32 a, CPUARMState *env, float_status 
*s)
+float16 HELPER(vfp_fcvt_f32_to_f16)(float32 a, void *fpstp, uint32_t ahp_mode)
 {
-int ieee = (env->vfp.xregs[ARM_VFP_FPSCR] & (1 << 26)) == 0;
-float16 r = float32_to_float16(a, ieee, s);
-if (ieee) {
-r = float16_maybe_silence_nan(r, s);
-}
-return float16_val(r);
+return float32_to_float16(a, !ahp_mode, fpstp);
 }
 
-float32 HELPER(neon_fcvt_f16_to_f32)(uint32_t a, CPUARMState *env)
+float64 HELPER(vfp_fcvt_f16_to_f64)(float16 a, void *fpstp, uint32_t ahp_mode)
 {
-return do_fcvt_f16_to_f32(a, env, >vfp.standard_fp_status);
+return float16_to_float64(a, !ahp_mode, fpstp);
 }
 
-uint32_t HELPER(neon_fcvt_f32_to_f16)(float32 a, CPUARMState *env)
+float16 HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode)
 {
-return do_fcvt_f32_to_f16(a, env, >vfp.standard_fp_status);
-}
-
-float32 HELPER(vfp_fcvt_f16_to_f32)(uint32_t a, CPUARMState *env)
-{
-return do_fcvt_f16_to_f32(a, env, >vfp.fp_status);
-}
-
-uint32_t HELPER(vfp_fcvt_f32_to_f16)(float32 a, CPUARMState *env)
-{
-return do_fcvt_f32_to_f16(a, env, >vfp.fp_status);
-}
-
-float64 HELPER(vfp_fcvt_f16_to_f64)(uint32_t a, CPUARMState *env)
-{
-int ieee = (env->vfp.xregs[ARM_VFP_FPSCR] & (1 << 26)) == 0;
-float64 r = 

[Qemu-devel] [PATCH v6 04/28] fpu/softfloat: Move softfloat-specialize.h below FloatParts definition

[Richard Henderson]

We want to be able to specialize on the canonical representation.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat.c | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index d07419324a..0d17027379 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -95,16 +95,6 @@ this code that are retained.
 **/
 #include "fpu/softfloat-macros.h"
 
-/*
-| Functions and definitions to determine:  (1) whether tininess for underflow
-| is detected before or after rounding by default, (2) what (if anything)
-| happens when exceptions are raised, (3) how signaling NaNs are distinguished
-| from quiet NaNs, (4) the default generated quiet NaNs, and (5) how NaNs
-| are propagated from function inputs to output.  These details are target-
-| specific.
-**/
-#include "softfloat-specialize.h"
-
 /*
 | Returns the fraction bits of the half-precision floating-point value `a'.
 **/
@@ -322,6 +312,16 @@ static inline float64 float64_pack_raw(FloatParts p)
 return make_float64(pack_raw(float64_params, p));
 }
 
+/*
+| Functions and definitions to determine:  (1) whether tininess for underflow
+| is detected before or after rounding by default, (2) what (if anything)
+| happens when exceptions are raised, (3) how signaling NaNs are distinguished
+| from quiet NaNs, (4) the default generated quiet NaNs, and (5) how NaNs
+| are propagated from function inputs to output.  These details are target-
+| specific.
+**/
+#include "softfloat-specialize.h"
+
 /* Canonicalize EXP and FRAC, setting CLS.  */
 static FloatParts canonicalize(FloatParts part, const FloatFmt *parm,
float_status *status)
-- 
2.17.0

[Qemu-devel] [PATCH v6 25/28] fpu/softfloat: Pass FloatClass to pickNaNMulAdd

[Richard Henderson]

For each operand, pass a single enumeration instead of a pair of booleans.
The commit also merges multiple different ifdef-selected implementations
of pickNaNMulAdd into a single function whose body is ifdef-selected.

Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 70 +++---
 fpu/softfloat.c|  5 +--
 2 files changed, 28 insertions(+), 47 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 2695183188..0399dfe011 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -594,15 +594,14 @@ static int pickNaN(FloatClass a_cls, FloatClass b_cls,
 | information.
 | Return values : 0 : a; 1 : b; 2 : c; 3 : default-NaN
 **/
-#if defined(TARGET_ARM)
-static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag 
bIsSNaN,
- flag cIsQNaN, flag cIsSNaN, flag infzero,
- float_status *status)
+static int pickNaNMulAdd(FloatClass a_cls, FloatClass b_cls, FloatClass c_cls,
+ bool infzero, float_status *status)
 {
+#if defined(TARGET_ARM)
 /* For ARM, the (inf,zero,qnan) case sets InvalidOp and returns
  * the default NaN
  */
-if (infzero && cIsQNaN) {
+if (infzero && is_qnan(c_cls)) {
 float_raise(float_flag_invalid, status);
 return 3;
 }
@@ -610,25 +609,20 @@ static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
 /* This looks different from the ARM ARM pseudocode, because the ARM ARM
  * puts the operands to a fused mac operation (a*b)+c in the order c,a,b.
  */
-if (cIsSNaN) {
+if (is_snan(c_cls)) {
 return 2;
-} else if (aIsSNaN) {
+} else if (is_snan(a_cls)) {
 return 0;
-} else if (bIsSNaN) {
+} else if (is_snan(b_cls)) {
 return 1;
-} else if (cIsQNaN) {
+} else if (is_qnan(c_cls)) {
 return 2;
-} else if (aIsQNaN) {
+} else if (is_qnan(a_cls)) {
 return 0;
 } else {
 return 1;
 }
-}
 #elif defined(TARGET_MIPS)
-static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag 
bIsSNaN,
- flag cIsQNaN, flag cIsSNaN, flag infzero,
- float_status *status)
-{
 /* For MIPS, the (inf,zero,qnan) case sets InvalidOp and returns
  * the default NaN
  */
@@ -639,41 +633,36 @@ static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
 
 if (snan_bit_is_one(status)) {
 /* Prefer sNaN over qNaN, in the a, b, c order. */
-if (aIsSNaN) {
+if (is_snan(a_cls)) {
 return 0;
-} else if (bIsSNaN) {
+} else if (is_snan(b_cls)) {
 return 1;
-} else if (cIsSNaN) {
+} else if (is_snan(c_cls)) {
 return 2;
-} else if (aIsQNaN) {
+} else if (is_qnan(a_cls)) {
 return 0;
-} else if (bIsQNaN) {
+} else if (is_qnan(b_cls)) {
 return 1;
 } else {
 return 2;
 }
 } else {
 /* Prefer sNaN over qNaN, in the c, a, b order. */
-if (cIsSNaN) {
+if (is_snan(c_cls)) {
 return 2;
-} else if (aIsSNaN) {
+} else if (is_snan(a_cls)) {
 return 0;
-} else if (bIsSNaN) {
+} else if (is_snan(b_cls)) {
 return 1;
-} else if (cIsQNaN) {
+} else if (is_qnan(c_cls)) {
 return 2;
-} else if (aIsQNaN) {
+} else if (is_qnan(a_cls)) {
 return 0;
 } else {
 return 1;
 }
 }
-}
 #elif defined(TARGET_PPC)
-static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag 
bIsSNaN,
- flag cIsQNaN, flag cIsSNaN, flag infzero,
- float_status *status)
-{
 /* For PPC, the (inf,zero,qnan) case sets InvalidOp, but we prefer
  * to return an input NaN if we have one (ie c) rather than generating
  * a default NaN
@@ -686,31 +675,26 @@ static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag 
bIsQNaN, flag bIsSNaN,
 /* If fRA is a NaN return it; otherwise if fRB is a NaN return it;
  * otherwise return fRC. Note that muladd on PPC is (fRA * fRC) + frB
  */
-if (aIsSNaN || aIsQNaN) {
+if (is_nan(a_cls)) {
 return 0;
-} else if (cIsSNaN || cIsQNaN) {
+} else if (is_nan(c_cls)) {
 return 2;
 } else {
 return 1;
 }
-}
 #else
-/* A default implementation: prefer a to b to c.
- * This is unlikely to actually match any real implementation.
- */
-static int pickNaNMulAdd(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag 
bIsSNaN,
- flag cIsQNaN, flag cIsSNaN, flag 

[Qemu-devel] [PATCH v6 07/28] fpu/softfloat: Replace float_class_dnan with parts_default_nan

[Richard Henderson]

With a canonical representation of NaNs, we can return the
default nan directly rather than delay the expansion until
the final format is known.

Note one case where we uselessly assigned to a.sign, which was
overwritten/ignored later when expanding float_class_dnan.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 37 +
 fpu/softfloat.c| 38 +++---
 2 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 515cb12cfa..0d3d81a52b 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -101,6 +101,43 @@ static bool parts_is_snan_frac(uint64_t frac, float_status 
*status)
 #endif
 }
 
+/*
+| The pattern for a default generated deconstructed floating-point NaN.
+**/
+
+static FloatParts parts_default_nan(float_status *status)
+{
+bool sign = 0;
+uint64_t frac;
+
+#if defined(TARGET_SPARC) || defined(TARGET_M68K)
+frac = (1ULL << DECOMPOSED_BINARY_POINT) - 1;
+#elif defined(TARGET_PPC) || defined(TARGET_ARM) || defined(TARGET_ALPHA) || \
+  defined(TARGET_S390X) || defined(TARGET_RISCV)
+frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
+#elif defined(TARGET_HPPA)
+frac = 1ULL << (DECOMPOSED_BINARY_POINT - 2);
+#else
+if (status->snan_bit_is_one) {
+frac = (1ULL << (DECOMPOSED_BINARY_POINT - 1)) - 1;
+} else {
+#if defined(TARGET_MIPS)
+frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
+#else
+frac = 1ULL << (DECOMPOSED_BINARY_POINT - 1);
+sign = 1;
+#endif
+}
+#endif
+
+return (FloatParts) {
+.cls = float_class_qnan,
+.sign = sign,
+.exp = INT_MAX,
+.frac = frac
+};
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 19f40d6932..51780b718f 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -188,7 +188,6 @@ typedef enum __attribute__ ((__packed__)) {
 float_class_inf,
 float_class_qnan,  /* all NaNs from here */
 float_class_snan,
-float_class_dnan,
 float_class_msnan, /* maybe silenced */
 } FloatClass;
 
@@ -494,8 +493,6 @@ static FloatParts float16_unpack_canonical(float16 f, 
float_status *s)
 static float16 float16_round_pack_canonical(FloatParts p, float_status *s)
 {
 switch (p.cls) {
-case float_class_dnan:
-return float16_default_nan(s);
 case float_class_msnan:
 p.frac >>= float16_params.frac_shift;
 return float16_maybe_silence_nan(float16_pack_raw(p), s);
@@ -513,8 +510,6 @@ static FloatParts float32_unpack_canonical(float32 f, 
float_status *s)
 static float32 float32_round_pack_canonical(FloatParts p, float_status *s)
 {
 switch (p.cls) {
-case float_class_dnan:
-return float32_default_nan(s);
 case float_class_msnan:
 p.frac >>= float32_params.frac_shift;
 return float32_maybe_silence_nan(float32_pack_raw(p), s);
@@ -532,8 +527,6 @@ static FloatParts float64_unpack_canonical(float64 f, 
float_status *s)
 static float64 float64_round_pack_canonical(FloatParts p, float_status *s)
 {
 switch (p.cls) {
-case float_class_dnan:
-return float64_default_nan(s);
 case float_class_msnan:
 p.frac >>= float64_params.frac_shift;
 return float64_maybe_silence_nan(float64_pack_raw(p), s);
@@ -566,7 +559,7 @@ static FloatParts return_nan(FloatParts a, float_status *s)
 /* fall through */
 case float_class_qnan:
 if (s->default_nan_mode) {
-a.cls = float_class_dnan;
+return parts_default_nan(s);
 }
 break;
 
@@ -583,7 +576,7 @@ static FloatParts pick_nan(FloatParts a, FloatParts b, 
float_status *s)
 }
 
 if (s->default_nan_mode) {
-a.cls = float_class_dnan;
+return parts_default_nan(s);
 } else {
 if (pickNaN(is_qnan(a.cls), is_snan(a.cls),
 is_qnan(b.cls), is_snan(b.cls),
@@ -614,8 +607,7 @@ static FloatParts pick_nan_muladd(FloatParts a, FloatParts 
b, FloatParts c,
 /* Note that this check is after pickNaNMulAdd so that function
  * has an opportunity to set the Invalid flag.
  */
-a.cls = float_class_dnan;
-return a;
+which = 3;
 }
 
 switch (which) {
@@ -628,8 +620,7 @@ static FloatParts pick_nan_muladd(FloatParts a, FloatParts 
b, FloatParts c,
 a = c;
 break;
 

[Qemu-devel] [PATCH v6 01/28] fpu/softfloat: Fix conversion from uint64 to float128

[Richard Henderson]

From: Petr Tesarik 

The significand is passed to normalizeRoundAndPackFloat128() as high
first, low second. The current code passes the integer first, so the
result is incorrectly shifted left by 64 bits.

This bug affects the emulation of s390x instruction CXLGBR (convert
from logical 64-bit binary-integer operand to extended BFP result).

Cc: qemu-sta...@nongnu.org
Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Petr Tesarik 
Message-Id: <20180511071052.1443-1-ptesa...@suse.com>
Signed-off-by: Richard Henderson 
---
 fpu/softfloat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index bc0f52fa54..d07419324a 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -3147,7 +3147,7 @@ float128 uint64_to_float128(uint64_t a, float_status 
*status)
 if (a == 0) {
 return float128_zero;
 }
-return normalizeRoundAndPackFloat128(0, 0x406E, a, 0, status);
+return normalizeRoundAndPackFloat128(0, 0x406E, 0, a, status);
 }
 
 
-- 
2.17.0

[Qemu-devel] [PATCH v6 10/28] target/arm: squash FZ16 behaviour for conversions

[Richard Henderson]

From: Alex Bennée 

The ARM ARM specifies FZ16 is suppressed for conversions. Rather than
pushing this logic into the softfloat code we can simply save the FZ
state and temporarily disable it for the softfloat call.

Reviewed-by: Peter Maydell 
Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 

---
v4
  - float16_to_floatX squished the wrong softfloat bit for FZ16;
need to adjust input denormals in this case.
---
 target/arm/helper.c | 40 
 1 file changed, 36 insertions(+), 4 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 1762042fc7..238a3ceba8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -11542,22 +11542,54 @@ uint32_t HELPER(set_neon_rmode)(uint32_t rmode, 
CPUARMState *env)
 /* Half precision conversions.  */
 float32 HELPER(vfp_fcvt_f16_to_f32)(float16 a, void *fpstp, uint32_t ahp_mode)
 {
-return float16_to_float32(a, !ahp_mode, fpstp);
+/* Squash FZ16 to 0 for the duration of conversion.  In this case,
+ * it would affect flushing input denormals.
+ */
+float_status *fpst = fpstp;
+flag save = get_flush_inputs_to_zero(fpst);
+set_flush_inputs_to_zero(false, fpst);
+float32 r = float16_to_float32(a, !ahp_mode, fpst);
+set_flush_inputs_to_zero(save, fpst);
+return r;
 }
 
 float16 HELPER(vfp_fcvt_f32_to_f16)(float32 a, void *fpstp, uint32_t ahp_mode)
 {
-return float32_to_float16(a, !ahp_mode, fpstp);
+/* Squash FZ16 to 0 for the duration of conversion.  In this case,
+ * it would affect flushing output denormals.
+ */
+float_status *fpst = fpstp;
+flag save = get_flush_to_zero(fpst);
+set_flush_to_zero(false, fpst);
+float16 r = float32_to_float16(a, !ahp_mode, fpst);
+set_flush_to_zero(save, fpst);
+return r;
 }
 
 float64 HELPER(vfp_fcvt_f16_to_f64)(float16 a, void *fpstp, uint32_t ahp_mode)
 {
-return float16_to_float64(a, !ahp_mode, fpstp);
+/* Squash FZ16 to 0 for the duration of conversion.  In this case,
+ * it would affect flushing input denormals.
+ */
+float_status *fpst = fpstp;
+flag save = get_flush_inputs_to_zero(fpst);
+set_flush_inputs_to_zero(false, fpst);
+float64 r = float16_to_float64(a, !ahp_mode, fpst);
+set_flush_inputs_to_zero(save, fpst);
+return r;
 }
 
 float16 HELPER(vfp_fcvt_f64_to_f16)(float64 a, void *fpstp, uint32_t ahp_mode)
 {
-return float64_to_float16(a, !ahp_mode, fpstp);
+/* Squash FZ16 to 0 for the duration of conversion.  In this case,
+ * it would affect flushing output denormals.
+ */
+float_status *fpst = fpstp;
+flag save = get_flush_to_zero(fpst);
+set_flush_to_zero(false, fpst);
+float16 r = float64_to_float16(a, !ahp_mode, fpst);
+set_flush_to_zero(save, fpst);
+return r;
 }
 
 #define float32_two make_float32(0x4000)
-- 
2.17.0

[Qemu-devel] [PATCH v6 08/28] fpu/softfloat: Replace float_class_msnan with parts_silence_nan

[Richard Henderson]

With a canonical representation of NaNs, we can silence an SNaN
immediately rather than delay until the final format is known.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 23 ++
 fpu/softfloat.c| 40 ++
 2 files changed, 33 insertions(+), 30 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 0d3d81a52b..571d1df378 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -138,6 +138,29 @@ static FloatParts parts_default_nan(float_status *status)
 };
 }
 
+/*
+| Returns a quiet NaN from a signalling NaN for the deconstructed
+| floating-point parts.
+**/
+
+static FloatParts parts_silence_nan(FloatParts a, float_status *status)
+{
+#ifdef NO_SIGNALING_NANS
+g_assert_not_reached();
+#elif defined(TARGET_HPPA)
+a.frac &= ~(1ULL << (DECOMPOSED_BINARY_POINT - 1));
+a.frac |= 1ULL << (DECOMPOSED_BINARY_POINT - 2);
+#else
+if (status->snan_bit_is_one) {
+return parts_default_nan(status);
+} else {
+a.frac |= 1ULL << (DECOMPOSED_BINARY_POINT - 1);
+}
+#endif
+a.cls = float_class_qnan;
+return a;
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 51780b718f..41253c6749 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -188,7 +188,6 @@ typedef enum __attribute__ ((__packed__)) {
 float_class_inf,
 float_class_qnan,  /* all NaNs from here */
 float_class_snan,
-float_class_msnan, /* maybe silenced */
 } FloatClass;
 
 /*
@@ -492,14 +491,7 @@ static FloatParts float16_unpack_canonical(float16 f, 
float_status *s)
 
 static float16 float16_round_pack_canonical(FloatParts p, float_status *s)
 {
-switch (p.cls) {
-case float_class_msnan:
-p.frac >>= float16_params.frac_shift;
-return float16_maybe_silence_nan(float16_pack_raw(p), s);
-default:
-p = round_canonical(p, s, _params);
-return float16_pack_raw(p);
-}
+return float16_pack_raw(round_canonical(p, s, _params));
 }
 
 static FloatParts float32_unpack_canonical(float32 f, float_status *s)
@@ -509,14 +501,7 @@ static FloatParts float32_unpack_canonical(float32 f, 
float_status *s)
 
 static float32 float32_round_pack_canonical(FloatParts p, float_status *s)
 {
-switch (p.cls) {
-case float_class_msnan:
-p.frac >>= float32_params.frac_shift;
-return float32_maybe_silence_nan(float32_pack_raw(p), s);
-default:
-p = round_canonical(p, s, _params);
-return float32_pack_raw(p);
-}
+return float32_pack_raw(round_canonical(p, s, _params));
 }
 
 static FloatParts float64_unpack_canonical(float64 f, float_status *s)
@@ -526,14 +511,7 @@ static FloatParts float64_unpack_canonical(float64 f, 
float_status *s)
 
 static float64 float64_round_pack_canonical(FloatParts p, float_status *s)
 {
-switch (p.cls) {
-case float_class_msnan:
-p.frac >>= float64_params.frac_shift;
-return float64_maybe_silence_nan(float64_pack_raw(p), s);
-default:
-p = round_canonical(p, s, _params);
-return float64_pack_raw(p);
-}
+return float64_pack_raw(round_canonical(p, s, _params));
 }
 
 /* Simple helpers for checking if what NaN we have */
@@ -555,7 +533,7 @@ static FloatParts return_nan(FloatParts a, float_status *s)
 switch (a.cls) {
 case float_class_snan:
 s->float_exception_flags |= float_flag_invalid;
-a.cls = float_class_msnan;
+a = parts_silence_nan(a, s);
 /* fall through */
 case float_class_qnan:
 if (s->default_nan_mode) {
@@ -584,7 +562,9 @@ static FloatParts pick_nan(FloatParts a, FloatParts b, 
float_status *s)
 (a.frac == b.frac && a.sign < b.sign))) {
 a = b;
 }
-a.cls = float_class_msnan;
+if (is_snan(a.cls)) {
+return parts_silence_nan(a, s);
+}
 }
 return a;
 }
@@ -624,8 +604,10 @@ static FloatParts pick_nan_muladd(FloatParts a, FloatParts 
b, FloatParts c,
 default:
 g_assert_not_reached();
 }
-a.cls = float_class_msnan;
 
+if (is_snan(a.cls)) {
+return parts_silence_nan(a, s);
+}
 return a;
 }
 
@@ -1334,7 +1316,6 @@ static int64_t round_to_int_and_pack(FloatParts in, int 
rmode,
 switch (p.cls) {
 case float_class_snan:
 case float_class_qnan:
-case 

[Qemu-devel] [PATCH v6 02/28] fpu/softfloat: Merge NO_SIGNALING_NANS definitions

[Richard Henderson]

Move the ifdef inside the relevant functions instead of
duplicating the function declarations.

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 100 +++--
 1 file changed, 40 insertions(+), 60 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 27834af0de..58b05718c8 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -233,17 +233,6 @@ typedef struct {
 uint64_t high, low;
 } commonNaNT;
 
-#ifdef NO_SIGNALING_NANS
-int float16_is_quiet_nan(float16 a_, float_status *status)
-{
-return float16_is_any_nan(a_);
-}
-
-int float16_is_signaling_nan(float16 a_, float_status *status)
-{
-return 0;
-}
-#else
 /*
 | Returns 1 if the half-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
@@ -251,12 +240,16 @@ int float16_is_signaling_nan(float16 a_, float_status 
*status)
 
 int float16_is_quiet_nan(float16 a_, float_status *status)
 {
+#ifdef NO_SIGNALING_NANS
+return float16_is_any_nan(a_);
+#else
 uint16_t a = float16_val(a_);
 if (status->snan_bit_is_one) {
 return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
 } else {
 return ((a & ~0x8000) >= 0x7C80);
 }
+#endif
 }
 
 /*
@@ -266,14 +259,17 @@ int float16_is_quiet_nan(float16 a_, float_status *status)
 
 int float16_is_signaling_nan(float16 a_, float_status *status)
 {
+#ifdef NO_SIGNALING_NANS
+return 0;
+#else
 uint16_t a = float16_val(a_);
 if (status->snan_bit_is_one) {
 return ((a & ~0x8000) >= 0x7C80);
 } else {
 return (((a >> 9) & 0x3F) == 0x3E) && (a & 0x1FF);
 }
-}
 #endif
+}
 
 /*
 | Returns a quiet NaN if the half-precision floating point value `a' is a
@@ -333,17 +329,6 @@ static float16 commonNaNToFloat16(commonNaNT a, 
float_status *status)
 }
 }
 
-#ifdef NO_SIGNALING_NANS
-int float32_is_quiet_nan(float32 a_, float_status *status)
-{
-return float32_is_any_nan(a_);
-}
-
-int float32_is_signaling_nan(float32 a_, float_status *status)
-{
-return 0;
-}
-#else
 /*
 | Returns 1 if the single-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
@@ -351,12 +336,16 @@ int float32_is_signaling_nan(float32 a_, float_status 
*status)
 
 int float32_is_quiet_nan(float32 a_, float_status *status)
 {
+#ifdef NO_SIGNALING_NANS
+return float32_is_any_nan(a_);
+#else
 uint32_t a = float32_val(a_);
 if (status->snan_bit_is_one) {
 return (((a >> 22) & 0x1FF) == 0x1FE) && (a & 0x003F);
 } else {
 return ((uint32_t)(a << 1) >= 0xFF80);
 }
+#endif
 }
 
 /*
@@ -366,14 +355,17 @@ int float32_is_quiet_nan(float32 a_, float_status *status)
 
 int float32_is_signaling_nan(float32 a_, float_status *status)
 {
+#ifdef NO_SIGNALING_NANS
+return 0;
+#else
 uint32_t a = float32_val(a_);
 if (status->snan_bit_is_one) {
 return ((uint32_t)(a << 1) >= 0xFF80);
 } else {
 return (((a >> 22) & 0x1FF) == 0x1FE) && (a & 0x003F);
 }
-}
 #endif
+}
 
 /*
 | Returns a quiet NaN if the single-precision floating point value `a' is a
@@ -744,17 +736,6 @@ static float32 propagateFloat32NaN(float32 a, float32 b, 
float_status *status)
 }
 }
 
-#ifdef NO_SIGNALING_NANS
-int float64_is_quiet_nan(float64 a_, float_status *status)
-{
-return float64_is_any_nan(a_);
-}
-
-int float64_is_signaling_nan(float64 a_, float_status *status)
-{
-return 0;
-}
-#else
 /*
 | Returns 1 if the double-precision floating-point value `a' is a quiet
 | NaN; otherwise returns 0.
@@ -762,6 +743,9 @@ int float64_is_signaling_nan(float64 a_, float_status 
*status)
 
 int float64_is_quiet_nan(float64 a_, float_status *status)
 {
+#ifdef NO_SIGNALING_NANS
+return float64_is_any_nan(a_);
+#else
 uint64_t a = float64_val(a_);
 if (status->snan_bit_is_one) {
 return (((a >> 51) & 0xFFF) == 0xFFE)
@@ -769,6 +753,7 @@ int float64_is_quiet_nan(float64 a_, float_status *status)
 } else {
 return ((a << 1) >= 0xFFF0ULL);
 }
+#endif
 }
 
 /*
@@ -778,6 +763,9 @@ int float64_is_quiet_nan(float64 a_, float_status *status)
 
 int float64_is_signaling_nan(float64 

[Qemu-devel] [PATCH v6 06/28] fpu/softfloat: Introduce parts_is_snan_frac

[Richard Henderson]

Tested-by: Alex Bennée 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 fpu/softfloat-specialize.h | 15 +++
 fpu/softfloat.c| 12 ++--
 2 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 4fc9ea4ac0..515cb12cfa 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -86,6 +86,21 @@ this code that are retained.
 #define NO_SIGNALING_NANS 1
 #endif
 
+/*
+| For the deconstructed floating-point with fraction FRAC, return true
+| if the fraction represents a signalling NaN; otherwise false.
+**/
+
+static bool parts_is_snan_frac(uint64_t frac, float_status *status)
+{
+#ifdef NO_SIGNALING_NANS
+return false;
+#else
+flag msb = extract64(frac, DECOMPOSED_BINARY_POINT - 1, 1);
+return msb == status->snan_bit_is_one;
+#endif
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 607c4a78d5..19f40d6932 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -331,16 +331,8 @@ static FloatParts canonicalize(FloatParts part, const 
FloatFmt *parm,
 part.cls = float_class_inf;
 } else {
 part.frac <<= parm->frac_shift;
-#ifdef NO_SIGNALING_NANS
-part.cls = float_class_qnan;
-#else
-int64_t msb = part.frac << 2;
-if ((msb < 0) == status->snan_bit_is_one) {
-part.cls = float_class_snan;
-} else {
-part.cls = float_class_qnan;
-}
-#endif
+part.cls = (parts_is_snan_frac(part.frac, status)
+? float_class_snan : float_class_qnan);
 }
 } else if (part.exp == 0) {
 if (likely(part.frac == 0)) {
-- 
2.17.0

[Qemu-devel] [PATCH v6 00/28] softfloat patch roundup

[Richard Henderson]

This is my SNaN patch set, Alex's float-float refactor, and a couple
of other random outstanding fpu patches.  This has been reordered so
as to be bisectable, since the float-float refactor requires the snan
work to avoid breakage.

The edition of pm215/target-arm.next upon which this was built has
now been merged to master, so there are no external dependencies.

Changes since v5:
  - Fix issues in the last couple of patches added for v5.
  - Incorporate feedback from pm215.

Changes since n-2:
  - Use v5 instead of v3, since Alex was up to v4 with float-float.
  - Incorporate feedback from pm215.
  - Include float128 fix from Petr.
  - Include further cleanups for default_nan and silence_nan.
The first of these was inspired by a comment from pm215; 
the rest follow as logical extensions.


r~


Alex Bennée (4):
  target/arm: convert conversion helpers to fpst/ahp_flag
  target/arm: squash FZ16 behaviour for conversions
  fpu/softfloat: Partial support for ARM Alternative half-precision
  fpu/softfloat: re-factor float to float conversions

Petr Tesarik (1):
  fpu/softfloat: Fix conversion from uint64 to float128

Richard Henderson (23):
  fpu/softfloat: Merge NO_SIGNALING_NANS definitions
  fpu/softfloat: Split floatXX_silence_nan from
floatXX_maybe_silence_nan
  fpu/softfloat: Move softfloat-specialize.h below FloatParts definition
  fpu/softfloat: Canonicalize NaN fraction
  fpu/softfloat: Introduce parts_is_snan_frac
  fpu/softfloat: Replace float_class_dnan with parts_default_nan
  fpu/softfloat: Replace float_class_msnan with parts_silence_nan
  target/arm: Use floatX_silence_nan when we have already checked for
SNaN
  target/arm: Remove floatX_maybe_silence_nan from conversions
  target/hppa: Remove floatX_maybe_silence_nan from conversions
  target/m68k: Use floatX_silence_nan when we have already checked for
SNaN
  target/mips: Remove floatX_maybe_silence_nan from conversions
  target/riscv: Remove floatX_maybe_silence_nan from conversions
  target/s390x: Remove floatX_maybe_silence_nan from conversions
  fpu/softfloat: Use float*_silence_nan in propagateFloat*NaN
  fpu/softfloat: Remove floatX_maybe_silence_nan
  fpu/softfloat: Specialize on snan_bit_is_one
  fpu/softfloat: Make is_nan et al available to softfloat-specialize.h
  fpu/softfloat: Pass FloatClass to pickNaN
  fpu/softfloat: Pass FloatClass to pickNaNMulAdd
  fpu/softfloat: Define floatN_default_nan in terms of parts_default_nan
  fpu/softfloat: Clean up parts_default_nan
  fpu/softfloat: Define floatN_silence_nan in terms of parts_silence_nan

 fpu/softfloat-specialize.h| 691 +---
 include/fpu/softfloat-types.h |   1 +
 include/fpu/softfloat.h   |  18 +-
 target/arm/helper.h   |  10 +-
 target/arm/translate.h|  12 +
 fpu/softfloat.c   | 726 --
 target/arm/helper-a64.c   |   7 +-
 target/arm/helper.c   | 114 +++---
 target/arm/translate-a64.c|  37 +-
 target/arm/translate.c|  74 +++-
 target/hppa/cpu.c |   1 -
 target/hppa/op_helper.c   |   2 -
 target/m68k/softfloat.c   |   3 +-
 target/mips/msa_helper.c  |   4 -
 target/mips/op_helper.c   |   2 -
 target/ppc/fpu_helper.c   |   1 -
 target/riscv/fpu_helper.c |   6 +-
 target/s390x/fpu_helper.c |  12 +-
 target/sh4/cpu.c  |   1 -
 target/unicore32/cpu.c|   2 -
 20 files changed, 718 insertions(+), 1006 deletions(-)

-- 
2.17.0

Re: [Qemu-devel] [PATCH v2 28/36] target-microblaze: Add support for extended access to TLBLO

[Edgar E. Iglesias]

On Wed, May 09, 2018 at 02:15:14PM -0700, Richard Henderson wrote:
> On 05/08/2018 10:31 AM, Edgar E. Iglesias wrote:
> > +if (to) {
> > +gen_helper_mmu_write(cpu_env, tcg_const_i32(extended),
> > + tcg_const_i32(sr), cpu_R[dc->ra]);
> > +} else {
> > +gen_helper_mmu_read(cpu_R[dc->rd], cpu_env,
> > +tcg_const_i32(extended), 
> > tcg_const_i32(sr));
> > +}
> 
> These tcg_const_i32 results should be tcg_temp_free'd.

Thanks, I've fixed this for v3.

[Qemu-devel] [PULL 7/7] i386: Add new property to control cache info

[Eduardo Habkost]

From: Babu Moger 

The property legacy-cache will be used to control the cache information.
If user passes "-cpu legacy-cache" then older information will
be displayed even if the hardware supports new information. Otherwise
use the statically loaded cache definitions if available.

Renamed the previous cache structures to legacy_*. If there is any change in
the cache information, then it needs to be initialized in builtin_x86_defs.

Signed-off-by: Babu Moger 
Tested-by: Geoffrey McRae 
Message-Id: <20180514164156.27034-3-babu.mo...@amd.com>
Reviewed-by: Eduardo Habkost 
Signed-off-by: Eduardo Habkost 
---
 include/hw/i386/pc.h |  5 +++
 target/i386/cpu.h|  5 +++
 target/i386/cpu.c| 97 ++--
 3 files changed, 81 insertions(+), 26 deletions(-)

diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index d4fe073cb8..a0c269fc34 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -298,6 +298,11 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);
 
 #define PC_COMPAT_2_12 \
 HW_COMPAT_2_12 \
+{\
+.driver   = TYPE_X86_CPU,\
+.property = "legacy-cache",\
+.value= "on",\
+},
 
 #define PC_COMPAT_2_11 \
 HW_COMPAT_2_11 \
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index ac94013c4a..8bc54d70bf 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1399,6 +1399,11 @@ struct X86CPU {
  */
 bool enable_l3_cache;
 
+/* Compatibility bits for old machine types.
+ * If true present the old cache topology information
+ */
+bool legacy_cache;
+
 /* Compatibility bits for old machine types: */
 bool enable_cpuid_0xb;
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 174a8f434b..e5e66a75d4 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -336,10 +336,14 @@ static void encode_cache_cpuid8006(CPUCacheInfo *l2,
 }
 }
 
-/* Definitions of the hardcoded cache entries we expose: */
+/*
+ * Definitions of the hardcoded cache entries we expose:
+ * These are legacy cache values. If there is a need to change any
+ * of these values please use builtin_x86_defs
+ */
 
 /* L1 data cache: */
-static CPUCacheInfo l1d_cache = {
+static CPUCacheInfo legacy_l1d_cache = {
 .type = DCACHE,
 .level = 1,
 .size = 32 * KiB,
@@ -352,7 +356,7 @@ static CPUCacheInfo l1d_cache = {
 };
 
 /*FIXME: CPUID leaf 0x8005 is inconsistent with leaves 2 & 4 */
-static CPUCacheInfo l1d_cache_amd = {
+static CPUCacheInfo legacy_l1d_cache_amd = {
 .type = DCACHE,
 .level = 1,
 .size = 64 * KiB,
@@ -366,7 +370,7 @@ static CPUCacheInfo l1d_cache_amd = {
 };
 
 /* L1 instruction cache: */
-static CPUCacheInfo l1i_cache = {
+static CPUCacheInfo legacy_l1i_cache = {
 .type = ICACHE,
 .level = 1,
 .size = 32 * KiB,
@@ -379,7 +383,7 @@ static CPUCacheInfo l1i_cache = {
 };
 
 /*FIXME: CPUID leaf 0x8005 is inconsistent with leaves 2 & 4 */
-static CPUCacheInfo l1i_cache_amd = {
+static CPUCacheInfo legacy_l1i_cache_amd = {
 .type = ICACHE,
 .level = 1,
 .size = 64 * KiB,
@@ -393,7 +397,7 @@ static CPUCacheInfo l1i_cache_amd = {
 };
 
 /* Level 2 unified cache: */
-static CPUCacheInfo l2_cache = {
+static CPUCacheInfo legacy_l2_cache = {
 .type = UNIFIED_CACHE,
 .level = 2,
 .size = 4 * MiB,
@@ -406,7 +410,7 @@ static CPUCacheInfo l2_cache = {
 };
 
 /*FIXME: CPUID leaf 2 descriptor is inconsistent with CPUID leaf 4 */
-static CPUCacheInfo l2_cache_cpuid2 = {
+static CPUCacheInfo legacy_l2_cache_cpuid2 = {
 .type = UNIFIED_CACHE,
 .level = 2,
 .size = 2 * MiB,
@@ -416,7 +420,7 @@ static CPUCacheInfo l2_cache_cpuid2 = {
 
 
 /*FIXME: CPUID leaf 0x8006 is inconsistent with leaves 2 & 4 */
-static CPUCacheInfo l2_cache_amd = {
+static CPUCacheInfo legacy_l2_cache_amd = {
 .type = UNIFIED_CACHE,
 .level = 2,
 .size = 512 * KiB,
@@ -428,7 +432,7 @@ static CPUCacheInfo l2_cache_amd = {
 };
 
 /* Level 3 unified cache: */
-static CPUCacheInfo l3_cache = {
+static CPUCacheInfo legacy_l3_cache = {
 .type = UNIFIED_CACHE,
 .level = 3,
 .size = 16 * MiB,
@@ -3338,6 +3342,10 @@ static void x86_cpu_load_def(X86CPU *cpu, 
X86CPUDefinition *def, Error **errp)
 env->features[w] = def->features[w];
 }
 
+/* Store Cache information from the X86CPUDefinition if available */
+env->cache_info = def->cache_info;
+cpu->legacy_cache = def->cache_info ? 0 : 1;
+
 /* Special cases not set in the X86CPUDefinition structs: */
 /* TODO: in-kernel irqchip for hvf */
 if (kvm_enabled()) {
@@ -3687,11 +3695,21 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
uint32_t count,
 if (!cpu->enable_l3_cache) {
 *ecx = 0;
 } else {
-*ecx = cpuid2_cache_descriptor(_cache);
+if (env->cache_info && !cpu->legacy_cache) {

[Qemu-devel] [PULL 2/7] x86/cpu: Enable CLDEMOTE(Demote Cache Line) cpu feature

[Eduardo Habkost]

From: Jingqi Liu 

The CLDEMOTE instruction hints to hardware that the cache line that
contains the linear address should be moved("demoted") from
the cache(s) closest to the processor core to a level more distant
from the processor core. This may accelerate subsequent accesses
to the line by other cores in the same coherence domain,
especially if the line was written by the core that demotes the line.

Intel Snow Ridge has added new cpu feature, CLDEMOTE.
The new cpu feature needs to be exposed to guest VM.

The bit definition:
CPUID.(EAX=7,ECX=0):ECX[bit 25] CLDEMOTE

The release document ref below link:
https://software.intel.com/sites/default/files/managed/c5/15/\
architecture-instruction-set-extensions-programming-reference.pdf

Signed-off-by: Jingqi Liu 
Message-Id: <1525406253-54846-1-git-send-email-jingqi@intel.com>
Reviewed-by: Eduardo Habkost 
Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.h | 1 +
 target/i386/cpu.c | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index b58b779bff..8fbe1537c1 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -680,6 +680,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) /* POPCNT for vectors of 
DW/QW */
 #define CPUID_7_0_ECX_LA57 (1U << 16)
 #define CPUID_7_0_ECX_RDPID(1U << 22)
+#define CPUID_7_0_ECX_CLDEMOTE (1U << 25)  /* CLDEMOTE Instruction */
 
 #define CPUID_7_0_EDX_AVX512_4VNNIW (1U << 2) /* AVX512 Neural Network 
Instructions */
 #define CPUID_7_0_EDX_AVX512_4FMAPS (1U << 3) /* AVX512 Multiply Accumulation 
Single Precision */
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 52fd35b6a1..4b39ab5dd4 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -494,7 +494,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
 "avx512bitalg", NULL, "avx512-vpopcntdq", NULL,
 "la57", NULL, NULL, NULL,
 NULL, NULL, "rdpid", NULL,
-NULL, NULL, NULL, NULL,
+NULL, "cldemote", NULL, NULL,
 NULL, NULL, NULL, NULL,
 },
 .cpuid_eax = 7,
-- 
2.14.3

[Qemu-devel] [PULL 3/7] i386: Helpers to encode cache information consistently

[Eduardo Habkost]

Instead of having a collection of macros that need to be used in
complex expressions to build CPUID data, define a CPUCacheInfo
struct that can hold information about a given cache.  Helper
functions will take a CPUCacheInfo struct as input to encode
CPUID leaves for a cache.

This will help us ensure consistency between cache information
CPUID leaves, and make the existing inconsistencies in CPUID info
more visible.

Signed-off-by: Eduardo Habkost 
Signed-off-by: Babu Moger 
Tested-by: Geoffrey McRae 
Message-Id: <20180510204148.11687-2-babu.mo...@amd.com>
Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.h |  53 ++
 target/i386/cpu.c | 495 --
 2 files changed, 424 insertions(+), 124 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 8fbe1537c1..512c69 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1045,6 +1045,59 @@ typedef enum TPRAccess {
 TPR_ACCESS_WRITE,
 } TPRAccess;
 
+/* Cache information data structures: */
+
+enum CacheType {
+DCACHE,
+ICACHE,
+UNIFIED_CACHE
+};
+
+typedef struct CPUCacheInfo {
+enum CacheType type;
+uint8_t level;
+/* Size in bytes */
+uint32_t size;
+/* Line size, in bytes */
+uint16_t line_size;
+/*
+ * Associativity.
+ * Note: representation of fully-associative caches is not implemented
+ */
+uint8_t associativity;
+/* Physical line partitions. CPUID[0x801D].EBX, CPUID[4].EBX */
+uint8_t partitions;
+/* Number of sets. CPUID[0x801D].ECX, CPUID[4].ECX */
+uint32_t sets;
+/*
+ * Lines per tag.
+ * AMD-specific: CPUID[0x8005], CPUID[0x8006].
+ * (Is this synonym to @partitions?)
+ */
+uint8_t lines_per_tag;
+
+/* Self-initializing cache */
+bool self_init;
+/*
+ * WBINVD/INVD is not guaranteed to act upon lower level caches of
+ * non-originating threads sharing this cache.
+ * CPUID[4].EDX[bit 0], CPUID[0x801D].EDX[bit 0]
+ */
+bool no_invd_sharing;
+/*
+ * Cache is inclusive of lower cache levels.
+ * CPUID[4].EDX[bit 1], CPUID[0x801D].EDX[bit 1].
+ */
+bool inclusive;
+/*
+ * A complex function is used to index the cache, potentially using all
+ * address bits.  CPUID[4].EDX[bit 2].
+ */
+bool complex_indexing;
+} CPUCacheInfo;
+
+
+
 typedef struct CPUX86State {
 /* standard registers */
 target_ulong regs[CPU_NB_REGS];
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 4b39ab5dd4..28bb93990e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -56,33 +56,240 @@
 
 #include "disas/capstone.h"
 
+/* Helpers for building CPUID[2] descriptors: */
+
+struct CPUID2CacheDescriptorInfo {
+enum CacheType type;
+int level;
+int size;
+int line_size;
+int associativity;
+};
 
-/* Cache topology CPUID constants: */
+#define KiB 1024
+#define MiB (1024 * 1024)
 
-/* CPUID Leaf 2 Descriptors */
+/*
+ * Known CPUID 2 cache descriptors.
+ * From Intel SDM Volume 2A, CPUID instruction
+ */
+struct CPUID2CacheDescriptorInfo cpuid2_cache_descriptors[] = {
+[0x06] = { .level = 1, .type = ICACHE,.size =   8 * KiB,
+   .associativity = 4,  .line_size = 32, },
+[0x08] = { .level = 1, .type = ICACHE,.size =  16 * KiB,
+   .associativity = 4,  .line_size = 32, },
+[0x09] = { .level = 1, .type = ICACHE,.size =  32 * KiB,
+   .associativity = 4,  .line_size = 64, },
+[0x0A] = { .level = 1, .type = DCACHE,.size =   8 * KiB,
+   .associativity = 2,  .line_size = 32, },
+[0x0C] = { .level = 1, .type = DCACHE,.size =  16 * KiB,
+   .associativity = 4,  .line_size = 32, },
+[0x0D] = { .level = 1, .type = DCACHE,.size =  16 * KiB,
+   .associativity = 4,  .line_size = 64, },
+[0x0E] = { .level = 1, .type = DCACHE,.size =  24 * KiB,
+   .associativity = 6,  .line_size = 64, },
+[0x1D] = { .level = 2, .type = UNIFIED_CACHE, .size = 128 * KiB,
+   .associativity = 2,  .line_size = 64, },
+[0x21] = { .level = 2, .type = UNIFIED_CACHE, .size = 256 * KiB,
+   .associativity = 8,  .line_size = 64, },
+/* lines per sector is not supported cpuid2_cache_descriptor(),
+* so descriptors 0x22, 0x23 are not included
+*/
+[0x24] = { .level = 2, .type = UNIFIED_CACHE, .size =   1 * MiB,
+   .associativity = 16, .line_size = 64, },
+/* lines per sector is not supported cpuid2_cache_descriptor(),
+* so descriptors 0x25, 0x20 are not included
+*/
+[0x2C] = { .level = 1, .type = DCACHE,.size =  32 * KiB,
+   .associativity = 8,  .line_size = 64, },
+[0x30] = { .level = 1, .type = ICACHE,.size =  32 * KiB,
+   .associativity = 8,  

[Qemu-devel] [PULL 1/7] i386: add KnightsMill cpu model

[Eduardo Habkost]

From: Boqun Feng 

A new cpu model called "KnightsMill" is added to model Knights Mill
processors.  Compared to "Skylake-Server" cpu model, the following
features are added:

avx512_4vnniw avx512_4fmaps avx512pf avx512er avx512_vpopcntdq

and the following features are removed:

pcid invpcid clflushopt avx512dq avx512bw clwb smap rtm mpx
xsavec xgetbv1 hle

Signed-off-by: Boqun Feng 
Message-Id: <2018032821.8337-1-boqun.f...@intel.com>
Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.c | 42 ++
 1 file changed, 42 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index b0a1c629a3..52fd35b6a1 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1839,6 +1839,48 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .xlevel = 0x8008,
 .model_id = "Intel Xeon Processor (Skylake, IBRS)",
 },
+{
+.name = "KnightsMill",
+.level = 0xd,
+.vendor = CPUID_VENDOR_INTEL,
+.family = 6,
+.model = 133,
+.stepping = 0,
+.features[FEAT_1_EDX] =
+CPUID_VME | CPUID_SS | CPUID_SSE2 | CPUID_SSE | CPUID_FXSR |
+CPUID_MMX | CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV |
+CPUID_MCA | CPUID_PGE | CPUID_MTRR | CPUID_SEP | CPUID_APIC |
+CPUID_CX8 | CPUID_MCE | CPUID_PAE | CPUID_MSR | CPUID_TSC |
+CPUID_PSE | CPUID_DE | CPUID_FP87,
+.features[FEAT_1_ECX] =
+CPUID_EXT_AVX | CPUID_EXT_XSAVE | CPUID_EXT_AES |
+CPUID_EXT_POPCNT | CPUID_EXT_X2APIC | CPUID_EXT_SSE42 |
+CPUID_EXT_SSE41 | CPUID_EXT_CX16 | CPUID_EXT_SSSE3 |
+CPUID_EXT_PCLMULQDQ | CPUID_EXT_SSE3 |
+CPUID_EXT_TSC_DEADLINE_TIMER | CPUID_EXT_FMA | CPUID_EXT_MOVBE |
+CPUID_EXT_F16C | CPUID_EXT_RDRAND,
+.features[FEAT_8000_0001_EDX] =
+CPUID_EXT2_LM | CPUID_EXT2_PDPE1GB | CPUID_EXT2_RDTSCP |
+CPUID_EXT2_NX | CPUID_EXT2_SYSCALL,
+.features[FEAT_8000_0001_ECX] =
+CPUID_EXT3_ABM | CPUID_EXT3_LAHF_LM | CPUID_EXT3_3DNOWPREFETCH,
+.features[FEAT_7_0_EBX] =
+CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_BMI1 | CPUID_7_0_EBX_AVX2 |
+CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_ERMS |
+CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_ADX | CPUID_7_0_EBX_AVX512F |
+CPUID_7_0_EBX_AVX512CD | CPUID_7_0_EBX_AVX512PF |
+CPUID_7_0_EBX_AVX512ER,
+.features[FEAT_7_0_ECX] =
+CPUID_7_0_ECX_AVX512_VPOPCNTDQ,
+.features[FEAT_7_0_EDX] =
+CPUID_7_0_EDX_AVX512_4VNNIW | CPUID_7_0_EDX_AVX512_4FMAPS,
+.features[FEAT_XSAVE] =
+CPUID_XSAVE_XSAVEOPT,
+.features[FEAT_6_EAX] =
+CPUID_6_EAX_ARAT,
+.xlevel = 0x8008,
+.model_id = "Intel Xeon Phi Processor (Knights Mill)",
+},
 {
 .name = "Opteron_G1",
 .level = 5,
-- 
2.14.3

[Qemu-devel] [PULL 5/7] i386: Initialize cache information for EPYC family processors

[Eduardo Habkost]

From: Babu Moger 

Initialize pre-determined cache information for EPYC processors.

Signed-off-by: Babu Moger 
Tested-by: Geoffrey McRae 
Message-Id: <20180510204148.11687-5-babu.mo...@amd.com>
Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.c | 52 
 1 file changed, 52 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 55685ed19d..174a8f434b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1109,6 +1109,56 @@ struct X86CPUDefinition {
 CPUCaches *cache_info;
 };
 
+static CPUCaches epyc_cache_info = {
+.l1d_cache = {
+.type = DCACHE,
+.level = 1,
+.size = 32 * KiB,
+.line_size = 64,
+.associativity = 8,
+.partitions = 1,
+.sets = 64,
+.lines_per_tag = 1,
+.self_init = 1,
+.no_invd_sharing = true,
+},
+.l1i_cache = {
+.type = ICACHE,
+.level = 1,
+.size = 64 * KiB,
+.line_size = 64,
+.associativity = 4,
+.partitions = 1,
+.sets = 256,
+.lines_per_tag = 1,
+.self_init = 1,
+.no_invd_sharing = true,
+},
+.l2_cache = {
+.type = UNIFIED_CACHE,
+.level = 2,
+.size = 512 * KiB,
+.line_size = 64,
+.associativity = 8,
+.partitions = 1,
+.sets = 1024,
+.lines_per_tag = 1,
+},
+.l3_cache = {
+.type = UNIFIED_CACHE,
+.level = 3,
+.size = 8 * MiB,
+.line_size = 64,
+.associativity = 16,
+.partitions = 1,
+.sets = 8192,
+.lines_per_tag = 1,
+.self_init = true,
+.inclusive = true,
+.complex_indexing = true,
+},
+};
+
 static X86CPUDefinition builtin_x86_defs[] = {
 {
 .name = "qemu64",
@@ -2345,6 +2395,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_6_EAX_ARAT,
 .xlevel = 0x800A,
 .model_id = "AMD EPYC Processor",
+.cache_info = _cache_info,
 },
 {
 .name = "EPYC-IBPB",
@@ -2391,6 +2442,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_6_EAX_ARAT,
 .xlevel = 0x800A,
 .model_id = "AMD EPYC Processor (with IBPB)",
+.cache_info = _cache_info,
 },
 };
 
-- 
2.14.3

[Qemu-devel] [PULL 6/7] pc: add 2.13 machine types

[Eduardo Habkost]

From: Babu Moger 

Add pc-q35-2.13 and pc-i440fx-2.13 machine types

Signed-off-by: Babu Moger 
Message-Id: <20180514164156.27034-2-babu.mo...@amd.com>
Reviewed-by: Eduardo Habkost 
Signed-off-by: Eduardo Habkost 
---
 include/hw/i386/pc.h |  3 +++
 hw/i386/pc_piix.c| 15 ---
 hw/i386/pc_q35.c | 13 +++--
 3 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 2a98e3ad68..d4fe073cb8 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -296,6 +296,9 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t);
 int e820_get_num_entries(void);
 bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);
 
+#define PC_COMPAT_2_12 \
+HW_COMPAT_2_12 \
+
 #define PC_COMPAT_2_11 \
 HW_COMPAT_2_11 \
 {\
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 729a0508aa..e36c7bbb40 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -425,21 +425,30 @@ static void pc_i440fx_machine_options(MachineClass *m)
 m->default_display = "std";
 }
 
-static void pc_i440fx_2_12_machine_options(MachineClass *m)
+static void pc_i440fx_2_13_machine_options(MachineClass *m)
 {
 pc_i440fx_machine_options(m);
 m->alias = "pc";
 m->is_default = 1;
 }
 
+DEFINE_I440FX_MACHINE(v2_13, "pc-i440fx-2.13", NULL,
+  pc_i440fx_2_13_machine_options);
+
+static void pc_i440fx_2_12_machine_options(MachineClass *m)
+{
+pc_i440fx_2_13_machine_options(m);
+m->is_default = 0;
+m->alias = NULL;
+SET_MACHINE_COMPAT(m, PC_COMPAT_2_12);
+}
+
 DEFINE_I440FX_MACHINE(v2_12, "pc-i440fx-2.12", NULL,
   pc_i440fx_2_12_machine_options);
 
 static void pc_i440fx_2_11_machine_options(MachineClass *m)
 {
 pc_i440fx_2_12_machine_options(m);
-m->is_default = 0;
-m->alias = NULL;
 SET_MACHINE_COMPAT(m, PC_COMPAT_2_11);
 }
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 9ae916327e..2372457c6a 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -308,12 +308,22 @@ static void pc_q35_machine_options(MachineClass *m)
 m->max_cpus = 288;
 }
 
-static void pc_q35_2_12_machine_options(MachineClass *m)
+static void pc_q35_2_13_machine_options(MachineClass *m)
 {
 pc_q35_machine_options(m);
 m->alias = "q35";
 }
 
+DEFINE_Q35_MACHINE(v2_13, "pc-q35-2.13", NULL,
+pc_q35_2_13_machine_options);
+
+static void pc_q35_2_12_machine_options(MachineClass *m)
+{
+pc_q35_2_13_machine_options(m);
+m->alias = NULL;
+SET_MACHINE_COMPAT(m, PC_COMPAT_2_12);
+}
+
 DEFINE_Q35_MACHINE(v2_12, "pc-q35-2.12", NULL,
pc_q35_2_12_machine_options);
 
@@ -323,7 +333,6 @@ static void pc_q35_2_11_machine_options(MachineClass *m)
 
 pc_q35_2_12_machine_options(m);
 pcmc->default_nic_model = "e1000";
-m->alias = NULL;
 SET_MACHINE_COMPAT(m, PC_COMPAT_2_11);
 }
 
-- 
2.14.3

[Qemu-devel] [PULL 4/7] i386: Add cache information in X86CPUDefinition

[Eduardo Habkost]

From: Babu Moger 

Add cache information in X86CPUDefinition and CPUX86State.

Signed-off-by: Babu Moger 
Tested-by: Geoffrey McRae 
Reviewed-by: Eduardo Habkost 
Message-Id: <20180510204148.11687-3-babu.mo...@amd.com>
Signed-off-by: Eduardo Habkost 
---
 target/i386/cpu.h | 7 +++
 target/i386/cpu.c | 1 +
 2 files changed, 8 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 512c69..ac94013c4a 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1097,6 +1097,12 @@ typedef struct CPUCacheInfo {
 } CPUCacheInfo;
 
 
+typedef struct CPUCaches {
+CPUCacheInfo l1d_cache;
+CPUCacheInfo l1i_cache;
+CPUCacheInfo l2_cache;
+CPUCacheInfo l3_cache;
+} CPUCaches;
 
 typedef struct CPUX86State {
 /* standard registers */
@@ -1286,6 +1292,7 @@ typedef struct CPUX86State {
 /* Features that were explicitly enabled/disabled */
 FeatureWordArray user_features;
 uint32_t cpuid_model[12];
+CPUCaches *cache_info;
 
 /* MTRRs */
 uint64_t mtrr_fixed[11];
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 28bb93990e..55685ed19d 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1106,6 +1106,7 @@ struct X86CPUDefinition {
 int stepping;
 FeatureWordArray features;
 const char *model_id;
+CPUCaches *cache_info;
 };
 
 static X86CPUDefinition builtin_x86_defs[] = {
-- 
2.14.3

[Qemu-devel] [PULL 0/7] x86 queue, 2018-05-15

[Eduardo Habkost]

The following changes since commit ad1b4ec39caa5b3f17cbd8160283a03a3dcfe2ae:

  Merge remote-tracking branch 
'remotes/kraxel/tags/input-20180515-pull-request' into staging (2018-05-15 
12:50:06 +0100)

are available in the Git repository at:

  git://github.com/ehabkost/qemu.git tags/x86-next-pull-request

for you to fetch changes up to ab8f992e3e63e91be257e4e343d386dae7be4bcb:

  i386: Add new property to control cache info (2018-05-15 11:33:33 -0300)


x86 queue, 2018-05-15

* KnightsMill CPU model
* CLDEMOTE(Demote Cache Line) cpu feature
* pc-i440fx-2.13 and pc-q35-2.13 machine-types
* Add model-specific cache information to EPYC CPU model



Babu Moger (4):
  i386: Add cache information in X86CPUDefinition
  i386: Initialize cache information for EPYC family processors
  pc: add 2.13 machine types
  i386: Add new property to control cache info

Boqun Feng (1):
  i386: add KnightsMill cpu model

Eduardo Habkost (1):
  i386: Helpers to encode cache information consistently

Jingqi Liu (1):
  x86/cpu: Enable CLDEMOTE(Demote Cache Line) cpu feature

 include/hw/i386/pc.h |   8 +
 target/i386/cpu.h|  66 ++
 hw/i386/pc_piix.c|  15 +-
 hw/i386/pc_q35.c |  13 +-
 target/i386/cpu.c| 635 +--
 5 files changed, 608 insertions(+), 129 deletions(-)

-- 
2.14.3

Re: [Qemu-devel] [PATCH v2 10/36] target-microblaze: Bypass MMU with MMU_NOMMU_IDX

[Edgar E. Iglesias]

On Wed, May 09, 2018 at 01:51:31PM -0700, Richard Henderson wrote:
> On 05/08/2018 10:31 AM, Edgar E. Iglesias wrote:
> > +if (cpu->cfg.use_mmu && (env->sregs[SR_MSR] & MSR_VM)
> > +&& mmu_idx != MMU_NOMMU_IDX) {
> 
> For future cleanup, the first condition should be moved to cpu_mmu_index (the
> second condition is already there).  At which point here you need check 
> nothing
> but mmu_idx.

Thanks, I've added a followup patch that does this.

Cheers,
Edgar

Re: [Qemu-devel] [PATCH] exec.c: Initialize sa_flags passed to sigaction()

[Eric Blake]

On 05/15/2018 03:53 PM, Philippe Mathieu-Daudé wrote:

On 05/15/2018 03:27 PM, Peter Maydell wrote:

Coverity points out that in the user-only version of cpu_abort() we
call sigaction() with a partially initialized struct sigaction
(CID 1005351). Correct the omission.

Signed-off-by: Peter Maydell 
---
  exec.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/exec.c b/exec.c
index ffa1099547..bd8833fc9d 100644
--- a/exec.c
+++ b/exec.c
@@ -1124,6 +1124,7 @@ void cpu_abort(CPUState *cpu, const char *fmt, ...)
  struct sigaction act;


I'd have used the more generic:

struct sigaction act = { };


That's a gcc/clang extension (although we have used it before, 
particularly to shut up buggy versions of clang); better is:


struct sigaction act = { 0 };

if that doesn't trigger the clang bug.



Reviewed-by: Philippe Mathieu-Daudé 


  sigfillset(_mask);
  act.sa_handler = SIG_DFL;


The sigfillset() has to be done after initialization, but you could also 
use:


struct sigaction act = {
  .sa_handler = SIG_DFL;
};
sigfillset(_mask);

as a way to zero-initialize all other fields.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] Problem with data miscompare using scsi-hd, cache=none and io=threads

[Daniel Henrique Barboza]

Hi,

I've been working in the last two months in a miscompare issue that 
happens when using a raid device and a SATA as scsi-hd (emulated SCSI) 
with cache=none and io=threads during a hardware stress test. I'll 
summarize it here as best as I can without creating a great wall of text 
- Red Hat folks can check [1] for all the details.


Using the following setup:

- Host is a POWER9 RHEL 7.5-alt: kernel 4.14.0-49.1.1.el7a.ppc64le, 
qemu-kvm-ma 2.10.0-20.el7 (also reproducible with upstream QEMU)


- Guest is RHEL 7.5-alt using the same kernel as the host, using two 
storage disks (a 1.8 Tb raid and a 446Gb SATA drive) as follows:


    
  
  dev='/dev/disk/by-id/scsi-3600605b000a2c110ff0004053d84a61b'/>

  
  
  
    

Both block devices have WCE off in the host.

With this env, we found problems when running a stress test called HTX 
[2]. At a given time (usually after 24+ hours of test) HTX finds a data 
miscompare in one of the devices. This is an example:


---

Device name: /dev/sdb
Total blocks: 0x74706daf, Block size: 0x200
Rule file name: /usr/lpp/htx/rules/reg/hxestorage/default.hdd
Number of Rulefile passes (cycle) completed: 0
Stanza running: rule_6, Thread no.: 8
Oper performed: wrc, Current seek type: SEQ
LBA no. where IO started: 0x94fa
Transfer size: 0x8400

Miscompare Summary:
===
LBA no. where miscomapre started: 0x94fa
LBA no. where miscomapre ended:   0x94ff
Miscompare start offset (in bytes):   0x8
Miscomapre end offset (in bytes): 0xbff
Miscompare size (in bytes):   0xbf8

Expected data (at miscomapre offset): 8c9aea5a736462007275
Actual data (at miscomapre offset): 889aea5a736462007275

-


This means that the test executed a write at  LBA 0x94fa and, after 
confirming that the write was completed, issue 2 reads in the same LBA 
to assert the written contents and found out a mismatch.



I've tested all sort of configurations between disk vs LUN, cache modes 
and AIO. My findings are:


- using device='lun' instead of device='disk', I can't reproduce the 
issue doesn't matter what other configurations are;
- using device='disk' but with cache='writethrough', issue doesn't 
happen (haven't checked other cache modes);

- using device='disk', cache='none' and io='native', issue doesn't happen.


The issue seems to be tied with the combination device=disk + cache=none 
+ io=threads. I've started digging into the SCSI layer all the way down 
to the block backend. With a shameful amount of logs I've discovered 
that, in the write that the test finds a miscompare, in block/file-posix.c:


- when doing the write, handle_aiocb_rw_vector() returns success, 
pwritev() reports that all bytes were written
- in both reads after the write, handle_aiocb_rw_vector returns success, 
all bytes read by preadv(). In both reads, the data read is different 
from the data written by  the pwritev() that happened before


In the discussions at [1], Fam Zheng suggested a test in which we would 
take down the number of threads created in the POSIX thread pool from 64 
to 1. The idea is to ensure that we're using the same thread to write 
and read. There was a suspicion that the kernel can't guarantee data 
coherency between different threads, even if using the same fd, when 
using pwritev() and preadv(). This would explain why the following reads 
in the same fd would fail to retrieve the same data that was written 
before. After doing this modification, the miscompare didn't reproduce.


After reverting the thread pool number change, I've made a couple of 
attempts trying to flush before read() and flushing after write(). Both 
attempts failed - the miscompare appears in both scenarios. This 
enforces the suspicion we have above - if data coherency can't be 
granted between different threads, flushing in different threads 
wouldn't make a difference too. I've also tested a suggestion from Fam 
where I started the disks with "cache.direct=on,cache.no-flush=off" - 
bug still reproduces.



This is the current status of this investigation. I decided to start a 
discussion here, see if someone can point me something that I overlooked 
or got it wrong, before I started changing the POSIX thread pool 
behavior to see if I can enforce one specific POSIX thread to do a 
read() if we had a write() done in the same fd. Any suggestions?




ps: it is worth mentioning that I was able to reproduce this same bug in 
a POWER8 system running Ubuntu 18.04. Given that the code we're dealing 
with doesn't have any arch-specific behavior I wouldn't be surprised if 
this bug is also reproducible in other archs like x86.



Thanks,

Daniel

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1561017
[2] https://github.com/open-power/HTX

Re: [Qemu-devel] [RFC PATCH 07/12] qcow2-bitmap: add basic bitmaps info

[John Snow]

On 05/14/2018 11:12 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.05.2018 04:25, John Snow wrote:
>> Add functions for querying the basic information inside of bitmaps.
>> Restructure the bitmaps flags masks to facilitate providing a list of
>> flags belonging to the bitmap(s) being queried.
>>
>> Signed-off-by: John Snow 
>> ---
>>   block/qcow2-bitmap.c | 81
>> ++--
>>   block/qcow2.c    |  7 +
>>   block/qcow2.h    |  1 +
>>   3 files changed, 87 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index 60e01abfd7..811b82743a 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
>> @@ -49,8 +49,28 @@
>>     /* Bitmap directory entry flags */
>>   #define BME_RESERVED_FLAGS 0xfffcU
>> -#define BME_FLAG_IN_USE (1U << 0)
>> -#define BME_FLAG_AUTO   (1U << 1)
>> +
>> +enum BME_FLAG_BITS {
>> +    BME_FLAG_BIT__BEGIN = 0,
>> +    BME_FLAG_BIT_IN_USE = BME_FLAG_BIT__BEGIN,
>> +    BME_FLAG_BIT_AUTO   = 1,
>> +    BME_FLAG_BIT_EXTRA  = 2,
>> +    BME_FLAG_BIT__MAX,
>> +};
>> +
>> +#define BME_FLAG_IN_USE (1U << BME_FLAG_BIT_IN_USE)
>> +#define BME_FLAG_AUTO   (1U << BME_FLAG_BIT_AUTO)
>> +#define BME_FLAG_EXTRA  (1U << BME_FLAG_BIT_EXTRA)
>> +
>> +/* Correlate canonical spec values to autogenerated QAPI values */
>> +struct {
>> +    uint32_t mask;
> 
> mask is unused in this patch
> 

It sure is! I'll remove it. It's an artifact from an earlier revision.

>> +    int qapi_value;
>> +} BMEFlagMap[BME_FLAG_BIT__MAX] = {
>> +    [BME_FLAG_BIT_IN_USE] = { BME_FLAG_IN_USE,
>> BITMAP_FLAG_ENUM_IN_USE },
>> +    [BME_FLAG_BIT_AUTO]   = { BME_FLAG_AUTO,   BITMAP_FLAG_ENUM_AUTO },
>> +    [BME_FLAG_BIT_EXTRA]  = { BME_FLAG_EXTRA, 
>> BITMAP_FLAG_ENUM_EXTRA_DATA_COMPATIBLE },
>> +};
>>     /* bits [1, 8] U [56, 63] are reserved */
>>   #define BME_TABLE_ENTRY_RESERVED_MASK 0xff0001feULL
>> @@ -663,6 +683,63 @@ static void del_bitmap_list(BlockDriverState *bs)
>>   }
>>   }
>>   +static BitmapFlagEnumList *get_bitmap_flags(uint32_t flags)
>> +{
>> +    int i;
>> +    BitmapFlagEnumList *flist = NULL;
>> +    BitmapFlagEnumList *ftmp;
>> +
>> +    while (flags) {
>> +    i = ctz32(flags);
>> +    ftmp = g_new0(BitmapFlagEnumList, 1);
>> +    if (i >= BME_FLAG_BIT__BEGIN && i < BME_FLAG_BIT__MAX) {
>> +    ftmp->value = BMEFlagMap[i].qapi_value;
>> +    } else {
>> +    ftmp->value = BITMAP_FLAG_ENUM_UNKNOWN;
> 
> so, there may be several "unknown" entries. It's inconsistent with
> "@unknown: This bitmap has unknown or reserved properties.".
> 

Based on your feedback from the previous patch I'll probably just call
this @reserved for any unrecognized flag.

> Finally, can we export values for unknown flags? It should be more
> informative.
> 

I'll see what I can do -- I hate losing this information too.

Re: [Qemu-devel] [RFC PATCH 03/12] block/qcow2-bitmap: avoid adjusting bm->flags for RO bitmaps

[John Snow]

On 05/14/2018 08:44 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.05.2018 04:25, John Snow wrote:
>> Instead of always setting IN_USE, handle whether or not the bitmap
>> is read-only instead of a two-loop pass. This will allow us to show
>> the flags exactly as they appear for bitmaps in `qemu-img info`.
>>
>> Signed-off-by: John Snow 
>> ---
>>   block/qcow2-bitmap.c | 48
>> 
>>   1 file changed, 20 insertions(+), 28 deletions(-)
>>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index d89758f235..fc8d52fc3e 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
>> @@ -942,13 +942,6 @@ fail:
>>   return ret;
>>   }
>>   -/* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */
>> -static void release_dirty_bitmap_helper(gpointer bitmap,
>> -    gpointer bs)
>> -{
>> -    bdrv_release_dirty_bitmap(bs, bitmap);
>> -}
>> -
>>   /* for g_slist_foreach for GSList of BdrvDirtyBitmap* elements */
>>   static void set_readonly_helper(gpointer bitmap, gpointer value)
>>   {
>> @@ -967,8 +960,8 @@ bool qcow2_load_dirty_bitmaps(BlockDriverState
>> *bs, Error **errp)
>>   BDRVQcow2State *s = bs->opaque;
>>   Qcow2BitmapList *bm_list;
>>   Qcow2Bitmap *bm;
>> -    GSList *created_dirty_bitmaps = NULL;
>>   bool header_updated = false;
>> +    bool needs_update = false;
>>     if (s->nb_bitmaps == 0) {
>>   /* No bitmaps - nothing to do */
>> @@ -992,33 +985,32 @@ bool qcow2_load_dirty_bitmaps(BlockDriverState
>> *bs, Error **errp)
>>   bdrv_disable_dirty_bitmap(bitmap);
>>   }
>>   bdrv_dirty_bitmap_set_persistance(bitmap, true);
>> -    bm->flags |= BME_FLAG_IN_USE;
>> -    created_dirty_bitmaps =
>> -    g_slist_append(created_dirty_bitmaps, bitmap);
>> -    }
>> -    }
>> -
>> -    if (created_dirty_bitmaps != NULL) {
>> -    if (can_write(bs)) {
>> -    /* in_use flags must be updated */
>> -    int ret = update_ext_header_and_dir_in_place(bs, bm_list);
>> -    if (ret < 0) {
>> -    error_setg_errno(errp, -ret, "Can't update bitmap
>> directory");
>> -    goto fail;
>> +    if (can_write(bs)) {
>> +    bm->flags |= BME_FLAG_IN_USE;
>> +    needs_update = true;
>> +    } else {
>> +    bdrv_dirty_bitmap_set_readonly(bitmap, true);
> 
> oops. This shows, that patch 01 was incorrect: before this (03) patch,
> in this case we'll have IN_USE set in cache but not in the image.
> 

Ah, I can try to order this patch first so that the cache stays
semantically correct.

>>   }
>> -    header_updated = true;
>> -    } else {
>> -    g_slist_foreach(created_dirty_bitmaps, set_readonly_helper,
>> -    (gpointer)true);
>>   }
>>   }
>>   -    g_slist_free(created_dirty_bitmaps);
>> +    /* in_use flags must be updated */
>> +    if (needs_update) {
>> +    int ret = update_ext_header_and_dir_in_place(bs, bm_list);
>> +    if (ret < 0) {
>> +    error_setg_errno(errp, -ret, "Can't update bitmap
>> directory");
>> +    goto fail;
>> +    }
>> +    header_updated = true;
>> +    }
>>   return header_updated;
> 
> may be safely changed to return needs_update, as if we failed to update,
> we'll go to fail.
> 

Ugh, yes.

>>     fail:
>> -    g_slist_foreach(created_dirty_bitmaps,
>> release_dirty_bitmap_helper, bs);
>> -    g_slist_free(created_dirty_bitmaps);
>> +    QSIMPLEQ_FOREACH(bm, bm_list, entry) {
>> +    if (bm->dirty_bitmap) {
>> +    bdrv_release_dirty_bitmap(bs, bm->dirty_bitmap);
>> +    }
>> +    }
>>   del_bitmap_list(bs);
>>     return false;
> 
> overall looks good, may be worth move it before 01.
>

Re: [Qemu-devel] [RFC PATCH 06/12] qapi: add bitmap info

[John Snow]

On 05/14/2018 10:30 AM, Vladimir Sementsov-Ogievskiy wrote:
> 12.05.2018 04:25, John Snow wrote:
>> Add some of the necessary scaffolding for reporting bitmap information.
>>
>> Signed-off-by: John Snow 
>> ---
>>   qapi/block-core.json | 60
>> +++-
>>   1 file changed, 59 insertions(+), 1 deletion(-)
>>
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index c50517bff3..8f33f41ce7 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -33,6 +33,61 @@
>>   'date-sec': 'int', 'date-nsec': 'int',
>>   'vm-clock-sec': 'int', 'vm-clock-nsec': 'int' } }
>>   +##
>> +# @BitmapTypeEnum:
>> +#
>> +# An enumeration of possible serialized bitmap types.
>> +#
>> +# @dirty-tracking: This bitmap records information on dirty
>> +#  segments within the file.
>> +#
>> +# @unknown: This bitmap is of an unknown or reserved type.
>> +#
>> +# Since: 2.13
>> +##
>> +{ 'enum': 'BitmapTypeEnum', 'data': [ 'dirty-tracking', 'unknown' ] }
>> +
>> +##
>> +# @BitmapFlagEnum:
>> +#
>> +# An enumeration of possible flags for serialized bitmaps.
>> +#
>> +# @in-use: This bitmap is considered to be in-use, and may now be
>> inconsistent.
>> +#
>> +# @auto: This bitmap must reflect any and all changes to the file it
>> describes.
>> +#    The type of this bitmap must be @DirtyTrackingBitmap.
> 
> logical, but I don't see this restriction in the spec. May be we need to
> update the spec
> 

Where did I even get this idea?

>> +#
>> +# @extra-data-compatible: This bitmap has extra information
>> associated with it.
> 
> no, this flag means, that extra data is compatible. So, if you don't
> know what is this extra data, you can read and modify the bitmap,
> leaving this data as is. If this flag is unset, and there are some extra
> data, bitmap must not be used.
> 

Oh, understood! Thank you for clearing that up.

> Finally, this spec should be consistent (or, may be better, duplicate)
> spec from docs/interop/qcow2.txt..

*cough* I guess if I get enough things wrong it's fair to say "RTFM!"

>> +#
>> +# @unknown: This bitmap has unknown or reserved properties.
> 
> Better is only "reserved flags" (not unknown and not properties), they
> are reserved by spec.
> 

Yeah, I'll take that stance.

>> +#
>> +# Since: 2.13
>> +##
>> +{ 'enum': 'BitmapFlagEnum', 'data': [ 'in-use', 'auto',
>> +  'extra-data-compatible',
>> 'unknown' ] }
>> +
>> +##
>> +# @BitmapInfo:
>> +#
>> +# @name: The name of the bitmap.
>> +#
>> +# @type: The type of bitmap.
>> +#
>> +# @granularity: Bitmap granularity, in bytes.
>> +#
>> +# @count: Overall bitmap dirtiness, in bytes.
>> +#
>> +# @flags: Bitmap flags, if any.
>> +#
>> +# Since: 2.13
>> +#
>> +##
>> +{ 'struct': 'BitmapInfo',
>> +  'data': { 'name': 'str', 'type': 'BitmapTypeEnum', 'granularity':
>> 'int',
>> +    'count': 'int', '*flags': ['BitmapFlagEnum']
> 
> may be worth add 'has-extra-data'
> 

Since you pointed out the difference, I agree.

>> +  }
>> +}
>> +
>>   ##
>>   # @ImageInfoSpecificQCow2EncryptionBase:
>>   #
>> @@ -69,6 +124,8 @@
>>   # @encrypt: details about encryption parameters; only set if image
>>   #   is encrypted (since 2.10)
>>   #
>> +# @bitmaps: list of image bitmaps (since 2.13)
>> +#
>>   # Since: 1.7
>>   ##
>>   { 'struct': 'ImageInfoSpecificQCow2',
>> @@ -77,7 +134,8 @@
>>     '*lazy-refcounts': 'bool',
>>     '*corrupt': 'bool',
>>     'refcount-bits': 'int',
>> -  '*encrypt': 'ImageInfoSpecificQCow2Encryption'
>> +  '*encrypt': 'ImageInfoSpecificQCow2Encryption',
>> +  '*bitmaps': ['BitmapInfo']
>>     } }
>>     ##
> 
>