Re: [Qemu-devel] [RFC PATCH qemu] qmp: Add qom-list-properties to list QOM object properties

[Paolo Bonzini]

On 21/02/2018 04:36, Alexey Kardashevskiy wrote:
> On 19/01/18 16:09, Alexey Kardashevskiy wrote:
>> There is already 'device-list-properties' which does most of the job,
>> however it does not handle everything returned by qom-list-types such
>> as machines as they inherit directly from TYPE_OBJECT and not TYPE_DEVICE.
>>
>> This adds a new qom-list-properties command which prints properties
>> of a specific class and its instance. It is pretty much a simplified copy
>> of the device-list-properties handler.
>>
>> Since it creates an object instance, device properties should appear
>> in the output as they are copied to QOM properties at the instance_init
>> hook.
>>
>> Signed-off-by: Alexey Kardashevskiy 
> 
> So is it ack or nack for the patch? Whose area is this? Thanks,

I think Markus, but I can queue it too because he's on leave.  Can you
please resubmit it?

Paolo

Re: [Qemu-devel] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

[Alberto Garcia]

On Tue 20 Feb 2018 11:24:59 PM CET, Eric Blake wrote:

I was also preparing a patch to change this, but you arrived first :-)

> So, it's time to cut back on the waste.  A compressed cluster
> will NEVER occupy more than an uncompressed cluster (okay, gzip
> DOES document that because the compression stream adds metadata,
> and because of the pigeonhole principle, there are worst case
> scenarios where attempts to compress will actually inflate an
> image - but in those cases, we would just write the cluster
> uncompressed instead of inflating it).  And as that is a smaller
> amount of memory, we can get by with the simpler g_malloc.

> -if (!s->cluster_cache) {
> -s->cluster_cache = g_malloc(s->cluster_size);
> +assert(!s->cluster_cache);
> +s->cluster_data = g_try_malloc(s->cluster_size);
> +s->cluster_cache = g_try_malloc(s->cluster_size);

There's a few things here:

- QEMU won't write compressed data if the size is >= s->cluster_size
  (there's an explicit check for that in qcow2_co_pwritev_compressed())

- The size field of the compressed cluster descriptor *does* allow
  larger sizes, so you can't simply read csize bytes into
  s->cluster_data becuase you could cause a buffer overflow.

- Solution a: check that csize < s->cluster_size and return an error if
  it's not. However! although QEMU won't produce an image with a
  compressed cluster that is larger than the uncompressed one, the qcow2
  on-disk format in principle allows for that, so arguably we should
  accept it.

- Solution b: the width of the 'compressed cluster size' field is
  (cluster_bits - 8), that's (cluster_size / 256) sectors. Since the
  size of each sector is 512 bytes, the maximum possible size that the
  field can store is (cluster_size * 2) bytes. So allocate that amount
  of space for s->cluster_data, read the data as it is on disk and let
  the decompression code return an error if the data is indeed
  corrupted or it doesn't fit in the output buffer.

Berto

Re: [Qemu-devel] [PATCH v2 8/8] qemu-doc: Make "-net" less prominent

[Paolo Bonzini]

On 21/02/2018 01:05, Thomas Huth wrote:
> On 20.02.2018 19:37, Paolo Bonzini wrote:
>> On 20/02/2018 18:40, Thomas Huth wrote:
>>> "-net" is clearly a legacy option. Yet we still use it in almost all
>>> examples in the qemu documentation, and many other spots in the network
>>> chapter. We should make it less prominent that users are not lured into
>>> using it so often anymore. So instead of starting the network chapter with
>>> "-net nic" and documenting "-net " below "-netdev "
>>> everywhere, all the "-net" related documentation is now moved to the end
>>> of the chapter. And the examples are changed to use the "--device" and
>>> "--netdev" options instead of "-net nic -net ".
>>
>> Do we want to change them to "-nic" instead?  The proof is in the
>> pudding, they say, :) and "-nic" is way easier to learn than "-device
>> -netdev".
> 
> While -nic is easier to use than -netdev, I don't think that we should
> put the focus in our main qemu-doc on -nic instead of -netdev. -nic is a
> convenience option, while -netdev is the "architected" way to configure
> network devices. We first should document how to do it "right", and
> teach the user to proper distinguish between emulated guest hardware and
> host network backend (with the old -net command, a lot of people seemed
> to have mixed that up IIRC), and then finally explain -nic on top of it.

Heh, that's a philosophy question regarding the organization of the
whole manual.  Currently the "architected" way is pretty much confined
to docs/qdev-device-use.txt.  The manual is full of uses of -drive or
-hda, and I think it makes sense because honestly that's what users use.
 I should have explained this in the previous message, sorry.

>> And maybe we *should* go the extra mile and deprecate "-net" altogether.
>>  The only case where the newer syntax is a bit more uncomfortable is for
>> "-net nic -net nic -net tap|user", which however does work with "-nic
>> hubport -nic hubport -netdev tap|user,id=x -netdev hubport,netdev=x".
> 
> I'd be glad to add such a deprecation patch to this series - I just
> thought it might have been too early so far, but if you feel confident
> that we can mark it as deprecated, I can spin a v3 with such a patch on
> top...

I can't deny it's going to be a lng deprecation.  But we have to
start somewhere, and -nic is a great start.

I think you should send v3 with the minimal changes required to accept
these patches, and then leave the rest to a separate submission, but of
course you don't have to do it that way.

Thanks,

Paolo

Re: [Qemu-devel] [PATCH] configure: fix sanitizers' test program to mend ASan detection

[Marc-André Lureau]

Hi

On Wed, Feb 21, 2018 at 2:03 AM, Emilio G. Cota  wrote:
> Commit 218bb57 ("build-sys: check static linking of UBSAN", 2018-02-13)
> adds a small test program to check whether ubsan works even when
> configuring with --static. This added program is used to
> detect all sanitizers, which breaks ASan's detection since the
> compilation fails with -fsanitize=address, at least with gcc 5.4.0
> and 7.2.0:
>
>   qemu-conf.c: In function ‘main’:
>   qemu-conf.c:3:20: error: integer overflow in expression [-Werror=overflow]
>  return INT32_MIN / -1;
>   ^
>   cc1: all warnings being treated as errors


I sent a patch a few days ago:
"[PATCH 1/6] build-sys: fix -fsanitize=address check"

>
> Fix it by:
>
> - Changing the test program to one that incurs undefined behaviour that isn't
>   detected at compile-time, even with -fsanitize=address.
>
> - To be extra safe (since compilers might evolve and eventually figure out
>   the UB at compile-time), use this newly-added test only when checking
>   for UBSan; use the skeleton otherwise.
>
> Signed-off-by: Emilio G. Cota 
> ---
>  configure | 20 
>  1 file changed, 12 insertions(+), 8 deletions(-)
>
> diff --git a/configure b/configure
> index 913e148..56f647f 100755
> --- a/configure
> +++ b/configure
> @@ -5306,13 +5306,13 @@ fi
>  ##
>  # checks for sanitizers
>
> -# we could use a simple skeleton for flags checks, but this also
> -# detect the static linking issue of ubsan, see also:
> +# Use this program to detect the static linking issue of ubsan; see
>  # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84285
>  cat > $TMPC << EOF
> -#include 
> -int main(void) {
> -  return INT32_MIN / -1;
> +int main(int argc, char **argv) {
> +  int k = 0x7fff;
> +  k += argc;
> +  return 0;
>  }
>  EOF
>
> @@ -5322,12 +5322,16 @@ have_asan_iface_h=no
>  have_asan_iface_fiber=no
>
>  if test "$sanitizers" = "yes" ; then
> +  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=undefined" ""; then
> +have_ubsan=yes
> +  fi
> +
> +  # Use the skeleton for all other sanitizer checks
> +  write_c_skeleton
> +
>if compile_prog "$CPU_CFLAGS -Werror -fsanitize=address" ""; then
>have_asan=yes
>fi
> -  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=undefined" ""; then
> -  have_ubsan=yes
> -  fi
>
>if check_include "sanitizer/asan_interface.h" ; then
>have_asan_iface_h=yes
> --
> 2.7.4
>

Re: [Qemu-devel] [PATCH v4 5/5] usb-mtp: Advertise SendObjectInfo for write support

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 05:59:04PM -0500, Bandan Das wrote:
> This patch implements a dummy ObjectInfo structure so that
> it's easy to typecast the incoming data. If the metadata is
> valid, write_pending is set. Also, the incoming filename
> is utf-16, so, instead of depending on external libraries, just
> implement a simple function to get the filename
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 132 
> ++-
>  1 file changed, 130 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 9b51708614..086296f415 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -47,6 +47,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
>  CMD_DELETE_OBJECT  = 0x100b,
> +CMD_SEND_OBJECT_INFO   = 0x100c,
>  CMD_SEND_OBJECT= 0x100d,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
> @@ -67,8 +68,10 @@ enum mtp_code {
>  RES_STORE_FULL = 0x200c,
>  RES_STORE_READ_ONLY= 0x200e,
>  RES_PARTIAL_DELETE = 0x2012,
> +RES_STORE_NOT_AVAILABLE= 0x2013,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
>  RES_INVALID_OBJECTINFO = 0x2015,
> +RES_DESTINATION_UNSUPPORTED= 0x2020,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
>  RES_SESSION_ALREADY_OPEN   = 0x201e,
> @@ -196,6 +199,34 @@ struct MTPState {
>  } dataset;
>  };
>  
> +/*
> + * ObjectInfo dataset received from initiator
> + * Fields we don't care about are ignored
> + */
> +typedef struct {
> +uint32_t storage_id; /*unused*/
> +uint16_t format;
> +uint16_t protection_status; /*unused*/
> +uint32_t size;
> +uint16_t thumb_format; /*unused*/
> +uint32_t thumb_comp_sz; /*unused*/
> +uint32_t thumb_pix_width; /*unused*/
> +uint32_t thumb_pix_height; /*unused*/
> +uint32_t image_pix_width; /*unused*/
> +uint32_t image_pix_height; /*unused*/
> +uint32_t image_bit_depth; /*unused*/
> +uint32_t parent; /*unused*/
> +uint16_t assoc_type;
> +uint32_t assoc_desc;
> +uint32_t seq_no; /*unused*/
> +uint8_t length; /*part of filename field*/
> +uint16_t filename[0];
> +char date_created[0]; /*unused*/
> +char date_modified[0]; /*unused*/
> +char keywords[0]; /*unused*/
> +/* string and other data follows */
> +} QEMU_PACKED ObjectInfo;
> +
>  #define TYPE_USB_MTP "usb-mtp"
>  #define USB_MTP(obj) OBJECT_CHECK(MTPState, (obj), TYPE_USB_MTP)
>  
> @@ -437,7 +468,6 @@ static MTPObject *usb_mtp_add_child(MTPState *s, 
> MTPObject *o,
>  return child;
>  }
>  
> -#ifdef CONFIG_INOTIFY1
>  static MTPObject *usb_mtp_object_lookup_name(MTPObject *parent,
>   char *name, int len)
>  {
> @@ -452,6 +482,7 @@ static MTPObject *usb_mtp_object_lookup_name(MTPObject 
> *parent,
>  return NULL;
>  }
>  
> +#ifdef CONFIG_INOTIFY1
>  static MTPObject *usb_mtp_object_lookup_wd(MTPState *s, int wd)
>  {
>  MTPObject *iter;
> @@ -815,6 +846,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
>  CMD_DELETE_OBJECT,
> +CMD_SEND_OBJECT_INFO,

Same question about filtering this out for read-only devices,
and somewhere else in this patch validating it too.

>  CMD_SEND_OBJECT,
>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
> @@ -1243,7 +1275,7 @@ static void usb_mtp_object_delete(MTPState *s, uint32_t 
> handle,
>  static void usb_mtp_command(MTPState *s, MTPControl *c)
>  {
>  MTPData *data_in = NULL;
> -MTPObject *o;
> +MTPObject *o = NULL;
>  uint32_t nres = 0, res0 = 0;
>  
>  /* sanity checks */
> @@ -1390,6 +1422,37 @@ static void usb_mtp_command(MTPState *s, MTPControl *c)
>  nres = 1;
>  res0 = data_in->length;
>  break;
> +case CMD_SEND_OBJECT_INFO:
> +/* First parameter points to storage id or is 0 */
> +if (c->argv[0] && (c->argv[0] != QEMU_STORAGE_ID)) {
> +usb_mtp_queue_result(s, RES_STORE_NOT_AVAILABLE, c->trans,
> + 0, 0, 0, 0);
> +} else if (c->argv[1] && !c->argv[0]) {
> +/* If second parameter is specified, first must also be 
> specified */
> +usb_mtp_queue_result(s, RES_DESTINATION_UNSUPPORTED, c->trans,
> + 0, 0, 0, 0);
> +} else {
> +uint32_t handle = c->argv[1];
> +if (handle == 0x || handle == 0) {
> +/* root object */
> +o = QTAILQ_FIRST(>objects);
> +} else {
> +o = usb_mtp_object_lookup(s, handle);
> +}
> +

Re: [Qemu-devel] [PATCH v4 3/5] usb-mtp: Support delete of mtp objects

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 05:59:02PM -0500, Bandan Das wrote:
> Write of existing objects by the initiator is acheived by
> making a temporary buffer with the new changes, deleting the
> old file and then writing a new file with the same name.
> 
> Also, add a "readonly" property which needs to be set to false
> for deletion to work.
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 123 
> +++
>  1 file changed, 123 insertions(+)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 63f8f3b90b..5ef77f3e9f 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -46,6 +46,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_HANDLES = 0x1007,
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
> +CMD_DELETE_OBJECT  = 0x100b,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
>  CMD_GET_OBJECT_PROP_DESC   = 0x9802,
> @@ -62,6 +63,8 @@ enum mtp_code {
>  RES_INVALID_STORAGE_ID = 0x2008,
>  RES_INVALID_OBJECT_HANDLE  = 0x2009,
>  RES_INVALID_OBJECT_FORMAT_CODE = 0x200b,
> +RES_STORE_READ_ONLY= 0x200e,
> +RES_PARTIAL_DELETE = 0x2012,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
> @@ -172,6 +175,7 @@ struct MTPState {
>  MTPControl   *result;
>  uint32_t session;
>  uint32_t next_handle;
> +bool readonly;
>  
>  QTAILQ_HEAD(, MTPObject) objects;
>  #ifdef CONFIG_INOTIFY1
> @@ -799,6 +803,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_NUM_OBJECTS,
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
> +CMD_DELETE_OBJECT,

Should we not advertize this in the first place if the device is readonly.

>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
>  CMD_GET_OBJECT_PROPS_SUPPORTED,
> @@ -1113,6 +1118,116 @@ static MTPData 
> *usb_mtp_get_object_prop_value(MTPState *s, MTPControl *c,
>  return d;
>  }
>  
> +/* Return correct return code for a delete event */
> +enum {
> +ALL_DELETE,
> +PARTIAL_DELETE,
> +READ_ONLY,
> +};
> +
> +/* Assumes that children, if any, have been already freed */
> +static void usb_mtp_object_free_one(MTPState *s, MTPObject *o)
> +{
> +#ifndef CONFIG_INOTIFY1
> +assert(o->nchildren == 0);
> +QTAILQ_REMOVE(>objects, o, next);
> +g_free(o->name);
> +g_free(o->path);
> +g_free(o);
> +#endif
> +}
> +
> +static int usb_mtp_deletefn(MTPState *s, MTPObject *o, uint32_t trans)
> +{
> +MTPObject *iter, *iter2;
> +bool partial_delete = false;
> +bool success = false;
> +
> +/*
> + * TODO: Add support for Protection Status
> + */
> +
> +QLIST_FOREACH(iter, >children, list) {
> +if (iter->format == FMT_ASSOCIATION) {
> +QLIST_FOREACH(iter2, >children, list) {
> +usb_mtp_deletefn(s, iter2, trans);
> +}
> +}
> +}
> +
> +if (o->format == FMT_UNDEFINED_OBJECT) {
> +if (remove(o->path)) {
> +partial_delete = true;
> +} else {
> +usb_mtp_object_free_one(s, o);
> +success = true;
> +}
> +}
> +
> +if (o->format == FMT_ASSOCIATION) {
> +if (rmdir(o->path)) {
> +partial_delete = true;
> +} else {
> +usb_mtp_object_free_one(s, o);
> +success = true;
> +}
> +}
> +
> +if (success && partial_delete) {
> +return PARTIAL_DELETE;
> +}
> +if (!success && partial_delete) {
> +return READ_ONLY;
> +}
> +return ALL_DELETE;
> +}
> +
> +static void usb_mtp_object_delete(MTPState *s, uint32_t handle,
> +  uint32_t format_code, uint32_t trans)
> +{
> +MTPObject *o;
> +int ret;
> +
> +/* Return error if store is read-only */
> +if (!FLAG_SET(s, MTP_FLAG_WRITABLE)) {
> +usb_mtp_queue_result(s, RES_STORE_READ_ONLY,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +if (format_code != 0) {
> +usb_mtp_queue_result(s, RES_SPEC_BY_FORMAT_UNSUPPORTED,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +if (handle == 0xFFF) {
> +o = QTAILQ_FIRST(>objects);
> +} else {
> +o = usb_mtp_object_lookup(s, handle);
> +}
> +if (o == NULL) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECT_HANDLE,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +ret = usb_mtp_deletefn(s, o, trans);
> +if (ret == PARTIAL_DELETE) {
> +usb_mtp_queue_result(s, RES_PARTIAL_DELETE,
> + trans, 0, 0, 0, 0);
> +return;
> +} else if (ret == 

Re: [Qemu-devel] [PATCH 1/2] qcow2: Prefer byte-based calls into bs->file

[Alberto Garcia]

On Tue 20 Feb 2018 11:24:58 PM CET, Eric Blake wrote:
> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
> index d46b69d7f34..3fefeb3dc50 100644
> --- a/block/qcow2-refcount.c
> +++ b/block/qcow2-refcount.c
> @@ -2310,8 +2310,8 @@ write_refblocks:
>  on_disk_refblock = (void *)((char *) *refcount_table +
>  refblock_index * s->cluster_size);
>
> -ret = bdrv_write(bs->file, refblock_offset / BDRV_SECTOR_SIZE,
> - on_disk_refblock, s->cluster_sectors);
> +ret = bdrv_pwrite(bs->file, refblock_offset,
> + on_disk_refblock, s->cluster_size);

It looks like the second line is not properly indented. I think you can
also move on_disk_refblock to the previous line.

Otherwise the patch looks good.

Reviewed-by: Alberto Garcia 

Berto

Re: [Qemu-devel] [qemu-web PATCH] Add a blog post documenting Spectre/Meltdown options for QEMU 2.11.1

[Paolo Bonzini]

On 16/02/2018 12:57, Dr. David Alan Gilbert wrote:
> It's a bit weird to end up pointing to them to a lkml post;
> Paolo: Any ideas on anything better to say?

IBRS is really that much slower than retpolines on pre-Skylake machines,
so it's a hard call to tell people to use it.  It looks like Intel is
going to add Yet Another Bit to clean up the Skylake mess; in the
meanwhile I would just use retpolines even on Skylake and newer
machines, because the Spectre attacks are much more theoretical than
Meltdown, but it's good to point out the issue for the paranoid folks.

Paolo

Re: [Qemu-devel] [Bug 1750229] Re: virtio-blk-pci regression: softlock in guest kernel at module loading

[Matwey V. Kornilov]

Well, last_avail_idx equals to shadow_avail_idx and both of them are 1
at the qemu side. So, only one request is transferred.
I wonder why, probably something is badly cached, but new avail_idx
(which is supposed to become 2) is never shown up.

2018-02-20 15:49 GMT+03:00 Matwey V. Kornilov :
> virtqueue_pop() returns NULL due to virtio_queue_empty_rcu() returns
> true all the time.
>
> 2018-02-20 14:47 GMT+03:00 Matwey V. Kornilov :
>> Well, I've found that on qemu side VirtQueue for virtio_blk device
>> infinitely calls virtio_blk_handle_vq() where virtio_blk_get_request()
>> (virtqueue_pop() in essens) always returns NULL.
>>
>> 2018-02-18 15:26 GMT+03:00 Matwey V. Kornilov :
>>> ** Attachment added: ".build.kernel.kvm"
>>>
>>> https://bugs.launchpad.net/qemu/+bug/1750229/+attachment/5057653/+files/.build.kernel.kvm
>>>
>>> --
>>> You received this bug notification because you are subscribed to the bug
>>> report.
>>> https://bugs.launchpad.net/bugs/1750229
>>>
>>> Title:
>>>   virtio-blk-pci regression: softlock in guest kernel at module loading
>>>
>>> Status in QEMU:
>>>   New
>>>
>>> Bug description:
>>>   Hello,
>>>
>>>   I am running qemu from master git branch on x86_64 host with kernel is
>>>   4.4.114. I've found that commit
>>>
>>>   9a4c0e220d8a "hw/virtio-pci: fix virtio behaviour"
>>>
>>>   introduces an regression with the following command:
>>>
>>>   qemu-system-x86_64 -enable-kvm -nodefaults -no-reboot -nographic
>>>   -vga none -runas qemu -kernel .build.kernel.kvm -initrd
>>>   .build.initrd.kvm -append 'panic=1 softlockup_panic=1 no-kvmclock
>>>   nmi_watchdog=0 console=ttyS0 root=/dev/disk/by-id/virtio-0' -m 2048
>>>   -drive file=./root,format=raw,if=none,id=disk,serial=0,cache=unsafe
>>>   -device virtio-blk-pci,drive=disk -serial stdio -smp 2
>>>
>>>   Starting from this commit to master the following happens with a wide
>>>   variety of guest kernels (4.4 to 4.15):
>>>
>>>   [   62.428107] BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 
>>> nice=-20 stuck for 59s!
>>>   [   62.437426] Showing busy workqueues and worker pools:
>>>   [   62.443117] workqueue events: flags=0x0
>>>   [   62.447512]   pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
>>>   [   62.448161] pending: check_corruption
>>>   [   62.458570] workqueue kblockd: flags=0x18
>>>   [   62.463082]   pwq 1: cpus=0 node=0 flags=0x0 nice=-20 active=3/256
>>>   [   62.463082] in-flight: 4:blk_mq_run_work_fn
>>>   [   62.463082] pending: blk_mq_run_work_fn, blk_mq_timeout_work
>>>   [   62.474831] pool 1: cpus=0 node=0 flags=0x0 nice=-20 hung=59s 
>>> workers=2 idle: 214
>>>   [   62.492121] INFO: rcu_preempt detected stalls on CPUs/tasks:
>>>   [   62.492121]  Tasks blocked on level-0 rcu_node (CPUs 0-1): P4
>>>   [   62.492121]  (detected by 0, t=15002 jiffies, g=-130, c=-131, q=32)
>>>   [   62.492121] kworker/0:0HR  running task0 4  2 
>>> 0x8000
>>>   [   62.492121] Workqueue: kblockd blk_mq_run_work_fn
>>>   [   62.492121] Call Trace:
>>>   [   62.492121]  
>>>   [   62.492121]  sched_show_task+0xdf/0x100
>>>   [   62.492121]  rcu_print_detail_task_stall_rnp+0x48/0x69
>>>   [   62.492121]  rcu_check_callbacks+0x93d/0x9d0
>>>   [   62.492121]  ? tick_sched_do_timer+0x40/0x40
>>>   [   62.492121]  update_process_times+0x28/0x50
>>>   [   62.492121]  tick_sched_handle+0x22/0x70
>>>   [   62.492121]  tick_sched_timer+0x34/0x70
>>>   [   62.492121]  __hrtimer_run_queues+0xcc/0x250
>>>   [   62.492121]  hrtimer_interrupt+0xab/0x1f0
>>>   [   62.492121]  smp_apic_timer_interrupt+0x62/0x150
>>>   [   62.492121]  apic_timer_interrupt+0xa2/0xb0
>>>   [   62.492121]  
>>>   [   62.492121] RIP: 0010:iowrite16+0x1d/0x30
>>>   [   62.492121] RSP: 0018:a477c034fcc8 EFLAGS: 00010292 ORIG_RAX: 
>>> ff11
>>>   [   62.492121] RAX: a24fbdb0 RBX: 92a1f8f82000 RCX: 
>>> 0001
>>>   [   62.492121] RDX: a477c0371000 RSI: a477c0371000 RDI: 
>>> 
>>>   [   62.492121] RBP: 0001 R08:  R09: 
>>> 01080020
>>>   [   62.492121] R10: dc7cc1e4fc00 R11:  R12: 
>>> 
>>>   [   62.492121] R13:  R14: 92a1f93f R15: 
>>> 92a1f8e1aa80
>>>   [   62.492121]  ? vp_synchronize_vectors+0x60/0x60
>>>   [   62.492121]  vp_notify+0x12/0x20
>>>   [   62.492121]  virtqueue_notify+0x18/0x30
>>>   [   62.492121]  virtio_queue_rq+0x2f5/0x300 [virtio_blk]
>>>   [   62.492121]  blk_mq_dispatch_rq_list+0x7e/0x4a0
>>>   [   62.492121]  blk_mq_do_dispatch_sched+0x4a/0xd0
>>>   [   62.492121]  blk_mq_sched_dispatch_requests+0x106/0x170
>>>   [   62.492121]  __blk_mq_run_hw_queue+0x80/0x90
>>>   [   62.492121]  process_one_work+0x1e3/0x420
>>>   [   62.492121]  worker_thread+0x2b/0x3d0
>>>   [   62.492121]  ? process_one_work+0x420/0x420
>>>   [   62.492121]  kthread+0x113/0x130

[Qemu-devel] SSD virtio-scsi passthrough

[Nitin Gupta]

Dear Qemu Team

Please let me know the qemu command for doing ssd virtio-scsi passthrough .
i am able to do the pass through with virsh .but same command when i am
trying with qemu , VM is not coming up

Please let me know how can i proceed further . any help will be appreciated

Regards
nitin

Re: [Qemu-devel] [PATCH v4 4/5] usb-mtp: Introduce write support for MTP objects

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 05:59:03PM -0500, Bandan Das wrote:
> Allow write operations on behalf of the initiator. The
> precursor to write is the sending of the write metadata
> that consists of the ObjectInfo dataset. This patch introduces
> a flag that is set when the responder is ready to receive
> write data based on a previous SendObjectInfo operation by
> the initiator (The SendObjectInfo implementation is in a
> later patch)
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 152 
> ++-
>  1 file changed, 150 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 5ef77f3e9f..9b51708614 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -47,6 +47,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
>  CMD_DELETE_OBJECT  = 0x100b,
> +CMD_SEND_OBJECT= 0x100d,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
>  CMD_GET_OBJECT_PROP_DESC   = 0x9802,
> @@ -63,9 +64,11 @@ enum mtp_code {
>  RES_INVALID_STORAGE_ID = 0x2008,
>  RES_INVALID_OBJECT_HANDLE  = 0x2009,
>  RES_INVALID_OBJECT_FORMAT_CODE = 0x200b,
> +RES_STORE_FULL = 0x200c,
>  RES_STORE_READ_ONLY= 0x200e,
>  RES_PARTIAL_DELETE = 0x2012,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
> +RES_INVALID_OBJECTINFO = 0x2015,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
>  RES_SESSION_ALREADY_OPEN   = 0x201e,
> @@ -183,6 +186,14 @@ struct MTPState {
>  int  inotifyfd;
>  QTAILQ_HEAD(events, MTPMonEntry) events;
>  #endif
> +/* Responder is expecting a write operation */
> +bool write_pending;
> +struct {
> +uint32_t parent_handle;
> +uint16_t format;
> +uint32_t size;
> +char *filename;
> +} dataset;
>  };
>  
>  #define TYPE_USB_MTP "usb-mtp"
> @@ -804,6 +815,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
>  CMD_DELETE_OBJECT,
> +CMD_SEND_OBJECT,

Seems we should not advertize this for readonly devices.

>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
>  CMD_GET_OBJECT_PROPS_SUPPORTED,
> @@ -1378,6 +1390,14 @@ static void usb_mtp_command(MTPState *s, MTPControl *c)
>  nres = 1;
>  res0 = data_in->length;
>  break;
> +case CMD_SEND_OBJECT:
> +if (!s->write_pending) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO,
> + c->trans, 0, 0, 0, 0);
> +return;
> +}
> +s->data_out = usb_mtp_data_alloc(c);
> +return;
>  case CMD_GET_OBJECT_PROPS_SUPPORTED:
>  if (c->argv[0] != FMT_UNDEFINED_OBJECT &&
>  c->argv[0] != FMT_ASSOCIATION) {
> @@ -1472,12 +1492,126 @@ static void usb_mtp_cancel_packet(USBDevice *dev, 
> USBPacket *p)
>  fprintf(stderr, "%s\n", __func__);
>  }
>  
> +static void usb_mtp_write_data(MTPState *s)
> +{
> +MTPData *d = s->data_out;
> +MTPObject *parent =
> +usb_mtp_object_lookup(s, s->dataset.parent_handle);
> +char *path = NULL;
> +int rc = -1;
> +mode_t mask = 0644;
> +
> +assert(d != NULL);
> +


Somewhere in here should surely be validating the "readonly" flag.

> +if (parent == NULL || !s->write_pending) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO, d->trans,
> + 0, 0, 0, 0);
> +return;
> +}
> +

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH v2 05/36] qcow2: Use BlockdevRef in qcow2_create2()

[Kevin Wolf]

Instead of passing a separate BlockDriverState* into qcow2_create2(),
make use of the BlockdevRef that is included in BlockdevCreateOptions.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 include/block/block.h |  1 +
 block.c   | 47 +++
 block/qcow2.c | 38 --
 3 files changed, 72 insertions(+), 14 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 947e8876cd..54fe8b7a0e 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -245,6 +245,7 @@ BdrvChild *bdrv_open_child(const char *filename,
BlockDriverState* parent,
const BdrvChildRole *child_role,
bool allow_none, Error **errp);
+BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
  Error **errp);
 int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
diff --git a/block.c b/block.c
index 814e5a02da..c0e343d278 100644
--- a/block.c
+++ b/block.c
@@ -35,6 +35,8 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/qobject-output-visitor.h"
+#include "qapi-visit.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/sysemu.h"
 #include "qemu/notify.h"
@@ -2408,6 +2410,51 @@ BdrvChild *bdrv_open_child(const char *filename,
 return c;
 }
 
+/* TODO Future callers may need to specify parent/child_role in order for
+ * option inheritance to work. Existing callers use it for the root node. */
+BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp)
+{
+BlockDriverState *bs = NULL;
+Error *local_err = NULL;
+QObject *obj = NULL;
+QDict *qdict = NULL;
+const char *reference = NULL;
+Visitor *v = NULL;
+
+if (ref->type == QTYPE_QSTRING) {
+reference = ref->u.reference;
+} else {
+BlockdevOptions *options = >u.definition;
+assert(ref->type == QTYPE_QDICT);
+
+v = qobject_output_visitor_new();
+visit_type_BlockdevOptions(v, NULL, , _err);
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+visit_complete(v, );
+
+qdict = qobject_to_qdict(obj);
+qdict_flatten(qdict);
+
+/* bdrv_open_inherit() defaults to the values in bdrv_flags (for
+ * compatibility with other callers) rather than what we want as the
+ * real defaults. Apply the defaults here instead. */
+qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
+qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
+qdict_set_default_str(qdict, BDRV_OPT_READ_ONLY, "off");
+}
+
+bs = bdrv_open_inherit(NULL, reference, qdict, 0, NULL, NULL, errp);
+obj = NULL;
+
+fail:
+qobject_decref(obj);
+visit_free(v);
+return bs;
+}
+
 static BlockDriverState *bdrv_append_temp_snapshot(BlockDriverState *bs,
int flags,
QDict *snapshot_options,
diff --git a/block/qcow2.c b/block/qcow2.c
index 22194180c6..b34924b0f0 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2732,8 +2732,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockDriverState *bs,
- BlockdevCreateOptions *create_options,
+static int qcow2_create2(BlockdevCreateOptions *create_options,
  QemuOpts *opts, const char *encryptfmt, Error **errp)
 {
 BlockdevCreateOptionsQcow2 *qcow2_opts;
@@ -2751,7 +2750,8 @@ static int qcow2_create2(BlockDriverState *bs,
  * 2 GB for 64k clusters, and we don't want to have a 2 GB initial file
  * size for any qcow2 image.
  */
-BlockBackend *blk;
+BlockBackend *blk = NULL;
+BlockDriverState *bs = NULL;
 QCowHeader *header;
 size_t cluster_size;
 int version;
@@ -2760,10 +2760,15 @@ static int qcow2_create2(BlockDriverState *bs,
 Error *local_err = NULL;
 int ret;
 
-/* Validate options and set default values */
 assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
 qcow2_opts = _options->u.qcow2;
 
+bs = bdrv_open_blockdev_ref(qcow2_opts->file, errp);
+if (bs == NULL) {
+return -EIO;
+}
+
+/* Validate options and set default values */
 if (!QEMU_IS_ALIGNED(qcow2_opts->size, BDRV_SECTOR_SIZE)) {
 error_setg(errp, "Image size must be a multiple of 512 bytes");
 ret = -EINVAL;
@@ -2792,7 +2797,8 @@ static int qcow2_create2(BlockDriverState *bs,
 }
 
 if (!validate_cluster_size(cluster_size, errp)) {
-return -EINVAL;
+

[Qemu-devel] [PATCH v2 00/36] x-blockdev-create for protocols and qcow2

[Kevin Wolf]

This series implements a minimal QMP command that allows to create an
image file on the protocol level or an image format on a given block
node.

Eventually, the interface is going to change to some kind of an async
command (possibly a (non-)block job), but that will require more work on
the job infrastructure first, so let's first QAPIfy image creation in
the block drivers. In this series, I'm going for a synchronous command
that is prefixed with x- for now.

This series converts qcow2 and all protocol drivers that allow an actual
image creation. This means that drivers which only check if the already
existing storage is good enough are not converted (e.g. host_device,
iscsi). The old behaviour was useful because 'qemu-img create' wants to
create both protocol and format layer, but with the separation in QMP,
you can just leave out the protocol layer creation when the device
already exists.

Please note that for some of the protocol drivers (gluster, rbd and
sheepdog) I don't have a test setup ready. For those, I only tested
with a fake server address to check that the option are parsed correctly
up to this point and an appropriate error is returned without crashing.

If you are a maintainer of one of these protocols and you are
interested in keeping image creation working for your protocol, you
probably want to test this series on a real setup and give me some
feedback. If you don't, I'll just merge the patches and hope that they
won't break anything.

v2:
- Patch 1 ('block/qapi: Introduce BlockdevCreateOptions'):
  Added nvme as unsupported driver

- Patch 8 ('util: Add qemu_opts_to_qdict_filtered'):
  Fixed use after free with QemuOpts that contained more than one option
  with the same name, documented the behaviour with them

- Patches 9 and 10 (new):
  Added unit tests for qemu_opts_append() and
  qemu_opts_to_qdict_filtered()

- Patch 11 ('qdict: Introduce qdict_rename_keys()'):
  Added unit test, improved documentation

- Patch 12 ('qcow2: Use visitor for options in qcow2_create()'):
  Improved commit message, removed unnecessary movement of declaration

- Patches 13 and 14 ('block: x-blockdev-create QMP command'):
  Move making bdrv_is_whitelisted() public into a separate patch,
  use read-write driver whitelist instead of read-only

- Patch 17 ('gluster: Support .bdrv_co_create'):
  Rebased on top of preallocated truncate

- Patches 18-24 ('rbd: Support .bdrv_co_create'):
  QAPIfied .bdrv_open() implementation so that it can be shared with
  .bdrv_co_create() and specified servers are actually used instead of
  silently ignored

- Patch 25 ('nfs: Use QAPI options in nfs_client_open()'):
  Fixed use of uninitialised variable in the error path

- Patch 27 ('sheepdog: QAPIfy "redundacy" create option'):
  Addressed FIXME to use qemu_strtol()

- Patch 28 ('sheepdog: Support .bdrv_co_create'):
  Renamed 'backing_file' to 'backing-file', rebased on top of
  preallocated truncate


git-backport-diff compared to v1:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/36:[0001] [FC] 'block/qapi: Introduce BlockdevCreateOptions'
002/36:[] [-C] 'block/qapi: Add qcow2 create options to schema'
003/36:[] [--] 'qcow2: Let qcow2_create() handle protocol layer'
004/36:[] [--] 'qcow2: Pass BlockdevCreateOptions to qcow2_create2()'
005/36:[] [-C] 'qcow2: Use BlockdevRef in qcow2_create2()'
006/36:[] [--] 'qcow2: Use QCryptoBlockCreateOptions in qcow2_create2()'
007/36:[] [--] 'qcow2: Handle full/falloc preallocation in qcow2_create2()'
008/36:[0007] [FC] 'util: Add qemu_opts_to_qdict_filtered()'
009/36:[down] 'test-qemu-opts: Test qemu_opts_append()'
010/36:[down] 'test-qemu-opts: Test qemu_opts_to_qdict_filtered()'
011/36:[0117] [FC] 'qdict: Introduce qdict_rename_keys()'
012/36:[0002] [FC] 'qcow2: Use visitor for options in qcow2_create()'
013/36:[down] 'block: Make bdrv_is_whitelisted() public'
014/36:[0006] [FC] 'block: x-blockdev-create QMP command'
015/36:[] [-C] 'file-posix: Support .bdrv_co_create'
016/36:[] [--] 'file-win32: Support .bdrv_co_create'
017/36:[0130] [FC] 'gluster: Support .bdrv_co_create'
018/36:[down] 'rbd: Fix use after free in qemu_rbd_set_keypairs() error path'
019/36:[down] 'rbd: Factor out qemu_rbd_connect()'
020/36:[down] 'rbd: Remove non-schema options from runtime_opts'
021/36:[down] 'rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()'
022/36:[0034] [FC] 'rbd: Support .bdrv_co_create'
023/36:[down] 'rbd: Assing s->snap/image_name in qemu_rbd_open()'
024/36:[down] 'rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()'
025/36:[0002] [FC] 'nfs: Use QAPI options in nfs_client_open()'
026/36:[] [-C] 'nfs: Support .bdrv_co_create'
027/36:[0013] [FC] 'sheepdog: QAPIfy "redundacy" create option'
028/36:[0049] [FC] 'sheepdog: Support .bdrv_co_create'
029/36:[] [-C] 'ssh: Use 

[Qemu-devel] [PATCH v2 03/36] qcow2: Let qcow2_create() handle protocol layer

[Kevin Wolf]

Currently, qcow2_create() only parses the QemuOpts and then calls
qcow2_create2() for the actual image creation, which includes both the
creation of the actual file on the file system and writing a valid empty
qcow2 image into that file.

The plan is that qcow2_create2() becomes the function that implements
the functionality for a future 'blockdev-create' QMP command, which only
creates the qcow2 layer on an already opened file node.

This is a first step towards that goal: Let's move out anything that
deals with the protocol layer from qcow2_create2() into qcow2_create().
This means that qcow2_create2() doesn't need a file name any more.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 64 +++
 1 file changed, 38 insertions(+), 26 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 288b5299d8..dc6cdea113 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2725,7 +2725,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(const char *filename, int64_t total_size,
+static int qcow2_create2(BlockDriverState *bs, int64_t total_size,
  const char *backing_file, const char *backing_format,
  int flags, size_t cluster_size, PreallocMode prealloc,
  QemuOpts *opts, int version, int refcount_order,
@@ -2751,28 +2751,11 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 Error *local_err = NULL;
 int ret;
 
-if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
-int64_t prealloc_size =
-qcow2_calc_prealloc_size(total_size, cluster_size, refcount_order);
-qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, _abort);
-qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
- _abort);
-}
-
-ret = bdrv_create_file(filename, opts, _err);
+blk = blk_new(BLK_PERM_WRITE | BLK_PERM_RESIZE, BLK_PERM_ALL);
+ret = blk_insert_bs(blk, bs, errp);
 if (ret < 0) {
-error_propagate(errp, local_err);
-return ret;
-}
-
-blk = blk_new_open(filename, NULL, NULL,
-   BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL,
-   _err);
-if (blk == NULL) {
-error_propagate(errp, local_err);
-return -EIO;
+goto out;
 }
-
 blk_set_allow_write_beyond_eof(blk, true);
 
 /* Write the header */
@@ -2827,7 +2810,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  */
 options = qdict_new();
 qdict_put_str(options, "driver", "qcow2");
-blk = blk_new_open(filename, NULL, options,
+qdict_put_str(options, "file", bs->node_name);
+blk = blk_new_open(NULL, NULL, options,
BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_NO_FLUSH,
_err);
 if (blk == NULL) {
@@ -2899,7 +2883,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  */
 options = qdict_new();
 qdict_put_str(options, "driver", "qcow2");
-blk = blk_new_open(filename, NULL, options,
+qdict_put_str(options, "file", bs->node_name);
+blk = blk_new_open(NULL, NULL, options,
BDRV_O_RDWR | BDRV_O_NO_BACKING | BDRV_O_NO_IO,
_err);
 if (blk == NULL) {
@@ -2929,6 +2914,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 uint64_t refcount_bits;
 int refcount_order;
 char *encryptfmt = NULL;
+BlockDriverState *bs = NULL;
 Error *local_err = NULL;
 int ret;
 
@@ -2997,12 +2983,38 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 refcount_order = ctz32(refcount_bits);
 
-ret = qcow2_create2(filename, size, backing_file, backing_fmt, flags,
+/* Create and open the file (protocol layer) */
+if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
+int64_t prealloc_size =
+qcow2_calc_prealloc_size(size, cluster_size, refcount_order);
+qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, _abort);
+qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
+ _abort);
+}
+
+ret = bdrv_create_file(filename, opts, errp);
+if (ret < 0) {
+goto finish;
+}
+
+bs = bdrv_open(filename, NULL, NULL,
+   BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL, errp);
+if (bs == NULL) {
+ret = -EIO;
+goto finish;
+}
+
+/* Create the qcow2 image (format layer) */
+ret = qcow2_create2(bs, size, backing_file, backing_fmt, flags,
 cluster_size, prealloc, opts, version, refcount_order,
-encryptfmt, 

[Qemu-devel] [PATCH v2 11/36] qdict: Introduce qdict_rename_keys()

[Kevin Wolf]

A few block drivers will need to rename .bdrv_create options for their
QAPIfication, so let's have a helper function for that.

Signed-off-by: Kevin Wolf 
---
 include/qapi/qmp/qdict.h |   6 +++
 qobject/qdict.c  |  34 ++
 tests/check-qdict.c  | 113 +++
 3 files changed, 153 insertions(+)

diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index ff6f7842c3..7c6d844549 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -81,4 +81,10 @@ QObject *qdict_crumple(const QDict *src, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
+typedef struct QDictRenames {
+const char *from;
+const char *to;
+} QDictRenames;
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error 
**errp);
+
 #endif /* QDICT_H */
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 23df84f9cd..229b8c840b 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -1072,3 +1072,37 @@ void qdict_join(QDict *dest, QDict *src, bool overwrite)
 entry = next;
 }
 }
+
+/**
+ * qdict_rename_keys(): Rename keys in qdict according to the replacements
+ * specified in the array renames. The array must be terminated by an entry
+ * with from = NULL.
+ *
+ * The renames are performed individually in the order of the array, so entries
+ * may be renamed multiple times and may or may not conflict depending on the
+ * order of the renames array.
+ *
+ * Returns true for success, false in error cases.
+ */
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
+{
+QObject *qobj;
+
+while (renames->from) {
+if (qdict_haskey(qdict, renames->from)) {
+if (qdict_haskey(qdict, renames->to)) {
+error_setg(errp, "'%s' and its alias '%s' can't be used at the 
"
+   "same time", renames->to, renames->from);
+return false;
+}
+
+qobj = qdict_get(qdict, renames->from);
+qobject_incref(qobj);
+qdict_put_obj(qdict, renames->to, qobj);
+qdict_del(qdict, renames->from);
+}
+
+renames++;
+}
+return true;
+}
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index ec628f3453..5f8f3be9ff 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -665,6 +665,117 @@ static void qdict_crumple_test_empty(void)
 QDECREF(dst);
 }
 
+static void qdict_rename_keys_test(void)
+{
+QDict *dict = qdict_new();
+QDict *copy;
+QDictRenames *renames;
+Error *local_err = NULL;
+
+qdict_put_str(dict, "abc", "foo");
+qdict_put_str(dict, "abcdef", "bar");
+qdict_put_int(dict, "number", 42);
+qdict_put_bool(dict, "flag", true);
+qdict_put_null(dict, "nothing");
+
+/* Empty rename list */
+renames = (QDictRenames[]) {
+{ NULL, "this can be anything" }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, _abort);
+
+g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+
+QDECREF(copy);
+
+/* Simple rename of all entries */
+renames = (QDictRenames[]) {
+{ "abc","str1" },
+{ "abcdef", "str2" },
+{ "number", "int" },
+{ "flag",   "bool" },
+{ "nothing","null" },
+{ NULL , NULL }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, _abort);
+
+g_assert(!qdict_haskey(copy, "abc"));
+g_assert(!qdict_haskey(copy, "abcdef"));
+g_assert(!qdict_haskey(copy, "number"));
+g_assert(!qdict_haskey(copy, "flag"));
+g_assert(!qdict_haskey(copy, "nothing"));
+
+g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
+g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
+
+QDECREF(copy);
+
+/* Renames are processed top to bottom */
+renames = (QDictRenames[]) {
+{ "abc","tmp" },
+{ "abcdef", "abc" },
+{ "number", "abcdef" },
+{ "flag",   "number" },
+{ "nothing","flag" },
+{ "tmp","nothing" },
+{ NULL , NULL }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, _abort);
+
+g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
+

[Qemu-devel] [PATCH v2 17/36] gluster: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to gluster, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json |  18 ++-
 block/gluster.c  | 135 ++-
 2 files changed, 108 insertions(+), 45 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0040795603..74021c51d7 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3377,6 +3377,22 @@
 '*nocow':   'bool' } }
 
 ##
+# @BlockdevCreateOptionsGluster:
+#
+# Driver specific image creation options for gluster.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+# @preallocationPreallocation mode for the new image (default: off)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsGluster',
+  'data': { 'location': 'BlockdevOptionsGluster',
+'size': 'size',
+'*preallocation':   'PreallocMode' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3450,7 +3466,7 @@
   'file':   'BlockdevCreateOptionsFile',
   'ftp':'BlockdevCreateNotSupported',
   'ftps':   'BlockdevCreateNotSupported',
-  'gluster':'BlockdevCreateNotSupported',
+  'gluster':'BlockdevCreateOptionsGluster',
   'host_cdrom': 'BlockdevCreateNotSupported',
   'host_device':'BlockdevCreateNotSupported',
   'http':   'BlockdevCreateNotSupported',
diff --git a/block/gluster.c b/block/gluster.c
index 1a07d221d1..6e2f0e3185 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -655,9 +655,11 @@ out:
 return -errno;
 }
 
-static struct glfs *qemu_gluster_init(BlockdevOptionsGluster *gconf,
-  const char *filename,
-  QDict *options, Error **errp)
+/* Converts options given in @filename and the @options QDict into the QAPI
+ * object @gconf. */
+static int qemu_gluster_parse(BlockdevOptionsGluster *gconf,
+  const char *filename,
+  QDict *options, Error **errp)
 {
 int ret;
 if (filename) {
@@ -668,8 +670,7 @@ static struct glfs 
*qemu_gluster_init(BlockdevOptionsGluster *gconf,
 "[host[:port]]volume/path[?socket=...]"
 "[,file.debug=N]"
 "[,file.logfile=/path/filename.log]\n");
-errno = -ret;
-return NULL;
+return ret;
 }
 } else {
 ret = qemu_gluster_parse_json(gconf, options, errp);
@@ -685,10 +686,23 @@ static struct glfs 
*qemu_gluster_init(BlockdevOptionsGluster *gconf,
  "file.server.1.transport=unix,"
  "file.server.1.socket=/var/run/glusterd.socket 
..."
  "\n");
-errno = -ret;
-return NULL;
+return ret;
 }
+}
 
+return 0;
+}
+
+static struct glfs *qemu_gluster_init(BlockdevOptionsGluster *gconf,
+  const char *filename,
+  QDict *options, Error **errp)
+{
+int ret;
+
+ret = qemu_gluster_parse(gconf, filename, options, errp);
+if (ret < 0) {
+errno = -ret;
+return NULL;
 }
 
 return qemu_gluster_glfs_init(gconf, errp);
@@ -1021,19 +1035,71 @@ static int qemu_gluster_do_truncate(struct glfs_fd *fd, 
int64_t offset,
 return 0;
 }
 
-static int qemu_gluster_create(const char *filename,
-   QemuOpts *opts, Error **errp)
+static int qemu_gluster_co_create(BlockdevCreateOptions *options,
+  Error **errp)
 {
-BlockdevOptionsGluster *gconf;
+BlockdevCreateOptionsGluster *opts = >u.gluster;
 struct glfs *glfs;
 struct glfs_fd *fd = NULL;
 int ret = 0;
-PreallocMode prealloc;
-int64_t total_size = 0;
+
+assert(options->driver == BLOCKDEV_DRIVER_GLUSTER);
+
+glfs = qemu_gluster_glfs_init(opts->location, errp);
+if (!glfs) {
+ret = -errno;
+goto out;
+}
+
+fd = glfs_creat(glfs, opts->location->path,
+O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, S_IRUSR | 
S_IWUSR);
+if (!fd) {
+ret = -errno;
+goto out;
+}
+
+ret = qemu_gluster_do_truncate(fd, opts->size, opts->preallocation, errp);
+
+out:
+if (fd) {
+if (glfs_close(fd) != 0 && ret == 0) {
+ret = -errno;
+}
+}
+glfs_clear_preopened(glfs);
+return ret;
+}
+
+static int qemu_gluster_create(const char *filename,
+   QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions *options;
+BlockdevCreateOptionsGluster *gopts;
+BlockdevOptionsGluster 

[Qemu-devel] [PATCH v2 14/36] block: x-blockdev-create QMP command

[Kevin Wolf]

This adds a synchronous x-blockdev-create QMP command that can create
qcow2 images on a given node name.

We don't want to block while creating an image, so this is not the final
interface in all aspects, but BlockdevCreateOptionsQcow2 and
.bdrv_co_create() are what they actually might look like in the end. In
any case, this should be good enough to test whether we interpret
BlockdevCreateOptions as we should.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json  | 12 
 include/block/block_int.h |  2 ++
 block/create.c| 76 +++
 block/qcow2.c |  3 +-
 block/Makefile.objs   |  2 +-
 5 files changed, 93 insertions(+), 2 deletions(-)
 create mode 100644 block/create.c

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 74b864d64e..359195a1a3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3464,6 +3464,18 @@
   } }
 
 ##
+# @x-blockdev-create:
+#
+# Create an image format on a given node.
+# TODO Replace with something asynchronous (block job?)
+#
+# Since: 2.12
+##
+{ 'command': 'x-blockdev-create',
+  'data': 'BlockdevCreateOptions',
+  'boxed': true }
+
+##
 # @blockdev-open-tray:
 #
 # Opens a block device's tray. If there is a block driver state tree inserted 
as
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 5ae7738cf8..0b43fae782 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -128,6 +128,8 @@ struct BlockDriver {
 int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
   Error **errp);
 void (*bdrv_close)(BlockDriverState *bs);
+int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
+   Error **errp);
 int (*bdrv_create)(const char *filename, QemuOpts *opts, Error **errp);
 int (*bdrv_make_empty)(BlockDriverState *bs);
 
diff --git a/block/create.c b/block/create.c
new file mode 100644
index 00..dfd31eca37
--- /dev/null
+++ b/block/create.c
@@ -0,0 +1,76 @@
+/*
+ * Block layer code related to image creation
+ *
+ * Copyright (c) 2018 Kevin Wolf 
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qmp-commands.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevCreateCo {
+BlockDriver *drv;
+BlockdevCreateOptions *opts;
+int ret;
+Error **errp;
+} BlockdevCreateCo;
+
+static void coroutine_fn bdrv_co_create_co_entry(void *opaque)
+{
+BlockdevCreateCo *cco = opaque;
+cco->ret = cco->drv->bdrv_co_create(cco->opts, cco->errp);
+}
+
+void qmp_x_blockdev_create(BlockdevCreateOptions *options, Error **errp)
+{
+const char *fmt = BlockdevDriver_str(options->driver);
+BlockDriver *drv = bdrv_find_format(fmt);
+Coroutine *co;
+BlockdevCreateCo cco;
+
+/* If the driver is in the schema, we know that it exists. But it may not
+ * be whitelisted. */
+assert(drv);
+if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+error_setg(errp, "Driver is not whitelisted");
+return;
+}
+
+/* Call callback if it exists */
+if (!drv->bdrv_co_create) {
+error_setg(errp, "Driver does not support blockdev-create");
+return;
+}
+
+cco = (BlockdevCreateCo) {
+.drv = drv,
+.opts = options,
+.ret = -EINPROGRESS,
+.errp = errp,
+};
+
+co = qemu_coroutine_create(bdrv_co_create_co_entry, );
+qemu_coroutine_enter(co);
+while (cco.ret == -EINPROGRESS) {
+aio_poll(qemu_get_aio_context(), true);
+}
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 58737d0833..8acb36b0af 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4463,7 +4463,8 @@ BlockDriver bdrv_qcow2 = {
 .bdrv_reopen_abort= qcow2_reopen_abort,
 .bdrv_join_options= 

[Qemu-devel] [PATCH v2 26/36] nfs: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to nfs, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 16 +++-
 block/nfs.c  | 74 +---
 2 files changed, 74 insertions(+), 16 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 6c0c16ebe3..085b791303 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3393,6 +3393,20 @@
 '*preallocation':   'PreallocMode' } }
 
 ##
+# @BlockdevCreateOptionsNfs:
+#
+# Driver specific image creation options for NFS.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsNfs',
+  'data': { 'location': 'BlockdevOptionsNfs',
+'size': 'size' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3491,7 +3505,7 @@
   'iscsi':  'BlockdevCreateNotSupported',
   'luks':   'BlockdevCreateNotSupported',
   'nbd':'BlockdevCreateNotSupported',
-  'nfs':'BlockdevCreateNotSupported',
+  'nfs':'BlockdevCreateOptionsNfs',
   'null-aio':   'BlockdevCreateNotSupported',
   'null-co':'BlockdevCreateNotSupported',
   'nvme':   'BlockdevCreateNotSupported',
diff --git a/block/nfs.c b/block/nfs.c
index 9283bfbaae..c0c153cadb 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -551,33 +551,45 @@ out:
 return ret;
 }
 
-static int64_t nfs_client_open_qdict(NFSClient *client, QDict *options,
- int flags, int open_flags, Error **errp)
+static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options,
+ Error **errp)
 {
 BlockdevOptionsNfs *opts = NULL;
 QObject *crumpled = NULL;
 Visitor *v;
 Error *local_err = NULL;
-int ret;
 
 crumpled = qdict_crumple(options, errp);
 if (crumpled == NULL) {
-return -EINVAL;
+return NULL;
 }
 
 v = qobject_input_visitor_new_keyval(crumpled);
 visit_type_BlockdevOptionsNfs(v, NULL, , _err);
 visit_free(v);
+qobject_decref(crumpled);
 
 if (local_err) {
-error_propagate(errp, local_err);
+return NULL;
+}
+
+return opts;
+}
+
+static int64_t nfs_client_open_qdict(NFSClient *client, QDict *options,
+ int flags, int open_flags, Error **errp)
+{
+BlockdevOptionsNfs *opts;
+int ret;
+
+opts = nfs_options_qdict_to_qapi(options, errp);
+if (opts == NULL) {
 ret = -EINVAL;
 goto fail;
 }
 
 ret = nfs_client_open(client, opts, flags, open_flags, errp);
 fail:
-qobject_decref(crumpled);
 qapi_free_BlockdevOptionsNfs(opts);
 return ret;
 }
@@ -614,17 +626,42 @@ static QemuOptsList nfs_create_opts = {
 }
 };
 
-static int nfs_file_create(const char *url, QemuOpts *opts, Error **errp)
+static int nfs_file_co_create(BlockdevCreateOptions *options, Error **errp)
 {
-int64_t ret, total_size;
+BlockdevCreateOptionsNfs *opts = >u.nfs;
 NFSClient *client = g_new0(NFSClient, 1);
-QDict *options = NULL;
+int ret;
+
+assert(options->driver == BLOCKDEV_DRIVER_NFS);
 
 client->aio_context = qemu_get_aio_context();
 
+ret = nfs_client_open(client, opts->location, O_CREAT, 0, errp);
+if (ret < 0) {
+goto out;
+}
+ret = nfs_ftruncate(client->context, client->fh, opts->size);
+nfs_client_close(client);
+
+out:
+g_free(client);
+return ret;
+}
+
+static int nfs_file_create(const char *url, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions *create_options;
+BlockdevCreateOptionsNfs *nfs_opts;
+QDict *options;
+int ret;
+
+create_options = g_new0(BlockdevCreateOptions, 1);
+create_options->driver = BLOCKDEV_DRIVER_NFS;
+nfs_opts = _options->u.nfs;
+
 /* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
+nfs_opts->size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
 
 options = qdict_new();
 ret = nfs_parse_uri(url, options, errp);
@@ -632,15 +669,21 @@ static int nfs_file_create(const char *url, QemuOpts 
*opts, Error **errp)
 goto out;
 }
 
-ret = nfs_client_open_qdict(client, options, O_CREAT, 0, errp);
+nfs_opts->location = nfs_options_qdict_to_qapi(options, errp);
+if (nfs_opts->location == NULL) {
+ret = -EINVAL;
+goto out;
+}
+
+ret = nfs_file_co_create(create_options, errp);
 if (ret < 0) {
 goto out;
 }
-ret = nfs_ftruncate(client->context, client->fh, total_size);
-

Re: [Qemu-devel] [PATCH] hw/acpi-build: build SRAT memory affinity structures for NVDIMM

[Igor Mammedov]

On Tue, 20 Feb 2018 17:17:58 -0800
Dan Williams  wrote:

> On Tue, Feb 20, 2018 at 6:10 AM, Igor Mammedov  wrote:
> > On Sat, 17 Feb 2018 14:31:35 +0800
> > Haozhong Zhang  wrote:
> >  
> >> ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity
> >> domain of a NVDIMM SPA range must match with corresponding entry in
> >> SRAT table.
> >>
> >> The address ranges of vNVDIMM in QEMU are allocated from the
> >> hot-pluggable address space, which is entirely covered by one SRAT
> >> memory affinity structure. However, users can set the vNVDIMM
> >> proximity domain in NFIT SPA range structure by the 'node' property of
> >> '-device nvdimm' to a value different than the one in the above SRAT
> >> memory affinity structure.
> >>
> >> In order to solve such proximity domain mismatch, this patch build one
> >> SRAT memory affinity structure for each NVDIMM device with the
> >> proximity domain used in NFIT. The remaining hot-pluggable address
> >> space is covered by one or multiple SRAT memory affinity structures
> >> with the proximity domain of the last node as before.
> >>
> >> Signed-off-by: Haozhong Zhang   
> > If we consider hotpluggable system, correctly implemented OS should
> > be able pull proximity from Device::_PXM and override any value from SRAT.
> > Do we really have a problem here (anything that breaks if we would use 
> > _PXM)?
> > Maybe we should add _PXM object to nvdimm device nodes instead of massaging 
> > SRAT?  
> 
> Unfortunately _PXM is an awkward fit. Currently the proximity domain
> is attached to the SPA range structure. The SPA range may be
> associated with multiple DIMM devices and those individual NVDIMMs may
> have conflicting _PXM properties.
There shouldn't be any conflict here as  NVDIMM device's _PXM method,
should override in runtime any proximity specified by parent scope.
(as parent scope I'd also count boot time NFIT/SRAT tables).

To make it more clear we could clear valid proximity domain flag in SPA
like this:

diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index 59d6e42..131bca5 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -260,9 +260,7 @@ nvdimm_build_structure_spa(GArray *structures, DeviceState 
*dev)
  */
 nfit_spa->flags = cpu_to_le16(1 /* Control region is strictly for
management during hot add/online
-   operation */ |
-  2 /* Data in Proximity Domain field is
-   valid*/);
+   operation */);
 
 /* NUMA node. */
 nfit_spa->proximity_domain = cpu_to_le32(node);

> Even if that was unified across
> DIMMs it is ambiguous whether a DIMM-device _PXM would relate to the
> device's control interface, or the assembled persistent memory SPA
> range.
I'm not sure what you mean under 'device's control interface',
could you clarify where the ambiguity comes from?

I read spec as: _PXM applies to address range covered by NVDIMM
device it belongs to.

As for assembled SPA, I'd assume that it applies to interleaved set
and all NVDIMMs with it should be on the same node. It's somewhat
irrelevant question though as QEMU so far implements only
  1:1:1/SPA:Region Mapping:NVDIMM Device/
mapping.

My main concern with using static configuration tables for proximity
mapping, we'd miss on hotplug side of equation. However if we start
from dynamic side first, we could later complement it with static
tables if there really were need for it.

[Qemu-devel] [PATCH v2 35/36] qemu-iotests: Test qcow2 over file image creation with QMP

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/206 | 436 +
 tests/qemu-iotests/206.out | 209 ++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 646 insertions(+)
 create mode 100755 tests/qemu-iotests/206
 create mode 100644 tests/qemu-iotests/206.out

diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
new file mode 100755
index 00..0a18b2b19a
--- /dev/null
+++ b/tests/qemu-iotests/206
@@ -0,0 +1,436 @@
+#!/bin/bash
+#
+# Test qcow2 and file image creation
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=kw...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+function do_run_qemu()
+{
+echo Testing: "$@"
+$QEMU -nographic -qmp stdio -serial none "$@"
+echo
+}
+
+function run_qemu()
+{
+do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \
+  | _filter_qemu | _filter_imgfmt \
+  | _filter_actual_image_size
+}
+
+echo
+echo "=== Successful image creation (defaults) ==="
+echo
+
+size=$((128 * 1024 * 1024))
+
+run_qemu <

Re: [Qemu-devel] [PATCH v4 4/5] usb-mtp: Introduce write support for MTP objects

[Daniel P . Berrangé]

On Wed, Feb 21, 2018 at 12:11:00PM +0100, Gerd Hoffmann wrote:
> > > +static void usb_mtp_write_data(MTPState *s)
> > > +{
> > > +MTPData *d = s->data_out;
> > > +MTPObject *parent =
> > > +usb_mtp_object_lookup(s, s->dataset.parent_handle);
> > > +char *path = NULL;
> > > +int rc = -1;
> > > +mode_t mask = 0644;
> > > +
> > > +assert(d != NULL);
> > > +
> > 
> > 
> > Somewhere in here should surely be validating the "readonly" flag.
> > 
> > > +if (parent == NULL || !s->write_pending) {
> 
> Does happens here.  With a readonly device write_pending should
> never be true.

Unless I'm mis-understanding the flow, the next patch appears to set
write_pending = true, in response to a guest command, without checking
the readonly flag.

> 
> > > +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO, d->trans,
> > > + 0, 0, 0, 0);
> > > +return;
> > > +}
> 
> But adding an "assert(!readonly)" here as double-check surely doesn't hurt.
> 
> cheers,
>   Gerd
> 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH] nbd: Honor server's advertised minimum block size

[Eric Blake]

On 02/16/2018 05:50 AM, Vladimir Sementsov-Ogievskiy wrote:

15.02.2018 06:29, Eric Blake wrote:

Commit 79ba8c98 (v2.7) changed the setting of request_alignment
to occur only during bdrv_refresh_limits(), rather than at at
bdrv_open() time; but at the time, NBD was unaffected, because
it still used sector-based callbacks, so the block layer
defaulted NBD to use 512 request_alignment.




Fix these issues by moving the assignment to request_alignment
to the right function, and by using a sane default when the
server does not advertise a minimum size.

CC: qemu-sta...@nongnu.org
Signed-off-by: Eric Blake 
---



Reviewed-by: Vladimir Sementsov-Ogievskiy


Thanks; applied to my NBD queue.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v2 2/5] keymap: use glib hash for kbd_layout_t

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:03:54PM +0100, Gerd Hoffmann wrote:
> Drop home-grown lookup code, which is a strange mix of a lookup table
> and a list.  Use standard glib hash instead.
> 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/keymaps.c| 79 
> +++--
>  ui/trace-events |  2 +-
>  2 files changed, 38 insertions(+), 43 deletions(-)


> 
> diff --git a/ui/keymaps.c b/ui/keymaps.c
> index 134958a197..bef1405576 100644
> --- a/ui/keymaps.c
> +++ b/ui/keymaps.c
> @@ -28,26 +28,28 @@
>  #include "trace.h"
>  #include "qemu/error-report.h"
>  
> -#define MAX_NORMAL_KEYCODE 512
> -#define MAX_EXTRA_COUNT 256
> -
>  struct key_range {
>  int start;
>  int end;
>  struct key_range *next;
>  };
>  
> +struct keysym2code {
> +uint16_t keycode;
> +};
> +
>  struct kbd_layout_t {
> -uint16_t keysym2keycode[MAX_NORMAL_KEYCODE];
> -struct {
> -int keysym;
> -uint16_t keycode;
> -} keysym2keycode_extra[MAX_EXTRA_COUNT];
> -int extra_count;
> +GHashTable *hash;
>  struct key_range *keypad_range;
>  struct key_range *numlock_range;
>  };
>  
> +static inline gpointer hashkey(int keysym)
> +{
> +intptr_t ptr = keysym;
> +return (gpointer)(ptr);
> +}
> +
>  static int get_keysym(const name2keysym_t *table,
>const char *name)
>  {
> @@ -91,23 +93,19 @@ static void add_to_key_range(struct key_range **krp, int 
> code) {
>  }
>  }
>  
> -static void add_keysym(char *line, int keysym, int keycode, kbd_layout_t *k) 
> {
> -if (keysym < MAX_NORMAL_KEYCODE) {
> -trace_keymap_add("normal", keysym, keycode, line);
> -k->keysym2keycode[keysym] = keycode;
> -} else {
> -if (k->extra_count >= MAX_EXTRA_COUNT) {
> -warn_report("Could not assign keysym %s (0x%x)"
> -" because of memory constraints.", line, keysym);
> -} else {
> -trace_keymap_add("extra", keysym, keycode, line);
> -k->keysym2keycode_extra[k->extra_count].
> -keysym = keysym;
> -k->keysym2keycode_extra[k->extra_count].
> -keycode = keycode;
> -k->extra_count++;
> -}
> +static void add_keysym(char *line, int keysym, int keycode, kbd_layout_t *k)
> +{
> +struct keysym2code *keysym2code;
> +
> +keysym2code = g_hash_table_lookup(k->hash, hashkey(keysym));

FYI, the glib2 GINT_TO_POINTER() macro is intended to cover this
usecase of storing integers as hash keys. 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

[Eric Blake]

On 02/21/2018 04:04 AM, Alberto Garcia wrote:

On Tue 20 Feb 2018 11:24:59 PM CET, Eric Blake wrote:

I was also preparing a patch to change this, but you arrived first :-)


So, it's time to cut back on the waste.  A compressed cluster
will NEVER occupy more than an uncompressed cluster (okay, gzip
DOES document that because the compression stream adds metadata,
and because of the pigeonhole principle, there are worst case
scenarios where attempts to compress will actually inflate an
image - but in those cases, we would just write the cluster
uncompressed instead of inflating it).  And as that is a smaller
amount of memory, we can get by with the simpler g_malloc.



-if (!s->cluster_cache) {
-s->cluster_cache = g_malloc(s->cluster_size);
+assert(!s->cluster_cache);
+s->cluster_data = g_try_malloc(s->cluster_size);
+s->cluster_cache = g_try_malloc(s->cluster_size);


Shoot - I made edits that I forgot to commit before sending; I meant for 
these to be g_malloc() rather than g_try_malloc().




There's a few things here:

- QEMU won't write compressed data if the size is >= s->cluster_size
   (there's an explicit check for that in qcow2_co_pwritev_compressed())


Correct, we never cause inflation (and even if we wanted to, we can't, 
because the qcow2 format doesn't have enough bits for us to record that 
many sectors for a compressed stream that occupies more space than the 
original cluster).




- The size field of the compressed cluster descriptor *does* allow
   larger sizes, so you can't simply read csize bytes into
   s->cluster_data becuase you could cause a buffer overflow.


Let's step through this:

  nb_csectors = ((cluster_offset >> s->csize_shift) & s->csize_mask + 1;

Since s->csize_mask is determined by s->cluster_size / 512, then after 
this assignment, the most nb_csectors can be is exactly s->cluster_size 
/ 512.


  sector_offset = coffset & 511;
  csize = nb_csectors * 512 - sector_offset;

And here, csize can only get smaller than the length picked by 
nb_csectors.  Therefore, csize is GUARANTEED to be <= c->sector_size.




- Solution a: check that csize < s->cluster_size and return an error if
   it's not. However! although QEMU won't produce an image with a
   compressed cluster that is larger than the uncompressed one, the qcow2
   on-disk format in principle allows for that, so arguably we should
   accept it.


No, the qcow2 on-disk format does not have enough bits reserved for 
that.  It is impossible to store an inflated cluster, because you don't 
have enough bits to record it.


That said, we MAY have a bug, more likely to be visible in 512-byte 
clusters but possible on any size.  While the 'number sectors' field IS 
sufficient for any compressed cluster starting at offset 0 in the 
cluster, we run into issues if the starting offset is later in the 
cluster.  That is, even though the compressed length of a 512-byte 
cluster is <= 512 (or we wouldn't compress it), if we start at offset 
510, we NEED to read the next cluster to get the rest of the compressed 
stream - but at 512-byte clusters, there are 0 bits reserved for 'number 
sectors'.  Per the above calculations with the example offset of 510, 
nb_csectors would be 1 (it can't be anything else for a 512-byte cluster 
image), and csize would then be 2 bytes, which is insufficient for 
reading back enough data to reconstruct the cluster.


We probably need to clarify in the spec that if the starting offset of a 
compressed cluster falls mid-sector, then the compressed size has to be 
smaller than cluster size - (offset % 512) (this requirement is already 
there implicitly due to the field widths, but being explicit can't 
hurt).  We probably also need to fix our compression code to actually do 
the right thing, particularly for 512-byte clusters where we are most 
likely to run into a compressed size that is likely to overflow the 
space available for nb_csectors.




- Solution b: the width of the 'compressed cluster size' field is
   (cluster_bits - 8), that's (cluster_size / 256) sectors.


Not true.  It is (cluster_bits - 9) or (cluster_size / 512).  Remember, 
x = 62 - (cluster_bits - 8); for a 512-byte cluster, x = 61.  The 
'number sectors' field is then bits x+1 - 61 (but you can't have a 
bitfield occupying bit 62 upto 61; especially since bit 62 is the bit 
for compressed cluster).



Since the
   size of each sector is 512 bytes, the maximum possible size that the
   field can store is (cluster_size * 2) bytes. So allocate that amount
   of space for s->cluster_data, read the data as it is on disk and let
   the decompression code return an error if the data is indeed
   corrupted or it doesn't fit in the output buffer.


Again, I argue that the qcow2 spec says that the maximum size for a 
compressed cluster is cluster_size, even if it spills over a host 
cluster boundary.  But if in practice, we HAVE allowed a spillover 
beyond the 'number fields' 

[Qemu-devel] [PATCH v3 6/7] net: Remove the deprecated 'host_net_add' and 'host_net_remove' HMP commands

[Thomas Huth]

They are deprecated since QEMU v2.10, and so far nobody complained that
these commands are still necessary for any reason - and since you can use
'netdev_add' and 'netdev_remove' instead, there also should not be any
real reason. Since they are also standing in the way for the upcoming
'vlan' clean-up, it's now time to remove them.

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 hmp-commands.hx  | 30 --
 hmp.h|  3 --
 monitor.c| 61 
 net/net.c| 94 
 qemu-doc.texi| 10 --
 tests/test-hmp.c |  2 --
 6 files changed, 200 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index d26eb41..964eb51 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1291,36 +1291,6 @@ Inject PCIe AER error
 ETEXI
 
 {
-.name   = "host_net_add",
-.args_type  = "device:s,opts:s?",
-.params = "tap|user|socket|vde|netmap|bridge|vhost-user|dump 
[options]",
-.help   = "add host VLAN client (deprecated, use netdev_add 
instead)",
-.cmd= hmp_host_net_add,
-.command_completion = host_net_add_completion,
-},
-
-STEXI
-@item host_net_add
-@findex host_net_add
-Add host VLAN client. Deprecated, please use @code{netdev_add} instead.
-ETEXI
-
-{
-.name   = "host_net_remove",
-.args_type  = "vlan_id:i,device:s",
-.params = "vlan_id name",
-.help   = "remove host VLAN client (deprecated, use netdev_del 
instead)",
-.cmd= hmp_host_net_remove,
-.command_completion = host_net_remove_completion,
-},
-
-STEXI
-@item host_net_remove
-@findex host_net_remove
-Remove host VLAN client. Deprecated, please use @code{netdev_del} instead.
-ETEXI
-
-{
 .name   = "netdev_add",
 .args_type  = "netdev:O",
 .params = 
"[user|tap|socket|vde|bridge|hubport|netmap|vhost-user],id=str[,prop=value][,...]",
diff --git a/hmp.h b/hmp.h
index 1143db4..b897338 100644
--- a/hmp.h
+++ b/hmp.h
@@ -132,9 +132,6 @@ void migrate_set_capability_completion(ReadLineState *rs, 
int nb_args,
const char *str);
 void migrate_set_parameter_completion(ReadLineState *rs, int nb_args,
   const char *str);
-void host_net_add_completion(ReadLineState *rs, int nb_args, const char *str);
-void host_net_remove_completion(ReadLineState *rs, int nb_args,
-const char *str);
 void delvm_completion(ReadLineState *rs, int nb_args, const char *str);
 void loadvm_completion(ReadLineState *rs, int nb_args, const char *str);
 void hmp_rocker(Monitor *mon, const QDict *qdict);
diff --git a/monitor.c b/monitor.c
index 373bb8d..a4f1f28 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3574,67 +3574,6 @@ void migrate_set_parameter_completion(ReadLineState *rs, 
int nb_args,
 }
 }
 
-void host_net_add_completion(ReadLineState *rs, int nb_args, const char *str)
-{
-int i;
-size_t len;
-if (nb_args != 2) {
-return;
-}
-len = strlen(str);
-readline_set_completion_index(rs, len);
-for (i = 0; host_net_devices[i]; i++) {
-if (!strncmp(host_net_devices[i], str, len)) {
-readline_add_completion(rs, host_net_devices[i]);
-}
-}
-}
-
-void host_net_remove_completion(ReadLineState *rs, int nb_args, const char 
*str)
-{
-NetClientState *ncs[MAX_QUEUE_NUM];
-int count, i, len;
-
-len = strlen(str);
-readline_set_completion_index(rs, len);
-if (nb_args == 2) {
-count = qemu_find_net_clients_except(NULL, ncs,
- NET_CLIENT_DRIVER_NONE,
- MAX_QUEUE_NUM);
-for (i = 0; i < MIN(count, MAX_QUEUE_NUM); i++) {
-int id;
-char name[16];
-
-if (net_hub_id_for_client(ncs[i], )) {
-continue;
-}
-snprintf(name, sizeof(name), "%d", id);
-if (!strncmp(str, name, len)) {
-readline_add_completion(rs, name);
-}
-}
-return;
-} else if (nb_args == 3) {
-count = qemu_find_net_clients_except(NULL, ncs,
- NET_CLIENT_DRIVER_NIC,
- MAX_QUEUE_NUM);
-for (i = 0; i < MIN(count, MAX_QUEUE_NUM); i++) {
-int id;
-const char *name;
-
-if (ncs[i]->info->type == NET_CLIENT_DRIVER_HUBPORT ||
-net_hub_id_for_client(ncs[i], )) {
-continue;
-}
-name = ncs[i]->name;
-if (!strncmp(str, name, len)) {
-readline_add_completion(rs, name);
-}
-}
-return;
-}
-}
-
 static void vm_completion(ReadLineState 

[Qemu-devel] [PATCH v3 0/7] Improvements and clean-ups related to -net

[Thomas Huth]

"-net" is a legacy option that often causes confusion and
misconfigurations for the users, since most people are not aware
of the underlying "vlan" (i.e. hub) concept that is used for this
parameter. The prefered way of configuring your network stack is
to use "--netdev" instead, which gives you a clean 1:1 connection
between your emulated guest hardware and the host network backend.

However, there are two reasons why we could not completely deprecate
"-net" yet:

1) Convenience:
In some cases, it's more convenient to use "-net" instead of "--netdev",
e.g. if you just want to have a "tap" network connection, it's faster
to type "-net nic -net tap" instead of "--device e1000,netdev=n1
--netdev tap,id=n1".

2) On-board NICs:
Currently the "-net nic" parameter is the only way to configure on-
board NICs on certain (embedded) machines via the nd_table[] array.

So beside some generic clean-ups and removal of code that has been
marked as deprecated since QEMU 2.10 already, this patch series intro-
duces a new parameter "--nic" (in patch 7) which should be able to re-
place "-net" in the long run completely: This new convenience parameter
can be used to configure the default/on-board guest HW together with a
host network backend in a very compact way. To configure a tap backend
for the default NIC, you just have to type "--nic tap" here for example.

Note that "-net" itself is not marked as deprecated yet - that is
subject to a later patch, since it likely needs some additional
discussion first (or we might rather want to wait for some more
releases first until --nic has been properly established instead)...

v3:
- Changed the comments in qapi/net.json in patch 5/7 according
  to the suggestion from Eric. Dropped the Reviewed-bys from this
  patch since the text changed a bit now.
- Dropped the final patch 8 with the updates to qemu-doc about
  making -net less prominent. I'll resubmit that patch together
  with a patch that deprecates "-net" in a separate patch series
  instead (which will likely both need some more discussion first).

v2:
- Renamed "-n" to "--nic" (suggested by Paolo)
- Improved the QAPI comment about the removal of 'dump (suggested by Eric)
- exit(0) instead of exit(1) after showing the "-netdev help" (Eric)
- Improved the documenation for hubport a little bit (suggested by Paolo)

Thomas Huth (7):
  net: Move error reporting from net_init_client/netdev to the calling
site
  net: List available netdevs with "-netdev help"
  net: Only show vhost-user in the help text if CONFIG_POSIX is defined
  net: Make net_client_init() static
  net: Remove the deprecated way of dumping network packets
  net: Remove the deprecated 'host_net_add' and 'host_net_remove' HMP
commands
  net: Add a new convenience option "--nic" to configure
default/on-board NICs

 hmp-commands.hx |  30 --
 hmp.h   |   3 -
 include/net/net.h   |   4 +-
 include/sysemu/sysemu.h |   1 +
 monitor.c   |  61 
 net/dump.c  | 102 +
 net/net.c   | 239 +++-
 qapi/net.json   |  29 ++
 qemu-doc.texi   |  16 
 qemu-options.hx |  48 +++---
 tests/test-hmp.c|   2 -
 vl.c|  10 +-
 12 files changed, 170 insertions(+), 375 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH v3 3/7] net: Only show vhost-user in the help text if CONFIG_POSIX is defined

[Thomas Huth]

According to net/Makefile.objs we only link in the vhost-user code
if CONFIG_POSIX has been set. So the help screen should also only
show this information if CONFIG_POSIX has been defined.

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 qemu-options.hx | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qemu-options.hx b/qemu-options.hx
index 8ccd5dc..9ae49a0 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1998,8 +1998,10 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
 "VALE port (created on the fly) called 'name' ('nmname' is 
name of the \n"
 "netmap device, defaults to '/dev/netmap')\n"
 #endif
+#ifdef CONFIG_POSIX
 "-netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off]\n"
 "configure a vhost-user network, backed by a chardev 
'dev'\n"
+#endif
 "-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
 "configure a hub port on QEMU VLAN 'n'\n", QEMU_ARCH_ALL)
 DEF("net", HAS_ARG, QEMU_OPTION_net,
-- 
1.8.3.1

[Qemu-devel] [PULL 00/22] re-factor softfloat and add fp16 functions

[Alex Bennée]

The following changes since commit a6e0344fa0e09413324835ae122c4cadd7890231:

  Merge remote-tracking branch 'remotes/kraxel/tags/ui-20180220-pull-request' 
into staging (2018-02-20 14:05:00 +)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-softfloat-refactor-210218-1

for you to fetch changes up to c13bb2da9eedfbc5886c8048df1bc1114b285fb0:

  fpu/softfloat: re-factor sqrt (2018-02-21 10:21:54 +)


This is the re-factor of softfloat:

  - shared common code path float16/32/64
  - well commented and easy to follow code
  - added a bunch of float16 support

While some operations are slower the key ones exercised by the
floating point dbt-bench are the same: https://i.imgur.com/oXNJNql.png


Alex Bennée (22):
  fpu/softfloat: implement float16_squash_input_denormal
  include/fpu/softfloat: remove USE_SOFTFLOAT_STRUCT_TYPES
  fpu/softfloat-types: new header to prevent excessive re-builds
  target/*/cpu.h: remove softfloat.h
  include/fpu/softfloat: implement float16_abs helper
  include/fpu/softfloat: implement float16_chs helper
  include/fpu/softfloat: implement float16_set_sign helper
  include/fpu/softfloat: add some float16 constants
  fpu/softfloat: improve comments on ARM NaN propagation
  fpu/softfloat: move the extract functions to the top of the file
  fpu/softfloat: define decompose structures
  fpu/softfloat: re-factor add/sub
  fpu/softfloat: re-factor mul
  fpu/softfloat: re-factor div
  fpu/softfloat: re-factor muladd
  fpu/softfloat: re-factor round_to_int
  fpu/softfloat: re-factor float to int/uint
  fpu/softfloat: re-factor int/uint to float
  fpu/softfloat: re-factor scalbn
  fpu/softfloat: re-factor minmax
  fpu/softfloat: re-factor compare
  fpu/softfloat: re-factor sqrt

 fpu/softfloat-macros.h  |   48 +
 fpu/softfloat-specialize.h  |  109 +-
 fpu/softfloat.c | 4550 ---
 include/fpu/softfloat-types.h   |  179 ++
 include/fpu/softfloat.h |  202 +-
 include/qemu/bswap.h|2 +-
 target/alpha/cpu.h  |2 -
 target/arm/cpu.c|1 +
 target/arm/cpu.h|2 -
 target/arm/helper-a64.c |1 +
 target/arm/helper.c |1 +
 target/arm/neon_helper.c|1 +
 target/hppa/cpu.c   |1 +
 target/hppa/cpu.h   |1 -
 target/hppa/op_helper.c |2 +-
 target/i386/cpu.h   |4 -
 target/i386/fpu_helper.c|1 +
 target/m68k/cpu.c   |2 +-
 target/m68k/cpu.h   |1 -
 target/m68k/fpu_helper.c|1 +
 target/m68k/helper.c|1 +
 target/m68k/translate.c |2 +
 target/microblaze/cpu.c |1 +
 target/microblaze/cpu.h |2 +-
 target/microblaze/op_helper.c   |1 +
 target/moxie/cpu.h  |1 -
 target/nios2/cpu.h  |1 -
 target/openrisc/cpu.h   |1 -
 target/openrisc/fpu_helper.c|1 +
 target/ppc/cpu.h|1 -
 target/ppc/fpu_helper.c |1 +
 target/ppc/int_helper.c |1 +
 target/ppc/translate_init.c |1 +
 target/s390x/cpu.c  |1 +
 target/s390x/cpu.h  |2 -
 target/s390x/fpu_helper.c   |1 +
 target/sh4/cpu.c|1 +
 target/sh4/cpu.h|2 -
 target/sh4/op_helper.c  |1 +
 target/sparc/cpu.h  |2 -
 target/sparc/fop_helper.c   |1 +
 target/tricore/cpu.h|1 -
 target/tricore/fpu_helper.c |1 +
 target/tricore/helper.c |1 +
 target/unicore32/cpu.c  |1 +
 target/unicore32/cpu.h  |1 -
 target/unicore32/ucf64_helper.c |1 +
 target/xtensa/cpu.h |1 -
 target/xtensa/op_helper.c   |1 +
 49 files changed, 2204 insertions(+), 2941 deletions(-)
 create mode 100644 include/fpu/softfloat-types.h


-- 
2.15.1

[Qemu-devel] [PATCH 02/12] sdl: switch over to new display registry

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 19 ---
 ui/sdl.c | 24 +---
 ui/sdl2.c| 17 +++--
 vl.c | 15 +--
 4 files changed, 29 insertions(+), 46 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index ce3589aadd..82bbea0242 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -441,25 +441,6 @@ void qemu_display_register(QemuDisplay *ui);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
-/* sdl.c */
-#ifdef CONFIG_SDL
-void sdl_display_early_init(DisplayOptions *opts);
-void sdl_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void sdl_display_early_init(DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_SDL is disabled */
-error_report("SDL support is disabled");
-abort();
-}
-static inline void sdl_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_SDL is disabled */
-error_report("SDL support is disabled");
-abort();
-}
-#endif
-
 /* cocoa.m */
 #ifdef CONFIG_COCOA
 void cocoa_display_init(DisplayState *ds, DisplayOptions *opts);
diff --git a/ui/sdl.c b/ui/sdl.c
index 963cdf77a7..153cbc6d1c 100644
--- a/ui/sdl.c
+++ b/ui/sdl.c
@@ -897,17 +897,7 @@ static const DisplayChangeListenerOps dcl_ops = {
 .dpy_cursor_define= sdl_mouse_define,
 };
 
-void sdl_display_early_init(DisplayOptions *opts)
-{
-if (opts->has_gl && opts->gl) {
-fprintf(stderr,
-"SDL1 display code has no opengl support.\n"
-"Please recompile qemu with SDL2, using\n"
-"./configure --enable-sdl --with-sdlabi=2.0\n");
-}
-}
-
-void sdl_display_init(DisplayState *ds, DisplayOptions *o)
+static void sdl1_display_init(DisplayState *ds, DisplayOptions *o)
 {
 int flags;
 uint8_t data = 0;
@@ -1019,3 +1009,15 @@ void sdl_display_init(DisplayState *ds, DisplayOptions 
*o)
 
 atexit(sdl_cleanup);
 }
+
+static QemuDisplay qemu_display_sdl1 = {
+.type   = DISPLAY_TYPE_SDL,
+.init   = sdl1_display_init,
+};
+
+static void register_sdl1(void)
+{
+qemu_display_register(_display_sdl1);
+}
+
+type_init(register_sdl1);
diff --git a/ui/sdl2.c b/ui/sdl2.c
index 6e96a4a24c..f17d039650 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -759,7 +759,7 @@ static const DisplayChangeListenerOps dcl_gl_ops = {
 };
 #endif
 
-void sdl_display_early_init(DisplayOptions *o)
+static void sdl2_display_early_init(DisplayOptions *o)
 {
 assert(o->type == DISPLAY_TYPE_SDL);
 if (o->has_gl && o->gl) {
@@ -769,7 +769,7 @@ void sdl_display_early_init(DisplayOptions *o)
 }
 }
 
-void sdl_display_init(DisplayState *ds, DisplayOptions *o)
+static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
 {
 int flags;
 uint8_t data = 0;
@@ -869,3 +869,16 @@ void sdl_display_init(DisplayState *ds, DisplayOptions *o)
 
 atexit(sdl_cleanup);
 }
+
+static QemuDisplay qemu_display_sdl2 = {
+.type   = DISPLAY_TYPE_SDL,
+.early_init = sdl2_display_early_init,
+.init   = sdl2_display_init,
+};
+
+static void register_sdl1(void)
+{
+qemu_display_register(_display_sdl2);
+}
+
+type_init(register_sdl1);
diff --git a/vl.c b/vl.c
index 70458ba708..45900ba7e6 100644
--- a/vl.c
+++ b/vl.c
@@ -2085,7 +2085,6 @@ static void parse_display(const char *p)
 const char *opts;
 
 if (strstart(p, "sdl", )) {
-#ifdef CONFIG_SDL
 dpy.type = DISPLAY_TYPE_SDL;
 while (*opts) {
 const char *nextopt;
@@ -2146,10 +2145,6 @@ static void parse_display(const char *p)
 }
 opts = nextopt;
 }
-#else
-error_report("SDL support is disabled");
-exit(1);
-#endif
 } else if (strstart(p, "vnc", )) {
 if (*opts == '=') {
 vnc_parse(opts + 1, _fatal);
@@ -4327,12 +4322,7 @@ int main(int argc, char **argv, char **envp)
  "ignoring option");
 }
 
-if (dpy.type == DISPLAY_TYPE_SDL) {
-sdl_display_early_init();
-} else {
-qemu_display_early_init();
-}
-
+qemu_display_early_init();
 qemu_console_early_init();
 
 if (dpy.has_gl && dpy.gl && display_opengl == 0) {
@@ -4664,9 +4654,6 @@ int main(int argc, char **argv, char **envp)
 case DISPLAY_TYPE_CURSES:
 curses_display_init(ds, );
 break;
-case DISPLAY_TYPE_SDL:
-sdl_display_init(ds, );
-break;
 case DISPLAY_TYPE_COCOA:
 cocoa_display_init(ds, );
 break;
-- 
2.9.3

[Qemu-devel] [PATCH 03/12] cocoa: switch over to new display registry

[Gerd Hoffmann]

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 12 
 vl.c |  3 ---
 ui/cocoa.m   | 14 +-
 3 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 82bbea0242..b97d9ccae4 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -441,18 +441,6 @@ void qemu_display_register(QemuDisplay *ui);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
-/* cocoa.m */
-#ifdef CONFIG_COCOA
-void cocoa_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_COCOA is disabled */
-error_report("Cocoa support is disabled");
-abort();
-}
-#endif
-
 /* vnc.c */
 void vnc_display_init(const char *id);
 void vnc_display_open(const char *id, Error **errp);
diff --git a/vl.c b/vl.c
index 45900ba7e6..2c3cb4651c 100644
--- a/vl.c
+++ b/vl.c
@@ -4654,9 +4654,6 @@ int main(int argc, char **argv, char **envp)
 case DISPLAY_TYPE_CURSES:
 curses_display_init(ds, );
 break;
-case DISPLAY_TYPE_COCOA:
-cocoa_display_init(ds, );
-break;
 default:
 qemu_display_init(ds, );
 break;
diff --git a/ui/cocoa.m b/ui/cocoa.m
index 90d9aa57ea..8b0dce90cb 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1683,7 +1683,7 @@ static void addRemovableDevicesMenuItems(void)
 qapi_free_BlockInfoList(pointerToFree);
 }
 
-void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
+static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
 {
 COCOA_DEBUG("qemu_cocoa: cocoa_display_init\n");
 
@@ -1713,3 +1713,15 @@ void cocoa_display_init(DisplayState *ds, DisplayOptions 
*opts)
  */
 addRemovableDevicesMenuItems();
 }
+
+static QemuDisplay qemu_display_cocoa = {
+.type   = DISPLAY_TYPE_COCOA,
+.init   = cocoa_display_init,
+};
+
+static void register_cocoa(void)
+{
+qemu_display_register(_display_cocoa);
+}
+
+type_init(register_cocoa);
-- 
2.9.3

[Qemu-devel] [PATCH 06/12] console: add and use qemu_display_find_default

[Gerd Hoffmann]

Using the new display registry instead of #ifdefs in vl.c.

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h |  1 +
 ui/console.c | 19 +++
 vl.c | 15 +--
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 1832c7eccf..a0d3330056 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -438,6 +438,7 @@ struct QemuDisplay {
 };
 
 void qemu_display_register(QemuDisplay *ui);
+bool qemu_display_find_default(DisplayOptions *opts);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
diff --git a/ui/console.c b/ui/console.c
index 8e55a05108..5a63e9dfa2 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -2178,6 +2178,25 @@ void qemu_display_register(QemuDisplay *ui)
 dpys[ui->type] = ui;
 }
 
+bool qemu_display_find_default(DisplayOptions *opts)
+{
+static DisplayType prio[] = {
+DISPLAY_TYPE_GTK,
+DISPLAY_TYPE_SDL,
+DISPLAY_TYPE_COCOA
+};
+int i;
+
+for (i = 0; i < ARRAY_SIZE(prio); i++) {
+if (dpys[prio[i]] == NULL) {
+continue;
+}
+opts->type = prio[i];
+return true;
+}
+return false;
+}
+
 void qemu_display_early_init(DisplayOptions *opts)
 {
 assert(opts->type < DISPLAY_TYPE__MAX);
diff --git a/vl.c b/vl.c
index 47c953f8dc..59e56593f8 100644
--- a/vl.c
+++ b/vl.c
@@ -4285,17 +4285,12 @@ int main(int argc, char **argv, char **envp)
 }
 #endif
 if (dpy.type == DISPLAY_TYPE_DEFAULT && !display_remote) {
-#if defined(CONFIG_GTK)
-dpy.type = DISPLAY_TYPE_GTK;
-#elif defined(CONFIG_SDL)
-dpy.type = DISPLAY_TYPE_SDL;
-#elif defined(CONFIG_COCOA)
-dpy.type = DISPLAY_TYPE_COCOA;
-#elif defined(CONFIG_VNC)
-vnc_parse("localhost:0,to=99,id=default", _abort);
-#else
-dpy.type = DISPLAY_TYPE_NONE;
+if (!qemu_display_find_default()) {
+dpy.type = DISPLAY_TYPE_NONE;
+#if defined(CONFIG_VNC)
+vnc_parse("localhost:0,to=99,id=default", _abort);
 #endif
+}
 }
 if (dpy.type == DISPLAY_TYPE_DEFAULT) {
 dpy.type = DISPLAY_TYPE_NONE;
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2] specs/qcow2: Fix documentation of the compressed cluster descriptor

[Alberto Garcia]

On Tue 20 Feb 2018 08:40:43 PM CET, Eric Blake wrote:
>>   Compressed Clusters Descriptor (x = 62 - (cluster_bits - 8)):
>
> I'm looking at how this works for different cluster sizes.  If we have
> 512-byte clusters, x is 61, and we DON'T have the 'number sectors'
> field at all!

Well, you can definitely have compressed images with 512-byte clusters.

So I think he have just found one more mistake in the documentation :)

 (x = 62 - (cluster_bits - 8)):

Bit  0 -  x:Host cluster offset.
   x+1 - 61:Number of 512-byte sectors

That's not how it works, it's rather [0, x-1], [x, 61]. For 512-byte
clusters x is 61 and we have 1 bit for the number of sectors, allowing
one or two sectors.

If you have a compressed image with 512-byte clusters you can also see
that since the compressed data is not aligned, some compressed clusters
span two different sectors (as expected).

That means that nb_csectors in the L2 entry is two (1+1), which is the
maximum allowed in this case, so that makes sense. And since the size of
our clusters is also 512 bytes, nb_csectors is twice the cluster size,
so we need s->cluster_data to be (cluster_size * 2) bytes (minus one,
strictly speaking).

> If we ever allowed a compressed cluster to spill across two host
> clusters, it would cause mayhem in trying to track refcounts and other
> things.

I haven't checked how this works in practice but it seems to work fine.
Note that those clusters are read-only so that surely makes things
easier.

Berto

Re: [Qemu-devel] [PATCH] tests/boot-serial-test: Fix problem with timeout due to dropped characters

[Paolo Bonzini]

On 16/02/2018 07:12, Thomas Huth wrote:
> Commit 92b540dac9fc3a5 introduce a counter to handle the timeouts in a
> better way. But in case ccnt reaches 512, the current read character is
> ignored - and if that character is part of the string that we are looking
> for, the test fails to match the string.
> 
> Almost all of the tests look for a string within the first 512 bytes of
> firmware output, so the problem never triggered there. But the hppa test
> that has been added recently looks for a longer string at the very end of
> a long output, thus there's a chance that we miss a character there so
> that the test fails unexpectedly. Fix it by *not* reading and dropping a
> character if the counter reaches 512.
> 
> Fixes: 92b540dac9fc3a572c7342edd0b073000f5a6abf
> Signed-off-by: Thomas Huth 
> ---
>  @Peter: Since this fixes the problem with running "make check", could
>  you maybe apply this directly to the master branch? Thanks, and sorry
>  for the inconvenience!
> 
>  tests/boot-serial-test.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
> index ea87a80..696f7a3 100644
> --- a/tests/boot-serial-test.c
> +++ b/tests/boot-serial-test.c
> @@ -101,7 +101,7 @@ static void check_guest_output(const testdef_t *test, int 
> fd)
>  /* Poll serial output... Wait at most 60 seconds */
>  for (i = 0; i < 6000; ++i) {
>  ccnt = 0;
> -while ((nbr = read(fd, , 1)) == 1 && ccnt++ < 512) {
> +while (ccnt++ < 512 && (nbr = read(fd, , 1)) == 1) {
>  if (ch == test->expect[pos]) {
>  pos += 1;
>  if (test->expect[pos] == '\0') {
> 

Queued in the meanwhile, thanks.

Paolo

[Qemu-devel] [PULL 07/22] include/fpu/softfloat: implement float16_set_sign helper

[Alex Bennée]

Signed-off-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index f75aa59100..59c06ef192 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -281,6 +281,11 @@ static inline float16 float16_chs(float16 a)
 return make_float16(float16_val(a) ^ 0x8000);
 }
 
+static inline float16 float16_set_sign(float16 a, int sign)
+{
+return make_float16((float16_val(a) & 0x7fff) | (sign << 15));
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
-- 
2.15.1

[Qemu-devel] [PULL 06/22] include/fpu/softfloat: implement float16_chs helper

[Alex Bennée]

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 1d34f2c3eb..f75aa59100 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -272,6 +272,15 @@ static inline float16 float16_abs(float16 a)
  */
 return make_float16(float16_val(a) & 0x7fff);
 }
+
+static inline float16 float16_chs(float16 a)
+{
+/* Note that chs does *not* handle NaN specially, nor does
+ * it flush denormal inputs to zero.
+ */
+return make_float16(float16_val(a) ^ 0x8000);
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
-- 
2.15.1

[Qemu-devel] [PULL 03/22] fpu/softfloat-types: new header to prevent excessive re-builds

[Alex Bennée]

The main culprit here is bswap.h which pulled in softfloat.h so it
could use the types in its CPU_Float* and ldfl/stfql functions. As
bswap.h is very widely included this added a compile dependency every
time we touch softfloat.h. Move the typedefs for each float type into
their own file so we don't re-build the world every time we tweak the
main softfloat.h header.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat-types.h b/include/fpu/softfloat-types.h
new file mode 100644
index 00..8210a94ea1
--- /dev/null
+++ b/include/fpu/softfloat-types.h
@@ -0,0 +1,115 @@
+/*
+ * QEMU float support
+ *
+ * The code in this source file is derived from release 2a of the SoftFloat
+ * IEC/IEEE Floating-point Arithmetic Package. Those parts of the code (and
+ * some later contributions) are provided under that license, as detailed 
below.
+ * It has subsequently been modified by contributors to the QEMU Project,
+ * so some portions are provided under:
+ *  the SoftFloat-2a license
+ *  the BSD license
+ *  GPL-v2-or-later
+ *
+ * This header holds definitions for code that might be dealing with
+ * softfloat types but not need access to the actual library functions.
+ */
+/*
+===
+This C header file is part of the SoftFloat IEC/IEEE Floating-point
+Arithmetic Package, Release 2a.
+
+Written by John R. Hauser.  This work was made possible in part by the
+International Computer Science Institute, located at Suite 600, 1947 Center
+Street, Berkeley, California 94704.  Funding was partially provided by the
+National Science Foundation under grant MIP-9311980.  The original version
+of this code was written as part of a project to build a fixed-point vector
+processor in collaboration with the University of California at Berkeley,
+overseen by Profs. Nelson Morgan and John Wawrzynek.  More information
+is available through the Web page `http://HTTP.CS.Berkeley.EDU/~jhauser/
+arithmetic/SoftFloat.html'.
+
+THIS SOFTWARE IS DISTRIBUTED AS IS, FOR FREE.  Although reasonable effort
+has been made to avoid it, THIS SOFTWARE MAY CONTAIN FAULTS THAT WILL AT
+TIMES RESULT IN INCORRECT BEHAVIOR.  USE OF THIS SOFTWARE IS RESTRICTED TO
+PERSONS AND ORGANIZATIONS WHO CAN AND WILL TAKE FULL RESPONSIBILITY FOR ANY
+AND ALL LOSSES, COSTS, OR OTHER PROBLEMS ARISING FROM ITS USE.
+
+Derivative works are acceptable, even for commercial purposes, so long as
+(1) they include prominent notice that the work is derivative, and (2) they
+include prominent notice akin to these four paragraphs for those parts of
+this code that are retained.
+
+===
+*/
+
+/* BSD licensing:
+ * Copyright (c) 2006, Fabrice Bellard
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ * this list of conditions and the following disclaimer in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ * 3. Neither the name of the copyright holder nor the names of its 
contributors
+ * may be used to endorse or promote products derived from this software 
without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* Portions of this work are licensed under the terms of the GNU GPL,
+ * version 2 or later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef SOFTFLOAT_TYPES_H
+#define SOFTFLOAT_TYPES_H
+
+/*
+ * Software IEC/IEEE floating-point types.
+ */
+
+typedef uint16_t float16;
+typedef uint32_t float32;
+typedef uint64_t float64;
+#define float16_val(x) (x)
+#define float32_val(x) (x)
+#define float64_val(x) (x)
+#define make_float16(x) (x)
+#define make_float32(x) (x)
+#define make_float64(x) (x)
+#define const_float16(x) (x)
+#define 

Re: [Qemu-devel] [PATCH] tests/boot-serial-test: Fix problem with timeout due to dropped characters

[Thomas Huth]

On 21.02.2018 12:05, Paolo Bonzini wrote:
> On 16/02/2018 07:12, Thomas Huth wrote:
>> Commit 92b540dac9fc3a5 introduce a counter to handle the timeouts in a
>> better way. But in case ccnt reaches 512, the current read character is
>> ignored - and if that character is part of the string that we are looking
>> for, the test fails to match the string.
>>
>> Almost all of the tests look for a string within the first 512 bytes of
>> firmware output, so the problem never triggered there. But the hppa test
>> that has been added recently looks for a longer string at the very end of
>> a long output, thus there's a chance that we miss a character there so
>> that the test fails unexpectedly. Fix it by *not* reading and dropping a
>> character if the counter reaches 512.
>>
>> Fixes: 92b540dac9fc3a572c7342edd0b073000f5a6abf
>> Signed-off-by: Thomas Huth 
>> ---
>>  @Peter: Since this fixes the problem with running "make check", could
>>  you maybe apply this directly to the master branch? Thanks, and sorry
>>  for the inconvenience!
>>
>>  tests/boot-serial-test.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
>> index ea87a80..696f7a3 100644
>> --- a/tests/boot-serial-test.c
>> +++ b/tests/boot-serial-test.c
>> @@ -101,7 +101,7 @@ static void check_guest_output(const testdef_t *test, 
>> int fd)
>>  /* Poll serial output... Wait at most 60 seconds */
>>  for (i = 0; i < 6000; ++i) {
>>  ccnt = 0;
>> -while ((nbr = read(fd, , 1)) == 1 && ccnt++ < 512) {
>> +while (ccnt++ < 512 && (nbr = read(fd, , 1)) == 1) {
>>  if (ch == test->expect[pos]) {
>>  pos += 1;
>>  if (test->expect[pos] == '\0') {
>>
> 
> Queued in the meanwhile, thanks.

Thanks, but Peter already applied it to fix the "make check" failures:

https://git.qemu.org/?p=qemu.git;a=commit;h=5e5432b766c424a5d1

 Thomas

[Qemu-devel] [PULL 18/22] fpu/softfloat: re-factor int/uint to float

[Alex Bennée]

These are considerably simpler as the lower order integers can just
use the higher order conversion function. As the decomposed fractional
part is a full 64 bit rounding and inexact handling comes from the
pack functions.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index da0c43c0e7..4313d3a602 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1500,6 +1500,169 @@ FLOAT_TO_UINT(64, 64)
 
 #undef FLOAT_TO_UINT
 
+/*
+ * Integer to float conversions
+ *
+ * Returns the result of converting the two's complement integer `a'
+ * to the floating-point format. The conversion is performed according
+ * to the IEC/IEEE Standard for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts int_to_float(int64_t a, float_status *status)
+{
+FloatParts r;
+if (a == 0) {
+r.cls = float_class_zero;
+r.sign = false;
+} else if (a == (1ULL << 63)) {
+r.cls = float_class_normal;
+r.sign = true;
+r.frac = DECOMPOSED_IMPLICIT_BIT;
+r.exp = 63;
+} else {
+uint64_t f;
+if (a < 0) {
+f = -a;
+r.sign = true;
+} else {
+f = a;
+r.sign = false;
+}
+int shift = clz64(f) - 1;
+r.cls = float_class_normal;
+r.exp = (DECOMPOSED_BINARY_POINT - shift);
+r.frac = f << shift;
+}
+
+return r;
+}
+
+float16 int64_to_float16(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float16_round_pack_canonical(pa, status);
+}
+
+float16 int32_to_float16(int32_t a, float_status *status)
+{
+return int64_to_float16(a, status);
+}
+
+float16 int16_to_float16(int16_t a, float_status *status)
+{
+return int64_to_float16(a, status);
+}
+
+float32 int64_to_float32(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float32_round_pack_canonical(pa, status);
+}
+
+float32 int32_to_float32(int32_t a, float_status *status)
+{
+return int64_to_float32(a, status);
+}
+
+float32 int16_to_float32(int16_t a, float_status *status)
+{
+return int64_to_float32(a, status);
+}
+
+float64 int64_to_float64(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float64_round_pack_canonical(pa, status);
+}
+
+float64 int32_to_float64(int32_t a, float_status *status)
+{
+return int64_to_float64(a, status);
+}
+
+float64 int16_to_float64(int16_t a, float_status *status)
+{
+return int64_to_float64(a, status);
+}
+
+
+/*
+ * Unsigned Integer to float conversions
+ *
+ * Returns the result of converting the unsigned integer `a' to the
+ * floating-point format. The conversion is performed according to the
+ * IEC/IEEE Standard for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts uint_to_float(uint64_t a, float_status *status)
+{
+FloatParts r = { .sign = false};
+
+if (a == 0) {
+r.cls = float_class_zero;
+} else {
+int spare_bits = clz64(a) - 1;
+r.cls = float_class_normal;
+r.exp = DECOMPOSED_BINARY_POINT - spare_bits;
+if (spare_bits < 0) {
+shift64RightJamming(a, -spare_bits, );
+r.frac = a;
+} else {
+r.frac = a << spare_bits;
+}
+}
+
+return r;
+}
+
+float16 uint64_to_float16(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float16_round_pack_canonical(pa, status);
+}
+
+float16 uint32_to_float16(uint32_t a, float_status *status)
+{
+return uint64_to_float16(a, status);
+}
+
+float16 uint16_to_float16(uint16_t a, float_status *status)
+{
+return uint64_to_float16(a, status);
+}
+
+float32 uint64_to_float32(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float32_round_pack_canonical(pa, status);
+}
+
+float32 uint32_to_float32(uint32_t a, float_status *status)
+{
+return uint64_to_float32(a, status);
+}
+
+float32 uint16_to_float32(uint16_t a, float_status *status)
+{
+return uint64_to_float32(a, status);
+}
+
+float64 uint64_to_float64(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float64_round_pack_canonical(pa, status);
+}
+
+float64 uint32_to_float64(uint32_t a, float_status *status)
+{
+return uint64_to_float64(a, status);
+}
+
+float64 uint16_to_float64(uint16_t a, float_status *status)
+{
+return uint64_to_float64(a, status);
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -2591,43 +2754,6 @@ static float128 normalizeRoundAndPackFloat128(flag 
zSign, int32_t zExp,
 
 }
 

Re: [Qemu-devel] [PATCH] migration: do not transfer ram during bulk storage migration

[Stefan Hajnoczi]

On Tue, Feb 20, 2018 at 04:10:03PM +0100, Peter Lieven wrote:
> this patch makes the bulk phase of a block migration to take
> place before we start transferring ram. As the bulk block migration
> can take a long time its pointless to transfer ram during that phase.
> 
> Signed-off-by: Peter Lieven 
> ---
>  migration/ram.c | 8 
>  1 file changed, 8 insertions(+)

This makes sense to me:

Reviewed-by: Stefan Hajnoczi 


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH 0/2] Firmware blob and git submodule for Sam460ex

[Peter Maydell]

On 20 February 2018 at 20:44, Emilio G. Cota  wrote:
> On Tue, Feb 20, 2018 at 18:31:17 +, Peter Maydell wrote:
>> On 20 February 2018 at 18:10, BALATON Zoltan  wrote:
>> > I've created a git repo for the Sam460ex u-boot sources and this adds
>> > that as a submodule and a separate patch to add the binary built from
>> > these sources. Feel free to keep this as two patches, squash them into
>> > one patch or take the git repo and commit the content under the QEMU
>> > repo and use that as a submodule as you see fit (or let me know if any
>> > changes are needed for these patches).
>> >
>> > BALATON Zoltan (2):
>> >   roms: Added git submodule for u-boot-sam460 (firmware for sam460ex)
>> >   pc-bios: Added u-boot-sam460 firmware binary
>>
>> We already have a submodule for u-boot. Is it not possible to
>> build this bios blob from those upstream u-boot sources?
>
> This is discussed in the following thread:
>   Re: [Qemu-ppc] [PATCH v3 2/2] ppc: Add aCube Sam460ex board
>   http://lists.gnu.org/archive/html/qemu-ppc/2018-02/msg00268.html

If upstream u-boot have abandoned the board support I'm not very
enthusiastic about our taking it on :-(

thanks
-- PMM

[Qemu-devel] [PATCH v2 04/36] qcow2: Pass BlockdevCreateOptions to qcow2_create2()

[Kevin Wolf]

All of the simple options are now passed to qcow2_create2() in a
BlockdevCreateOptions object. Still missing: node-name and the
encryption options.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 190 ++
 1 file changed, 152 insertions(+), 38 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index dc6cdea113..22194180c6 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2665,19 +2665,26 @@ static int64_t qcow2_calc_prealloc_size(int64_t 
total_size,
 return meta_size + aligned_total_size;
 }
 
-static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp)
+static bool validate_cluster_size(size_t cluster_size, Error **errp)
 {
-size_t cluster_size;
-int cluster_bits;
-
-cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
- DEFAULT_CLUSTER_SIZE);
-cluster_bits = ctz32(cluster_size);
+int cluster_bits = ctz32(cluster_size);
 if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS ||
 (1 << cluster_bits) != cluster_size)
 {
 error_setg(errp, "Cluster size must be a power of two between %d and "
"%dk", 1 << MIN_CLUSTER_BITS, 1 << (MAX_CLUSTER_BITS - 10));
+return false;
+}
+return true;
+}
+
+static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp)
+{
+size_t cluster_size;
+
+cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
+ DEFAULT_CLUSTER_SIZE);
+if (!validate_cluster_size(cluster_size, errp)) {
 return 0;
 }
 return cluster_size;
@@ -2725,12 +2732,11 @@ static uint64_t 
qcow2_opt_get_refcount_bits_del(QemuOpts *opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockDriverState *bs, int64_t total_size,
- const char *backing_file, const char *backing_format,
- int flags, size_t cluster_size, PreallocMode prealloc,
- QemuOpts *opts, int version, int refcount_order,
- const char *encryptfmt, Error **errp)
+static int qcow2_create2(BlockDriverState *bs,
+ BlockdevCreateOptions *create_options,
+ QemuOpts *opts, const char *encryptfmt, Error **errp)
 {
+BlockdevCreateOptionsQcow2 *qcow2_opts;
 QDict *options;
 
 /*
@@ -2747,10 +2753,92 @@ static int qcow2_create2(BlockDriverState *bs, int64_t 
total_size,
  */
 BlockBackend *blk;
 QCowHeader *header;
+size_t cluster_size;
+int version;
+int refcount_order;
 uint64_t* refcount_table;
 Error *local_err = NULL;
 int ret;
 
+/* Validate options and set default values */
+assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
+qcow2_opts = _options->u.qcow2;
+
+if (!QEMU_IS_ALIGNED(qcow2_opts->size, BDRV_SECTOR_SIZE)) {
+error_setg(errp, "Image size must be a multiple of 512 bytes");
+ret = -EINVAL;
+goto out;
+}
+
+if (qcow2_opts->has_version) {
+switch (qcow2_opts->version) {
+case BLOCKDEV_QCOW2_VERSION_V2:
+version = 2;
+break;
+case BLOCKDEV_QCOW2_VERSION_V3:
+version = 3;
+break;
+default:
+g_assert_not_reached();
+}
+} else {
+version = 3;
+}
+
+if (qcow2_opts->has_cluster_size) {
+cluster_size = qcow2_opts->cluster_size;
+} else {
+cluster_size = DEFAULT_CLUSTER_SIZE;
+}
+
+if (!validate_cluster_size(cluster_size, errp)) {
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_preallocation) {
+qcow2_opts->preallocation = PREALLOC_MODE_OFF;
+}
+if (qcow2_opts->has_backing_file &&
+qcow2_opts->preallocation != PREALLOC_MODE_OFF)
+{
+error_setg(errp, "Backing file and preallocation cannot be used at "
+   "the same time");
+return -EINVAL;
+}
+if (qcow2_opts->has_backing_fmt && !qcow2_opts->has_backing_file) {
+error_setg(errp, "Backing format cannot be used without backing file");
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_lazy_refcounts) {
+qcow2_opts->lazy_refcounts = false;
+}
+if (version < 3 && qcow2_opts->lazy_refcounts) {
+error_setg(errp, "Lazy refcounts only supported with compatibility "
+   "level 1.1 and above (use compat=1.1 or greater)");
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_refcount_bits) {
+qcow2_opts->refcount_bits = 16;
+}
+if (qcow2_opts->refcount_bits > 64 ||
+!is_power_of_2(qcow2_opts->refcount_bits))
+{
+error_setg(errp, "Refcount width must be a power of two and may not "
+   "exceed 64 bits");
+return -EINVAL;
+}
+

[Qemu-devel] [PATCH v2 09/36] test-qemu-opts: Test qemu_opts_append()

[Kevin Wolf]

Basic test for merging two QemuOptsLists.

Signed-off-by: Kevin Wolf 
---
 tests/test-qemu-opts.c | 128 +
 1 file changed, 128 insertions(+)

diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
index 5d5a3daa7b..6c3183390b 100644
--- a/tests/test-qemu-opts.c
+++ b/tests/test-qemu-opts.c
@@ -23,6 +23,8 @@ static QemuOptsList opts_list_01 = {
 {
 .name = "str1",
 .type = QEMU_OPT_STRING,
+.help = "Help texts are preserved in qemu_opts_append",
+.def_value_str = "default",
 },{
 .name = "str2",
 .type = QEMU_OPT_STRING,
@@ -32,6 +34,7 @@ static QemuOptsList opts_list_01 = {
 },{
 .name = "number1",
 .type = QEMU_OPT_NUMBER,
+.help = "Having help texts only for some options is okay",
 },{
 .name = "number2",
 .type = QEMU_OPT_NUMBER,
@@ -743,6 +746,129 @@ static void test_opts_parse_size(void)
 qemu_opts_reset(_list_02);
 }
 
+static void append_verify_list_01(QemuOptDesc *desc, bool with_overlapping)
+{
+int i = 0;
+
+if (with_overlapping) {
+g_assert_cmpstr(desc[i].name, ==, "str1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==,
+"Help texts are preserved in qemu_opts_append");
+g_assert_cmpstr(desc[i].def_value_str, ==, "default");
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "str2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+}
+
+g_assert_cmpstr(desc[i].name, ==, "str3");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "number1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_NUMBER);
+g_assert_cmpstr(desc[i].help, ==,
+"Having help texts only for some options is okay");
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "number2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_NUMBER);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, NULL);
+}
+
+static void append_verify_list_02(QemuOptDesc *desc)
+{
+int i = 0;
+
+g_assert_cmpstr(desc[i].name, ==, "str1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "str2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "bool1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_BOOL);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "bool2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_BOOL);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size3");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+}
+
+static void test_opts_append_to_null(void)
+{
+QemuOptsList *merged;
+
+merged = qemu_opts_append(NULL, _list_01);
+g_assert(merged != _list_01);
+
+g_assert_cmpstr(merged->name, ==, NULL);
+g_assert_cmpstr(merged->implied_opt_name, ==, NULL);
+g_assert_false(merged->merge_lists);
+
+append_verify_list_01(merged->desc, true);
+
+qemu_opts_free(merged);
+}
+
+static void test_opts_append(void)
+{
+QemuOptsList *first, *merged;
+
+first = qemu_opts_append(NULL, _list_02);
+merged = qemu_opts_append(first, _list_01);
+g_assert(first != _list_02);
+g_assert(merged != _list_01);
+
+g_assert_cmpstr(merged->name, ==, NULL);
+g_assert_cmpstr(merged->implied_opt_name, ==, NULL);
+g_assert_false(merged->merge_lists);
+
+append_verify_list_02(>desc[0]);
+append_verify_list_01(>desc[7], false);
+
+

[Qemu-devel] [PATCH v2 01/36] block/qapi: Introduce BlockdevCreateOptions

[Kevin Wolf]

This creates a BlockdevCreateOptions union type that will contain all of
the options for image creation. We'll start out with an empty struct
type BlockdevCreateNotSupported for all drivers.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 62 
 1 file changed, 62 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 5c5921bfb7..d256cefc79 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,68 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevCreateNotSupported:
+#
+# This is used for all drivers that don't support creating images.
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateNotSupported', 'data': {}}
+
+##
+# @BlockdevCreateOptions:
+#
+# Options for creating an image format on a given node.
+#
+# @driver   block driver to create the image format
+#
+# Since: 2.12
+##
+{ 'union': 'BlockdevCreateOptions',
+  'base': {
+  'driver': 'BlockdevDriver' },
+  'discriminator': 'driver',
+  'data': {
+  'blkdebug':   'BlockdevCreateNotSupported',
+  'blkverify':  'BlockdevCreateNotSupported',
+  'bochs':  'BlockdevCreateNotSupported',
+  'cloop':  'BlockdevCreateNotSupported',
+  'dmg':'BlockdevCreateNotSupported',
+  'file':   'BlockdevCreateNotSupported',
+  'ftp':'BlockdevCreateNotSupported',
+  'ftps':   'BlockdevCreateNotSupported',
+  'gluster':'BlockdevCreateNotSupported',
+  'host_cdrom': 'BlockdevCreateNotSupported',
+  'host_device':'BlockdevCreateNotSupported',
+  'http':   'BlockdevCreateNotSupported',
+  'https':  'BlockdevCreateNotSupported',
+  'iscsi':  'BlockdevCreateNotSupported',
+  'luks':   'BlockdevCreateNotSupported',
+  'nbd':'BlockdevCreateNotSupported',
+  'nfs':'BlockdevCreateNotSupported',
+  'null-aio':   'BlockdevCreateNotSupported',
+  'null-co':'BlockdevCreateNotSupported',
+  'nvme':   'BlockdevCreateNotSupported',
+  'parallels':  'BlockdevCreateNotSupported',
+  'qcow2':  'BlockdevCreateNotSupported',
+  'qcow':   'BlockdevCreateNotSupported',
+  'qed':'BlockdevCreateNotSupported',
+  'quorum': 'BlockdevCreateNotSupported',
+  'raw':'BlockdevCreateNotSupported',
+  'rbd':'BlockdevCreateNotSupported',
+  'replication':'BlockdevCreateNotSupported',
+  'sheepdog':   'BlockdevCreateNotSupported',
+  'ssh':'BlockdevCreateNotSupported',
+  'throttle':   'BlockdevCreateNotSupported',
+  'vdi':'BlockdevCreateNotSupported',
+  'vhdx':   'BlockdevCreateNotSupported',
+  'vmdk':   'BlockdevCreateNotSupported',
+  'vpc':'BlockdevCreateNotSupported',
+  'vvfat':  'BlockdevCreateNotSupported',
+  'vxhs':   'BlockdevCreateNotSupported'
+  } }
+
+##
 # @blockdev-open-tray:
 #
 # Opens a block device's tray. If there is a block driver state tree inserted 
as
-- 
2.13.6

[Qemu-devel] [PATCH v2 02/36] block/qapi: Add qcow2 create options to schema

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 45 -
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index d256cefc79..74b864d64e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,49 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevQcow2Version:
+#
+# @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
+# @v3:  The extended QCOW2 format as introduced in qemu 1.1 (version 3)
+#
+# Since: 2.12
+##
+{ 'enum': 'BlockdevQcow2Version',
+  'data': [ 'v2', 'v3' ] }
+
+
+##
+# @BlockdevCreateOptionsQcow2:
+#
+# Driver specific image creation options for qcow2.
+#
+# @file Node to create the image format on
+# @size Size of the virtual disk in bytes
+# @version  Compatibility level (default: v3)
+# @backing-file File name of the backing file if a backing file
+#   should be used
+# @backing-fmt  Name of the block driver to use for the backing file
+# @encrypt  Encryption options if the image should be encrypted
+# @cluster-size qcow2 cluster size in bytes (default: 65536)
+# @preallocationPreallocation mode for the new image (default: off)
+# @lazy-refcounts   True if refcounts may be updated lazily (default: off)
+# @refcount-bitsWidth of reference counts in bits (default: 16)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsQcow2',
+  'data': { 'file': 'BlockdevRef',
+'size': 'size',
+'*version': 'BlockdevQcow2Version',
+'*backing-file':'str',
+'*backing-fmt': 'BlockdevDriver',
+'*encrypt': 'QCryptoBlockCreateOptions',
+'*cluster-size':'size',
+'*preallocation':   'PreallocMode',
+'*lazy-refcounts':  'bool',
+'*refcount-bits':   'int' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3402,7 +3445,7 @@
   'null-co':'BlockdevCreateNotSupported',
   'nvme':   'BlockdevCreateNotSupported',
   'parallels':  'BlockdevCreateNotSupported',
-  'qcow2':  'BlockdevCreateNotSupported',
+  'qcow2':  'BlockdevCreateOptionsQcow2',
   'qcow':   'BlockdevCreateNotSupported',
   'qed':'BlockdevCreateNotSupported',
   'quorum': 'BlockdevCreateNotSupported',
-- 
2.13.6

[Qemu-devel] [PATCH v2 06/36] qcow2: Use QCryptoBlockCreateOptions in qcow2_create2()

[Kevin Wolf]

Instead of passing the encryption format name and the QemuOpts down, use
the QCryptoBlockCreateOptions contained in BlockdevCreateOptions.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 62 +++
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index b34924b0f0..9a2028b3cf 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2414,13 +2414,10 @@ static int qcow2_crypt_method_from_format(const char 
*encryptfmt)
 }
 }
 
-static int qcow2_set_up_encryption(BlockDriverState *bs, const char 
*encryptfmt,
-   QemuOpts *opts, Error **errp)
+static QCryptoBlockCreateOptions *
+qcow2_parse_encryption(const char *encryptfmt, QemuOpts *opts, Error **errp)
 {
-BDRVQcow2State *s = bs->opaque;
 QCryptoBlockCreateOptions *cryptoopts = NULL;
-QCryptoBlock *crypto = NULL;
-int ret = -EINVAL;
 QDict *options, *encryptopts;
 int fmt;
 
@@ -2443,10 +2440,31 @@ static int qcow2_set_up_encryption(BlockDriverState 
*bs, const char *encryptfmt,
 error_setg(errp, "Unknown encryption format '%s'", encryptfmt);
 break;
 }
-if (!cryptoopts) {
-ret = -EINVAL;
-goto out;
+
+QDECREF(encryptopts);
+return cryptoopts;
+}
+
+static int qcow2_set_up_encryption(BlockDriverState *bs,
+   QCryptoBlockCreateOptions *cryptoopts,
+   Error **errp)
+{
+BDRVQcow2State *s = bs->opaque;
+QCryptoBlock *crypto = NULL;
+int fmt, ret;
+
+switch (cryptoopts->format) {
+case Q_CRYPTO_BLOCK_FORMAT_LUKS:
+fmt = QCOW_CRYPT_LUKS;
+break;
+case Q_CRYPTO_BLOCK_FORMAT_QCOW:
+fmt = QCOW_CRYPT_AES;
+break;
+default:
+error_setg(errp, "Crypto format not supported in qcow2");
+return -EINVAL;
 }
+
 s->crypt_method_header = fmt;
 
 crypto = qcrypto_block_create(cryptoopts, "encrypt.",
@@ -2454,8 +2472,7 @@ static int qcow2_set_up_encryption(BlockDriverState *bs, 
const char *encryptfmt,
   qcow2_crypto_hdr_write_func,
   bs, errp);
 if (!crypto) {
-ret = -EINVAL;
-goto out;
+return -EINVAL;
 }
 
 ret = qcow2_update_header(bs);
@@ -2464,10 +2481,9 @@ static int qcow2_set_up_encryption(BlockDriverState *bs, 
const char *encryptfmt,
 goto out;
 }
 
+ret = 0;
  out:
-QDECREF(encryptopts);
 qcrypto_block_free(crypto);
-qapi_free_QCryptoBlockCreateOptions(cryptoopts);
 return ret;
 }
 
@@ -2732,8 +2748,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockdevCreateOptions *create_options,
- QemuOpts *opts, const char *encryptfmt, Error **errp)
+static int qcow2_create2(BlockdevCreateOptions *create_options, Error **errp)
 {
 BlockdevCreateOptionsQcow2 *qcow2_opts;
 QDict *options;
@@ -2963,8 +2978,8 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options,
 }
 
 /* Want encryption? There you go. */
-if (encryptfmt) {
-ret = qcow2_set_up_encryption(blk_bs(blk), encryptfmt, opts, errp);
+if (qcow2_opts->has_encrypt) {
+ret = qcow2_set_up_encryption(blk_bs(blk), qcow2_opts->encrypt, errp);
 if (ret < 0) {
 goto out;
 }
@@ -3021,6 +3036,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 int version;
 uint64_t refcount_bits;
 char *encryptfmt = NULL;
+QCryptoBlockCreateOptions *cryptoopts = NULL;
 BlockDriverState *bs = NULL;
 Error *local_err = NULL;
 int ret;
@@ -3037,6 +3053,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 ret = -EINVAL;
 goto finish;
 }
+
 encryptfmt = qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT_FORMAT);
 if (encryptfmt) {
 if (qemu_opt_get(opts, BLOCK_OPT_ENCRYPT)) {
@@ -3048,6 +3065,14 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 } else if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) {
 encryptfmt = g_strdup("aes");
 }
+if (encryptfmt) {
+cryptoopts = qcow2_parse_encryption(encryptfmt, opts, errp);
+if (cryptoopts == NULL) {
+ret = -EINVAL;
+goto finish;
+}
+}
+
 cluster_size = qcow2_opt_get_cluster_size_del(opts, _err);
 if (local_err) {
 error_propagate(errp, local_err);
@@ -3121,6 +3146,8 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 .backing_file   = backing_file,
 .has_backing_fmt= (backing_fmt != NULL),
 .backing_fmt= backing_drv,
+ 

[Qemu-devel] [PATCH v2 07/36] qcow2: Handle full/falloc preallocation in qcow2_create2()

[Kevin Wolf]

Once qcow2_create2() can be called directly on an already existing node,
we must provide the 'full' and 'falloc' preallocation modes outside of
creating the image on the protocol layer. Fortunately, we have
preallocated truncate now which can provide this functionality.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 28 +++-
 1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 9a2028b3cf..64bf2863cd 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2872,6 +2872,25 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 blk_set_allow_write_beyond_eof(blk, true);
 
+/* Clear the protocol layer and preallocate it if necessary */
+ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp);
+if (ret < 0) {
+goto out;
+}
+
+if (qcow2_opts->preallocation == PREALLOC_MODE_FULL ||
+qcow2_opts->preallocation == PREALLOC_MODE_FALLOC)
+{
+int64_t prealloc_size =
+qcow2_calc_prealloc_size(qcow2_opts->size, cluster_size,
+ refcount_order);
+
+ret = blk_truncate(blk, prealloc_size, qcow2_opts->preallocation, 
errp);
+if (ret < 0) {
+goto out;
+}
+}
+
 /* Write the header */
 QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header));
 header = g_malloc0(cluster_size);
@@ -3108,15 +3127,6 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 
 /* Create and open the file (protocol layer) */
-if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
-int refcount_order = ctz32(refcount_bits);
-int64_t prealloc_size =
-qcow2_calc_prealloc_size(size, cluster_size, refcount_order);
-qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, _abort);
-qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
- _abort);
-}
-
 ret = bdrv_create_file(filename, opts, errp);
 if (ret < 0) {
 goto finish;
-- 
2.13.6

[Qemu-devel] [PATCH v2 08/36] util: Add qemu_opts_to_qdict_filtered()

[Kevin Wolf]

This allows, given a QemuOpts for a QemuOptsList that was merged from
multiple QemuOptsList, to only consider those options that exist in one
specific list. Block drivers need this to separate format-layer create
options from protocol-level options.

Signed-off-by: Kevin Wolf 
---
 include/qemu/option.h |  2 ++
 util/qemu-option.c| 42 +-
 2 files changed, 39 insertions(+), 5 deletions(-)

diff --git a/include/qemu/option.h b/include/qemu/option.h
index b127fb6db6..306fdb5f7a 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -124,6 +124,8 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char 
*params,
 int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
Error **errp);
+QDict *qemu_opts_to_qdict_filtered(QemuOpts *opts, QDict *qdict,
+   QemuOptsList *list, bool del);
 QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
 void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
 
diff --git a/util/qemu-option.c b/util/qemu-option.c
index a401e936da..2b412eff5e 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -1007,14 +1007,23 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict 
*qdict, Error **errp)
 }
 
 /*
- * Convert from QemuOpts to QDict.
- * The QDict values are of type QString.
+ * Convert from QemuOpts to QDict. The QDict values are of type QString.
+ *
+ * If @list is given, only add those options to the QDict that are contained in
+ * the list. If @del is true, any options added to the QDict are removed from
+ * the QemuOpts, otherwise they remain there.
+ *
+ * If two options in @opts have the same name, they are processed in order
+ * so that the last one wins (consistent with the reverse iteration in
+ * qemu_opt_find()), but all of them are deleted if @del is true.
+ *
  * TODO We'll want to use types appropriate for opt->desc->type, but
  * this is enough for now.
  */
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+QDict *qemu_opts_to_qdict_filtered(QemuOpts *opts, QDict *qdict,
+   QemuOptsList *list, bool del)
 {
-QemuOpt *opt;
+QemuOpt *opt, *next;
 
 if (!qdict) {
 qdict = qdict_new();
@@ -1022,12 +1031,35 @@ QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
 if (opts->id) {
 qdict_put_str(qdict, "id", opts->id);
 }
-QTAILQ_FOREACH(opt, >head, next) {
+QTAILQ_FOREACH_SAFE(opt, >head, next, next) {
+if (list) {
+QemuOptDesc *desc;
+bool found = false;
+for (desc = list->desc; desc->name; desc++) {
+if (!strcmp(desc->name, opt->name)) {
+found = true;
+break;
+}
+}
+if (!found) {
+continue;
+}
+}
 qdict_put_str(qdict, opt->name, opt->str);
+if (del) {
+qemu_opt_del(opt);
+}
 }
 return qdict;
 }
 
+/* Copy all options in a QemuOpts to the given QDict. See
+ * qemu_opts_to_qdict_filtered() for details. */
+QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+{
+return qemu_opts_to_qdict_filtered(opts, qdict, NULL, false);
+}
+
 /* Validate parsed opts against descriptions where no
  * descriptions were provided in the QemuOptsList.
  */
-- 
2.13.6

[Qemu-devel] [PATCH v2 27/36] sheepdog: QAPIfy "redundacy" create option

[Kevin Wolf]

The "redundacy" option for Sheepdog image creation is currently a string
that can encode one or two integers depending on its format, which at
the same time implicitly selects a mode.

This patch turns it into a QAPI union and converts the string into such
a QAPI object before interpreting the values.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json | 45 +
 block/sheepdog.c | 94 +---
 2 files changed, 112 insertions(+), 27 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 085b791303..2b249c9e3d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3467,6 +3467,51 @@
 '*cluster-size' :   'size' } }
 
 ##
+# @SheepdogRedundancyType:
+#
+# @full Create a fully replicated vdi with x copies
+# @erasure-codedCreate an erasure coded vdi with x data strips and
+#   y parity strips
+#
+# Since: 2.12
+##
+{ 'enum': 'SheepdogRedundancyType',
+  'data': [ 'full', 'erasure-coded' ] }
+
+##
+# @SheepdogRedundancyFull:
+#
+# @copies   Number of copies to use (between 1 and 31)
+#
+# Since: 2.12
+##
+{ 'struct': 'SheepdogRedundancyFull',
+  'data': { 'copies': 'int' }}
+
+##
+# @SheepdogRedundancyErasureCoded:
+#
+# @data-strips  Number of data strips to use (one of {2,4,8,16})
+# @parity-stripsNumber of parity strips to use (between 1 and 15)
+#
+# Since: 2.12
+##
+{ 'struct': 'SheepdogRedundancyErasureCoded',
+  'data': { 'data-strips': 'int',
+'parity-strips': 'int' }}
+
+##
+# @SheepdogRedundancy:
+#
+# Since: 2.12
+##
+{ 'union': 'SheepdogRedundancy',
+  'base': { 'type': 'SheepdogRedundancyType' },
+  'discriminator': 'type',
+  'data': { 'full': 'SheepdogRedundancyFull',
+'erasure-coded': 'SheepdogRedundancyErasureCoded' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 3c3becf94d..22df2ba9d0 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1882,6 +1882,48 @@ out_with_err_set:
 return ret;
 }
 
+static int parse_redundancy(BDRVSheepdogState *s, SheepdogRedundancy *opt)
+{
+struct SheepdogInode *inode = >inode;
+
+switch (opt->type) {
+case SHEEPDOG_REDUNDANCY_TYPE_FULL:
+if (opt->u.full.copies > SD_MAX_COPIES || opt->u.full.copies < 1) {
+return -EINVAL;
+}
+inode->copy_policy = 0;
+inode->nr_copies = opt->u.full.copies;
+return 0;
+
+case SHEEPDOG_REDUNDANCY_TYPE_ERASURE_CODED:
+{
+int64_t copy = opt->u.erasure_coded.data_strips;
+int64_t parity = opt->u.erasure_coded.parity_strips;
+
+if (copy != 2 && copy != 4 && copy != 8 && copy != 16) {
+return -EINVAL;
+}
+
+if (parity >= SD_EC_MAX_STRIP || parity < 1) {
+return -EINVAL;
+}
+
+/*
+ * 4 bits for parity and 4 bits for data.
+ * We have to compress upper data bits because it can't represent 16
+ */
+inode->copy_policy = ((copy / 2) << 4) + parity;
+inode->nr_copies = copy + parity;
+return 0;
+}
+
+default:
+g_assert_not_reached();
+}
+
+return -EINVAL;
+}
+
 /*
  * Sheepdog support two kinds of redundancy, full replication and erasure
  * coding.
@@ -1892,12 +1934,13 @@ out_with_err_set:
  * # create a erasure coded vdi with x data strips and y parity strips
  * -o redundancy=x:y (x must be one of {2,4,8,16} and 1 <= y < SD_EC_MAX_STRIP)
  */
-static int parse_redundancy(BDRVSheepdogState *s, const char *opt)
+static int parse_redundancy_str(BDRVSheepdogState *s, const char *opt)
 {
-struct SheepdogInode *inode = >inode;
+struct SheepdogRedundancy redundancy;
 const char *n1, *n2;
 long copy, parity;
 char p[10];
+int ret;
 
 pstrcpy(p, sizeof(p), opt);
 n1 = strtok(p, ":");
@@ -1907,35 +1950,32 @@ static int parse_redundancy(BDRVSheepdogState *s, const 
char *opt)
 return -EINVAL;
 }
 
-copy = strtol(n1, NULL, 10);
-/* FIXME fix error checking by switching to qemu_strtol() */
-if (copy > SD_MAX_COPIES || copy < 1) {
-return -EINVAL;
-}
-if (!n2) {
-inode->copy_policy = 0;
-inode->nr_copies = copy;
-return 0;
+ret = qemu_strtol(n1, NULL, 10, );
+if (ret < 0) {
+return ret;
 }
 
-if (copy != 2 && copy != 4 && copy != 8 && copy != 16) {
-return -EINVAL;
-}
+if (!n2) {
+redundancy = (SheepdogRedundancy) {
+.type   = SHEEPDOG_REDUNDANCY_TYPE_FULL,
+.u.full.copies  = copy,
+};
+} else {
+ret = qemu_strtol(n2, NULL, 10, );
+if (ret < 0) {
+return ret;
+}
 
-parity = strtol(n2, NULL, 10);
-/* FIXME fix error checking by switching to qemu_strtol() */
-if (parity 

[Qemu-devel] [PATCH v2 20/36] rbd: Remove non-schema options from runtime_opts

[Kevin Wolf]

Instead of the QemuOpts in qemu_rbd_connect(), we want to use QAPI
objects. As a preparation, fetch those options directly from the QDict
that .bdrv_open() supports in the rbd driver and that are not in the
schema.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 55 ---
 1 file changed, 24 insertions(+), 31 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 4bbcce4eca..2e79c2d1fd 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -326,28 +326,6 @@ static QemuOptsList runtime_opts = {
 /*
  * server.* extracted manually, see qemu_rbd_mon_host()
  */
-{
-.name = "password-secret",
-.type = QEMU_OPT_STRING,
-.help = "ID of secret providing the password",
-},
-
-/*
- * Keys for qemu_rbd_parse_filename(), not in the QAPI schema
- */
-{
-/*
- * HACK: name starts with '=' so that qemu_opts_parse()
- * can't set it
- */
-.name = "=keyvalue-pairs",
-.type = QEMU_OPT_STRING,
-.help = "Legacy rados key/value option parameters",
-},
-{
-.name = "filename",
-.type = QEMU_OPT_STRING,
-},
 { /* end of list */ }
 },
 };
@@ -546,12 +524,13 @@ out:
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
 char **s_snap, char **s_image_name,
-QDict *options, bool cache, Error **errp)
+QDict *options, bool cache,
+const char *keypairs, const char *secretid,
+Error **errp)
 {
 QemuOpts *opts;
 char *mon_host = NULL;
-const char *pool, *snap, *conf, *user, *image_name, *keypairs;
-const char *secretid;
+const char *pool, *snap, *conf, *user, *image_name;
 Error *local_err = NULL;
 int r;
 
@@ -570,14 +549,11 @@ static int qemu_rbd_connect(rados_t *cluster, 
rados_ioctx_t *io_ctx,
 goto failed_opts;
 }
 
-secretid = qemu_opt_get(opts, "password-secret");
-
 pool   = qemu_opt_get(opts, "pool");
 conf   = qemu_opt_get(opts, "conf");
 snap   = qemu_opt_get(opts, "snapshot");
 user   = qemu_opt_get(opts, "user");
 image_name = qemu_opt_get(opts, "image");
-keypairs   = qemu_opt_get(opts, "=keyvalue-pairs");
 
 if (!pool || !image_name) {
 error_setg(errp, "Parameters 'pool' and 'image' are required");
@@ -662,6 +638,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 BDRVRBDState *s = bs->opaque;
 Error *local_err = NULL;
 const char *filename;
+char *keypairs, *secretid;
 int r;
 
 /* If we are given a filename, parse the filename, with precedence given to
@@ -672,16 +649,28 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 "This is an unsupported option, and may be deprecated "
 "in the future");
 qemu_rbd_parse_filename(filename, options, _err);
+qdict_del(options, "filename");
 if (local_err) {
 error_propagate(errp, local_err);
 return -EINVAL;
 }
 }
 
+keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs"));
+if (keypairs) {
+qdict_del(options, "=keyvalue-pairs");
+}
+
+secretid = g_strdup(qdict_get_try_str(options, "password-secret"));
+if (secretid) {
+qdict_del(options, "password-secret");
+}
+
 r = qemu_rbd_connect(>cluster, >io_ctx, >snap, >image_name,
- options, !(flags & BDRV_O_NOCACHE), errp);
+ options, !(flags & BDRV_O_NOCACHE), keypairs, 
secretid,
+ errp);
 if (r < 0) {
-return r;
+goto out;
 }
 
 /* rbd_open is always r/w */
@@ -708,13 +697,17 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 }
 
-return 0;
+r = 0;
+goto out;
 
 failed_open:
 rados_ioctx_destroy(s->io_ctx);
 g_free(s->snap);
 g_free(s->image_name);
 rados_shutdown(s->cluster);
+out:
+g_free(keypairs);
+g_free(secretid);
 return r;
 }
 
-- 
2.13.6

[Qemu-devel] [PATCH v2 19/36] rbd: Factor out qemu_rbd_connect()

[Kevin Wolf]

The code to establish an RBD connection is duplicated between open and
create. In order to be able to share the code, factor out the code from
qemu_rbd_open() as a first step.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 100 
 1 file changed, 60 insertions(+), 40 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 27fa11b473..4bbcce4eca 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -544,32 +544,17 @@ out:
 return rados_str;
 }
 
-static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
- Error **errp)
+static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
+char **s_snap, char **s_image_name,
+QDict *options, bool cache, Error **errp)
 {
-BDRVRBDState *s = bs->opaque;
-const char *pool, *snap, *conf, *user, *image_name, *keypairs;
-const char *secretid, *filename;
 QemuOpts *opts;
-Error *local_err = NULL;
 char *mon_host = NULL;
+const char *pool, *snap, *conf, *user, *image_name, *keypairs;
+const char *secretid;
+Error *local_err = NULL;
 int r;
 
-/* If we are given a filename, parse the filename, with precedence given to
- * filename encoded options */
-filename = qdict_get_try_str(options, "filename");
-if (filename) {
-warn_report("'filename' option specified. "
-"This is an unsupported option, and may be deprecated "
-"in the future");
-qemu_rbd_parse_filename(filename, options, _err);
-if (local_err) {
-r = -EINVAL;
-error_propagate(errp, local_err);
-goto exit;
-}
-}
-
 opts = qemu_opts_create(_opts, NULL, 0, _abort);
 qemu_opts_absorb_qdict(opts, options, _err);
 if (local_err) {
@@ -600,35 +585,35 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 goto failed_opts;
 }
 
-r = rados_create(>cluster, user);
+r = rados_create(cluster, user);
 if (r < 0) {
 error_setg_errno(errp, -r, "error initializing");
 goto failed_opts;
 }
 
-s->snap = g_strdup(snap);
-s->image_name = g_strdup(image_name);
+*s_snap = g_strdup(snap);
+*s_image_name = g_strdup(image_name);
 
 /* try default location when conf=NULL, but ignore failure */
-r = rados_conf_read_file(s->cluster, conf);
+r = rados_conf_read_file(*cluster, conf);
 if (conf && r < 0) {
 error_setg_errno(errp, -r, "error reading conf file %s", conf);
 goto failed_shutdown;
 }
 
-r = qemu_rbd_set_keypairs(s->cluster, keypairs, errp);
+r = qemu_rbd_set_keypairs(*cluster, keypairs, errp);
 if (r < 0) {
 goto failed_shutdown;
 }
 
 if (mon_host) {
-r = rados_conf_set(s->cluster, "mon_host", mon_host);
+r = rados_conf_set(*cluster, "mon_host", mon_host);
 if (r < 0) {
 goto failed_shutdown;
 }
 }
 
-if (qemu_rbd_set_auth(s->cluster, secretid, errp) < 0) {
+if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) {
 r = -EIO;
 goto failed_shutdown;
 }
@@ -640,24 +625,65 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  * librbd defaults to no caching. If write through caching cannot
  * be set up, fall back to no caching.
  */
-if (flags & BDRV_O_NOCACHE) {
-rados_conf_set(s->cluster, "rbd_cache", "false");
+if (cache) {
+rados_conf_set(*cluster, "rbd_cache", "true");
 } else {
-rados_conf_set(s->cluster, "rbd_cache", "true");
+rados_conf_set(*cluster, "rbd_cache", "false");
 }
 
-r = rados_connect(s->cluster);
+r = rados_connect(*cluster);
 if (r < 0) {
 error_setg_errno(errp, -r, "error connecting");
 goto failed_shutdown;
 }
 
-r = rados_ioctx_create(s->cluster, pool, >io_ctx);
+r = rados_ioctx_create(*cluster, pool, io_ctx);
 if (r < 0) {
 error_setg_errno(errp, -r, "error opening pool %s", pool);
 goto failed_shutdown;
 }
 
+qemu_opts_del(opts);
+return 0;
+
+failed_shutdown:
+rados_shutdown(*cluster);
+g_free(*s_snap);
+g_free(*s_image_name);
+failed_opts:
+qemu_opts_del(opts);
+g_free(mon_host);
+return r;
+}
+
+static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
+{
+BDRVRBDState *s = bs->opaque;
+Error *local_err = NULL;
+const char *filename;
+int r;
+
+/* If we are given a filename, parse the filename, with precedence given to
+ * filename encoded options */
+filename = qdict_get_try_str(options, "filename");
+if (filename) {
+warn_report("'filename' option specified. "
+"This is an unsupported option, and may be deprecated "
+"in the 

[Qemu-devel] [PATCH v2 28/36] sheepdog: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to sheepdog, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json |  24 +-
 block/sheepdog.c | 240 +++
 2 files changed, 189 insertions(+), 75 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2b249c9e3d..f7679fce53 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3512,6 +3512,28 @@
 'erasure-coded': 'SheepdogRedundancyErasureCoded' } }
 
 ##
+# @BlockdevCreateOptionsSheepdog:
+#
+# Driver specific image creation options for Sheepdog.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+# @backing-file File name of a base image
+# @preallocationPreallocation mode (allowed values: off, full)
+# @redundancy   Redundancy of the image
+# @object-size  Object size of the image
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsSheepdog',
+  'data': { 'location': 'BlockdevOptionsSheepdog',
+'size': 'size',
+'*backing-file':'str',
+'*preallocation':   'PreallocMode',
+'*redundancy':  'SheepdogRedundancy',
+'*object-size': 'size' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3562,7 +3584,7 @@
   'raw':'BlockdevCreateNotSupported',
   'rbd':'BlockdevCreateOptionsRbd',
   'replication':'BlockdevCreateNotSupported',
-  'sheepdog':   'BlockdevCreateNotSupported',
+  'sheepdog':   'BlockdevCreateOptionsSheepdog',
   'ssh':'BlockdevCreateNotSupported',
   'throttle':   'BlockdevCreateNotSupported',
   'vdi':'BlockdevCreateNotSupported',
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 22df2ba9d0..d45cf68ff2 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -17,6 +17,7 @@
 #include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qobject-input-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qemu/uri.h"
 #include "qemu/error-report.h"
 #include "qemu/option.h"
@@ -533,23 +534,6 @@ static void sd_aio_setup(SheepdogAIOCB *acb, 
BDRVSheepdogState *s,
 qemu_co_mutex_unlock(>queue_lock);
 }
 
-static SocketAddress *sd_socket_address(const char *path,
-const char *host, const char *port)
-{
-SocketAddress *addr = g_new0(SocketAddress, 1);
-
-if (path) {
-addr->type = SOCKET_ADDRESS_TYPE_UNIX;
-addr->u.q_unix.path = g_strdup(path);
-} else {
-addr->type = SOCKET_ADDRESS_TYPE_INET;
-addr->u.inet.host = g_strdup(host ?: SD_DEFAULT_ADDR);
-addr->u.inet.port = g_strdup(port ?: stringify(SD_DEFAULT_PORT));
-}
-
-return addr;
-}
-
 static SocketAddress *sd_server_config(QDict *options, Error **errp)
 {
 QDict *server = NULL;
@@ -1882,6 +1866,42 @@ out_with_err_set:
 return ret;
 }
 
+static int sd_create_prealloc(BlockdevOptionsSheepdog *location, int64_t size,
+  Error **errp)
+{
+BlockDriverState *bs;
+Visitor *v;
+QObject *obj = NULL;
+QDict *qdict;
+Error *local_err = NULL;
+int ret;
+
+v = qobject_output_visitor_new();
+visit_type_BlockdevOptionsSheepdog(v, NULL, , _err);
+visit_free(v);
+
+if (local_err) {
+error_propagate(errp, local_err);
+qobject_decref(obj);
+return -EINVAL;
+}
+
+qdict = qobject_to_qdict(obj);
+qdict_flatten(qdict);
+
+bs = bdrv_open(NULL, NULL, qdict, BDRV_O_PROTOCOL | BDRV_O_RDWR, errp);
+if (bs == NULL) {
+ret = -EIO;
+goto fail;
+}
+
+ret = sd_prealloc(bs, 0, size, errp);
+fail:
+bdrv_unref(bs);
+QDECREF(qdict);
+return ret;
+}
+
 static int parse_redundancy(BDRVSheepdogState *s, SheepdogRedundancy *opt)
 {
 struct SheepdogInode *inode = >inode;
@@ -1934,9 +1954,9 @@ static int parse_redundancy(BDRVSheepdogState *s, 
SheepdogRedundancy *opt)
  * # create a erasure coded vdi with x data strips and y parity strips
  * -o redundancy=x:y (x must be one of {2,4,8,16} and 1 <= y < SD_EC_MAX_STRIP)
  */
-static int parse_redundancy_str(BDRVSheepdogState *s, const char *opt)
+static SheepdogRedundancy *parse_redundancy_str(const char *opt)
 {
-struct SheepdogRedundancy redundancy;
+SheepdogRedundancy *redundancy;
 const char *n1, *n2;
 long copy, parity;
 char p[10];
@@ -1947,26 +1967,27 @@ static int parse_redundancy_str(BDRVSheepdogState *s, 
const char *opt)
 n2 = strtok(NULL, ":");
 
 if (!n1) {
-return -EINVAL;
+return NULL;
 }
 
 ret = qemu_strtol(n1, NULL, 10, );
 if (ret < 0) {
-return ret;
+return NULL;
 }
 
+redundancy = g_new0(SheepdogRedundancy, 1);
 if (!n2) {
-

[Qemu-devel] [PATCH v2 16/36] file-win32: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to file-win32, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/file-win32.c | 45 +
 1 file changed, 37 insertions(+), 8 deletions(-)

diff --git a/block/file-win32.c b/block/file-win32.c
index f24c7bb92c..d572cde357 100644
--- a/block/file-win32.c
+++ b/block/file-win32.c
@@ -553,29 +553,58 @@ static int64_t 
raw_get_allocated_file_size(BlockDriverState *bs)
 return st.st_size;
 }
 
-static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+static int raw_co_create(BlockdevCreateOptions *options, Error **errp)
 {
+BlockdevCreateOptionsFile *file_opts;
 int fd;
-int64_t total_size = 0;
 
-strstart(filename, "file:", );
+assert(options->driver == BLOCKDEV_DRIVER_FILE);
+file_opts = >u.file;
 
-/* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
+if (file_opts->has_preallocation) {
+error_setg(errp, "Preallocation is not supported on Windows");
+return -EINVAL;
+}
+if (file_opts->has_nocow) {
+error_setg(errp, "nocow is not supported on Windows");
+return -EINVAL;
+}
 
-fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
+fd = qemu_open(file_opts->filename, O_WRONLY | O_CREAT | O_TRUNC | 
O_BINARY,
0644);
 if (fd < 0) {
 error_setg_errno(errp, errno, "Could not create file");
 return -EIO;
 }
 set_sparse(fd);
-ftruncate(fd, total_size);
+ftruncate(fd, file_opts->size);
 qemu_close(fd);
+
 return 0;
 }
 
+static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions options;
+int64_t total_size = 0;
+
+strstart(filename, "file:", );
+
+/* Read out options */
+total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
+
+options = (BlockdevCreateOptions) {
+.driver = BLOCKDEV_DRIVER_FILE,
+.u.file = {
+.filename   = (char *) filename,
+.size   = total_size,
+.has_preallocation  = false,
+.has_nocow  = false,
+},
+};
+return raw_co_create(, errp);
+}
 
 static QemuOptsList raw_create_opts = {
 .name = "raw-create-opts",
-- 
2.13.6

[Qemu-devel] [PATCH v2 29/36] ssh: Use QAPI BlockdevOptionsSsh object

[Kevin Wolf]

Create a BlockdevOptionsSsh object in connect_to_ssh() and take the
options from there. 'host_key_check' is still processed separately
because it's not in the schema yet.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/ssh.c | 136 +++-
 1 file changed, 61 insertions(+), 75 deletions(-)

diff --git a/block/ssh.c b/block/ssh.c
index b63addcf94..9a89b7f350 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -543,21 +543,6 @@ static QemuOptsList ssh_runtime_opts = {
 .type = QEMU_OPT_NUMBER,
 .help = "Port to connect to",
 },
-{
-.name = "path",
-.type = QEMU_OPT_STRING,
-.help = "Path of the image on the host",
-},
-{
-.name = "user",
-.type = QEMU_OPT_STRING,
-.help = "User as which to connect",
-},
-{
-.name = "host_key_check",
-.type = QEMU_OPT_STRING,
-.help = "Defines how and what to check the host key against",
-},
 { /* end of list */ }
 },
 };
@@ -582,23 +567,31 @@ static bool ssh_process_legacy_socket_options(QDict 
*output_opts,
 return true;
 }
 
-static InetSocketAddress *ssh_config(QDict *options, Error **errp)
+static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp)
 {
-InetSocketAddress *inet = NULL;
-QDict *addr = NULL;
-QObject *crumpled_addr = NULL;
-Visitor *iv = NULL;
-Error *local_error = NULL;
-
-qdict_extract_subqdict(options, , "server.");
-if (!qdict_size(addr)) {
-error_setg(errp, "SSH server address missing");
-goto out;
+BlockdevOptionsSsh *result = NULL;
+QemuOpts *opts = NULL;
+Error *local_err = NULL;
+QObject *crumpled;
+const QDictEntry *e;
+Visitor *v;
+
+/* Translate legacy options */
+opts = qemu_opts_create(_runtime_opts, NULL, 0, _abort);
+qemu_opts_absorb_qdict(opts, options, _err);
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
 }
 
-crumpled_addr = qdict_crumple(addr, errp);
-if (!crumpled_addr) {
-goto out;
+if (!ssh_process_legacy_socket_options(options, opts, errp)) {
+goto fail;
+}
+
+/* Create the QAPI object */
+crumpled = qdict_crumple(options, errp);
+if (crumpled == NULL) {
+goto fail;
 }
 
 /*
@@ -609,51 +602,50 @@ static InetSocketAddress *ssh_config(QDict *options, 
Error **errp)
  * but when they come from -drive, they're all QString.  The
  * visitor expects the former.
  */
-iv = qobject_input_visitor_new(crumpled_addr);
-visit_type_InetSocketAddress(iv, NULL, , _error);
-if (local_error) {
-error_propagate(errp, local_error);
-goto out;
+v = qobject_input_visitor_new(crumpled);
+visit_type_BlockdevOptionsSsh(v, NULL, , _err);
+visit_free(v);
+qobject_decref(crumpled);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
 }
 
-out:
-QDECREF(addr);
-qobject_decref(crumpled_addr);
-visit_free(iv);
-return inet;
+/* Remove the processed options from the QDict (the visitor processes
+ * _all_ options in the QDict) */
+while ((e = qdict_first(options))) {
+qdict_del(options, e->key);
+}
+
+fail:
+qemu_opts_del(opts);
+return result;
 }
 
 static int connect_to_ssh(BDRVSSHState *s, QDict *options,
   int ssh_flags, int creat_mode, Error **errp)
 {
+BlockdevOptionsSsh *opts;
 int r, ret;
-QemuOpts *opts = NULL;
-Error *local_err = NULL;
-const char *user, *path, *host_key_check;
+const char *user, *host_key_check;
 long port = 0;
 
-opts = qemu_opts_create(_runtime_opts, NULL, 0, _abort);
-qemu_opts_absorb_qdict(opts, options, _err);
-if (local_err) {
-ret = -EINVAL;
-error_propagate(errp, local_err);
-goto err;
-}
-
-if (!ssh_process_legacy_socket_options(options, opts, errp)) {
-ret = -EINVAL;
-goto err;
+host_key_check = qdict_get_try_str(options, "host_key_check");
+if (!host_key_check) {
+host_key_check = "yes";
+} else {
+qdict_del(options, "host_key_check");
 }
 
-path = qemu_opt_get(opts, "path");
-if (!path) {
-ret = -EINVAL;
-error_setg(errp, "No path was specified");
-goto err;
+opts = ssh_parse_options(options, errp);
+if (opts == NULL) {
+return -EINVAL;
 }
 
-user = qemu_opt_get(opts, "user");
-if (!user) {
+if (opts->has_user) {
+user = opts->user;
+} else {
 user = g_get_user_name();
 if (!user) {
 error_setg_errno(errp, errno, "Can't get user name");
@@ -662,17 +654,9 @@ static int connect_to_ssh(BDRVSSHState *s, QDict *options,
 }
 }
 
-

[Qemu-devel] [PATCH v2 33/36] file-posix: Fix no-op bdrv_truncate() with falloc preallocation

[Kevin Wolf]

If bdrv_truncate() is called, but the requested size is the same as
before, don't call posix_fallocate(), which returns -EINVAL for length
zero and would therefore make bdrv_truncate() fail.

The problem can be triggered by creating a zero-sized raw image with
'falloc' preallocation mode.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/file-posix.c | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index ba14ed9459..6aed5bca0b 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1686,11 +1686,15 @@ static int raw_regular_truncate(int fd, int64_t offset, 
PreallocMode prealloc,
  * file systems that do not support fallocate(), trying to check if a
  * block is allocated before allocating it, so don't do that here.
  */
-result = -posix_fallocate(fd, current_length, offset - current_length);
-if (result != 0) {
-/* posix_fallocate() doesn't set errno. */
-error_setg_errno(errp, -result,
- "Could not preallocate new data");
+if (offset != current_length) {
+result = -posix_fallocate(fd, current_length, offset - 
current_length);
+if (result != 0) {
+/* posix_fallocate() doesn't set errno. */
+error_setg_errno(errp, -result,
+ "Could not preallocate new data");
+}
+} else {
+result = 0;
 }
 goto out;
 #endif
-- 
2.13.6

[Qemu-devel] [PATCH v2 31/36] ssh: Pass BlockdevOptionsSsh to connect_to_ssh()

[Kevin Wolf]

Move the parsing of the QDict options up to the callers, in preparation
for the .bdrv_co_create implementation that directly gets a QAPI type.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/ssh.c | 34 +-
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/block/ssh.c b/block/ssh.c
index dcf766c213..77bc20041f 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -655,19 +655,13 @@ fail:
 return result;
 }
 
-static int connect_to_ssh(BDRVSSHState *s, QDict *options,
+static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
   int ssh_flags, int creat_mode, Error **errp)
 {
-BlockdevOptionsSsh *opts;
 int r, ret;
 const char *user;
 long port = 0;
 
-opts = ssh_parse_options(options, errp);
-if (opts == NULL) {
-return -EINVAL;
-}
-
 if (opts->has_user) {
 user = opts->user;
 } else {
@@ -747,8 +741,6 @@ static int connect_to_ssh(BDRVSSHState *s, QDict *options,
 goto err;
 }
 
-qapi_free_BlockdevOptionsSsh(opts);
-
 r = libssh2_sftp_fstat(s->sftp_handle, >attrs);
 if (r < 0) {
 sftp_error_setg(errp, s, "failed to read file attributes");
@@ -774,8 +766,6 @@ static int connect_to_ssh(BDRVSSHState *s, QDict *options,
 }
 s->session = NULL;
 
-qapi_free_BlockdevOptionsSsh(opts);
-
 return ret;
 }
 
@@ -783,6 +773,7 @@ static int ssh_file_open(BlockDriverState *bs, QDict 
*options, int bdrv_flags,
  Error **errp)
 {
 BDRVSSHState *s = bs->opaque;
+BlockdevOptionsSsh *opts;
 int ret;
 int ssh_flags;
 
@@ -793,8 +784,13 @@ static int ssh_file_open(BlockDriverState *bs, QDict 
*options, int bdrv_flags,
 ssh_flags |= LIBSSH2_FXF_WRITE;
 }
 
+opts = ssh_parse_options(options, errp);
+if (opts == NULL) {
+return -EINVAL;
+}
+
 /* Start up SSH. */
-ret = connect_to_ssh(s, options, ssh_flags, 0, errp);
+ret = connect_to_ssh(s, opts, ssh_flags, 0, errp);
 if (ret < 0) {
 goto err;
 }
@@ -802,6 +798,8 @@ static int ssh_file_open(BlockDriverState *bs, QDict 
*options, int bdrv_flags,
 /* Go non-blocking. */
 libssh2_session_set_blocking(s->session, 0);
 
+qapi_free_BlockdevOptionsSsh(opts);
+
 return 0;
 
  err:
@@ -810,6 +808,8 @@ static int ssh_file_open(BlockDriverState *bs, QDict 
*options, int bdrv_flags,
 }
 s->sock = -1;
 
+qapi_free_BlockdevOptionsSsh(opts);
+
 return ret;
 }
 
@@ -831,6 +831,7 @@ static int ssh_create(const char *filename, QemuOpts *opts, 
Error **errp)
 int r, ret;
 int64_t total_size = 0;
 QDict *uri_options = NULL;
+BlockdevOptionsSsh *ssh_opts = NULL;
 BDRVSSHState s;
 ssize_t r2;
 char c[1] = { '\0' };
@@ -849,7 +850,13 @@ static int ssh_create(const char *filename, QemuOpts 
*opts, Error **errp)
 goto out;
 }
 
-r = connect_to_ssh(, uri_options,
+ssh_opts = ssh_parse_options(uri_options, errp);
+if (ssh_opts == NULL) {
+ret = -EINVAL;
+goto out;
+}
+
+r = connect_to_ssh(, ssh_opts,
LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
0644, errp);
@@ -876,6 +883,7 @@ static int ssh_create(const char *filename, QemuOpts *opts, 
Error **errp)
 if (uri_options != NULL) {
 QDECREF(uri_options);
 }
+qapi_free_BlockdevOptionsSsh(ssh_opts);
 return ret;
 }
 
-- 
2.13.6

Re: [Qemu-devel] [PATCH v2 3/5] keymap: numpad keysyms and keycodes are fixed

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:03:55PM +0100, Gerd Hoffmann wrote:
> No need to figure them at runtime from the keymap.
> 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/keymaps.c | 61 
> +---
>  1 file changed, 9 insertions(+), 52 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v2 5/5] keymap: consider modifier state when picking a mapping

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:03:57PM +0100, Gerd Hoffmann wrote:
> Pass the modifier state to the keymap lookup function.  In case multiple
> keysym -> keycode mappings exist look at the modifier state and prefer
> the mapping where the modifier state matches.
> 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/keymaps.h |  3 ++-
>  ui/curses.c  |  3 ++-
>  ui/keymaps.c | 33 -
>  ui/sdl.c |  6 +-
>  ui/vnc.c |  9 +++--
>  5 files changed, 48 insertions(+), 6 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH] sdl2: fix hotkey keyup

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:04:44PM +0100, Gerd Hoffmann wrote:
> After some hotkey was pressed sdl2 doesn't forward the first modifier
> keyup event to the guest, resulting in stuck modifier keys.
> 
> Fix the logic in handle_keyup().  Also gui_key_modifier_pressed doesn't
> need to be a global variable.
> 
> Reported-by: Howard Spoelstra 
> Tested-by: Howard Spoelstra 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/sdl2.c | 14 +++---
>  1 file changed, 3 insertions(+), 11 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH] hw/acpi-build: build SRAT memory affinity structures for NVDIMM

[Dan Williams]

On Wed, Feb 21, 2018 at 5:55 AM, Igor Mammedov  wrote:
> On Tue, 20 Feb 2018 17:17:58 -0800
> Dan Williams  wrote:
>
>> On Tue, Feb 20, 2018 at 6:10 AM, Igor Mammedov  wrote:
>> > On Sat, 17 Feb 2018 14:31:35 +0800
>> > Haozhong Zhang  wrote:
>> >
>> >> ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity
>> >> domain of a NVDIMM SPA range must match with corresponding entry in
>> >> SRAT table.
>> >>
>> >> The address ranges of vNVDIMM in QEMU are allocated from the
>> >> hot-pluggable address space, which is entirely covered by one SRAT
>> >> memory affinity structure. However, users can set the vNVDIMM
>> >> proximity domain in NFIT SPA range structure by the 'node' property of
>> >> '-device nvdimm' to a value different than the one in the above SRAT
>> >> memory affinity structure.
>> >>
>> >> In order to solve such proximity domain mismatch, this patch build one
>> >> SRAT memory affinity structure for each NVDIMM device with the
>> >> proximity domain used in NFIT. The remaining hot-pluggable address
>> >> space is covered by one or multiple SRAT memory affinity structures
>> >> with the proximity domain of the last node as before.
>> >>
>> >> Signed-off-by: Haozhong Zhang 
>> > If we consider hotpluggable system, correctly implemented OS should
>> > be able pull proximity from Device::_PXM and override any value from SRAT.
>> > Do we really have a problem here (anything that breaks if we would use 
>> > _PXM)?
>> > Maybe we should add _PXM object to nvdimm device nodes instead of 
>> > massaging SRAT?
>>
>> Unfortunately _PXM is an awkward fit. Currently the proximity domain
>> is attached to the SPA range structure. The SPA range may be
>> associated with multiple DIMM devices and those individual NVDIMMs may
>> have conflicting _PXM properties.
> There shouldn't be any conflict here as  NVDIMM device's _PXM method,
> should override in runtime any proximity specified by parent scope.
> (as parent scope I'd also count boot time NFIT/SRAT tables).
>
> To make it more clear we could clear valid proximity domain flag in SPA
> like this:
>
> diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
> index 59d6e42..131bca5 100644
> --- a/hw/acpi/nvdimm.c
> +++ b/hw/acpi/nvdimm.c
> @@ -260,9 +260,7 @@ nvdimm_build_structure_spa(GArray *structures, 
> DeviceState *dev)
>   */
>  nfit_spa->flags = cpu_to_le16(1 /* Control region is strictly for
> management during hot add/online
> -   operation */ |
> -  2 /* Data in Proximity Domain field is
> -   valid*/);
> +   operation */);
>
>  /* NUMA node. */
>  nfit_spa->proximity_domain = cpu_to_le32(node);
>
>> Even if that was unified across
>> DIMMs it is ambiguous whether a DIMM-device _PXM would relate to the
>> device's control interface, or the assembled persistent memory SPA
>> range.
> I'm not sure what you mean under 'device's control interface',
> could you clarify where the ambiguity comes from?

There are multiple SPA range types. In addition to the typical
Persistent Memory SPA range there are also Control Region SPA ranges
for MMIO registers on the DIMM for Block Apertures and other purposes.

>
> I read spec as: _PXM applies to address range covered by NVDIMM
> device it belongs to.

No, an NVDIMM may contribute to multiple SPA ranges and those ranges
may span sockets.

>
> As for assembled SPA, I'd assume that it applies to interleaved set
> and all NVDIMMs with it should be on the same node. It's somewhat
> irrelevant question though as QEMU so far implements only
>   1:1:1/SPA:Region Mapping:NVDIMM Device/
> mapping.
>
> My main concern with using static configuration tables for proximity
> mapping, we'd miss on hotplug side of equation. However if we start
> from dynamic side first, we could later complement it with static
> tables if there really were need for it.

Especially when you consider the new HMAT table that wants to have
proximity domains for describing performance characteristics of an
address range relative to an initiator, the _PXM method on an
individual NVDIMM device is a poor fit for describing a wider set.

[Qemu-devel] [PATCH v2 10/36] test-qemu-opts: Test qemu_opts_to_qdict_filtered()

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 tests/test-qemu-opts.c | 125 +
 1 file changed, 125 insertions(+)

diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
index 6c3183390b..2c422abcd4 100644
--- a/tests/test-qemu-opts.c
+++ b/tests/test-qemu-opts.c
@@ -10,6 +10,7 @@
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
 #include "qemu/option.h"
+#include "qemu/option_int.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qstring.h"
@@ -868,6 +869,127 @@ static void test_opts_append(void)
 qemu_opts_free(merged);
 }
 
+static void test_opts_to_qdict_basic(void)
+{
+QemuOpts *opts;
+QDict *dict;
+
+opts = qemu_opts_parse(_list_01, "str1=foo,str2=,str3=bar,number1=42",
+   false, _abort);
+g_assert(opts != NULL);
+
+dict = qemu_opts_to_qdict(opts, NULL);
+g_assert(dict != NULL);
+
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+
+QDECREF(dict);
+qemu_opts_del(opts);
+}
+
+static void test_opts_to_qdict_filtered(void)
+{
+QemuOptsList *first, *merged;
+QemuOpts *opts;
+QDict *dict;
+
+first = qemu_opts_append(NULL, _list_02);
+merged = qemu_opts_append(first, _list_01);
+
+opts = qemu_opts_parse(merged,
+   "str1=foo,str2=,str3=bar,bool1=off,number1=42",
+   false, _abort);
+g_assert(opts != NULL);
+
+/* Convert to QDict without deleting from opts */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, _list_01, false);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+g_assert_false(qdict_haskey(dict, "bool1"));
+QDECREF(dict);
+
+dict = qemu_opts_to_qdict_filtered(opts, NULL, _list_02, false);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "bool1"), ==, "off");
+g_assert_false(qdict_haskey(dict, "str3"));
+g_assert_false(qdict_haskey(dict, "number1"));
+g_assert_false(qdict_haskey(dict, "number2"));
+QDECREF(dict);
+
+/* Now delete converted options from opts */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, _list_01, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+g_assert_false(qdict_haskey(dict, "bool1"));
+QDECREF(dict);
+
+dict = qemu_opts_to_qdict_filtered(opts, NULL, _list_02, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "bool1"), ==, "off");
+g_assert_false(qdict_haskey(dict, "str1"));
+g_assert_false(qdict_haskey(dict, "str2"));
+g_assert_false(qdict_haskey(dict, "str3"));
+g_assert_false(qdict_haskey(dict, "number1"));
+g_assert_false(qdict_haskey(dict, "number2"));
+QDECREF(dict);
+
+g_assert_true(QTAILQ_EMPTY(>head));
+
+qemu_opts_del(opts);
+qemu_opts_free(merged);
+}
+
+static void test_opts_to_qdict_duplicates(void)
+{
+QemuOpts *opts;
+QemuOpt *opt;
+QDict *dict;
+
+opts = qemu_opts_parse(_list_03, "foo=a,foo=b", false, _abort);
+g_assert(opts != NULL);
+
+/* Verify that opts has two options with the same name */
+opt = QTAILQ_FIRST(>head);
+g_assert_cmpstr(opt->name, ==, "foo");
+g_assert_cmpstr(opt->str , ==, "a");
+
+opt = QTAILQ_NEXT(opt, next);
+g_assert_cmpstr(opt->name, ==, "foo");
+g_assert_cmpstr(opt->str , ==, "b");
+
+opt = QTAILQ_NEXT(opt, next);
+g_assert(opt == NULL);
+
+/* In the conversion to QDict, the last one wins */
+dict = qemu_opts_to_qdict(opts, NULL);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b");
+QDECREF(dict);
+
+/* The last one still wins if entries are deleted, and both are deleted */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, NULL, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b");
+QDECREF(dict);
+
+g_assert_true(QTAILQ_EMPTY(>head));
+
+qemu_opts_del(opts);
+}
 
 int main(int argc, char *argv[])
 {
@@ -889,6 +1011,9 @@ int main(int argc, char *argv[])
 

[Qemu-devel] [PATCH v2 24/36] rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()

[Kevin Wolf]

This is almost exactly the same code. The differences are that
qemu_rbd_connect() supports BlockdevOptionsRbd.server and that the cache
mode is set explicitly.

Supporting 'server' is a welcome new feature for image creation.
Caching is disabled by default, so leave it that way.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 54 ++
 1 file changed, 10 insertions(+), 44 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index a34bf0be46..af8e186106 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -103,6 +103,11 @@ typedef struct BDRVRBDState {
 char *snap;
 } BDRVRBDState;
 
+static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
+BlockdevOptionsRbd *opts, bool cache,
+const char *keypairs, const char *secretid,
+Error **errp);
+
 static char *qemu_rbd_next_tok(char *src, char delim, char **p)
 {
 char *end;
@@ -351,12 +356,6 @@ static int qemu_rbd_do_create(BlockdevCreateOptions 
*options,
 return -EINVAL;
 }
 
-/* TODO Remove the limitation */
-if (opts->location->has_server) {
-error_setg(errp, "Can't specify server for image creation");
-return -EINVAL;
-}
-
 if (opts->has_cluster_size) {
 int64_t objsize = opts->cluster_size;
 if ((objsize - 1) & objsize) {/* not a power of 2? */
@@ -370,54 +369,21 @@ static int qemu_rbd_do_create(BlockdevCreateOptions 
*options,
 obj_order = ctz32(objsize);
 }
 
-ret = rados_create(, opts->location->user);
+ret = qemu_rbd_connect(, _ctx, opts->location, false, keypairs,
+   NULL, errp);
 if (ret < 0) {
-error_setg_errno(errp, -ret, "error initializing");
 return ret;
 }
 
-/* try default location when conf=NULL, but ignore failure */
-ret = rados_conf_read_file(cluster, opts->location->conf);
-if (opts->location->conf && ret < 0) {
-error_setg_errno(errp, -ret, "error reading conf file %s",
- opts->location->conf);
-ret = -EIO;
-goto shutdown;
-}
-
-ret = qemu_rbd_set_keypairs(cluster, keypairs, errp);
-if (ret < 0) {
-ret = -EIO;
-goto shutdown;
-}
-
-if (qemu_rbd_set_auth(cluster, password_secret, errp) < 0) {
-ret = -EIO;
-goto shutdown;
-}
-
-ret = rados_connect(cluster);
-if (ret < 0) {
-error_setg_errno(errp, -ret, "error connecting");
-goto shutdown;
-}
-
-ret = rados_ioctx_create(cluster, opts->location->pool, _ctx);
-if (ret < 0) {
-error_setg_errno(errp, -ret, "error opening pool %s",
- opts->location->pool);
-goto shutdown;
-}
-
 ret = rbd_create(io_ctx, opts->location->image, opts->size, _order);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "error rbd create");
+goto out;
 }
 
-rados_ioctx_destroy(io_ctx);
-
 ret = 0;
-shutdown:
+out:
+rados_ioctx_destroy(io_ctx);
 rados_shutdown(cluster);
 return ret;
 }
-- 
2.13.6

[Qemu-devel] [PATCH v2 34/36] block: Fail bdrv_truncate() with negative size

[Kevin Wolf]

Most callers have their own checks, but something like this should also
be checked centrally. As it happens, x-blockdev-create can pass negative
image sizes to format drivers (because there is no QAPI type that would
reject negative numbers) and triggers the check added by this patch.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/block.c b/block.c
index 4a7e448226..5c874aefa1 100644
--- a/block.c
+++ b/block.c
@@ -3684,6 +3684,11 @@ int bdrv_truncate(BdrvChild *child, int64_t offset, 
PreallocMode prealloc,
 error_setg(errp, "No medium inserted");
 return -ENOMEDIUM;
 }
+if (offset < 0) {
+error_setg(errp, "Image size cannot be negative");
+return -EINVAL;
+}
+
 if (!drv->bdrv_truncate) {
 if (bs->file && drv->is_filter) {
 return bdrv_truncate(bs->file, offset, prealloc, errp);
-- 
2.13.6

[Qemu-devel] [PATCH v2 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

[Kevin Wolf]

With the conversion to a QAPI options object, the function is now
prepared to be used in a .bdrv_co_create implementation.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 102 +++-
 1 file changed, 52 insertions(+), 50 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 2e79c2d1fd..26641e53e0 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -24,6 +24,8 @@
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/qlist.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qapi-visit.h"
 
 /*
  * When specifying the image filename use:
@@ -482,24 +484,27 @@ static void qemu_rbd_complete_aio(RADOSCB *rcb)
 qemu_aio_unref(acb);
 }
 
-static char *qemu_rbd_mon_host(QDict *options, Error **errp)
+static char *qemu_rbd_mon_host(BlockdevOptionsRbd *opts, Error **errp)
 {
-const char **vals = g_new(const char *, qdict_size(options) + 1);
-char keybuf[32];
+const char **vals;
 const char *host, *port;
 char *rados_str;
-int i;
-
-for (i = 0;; i++) {
-sprintf(keybuf, "server.%d.host", i);
-host = qdict_get_try_str(options, keybuf);
-qdict_del(options, keybuf);
-sprintf(keybuf, "server.%d.port", i);
-port = qdict_get_try_str(options, keybuf);
-qdict_del(options, keybuf);
-if (!host && !port) {
-break;
-}
+InetSocketAddressBaseList *p;
+int i, cnt;
+
+if (!opts->has_server) {
+return NULL;
+}
+
+for (cnt = 0, p = opts->server; p; p = p->next) {
+cnt++;
+}
+
+vals = g_new(const char *, cnt + 1);
+
+for (i = 0, p = opts->server; p; p = p->next, i++) {
+host = p->value->host;
+port = p->value->port;
 if (!host) {
 error_setg(errp, "Parameter server.%d.host is missing", i);
 rados_str = NULL;
@@ -524,56 +529,34 @@ out:
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
 char **s_snap, char **s_image_name,
-QDict *options, bool cache,
+BlockdevOptionsRbd *opts, bool cache,
 const char *keypairs, const char *secretid,
 Error **errp)
 {
-QemuOpts *opts;
 char *mon_host = NULL;
-const char *pool, *snap, *conf, *user, *image_name;
 Error *local_err = NULL;
 int r;
 
-opts = qemu_opts_create(_opts, NULL, 0, _abort);
-qemu_opts_absorb_qdict(opts, options, _err);
+mon_host = qemu_rbd_mon_host(opts, _err);
 if (local_err) {
 error_propagate(errp, local_err);
 r = -EINVAL;
 goto failed_opts;
 }
 
-mon_host = qemu_rbd_mon_host(options, _err);
-if (local_err) {
-error_propagate(errp, local_err);
-r = -EINVAL;
-goto failed_opts;
-}
-
-pool   = qemu_opt_get(opts, "pool");
-conf   = qemu_opt_get(opts, "conf");
-snap   = qemu_opt_get(opts, "snapshot");
-user   = qemu_opt_get(opts, "user");
-image_name = qemu_opt_get(opts, "image");
-
-if (!pool || !image_name) {
-error_setg(errp, "Parameters 'pool' and 'image' are required");
-r = -EINVAL;
-goto failed_opts;
-}
-
-r = rados_create(cluster, user);
+r = rados_create(cluster, opts->user);
 if (r < 0) {
 error_setg_errno(errp, -r, "error initializing");
 goto failed_opts;
 }
 
-*s_snap = g_strdup(snap);
-*s_image_name = g_strdup(image_name);
+*s_snap = g_strdup(opts->snapshot);
+*s_image_name = g_strdup(opts->image);
 
 /* try default location when conf=NULL, but ignore failure */
-r = rados_conf_read_file(*cluster, conf);
-if (conf && r < 0) {
-error_setg_errno(errp, -r, "error reading conf file %s", conf);
+r = rados_conf_read_file(*cluster, opts->conf);
+if (opts->has_conf && r < 0) {
+error_setg_errno(errp, -r, "error reading conf file %s", opts->conf);
 goto failed_shutdown;
 }
 
@@ -613,13 +596,12 @@ static int qemu_rbd_connect(rados_t *cluster, 
rados_ioctx_t *io_ctx,
 goto failed_shutdown;
 }
 
-r = rados_ioctx_create(*cluster, pool, io_ctx);
+r = rados_ioctx_create(*cluster, opts->pool, io_ctx);
 if (r < 0) {
-error_setg_errno(errp, -r, "error opening pool %s", pool);
+error_setg_errno(errp, -r, "error opening pool %s", opts->pool);
 goto failed_shutdown;
 }
 
-qemu_opts_del(opts);
 return 0;
 
 failed_shutdown:
@@ -627,7 +609,6 @@ failed_shutdown:
 g_free(*s_snap);
 g_free(*s_image_name);
 failed_opts:
-qemu_opts_del(opts);
 g_free(mon_host);
 return r;
 }
@@ -636,6 +617,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  Error **errp)
 {
 BDRVRBDState *s = bs->opaque;
+BlockdevOptionsRbd *opts = NULL;
+

[Qemu-devel] [PATCH v3] specs/qcow2: Fix documentation of the compressed cluster descriptor

[Alberto Garcia]

This patch fixes several mistakes in the documentation of the
compressed cluster descriptor:

1) the documentation claims that the cluster descriptor contains the
   number of sectors used to store the compressed data, but what it
   actually contains is the number of sectors *minus one* or, in other
   words, the number of additional sectors after the first one.

2) the width of the fields is incorrectly specified. The number of bits
   used by each field is

  x = 62 - (cluster_bits - 8)   for the offset field
  y = (cluster_bits - 8)for the size field

   So the offset field's location is [0, x-1], not [0, x] as stated.

3) the size field does not contain the size of the compressed data,
   but rather the number of sectors where that data is stored. The
   compressed data starts at the exact point specified in the offset
   field and ends when there's enough data to produce a cluster of
   decompressed data. Both points can be in the middle of a sector,
   allowing several compressed clusters to be stored next to one
   another, sharing sectors if necessary.

Signed-off-by: Alberto Garcia 
---

v3: Fix the specification of the width of the fields, and update the
explanation of how the compressed data is stored [Eric].

v2: I realized that the documentation is not completely clear about
the exact location and size of the compressed data, so I updated
the patch to clarify this.

---
 docs/interop/qcow2.txt | 16 +---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt
index d7fdb1fee3..feb711fb6a 100644
--- a/docs/interop/qcow2.txt
+++ b/docs/interop/qcow2.txt
@@ -426,10 +426,20 @@ Standard Cluster Descriptor:
 
 Compressed Clusters Descriptor (x = 62 - (cluster_bits - 8)):
 
-Bit  0 -  x:Host cluster offset. This is usually _not_ aligned to a
-cluster boundary!
+Bit  0 - x-1:   Host cluster offset. This is usually _not_ aligned to a
+cluster or sector boundary!
 
-   x+1 - 61:Compressed size of the images in sectors of 512 bytes
+ x - 61:Number of additional 512-byte sectors used for the
+compressed data, beyond the sector containing the offset
+in the previous field. Some of these sectors may reside
+in the next contiguous host cluster.
+
+Note that the compressed data does not necessarily occupy
+all of the bytes in the final sector; rather, decompression
+stops when it has produced a cluster of data.
+
+Another compressed cluster may map to the tail of the final
+sector used by this compressed cluster.
 
 If a cluster is unallocated, read requests shall read the data from the backing
 file (except if bit 0 in the Standard Cluster Descriptor is set). If there is
-- 
2.11.0

[Qemu-devel] [PATCH v2 25/36] nfs: Use QAPI options in nfs_client_open()

[Kevin Wolf]

Using the QAPI visitor to turn all options into QAPI BlockdevOptionsNfs
simplifies the code a lot. It will also be useful for implementing the
QAPI based .bdrv_co_create callback.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/nfs.c | 176 ++--
 1 file changed, 53 insertions(+), 123 deletions(-)

diff --git a/block/nfs.c b/block/nfs.c
index 6576a73d6e..9283bfbaae 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -367,49 +367,6 @@ static int coroutine_fn nfs_co_flush(BlockDriverState *bs)
 return task.ret;
 }
 
-static QemuOptsList runtime_opts = {
-.name = "nfs",
-.head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head),
-.desc = {
-{
-.name = "path",
-.type = QEMU_OPT_STRING,
-.help = "Path of the image on the host",
-},
-{
-.name = "user",
-.type = QEMU_OPT_NUMBER,
-.help = "UID value to use when talking to the server",
-},
-{
-.name = "group",
-.type = QEMU_OPT_NUMBER,
-.help = "GID value to use when talking to the server",
-},
-{
-.name = "tcp-syn-count",
-.type = QEMU_OPT_NUMBER,
-.help = "Number of SYNs to send during the session establish",
-},
-{
-.name = "readahead-size",
-.type = QEMU_OPT_NUMBER,
-.help = "Set the readahead size in bytes",
-},
-{
-.name = "page-cache-size",
-.type = QEMU_OPT_NUMBER,
-.help = "Set the pagecache size in bytes",
-},
-{
-.name = "debug",
-.type = QEMU_OPT_NUMBER,
-.help = "Set the NFS debug level (max 2)",
-},
-{ /* end of list */ }
-},
-};
-
 static void nfs_detach_aio_context(BlockDriverState *bs)
 {
 NFSClient *client = bs->opaque;
@@ -452,71 +409,16 @@ static void nfs_file_close(BlockDriverState *bs)
 nfs_client_close(client);
 }
 
-static NFSServer *nfs_config(QDict *options, Error **errp)
-{
-NFSServer *server = NULL;
-QDict *addr = NULL;
-QObject *crumpled_addr = NULL;
-Visitor *iv = NULL;
-Error *local_error = NULL;
-
-qdict_extract_subqdict(options, , "server.");
-if (!qdict_size(addr)) {
-error_setg(errp, "NFS server address missing");
-goto out;
-}
-
-crumpled_addr = qdict_crumple(addr, errp);
-if (!crumpled_addr) {
-goto out;
-}
-
-/*
- * Caution: this works only because all scalar members of
- * NFSServer are QString in @crumpled_addr.  The visitor expects
- * @crumpled_addr to be typed according to the QAPI schema.  It
- * is when @options come from -blockdev or blockdev_add.  But when
- * they come from -drive, they're all QString.
- */
-iv = qobject_input_visitor_new(crumpled_addr);
-visit_type_NFSServer(iv, NULL, , _error);
-if (local_error) {
-error_propagate(errp, local_error);
-goto out;
-}
-
-out:
-QDECREF(addr);
-qobject_decref(crumpled_addr);
-visit_free(iv);
-return server;
-}
-
-
-static int64_t nfs_client_open(NFSClient *client, QDict *options,
+static int64_t nfs_client_open(NFSClient *client, BlockdevOptionsNfs *opts,
int flags, int open_flags, Error **errp)
 {
 int64_t ret = -EINVAL;
-QemuOpts *opts = NULL;
-Error *local_err = NULL;
 struct stat st;
 char *file = NULL, *strp = NULL;
 
 qemu_mutex_init(>mutex);
-opts = qemu_opts_create(_opts, NULL, 0, _abort);
-qemu_opts_absorb_qdict(opts, options, _err);
-if (local_err) {
-error_propagate(errp, local_err);
-ret = -EINVAL;
-goto fail;
-}
 
-client->path = g_strdup(qemu_opt_get(opts, "path"));
-if (!client->path) {
-ret = -EINVAL;
-error_setg(errp, "No path was specified");
-goto fail;
-}
+client->path = g_strdup(opts->path);
 
 strp = strrchr(client->path, '/');
 if (strp == NULL) {
@@ -526,12 +428,10 @@ static int64_t nfs_client_open(NFSClient *client, QDict 
*options,
 file = g_strdup(strp);
 *strp = 0;
 
-/* Pop the config into our state object, Exit if invalid */
-client->server = nfs_config(options, errp);
-if (!client->server) {
-ret = -EINVAL;
-goto fail;
-}
+/* Steal the NFSServer object from opts; set the original pointer to NULL
+ * to avoid use after free and double free. */
+client->server = opts->server;
+opts->server = NULL;
 
 client->context = nfs_init_context();
 if (client->context == NULL) {
@@ -539,29 +439,29 @@ static int64_t nfs_client_open(NFSClient *client, QDict 
*options,
 goto fail;
 }
 
-if (qemu_opt_get(opts, "user")) {
-client->uid = qemu_opt_get_number(opts, "user", 0);
+if (opts->has_user) {
+

[Qemu-devel] [PATCH v2 36/36] qemu-iotests: Test ssh image creation over QMP

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/207 | 261 +
 tests/qemu-iotests/207.out |  75 +
 tests/qemu-iotests/group   |   1 +
 3 files changed, 337 insertions(+)
 create mode 100755 tests/qemu-iotests/207
 create mode 100644 tests/qemu-iotests/207.out

diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
new file mode 100755
index 00..f5c77852d1
--- /dev/null
+++ b/tests/qemu-iotests/207
@@ -0,0 +1,261 @@
+#!/bin/bash
+#
+# Test ssh image creation
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=kw...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt raw
+_supported_proto ssh
+_supported_os Linux
+
+function do_run_qemu()
+{
+echo Testing: "$@"
+$QEMU -nographic -qmp stdio -serial none "$@"
+echo
+}
+
+function run_qemu()
+{
+do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \
+  | _filter_qemu | _filter_imgfmt \
+  | _filter_actual_image_size
+}
+
+echo
+echo "=== Successful image creation (defaults) ==="
+echo
+
+run_qemu 

Re: [Qemu-devel] [PATCH] Fix ast2500 protection register emulation

[Cédric Le Goater]

On 02/20/2018 03:19 PM, Hugo Landau wrote:
>> I also gave it a test on an OpenBMC romulus image. Looks fine, but that's 
>> an old custom U-Boot. Which defconfig did you use for U-Boot HEAD ? 
> evb-ast2500_defconfig.

ok

> FYI, these changes are necessary, but not sufficient to get u-boot HEAD
> (or for that matter u-boot 2017.11, another version tested) running.
> 
> The other issues were
>   - the tests
>   while (!(readl(>ecc_test_ctrl) & SDRAM_TEST_DONE));
> and
>   while (!(readl(>regs->config) & SDRAM_CONF_CACHE_INIT_DONE));
> which appear in various places in the u-boot source and which spin
> forever. I made u-boot work by commenting these out in u-boot rather
> than patching qemu, not familiar enough with qemu to implement this.

This patch :

  
https://github.com/openbmc/qemu/commit/4fb98fffd3115d8d3d0a16a1033f5335b5c0fd9b

fakes some more SDMC registers to let the SDRAM initialization run. you
might want to take a look at it.

Thanks,

C.  

>   - the call to reset_assert in ast2500_sdrammc_probe seems to actually
> reset the machine rather than just initialize SDRAM as it is
> apparently supposed to, leading to an infinite cycle of resets.
> Couldn't quite figure out how it was supposed to work, so I
> commented this out, since obviously qemu doesn't actually have SDRAM
> initialization requirements.
> 
> The above changes plus this patch allowed u-boot to get to the u-boot
> CLI. Haven't tried booting anything with it yet though.
> 

Re: [Qemu-devel] [PATCH v2 4/5] keymap: record multiple keysym -> keycode mappings

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:03:56PM +0100, Gerd Hoffmann wrote:
> Sometimes the same keysym can be created using different key
> combinations.  Record them all in the reverse keymap, not only
> the first one.
> 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/keymaps.c | 14 +++---
>  1 file changed, 11 insertions(+), 3 deletions(-)

Reviewed-by: Daniel P. Berrangé 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

[Qemu-devel] [PATCH v2 13/36] block: Make bdrv_is_whitelisted() public

[Kevin Wolf]

We'll use a separate source file for image creation, and we need to
check there whether the requested driver is whitelisted.

Signed-off-by: Kevin Wolf 
---
 include/block/block.h | 1 +
 block.c   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/block/block.h b/include/block/block.h
index 54fe8b7a0e..cfce88cbda 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -225,6 +225,7 @@ char *bdrv_perm_names(uint64_t perm);
 void bdrv_init(void);
 void bdrv_init_with_whitelist(void);
 bool bdrv_uses_whitelist(void);
+int bdrv_is_whitelisted(BlockDriver *drv, bool read_only);
 BlockDriver *bdrv_find_protocol(const char *filename,
 bool allow_protocol_prefix,
 Error **errp);
diff --git a/block.c b/block.c
index c0e343d278..4a7e448226 100644
--- a/block.c
+++ b/block.c
@@ -372,7 +372,7 @@ BlockDriver *bdrv_find_format(const char *format_name)
 return bdrv_do_find_format(format_name);
 }
 
-static int bdrv_is_whitelisted(BlockDriver *drv, bool read_only)
+int bdrv_is_whitelisted(BlockDriver *drv, bool read_only)
 {
 static const char *whitelist_rw[] = {
 CONFIG_BDRV_RW_WHITELIST
-- 
2.13.6

[Qemu-devel] [PATCH v2 15/36] file-posix: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to file, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 20 +-
 block/file-posix.c   | 77 +---
 2 files changed, 74 insertions(+), 23 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 359195a1a3..0040795603 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,24 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevCreateOptionsFile:
+#
+# Driver specific image creation options for file.
+#
+# @filename Filename for the new image file
+# @size Size of the virtual disk in bytes
+# @preallocationPreallocation mode for the new image (default: off)
+# @nocowTurn off copy-on-write (valid only on btrfs; default: off)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsFile',
+  'data': { 'filename': 'str',
+'size': 'size',
+'*preallocation':   'PreallocMode',
+'*nocow':   'bool' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3429,7 +3447,7 @@
   'bochs':  'BlockdevCreateNotSupported',
   'cloop':  'BlockdevCreateNotSupported',
   'dmg':'BlockdevCreateNotSupported',
-  'file':   'BlockdevCreateNotSupported',
+  'file':   'BlockdevCreateOptionsFile',
   'ftp':'BlockdevCreateNotSupported',
   'ftps':   'BlockdevCreateNotSupported',
   'gluster':'BlockdevCreateNotSupported',
diff --git a/block/file-posix.c b/block/file-posix.c
index f1591c3849..ba14ed9459 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1982,33 +1982,25 @@ static int64_t 
raw_get_allocated_file_size(BlockDriverState *bs)
 return (int64_t)st.st_blocks * 512;
 }
 
-static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+static int raw_co_create(BlockdevCreateOptions *options, Error **errp)
 {
+BlockdevCreateOptionsFile *file_opts;
 int fd;
 int result = 0;
-int64_t total_size = 0;
-bool nocow = false;
-PreallocMode prealloc;
-char *buf = NULL;
-Error *local_err = NULL;
 
-strstart(filename, "file:", );
+/* Validate options and set default values */
+assert(options->driver == BLOCKDEV_DRIVER_FILE);
+file_opts = >u.file;
 
-/* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
-nocow = qemu_opt_get_bool(opts, BLOCK_OPT_NOCOW, false);
-buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
-prealloc = qapi_enum_parse(_lookup, buf,
-   PREALLOC_MODE_OFF, _err);
-g_free(buf);
-if (local_err) {
-error_propagate(errp, local_err);
-result = -EINVAL;
-goto out;
+if (!file_opts->has_nocow) {
+file_opts->nocow = false;
+}
+if (!file_opts->has_preallocation) {
+file_opts->preallocation = PREALLOC_MODE_OFF;
 }
 
-fd = qemu_open(filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY,
+/* Create file */
+fd = qemu_open(file_opts->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY,
0644);
 if (fd < 0) {
 result = -errno;
@@ -2016,7 +2008,7 @@ static int raw_create(const char *filename, QemuOpts 
*opts, Error **errp)
 goto out;
 }
 
-if (nocow) {
+if (file_opts->nocow) {
 #ifdef __linux__
 /* Set NOCOW flag to solve performance issue on fs like btrfs.
  * This is an optimisation. The FS_IOC_SETFLAGS ioctl return value
@@ -2031,7 +2023,8 @@ static int raw_create(const char *filename, QemuOpts 
*opts, Error **errp)
 #endif
 }
 
-result = raw_regular_truncate(fd, total_size, prealloc, errp);
+result = raw_regular_truncate(fd, file_opts->size, 
file_opts->preallocation,
+  errp);
 if (result < 0) {
 goto out_close;
 }
@@ -2045,6 +2038,45 @@ out:
 return result;
 }
 
+static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions options;
+int64_t total_size = 0;
+bool nocow = false;
+PreallocMode prealloc;
+char *buf = NULL;
+Error *local_err = NULL;
+
+/* Skip file: protocol prefix */
+strstart(filename, "file:", );
+
+/* Read out options */
+total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
+nocow = qemu_opt_get_bool(opts, BLOCK_OPT_NOCOW, false);
+buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
+prealloc = qapi_enum_parse(_lookup, buf,
+   PREALLOC_MODE_OFF, _err);
+g_free(buf);
+if (local_err) {
+

[Qemu-devel] [PATCH v2 12/36] qcow2: Use visitor for options in qcow2_create()

[Kevin Wolf]

Instead of manually creating the BlockdevCreateOptions object, use a
visitor to parse the given options into the QAPI object.

This involves translation from the old command line syntax to the syntax
mandated by the QAPI schema. Option names are still checked against
qcow2_create_opts, so only the old option names are allowed on the
command line, even if they are translated in qcow2_create().

In contrast, new option values are optionally recognised besides the old
values: 'compat' accepts 'v2'/'v3' as an alias for '0.10'/'1.1', and
'encrypt.format' accepts 'qcow' as an alias for 'aes' now.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 block/qcow2.c  | 217 -
 tests/qemu-iotests/049.out |   8 +-
 tests/qemu-iotests/112.out |   4 +-
 3 files changed, 83 insertions(+), 146 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 64bf2863cd..58737d0833 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -38,7 +38,7 @@
 #include "qemu/option_int.h"
 #include "qemu/cutils.h"
 #include "qemu/bswap.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi-visit.h"
 #include "block/crypto.h"
 
@@ -2414,37 +2414,6 @@ static int qcow2_crypt_method_from_format(const char 
*encryptfmt)
 }
 }
 
-static QCryptoBlockCreateOptions *
-qcow2_parse_encryption(const char *encryptfmt, QemuOpts *opts, Error **errp)
-{
-QCryptoBlockCreateOptions *cryptoopts = NULL;
-QDict *options, *encryptopts;
-int fmt;
-
-options = qemu_opts_to_qdict(opts, NULL);
-qdict_extract_subqdict(options, , "encrypt.");
-QDECREF(options);
-
-fmt = qcow2_crypt_method_from_format(encryptfmt);
-
-switch (fmt) {
-case QCOW_CRYPT_LUKS:
-cryptoopts = block_crypto_create_opts_init(
-Q_CRYPTO_BLOCK_FORMAT_LUKS, encryptopts, errp);
-break;
-case QCOW_CRYPT_AES:
-cryptoopts = block_crypto_create_opts_init(
-Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp);
-break;
-default:
-error_setg(errp, "Unknown encryption format '%s'", encryptfmt);
-break;
-}
-
-QDECREF(encryptopts);
-return cryptoopts;
-}
-
 static int qcow2_set_up_encryption(BlockDriverState *bs,
QCryptoBlockCreateOptions *cryptoopts,
Error **errp)
@@ -2838,7 +2807,7 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 if (version < 3 && qcow2_opts->lazy_refcounts) {
 error_setg(errp, "Lazy refcounts only supported with compatibility "
-   "level 1.1 and above (use compat=1.1 or greater)");
+   "level 1.1 and above (use version=v3 or greater)");
 ret = -EINVAL;
 goto out;
 }
@@ -2856,7 +2825,7 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 if (version < 3 && qcow2_opts->refcount_bits != 16) {
 error_setg(errp, "Different refcount widths than 16 bits require "
-   "compatibility level 1.1 or above (use compat=1.1 or "
+   "compatibility level 1.1 or above (use version=v3 or "
"greater)");
 ret = -EINVAL;
 goto out;
@@ -3043,144 +3012,112 @@ out:
 
 static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
 {
-BlockdevCreateOptions create_options;
-char *backing_file = NULL;
-char *backing_fmt = NULL;
-BlockdevDriver backing_drv;
-char *buf = NULL;
-uint64_t size = 0;
-int flags = 0;
-size_t cluster_size = DEFAULT_CLUSTER_SIZE;
-PreallocMode prealloc;
-int version;
-uint64_t refcount_bits;
-char *encryptfmt = NULL;
-QCryptoBlockCreateOptions *cryptoopts = NULL;
+BlockdevCreateOptions *create_options = NULL;
+QDict *qdict = NULL;
+QObject *qobj;
+Visitor *v;
 BlockDriverState *bs = NULL;
 Error *local_err = NULL;
+const char *val;
 int ret;
 
-/* Read out options */
-size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-BDRV_SECTOR_SIZE);
-backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
-backing_fmt = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FMT);
-backing_drv = qapi_enum_parse(_lookup, backing_fmt,
-  0, _err);
-if (local_err) {
-error_propagate(errp, local_err);
+/* Only the keyval visitor supports the dotted syntax needed for
+ * encryption, so go through a QDict before getting a QAPI type. Ignore
+ * options meant for the protocol layer so that the visitor doesn't
+ * complain. */
+qdict = qemu_opts_to_qdict_filtered(opts, NULL, bdrv_qcow2.create_opts,
+true);
+
+/* Handle encryption options */
+val = qdict_get_try_str(qdict, 

[Qemu-devel] [PATCH v2 18/36] rbd: Fix use after free in qemu_rbd_set_keypairs() error path

[Kevin Wolf]

If we want to include the invalid option name in the error message, we
can't free the string earlier than that.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/block/rbd.c b/block/rbd.c
index 8474b0ba11..27fa11b473 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -268,13 +268,14 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const 
char *keypairs_json,
 key = qstring_get_str(name);
 
 ret = rados_conf_set(cluster, key, qstring_get_str(value));
-QDECREF(name);
 QDECREF(value);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "invalid conf option %s", key);
+QDECREF(name);
 ret = -EINVAL;
 break;
 }
+QDECREF(name);
 }
 
 QDECREF(keypairs);
-- 
2.13.6

[Qemu-devel] [PATCH v2 23/36] rbd: Assing s->snap/image_name in qemu_rbd_open()

[Kevin Wolf]

Now that the options are already available in qemu_rbd_open() and not
only parsed in qemu_rbd_connect(), we can assign s->snap and
s->image_name there instead of passing the fields by reference to
qemu_rbd_connect().

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 14 +-
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 82f03505a9..a34bf0be46 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -577,7 +577,6 @@ out:
 }
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
-char **s_snap, char **s_image_name,
 BlockdevOptionsRbd *opts, bool cache,
 const char *keypairs, const char *secretid,
 Error **errp)
@@ -599,9 +598,6 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t 
*io_ctx,
 goto failed_opts;
 }
 
-*s_snap = g_strdup(opts->snapshot);
-*s_image_name = g_strdup(opts->image);
-
 /* try default location when conf=NULL, but ignore failure */
 r = rados_conf_read_file(*cluster, opts->conf);
 if (opts->has_conf && r < 0) {
@@ -655,8 +651,6 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t 
*io_ctx,
 
 failed_shutdown:
 rados_shutdown(*cluster);
-g_free(*s_snap);
-g_free(*s_image_name);
 failed_opts:
 g_free(mon_host);
 return r;
@@ -716,13 +710,15 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 goto out;
 }
 
-r = qemu_rbd_connect(>cluster, >io_ctx, >snap, >image_name,
- opts, !(flags & BDRV_O_NOCACHE), keypairs, secretid,
- errp);
+r = qemu_rbd_connect(>cluster, >io_ctx, opts,
+ !(flags & BDRV_O_NOCACHE), keypairs, secretid, errp);
 if (r < 0) {
 goto out;
 }
 
+s->snap = g_strdup(opts->snapshot);
+s->image_name = g_strdup(opts->image);
+
 /* rbd_open is always r/w */
 r = rbd_open(s->io_ctx, s->image_name, >image, s->snap);
 if (r < 0) {
-- 
2.13.6

[Qemu-devel] [PATCH v2 22/36] rbd: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to rbd, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json |  19 ++-
 block/rbd.c  | 146 ++-
 2 files changed, 116 insertions(+), 49 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 74021c51d7..6c0c16ebe3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3436,6 +3436,23 @@
 '*refcount-bits':   'int' } }
 
 ##
+# @BlockdevCreateOptionsRbd:
+#
+# Driver specific image creation options for rbd/Ceph.
+#
+# @location Where to store the new image file. This location cannot
+#   point to a snapshot.
+# @size Size of the virtual disk in bytes
+# @cluster-size RBD object size
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsRbd',
+  'data': { 'location': 'BlockdevOptionsRbd',
+'size': 'size',
+'*cluster-size' :   'size' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3484,7 +3501,7 @@
   'qed':'BlockdevCreateNotSupported',
   'quorum': 'BlockdevCreateNotSupported',
   'raw':'BlockdevCreateNotSupported',
-  'rbd':'BlockdevCreateNotSupported',
+  'rbd':'BlockdevCreateOptionsRbd',
   'replication':'BlockdevCreateNotSupported',
   'sheepdog':   'BlockdevCreateNotSupported',
   'ssh':'BlockdevCreateNotSupported',
diff --git a/block/rbd.c b/block/rbd.c
index 26641e53e0..82f03505a9 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -332,69 +332,55 @@ static QemuOptsList runtime_opts = {
 },
 };
 
-static int qemu_rbd_create(const char *filename, QemuOpts *opts, Error **errp)
+/* FIXME Deprecate and remove keypairs or make it available in QMP.
+ * password_secret should eventually be configurable in opts->location. Support
+ * for it in .bdrv_open will make it work here as well. */
+static int qemu_rbd_do_create(BlockdevCreateOptions *options,
+  const char *keypairs, const char 
*password_secret,
+  Error **errp)
 {
-Error *local_err = NULL;
-int64_t bytes = 0;
-int64_t objsize;
-int obj_order = 0;
-const char *pool, *image_name, *conf, *user, *keypairs;
-const char *secretid;
+BlockdevCreateOptionsRbd *opts = >u.rbd;
 rados_t cluster;
 rados_ioctx_t io_ctx;
-QDict *options = NULL;
-int ret = 0;
+int obj_order = 0;
+int ret;
 
-secretid = qemu_opt_get(opts, "password-secret");
+assert(options->driver == BLOCKDEV_DRIVER_RBD);
+if (opts->location->has_snapshot) {
+error_setg(errp, "Can't use snapshot name for image creation");
+return -EINVAL;
+}
 
-/* Read out options */
-bytes = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
- BDRV_SECTOR_SIZE);
-objsize = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE, 0);
-if (objsize) {
+/* TODO Remove the limitation */
+if (opts->location->has_server) {
+error_setg(errp, "Can't specify server for image creation");
+return -EINVAL;
+}
+
+if (opts->has_cluster_size) {
+int64_t objsize = opts->cluster_size;
 if ((objsize - 1) & objsize) {/* not a power of 2? */
 error_setg(errp, "obj size needs to be power of 2");
-ret = -EINVAL;
-goto exit;
+return -EINVAL;
 }
 if (objsize < 4096) {
 error_setg(errp, "obj size too small");
-ret = -EINVAL;
-goto exit;
+return -EINVAL;
 }
 obj_order = ctz32(objsize);
 }
 
-options = qdict_new();
-qemu_rbd_parse_filename(filename, options, _err);
-if (local_err) {
-ret = -EINVAL;
-error_propagate(errp, local_err);
-goto exit;
-}
-
-/*
- * Caution: while qdict_get_try_str() is fine, getting non-string
- * types would require more care.  When @options come from -blockdev
- * or blockdev_add, its members are typed according to the QAPI
- * schema, but when they come from -drive, they're all QString.
- */
-pool   = qdict_get_try_str(options, "pool");
-conf   = qdict_get_try_str(options, "conf");
-user   = qdict_get_try_str(options, "user");
-image_name = qdict_get_try_str(options, "image");
-keypairs   = qdict_get_try_str(options, "=keyvalue-pairs");
-
-ret = rados_create(, user);
+ret = rados_create(, opts->location->user);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "error initializing");
-goto exit;
+return ret;
 }
 
 /* try default location when conf=NULL, but ignore failure */
-ret = rados_conf_read_file(cluster, conf);
-if (conf && ret < 0) {
-

[Qemu-devel] [PATCH v2 30/36] ssh: QAPIfy host-key-check option

[Kevin Wolf]

This makes the host-key-check option available in blockdev-add.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 63 +++--
 block/ssh.c  | 88 +---
 2 files changed, 117 insertions(+), 34 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index f7679fce53..431d4a4fb2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2553,6 +2553,63 @@
 '*encrypt': 'BlockdevQcow2Encryption' } }
 
 ##
+# @SshHostKeyCheckMode:
+#
+# @none Don't check the host key at all
+# @hash Compare the host key with a given hash
+# @known_hosts  Check the host key against the known_hosts file
+#
+# Since: 2.12
+##
+{ 'enum': 'SshHostKeyCheckMode',
+  'data': [ 'none', 'hash', 'known_hosts' ] }
+
+##
+# @SshHostKeyCheckHashType:
+#
+# @md5  The given hash is an md5 hash
+# @sha1 The given hash is an sha1 hash
+#
+# Since: 2.12
+##
+{ 'enum': 'SshHostKeyCheckHashType',
+  'data': [ 'md5', 'sha1' ] }
+
+##
+# @SshHostKeyHash:
+#
+# @type The hash algorithm used for the hash
+# @hash The expected hash value
+#
+# Since: 2.12
+##
+{ 'struct': 'SshHostKeyHash',
+  'data': { 'type': 'SshHostKeyCheckHashType',
+'hash': 'str' }}
+
+##
+# @SshHostKeyDummy:
+#
+# For those union branches that don't need additional fields.
+#
+# Since: 2.12
+##
+{ 'struct': 'SshHostKeyDummy',
+  'data': {} }
+
+##
+# @SshHostKeyCheck:
+#
+# Since: 2.12
+##
+{ 'union': 'SshHostKeyCheck',
+  'base': { 'mode': 'SshHostKeyCheckMode' },
+  'discriminator': 'mode',
+  'data': { 'none': 'SshHostKeyDummy',
+'hash': 'SshHostKeyHash',
+'known_hosts': 'SshHostKeyDummy' } }
+
+##
 # @BlockdevOptionsSsh:
 #
 # @server:  host address
@@ -2562,14 +2619,16 @@
 # @user:user as which to connect, defaults to current
 #   local user name
 #
-# TODO: Expose the host_key_check option in QMP
+# @host-key-check:  Defines how and what to check the host key against
+#   (default: known_hosts)
 #
 # Since: 2.9
 ##
 { 'struct': 'BlockdevOptionsSsh',
   'data': { 'server': 'InetSocketAddress',
 'path': 'str',
-'*user': 'str' } }
+'*user': 'str',
+'*host-key-check': 'SshHostKeyCheck' } }
 
 
 ##
diff --git a/block/ssh.c b/block/ssh.c
index 9a89b7f350..dcf766c213 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -430,31 +430,35 @@ check_host_key_hash(BDRVSSHState *s, const char *hash,
 }
 
 static int check_host_key(BDRVSSHState *s, const char *host, int port,
-  const char *host_key_check, Error **errp)
+  SshHostKeyCheck *hkc, Error **errp)
 {
-/* host_key_check=no */
-if (strcmp(host_key_check, "no") == 0) {
-return 0;
-}
+SshHostKeyCheckMode mode;
 
-/* host_key_check=md5:xx:yy:zz:... */
-if (strncmp(host_key_check, "md5:", 4) == 0) {
-return check_host_key_hash(s, _key_check[4],
-   LIBSSH2_HOSTKEY_HASH_MD5, 16, errp);
-}
-
-/* host_key_check=sha1:xx:yy:zz:... */
-if (strncmp(host_key_check, "sha1:", 5) == 0) {
-return check_host_key_hash(s, _key_check[5],
-   LIBSSH2_HOSTKEY_HASH_SHA1, 20, errp);
+if (hkc) {
+mode = hkc->mode;
+} else {
+mode = SSH_HOST_KEY_CHECK_MODE_KNOWN_HOSTS;
 }
 
-/* host_key_check=yes */
-if (strcmp(host_key_check, "yes") == 0) {
+switch (mode) {
+case SSH_HOST_KEY_CHECK_MODE_NONE:
+return 0;
+case SSH_HOST_KEY_CHECK_MODE_HASH:
+if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_MD5) {
+return check_host_key_hash(s, hkc->u.hash.hash,
+   LIBSSH2_HOSTKEY_HASH_MD5, 16, errp);
+} else if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_SHA1) {
+return check_host_key_hash(s, hkc->u.hash.hash,
+   LIBSSH2_HOSTKEY_HASH_SHA1, 20, errp);
+}
+g_assert_not_reached();
+break;
+case SSH_HOST_KEY_CHECK_MODE_KNOWN_HOSTS:
 return check_host_key_knownhosts(s, host, port, errp);
+default:
+g_assert_not_reached();
 }
 
-error_setg(errp, "unknown host_key_check setting (%s)", host_key_check);
 return -EINVAL;
 }
 
@@ -543,16 +547,22 @@ static QemuOptsList ssh_runtime_opts = {
 .type = QEMU_OPT_NUMBER,
 .help = "Port to connect to",
 },
+{
+.name = "host_key_check",
+.type = QEMU_OPT_STRING,
+.help = "Defines how and what to check the host key against",
+},
 { /* end of list */ }
 },
 };
 
-static bool ssh_process_legacy_socket_options(QDict *output_opts,
-

[Qemu-devel] [PATCH v2 32/36] ssh: Support .bdrv_co_create

[Kevin Wolf]

This adds the .bdrv_co_create driver callback to ssh, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 16 -
 block/ssh.c  | 92 +---
 2 files changed, 67 insertions(+), 41 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 431d4a4fb2..2f7fab46eb 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3593,6 +3593,20 @@
 '*object-size': 'size' } }
 
 ##
+# @BlockdevCreateOptionsSsh:
+#
+# Driver specific image creation options for SSH.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsSsh',
+  'data': { 'location': 'BlockdevOptionsSsh',
+'size': 'size' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3644,7 +3658,7 @@
   'rbd':'BlockdevCreateOptionsRbd',
   'replication':'BlockdevCreateNotSupported',
   'sheepdog':   'BlockdevCreateOptionsSheepdog',
-  'ssh':'BlockdevCreateNotSupported',
+  'ssh':'BlockdevCreateOptionsSsh',
   'throttle':   'BlockdevCreateNotSupported',
   'vdi':'BlockdevCreateNotSupported',
   'vhdx':   'BlockdevCreateNotSupported',
diff --git a/block/ssh.c b/block/ssh.c
index 77bc20041f..bd3044e5f6 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -826,64 +826,75 @@ static QemuOptsList ssh_create_opts = {
 }
 };
 
-static int ssh_create(const char *filename, QemuOpts *opts, Error **errp)
+static int ssh_co_create(BlockdevCreateOptions *options, Error **errp)
 {
-int r, ret;
-int64_t total_size = 0;
-QDict *uri_options = NULL;
-BlockdevOptionsSsh *ssh_opts = NULL;
+BlockdevCreateOptionsSsh *opts = >u.ssh;
 BDRVSSHState s;
-ssize_t r2;
 char c[1] = { '\0' };
+int ret;
+
+assert(options->driver == BLOCKDEV_DRIVER_SSH);
 
 ssh_state_init();
 
+ret = connect_to_ssh(, opts->location,
+ LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
+ LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
+ 0644, errp);
+if (ret < 0) {
+goto fail;
+}
+
+if (opts->size > 0) {
+libssh2_sftp_seek64(s.sftp_handle, opts->size - 1);
+ret = libssh2_sftp_write(s.sftp_handle, c, 1);
+if (ret < 0) {
+sftp_error_setg(errp, , "truncate failed");
+ret = -EINVAL;
+goto fail;
+}
+s.attrs.filesize = opts->size;
+}
+
+ret = 0;
+fail:
+ssh_state_free();
+return ret;
+}
+
+static int ssh_create(const char *filename, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions *create_options;
+BlockdevCreateOptionsSsh *ssh_opts;
+int ret;
+QDict *uri_options = NULL;
+
+create_options = g_new0(BlockdevCreateOptions, 1);
+create_options->driver = BLOCKDEV_DRIVER_SSH;
+ssh_opts = _options->u.ssh;
+
 /* Get desired file size. */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
-DPRINTF("total_size=%" PRIi64, total_size);
+ssh_opts->size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
+DPRINTF("total_size=%" PRIi64, ssh_opts->size);
 
 uri_options = qdict_new();
-r = parse_uri(filename, uri_options, errp);
-if (r < 0) {
-ret = r;
+ret = parse_uri(filename, uri_options, errp);
+if (ret < 0) {
 goto out;
 }
 
-ssh_opts = ssh_parse_options(uri_options, errp);
-if (ssh_opts == NULL) {
+ssh_opts->location = ssh_parse_options(uri_options, errp);
+if (ssh_opts->location == NULL) {
 ret = -EINVAL;
 goto out;
 }
 
-r = connect_to_ssh(, ssh_opts,
-   LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
-   LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
-   0644, errp);
-if (r < 0) {
-ret = r;
-goto out;
-}
-
-if (total_size > 0) {
-libssh2_sftp_seek64(s.sftp_handle, total_size-1);
-r2 = libssh2_sftp_write(s.sftp_handle, c, 1);
-if (r2 < 0) {
-sftp_error_setg(errp, , "truncate failed");
-ret = -EINVAL;
-goto out;
-}
-s.attrs.filesize = total_size;
-}
-
-ret = 0;
+ret = ssh_co_create(create_options, errp);
 
  out:
-ssh_state_free();
-if (uri_options != NULL) {
-QDECREF(uri_options);
-}
-qapi_free_BlockdevOptionsSsh(ssh_opts);
+QDECREF(uri_options);
+qapi_free_BlockdevCreateOptions(create_options);
 return ret;
 }
 
@@ -1223,6 +1234,7 @@ static BlockDriver bdrv_ssh = {
 

Re: [Qemu-devel] [PATCH v2 1/5] keymap: make struct kbd_layout_t private to ui/keymaps.c

[Daniel P . Berrangé]

On Tue, Feb 20, 2018 at 04:03:53PM +0100, Gerd Hoffmann wrote:
> Also use kbd_layout_t pointers instead of void pointers.
> 
> Signed-off-by: Gerd Hoffmann 
> ---
>  ui/keymaps.h | 29 ++---
>  ui/keymaps.c | 32 +---
>  2 files changed, 31 insertions(+), 30 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v3] specs/qcow2: Fix documentation of the compressed cluster descriptor

[Eric Blake]

On 02/21/2018 08:08 AM, Alberto Garcia wrote:

This patch fixes several mistakes in the documentation of the
compressed cluster descriptor:



More things to consider, as followup patches:

Note that both the L1 table, and the standard L2 descriptors, have a cap 
on bit 55 as the maximum host offset (< 64PB).  But for compressed 
clusters, our maximum depends on cluster_bits, as follows:


512 byte cluster: bit 61 forms 'number clusters', leaving bits 60-0 for 
computing the host offset.  But even though this looks on the surface 
like you could allocate 2EB of compressed clusters, it does not play 
well with the rest of the L1/L2 system, so we should probably explicitly 
document that bits 60-56 MUST be 0, if they are assigned to the 'host 
offset field', making the maximum compressed cluster offset at 64PB.


2M cluster: bits 61-50 form 'number clusters', leaving bit 49 as the 
maximum bit in the host offset (< 512 TB).  But we never validate that 
we don't overflow this value!  I'm working on a patch.


Meanwhile, the refcount table currently allows all the way up to bit 63 
to form an offset to a refcount block, although capping that at 55 the 
way L1/L2 are capped would be reasonable (it gets weird to state that 
your metadata must live below 64PB but that your data pointed to by the 
metadata can live beyond that point).  So it may also be worth 
considering a spec patch that points out the 64PB maximum useful size, 
and maybe even a comment that the maximum size may be further 
constrained by the protocol layer (for example, ext4 has a 16TB cap on 
individual file size).


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [RFC PATCH v6 00/20] replay additions

[Ciro Santilli]

On Wed, Feb 21, 2018 at 6:41 AM, Pavel Dovgalyuk  wrote:
>> From: Ciro Santilli [mailto:ciro.santi...@gmail.com]
>> On Tue, Feb 20, 2018 at 9:46 AM, Pavel Dovgalyuk  wrote:
>> >
>> > Updated the branch on github.
>> > You may try it.
>>
>> At 8a482834780a131e7747c1c3c1931379ed0beedc ARM initrd record runs,
>> but replay is getting stuck at:
>>
>> [   12.120424] scsi host0: sym-2.2.3
>>
>> Neighboring lines on record:
>>
>> [   11.346357] sym53c8xx :00:0c.0: enabling device (0100 -> 0103)
>> [   11.536683] sym0: <895a> rev 0x0 at pci :00:0c.0 irq 66
>> [   11.731679] sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
>> [   11.930599] sym0: SCSI BUS has been reset.
>> [   12.120424] scsi host0: sym-2.2.3
>> [   15.451809] scsi 0:0:2:0: CD-ROMQEMU QEMU CD-ROM
>>   2.5+ PQ: 0 ANSI: 5
>> [   15.847227] scsi target0:0:2: tagged command queuing enabled,
>> command queue depth 16.
>> [   16.256585] scsi target0:0:2: Beginning Domain Validation
>> [   16.482189] scsi target0:0:2: Domain Validation skipping write tests
>> [   16.699445] scsi target0:0:2: Ending Domain Validation
>>
>> My QEMU command:
>>
>> time ./buildroot/output.arm~/host/usr/bin/qemu-system-arm -M
>> versatilepb -append 'root=/dev/sda nokaslr norandmaps
>> printk.devkmsg=on printk.time=y - lkmc_eval="/rand_check.out;wget -S
>> google.com;/poweroff.out;"'
>>  -kernel ./buildroot/output.arm~/images/zImage -dtb
>> ./buildroot/output.arm~/images/versatile-pb.dtb -nographic -initrd
>> ./buildroot/output.arm~/images/rootfs.cpio -netdev user,id=net1
>> -device rtl8139,netdev=net1
>> -object filter-replay,id=replay,netdev=net1
>>
>> What is your full QEMU command?
>
> I used your previous command and encountered kernel panic in guest.
> What is rootfs.cpio file? Is it the renamed rootfs.ext2 from your images.zip?
>

rootfs.cpio is the cpio version of rootfs.ext2, both should represent
the same filesystem in different serialized formats.

I've added it to this updated zip:
https://github.com/cirosantilli/linux-kernel-module-cheat/releases/download/test-replay-arm/images2.zip

The previous .zip didn't have the .cpio, then I later made Buildroot
generate it as well to try out the -initrd method.

I'm using this branch for my testing:
https://github.com/cirosantilli/linux-kernel-module-cheat/tree/rr with
"./build -a arm && ./recarm"

>> Also I think the patch to fix qmeu-img was not included in the branch.
>
> Forgot about it. Now it should be ok.
>
>
> Pavel Dovgalyuk
>

[Qemu-devel] [PATCH v2 1/3] qcow2: Prefer byte-based calls into bs->file

[Eric Blake]

We had only three sector-based stragglers left; convert them to use
our preferred byte-based accesses.

Signed-off-by: Eric Blake 
Reviewed-by: Alberto Garcia 

---
v2: indentation fix
---
 block/qcow2-cluster.c  | 5 ++---
 block/qcow2-refcount.c | 6 +++---
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index e406b0f3b9e..85be7d5e340 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1615,13 +1615,12 @@ int qcow2_decompress_cluster(BlockDriverState *bs, 
uint64_t cluster_offset)
 }

 BLKDBG_EVENT(bs->file, BLKDBG_READ_COMPRESSED);
-ret = bdrv_read(bs->file, coffset >> 9, s->cluster_data,
-nb_csectors);
+ret = bdrv_pread(bs->file, coffset, s->cluster_data, csize);
 if (ret < 0) {
 return ret;
 }
 if (decompress_buffer(s->cluster_cache, s->cluster_size,
-  s->cluster_data + sector_offset, csize) < 0) {
+  s->cluster_data, csize) < 0) {
 return -EIO;
 }
 s->cluster_cache_offset = coffset;
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index d46b69d7f34..28afbb1b5ea 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -2310,8 +2310,8 @@ write_refblocks:
 on_disk_refblock = (void *)((char *) *refcount_table +
 refblock_index * s->cluster_size);

-ret = bdrv_write(bs->file, refblock_offset / BDRV_SECTOR_SIZE,
- on_disk_refblock, s->cluster_sectors);
+ret = bdrv_pwrite(bs->file, refblock_offset, on_disk_refblock,
+  s->cluster_size);
 if (ret < 0) {
 fprintf(stderr, "ERROR writing refblock: %s\n", strerror(-ret));
 goto fail;
@@ -2533,7 +2533,7 @@ fail:
  * - 0 if writing to this offset will not affect the mentioned metadata
  * - a positive QCow2MetadataOverlap value indicating one overlapping section
  * - a negative value (-errno) indicating an error while performing a check,
- *   e.g. when bdrv_read failed on QCOW2_OL_INACTIVE_L2
+ *   e.g. when bdrv_pread failed on QCOW2_OL_INACTIVE_L2
  */
 int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
  int64_t size)
-- 
2.14.3

Re: [Qemu-devel] [PATCHv3 1/2] translator: merge max_insns into DisasContextBase

[Emilio G. Cota]

On Wed, Feb 21, 2018 at 13:05:45 -0800, Richard Henderson wrote:
> On 02/21/2018 12:55 PM, Emilio G. Cota wrote:
> > While at it, use int for both num_insns and max_insns to make
> > sure we have same-type comparisons.
> > 
> > Signed-off-by: Emilio G. Cota 
> > ---
> >  accel/tcg/translator.c | 21 ++---
> >  include/exec/translator.h  |  8 
> >  target/alpha/translate.c   |  6 ++
> >  target/arm/translate-a64.c |  8 +++-
> >  target/arm/translate.c |  9 +++--
> >  target/hppa/translate.c|  7 ++-
> >  target/i386/translate.c|  5 +
> >  target/ppc/translate.c |  5 ++---
> >  8 files changed, 27 insertions(+), 42 deletions(-)
> 
> Reviewed-by: Richard Henderson 

Thanks.

To avoid merge conflicts, I'll send v2's of the other conversions
once this patch goes in.

Emilio

[Qemu-devel] [PATCH v2 3/3] qcow2: Avoid memory over-allocation on compressed images

[Eric Blake]

When reading a compressed image, we were allocating s->cluster_data
to 32*cluster_size + 512 (possibly over 64 megabytes, for an image
with 2M clusters).  Let's check out the history:

Back when qcow2 was first written, we used s->cluster_data for
everything, including copy_sectors() and encryption, where we want
to operate on more than one cluster at once.  Obviously, at that
point, the buffer had to be aligned for other users, even though
compression itself doesn't require any alignment (the fact that
the compressed data generally starts mid-sector means that aligning
our buffer buys us nothing - either the protocol already supports
byte-based access into whatever offset we want, or we are already
using a bounce buffer to read a full sector, and copying into
our destination no longer requires alignment).

But commit 1b9f1491 (v1.1!) changed things to allocate parallel
buffers on demand rather than sharing a single buffer, for encryption
and COW, leaving compression as the final client of s->cluster_data.
That use was still preserved, because if a single compressed cluster
is read more than once, we reuse the cache instead of decompressing
it a second time (someday, we may come up with better caching to
avoid wasting repeated decompressions while still being more parallel,
but that is a task for another patch; the XXX comment in
qcow2_co_preadv for QCOW2_CLUSTER_COMPRESSED is telling).

Much later, in commit de82815d (v2.2), we noticed that a 64M
allocation is prone to failure, so we switched over to a graceful
memory allocation error message.  Elsewhere in the code, we do
g_malloc(2 * cluster_size) without ever checking for failure, but
even 4M starts to be large enough that trying to be nice is worth
the effort, so we want to keep that aspect.

Then even later, in 3e4c7052 (2.11), we realized that allocating
a large buffer up front for every qcow2 image is expensive, and
switched to lazy allocation only for images that actually had
compressed clusters.  But in the process, we never even bothered
to check whether what we were allocating still made sense in its
new context!

So, it's time to cut back on the waste.  A compressed cluster
written by qemu will NEVER occupy more than an uncompressed
cluster, but based on mid-sector alignment, we may still need
to read 1 cluster + 1 sector in order to recover enough bytes
for the decompression.  But third-party producers of qcow2 may
not be as smart, and gzip DOES document that because the
compression stream adds metadata, and because of the pigeonhole
principle, there are worst case scenarios where attempts to
compress will actually inflate an image, by up to 0.015% (or 62
sectors larger for an unfortunate 2M compression).  In fact,
the qcow2 spec permits up to 2 full clusters of sectors beyond
the initial offset; and the way decompression works, it really
doesn't matter if we read too much (gzip ignores slop, once it
has decoded a full cluster), so it's feasible to encounter a
third-party image that reports the maximum 'nb_csectors'
possible, even if it no longer has any bearing to the actual
compressed size.  So it's easier to just allocate cluster_data
to be as large as we can ever possibly see; even if it still
wastes up to 2M on any image created by qcow2, that's still an
improvment of 60M less waste than pre-patch.

Signed-off-by: Eric Blake 

---
v2: actually check allocation failure (previous version meant
to use g_malloc, but ended up posted with g_try_malloc without
checking); add assertions outside of conditional, improve
commit message to better match reality now that qcow2 spec bug
has been fixed
---
 block/qcow2-cluster.c | 27 ++-
 block/qcow2.c |  2 +-
 2 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 85be7d5e340..7d5276b5f6b 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1598,20 +1598,29 @@ int qcow2_decompress_cluster(BlockDriverState *bs, 
uint64_t cluster_offset)
 sector_offset = coffset & 511;
 csize = nb_csectors * 512 - sector_offset;

-/* Allocate buffers on first decompress operation, most images are
- * uncompressed and the memory overhead can be avoided.  The buffers
- * are freed in .bdrv_close().
+/* Allocate buffers on the first decompress operation; most
+ * images are uncompressed and the memory overhead can be
+ * avoided.  The buffers are freed in .bdrv_close().  qemu
+ * never writes an inflated cluster, and gzip itself never
+ * inflates a problematic cluster by more than 0.015%, but the
+ * qcow2 format allows up to 2 full clusters beyond the sector
+ * containing offset, and gzip ignores trailing slop, so it's
+ * easier to just allocate that much up front than to reject
+ * third-party images with overlarge csize.
  */
+assert(!!s->cluster_data == 

[Qemu-devel] [PATCH v2 0/3] qcow2: minor compression improvements

[Eric Blake]

Updates to v1:
- fix whitespace [Berto]
- fix g_try_malloc usage [Berto, Kevin]
- improve comments [Berto, Kevin]
- add a patch to avoid overflow on 512TB images with 2M clusters

Eric Blake (3):
  qcow2: Prefer byte-based calls into bs->file
  qcow2: Don't allow overflow during cluster allocation
  qcow2: Avoid memory over-allocation on compressed images

 block/qcow2.h  |  6 ++
 block/qcow2-cluster.c  | 32 
 block/qcow2-refcount.c | 26 --
 block/qcow2.c  |  2 +-
 4 files changed, 43 insertions(+), 23 deletions(-)

-- 
2.14.3

Re: [Qemu-devel] [PATCH v4 00/22] re-factor softfloat and add fp16 functions

[Fam Zheng]

On Mon, 02/19 13:56, Peter Maydell wrote:
> On 17 February 2018 at 13:23, Alex Bennée  wrote:
> > Peter Maydell  writes:
> >> If you persuade git to use the --minimal, --patience or --histogram
> >> git diff option when generating these patches you'll find that it
> >> doesn't produce unreadable patches that provoke all the checkpatch
> >> warnings.
> >
> > I think this is patchew getting confused
> 
> Oh yes, sorry. Fam, can we update patchew's git config to use
>   git config --global diff.algorithm histogram
> (equivalently, algorithm = histogram in the .gitconfig) -- that
> way the patches it runs checkpatch on are more likely to be
> formatted so that they minimise spurious checkpatch output, I think.

Updated. (With s/--global/--local/ along with other diff options, in the testing
command list:

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

)

Fam

Re: [Qemu-devel] [PATCH] hw/acpi-build: build SRAT memory affinity structures for NVDIMM

[Haozhong Zhang]

On 02/21/18 14:55 +0100, Igor Mammedov wrote:
> On Tue, 20 Feb 2018 17:17:58 -0800
> Dan Williams  wrote:
> 
> > On Tue, Feb 20, 2018 at 6:10 AM, Igor Mammedov  wrote:
> > > On Sat, 17 Feb 2018 14:31:35 +0800
> > > Haozhong Zhang  wrote:
> > >  
> > >> ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity
> > >> domain of a NVDIMM SPA range must match with corresponding entry in
> > >> SRAT table.
> > >>
> > >> The address ranges of vNVDIMM in QEMU are allocated from the
> > >> hot-pluggable address space, which is entirely covered by one SRAT
> > >> memory affinity structure. However, users can set the vNVDIMM
> > >> proximity domain in NFIT SPA range structure by the 'node' property of
> > >> '-device nvdimm' to a value different than the one in the above SRAT
> > >> memory affinity structure.
> > >>
> > >> In order to solve such proximity domain mismatch, this patch build one
> > >> SRAT memory affinity structure for each NVDIMM device with the
> > >> proximity domain used in NFIT. The remaining hot-pluggable address
> > >> space is covered by one or multiple SRAT memory affinity structures
> > >> with the proximity domain of the last node as before.
> > >>
> > >> Signed-off-by: Haozhong Zhang   
> > > If we consider hotpluggable system, correctly implemented OS should
> > > be able pull proximity from Device::_PXM and override any value from SRAT.
> > > Do we really have a problem here (anything that breaks if we would use 
> > > _PXM)?
> > > Maybe we should add _PXM object to nvdimm device nodes instead of 
> > > massaging SRAT?  
> > 
> > Unfortunately _PXM is an awkward fit. Currently the proximity domain
> > is attached to the SPA range structure. The SPA range may be
> > associated with multiple DIMM devices and those individual NVDIMMs may
> > have conflicting _PXM properties.
> There shouldn't be any conflict here as  NVDIMM device's _PXM method,
> should override in runtime any proximity specified by parent scope.
> (as parent scope I'd also count boot time NFIT/SRAT tables).
> 
> To make it more clear we could clear valid proximity domain flag in SPA
> like this:
> 
> diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
> index 59d6e42..131bca5 100644
> --- a/hw/acpi/nvdimm.c
> +++ b/hw/acpi/nvdimm.c
> @@ -260,9 +260,7 @@ nvdimm_build_structure_spa(GArray *structures, 
> DeviceState *dev)
>   */
>  nfit_spa->flags = cpu_to_le16(1 /* Control region is strictly for
> management during hot add/online
> -   operation */ |
> -  2 /* Data in Proximity Domain field is
> -   valid*/);
> +   operation */);
>  
>  /* NUMA node. */
>  nfit_spa->proximity_domain = cpu_to_le32(node);
> 
> > Even if that was unified across
> > DIMMs it is ambiguous whether a DIMM-device _PXM would relate to the
> > device's control interface, or the assembled persistent memory SPA
> > range.
> I'm not sure what you mean under 'device's control interface',
> could you clarify where the ambiguity comes from?
> 
> I read spec as: _PXM applies to address range covered by NVDIMM
> device it belongs to.
> 
> As for assembled SPA, I'd assume that it applies to interleaved set
> and all NVDIMMs with it should be on the same node. It's somewhat
> irrelevant question though as QEMU so far implements only
>   1:1:1/SPA:Region Mapping:NVDIMM Device/
> mapping.
> 
> My main concern with using static configuration tables for proximity
> mapping, we'd miss on hotplug side of equation. However if we start
> from dynamic side first, we could later complement it with static
> tables if there really were need for it.

This patch affects only the static tables and static-plugged NVDIMM.
For hot-plugged NVDIMMs, guest OSPM still needs to evaluate _FIT to
get the information of the new NVDIMMs including their proximity
domains.

One intention of this patch is to simulate the bare metal as much as
possible. I have been using this patch to develop and test NVDIMM
enabling work on Xen, and think it might be useful for developers of
other OS and hypervisors.


Haozhong

Re: [Qemu-devel] [PATCH 10/11] macio: move setting of CUDA timebase frequency to macio_common_realize()

[David Gibson]

On Mon, Feb 19, 2018 at 06:19:21PM +, Mark Cave-Ayland wrote:
> This removes the last of the functionality from macio_init() in preparation
> for its subsequent removal.
> 
> Signed-off-by: Mark Cave-Ayland 

Reviewed-by: David Gibson 

> ---
>  hw/misc/macio/macio.c | 7 ++-
>  1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/misc/macio/macio.c b/hw/misc/macio/macio.c
> index e5288f1084..f71ed61819 100644
> --- a/hw/misc/macio/macio.c
> +++ b/hw/misc/macio/macio.c
> @@ -101,6 +101,8 @@ static void macio_common_realize(PCIDevice *d, Error 
> **errp)
>  memory_region_add_subregion(>bar, 0x08000,
>  sysbus_mmio_get_region(sysbus_dev, 0));
>  
> +qdev_prop_set_uint64(DEVICE(>cuda), "timebase-frequency",
> + s->frequency);
>  object_property_set_bool(OBJECT(>cuda), true, "realized", );
>  if (err) {
>  error_propagate(errp, err);
> @@ -444,12 +446,7 @@ type_init(macio_register_types)
>  void macio_init(PCIDevice *d,
>  MemoryRegion *pic_mem)
>  {
> -MacIOState *macio_state = MACIO(d);
> -
>  /* Note: this code is strongly inspirated from the corresponding code
> in PearPC */
> -qdev_prop_set_uint64(DEVICE(_state->cuda), "timebase-frequency",
> - macio_state->frequency);
> -
>  qdev_init_nofail(DEVICE(d));
>  }

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

[Qemu-devel] [PATCH V4 0/3] tests: Add migration test for aarch64

[Wei Huang]

This patchset adds a migration test for aarch64. It leverages
Dave Gilbert's recent patch "tests/migration: Add source to PC boot block"
to create a new test case for aarch64.

V3->V4:
 * Rename .s to .S, allowing assembly to include C-style header file
 * Move test defines into a new migration-test.h file
 * Use different cpu & gic settings for kvm and tcg modes on aarch64
 * Clean up aarch64-a-b-kernel.S based on Andrew Jones' comments
 
V2->V3:
 * Convert build script to Makefile
 * Add cross-compilation support
 * Fix CPU type for "tcg" machine type
 * Revise asm code and the compilation process from asm to header file

V1->V2:
 * Similar to Dave Gilbert's recent changes to migration-test, we
   provide the test source and a build script in V2.
 * aarch64 kernel blob is defined as "unsigned char" because the source
   is now provided in V2.
 * Add "-machine none" to test_deprecated() because aarch64 doesn't have
   a default machine type.

RFC->V1:
 * aarch64 kernel blob is defined as an uint32_t array
 * The test code is re-written to address a data caching issue under KVM.
   Tests passed under both x86 and aarch64.
 * Re-use init_bootfile_x86() for both x86 and aarch64
 * Other minor fixes

Thanks,
-Wei

Wei Huang (3):
  tests/migration: Convert the boot block compilation script into
Makefile
  tests/migration: Add migration-test header file
  tests: Add migration test for aarch64

 tests/Makefile.include |  1 +
 tests/migration-test.c | 52 +---
 tests/migration/Makefile   | 46 ++
 tests/migration/aarch64-a-b-kernel.S   | 71 ++
 tests/migration/aarch64-a-b-kernel.h   | 19 ++
 tests/migration/migration-test.h   | 24 
 tests/migration/rebuild-x86-bootblock.sh   | 33 --
 .../{x86-a-b-bootblock.s => x86-a-b-bootblock.S}   | 12 ++--
 tests/migration/x86-a-b-bootblock.h|  4 +-
 9 files changed, 213 insertions(+), 49 deletions(-)
 create mode 100644 tests/migration/Makefile
 create mode 100644 tests/migration/aarch64-a-b-kernel.S
 create mode 100644 tests/migration/aarch64-a-b-kernel.h
 create mode 100644 tests/migration/migration-test.h
 delete mode 100755 tests/migration/rebuild-x86-bootblock.sh
 rename tests/migration/{x86-a-b-bootblock.s => x86-a-b-bootblock.S} (88%)

-- 
2.14.3

Re: [Qemu-devel] [qemu-s390x] [PATCH v8 11/13] s390-ccw: set cp_receive mask only when needed and consume pending service irqs

[Thomas Huth]

On 21.02.2018 20:35, Collin L. Walling wrote:
> It is possible while waiting for multiple types of external
> interrupts that we might have pending irqs remaining between
> irq consumption and irq-type disabling. Those interrupts
> could potentially propagate to the guest after IPL completes
> and cause unwanted behavior.
> 
> As it is today, the SCLP will only recognize write events that
> are enabled by the control program's send and receive masks. To
> limit the window for, and prevent further irqs from, ASCII
> console events (specifically keystrokes), we should only enable
> the control program's receive mask when we need it.
> 
> While we're at it, remove assignment of the (non control program)
> send and receive masks, as those are actually set by the SCLP.
> 
> Signed-off-by: Collin L. Walling 
> ---
>  pc-bios/s390-ccw/menu.c |  5 +
>  pc-bios/s390-ccw/s390-ccw.h |  1 +
>  pc-bios/s390-ccw/sclp.c | 10 --
>  3 files changed, 10 insertions(+), 6 deletions(-)

Reviewed-by: Thomas Huth 

Re: [Qemu-devel] [qemu-s390x] [PATCH v8 12/13] s390-ccw: use zipl values when no boot menu options are present

[Thomas Huth]

On 21.02.2018 20:35, Collin L. Walling wrote:
> If no boot menu options are present, then flag the boot menu to
> use the zipl options that were set in the zipl configuration file
> (and stored on disk by zipl). These options are found at some
> offset prior to the start of the zipl boot menu banner. The zipl
> timeout value is limited to a 16-bit unsigned integer and stored
> as seconds, so we take care to convert it to milliseconds in order
> to conform to the rest of the boot menu functionality. This is
> limited to CCW devices.
> 
> For reference, the zipl configuration file uses the following
> fields in the menu section:
> 
>   prompt=1  enable the boot menu
>   timeout=X set the timeout to X seconds
> 
> To explicitly disregard any boot menu options, then menu=off or
>  must be specified.
> 
> Signed-off-by: Collin L. Walling 
> ---
>  hw/s390x/ipl.c  |  5 +
>  hw/s390x/ipl.h  |  1 +
>  pc-bios/s390-ccw/iplb.h |  1 +
>  pc-bios/s390-ccw/main.c |  3 ++-
>  pc-bios/s390-ccw/menu.c | 14 ++
>  5 files changed, 23 insertions(+), 1 deletion(-)

Reviewed-by: Thomas Huth 

Re: [Qemu-devel] [Qemu-ppc] [PATCH 0/2] Firmware blob and git submodule for Sam460ex

[Thomas Huth]

On 21.02.2018 19:33, Peter Maydell wrote:
> On 21 February 2018 at 17:06, BALATON Zoltan  wrote:
>> It's not that upstream u-boot has abandoned board support (it only removed
>> support for the PPC440 CPU it once had). The board itself never had support
>> in upstream u-boot, it only exists in vendor's fork which is the reason we
>> need a separate source and cannot use upstream u-boot source we already
>> have.
>>
>> In my opinion we don't aim to take on support for this board in u-boot, we
>> only need to include the firmware binary for the emulation to be useful
>> which then requires us to also include the source for the GPL it's licensed
>> under. I've also found a few bugs in the firmware which I've fixed but apart
>> from such occasional bug fixes when needed I don't expect to take over
>> support for the board from the hardware vendor so this source is only so we
>> can include the firmware binary which is needed for the board emulation.
>> Does this answer your concerns?
> 
> We have lots of boards we don't ship firmware blobs for and
> which we expect the users to provide the guest code for
> if they're going to use them.

... which is also somewhat unfortunate. Have you ever tried to run one
of those boards to see whether you've broken something with your code
changes or not? Hunting the firmware for such a board can be quite
challenging. I'm not saying that we should now try to include way more
firmware blobs in our repository (its size would explode, I guess), but
maybe we should at least start a Wiki page with links to the various
firmware images or so?

 Thomas

Re: [Qemu-devel] [PATCH v6 27/28] migration/qmp: add command migrate-pause

[Peter Xu]

On Wed, Feb 14, 2018 at 06:56:59PM +, Dr. David Alan Gilbert wrote:
> * Peter Xu (pet...@redhat.com) wrote:
> > On Tue, Feb 13, 2018 at 08:11:00PM +, Dr. David Alan Gilbert wrote:
> > > * Peter Xu (pet...@redhat.com) wrote:
> > > > It pauses an ongoing migration.  Currently it only supports postcopy.
> > > > Note that this command will work on either side of the migration.
> > > > Basically when we trigger this on one side, it'll interrupt the other
> > > > side as well since the other side will get notified on the disconnect
> > > > event.
> > > > 
> > > > However, it's still possible that the other side is not notified, for
> > > > example, when the network is totally broken, or due to some firewall
> > > > configuration changes.  In that case, we will also need to run the same
> > > > command on the other side so both sides will go into the paused state.
> > > > 
> > > > Signed-off-by: Peter Xu 
> > > > ---
> > > >  migration/migration.c | 27 +++
> > > >  qapi/migration.json   | 16 
> > > >  2 files changed, 43 insertions(+)
> > > > 
> > > > diff --git a/migration/migration.c b/migration/migration.c
> > > > index bb57ed9ade..139abec0c3 100644
> > > > --- a/migration/migration.c
> > > > +++ b/migration/migration.c
> > > > @@ -1448,6 +1448,33 @@ void qmp_migrate_recover(const char *uri, Error 
> > > > **errp)
> > > >  qemu_start_incoming_migration(uri, errp);
> > > >  }
> > > >  
> > > > +void qmp_migrate_pause(Error **errp)
> > > > +{
> > > > +MigrationState *ms = migrate_get_current();
> > > > +MigrationIncomingState *mis = migration_incoming_get_current();
> > > > +int ret;
> > > > +
> > > > +if (ms->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
> > > > +/* Source side, during postcopy */
> > > > +ret = qemu_file_shutdown(ms->to_dst_file);
> > > 
> > > This doesn't feel thread safe; although I'm not sure how to make it so.
> > > If the migration finishes just after we check the state but before the
> > > shutdown we end up using a bogus QEMUFile*
> > > Making all the places that close a QEMUFile* set hte pointer Null before
> > > they do the close doesn't help because you still race with that.
> > > 
> > > (The race is small, but still)
> > 
> > IMHO we can fix it by adding a migration lock for management code. If
> > you see my previous migrate cleanup series, it's in my todo. ;)
> > 
> > The basic idea is that we take the lock for critical paths (but not
> > during most of the migration process).  E.g., we may need the lock
> > for:
> > 
> > - very beginning of migration, during setup
> > - reaching the end of migration
> > - every single migration QMP command (since HMP calls them so HMP will
> >   also acquire the lock)
> > - anywhere else I didn't mention that may necessary, e.g., when we
> >   change migrate state, meanwhile we do something else - basically
> >   that should be an "atomic operation", and we need the lock to make
> >   sure of that.
> 
> But then we couldn't take that lock in an OOB command, you'd have to
> audit all of those places that took it to make sure it didn't do any of
> the things OOB commands aren't allowed to do.

Yeah OOB commands will be special - my plan is that they just never
take the lock.  E.g., they only touches FDs, and FDs are naturally
thread safe (like this command).

And some major migration commands (like "migrate" itself) should never
be an OOB command.

> 
> > For the recovery series, I would prefer that we ignore this issue for
> > now - since this problem is there for quite a long time AFAICT in the
> > whole migration code rather than this series only, and we need to
> > solve it once and for all.
> 
> I don't think those problems happen (much) in the existing code, because
> everything is done in the main thread.

But migration is running in its own thread (migration_thread)?

For example: What if we send migration commands during the end of
migration or a failing migration?  Could there be risk in old code
too since both main thread and migration thread may be manipulating
MigrationState object?

> That's one reason why the to_dst_file is closed in migrate_fd_cleanup
> which is normally closed in the back-half run on the main thread.
> 
> One way would be to make the state go to POSTCOPY_PAUSED here;
> note that migrate_set_state already uses an atomic_cmpxchg to do the
> update.

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH v6 21/28] migration: setup ramstate for resume

[Peter Xu]

On Wed, Feb 14, 2018 at 06:40:46PM +, Dr. David Alan Gilbert wrote:
> * Peter Xu (pet...@redhat.com) wrote:
> > On Tue, Feb 13, 2018 at 06:17:51PM +, Dr. David Alan Gilbert wrote:
> > > * Peter Xu (pet...@redhat.com) wrote:
> > > > After we updated the dirty bitmaps of ramblocks, we also need to update
> > > > the critical fields in RAMState to make sure it is ready for a resume.
> > > > 
> > > > Signed-off-by: Peter Xu 
> > > > ---
> > > >  migration/ram.c| 40 +++-
> > > >  migration/trace-events |  1 +
> > > >  2 files changed, 40 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/migration/ram.c b/migration/ram.c
> > > > index a2a4b05d5c..d275875f54 100644
> > > > --- a/migration/ram.c
> > > > +++ b/migration/ram.c
> > > > @@ -2250,6 +2250,36 @@ static int ram_init_all(RAMState **rsp)
> > > >  return 0;
> > > >  }
> > > >  
> > > > +static void ram_state_resume_prepare(RAMState *rs, QEMUFile *out)
> > > > +{
> > > > +RAMBlock *block;
> > > > +long pages = 0;
> > > > +
> > > > +/*
> > > > + * Postcopy is not using xbzrle/compression, so no need for that.
> > > > + * Also, since source are already halted, we don't need to care
> > > > + * about dirty page logging as well.
> > > > + */
> > > > +
> > > > +RAMBLOCK_FOREACH(block) {
> > > > +pages += bitmap_count_one(block->bmap,
> > > > +  block->used_length >> 
> > > > TARGET_PAGE_BITS);
> > > > +}
> > > > +
> > > > +/* This may not be aligned with current bitmaps. Recalculate. */
> > > > +rs->migration_dirty_pages = pages;
> > > 
> > > migration_dirty_pages is uint64_t - so we should probably do the cast
> > > above and keep 'pages' as uint64_t.
> > 
> > Sure.
> > 
> > > 
> > > > +rs->last_seen_block = NULL;
> > > > +rs->last_sent_block = NULL;
> > > > +rs->last_page = 0;
> > > > +rs->last_version = ram_list.version;
> > > 
> > > Do you need to explicitly set
> > >rs->ram_bulk_stage = false;
> > > 
> > > if the failure happened just after the start of postcopy and no
> > > requested pages had been sent, I think it might still  be set?
> > 
> > Could you elaborate what would go wrong even if it's still set?
> 
> I think it might start sending all pages rather than just those
> that are dirty/needed;  see migration_bitmap_find_dirty.

Ah yes.  I should turn it off.

-- 
Peter Xu

Re: [Qemu-devel] [Resend][PATCH] qga: unset frozen state if no mount points are frozen

[Chen Hanxiao]

At 2018-02-16 02:41:25, "Michael Roth"  wrote:
>Quoting Chen Hanxiao (2018-02-08 19:35:42)
>> From: Chen Hanxiao 
>> 
>> If we set mountpoints to qmp_guest_fsfreeze_freeze_list,
>> we may got nothing to freeze as all mountpoints are
>> not valid.
>> Call ga_unset_frozen in this senario.
>> 
>> Cc: Michael Roth 
>> Signed-off-by: Chen Hanxiao 
>> ---
>> Rebase on master
>> 
>>  qga/commands-posix.c | 6 ++
>>  1 file changed, 6 insertions(+)
>> 
>> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
>> index e809e382eb..9fd51f1d7a 100644
>> --- a/qga/commands-posix.c
>> +++ b/qga/commands-posix.c
>> @@ -1273,6 +1273,12 @@ int64_t qmp_guest_fsfreeze_freeze_list(bool 
>> has_mountpoints,
>>  }
>> 
>>  free_fs_mount_list();
>> +/* We may not issue any FIFREEZE here when had mountpoints.
>> + * Just unset ga_state here and ready for the next call.
>> + */
>> +if (has_mountpoints && i == 0) {
>> +ga_unset_frozen(ga_state);
>> +}
>
>It seems odd to special-case has_mountpoints. Wouldn't:
>
>  if (i == 0) {
>...
>  }
>
>be more consistent? Then management could infer i==0 leaves qga in
>unfrozen state. Otherwise I'd rather just stick with expecting a
>gratuitous unfreeze() since it requires less special-casing on the
>management/ side.
>
>And if we do change this, we'd probably want to update
>qga/qapi-schema.json to reflect the behavior. I.e.:
>
>##
># @guest-fsfreeze-freeze:
>#
># Sync and freeze all freezable, local guest filesystems. If this
># command succeeded, you may call @guest-fsfreeze-thaw later to
># unfreeze.
>...
># Returns: Number of file systems currently frozen. On error, all filesystems
>-# will be thawed.
>+# will be thawed. If no filesystems are frozen as a result of this call,
>+# then @guest-fsfreeze-status will remain "thawed" and calling
>+# @guest-fsfreeze-thaw is not necessary.
>

Thanks for the review.
Will be fixed in v2.

Regards,
- Chen

Re: [Qemu-devel] [qemu-s390x] [PATCH v8 04/13] s390-ccw: update libc

[Thomas Huth]

On 21.02.2018 20:35, Collin L. Walling wrote:
> Moved:
>   memcmp from bootmap.h to libc.h (renamed from _memcmp)
>   strlen from sclp.c to libc.h (renamed from _strlen)
> 
> Added C standard functions:
>   isdigit
> 
> Added non C-standard function:
>   uitoa
>   atoui
> 
> Signed-off-by: Collin L. Walling 
> Acked-by: Christian Borntraeger 
> Reviewed-by: Janosch Frank 
> ---
>  pc-bios/s390-ccw/Makefile  |  2 +-
>  pc-bios/s390-ccw/bootmap.c |  4 +--
>  pc-bios/s390-ccw/bootmap.h | 16 +
>  pc-bios/s390-ccw/libc.c| 88 
> ++
>  pc-bios/s390-ccw/libc.h| 37 +--
>  pc-bios/s390-ccw/main.c| 17 +
>  pc-bios/s390-ccw/sclp.c| 10 +-
>  7 files changed, 129 insertions(+), 45 deletions(-)
>  create mode 100644 pc-bios/s390-ccw/libc.c

Reviewed-by: Thomas Huth 

[Qemu-devel] [PATCH qemu repost] qmp: Add qom-list-properties to list QOM object properties

[Alexey Kardashevskiy]

There is already 'device-list-properties' which does most of the job,
however it does not handle everything returned by qom-list-types such
as machines as they inherit directly from TYPE_OBJECT and not TYPE_DEVICE.

This adds a new qom-list-properties command which prints properties
of a specific class and its instance. It is pretty much a simplified copy
of the device-list-properties handler.

Since it creates an object instance, device properties should appear
in the output as they are copied to QOM properties at the instance_init
hook.

Signed-off-by: Alexey Kardashevskiy 
---

This is a simple rebase on top of the current upstream.


---
 qapi-schema.json | 29 +
 qmp.c| 52 
 2 files changed, 81 insertions(+)

diff --git a/qapi-schema.json b/qapi-schema.json
index 0262b9f..fa5f189 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1455,6 +1455,35 @@
   'returns': [ 'DevicePropertyInfo' ] }
 
 ##
+# @QOMPropertyInfo:
+#
+# Information about object properties.
+#
+# @name: the name of the property
+# @type: the typename of the property
+# @description: if specified, the description of the property.
+#
+# Since: 2.12
+##
+{ 'struct': 'QOMPropertyInfo',
+  'data': { 'name': 'str', 'type': 'str', '*description': 'str' } }
+
+##
+# @qom-list-properties:
+#
+# List properties associated with a QOM object.
+#
+# @typename: the type name of an object
+#
+# Returns: a list of QOMPropertyInfo describing object properties
+#
+# Since: 2.12
+##
+{ 'command': 'qom-list-properties',
+  'data': { 'typename': 'str'},
+  'returns': [ 'QOMPropertyInfo' ] }
+
+##
 # @xen-set-global-dirty-log:
 #
 # Enable or disable the global dirty log mode.
diff --git a/qmp.c b/qmp.c
index 793f6f3..f2d4781 100644
--- a/qmp.c
+++ b/qmp.c
@@ -576,6 +576,58 @@ DevicePropertyInfoList *qmp_device_list_properties(const 
char *typename,
 return prop_list;
 }
 
+QOMPropertyInfoList *qmp_qom_list_properties(const char *typename,
+ Error **errp)
+{
+ObjectClass *klass;
+Object *obj;
+ObjectProperty *prop;
+ObjectPropertyIterator iter;
+QOMPropertyInfoList *prop_list = NULL;
+
+klass = object_class_by_name(typename);
+if (klass == NULL) {
+error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+  "Class '%s' not found", typename);
+return NULL;
+}
+
+klass = object_class_dynamic_cast(klass, TYPE_OBJECT);
+if (klass == NULL) {
+error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "typename", 
TYPE_OBJECT);
+return NULL;
+}
+
+if (object_class_is_abstract(klass)) {
+error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "typename",
+   "non-abstract class");
+return NULL;
+}
+
+obj = object_new(typename);
+
+object_property_iter_init(, obj);
+while ((prop = object_property_iter_next())) {
+QOMPropertyInfo *info;
+QOMPropertyInfoList *entry;
+
+info = g_malloc0(sizeof(*info));
+info->name = g_strdup(prop->name);
+info->type = g_strdup(prop->type);
+info->has_description = !!prop->description;
+info->description = g_strdup(prop->description);
+
+entry = g_malloc0(sizeof(*entry));
+entry->value = info;
+entry->next = prop_list;
+prop_list = entry;
+}
+
+object_unref(obj);
+
+return prop_list;
+}
+
 CpuDefinitionInfoList *qmp_query_cpu_definitions(Error **errp)
 {
 return arch_query_cpu_definitions(errp);
-- 
2.11.0

[Qemu-devel] [PATCH v3 1/5] keymap: make struct kbd_layout_t private to ui/keymaps.c

[Gerd Hoffmann]

Also use kbd_layout_t pointers instead of void pointers.

Signed-off-by: Gerd Hoffmann 
Reviewed-by: Daniel P. Berrangé 
---
 ui/keymaps.h | 29 ++---
 ui/keymaps.c | 32 +---
 2 files changed, 31 insertions(+), 30 deletions(-)

diff --git a/ui/keymaps.h b/ui/keymaps.h
index 8757465529..17ec03387a 100644
--- a/ui/keymaps.h
+++ b/ui/keymaps.h
@@ -32,25 +32,6 @@ typedef struct {
int keysym;
 } name2keysym_t;
 
-struct key_range {
-int start;
-int end;
-struct key_range *next;
-};
-
-#define MAX_NORMAL_KEYCODE 512
-#define MAX_EXTRA_COUNT 256
-typedef struct {
-uint16_t keysym2keycode[MAX_NORMAL_KEYCODE];
-struct {
-   int keysym;
-   uint16_t keycode;
-} keysym2keycode_extra[MAX_EXTRA_COUNT];
-int extra_count;
-struct key_range *keypad_range;
-struct key_range *numlock_range;
-} kbd_layout_t;
-
 /* scancode without modifiers */
 #define SCANCODE_KEYMASK 0xff
 /* scancode without grey or up bit */
@@ -69,10 +50,12 @@ typedef struct {
 #define SCANCODE_ALT0x400
 #define SCANCODE_ALTGR  0x800
 
+typedef struct kbd_layout_t kbd_layout_t;
 
-void *init_keyboard_layout(const name2keysym_t *table, const char *language);
-int keysym2scancode(void *kbd_layout, int keysym);
-int keycode_is_keypad(void *kbd_layout, int keycode);
-int keysym_is_numlock(void *kbd_layout, int keysym);
+kbd_layout_t *init_keyboard_layout(const name2keysym_t *table,
+   const char *language);
+int keysym2scancode(kbd_layout_t *k, int keysym);
+int keycode_is_keypad(kbd_layout_t *k, int keycode);
+int keysym_is_numlock(kbd_layout_t *k, int keysym);
 
 #endif /* QEMU_KEYMAPS_H */
diff --git a/ui/keymaps.c b/ui/keymaps.c
index f9762d1497..134958a197 100644
--- a/ui/keymaps.c
+++ b/ui/keymaps.c
@@ -28,6 +28,26 @@
 #include "trace.h"
 #include "qemu/error-report.h"
 
+#define MAX_NORMAL_KEYCODE 512
+#define MAX_EXTRA_COUNT 256
+
+struct key_range {
+int start;
+int end;
+struct key_range *next;
+};
+
+struct kbd_layout_t {
+uint16_t keysym2keycode[MAX_NORMAL_KEYCODE];
+struct {
+int keysym;
+uint16_t keycode;
+} keysym2keycode_extra[MAX_EXTRA_COUNT];
+int extra_count;
+struct key_range *keypad_range;
+struct key_range *numlock_range;
+};
+
 static int get_keysym(const name2keysym_t *table,
   const char *name)
 {
@@ -186,15 +206,15 @@ static kbd_layout_t *parse_keyboard_layout(const 
name2keysym_t *table,
 }
 
 
-void *init_keyboard_layout(const name2keysym_t *table, const char *language)
+kbd_layout_t *init_keyboard_layout(const name2keysym_t *table,
+   const char *language)
 {
 return parse_keyboard_layout(table, language, NULL);
 }
 
 
-int keysym2scancode(void *kbd_layout, int keysym)
+int keysym2scancode(kbd_layout_t *k, int keysym)
 {
-kbd_layout_t *k = kbd_layout;
 if (keysym < MAX_NORMAL_KEYCODE) {
 if (k->keysym2keycode[keysym] == 0) {
 trace_keymap_unmapped(keysym);
@@ -217,9 +237,8 @@ int keysym2scancode(void *kbd_layout, int keysym)
 return 0;
 }
 
-int keycode_is_keypad(void *kbd_layout, int keycode)
+int keycode_is_keypad(kbd_layout_t *k, int keycode)
 {
-kbd_layout_t *k = kbd_layout;
 struct key_range *kr;
 
 for (kr = k->keypad_range; kr; kr = kr->next) {
@@ -230,9 +249,8 @@ int keycode_is_keypad(void *kbd_layout, int keycode)
 return 0;
 }
 
-int keysym_is_numlock(void *kbd_layout, int keysym)
+int keysym_is_numlock(kbd_layout_t *k, int keysym)
 {
-kbd_layout_t *k = kbd_layout;
 struct key_range *kr;
 
 for (kr = k->numlock_range; kr; kr = kr->next) {
-- 
2.9.3

Re: [Qemu-devel] [RFC PATCH v6 00/20] replay additions

[Pavel Dovgalyuk]

> From: Pavel Dovgalyuk [mailto:dovga...@ispras.ru]
> > From: Ciro Santilli [mailto:ciro.santi...@gmail.com]
> > On Wed, Feb 21, 2018 at 6:41 AM, Pavel Dovgalyuk  wrote:
> > >> From: Ciro Santilli [mailto:ciro.santi...@gmail.com]
> > >> On Tue, Feb 20, 2018 at 9:46 AM, Pavel Dovgalyuk  
> > >> wrote:
> > >> >
> > >> > Updated the branch on github.
> > >> > You may try it.
> > >>
> > >> At 8a482834780a131e7747c1c3c1931379ed0beedc ARM initrd record runs,
> > >> but replay is getting stuck at:
> > >>
> > >> [   12.120424] scsi host0: sym-2.2.3
> 
> Yes, I've got the same.
> But when I use -serial stdio instead of -nographic, replay makes a pause
> at this line and then continues and finishes successfully.

It doesn't stuck, but works much slower with -nographic. 
You just have to wait for 5 minutes or something to notice the progress.

Pavel Dovgalyuk

[Qemu-devel] [PATCH v2 2/3] qcow2: Don't allow overflow during cluster allocation

[Eric Blake]

Our code was already checking that we did not attempt to
allocate more clusters than what would fit in an INT64 (the
physical maximimum if we can access a full off_t's worth of
data).  But this does not catch smaller limits enforced by
various spots in the qcow2 image description: L1 and normal
clusters of L2 are documented as having bits 63-56 reserved
for other purposes, capping our maximum offset at 64PB (bit
55 is the maximum bit set).  And for compressed images with
2M clusters, the cap drops the maximum offset to bit 48, or
a maximum offset of 512TB.  If we overflow that offset, we
would write compressed data into one place, but try to
decompress from another, which won't work.

I don't have 512TB handy to prove whether things break if we
compress so much data that we overflow that limit, and don't
think that iotests can (quickly) test it either.  Test 138
comes close (it corrupts an image into thinking something lives
at 32PB, which is half the maximum for L1 sizing - although
it relies on 512-byte clusters).  But that test points out
that we will generally hit other limits first (such as running
out of memory for the refcount table, or exceeding file system
limits like 16TB on ext4, etc), so this is more a theoretical
safety valve than something likely to be hit.

Signed-off-by: Eric Blake 
---
 block/qcow2.h  |  6 ++
 block/qcow2-refcount.c | 20 +---
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index 883802241fb..560008c331d 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -41,6 +41,12 @@
 #define QCOW_MAX_CRYPT_CLUSTERS 32
 #define QCOW_MAX_SNAPSHOTS 65536

+/* Field widths in qcow2 mean normal cluster offsets cannot reach
+ * 64PB; depending on cluster size, compressed clusters can have a
+ * smaller limit (64PB for up to 16k clusters, then ramps down to
+ * 512TB for 2M clusters).  */
+#define QCOW_MAX_CLUSTER_OFFSET ((1ULL << 56) - 1)
+
 /* 8 MB refcount table is enough for 2 PB images at 64k cluster size
  * (128 GB for 512 byte clusters, 2 EB for 2 MB clusters) */
 #define QCOW_MAX_REFTABLE_SIZE 0x80
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 28afbb1b5ea..58c19789a22 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -31,7 +31,8 @@
 #include "qemu/bswap.h"
 #include "qemu/cutils.h"

-static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size);
+static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size,
+uint64_t max);
 static int QEMU_WARN_UNUSED_RESULT update_refcount(BlockDriverState *bs,
 int64_t offset, int64_t length, uint64_t addend,
 bool decrease, enum qcow2_discard_type type);
@@ -362,7 +363,8 @@ static int alloc_refcount_block(BlockDriverState *bs,
 }

 /* Allocate the refcount block itself and mark it as used */
-int64_t new_block = alloc_clusters_noref(bs, s->cluster_size);
+int64_t new_block = alloc_clusters_noref(bs, s->cluster_size,
+ QCOW_MAX_CLUSTER_OFFSET);
 if (new_block < 0) {
 return new_block;
 }
@@ -947,7 +949,8 @@ int qcow2_update_cluster_refcount(BlockDriverState *bs,


 /* return < 0 if error */
-static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size)
+static int64_t alloc_clusters_noref(BlockDriverState *bs, uint64_t size,
+uint64_t max)
 {
 BDRVQcow2State *s = bs->opaque;
 uint64_t i, nb_clusters, refcount;
@@ -972,9 +975,9 @@ retry:
 }

 /* Make sure that all offsets in the "allocated" range are representable
- * in an int64_t */
+ * in the requested max */
 if (s->free_cluster_index > 0 &&
-s->free_cluster_index - 1 > (INT64_MAX >> s->cluster_bits))
+s->free_cluster_index - 1 > (max >> s->cluster_bits))
 {
 return -EFBIG;
 }
@@ -994,7 +997,7 @@ int64_t qcow2_alloc_clusters(BlockDriverState *bs, uint64_t 
size)

 BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC);
 do {
-offset = alloc_clusters_noref(bs, size);
+offset = alloc_clusters_noref(bs, size, QCOW_MAX_CLUSTER_OFFSET);
 if (offset < 0) {
 return offset;
 }
@@ -1076,7 +1079,10 @@ int64_t qcow2_alloc_bytes(BlockDriverState *bs, int size)
 free_in_cluster = s->cluster_size - offset_into_cluster(s, offset);
 do {
 if (!offset || free_in_cluster < size) {
-int64_t new_cluster = alloc_clusters_noref(bs, s->cluster_size);
+int64_t new_cluster;
+
+new_cluster = alloc_clusters_noref(bs, s->cluster_size,
+   (1ULL << s->csize_shift) - 1);
 if (new_cluster < 0) {
 return new_cluster;
 }
-- 
2.14.3

[Qemu-devel] [PATCH v3 2/5] keymap: use glib hash for kbd_layout_t

[Gerd Hoffmann]

Drop home-grown lookup code, which is a strange mix of a lookup table
and a list.  Use standard glib hash instead.

Signed-off-by: Gerd Hoffmann 
---
 ui/keymaps.c| 73 -
 ui/trace-events |  2 +-
 2 files changed, 32 insertions(+), 43 deletions(-)

diff --git a/ui/keymaps.c b/ui/keymaps.c
index 134958a197..449c3dec22 100644
--- a/ui/keymaps.c
+++ b/ui/keymaps.c
@@ -28,22 +28,18 @@
 #include "trace.h"
 #include "qemu/error-report.h"
 
-#define MAX_NORMAL_KEYCODE 512
-#define MAX_EXTRA_COUNT 256
-
 struct key_range {
 int start;
 int end;
 struct key_range *next;
 };
 
+struct keysym2code {
+uint16_t keycode;
+};
+
 struct kbd_layout_t {
-uint16_t keysym2keycode[MAX_NORMAL_KEYCODE];
-struct {
-int keysym;
-uint16_t keycode;
-} keysym2keycode_extra[MAX_EXTRA_COUNT];
-int extra_count;
+GHashTable *hash;
 struct key_range *keypad_range;
 struct key_range *numlock_range;
 };
@@ -91,23 +87,19 @@ static void add_to_key_range(struct key_range **krp, int 
code) {
 }
 }
 
-static void add_keysym(char *line, int keysym, int keycode, kbd_layout_t *k) {
-if (keysym < MAX_NORMAL_KEYCODE) {
-trace_keymap_add("normal", keysym, keycode, line);
-k->keysym2keycode[keysym] = keycode;
-} else {
-if (k->extra_count >= MAX_EXTRA_COUNT) {
-warn_report("Could not assign keysym %s (0x%x)"
-" because of memory constraints.", line, keysym);
-} else {
-trace_keymap_add("extra", keysym, keycode, line);
-k->keysym2keycode_extra[k->extra_count].
-keysym = keysym;
-k->keysym2keycode_extra[k->extra_count].
-keycode = keycode;
-k->extra_count++;
-}
+static void add_keysym(char *line, int keysym, int keycode, kbd_layout_t *k)
+{
+struct keysym2code *keysym2code;
+
+keysym2code = g_hash_table_lookup(k->hash, GINT_TO_POINTER(keysym));
+if (keysym2code) {
+return;
 }
+
+keysym2code = g_new0(struct keysym2code, 1);
+keysym2code->keycode = keycode;
+g_hash_table_replace(k->hash, GINT_TO_POINTER(keysym), keysym2code);
+trace_keymap_add(keysym, keycode, line);
 }
 
 static kbd_layout_t *parse_keyboard_layout(const name2keysym_t *table,
@@ -131,6 +123,7 @@ static kbd_layout_t *parse_keyboard_layout(const 
name2keysym_t *table,
 
 if (!k) {
 k = g_new0(kbd_layout_t, 1);
+k->hash = g_hash_table_new(NULL, NULL);
 }
 
 for(;;) {
@@ -215,26 +208,22 @@ kbd_layout_t *init_keyboard_layout(const name2keysym_t 
*table,
 
 int keysym2scancode(kbd_layout_t *k, int keysym)
 {
-if (keysym < MAX_NORMAL_KEYCODE) {
-if (k->keysym2keycode[keysym] == 0) {
-trace_keymap_unmapped(keysym);
-warn_report("no scancode found for keysym %d", keysym);
-}
-return k->keysym2keycode[keysym];
-} else {
-int i;
+struct keysym2code *keysym2code;
+
 #ifdef XK_ISO_Left_Tab
-if (keysym == XK_ISO_Left_Tab) {
-keysym = XK_Tab;
-}
+if (keysym == XK_ISO_Left_Tab) {
+keysym = XK_Tab;
+}
 #endif
-for (i = 0; i < k->extra_count; i++) {
-if (k->keysym2keycode_extra[i].keysym == keysym) {
-return k->keysym2keycode_extra[i].keycode;
-}
-}
+
+keysym2code = g_hash_table_lookup(k->hash, GINT_TO_POINTER(keysym));
+if (!keysym2code) {
+trace_keymap_unmapped(keysym);
+warn_report("no scancode found for keysym %d", keysym);
+return 0;
 }
-return 0;
+
+return keysym2code->keycode;
 }
 
 int keycode_is_keypad(kbd_layout_t *k, int keycode)
diff --git a/ui/trace-events b/ui/trace-events
index 34229e6747..861b68a305 100644
--- a/ui/trace-events
+++ b/ui/trace-events
@@ -78,7 +78,7 @@ qemu_spice_create_update(uint32_t left, uint32_t right, 
uint32_t top, uint32_t b
 
 # ui/keymaps.c
 keymap_parse(const char *file) "file %s"
-keymap_add(const char *type, int sym, int code, const char *line) "%-6s 
sym=0x%04x code=0x%04x (line: %s)"
+keymap_add(int sym, int code, const char *line) "sym=0x%04x code=0x%04x (line: 
%s)"
 keymap_unmapped(int sym) "sym=0x%04x"
 
 # ui/x_keymap.c
-- 
2.9.3

[Qemu-devel] [PATCH V4 1/3] tests/migration: Convert the boot block compilation script into Makefile

[Wei Huang]

The x86 boot block header currently is generated with a shell script.
To better support other CPUs (e.g. aarch64), we convert the script
into Makefile. This allows us to 1) support cross-compilation easily,
and 2) avoid creating a script file for every architecture.

Signed-off-by: Wei Huang 
---
 tests/migration/Makefile | 38 
 tests/migration/rebuild-x86-bootblock.sh | 33 ---
 tests/migration/x86-a-b-bootblock.h  |  2 +-
 tests/migration/x86-a-b-bootblock.s  |  5 ++---
 4 files changed, 41 insertions(+), 37 deletions(-)
 create mode 100644 tests/migration/Makefile
 delete mode 100755 tests/migration/rebuild-x86-bootblock.sh

diff --git a/tests/migration/Makefile b/tests/migration/Makefile
new file mode 100644
index 00..1c07dd7be9
--- /dev/null
+++ b/tests/migration/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright (c) 2016-2018 Red Hat, Inc. and/or its affiliates
+#
+# Authors:
+#   Dave Gilbert 
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+path := $(subst :, ,$(PATH))
+system := $(shell uname -s | tr "A-Z" "a-z")
+
+cross-ld = $(firstword $(wildcard $(patsubst %,%/$(1)-*$(system)*-ld,$(path
+cross-gcc = $(firstword $(wildcard $(patsubst %ld,%gcc,$(call cross-ld,$(1)
+find-cross-prefix = $(subst gcc,,$(notdir $(call cross-gcc,$(1
+
+x86_64_cross_prefix := $(call find-cross-prefix,x86_64)
+
+export __note
+override define __note
+/* This file is automatically generated from
+ * tests/migration/$<, edit that and then run
+ * "make $@" inside tests/migration to update,
+ * and then remember to send both in your patch submission.
+ */
+endef
+
+all: x86-a-b-bootblock.h
+
+x86-a-b-bootblock.h: x86-a-b-bootblock.s
+   $(x86_64_cross_prefix)as --32 -march=i486 $< -o x86.o
+   $(x86_64_cross_prefix)objcopy -O binary x86.o x86.boot
+   dd if=x86.boot of=x86.bootsect bs=256 count=2 skip=124
+   echo "$$__note" > $@
+   xxd -i x86.bootsect | sed -e 's/.*int.*//' >> $@
+
+clean:
+   rm -f *.bootsect *.boot *.o
diff --git a/tests/migration/rebuild-x86-bootblock.sh 
b/tests/migration/rebuild-x86-bootblock.sh
deleted file mode 100755
index 86cec5d284..00
--- a/tests/migration/rebuild-x86-bootblock.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/bin/sh
-# Copyright (c) 2016-2018 Red Hat, Inc. and/or its affiliates
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
-# See the COPYING file in the top-level directory.
-#
-# Author: dgilb...@redhat.com
-
-ASMFILE=$PWD/tests/migration/x86-a-b-bootblock.s
-HEADER=$PWD/tests/migration/x86-a-b-bootblock.h
-
-if [ ! -e "$ASMFILE" ]
-then
-  echo "Couldn't find $ASMFILE" >&2
-  exit 1
-fi
-
-ASM_WORK_DIR=$(mktemp -d --tmpdir X86BB.XX)
-cd "$ASM_WORK_DIR" &&
-as --32 -march=i486 "$ASMFILE" -o x86.o &&
-objcopy -O binary x86.o x86.boot &&
-dd if=x86.boot of=x86.bootsect bs=256 count=2 skip=124 &&
-xxd -i x86.bootsect |
-sed -e 's/.*int.*//' > x86.hex &&
-cat - x86.hex < "$HEADER"
-/* This file is automatically generated from
- * tests/migration/x86-a-b-bootblock.s, edit that and then run
- * tests/migration/rebuild-x86-bootblock.sh to update,
- * and then remember to send both in your patch submission.
- */
-HERE
-
-rm x86.hex x86.bootsect x86.boot x86.o
-cd .. && rmdir "$ASM_WORK_DIR"
diff --git a/tests/migration/x86-a-b-bootblock.h 
b/tests/migration/x86-a-b-bootblock.h
index 78a151fe2a..9e8e2e028b 100644
--- a/tests/migration/x86-a-b-bootblock.h
+++ b/tests/migration/x86-a-b-bootblock.h
@@ -1,6 +1,6 @@
 /* This file is automatically generated from
  * tests/migration/x86-a-b-bootblock.s, edit that and then run
- * tests/migration/rebuild-x86-bootblock.sh to update,
+ * "make x86-a-b-bootblock.h" inside tests/migration to update,
  * and then remember to send both in your patch submission.
  */
 unsigned char x86_bootsect[] = {
diff --git a/tests/migration/x86-a-b-bootblock.s 
b/tests/migration/x86-a-b-bootblock.s
index b1642641a7..98dbfab084 100644
--- a/tests/migration/x86-a-b-bootblock.s
+++ b/tests/migration/x86-a-b-bootblock.s
@@ -3,9 +3,8 @@
 #  range.
 #  Outputs an initial 'A' on serial followed by repeated 'B's
 #
-# run   tests/migration/rebuild-x86-bootblock.sh
-#   to regenerate the hex, and remember to include both the .h and .s
-#   in any patches.
+#  In tests/migration dir, run 'make x86-a-b-bootblock.h' to regenerate
+#  the hex, and remember to include both the .h and .s in any patches.
 #
 # Copyright (c) 2016 Red Hat, Inc. and/or its affiliates
 # This work is licensed under the terms of the GNU GPL, version 2 or later.
-- 
2.14.3