Re: [Qemu-devel] [PATCH v11 0/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Thanks Jeff!

On Thu, Apr 20, 2017 at 8:03 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Mon, Apr 03, 2017 at 08:48:07PM -0700, Ashish Mittal wrote:
>> - Veritas HyperScale block driver in QEMU is designed to provide an 
>> accelerated
>>   IO path from KVM virtual machines to Veritas HyperScale storage service.
>>
>> - A network IO transfer library that translates block IO from HyperScale 
>> block
>>   driver to a network IO format to send it to Veritas HyperScale storage
>>   service. This library (libvxhs) has been open sourced and is available on
>>   github here:  https://github.com/VeritasHyperScale/libqnio.git
>>
>> Ashish Mittal (2):
>>   block/vxhs.c: Add support for a new block device type called "vxhs"
>>   block/vxhs.c: Add qemu-iotests for new block device type "vxhs"
>>
>>  block/Makefile.objs  |   2 +
>>  block/trace-events   |  17 ++
>>  block/vxhs.c | 575 
>> +++
>>  configure|  39 +++
>>  qapi/block-core.json |  23 +-
>>  tests/qemu-iotests/common|   6 +
>>  tests/qemu-iotests/common.config |  13 +
>>  tests/qemu-iotests/common.filter |   1 +
>>  tests/qemu-iotests/common.rc |  19 ++
>>  9 files changed, 693 insertions(+), 2 deletions(-)
>>  create mode 100644 block/vxhs.c
>>
>> --
>> 2.5.5
>>
>
> Thanks,
>
> Applied to my block branch:
>
> git://github.com/codyprime/qemu-kvm-jtc block
>
> -Jeff

Re: [Qemu-devel] [PATCH v11 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Apr 19, 2017 at 9:27 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Mon, Apr 03, 2017 at 08:48:08PM -0700, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/VeritasHyperScale/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
>> -k en-us -vga cirrus -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Sample command line using TLS credentials (run in secure mode):
>> ./qemu-io --object
>> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
>> -v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
>> "vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v11 changelog:
>> (1) Replaced InetSocketAddress with InetSocketAddressBase.
>> (2) Removed access to qemu_uuid.
>> (3) Removed unnecessary g_strdup()/g_free().
>> (4) Removed unused acb->qiov.
>> (5) Changed vxhs_init_and_ref() and vxhs_unref() per suggestion.
>> (6) Removed unnecessary initializations from local variables.
>
> QEMU code:
>
> Reviewed-by: Stefan Hajnoczi <stefa...@redhat.com>
>
Thanks!

> libvxhs is not robust yet.  Here are two examples:
>
> 1. If the response from the server has an invalid magic number then the
>whole VM is aborted:
>
>QNIO_API_(void) kvset_unmarshal(qnio_byte_t * bs, kvset_t * *p)
>{
>...
>assert(magic == kvset_magic);
>
> 2. There are buffer overflows and other memory corruptions.  For example
>when kv_binary_unpack() gets size=-1 over the wire.
>

We will continue to audit the library code and fix potential security
issues with libvxhs.

> The code needs to be audited line-by-line by someone aware of secure
> coding practices.  Please look into this.
>
> Also, is there a plan for getting libvxhs into Fedora and Debian?

At the moment, our requirement is only RHEL. The library should build
and work fine with any distro.
We also plan to build source tarballs and checkin to github for every
release of libvxhs.

Re: [Qemu-devel] [PATCH v11 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Tue, Apr 11, 2017 at 12:47 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Mon, Apr 03, 2017 at 08:48:08PM -0700, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/VeritasHyperScale/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
>> -k en-us -vga cirrus -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Sample command line using TLS credentials (run in secure mode):
>> ./qemu-io --object
>> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
>> -v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
>> "vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>
> I was testing this some with blockdev-add and blockdev-del, and this
> sequence causes a segfault:
>
> 1. blockdev-add vxhs image
> 2. blockdev-del above image
> 3. blockdev-add vxhs image  <--- segfaults
>
> Looking at it in gdb, this is an issue with libqnio.  The call to iio_fini()
> is not sufficiently thorough in cleaning up resources.
>
> In nio_client.c, qnc_ctx is never freed, because there does not
> seem to be a call such as 'qnc_driver_fini' that cleans up the allocated
> qnio_client_ctx.
>
> Therefore, on the second call to iio_init, the libqnio internal variable
> network_driver is NULL, because qnc_driver_init() returns NULL if it is
> called when qnc_ctx is still initialized:
>
>
>
> lib/qnio/nio_client.c:
>
> 411 int
> 412 iio_init(int32_t version, iio_cb_t cb)
> 413 {
>
> [...]
>
> 432 apictx->network_driver = qnc_secure_driver_init(client_callback);
> 433 nioDbg("Created API context.\n");
> 434 return 0;
> 435 }
>
> [...]
>
> 779 struct channel_driver *
> 780 qnc_driver_init(qnio_notify client_notify)
> 781 {
> 782 if (qnc_ctx) {
> 783 nioDbg("Driver already initialized");
> 784 return NULL;
> 785 }
> 786
>
>
> So two issues:
>
> A. iio_init() should check the returned pointer, and fail if NULL
>
> B. iio_fini() needs to clean everything up so that a new vxhs connection is
>possible. This likely means at least one new function in nio_client.c to
>clean up qnc_ctx.
>
> -Jeff

Thanks for reporting the issue and also suggesting the fix.
I was able to reproduce the issue and have checked in a fix to libqnio.

This was the stack trace from the core -

Program terminated with signal 11, Segmentation fault.
#0  0x7f12083a3afa in iio_channel_open (uri=0x7f120c57eb70
"of://127.0.0.1:", cacert=0x0, client_key=0x0, client_cert=0x0) at
lib/qnio/iioapi.c:105
#1  0x7f12083a470a in iio_open (uri=0x7f120d121be0
"of://127.0.0.1:", devid=0x7f120baa4bb0 "/test.raw", flags=0,
cacert=0x0, client_key=0x0, client_cert=0x0) at lib/qnio/iioapi.c:520
#2  0x7f12091bf7a3 in vxhs_open (bs=,
options=, bdrv_flags=,
errp=0x7ffdf1cafd70) at block/vxhs.c:388
#3  0x7f120916cf14 in bdrv_open_driver
(bs=bs@entry=0x7f120b1481b0, drv=drv@entry=0x7f1209843580 ,
node_name=, options=options@entry=0x7f120daa6840,
open_flags=8194,
errp=errp@entry=0x7ffdf1cafe28) at block.c:1011
#4  0x7f120917079f in bdrv_open_common (errp=0x7ffdf1cafe28,
options=0x7f120daa6840, file=0x0, bs=0x7f120b1481b0) at block.c:1250
#5  bdrv_open_inherit (filename=, filename@entry=0x0,
reference=reference@entry=0x0, options=0x7f120daa6840,
options@entry=0x7f120daa4800, flags=40962, parent=parent@entry=0x0,
child_role=child_role@entry=0x0, errp=errp@entry=0x7ffdf1caff18)
at block.c:2413
#6  0x7f1209171663 in bdrv_open (filename=filename@entry=0x0,
reference=reference@entry=0x0, options=options@entry=0x7f120daa4800,
flags=, errp=errp@entry=0x7ffdf1caff18)
at block.c:2505
#7  0x7f1208f9f026 in bds_tree_init
(bs_opts=bs_opts@entry=0x7f120daa4800, errp=errp@entry=0x7ffdf1caff18)
at blockdev.c:656
#8  0x7f1208fa4e5b in qmp_blockdev_add
(options=options@entry=0x7ffdf1caff20, errp=errp@entry=0x7ffdf1caff18)
at blockdev.c:3885
...

Please test again with refreshed libqnio and let me know in case of issues.

Thanks,
Ashish

[Qemu-devel] [PATCH v11 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/VeritasHyperScale/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
Reviewed-by: Stefan Hajnoczi <stefa...@redhat.com>
---
v11 changelog:
(1) No changes.

v10 changelog:
(1) Redirect o/p of "$QEMU_VXHS -d  $TEST_DIR" to /dev/null

v9 changelog:
(1) Dropped second argument to set_prog_path(). We will pick up the test
server location from the user's PATH env setting.

v8/v7 changelog:
(1) No changes.

v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index 4d5650d..9c6f972 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -157,6 +157,7 @@ check options
 -sshtest ssh
 -nfstest nfs
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -260,6 +261,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index 55527aa..c4b51b3 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 1040013..c9a2d5c 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -122,6 +122,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 7d4781d..62529ee 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -85,6 +85,9 @@ else
 elif [ "$IMGPROTO" = "nfs" ]; then
 TEST_DIR="nfs://127.0.0.1/$TEST_DIR"
 TEST_IMG=$TEST_DIR/t.$IMGFMT
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -171,6 +174,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
>/dev/null &"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR > /dev/null &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -197,6 +206,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
2.5.5

[Qemu-devel] [PATCH v11 0/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

- Veritas HyperScale block driver in QEMU is designed to provide an accelerated
  IO path from KVM virtual machines to Veritas HyperScale storage service.

- A network IO transfer library that translates block IO from HyperScale block
  driver to a network IO format to send it to Veritas HyperScale storage
  service. This library (libvxhs) has been open sourced and is available on
  github here:  https://github.com/VeritasHyperScale/libqnio.git

Ashish Mittal (2):
  block/vxhs.c: Add support for a new block device type called "vxhs"
  block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

 block/Makefile.objs  |   2 +
 block/trace-events   |  17 ++
 block/vxhs.c | 575 +++
 configure|  39 +++
 qapi/block-core.json |  23 +-
 tests/qemu-iotests/common|   6 +
 tests/qemu-iotests/common.config |  13 +
 tests/qemu-iotests/common.filter |   1 +
 tests/qemu-iotests/common.rc |  19 ++
 9 files changed, 693 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

-- 
2.5.5

[Qemu-devel] [PATCH v11 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/VeritasHyperScale/libqnio.git

Sample command line using JSON syntax:
./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
-k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Sample command line using TLS credentials (run in secure mode):
./qemu-io --object
tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
-v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
"vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v11 changelog:
(1) Replaced InetSocketAddress with InetSocketAddressBase.
(2) Removed access to qemu_uuid.
(3) Removed unnecessary g_strdup()/g_free().
(4) Removed unused acb->qiov.
(5) Changed vxhs_init_and_ref() and vxhs_unref() per suggestion.
(6) Removed unnecessary initializations from local variables.

v10 changelog:
(1) Implemented accepting TLS creds per block device via the CLI
(see 3rd e.g in commit log). Corresponding changes made to the
libqnio library.
(2) iio_open() changed to accept TLS creds and use these internally
to set up SSL connections.
(3) Got rid of hard-coded VXHS_UUID_DEF. qemu_uuid is no longer used
for authentication in any way.
(4) Removed unnecessary qdict_del(backing_options, str).
(5) Added '*tls-creds' to BlockdevOptionsVxHS.

v9 changelog:
(1) Fixes for all the review comments from v8. I have left the definition
of VXHS_UUID_DEF unchanged pending a better suggestion.
(2) qcow2 tests now pass on the vxhs test server.
(3) Packaging changes for libvxhs will be checked in to the git repo soon.
(4) I have not moved extern QemuUUID qemu_uuid to a separate header file.

v8 changelog:
(1) Security implementation for libqnio present in branch 'securify'.
Please use 'securify' branch for building libqnio and testing
with this patch.
(2) Renamed libqnio to libvxhs.
(3) Pass instance ID to libvxhs for SSL authentication.

v7 changelog:
(1) IO failover code has moved out to the libqnio library.
(2) Fixes for issues reported by Stefan on v6.
(3) Incorporated the QEMUBH patch provided by Stefan.
This is a replacement for the pipe mechanism used earlier.
(4) Fixes to the buffer overflows reported in libqnio.
(5) Input validations in vxhs.c to prevent any buffer overflows for 
arguments passed to libqnio.

v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  17 ++
 block/vxhs.c | 575 +++
 configure|  39 
 qapi/block-core.json |  23 ++-
 5 files changed, 654 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index de96f8e..ea95530 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 block-obj-$(if $(CONFIG_BZIP2),m,n) += dmg-bz2.o
diff --git a/block/trace-events b/block/trace-events
index 0bc5c0a..7758ec3 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -110,3 +110,20 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_po

Re: [Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Apr 3, 2017 at 8:11 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Fri, Mar 31, 2017 at 11:25:02AM -0700, ashish mittal wrote:
>> On Mon, Mar 27, 2017 at 6:04 PM, ashish mittal <ashmit...@gmail.com> wrote:
>> > On Mon, Mar 27, 2017 at 10:27 AM, Stefan Hajnoczi <stefa...@gmail.com> 
>> > wrote:
>> >> On Sun, Mar 26, 2017 at 07:50:35PM -0700, Ashish Mittal wrote:
>> >>
>> >> Have you tested live migration?
>> >>
>> >> If live migration is not supported then a migration blocker should be
>> >> added using migrate_add_blocker().
>> >>
>> >
>> > We do support live migration. We have been testing a fork of this code
>> > (slightly different version) with live migration.
>
> The reason I ask is because this patch doesn't implement the
> BlockDriver bdrv_invalidate_cache()/bdrv_inactivate() callbacks.  These
> functions are invoked during live migration so that the block driver can
> ensure data consistency.
>
> Since the destination QEMU process is launched while the source QEMU is
> still running and making changes to the disk, some block drivers need to
> discard metadata at migration handover time that was read upon opening
> the image on the destination.  The guarantees that they see the latest
> metadata.
>

Thanks! During live migration, we point the vdisks to the same host as
the migration source (specified on the vxhs command line), thereby
insuring consistent view of data.

> Not sure if libvxhs caches anything that might get stale during live
> migration, but I wanted to raise this question?
>
> Regarding a fork of this code that you haven't posted to the mailing
> list, it doesn't exist as far as anyone here is concerned :).  Therefore
> either the code on the mailing list needs to support migration or it
> must register a migration blocker to prevent migration.
>

The code on the mailing list does support live migration. Live
migration requires the support of proprietary vxhs bits and also our
orchestration code within OpenStack. We have been testing live
migration and it works without any data consistency issues.

>> >>> +static BlockDriver bdrv_vxhs = {
>> >>> +.format_name  = "vxhs",
>> >>> +.protocol_name= "vxhs",
>> >>> +.instance_size= sizeof(BDRVVXHSState),
>> >>> +.bdrv_file_open   = vxhs_open,
>> >>> +.bdrv_parse_filename  = vxhs_parse_filename,
>> >>> +.bdrv_close   = vxhs_close,
>> >>> +.bdrv_getlength   = vxhs_getlength,
>> >>> +.bdrv_aio_readv   = vxhs_aio_readv,
>> >>> +.bdrv_aio_writev  = vxhs_aio_writev,
>> >>
>> >> Missing .bdrv_aio_flush().  Does VxHS promise that every completed write
>> >> request is persistent?
>> >>
>> >
>> > Yes, every acknowledged write request is persistent.
>> >
>> >> In that case it may be better to disable the emulated disk write cache
>> >> so the guest operating system and application know not to send flush
>> >> commands.
>> >
>> > We do pass "cache=none" on the qemu command line for every block
>> > device. Are there any other code changes necessary? Any pointers will
>> > help.
>> >
>>
>> Upon further reading, I now understand that cache=none will not
>> disable the emulated disk write cache. I am trying to understand if -
>> (1) It should still not be a problem since flush will just be a no-op for us.
>
> The guest operating system and applications may take different code
> paths depending on the state of the disk write cache.
>
> Useless vmexits can be eliminated if the guest doesn't need to send
> flush commands.  Hence the file system and applications may perform
> better.
>
>> (2) Is there a way, or reason, to disable the emulated disk write
>> cache in the code for vxhs? I think passing WCE=0 to the guest has
>> something to do with this, although I have yet to figure out what that
>> means.
>
> Right, WCE == "Write Cache Enable".  If you disable the write cache then
> the guest's SCSI disk or virtio-blk drivers will notice that the disk
> does not require flush commands.
>
> Try launching a guest with -drive if=none,id=drive0,cache=directsync,...
> and you should see that the write cache is disabled:
>
>   # cat /sys/block/vda/queue/write_cache
>

Thanks! Will try this. As you mentioned, this might give us a slightly
better performance as it will avoid unnecessary flush from the guest
OS/app. I was wondering if there was a separate command line option
(other than cache=directsync) for passing WCE=0? Reading some of the
articles out there almost suggested as if there was! Maybe I was just
confused.

>> (3) Is this a must for merge?
>
> This doesn't affect the block driver code so no change is necessary.

Re: [Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Mar 27, 2017 at 6:04 PM, ashish mittal <ashmit...@gmail.com> wrote:
> On Mon, Mar 27, 2017 at 10:27 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
>> On Sun, Mar 26, 2017 at 07:50:35PM -0700, Ashish Mittal wrote:
>>
>> Have you tested live migration?
>>
>> If live migration is not supported then a migration blocker should be
>> added using migrate_add_blocker().
>>
>
> We do support live migration. We have been testing a fork of this code
> (slightly different version) with live migration.
>
>>> v10 changelog:
>>> (1) Implemented accepting TLS creds per block device via the CLI
>>> (see 3rd e.g in commit log). Corresponding changes made to the
>>> libqnio library.
>>> (2) iio_open() changed to accept TLS creds and use these internally
>>> to set up SSL connections.
>>> (3) Got rid of hard-coded VXHS_UUID_DEF. qemu_uuid is no longer used
>>> for authentication in any way.
>>
>> Why does the code still access qemu_uuid and pass the UUID string to
>> iio_init()?
>>
>
> I was of the opinion that knowing what instance (for qemu-kvm case)
> was opening a block device could be a useful piece of information for
> the block device to have in the future.
>
>> In libqnio.git (66698ca47bc594a9f623c240d63ea535f5a42b47) the 'instance'
>> field is unused and not sent over the wire.  Please drop it.
>>
>
> It is not used at present. I will drop it.
>
>>> diff --git a/block/vxhs.c b/block/vxhs.c
>>> new file mode 100644
>>> index 000..b98b535
>>> --- /dev/null
>>> +++ b/block/vxhs.c
>>> @@ -0,0 +1,595 @@
>>> +/*
>>> + * QEMU Block driver for Veritas HyperScale (VxHS)
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or 
>>> later.
>>> + * See the COPYING file in the top-level directory.
>>> + *
>>> + */
>>> +
>>> +#include "qemu/osdep.h"
>>> +#include 
>>> +#include 
>>> +#include "block/block_int.h"
>>> +#include "qapi/qmp/qerror.h"
>>> +#include "qapi/qmp/qdict.h"
>>> +#include "qapi/qmp/qstring.h"
>>> +#include "trace.h"
>>> +#include "qemu/uri.h"
>>> +#include "qapi/error.h"
>>> +#include "qemu/uuid.h"
>>> +#include "crypto/tlscredsx509.h"
>>> +
>>> +#define VXHS_OPT_FILENAME   "filename"
>>> +#define VXHS_OPT_VDISK_ID   "vdisk-id"
>>> +#define VXHS_OPT_SERVER "server"
>>> +#define VXHS_OPT_HOST   "host"
>>> +#define VXHS_OPT_PORT   "port"
>>> +
>>> +QemuUUID qemu_uuid __attribute__ ((weak));
>>> +
>>> +static uint32_t vxhs_ref;
>>
>> It would be nice to add:
>> /* Only accessed under QEMU global mutex */
>>
> Will do.
>
>>> +/*
>>> + * Parse the incoming URI and populate *options with the host information.
>>> + * URI syntax has the limitation of supporting only one host info.
>>> + * To pass multiple host information, use the JSON syntax.
>>
>> References to multiple hosts are out of date.  The driver only supports
>> a single host now.
>>
> Will change.
>
>>> + */
>>> +static int vxhs_parse_uri(const char *filename, QDict *options)
>>> +{
>>> +URI *uri = NULL;
>>> +char *hoststr, *portstr;
>>> +char *port;
>>> +int ret = 0;
>>> +
>>> +trace_vxhs_parse_uri_filename(filename);
>>> +uri = uri_parse(filename);
>>> +if (!uri || !uri->server || !uri->path) {
>>> +uri_free(uri);
>>> +return -EINVAL;
>>> +}
>>> +
>>> +hoststr = g_strdup(VXHS_OPT_SERVER".host");
>>> +qdict_put(options, hoststr, qstring_from_str(uri->server));
>>> +g_free(hoststr);
>>> +
>>> +portstr = g_strdup(VXHS_OPT_SERVER".port");
>>> +if (uri->port) {
>>> +port = g_strdup_printf("%d", uri->port);
>>> +qdict_put(options, portstr, qstring_from_str(port));
>>> +g_free(port);
>>> +}
>>> +g_free(portstr);
>>
>> The g_strdup()/g_free() isn't necessary for the qdict_put() key
>> argument.  The key belongs to the caller so we can pass a string
>> literal:
>
> Will Change.
&

Re: [Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Hi Jeff,

Will incorporate all your review comments with two small dependent
changes as follows:

@@ -302,14 +286,14 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,
  int bdrv_flags, Error **errp)
 {
 BDRVVXHSState *s = bs->opaque;
-void *dev_handlep = NULL;
+void *dev_handlep;
 QDict *backing_options = NULL;
-QemuOpts *opts, *tcp_opts;   <== These two have to be init to
NULL because we potentially gfree them before assignment..
+QemuOpts *opts = NULL;
+QemuOpts *tcp_opts = NULL;
 char *of_vsa_addr = NULL;
 Error *local_err = NULL;
 const char *vdisk_id_opt;
 const char *server_host_opt;
-char *str = NULL;
 int ret = 0;
 char *cacert = NULL;
 char *client_key = NULL;
@@ -317,7 +301,8 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,

 ret = vxhs_init_and_ref();
 if (ret < 0) {
-return ret;
+ret = -EINVAL;
+goto out;  <== have to call vxhs_unref() in case of an error
to decrement the refcount..
 }

Would it help if I send out a diff over the last patch (vxhs.c only)
after incorporating changes from all three reviews? I can then send
v11 series if everyone okays the diff?

Regards,
Ashish

On Tue, Mar 28, 2017 at 10:03 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Sun, Mar 26, 2017 at 07:50:35PM -0700, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/VeritasHyperScale/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
>> -k en-us -vga cirrus -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Sample command line using TLS credentials (run in secure mode):
>> ./qemu-io --object
>> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
>> -v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
>> "vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>>
>> v10 changelog:
>> (1) Implemented accepting TLS creds per block device via the CLI
>> (see 3rd e.g in commit log). Corresponding changes made to the
>> libqnio library.
>> (2) iio_open() changed to accept TLS creds and use these internally
>> to set up SSL connections.
>> (3) Got rid of hard-coded VXHS_UUID_DEF. qemu_uuid is no longer used
>> for authentication in any way.
>> (4) Removed unnecessary qdict_del(backing_options, str).
>> (5) Added '*tls-creds' to BlockdevOptionsVxHS.
>>
>> v9 changelog:
>> (1) Fixes for all the review comments from v8. I have left the definition
>> of VXHS_UUID_DEF unchanged pending a better suggestion.
>> (2) qcow2 tests now pass on the vxhs test server.
>> (3) Packaging changes for libvxhs will be checked in to the git repo soon.
>> (4) I have not moved extern QemuUUID qemu_uuid to a separate header file.
>>
>> v8 changelog:
>> (1) Security implementation for libqnio present in branch 'securify'.
>> Please use 'securify' branch for building libqnio and testing
>> with this patch.
>> (2) Renamed libqnio to libvxhs.
>> (3) Pass instance ID to libvxhs for SSL authentication.
>>
>> v7 changelog:
>> (1) IO failover code has moved out to the libqnio library.
>> (2) Fixes for issues reported by Stefan on v6.
>> (3) Incorporated the QEMUBH patch provided by Stefan.
>> This is a replacement for the pipe mechanism used earlier.
>> (4) Fixes to the buffer overflows reported in libqnio.
>> (5) Input validations in vxhs.c to prevent any buffer overflows for
>> arguments passed to libqnio.
>>
>> v6 changelog:
>> (1) Added qemu-iotests for VxHS as a new patch in the series.
>> (2) Replaced release version from 2.8 to 2.9 in block-core.json.
>>
>> v5 changelog:
>> (1) Incorporated v4 review comments.
>>
>> v4 changelog:
>> (1) Incorporated v3 review comments on QAPI changes.
>> (2) Added r

Re: [Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Mar 27, 2017 at 10:27 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Sun, Mar 26, 2017 at 07:50:35PM -0700, Ashish Mittal wrote:
>
> Have you tested live migration?
>
> If live migration is not supported then a migration blocker should be
> added using migrate_add_blocker().
>

We do support live migration. We have been testing a fork of this code
(slightly different version) with live migration.

>> v10 changelog:
>> (1) Implemented accepting TLS creds per block device via the CLI
>> (see 3rd e.g in commit log). Corresponding changes made to the
>> libqnio library.
>> (2) iio_open() changed to accept TLS creds and use these internally
>> to set up SSL connections.
>> (3) Got rid of hard-coded VXHS_UUID_DEF. qemu_uuid is no longer used
>> for authentication in any way.
>
> Why does the code still access qemu_uuid and pass the UUID string to
> iio_init()?
>

I was of the opinion that knowing what instance (for qemu-kvm case)
was opening a block device could be a useful piece of information for
the block device to have in the future.

> In libqnio.git (66698ca47bc594a9f623c240d63ea535f5a42b47) the 'instance'
> field is unused and not sent over the wire.  Please drop it.
>

It is not used at present. I will drop it.

>> diff --git a/block/vxhs.c b/block/vxhs.c
>> new file mode 100644
>> index 000..b98b535
>> --- /dev/null
>> +++ b/block/vxhs.c
>> @@ -0,0 +1,595 @@
>> +/*
>> + * QEMU Block driver for Veritas HyperScale (VxHS)
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
>> + * See the COPYING file in the top-level directory.
>> + *
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include 
>> +#include 
>> +#include "block/block_int.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qapi/qmp/qdict.h"
>> +#include "qapi/qmp/qstring.h"
>> +#include "trace.h"
>> +#include "qemu/uri.h"
>> +#include "qapi/error.h"
>> +#include "qemu/uuid.h"
>> +#include "crypto/tlscredsx509.h"
>> +
>> +#define VXHS_OPT_FILENAME   "filename"
>> +#define VXHS_OPT_VDISK_ID   "vdisk-id"
>> +#define VXHS_OPT_SERVER "server"
>> +#define VXHS_OPT_HOST   "host"
>> +#define VXHS_OPT_PORT   "port"
>> +
>> +QemuUUID qemu_uuid __attribute__ ((weak));
>> +
>> +static uint32_t vxhs_ref;
>
> It would be nice to add:
> /* Only accessed under QEMU global mutex */
>
Will do.

>> +/*
>> + * Parse the incoming URI and populate *options with the host information.
>> + * URI syntax has the limitation of supporting only one host info.
>> + * To pass multiple host information, use the JSON syntax.
>
> References to multiple hosts are out of date.  The driver only supports
> a single host now.
>
Will change.

>> + */
>> +static int vxhs_parse_uri(const char *filename, QDict *options)
>> +{
>> +URI *uri = NULL;
>> +char *hoststr, *portstr;
>> +char *port;
>> +int ret = 0;
>> +
>> +trace_vxhs_parse_uri_filename(filename);
>> +uri = uri_parse(filename);
>> +if (!uri || !uri->server || !uri->path) {
>> +uri_free(uri);
>> +return -EINVAL;
>> +}
>> +
>> +hoststr = g_strdup(VXHS_OPT_SERVER".host");
>> +qdict_put(options, hoststr, qstring_from_str(uri->server));
>> +g_free(hoststr);
>> +
>> +portstr = g_strdup(VXHS_OPT_SERVER".port");
>> +if (uri->port) {
>> +port = g_strdup_printf("%d", uri->port);
>> +qdict_put(options, portstr, qstring_from_str(port));
>> +g_free(port);
>> +}
>> +g_free(portstr);
>
> The g_strdup()/g_free() isn't necessary for the qdict_put() key
> argument.  The key belongs to the caller so we can pass a string
> literal:

Will Change.

>
>   qdict_put(options, VXHS_OPT_SERVER ".host", qstring_from_str(uri->server));
>   if (uri->port) {
>   port = g_strdup_printf("%d", uri->port);
>   qdict_put(options, VXHS_OPT_SERVER ".port", qstring_from_str(port));
>   g_free(port);
>   }
>
>> +
>> +if (strstr(uri->path, "vxhs") == NULL) {
>
> What does this check do?
>

Not sure about the history, but it's been there since first code
draft. Will check if it serves any purpose, or remove it.

>> +static int

Re: [Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Mar 27, 2017 at 8:56 AM, Eric Blake <ebl...@redhat.com> wrote:
> On 03/26/2017 09:50 PM, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/VeritasHyperScale/libqnio.git
>
> When sending a multi-patch series, please include a 0/2 cover letter
> ('git config format.coverletter auto' can help).
>

Will do!

>>
>> Sample command line using JSON syntax:
>> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
>> -k en-us -vga cirrus -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>
> Do we really need URI syntax, now that we have -blockdev-add going into 2.9?
>

Having it just makes manual testing with CLI easier (less to type).
Would like to retain it for now, unless having this is a problem.
Removing it will also require rework and retesting of the qemu-iotests
patch.

>>
>> Sample command line using TLS credentials (run in secure mode):
>> ./qemu-io --object
>> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
>> -v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
>> "vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>>
>
> I'm just doing a high-level review of the interface, and leaving the
> actual code review to others.
>
>> +++ b/block/vxhs.c
>> @@ -0,0 +1,595 @@
>> +/*
>> + * QEMU Block driver for Veritas HyperScale (VxHS)
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
>> + * See the COPYING file in the top-level directory.
>> + *
>
> No Copyright notice?  The GPL works by virtue of copyright law, so
> generally a copyright holder should be mentioned.
>

I guess what's missing is the "Copyright (c) Veritas, LLC. 2017" line.
Is that correct?

>
>> +++ b/qapi/block-core.json
>> @@ -2118,6 +2118,7 @@
>>  # @iscsi: Since 2.9
>>  # @rbd: Since 2.9
>>  # @sheepdog: Since 2.9
>> +# @vxhs: Since 2.10
>>  #
>>  # Since: 2.0
>>  ##
>> @@ -2127,7 +2128,7 @@
>>  'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs',
>>  'null-aio', 'null-co', 'parallels', 'qcow', 'qcow2', 'qed',
>>  'quorum', 'raw', 'rbd', 'replication', 'sheepdog', 'ssh',
>> -'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
>> +'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs' ] }
>
> Markus has patches pending (to promote x-blockdev-del over to
> blockdev-del) that will conflict in context with this patch.  You may
> have to rebase again, but hopefully the conflicts should be easy to
> figure out.
>

I will rebase again as needed. Not having vxhs code merged yet does
create conflicts while rebasing. It would have been much easier to
just send out patches for the review comments :)

>>
>>  ##
>>  # @BlockdevOptionsFile:
>> @@ -2820,6 +2821,22 @@
>>'data': { '*offset': 'int', '*size': 'int' } }
>>
>>  ##
>> +# @BlockdevOptionsVxHS:
>> +#
>> +# Driver specific block device options for VxHS
>> +#
>> +# @vdisk-id:UUID of VxHS volume
>> +# @server:  vxhs server IP, port
>> +# @tls-creds:   TLS credentials ID
>> +#
>> +# Since: 2.10
>> +##
>> +{ 'struct': 'BlockdevOptionsVxHS',
>> +  'data': { 'vdisk-id': 'str',
>> +'server': 'InetSocketAddress',
>
> Do you want to use the new InetSocketAddressBase (just host and port,
> eliminating things like 'to' that don't make much sense in this context)?
>

Don't see InetSocketAddressBase in the latest code. Could you please
give me a usage example for reference? Thanks!

>> +'*tls-creds': 'str' } }
>> +
>> +##
>>  # @BlockdevOptions:
>>  #
>>  # Options for creating a block device.  Many options are available for all
>> @@ -2881,7 +2898,8 @@
>>'vhdx':   'BlockdevOptionsGenericFormat',
>>'vmdk':   'BlockdevOptionsGenericCOWFormat',
>>'vpc':'BlockdevOptionsGenericFormat',
>> -  'vvfat':  'BlockdevOptionsVVFAT'
>> +  'vvfat':  'BlockdevOptionsVVFAT',
>> +  'vxhs':   'BlockdevOptionsVxHS'
>>} }
>>
>>  ##
>>
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Mar 22, 2017 at 5:03 PM, ashish mittal <ashmit...@gmail.com> wrote:
> On Mon, Mar 20, 2017 at 5:55 AM, Daniel P. Berrange <berra...@redhat.com> 
> wrote:
>> On Fri, Mar 17, 2017 at 06:44:56PM -0700, ashish mittal wrote:
>>> On Thu, Mar 16, 2017 at 5:29 PM, ashish mittal <ashmit...@gmail.com> wrote:
>>> > On Mon, Mar 13, 2017 at 2:57 AM, Daniel P. Berrange <berra...@redhat.com> 
>>> > wrote:
>>> >> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>>> >>> Thanks! There is one more input I need some help with!
>>> >>>
>>> >>> VxHS network library opens a fixed number of connection channels to a
>>> >>> given host, and all the vdisks (that connect to the same host) share
>>> >>> these connection channels.
>>> >>>
>>> >>> Therefore, we need to open secure channels to a specific target host
>>> >>> only once for the first vdisk that connects to that host. All the
>>> >>> other vdisks that connect to the same target host will share the same
>>> >>> set of secure communication channels.
>>> >>>
>>> >>> I hope the above scheme is acceptable?
>>> >>>
>>> >>> If yes, then we have a couple of options to implement this:
>>> >>>
>>> >>> (1) Accept the TLS credentials per vdisk using the previously
>>> >>> discussed --object tls-creds-x509 mechanism. In this case, if more
>>> >>> than one vdisk have the same host info, then we will use only the
>>> >>> first one's creds to set up the secure connection, and ignore the
>>> >>> others. vdisks that connect to different target hosts will use their
>>> >>> individual tls-creds-x509 to set up the secure channels. This is, of
>>> >>> course, not relevant for qemu-img/qemu-io as they can open only one
>>> >>> vdisk at a time.
>>> >>
>>> >> It looks like option 1 here is the way to go. Just report an error if
>>> >> there are multiple creds provided for the same host and they don't
>>> >> match.
>>> >>
>>> >
>>> > I have made changes to implement option 1 in the library (branch
>>> > per_channel_ssl).
>>> > Can you please help review it?
>>> > https://github.com/VeritasHyperScale/libqnio/compare/per_channel_ssl
>>> >
>>> > Here's the changelog:
>>> > (1) Changed code to not use instance UUID for setting up SSL context.
>>> > (2) User is supposed to pass the cacert, client_key and client_cert
>>> > files to iio_open(). These will be used to set up a per-channel 
>>> > secure SSL
>>> > connection to the server. All three values are needed to set up a
>>> > secure connection.
>>> > (3) If the secure channel to a particular host is already open, other
>>> > block device connections to the same host will have to provide
>>> > TLS/SSL credentials that match the original one.
>>> > (4) Set default locations for trusted client CA certificates
>>> >  based on user specified cacert file.
>>> >
>>> > NB - I have given steps to test SSL communication (using the supplied
>>> > test client/server programs) in the commit log. I have not tested
>>> > using qemu binary yet. Will run more tests in the days to come.
>>> >
>>> > qemu vxhs patch changes should be pretty straightforward, given that
>>> > we have already discussed how to implement passing --object
>>> > tls-creds-x509 per block device.
>>> >
>>> > Thanks!
>>> >
>>>
>>> Update -
>>> (1) Successfully tested SSL communication using qemu-io and the test server.
>>> (2) Minor changes to per_channel_ssl branch.
>>> (3) Created a pull request.
>>>
>>> Please review per convenience. Thanks!
>>
>> IIUC, on that branch the 'is_secure()' method is still looking for the
>> directory /var/lib/libvxhs/secure to exist on the host. If that is not
>> present, then it appears to be silently ignoring the SSL certs passed
>> in from QEMU.
>>
>> IMHO it should enable TLS when 'cacert' passed to iio_open is not NULL,
>> not relying on a magic directory to exist.
>>
>
> I have made changes per above to the library. Please see commits -
>
> https://github.com/VeritasHyperScale/libqnio

[Qemu-devel] [PATCH v10 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/VeritasHyperScale/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v10 changelog:
(1) Redirect o/p of "$QEMU_VXHS -d  $TEST_DIR" to /dev/null

v9 changelog:
(1) Dropped second argument to set_prog_path(). We will pick up the test
server location from the user's PATH env setting.

v8/v7 changelog:
(1) No changes.

v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index 4d5650d..9c6f972 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -157,6 +157,7 @@ check options
 -sshtest ssh
 -nfstest nfs
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -260,6 +261,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index 55527aa..c4b51b3 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 1040013..c9a2d5c 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -122,6 +122,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 7d4781d..62529ee 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -85,6 +85,9 @@ else
 elif [ "$IMGPROTO" = "nfs" ]; then
 TEST_DIR="nfs://127.0.0.1/$TEST_DIR"
 TEST_IMG=$TEST_DIR/t.$IMGFMT
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -171,6 +174,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
>/dev/null &"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR > /dev/null &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -197,6 +206,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
2.5.5

[Qemu-devel] [PATCH v10 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/VeritasHyperScale/libqnio.git

Sample command line using JSON syntax:
./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
-k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Sample command line using TLS credentials (run in secure mode):
./qemu-io --object
tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
-v 66000 2.5k' 'json:{"server.host": "127.0.0.1", "server.port": "",
"vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---

v10 changelog:
(1) Implemented accepting TLS creds per block device via the CLI
(see 3rd e.g in commit log). Corresponding changes made to the
libqnio library.
(2) iio_open() changed to accept TLS creds and use these internally
to set up SSL connections.
(3) Got rid of hard-coded VXHS_UUID_DEF. qemu_uuid is no longer used
for authentication in any way.
(4) Removed unnecessary qdict_del(backing_options, str).
(5) Added '*tls-creds' to BlockdevOptionsVxHS.

v9 changelog:
(1) Fixes for all the review comments from v8. I have left the definition
of VXHS_UUID_DEF unchanged pending a better suggestion.
(2) qcow2 tests now pass on the vxhs test server.
(3) Packaging changes for libvxhs will be checked in to the git repo soon.
(4) I have not moved extern QemuUUID qemu_uuid to a separate header file.

v8 changelog:
(1) Security implementation for libqnio present in branch 'securify'.
Please use 'securify' branch for building libqnio and testing
with this patch.
(2) Renamed libqnio to libvxhs.
(3) Pass instance ID to libvxhs for SSL authentication.

v7 changelog:
(1) IO failover code has moved out to the libqnio library.
(2) Fixes for issues reported by Stefan on v6.
(3) Incorporated the QEMUBH patch provided by Stefan.
This is a replacement for the pipe mechanism used earlier.
(4) Fixes to the buffer overflows reported in libqnio.
(5) Input validations in vxhs.c to prevent any buffer overflows for 
arguments passed to libqnio.

v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  17 ++
 block/vxhs.c | 595 +++
 configure|  39 
 qapi/block-core.json |  22 +-
 5 files changed, 673 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index de96f8e..ea95530 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 block-obj-$(if $(CONFIG_BZIP2),m,n) += dmg-bz2.o
diff --git a/block/trace-events b/block/trace-events
index 0bc5c0a..7758ec3 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -110,3 +110,20 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_ii

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Mar 20, 2017 at 5:55 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Fri, Mar 17, 2017 at 06:44:56PM -0700, ashish mittal wrote:
>> On Thu, Mar 16, 2017 at 5:29 PM, ashish mittal <ashmit...@gmail.com> wrote:
>> > On Mon, Mar 13, 2017 at 2:57 AM, Daniel P. Berrange <berra...@redhat.com> 
>> > wrote:
>> >> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>> >>> Thanks! There is one more input I need some help with!
>> >>>
>> >>> VxHS network library opens a fixed number of connection channels to a
>> >>> given host, and all the vdisks (that connect to the same host) share
>> >>> these connection channels.
>> >>>
>> >>> Therefore, we need to open secure channels to a specific target host
>> >>> only once for the first vdisk that connects to that host. All the
>> >>> other vdisks that connect to the same target host will share the same
>> >>> set of secure communication channels.
>> >>>
>> >>> I hope the above scheme is acceptable?
>> >>>
>> >>> If yes, then we have a couple of options to implement this:
>> >>>
>> >>> (1) Accept the TLS credentials per vdisk using the previously
>> >>> discussed --object tls-creds-x509 mechanism. In this case, if more
>> >>> than one vdisk have the same host info, then we will use only the
>> >>> first one's creds to set up the secure connection, and ignore the
>> >>> others. vdisks that connect to different target hosts will use their
>> >>> individual tls-creds-x509 to set up the secure channels. This is, of
>> >>> course, not relevant for qemu-img/qemu-io as they can open only one
>> >>> vdisk at a time.
>> >>
>> >> It looks like option 1 here is the way to go. Just report an error if
>> >> there are multiple creds provided for the same host and they don't
>> >> match.
>> >>
>> >
>> > I have made changes to implement option 1 in the library (branch
>> > per_channel_ssl).
>> > Can you please help review it?
>> > https://github.com/VeritasHyperScale/libqnio/compare/per_channel_ssl
>> >
>> > Here's the changelog:
>> > (1) Changed code to not use instance UUID for setting up SSL context.
>> > (2) User is supposed to pass the cacert, client_key and client_cert
>> > files to iio_open(). These will be used to set up a per-channel secure 
>> > SSL
>> > connection to the server. All three values are needed to set up a
>> > secure connection.
>> > (3) If the secure channel to a particular host is already open, other
>> > block device connections to the same host will have to provide
>> > TLS/SSL credentials that match the original one.
>> > (4) Set default locations for trusted client CA certificates
>> >  based on user specified cacert file.
>> >
>> > NB - I have given steps to test SSL communication (using the supplied
>> > test client/server programs) in the commit log. I have not tested
>> > using qemu binary yet. Will run more tests in the days to come.
>> >
>> > qemu vxhs patch changes should be pretty straightforward, given that
>> > we have already discussed how to implement passing --object
>> > tls-creds-x509 per block device.
>> >
>> > Thanks!
>> >
>>
>> Update -
>> (1) Successfully tested SSL communication using qemu-io and the test server.
>> (2) Minor changes to per_channel_ssl branch.
>> (3) Created a pull request.
>>
>> Please review per convenience. Thanks!
>
> IIUC, on that branch the 'is_secure()' method is still looking for the
> directory /var/lib/libvxhs/secure to exist on the host. If that is not
> present, then it appears to be silently ignoring the SSL certs passed
> in from QEMU.
>
> IMHO it should enable TLS when 'cacert' passed to iio_open is not NULL,
> not relying on a magic directory to exist.
>

I have made changes per above to the library. Please see commits -

https://github.com/VeritasHyperScale/libqnio/commit/6c3261e9c9bb1350f4433a1ae4fcd98f7692cf39
https://github.com/VeritasHyperScale/libqnio/commit/502c74278457e6ac86a4ee4ad9102e56ff3be5d4

Commit log: Enable secure mode based on the SSL/TLS args passed in iio_open()

(1) Do not use /var/lib/libvxhs/secure to enable secure SSL mode on
the client side.
(2) Instead, enable SSL mode if the user passes TLS/SSL creds for
the block device on the qemu CLI.

Will be posting v10 qemu vxhs patch soon. v10 will work with the
latest library changes, and will support passing tls-creds-x509 creds
for every vxhs block device.


> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Thu, Mar 16, 2017 at 5:29 PM, ashish mittal <ashmit...@gmail.com> wrote:
> On Mon, Mar 13, 2017 at 2:57 AM, Daniel P. Berrange <berra...@redhat.com> 
> wrote:
>> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>>> Thanks! There is one more input I need some help with!
>>>
>>> VxHS network library opens a fixed number of connection channels to a
>>> given host, and all the vdisks (that connect to the same host) share
>>> these connection channels.
>>>
>>> Therefore, we need to open secure channels to a specific target host
>>> only once for the first vdisk that connects to that host. All the
>>> other vdisks that connect to the same target host will share the same
>>> set of secure communication channels.
>>>
>>> I hope the above scheme is acceptable?
>>>
>>> If yes, then we have a couple of options to implement this:
>>>
>>> (1) Accept the TLS credentials per vdisk using the previously
>>> discussed --object tls-creds-x509 mechanism. In this case, if more
>>> than one vdisk have the same host info, then we will use only the
>>> first one's creds to set up the secure connection, and ignore the
>>> others. vdisks that connect to different target hosts will use their
>>> individual tls-creds-x509 to set up the secure channels. This is, of
>>> course, not relevant for qemu-img/qemu-io as they can open only one
>>> vdisk at a time.
>>
>> It looks like option 1 here is the way to go. Just report an error if
>> there are multiple creds provided for the same host and they don't
>> match.
>>
>
> I have made changes to implement option 1 in the library (branch
> per_channel_ssl).
> Can you please help review it?
> https://github.com/VeritasHyperScale/libqnio/compare/per_channel_ssl
>
> Here's the changelog:
> (1) Changed code to not use instance UUID for setting up SSL context.
> (2) User is supposed to pass the cacert, client_key and client_cert
> files to iio_open(). These will be used to set up a per-channel secure SSL
> connection to the server. All three values are needed to set up a
> secure connection.
> (3) If the secure channel to a particular host is already open, other
> block device connections to the same host will have to provide
> TLS/SSL credentials that match the original one.
> (4) Set default locations for trusted client CA certificates
>  based on user specified cacert file.
>
> NB - I have given steps to test SSL communication (using the supplied
> test client/server programs) in the commit log. I have not tested
> using qemu binary yet. Will run more tests in the days to come.
>
> qemu vxhs patch changes should be pretty straightforward, given that
> we have already discussed how to implement passing --object
> tls-creds-x509 per block device.
>
> Thanks!
>

Update -
(1) Successfully tested SSL communication using qemu-io and the test server.
(2) Minor changes to per_channel_ssl branch.
(3) Created a pull request.

Please review per convenience. Thanks!

>>>
>>> (2) Instead of having a per-vdisk --object tls-creds-x509, have a
>>> single such argument on the command line for vxhs block device code to
>>> consume - if that is possible! One way to achieve this could be the
>>> user/password authentication we discussed earlier, which we could use
>>> to pass the directory where cert/keys are kept.
>>>
>>> (3) Use the instance UUID, when available, to lookup the cert files
>>> per instance (i.e. for qemu-kvm), and use the --object tls-creds-x509
>>> mechanism, when instance UUID is NULL (i.e. qemu-io, qemu-img etc).
>>> The cert/key files are anyway protected by file permissions in either
>>> case, so I guess there is no additional security provided by either
>>> method.
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org  -o- http://virt-manager.org :|
>> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Mar 13, 2017 at 2:57 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>> Thanks! There is one more input I need some help with!
>>
>> VxHS network library opens a fixed number of connection channels to a
>> given host, and all the vdisks (that connect to the same host) share
>> these connection channels.
>>
>> Therefore, we need to open secure channels to a specific target host
>> only once for the first vdisk that connects to that host. All the
>> other vdisks that connect to the same target host will share the same
>> set of secure communication channels.
>>
>> I hope the above scheme is acceptable?
>>
>> If yes, then we have a couple of options to implement this:
>>
>> (1) Accept the TLS credentials per vdisk using the previously
>> discussed --object tls-creds-x509 mechanism. In this case, if more
>> than one vdisk have the same host info, then we will use only the
>> first one's creds to set up the secure connection, and ignore the
>> others. vdisks that connect to different target hosts will use their
>> individual tls-creds-x509 to set up the secure channels. This is, of
>> course, not relevant for qemu-img/qemu-io as they can open only one
>> vdisk at a time.
>
> It looks like option 1 here is the way to go. Just report an error if
> there are multiple creds provided for the same host and they don't
> match.
>

I have made changes to implement option 1 in the library (branch
per_channel_ssl).
Can you please help review it?
https://github.com/VeritasHyperScale/libqnio/compare/per_channel_ssl

Here's the changelog:
(1) Changed code to not use instance UUID for setting up SSL context.
(2) User is supposed to pass the cacert, client_key and client_cert
files to iio_open(). These will be used to set up a per-channel secure SSL
connection to the server. All three values are needed to set up a
secure connection.
(3) If the secure channel to a particular host is already open, other
block device connections to the same host will have to provide
TLS/SSL credentials that match the original one.
(4) Set default locations for trusted client CA certificates
 based on user specified cacert file.

NB - I have given steps to test SSL communication (using the supplied
test client/server programs) in the commit log. I have not tested
using qemu binary yet. Will run more tests in the days to come.

qemu vxhs patch changes should be pretty straightforward, given that
we have already discussed how to implement passing --object
tls-creds-x509 per block device.

Thanks!

>>
>> (2) Instead of having a per-vdisk --object tls-creds-x509, have a
>> single such argument on the command line for vxhs block device code to
>> consume - if that is possible! One way to achieve this could be the
>> user/password authentication we discussed earlier, which we could use
>> to pass the directory where cert/keys are kept.
>>
>> (3) Use the instance UUID, when available, to lookup the cert files
>> per instance (i.e. for qemu-kvm), and use the --object tls-creds-x509
>> mechanism, when instance UUID is NULL (i.e. qemu-io, qemu-img etc).
>> The cert/key files are anyway protected by file permissions in either
>> case, so I guess there is no additional security provided by either
>> method.
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Mar 8, 2017 at 10:11 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Wed, Mar 08, 2017 at 09:59:32AM -0800, ashish mittal wrote:
>> On Wed, Mar 8, 2017 at 5:04 AM, Ketan Nilangekar
>> <ketan.nilange...@veritas.com> wrote:
>> >
>> >
>> >> On Mar 8, 2017, at 1:13 AM, Daniel P. Berrange <berra...@redhat.com> 
>> >> wrote:
>> >>
>> >>> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>> >>> Thanks! There is one more input I need some help with!
>> >>>
>> >>> VxHS network library opens a fixed number of connection channels to a
>> >>> given host, and all the vdisks (that connect to the same host) share
>> >>> these connection channels.
>> >>>
>> >>> Therefore, we need to open secure channels to a specific target host
>> >>> only once for the first vdisk that connects to that host. All the
>> >>> other vdisks that connect to the same target host will share the same
>> >>> set of secure communication channels.
>> >>>
>> >>> I hope the above scheme is acceptable?
>> >>
>> >> I don't think I'm in favour of such an approach, as it forces a single
>> >> QEMU process to use the same privileges for all disks it uses.
>> >>
>> >> Consider an example where a QEMU process has two disks, one shared
>> >> readonly disk and one exclusive writable disk, both on the same host.
>> >>
>> >
>> > This is not a use case for VxHS as a solution. We do not support sharing 
>> > of vdisks across QEMU instances.
>> >
>> > Vxhs library was thus not designed to open different connections for 
>> > individual vdisks within a QEMU instance.
>> >
>> > Implementing this will involve rewrite of significant parts of libvxhs 
>> > client and server. Is this a new requirement for acceptance into QEMU?
>> >
>> >
>> >> It is reasonable as an administrator to want to use different credentials
>> >> for each of these. ie, they might have a set of well known credentials to
>> >> authenticate to get access to the read-only disk, but have a different set
>> >> of strictly limited access credentials to get access to the writable disk
>> >>
>> >> Trying to re-use the same connection for multiple cause prevents QEMU from
>> >> authenticating with different credentials per disk, so I don't think that
>> >> is a good approach - each disk should have totally independant state.
>> >>
>>
>> libvxhs does not make any claim to fit all the general purpose
>> use-cases. It was purpose-built to be the communication channel for
>> our block device service. As such, we do not need/support all the
>> general use-cases. For the same reason we changed the name of the
>> library from linqnio to libvxhs (v8 changelog, #2).
>
> I raise these kind of points because they are relevant to apps like
> OpenStack, against which you've proposed VHXS support. OpenStack
> intends to allow a single volume to be shared by multiple guests,
> so declare that out of scope you're crippling certain use cases
> within openstack. Of course you're free to make such a decision,
> but it makes VHXS a less compelling technology to use IMHO.
>

Fair point! Sharing of the same disk across multiple guests would
require significant work from our side, and we would like to evaluate
that after OpenStack starts supporting the feature. Would adding this
support now, be a prerequisite for merging vxhs code to qemu?

>> Having dedicated communication channels for each device, or sharing
>> the channels between multiple devices, should both be acceptable
>> choices. The latter, IO multiplexing, is also a widely adopted IO
>> model. It just happens to fit our use-cases better.
>>
>> Binding access control to a communication channel would prevent
>> anybody from using the latter approach. Having a separate way to allow
>> access-control would permit the use of latter also.
>
> Sharing access control across multiple disks does not fit in effectively
> with the model used by apps that manage QEMU. Libvirt, and apps above
> libvirt, like OpenStack, oVirt, and things like Kubernetes all represent
> the information required to connect to a network block device, on a
> per-disk basis - there's no sense of having some information that is
> shared across all disks associated with a VM.
>
> So from the POV of modelling this in QEMU, all information needs to be
> specified against the individual -drive / -blockdev. If you really must,
> you will just have to reject configurations which imply talking to the
> same host, with incompatible parameters. Better would be to dynamically
> determine if you can re-use connections, vs spawn new connection based
> on the config.
>

Would this be a prerequisite for merging vxhs?

> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Mar 8, 2017 at 5:04 AM, Ketan Nilangekar
<ketan.nilange...@veritas.com> wrote:
>
>
>> On Mar 8, 2017, at 1:13 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
>>
>>> On Tue, Mar 07, 2017 at 05:27:55PM -0800, ashish mittal wrote:
>>> Thanks! There is one more input I need some help with!
>>>
>>> VxHS network library opens a fixed number of connection channels to a
>>> given host, and all the vdisks (that connect to the same host) share
>>> these connection channels.
>>>
>>> Therefore, we need to open secure channels to a specific target host
>>> only once for the first vdisk that connects to that host. All the
>>> other vdisks that connect to the same target host will share the same
>>> set of secure communication channels.
>>>
>>> I hope the above scheme is acceptable?
>>
>> I don't think I'm in favour of such an approach, as it forces a single
>> QEMU process to use the same privileges for all disks it uses.
>>
>> Consider an example where a QEMU process has two disks, one shared
>> readonly disk and one exclusive writable disk, both on the same host.
>>
>
> This is not a use case for VxHS as a solution. We do not support sharing of 
> vdisks across QEMU instances.
>
> Vxhs library was thus not designed to open different connections for 
> individual vdisks within a QEMU instance.
>
> Implementing this will involve rewrite of significant parts of libvxhs client 
> and server. Is this a new requirement for acceptance into QEMU?
>
>
>> It is reasonable as an administrator to want to use different credentials
>> for each of these. ie, they might have a set of well known credentials to
>> authenticate to get access to the read-only disk, but have a different set
>> of strictly limited access credentials to get access to the writable disk
>>
>> Trying to re-use the same connection for multiple cause prevents QEMU from
>> authenticating with different credentials per disk, so I don't think that
>> is a good approach - each disk should have totally independant state.
>>

libvxhs does not make any claim to fit all the general purpose
use-cases. It was purpose-built to be the communication channel for
our block device service. As such, we do not need/support all the
general use-cases. For the same reason we changed the name of the
library from linqnio to libvxhs (v8 changelog, #2).

Having dedicated communication channels for each device, or sharing
the channels between multiple devices, should both be acceptable
choices. The latter, IO multiplexing, is also a widely adopted IO
model. It just happens to fit our use-cases better.

Binding access control to a communication channel would prevent
anybody from using the latter approach. Having a separate way to allow
access-control would permit the use of latter also.

>>>
>>> If yes, then we have a couple of options to implement this:
>>>
>>> (1) Accept the TLS credentials per vdisk using the previously
>>> discussed --object tls-creds-x509 mechanism. In this case, if more
>>> than one vdisk have the same host info, then we will use only the
>>> first one's creds to set up the secure connection, and ignore the
>>> others. vdisks that connect to different target hosts will use their
>>> individual tls-creds-x509 to set up the secure channels. This is, of
>>> course, not relevant for qemu-img/qemu-io as they can open only one
>>> vdisk at a time.
>>>
>>> (2) Instead of having a per-vdisk --object tls-creds-x509, have a
>>> single such argument on the command line for vxhs block device code to
>>> consume - if that is possible! One way to achieve this could be the
>>> user/password authentication we discussed earlier, which we could use
>>> to pass the directory where cert/keys are kept.
>>>
>>> (3) Use the instance UUID, when available, to lookup the cert files
>>> per instance (i.e. for qemu-kvm), and use the --object tls-creds-x509
>>> mechanism, when instance UUID is NULL (i.e. qemu-io, qemu-img etc).
>>> The cert/key files are anyway protected by file permissions in either
>>> case, so I guess there is no additional security provided by either
>>> method.
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org  -o- http://virt-manager.org :|
>> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Thanks! There is one more input I need some help with!

VxHS network library opens a fixed number of connection channels to a
given host, and all the vdisks (that connect to the same host) share
these connection channels.

Therefore, we need to open secure channels to a specific target host
only once for the first vdisk that connects to that host. All the
other vdisks that connect to the same target host will share the same
set of secure communication channels.

I hope the above scheme is acceptable?

If yes, then we have a couple of options to implement this:

(1) Accept the TLS credentials per vdisk using the previously
discussed --object tls-creds-x509 mechanism. In this case, if more
than one vdisk have the same host info, then we will use only the
first one's creds to set up the secure connection, and ignore the
others. vdisks that connect to different target hosts will use their
individual tls-creds-x509 to set up the secure channels. This is, of
course, not relevant for qemu-img/qemu-io as they can open only one
vdisk at a time.

(2) Instead of having a per-vdisk --object tls-creds-x509, have a
single such argument on the command line for vxhs block device code to
consume - if that is possible! One way to achieve this could be the
user/password authentication we discussed earlier, which we could use
to pass the directory where cert/keys are kept.

(3) Use the instance UUID, when available, to lookup the cert files
per instance (i.e. for qemu-kvm), and use the --object tls-creds-x509
mechanism, when instance UUID is NULL (i.e. qemu-io, qemu-img etc).
The cert/key files are anyway protected by file permissions in either
case, so I guess there is no additional security provided by either
method.

Regards,
Ashish

On Mon, Mar 6, 2017 at 1:23 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Sun, Mar 05, 2017 at 04:26:05PM -0800, ashish mittal wrote:
>> On Wed, Mar 1, 2017 at 1:18 AM, Daniel P. Berrange <berra...@redhat.com> 
>> wrote:
>> >
>> > Yes, that's how other parts of QEMU deal with SSL
>> >
>> > NB, QEMU needs to pass 3 paths to libqnoio - the client cert, the
>> > client key, and the certificate authority certlist
>> >
>>
>> I see that the QEMU TLS code internally does expect to find cacert,
>> and errors out if this is missing. I did have to create one and place
>> it in the dir path where we are keeping the client key,cert files. The
>> code diff below requires all the three files.
>
> Yes, the client cert/key have to be paired with the correct CA cert. We cannot
> assume that the default CA cert bundle contains the right CA, for use with the
> cert/key. The default CA bundle contains all the public commercial CAs, and in
> general a QEMU deployment will never use any of those - the site will use a
> private internal only CA. While you could add the private CA to the global CA
> bundle, that is not desirable from a security POV, as it opens the deployment
> to risk from a rogue CA.
>
>> > The include/crypto/tlscredsx509.h file has constants defined for the
>> > standard filenames - eg you would concatenate the directory with
>> > the constants QCRYPTO_TLS_CREDS_X509_CLIENT_KEY.
>> >
>> > This gives the filenames you can then pass to libqnio
>> >
>>
>> I am using function qcrypto_tls_creds_get_path() to achieve the same.
>> Hope this is OK.
>
> Yep, that's fine.
>
>> Example CLI accepting the new TLS credentials:
>>
>> [amittal2@camshaft qemu] 2017-03-05 15:54:55$ ./qemu-io --trace
>> enable=vxhs* --object
>> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
>> 66000 128k' 'json:{"server.host": "127.0.0.1", "server.port": "",
>> "vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
>> 15116@1488758101.084355:vxhs_open_vdiskid Opening vdisk-id /test.raw
>> 15116@1488758101.084396:vxhs_get_creds cacert
>> /etc/pki/qemu/vxhs/ca-cert.pem, client_key
>> /etc/pki/qemu/vxhs/client-key.pem, client_cert
>> /etc/pki/qemu/vxhs/client-cert.pem   <=  NOTE 
>> 15116@1488758101.084402:vxhs_open_hostinfo Adding host 127.0.0.1:
>> to BDRVVXHSState
>> 15116@1488758101.092060:vxhs_get_vdisk_stat vDisk /test.raw stat ioctl
>> returned size 1048576
>> read 131072/131072 bytes at offset 66000
>> 128 KiB, 1 ops; 0.0006 sec (188.537 MiB/sec and 1508.2956 ops/sec)
>> 15116@1488758101.094643:vxhs_close Closing vdisk /test.raw
>> [amittal2@camshaft qemu] 2017-03-05 15:55:01$
>
> That looks ok.
>
>> @@ -315,33 +374,39 @@ static int vxhs_open(BlockDriverState *bs, QDict 
>> *options,
>>  

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Mar 1, 2017 at 1:18 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Tue, Feb 28, 2017 at 02:51:39PM -0800, ashish mittal wrote:
>> On Mon, Feb 27, 2017 at 1:22 AM, Daniel P. Berrange <berra...@redhat.com> 
>> wrote:
>
>> >> +ret = -EINVAL;
>> >> +goto out;
>> >> +}
>> >> +
>> >> +secretid = qemu_opt_get(opts, VXHS_OPT_SECRETID);
>> >> +if (!secretid) {
>> >> +error_setg(_err, "please specify the ID of the secret to 
>> >> be "
>> >> +   "used for authenticating to target");
>> >> +qdict_del(backing_options, str);
>> >> +ret = -EINVAL;
>> >> +goto out;
>> >> +}
>> >> +
>> >> +/* check if we got password via the --object argument */
>> >> +password = qcrypto_secret_lookup_as_utf8(secretid, _err);
>> >> +if (local_err != NULL) {
>> >> +trace_vxhs_get_creds(user, secretid, password);
>> >> +qdict_del(backing_options, str);
>> >> +ret = -EINVAL;
>> >> +goto out;
>> >> +}
>> >> +trace_vxhs_get_creds(user, secretid, password);
>> >> +
>> >>  s->vdisk_hostinfo.host = g_strdup(server_host_opt);
>> >>
>> >>  s->vdisk_hostinfo.port = g_ascii_strtoll(qemu_opt_get(tcp_opts,
>>
>> The next thing we need consensus on, is to decide exactly what
>> additional information to pass.
>>
>> (1) Current security implementation in VxHS uses the SSL key and
>> certificate files. Do you think we can achieve all the intended
>> security goals if we pass these two files paths (and names?) from the
>> command line?
>
> Yes, that's how other parts of QEMU deal with SSL
>
> NB, QEMU needs to pass 3 paths to libqnoio - the client cert, the
> client key, and the certificate authority certlist
>

I see that the QEMU TLS code internally does expect to find cacert,
and errors out if this is missing. I did have to create one and place
it in the dir path where we are keeping the client key,cert files. The
code diff below requires all the three files.

Here are the files I had to create -
$ ll /etc/pki/qemu/vxhs
total 12
-r--r--r--. 1 root root 2094 Mar  4 18:02 ca-cert.pem
-r--r--r--. 1 root root 1899 Mar  4 18:00 client-cert.pem
-r--r--r--. 1 root root 1679 Mar  4 17:59 client-key.pem

>> (2) If we are OK to continue with the present security scheme (using
>> key and cert), then the only additional change needed might be to
>> accept these files on the command line.
>
> Yep, I think that's the minimum required change to make this mergable.
>
>> (3) If we decide to rely on file permissions and SELinux policy to
>> protect these key/cert files, then we don't need to pass the file
>> names as a secret, instead, passing them as regular qemu_opt_get()
>> options might be enough.
>
> That's correct - you can assume file permissions protect the cert
> files. I would expect the syntax to work as follows
>
>   $QEMU  -object 
> tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client \
>  -drive  driver=vxhs,...other..opts...,tls-creds=tls0
>
> When you see the 'tls-creds' flag, you can lookup the corresponding
> QCryptoTLSCredsX509 object and extract the 'dir' property from it.
>

The code diff below works as above. Please verify.

> The include/crypto/tlscredsx509.h file has constants defined for the
> standard filenames - eg you would concatenate the directory with
> the constants QCRYPTO_TLS_CREDS_X509_CLIENT_KEY.
>
> This gives the filenames you can then pass to libqnio
>

I am using function qcrypto_tls_creds_get_path() to achieve the same.
Hope this is OK.

> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|


Example CLI accepting the new TLS credentials:

[amittal2@camshaft qemu] 2017-03-05 15:54:55$ ./qemu-io --trace
enable=vxhs* --object
tls-creds-x509,id=tls0,dir=/etc/pki/qemu/vxhs,endpoint=client -c 'read
66000 128k' 'json:{"server.host": "127.0.0.1", "server.port": "",
"vdisk-id": "/test.raw", "driver": "vxhs", "tls-creds":"tls0"}'
15116@1488758101.084355:vxhs_open_vdiskid Opening vdisk-id /test.raw
15116@1488758101.084396:vxhs_get_creds cacert
/etc/pki/qemu/vxhs/ca-cert.pem, client_key
/etc/pki/qemu/vxh

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Feb 27, 2017 at 1:22 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Fri, Feb 24, 2017 at 03:30:21PM -0800, ashish mittal wrote:
>> Thanks!
>>
>> I hope the following is in line with what you suggested -
>
> Yes, that looks suitable for password auth
>
Thanks!

>>
>> We will error out in case either of username, secret-id, or password
>> are missing.
>>
>> Good case, passing password via a file -
>> $ ./qemu-io --trace enable=vxhs* --object
>> secret,id=xvxhspasswd,file=/tmp/some/file/path  -c 'read 66000 128k'
>> 'json:{"server.host": "127.0.0.1", "server.port": "", "vdisk-id":
>> "/test.raw", "driver": "vxhs", "user": "ashish",  "password-secret":
>> "xvxhspasswd"}'
>> 1132@1487977829.151064:vxhs_open_vdiskid Opening vdisk-id /test.raw
>>
>> 1132@1487977829.151141:vxhs_get_creds User ashish, SecretID
>> xvxhspasswd, Password Str0ngP@ssw0rd   <===   NOTE WILL NOT PRINT
>> PASSWORD IN FINAL CODE 
>>
>> 1132@1487977829.151168:vxhs_open_hostinfo Adding host 127.0.0.1:
>> to BDRVVXHSState
>> 1132@1487977829.173062:vxhs_get_vdisk_stat vDisk /test.raw stat ioctl
>> returned size 196616
>> read 131072/131072 bytes at offset 66000
>> 128 KiB, 1 ops; 0.0012 sec (99.049 MiB/sec and 792.3930 ops/sec)
>> 1132@1487977829.175141:vxhs_close Closing vdisk /test.raw
>>
>>
>> Bad case, missing user -
>> $ ./qemu-io --trace enable=vxhs* --object
>> secret,id=xvxhspasswd,data=/tmp/some/file/path  -c 'read 66000 128k'
>> 'json:{"server.host": "127.0.0.1", "server.port": "", "vdisk-id":
>> "/test.raw", "driver": "vxhs"}'
>> 1310@1487978547.771234:vxhs_open_vdiskid Opening vdisk-id /test.raw
>> can't open device json:{"server.host": "127.0.0.1", "server.port":
>> "", "vdisk-id": "/test.raw", "driver": "vxhs"}: please specify the
>> user for authenticating to target
>>
>> diff --git a/block/vxhs.c b/block/vxhs.c
>> index 4f0633e..9b60ddf 100644
>> --- a/block/vxhs.c
>> +++ b/block/vxhs.c
>> @@ -17,12 +17,16 @@
>>  #include "qemu/uri.h"
>>  #include "qapi/error.h"
>>  #include "qemu/uuid.h"
>> +#include "crypto/secret.h"
>>
>>  #define VXHS_OPT_FILENAME   "filename"
>>  #define VXHS_OPT_VDISK_ID   "vdisk-id"
>>  #define VXHS_OPT_SERVER "server"
>>  #define VXHS_OPT_HOST   "host"
>>  #define VXHS_OPT_PORT   "port"
>> +#define VXHS_OPT_USER   "user"
>> +#define VXHS_OPT_PASSWORD   "password"
>> +#define VXHS_OPT_SECRETID   "password-secret"
>>  #define VXHS_UUID_DEF "12345678-1234-1234-1234-123456789012"
>>
>>  QemuUUID qemu_uuid __attribute__ ((weak));
>> @@ -136,6 +140,22 @@ static QemuOptsList runtime_opts = {
>>  .type = QEMU_OPT_STRING,
>>  .help = "UUID of the VxHS vdisk",
>>  },
>> +{
>> +.name = VXHS_OPT_USER,
>> +.type = QEMU_OPT_STRING,
>> +.help = "username for authentication to target",
>> +},
>> +{
>> +.name = VXHS_OPT_PASSWORD,
>> +.type = QEMU_OPT_STRING,
>> +.help = "password for authentication to target",
>> +},
>> +{
>> +.name = VXHS_OPT_SECRETID,
>> +.type = QEMU_OPT_STRING,
>> +.help = "ID of the secret providing password for"
>> +"authentication to target",
>> +},
>>  { /* end of list */ }
>>  },
>>  };
>> @@ -257,6 +277,9 @@ static int vxhs_open(BlockDriverState *bs, QDict 
>> *options,
>>  const char *server_host_opt;
>>  char *str = NULL;
>>  int ret = 0;
>> +const char *user = NULL;
>> +const char *secretid = NULL;
>> +const char *password = NULL;
>>
>>  ret = vxhs_init_and_ref();
>>  if (ret < 0) {
>> @@ -320,6 +343,35 @@ static int vxhs_open(BlockDriverState *bs, QDict 
>> *options,
>>  goto out;
>>  }
>>
>> +/* check if we got username and secretid via the optio

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Thanks!

I hope the following is in line with what you suggested -

We will error out in case either of username, secret-id, or password
are missing.

Good case, passing password via a file -
$ ./qemu-io --trace enable=vxhs* --object
secret,id=xvxhspasswd,file=/tmp/some/file/path  -c 'read 66000 128k'
'json:{"server.host": "127.0.0.1", "server.port": "", "vdisk-id":
"/test.raw", "driver": "vxhs", "user": "ashish",  "password-secret":
"xvxhspasswd"}'
1132@1487977829.151064:vxhs_open_vdiskid Opening vdisk-id /test.raw

1132@1487977829.151141:vxhs_get_creds User ashish, SecretID
xvxhspasswd, Password Str0ngP@ssw0rd   <===   NOTE WILL NOT PRINT
PASSWORD IN FINAL CODE 

1132@1487977829.151168:vxhs_open_hostinfo Adding host 127.0.0.1:
to BDRVVXHSState
1132@1487977829.173062:vxhs_get_vdisk_stat vDisk /test.raw stat ioctl
returned size 196616
read 131072/131072 bytes at offset 66000
128 KiB, 1 ops; 0.0012 sec (99.049 MiB/sec and 792.3930 ops/sec)
1132@1487977829.175141:vxhs_close Closing vdisk /test.raw


Bad case, missing user -
$ ./qemu-io --trace enable=vxhs* --object
secret,id=xvxhspasswd,data=/tmp/some/file/path  -c 'read 66000 128k'
'json:{"server.host": "127.0.0.1", "server.port": "", "vdisk-id":
"/test.raw", "driver": "vxhs"}'
1310@1487978547.771234:vxhs_open_vdiskid Opening vdisk-id /test.raw
can't open device json:{"server.host": "127.0.0.1", "server.port":
"", "vdisk-id": "/test.raw", "driver": "vxhs"}: please specify the
user for authenticating to target

diff --git a/block/vxhs.c b/block/vxhs.c
index 4f0633e..9b60ddf 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -17,12 +17,16 @@
 #include "qemu/uri.h"
 #include "qapi/error.h"
 #include "qemu/uuid.h"
+#include "crypto/secret.h"

 #define VXHS_OPT_FILENAME   "filename"
 #define VXHS_OPT_VDISK_ID   "vdisk-id"
 #define VXHS_OPT_SERVER "server"
 #define VXHS_OPT_HOST   "host"
 #define VXHS_OPT_PORT   "port"
+#define VXHS_OPT_USER   "user"
+#define VXHS_OPT_PASSWORD   "password"
+#define VXHS_OPT_SECRETID   "password-secret"
 #define VXHS_UUID_DEF "12345678-1234-1234-1234-123456789012"

 QemuUUID qemu_uuid __attribute__ ((weak));
@@ -136,6 +140,22 @@ static QemuOptsList runtime_opts = {
 .type = QEMU_OPT_STRING,
 .help = "UUID of the VxHS vdisk",
 },
+{
+.name = VXHS_OPT_USER,
+.type = QEMU_OPT_STRING,
+.help = "username for authentication to target",
+},
+{
+.name = VXHS_OPT_PASSWORD,
+.type = QEMU_OPT_STRING,
+.help = "password for authentication to target",
+},
+{
+.name = VXHS_OPT_SECRETID,
+.type = QEMU_OPT_STRING,
+.help = "ID of the secret providing password for"
+"authentication to target",
+},
 { /* end of list */ }
 },
 };
@@ -257,6 +277,9 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,
 const char *server_host_opt;
 char *str = NULL;
 int ret = 0;
+const char *user = NULL;
+const char *secretid = NULL;
+const char *password = NULL;

 ret = vxhs_init_and_ref();
 if (ret < 0) {
@@ -320,6 +343,35 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,
 goto out;
 }

+/* check if we got username and secretid via the options */
+user = qemu_opt_get(opts, VXHS_OPT_USER);
+if (!user) {
+error_setg(_err, "please specify the user for authenticating to "
+   "target");
+qdict_del(backing_options, str);
+ret = -EINVAL;
+goto out;
+}
+
+secretid = qemu_opt_get(opts, VXHS_OPT_SECRETID);
+if (!secretid) {
+error_setg(_err, "please specify the ID of the secret to be "
+   "used for authenticating to target");
+qdict_del(backing_options, str);
+ret = -EINVAL;
+goto out;
+}
+
+/* check if we got password via the --object argument */
+password = qcrypto_secret_lookup_as_utf8(secretid, _err);
+if (local_err != NULL) {
+trace_vxhs_get_creds(user, secretid, password);
+qdict_del(backing_options, str);
+ret = -EINVAL;
+goto out;
+}
+trace_vxhs_get_creds(user, secretid, password);
+
 s->vdisk_hostinfo.host = g_strdup(server_host_opt);

 s->vdisk_hostinfo.port = g_ascii_strtoll(qemu_opt_get(tcp_opts,

Re

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Hi,

Just want to check if the following mechanism for accepting the secret
password looks OK?

We have yet to internally discuss the semantics of how we plan to use
the user-ID/password for authentication. This diff is just to
understand how I am expected to accept the secret from the command
line.

Example specifying the username and password:

$ ./qemu-io --trace enable=vxhs* --object
secret,id=ashish,data=asd888d989s  -c 'read 66000 128k'
'json:{"server.host": "127.0.0.1", "server.port": "", "vdisk-id":
"/test.raw", "driver": "vxhs", "user": "ashish"}'
17396@1487908905.546890:vxhs_open_vdiskid Opening vdisk-id /test.raw
17396@1487908905.546969:vxhs_get_creds SecretID ashish, Password
asd888d989s   < NOTE!
17396@1487908905.546994:vxhs_open_hostinfo Adding host 127.0.0.1:
to BDRVVXHSState
17396@1487908905.568370:vxhs_get_vdisk_stat vDisk /test.raw stat ioctl
returned size 196616
read 131072/131072 bytes at offset 66000
128 KiB, 1 ops; 0.0017 sec (72.844 MiB/sec and 582.7506 ops/sec)
17396@1487908905.570917:vxhs_close Closing vdisk /test.raw
[root@rhel72ga-build-upstream qemu] 2017-02-23 20:01:45$

Example where username and password are missing:

[root@rhel72ga-build-upstream qemu] 2017-02-23 19:30:16$ ./qemu-io
--trace enable=vxhs* -c 'read 66000 128k' 'json:{"server.host":
"127.0.0.1", "server.port": "", "vdisk-id": "/test.raw", "driver":
"vxhs"}'
16120@1487907025.251167:vxhs_open_vdiskid Opening vdisk-id /test.raw
16120@1487907025.251245:vxhs_get_creds SecretID user, Password (null)
can't open device json:{"server.host": "127.0.0.1", "server.port":
"", "vdisk-id": "/test.raw", "driver": "vxhs"}: No secret with id
'user'  < NOTE!
[root@rhel72ga-build-upstream qemu] 2017-02-23 19:30:25$


diff --git a/block/vxhs.c b/block/vxhs.c
index 4f0633e..f3e70ce 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -17,6 +17,7 @@
 #include "qemu/uri.h"
 #include "qapi/error.h"
 #include "qemu/uuid.h"
+#include "crypto/secret.h"

 #define VXHS_OPT_FILENAME   "filename"
 #define VXHS_OPT_VDISK_ID   "vdisk-id"
@@ -136,6 +137,14 @@ static QemuOptsList runtime_opts = {
 .type = QEMU_OPT_STRING,
 .help = "UUID of the VxHS vdisk",
 },
+{
+.name = "user",
+.type = QEMU_OPT_STRING,
+},
+{
+.name = "password",
+.type = QEMU_OPT_STRING,
+},
 { /* end of list */ }
 },
 };
@@ -257,6 +266,8 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,
 const char *server_host_opt;
 char *str = NULL;
 int ret = 0;
+const char *user = NULL;
+const char *password = NULL;

 ret = vxhs_init_and_ref();
 if (ret < 0) {
@@ -320,6 +331,23 @@ static int vxhs_open(BlockDriverState *bs, QDict *options,
 goto out;
 }

+/* check if we got username via the options */
+user = qemu_opt_get(opts, "user");
+if (!user) {
+//trace_vxhs_get_creds(user, password);
+user = "user";
+//return;
+}
+
+password = qcrypto_secret_lookup_as_utf8(user, _err);
+if (local_err != NULL) {
+trace_vxhs_get_creds(user, password);
+qdict_del(backing_options, str);
+ret = -EINVAL;
+goto out;
+}
+trace_vxhs_get_creds(user, password);
+
 s->vdisk_hostinfo.host = g_strdup(server_host_opt);

 s->vdisk_hostinfo.port = g_ascii_strtoll(qemu_opt_get(tcp_opts,

Thanks,
Ashish

On Wed, Feb 22, 2017 at 6:44 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Wed, Feb 22, 2017 at 02:22:30PM +, Daniel P. Berrange wrote:
>> On Wed, Feb 22, 2017 at 02:09:20PM +, Stefan Hajnoczi wrote:
>> > On Tue, Feb 21, 2017 at 11:33:53AM +, Daniel P. Berrange wrote:
>> > > On Tue, Feb 21, 2017 at 10:59:18AM +, Stefan Hajnoczi wrote:
>> > > > On Mon, Feb 20, 2017 at 03:34:57AM -0800, ashish mittal wrote:
>> > > > > On Mon, Feb 20, 2017 at 3:02 AM, Stefan Hajnoczi 
>> > > > > <stefa...@gmail.com> wrote:
>> > > > > > On Sat, Feb 18, 2017 at 12:30:31AM +, Ketan Nilangekar wrote:
>> > > > > >> On 2/17/17, 1:42 PM, "Jeff Cody" <jc...@redhat.com> wrote:
>> > > > > >>
>> > > > > >> On Thu, Feb 16, 2017 at 02:24:19PM -0800, ashish mittal wrote:
>> > > > > >> > Hi,
>> > > > > >> >
>> > > > > >> > I am getting the followin

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Feb 20, 2017 at 3:02 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Sat, Feb 18, 2017 at 12:30:31AM +, Ketan Nilangekar wrote:
>> On 2/17/17, 1:42 PM, "Jeff Cody" <jc...@redhat.com> wrote:
>>
>> On Thu, Feb 16, 2017 at 02:24:19PM -0800, ashish mittal wrote:
>> > Hi,
>> >
>> > I am getting the following error with checkpatch.pl
>> >
>> > ERROR: externs should be avoided in .c files
>> > #78: FILE: block/vxhs.c:28:
>> > +QemuUUID qemu_uuid __attribute__ ((weak));
>> >
>> > Is there any way to get around this, or does it mean that I would have
>> > to add a vxhs.h just for this one entry?
>> >
>>
>> I remain skeptical on the use of the qemu_uuid as a way to select the TLS
>> cert.
>>
>> [ketan]
>> Is there another identity that can be used for uniquely identifying 
>> instances?
>> The requirement was to enforce vdisk access to owner instances.
>
> The qemu_uuid weak attribute looks suspect.  What is going to provide a
> strong qemu_uuid symbol?
>
> Why aren't configuration parameters like the UUID coming from the QEMU
> command-line?
>
> Stefan

UUID will in fact come from the QEMU command line. VxHS is not doing
anything special here. It will just use the value already available to
qemu-kvm process.

QemuUUID qemu_uuid;
bool qemu_uuid_set;

Both the above are defined in vl.c. vl.c will provide the strong
symbol when available. There are certain binaries that do not get
linked with vl.c (e.g. qemu-img). The weak symbol will come into
affect for such binaries, and in this case, the default VXHS UUID will
get picked up. I had, in a previous email, explained how we plan to
use the default UUID. In the regular case, the VxHS controller will
not allow access to the default UUID (non qemu-kvm) binaries, but it
may choose to grant temporary access to specific vdisks for these
binaries depending on the workflow.

Regards,
Ashish

[Qemu-devel] [PATCH v9 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/VeritasHyperScale/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v9 changelog:
(1) Dropped second argument to set_prog_path(). We will pick up the test
server location from the user's PATH env setting.

v8/v7 changelog:
(1) No changes.

v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index b6274be..318a81d 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -158,6 +158,7 @@ check options
 -nfstest nfs
 -archipelagotest archipelago
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -261,6 +262,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index 55527aa..c4b51b3 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 4befd86..9766a48 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -123,6 +123,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index a3d904f..1eb735e 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -89,6 +89,9 @@ else
 TEST_IMG=$TEST_DIR/t.$IMGFMT
 elif [ "$IMGPROTO" = "archipelago" ]; then
 TEST_IMG="archipelago:at.$IMGFMT"
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -175,6 +178,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
&"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -201,6 +210,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
2.5.5

[Qemu-devel] [PATCH v9 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/VeritasHyperScale/libqnio.git

Sample command line using JSON syntax:
./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
-k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---

v9 changelog:
(1) Fixes for all the review comments from v8. I have left the definition
of VXHS_UUID_DEF unchanged pending a better suggestion.
(2) qcow2 tests now pass on the vxhs test server.
(3) Packaging changes for libvxhs will be checked in to the git repo soon.
(4) I have not moved extern QemuUUID qemu_uuid to a separate header file.

v8 changelog:
(1) Security implementation for libqnio present in branch 'securify'.
Please use 'securify' branch for building libqnio and testing
with this patch.
(2) Renamed libqnio to libvxhs.
(3) Pass instance ID to libvxhs for SSL authentication.

v7 changelog:
(1) IO failover code has moved out to the libqnio library.
(2) Fixes for issues reported by Stefan on v6.
(3) Incorporated the QEMUBH patch provided by Stefan.
This is a replacement for the pipe mechanism used earlier.
(4) Fixes to the buffer overflows reported in libqnio.
(5) Input validations in vxhs.c to prevent any buffer overflows for 
arguments passed to libqnio.

v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  16 ++
 block/vxhs.c | 527 +++
 configure|  40 
 qapi/block-core.json |  20 +-
 5 files changed, 603 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index c6bd14e..75675b4 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -39,6 +40,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 0bc5c0a..f193079 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -110,3 +110,19 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error) "ctx is NULL: error %d"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filena

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Hi,

I am getting the following error with checkpatch.pl

ERROR: externs should be avoided in .c files
#78: FILE: block/vxhs.c:28:
+QemuUUID qemu_uuid __attribute__ ((weak));

Is there any way to get around this, or does it mean that I would have
to add a vxhs.h just for this one entry?

Thanks,
Ashish

Re: [Qemu-devel] [PATCH v7 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[ashish mittal]

Sorry, pressed the "send" button instead of "expand text" on the
previous email ...

On Mon, Feb 13, 2017 at 6:43 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Feb 07, 2017 at 08:18:14PM -0800, Ashish Mittal wrote:
>> diff --git a/tests/qemu-iotests/common.config 
>> b/tests/qemu-iotests/common.config
>> index f6384fb..c7a80c0 100644
>> --- a/tests/qemu-iotests/common.config
>> +++ b/tests/qemu-iotests/common.config
>> @@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
>>  export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
>>  fi
>>
>> +if [ -z "$QEMU_VXHS_PROG" ]; then
>> +export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
>
> Did you test this with /usr/local/bin/qnio_server?
>
> I think it will evaluate to QEMU_VXHS_PROG=/usr/local/bin when qnio_server
> isn't found in PATH.  You probably wanted /usr/local/bin/qnio_server instead.
>
> I suggest dropping the second argument completely and letting the user set 
> PATH
> themselves.  No existing set_prog_path caller uses the second argument.
>

You're right. Will drop the second argument.

> # $1 = prog to look for, $2* = default pathnames if not found in $PATH
> set_prog_path()
> {
> p=`command -v $1 2> /dev/null`
> if [ -n "$p" -a -x "$p" ]; then
> echo $p
> return 0
> fi
> p=$1
>
> shift
> for f; do
> if [ -x $f ]; then
> echo $f
> return 0
> fi
> done
>
> echo ""
> return 1
> }
>
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 3213765..06a3164 100644
>> --- a/tests/qemu-iotests/common.rc
>> +++ b/tests/qemu-iotests/common.rc
>> @@ -89,6 +89,9 @@ else
>>  TEST_IMG=$TEST_DIR/t.$IMGFMT
>>  elif [ "$IMGPROTO" = "archipelago" ]; then
>>  TEST_IMG="archipelago:at.$IMGFMT"
>> +elif [ "$IMGPROTO" = "vxhs" ]; then
>> +TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
>> +TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
>>  else
>>  TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
>>  fi
>> @@ -175,6 +178,12 @@ _make_test_img()
>>  eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  
>> $TEST_IMG_FILE &"
>>  sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
>>  fi
>> +
>> +# Start QNIO server on image directory for vxhs protocol
>> +if [ $IMGPROTO = "vxhs" ]; then
>> +eval "$QEMU_VXHS -d  $TEST_DIR &"
>> +sleep 1 # Wait for server to come up.
>
> This is a pre-existing problem and you don't need to fix it now:
>
> We should replace sleep 1 with a function that probes the TCP port until the
> connection can be established or a timeout is reached.  The netcat (nc) 
> utility
> is often used for this.
>
> sleep 1 is not reliable and may fail on a heavily loaded machine like the
> Travis-CI build machines that are used.

Will skip this one for now.

Thanks,
Ashish

Re: [Qemu-devel] [PATCH v7 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[ashish mittal]

On Mon, Feb 13, 2017 at 6:43 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Feb 07, 2017 at 08:18:14PM -0800, Ashish Mittal wrote:
>> diff --git a/tests/qemu-iotests/common.config 
>> b/tests/qemu-iotests/common.config
>> index f6384fb..c7a80c0 100644
>> --- a/tests/qemu-iotests/common.config
>> +++ b/tests/qemu-iotests/common.config
>> @@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
>>  export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
>>  fi
>>
>> +if [ -z "$QEMU_VXHS_PROG" ]; then
>> +export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
>
> Did you test this with /usr/local/bin/qnio_server?
>
> I think it will evaluate to QEMU_VXHS_PROG=/usr/local/bin when qnio_server
> isn't found in PATH.  You probably wanted /usr/local/bin/qnio_server instead.
>
> I suggest dropping the second argument completely and letting the user set 
> PATH
> themselves.  No existing set_prog_path caller uses the second argument.
>
> # $1 = prog to look for, $2* = default pathnames if not found in $PATH
> set_prog_path()
> {
> p=`command -v $1 2> /dev/null`
> if [ -n "$p" -a -x "$p" ]; then
> echo $p
> return 0
> fi
> p=$1
>
> shift
> for f; do
> if [ -x $f ]; then
> echo $f
> return 0
> fi
> done
>
> echo ""
> return 1
> }
>
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 3213765..06a3164 100644
>> --- a/tests/qemu-iotests/common.rc
>> +++ b/tests/qemu-iotests/common.rc
>> @@ -89,6 +89,9 @@ else
>>  TEST_IMG=$TEST_DIR/t.$IMGFMT
>>  elif [ "$IMGPROTO" = "archipelago" ]; then
>>  TEST_IMG="archipelago:at.$IMGFMT"
>> +elif [ "$IMGPROTO" = "vxhs" ]; then
>> +TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
>> +TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
>>  else
>>  TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
>>  fi
>> @@ -175,6 +178,12 @@ _make_test_img()
>>  eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  
>> $TEST_IMG_FILE &"
>>  sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
>>  fi
>> +
>> +# Start QNIO server on image directory for vxhs protocol
>> +if [ $IMGPROTO = "vxhs" ]; then
>> +eval "$QEMU_VXHS -d  $TEST_DIR &"
>> +sleep 1 # Wait for server to come up.
>
> This is a pre-existing problem and you don't need to fix it now:
>
> We should replace sleep 1 with a function that probes the TCP port until the
> connection can be established or a timeout is reached.  The netcat (nc) 
> utility
> is often used for this.
>
> sleep 1 is not reliable and may fail on a heavily loaded machine like the
> Travis-CI build machines that are used.

Re: [Qemu-devel] [PATCH v7 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Feb 13, 2017 at 6:57 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Feb 07, 2017 at 08:18:13PM -0800, Ashish Mittal wrote:
>> +static int vxhs_parse_uri(const char *filename, QDict *options)
>> +{
>> +URI *uri = NULL;
>> +char *hoststr, *portstr;
>> +char *port;
>> +int ret = 0;
>> +
>> +trace_vxhs_parse_uri_filename(filename);
>> +uri = uri_parse(filename);
>> +if (!uri || !uri->server || !uri->path) {
>> +uri_free(uri);
>> +return -EINVAL;
>> +}
>> +
>> +hoststr = g_strdup(VXHS_OPT_SERVER".host");
>> +qdict_put(options, hoststr, qstring_from_str(uri->server));
>> +g_free(hoststr);
>> +
>> +portstr = g_strdup(VXHS_OPT_SERVER".port");
>> +if (uri->port) {
>> +port = g_strdup_printf("%d", uri->port);
>> +qdict_put(options, portstr, qstring_from_str(port));
>> +g_free(port);
>> +}
>> +g_free(portstr);
>> +
>> +if (strstr(uri->path, "vxhs") == NULL) {
>> +qdict_put(options, "vdisk-id", qstring_from_str(uri->path));
>> +}
>> +
>> +trace_vxhs_parse_uri_hostinfo(1, uri->server, uri->port);
>
> What is the purpose of the first argument?
>

It used to be a placeholder for the host index, which is now only 1. I
will remove it.

>> +str = g_strdup_printf(VXHS_OPT_SERVER".");
>> +qdict_extract_subqdict(options, _options, str);
>> +
>> +/* Create opts info from runtime_tcp_opts list */
>> +tcp_opts = qemu_opts_create(_tcp_opts, NULL, 0, _abort);
>> +qemu_opts_absorb_qdict(tcp_opts, backing_options, _err);
>> +if (local_err) {
>> +qdict_del(backing_options, str);
>
> What is qdict_del(backing_options, VXHS_OPT_SERVER".") supposed to do?
> The same call is made further down too.
>

Per my understanding, qdict_del() is to free the 'server.' entries
within the subqdict.

qdict_extract_subqdict() allocates a subqdict and populates it with
the entries based on the pattern we pass. In this case 'server.'.

>> +qemu_opts_del(tcp_opts);
>> +ret = -EINVAL;
>> +goto out;
>> +}
>> +
>> +server_host_opt = qemu_opt_get(tcp_opts, VXHS_OPT_HOST);
>> +if (!server_host_opt) {
>> +error_setg(_err, QERR_MISSING_PARAMETER,
>> +   VXHS_OPT_SERVER"."VXHS_OPT_HOST);
>> +ret = -EINVAL;
>> +goto out;
>
> Missing qemu_opts_del(tcp_opts).
>

Will fix this!

>> +}
>> +
>> +if (strlen(server_host_opt) > MAXHOSTNAMELEN) {
>> +error_setg(errp, "server.host cannot be more than %d characters",
>> +   MAXHOSTNAMELEN);
>> +ret = -EINVAL;
>> +goto out;
>
> Missing qemu_opts_del(tcp_opts).
>

Will fix this!

>> @@ -5114,6 +5147,7 @@ echo "tcmalloc support  $tcmalloc"
>>  echo "jemalloc support  $jemalloc"
>>  echo "avx2 optimization $avx2_opt"
>>  echo "replication support $replication"
>> +echo "VxHS block device $vxhs"
>>
>>  if test "$sdl_too_old" = "yes"; then
>>  echo "-> Your SDL version is too old - please upgrade to have SDL support"
>> @@ -5729,6 +5763,12 @@ if test "$pthread_setname_np" = "yes" ; then
>>echo "CONFIG_PTHREAD_SETNAME_NP=y" >> $config_host_mak
>>  fi
>>
>> +if test "$vxhs" = "yes" ; then
>> +  echo "CONFIG_VXHS=y" >> $config_host_mak
>> +  echo "VXHS_CFLAGS=$vxhs_cflags" >> $config_host_mak
>
> Please drop this unused variable.

Will fix this!


Thanks,
Ashish

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Tue, Feb 14, 2017 at 12:51 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Thu, Feb 09, 2017 at 01:24:58AM -0800, ashish mittal wrote:
>> On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
>> > On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>> >> From: Ashish Mittal <ashish.mit...@veritas.com>
>> >>
>> >> Source code for the qnio library that this code loads can be downloaded 
>> >> from:
>> >> https://github.com/VeritasHyperScale/libqnio.git
>> >>
>> >> Sample command line using JSON syntax:
>> >> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
>> >> 0.0.0.0:0
>> >> -k en-us -vga cirrus -device 
>> >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> >> -msg timestamp=on
>> >> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> >> "server":{"host":"172.172.17.4","port":""}}'
>> >>
>> >> Sample command line using URI syntax:
>> >> qemu-img convert -f raw -O raw -n
>> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> >> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>> >>
>
> [...]
>
>> >> +#define VXHS_OPT_FILENAME   "filename"
>> >> +#define VXHS_OPT_VDISK_ID   "vdisk-id"
>> >> +#define VXHS_OPT_SERVER "server"
>> >> +#define VXHS_OPT_HOST   "host"
>> >> +#define VXHS_OPT_PORT   "port"
>> >> +#define VXHS_UUID_DEF "12345678-1234-1234-1234-123456789012"
>> >
>> > What is this?  It is not a valid UUID; is the value significant?
>> >
>>
>> This value gets passed to libvxhs for binaries like qemu-io, qemu-img
>> that do not have an Instance ID. We can use this default ID to control
>> access to specific vdisks by these binaries. qemu-kvm will pass the
>> actual instance ID, and therefore will not use this default value.
>>
>> Will reply to other queries soon.
>>
>
> If you are going to call it a UUID, it should adhere to the RFC 4122 spec.
> You can easily generate a compliant UUID with uuidgen.  However:
>
> Can you explain more about how you are using this to control access by
> qemu-img and qemu-io?  Looking at libqnio, it looks like this is used to
> determine at runtime which TLS certs to use based off of a
> pathname/filename, which is then how I presume you are controlling access.
> See Daniel's email regarding TLS certificates.
>

(1) The default behavior would be to disallow access to any vdisks by
the non qemu-kvm binaries. qemu-kvm would use the actual instance ID
for authentication.
(2) Depending on the workflow, HyperScale controller can choose to
grant *temporary* access to specific vdisks by qemu-img, qemu-io
binaries (identified by the default VXHS_UUID_DEF above).
(3) This information, described in #2, would be communicated by the
HyperScale controller to the actual proprietary VxHS server (running
on each compute) that does the authentication/SSL.
(4) The HyperScale controller, in this way, can grant/revoke access
for specific vdisks not just to clients with VXHS_UUID_DEF instance
ID, but also the actual VM instances.

> [...]
>
>> >> +
>> >> +static void bdrv_vxhs_init(void)
>> >> +{
>> >> +char out[37];
>
> Additional point: this should be sized as UUID_FMT_LEN + 1, not 37, but I
> suspect this code is changing anyways.
>
>> >> +
>> >> +if (qemu_uuid_is_null(_uuid)) {
>> >> +lib_init_failed = iio_init(QNIO_VERSION, vxhs_iio_callback, 
>> >> VXHS_UUID_DEF);
>> >> +} else {
>> >> +qemu_uuid_unparse(_uuid, out);
>> >> +lib_init_failed = iio_init(QNIO_VERSION, vxhs_iio_callback, out);
>> >> +}
>> >> +
>> >
>> > [1]
>> >
>> > Can you explain what is going on here with the qemu_uuid check?
>> >

(1) qemu_uuid_is_null(_uuid) is true for qemu-io, qemu-img that
do not define a instance ID. We end up using the default VXHS_UUID_DEF
ID for them, and authenticating them as described above.

(2) For the other case 'else', we convert the uuid to a char * using
qemu_uuid_unparse(), and pass the resulting char * uuid in variable
'out' to libvxhs.

>> >
>> > You also can't do this here.  This init function is just to register the
>> > driver (e.g. populate the BlockDriver list).  You shouldn't be doing
>> > anything other than the bdrv_register() call here.
>> >
>> > Since you want to run this iio_init only once, I would recommend doing it 
>> > in
>> > the vxhs_open() call, and using a ref counter.  That way, you can also call
>> > iio_fini() to finish releasing resources once the last device is closed.
>> >
>> > This was actually a suggestion I had before, which you then incorporated
>> > into v6, but it appears all the refcnt code has gone away for v7/v8.
>> >
>> >
>> >> +bdrv_register(_vxhs);
>> >> +}
>> >> +

Will consider this in the next patch.

Regards,
Ashish

Re: [Qemu-devel] [PATCH v6 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[ashish mittal]

On Tue, Feb 14, 2017 at 10:12 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Mon, Nov 07, 2016 at 04:59:45PM -0800, Ashish Mittal wrote:
>> These changes use a vxhs test server that is a part of the following
>> repository:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v6 changelog:
>> (1) Added iotests for VxHS block device.
>>
>>  tests/qemu-iotests/common|  6 ++
>>  tests/qemu-iotests/common.config | 13 +
>>  tests/qemu-iotests/common.filter |  1 +
>>  tests/qemu-iotests/common.rc | 19 +++
>>  4 files changed, 39 insertions(+)
>>
>
> [...]
>
>> diff --git a/tests/qemu-iotests/common.filter 
>> b/tests/qemu-iotests/common.filter
>> index 240ed06..a8a4d0e 100644
>> --- a/tests/qemu-iotests/common.filter
>> +++ b/tests/qemu-iotests/common.filter
>> @@ -123,6 +123,7 @@ _filter_img_info()
>>  -e "s#$TEST_DIR#TEST_DIR#g" \
>>  -e "s#$IMGFMT#IMGFMT#g" \
>>  -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
>> +-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
>>  -e "/encrypted: yes/d" \
>>  -e "/cluster_size: [0-9]\\+/d" \
>>  -e "/table_size: [0-9]\\+/d" \
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 3213765..06a3164 100644
>> --- a/tests/qemu-iotests/common.rc
>> +++ b/tests/qemu-iotests/common.rc
>> @@ -89,6 +89,9 @@ else
>>  TEST_IMG=$TEST_DIR/t.$IMGFMT
>>  elif [ "$IMGPROTO" = "archipelago" ]; then
>>  TEST_IMG="archipelago:at.$IMGFMT"
>> +elif [ "$IMGPROTO" = "vxhs" ]; then
>> +TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
>> +TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
>>  else
>>  TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
>>  fi
>> @@ -175,6 +178,12 @@ _make_test_img()
>>  eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  
>> $TEST_IMG_FILE &"
>>  sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
>>  fi
>> +
>> +# Start QNIO server on image directory for vxhs protocol
>> +if [ $IMGPROTO = "vxhs" ]; then
>> +eval "$QEMU_VXHS -d  $TEST_DIR &"
>
> So I spoke too soon about tests passing, but it is not really your fault :)
>
> After rebasing to master, there is a new test 174 that now hangs (and hangs
> for nbd, as well).  This is because the test is piping the results of
> _make_test_image to sed to filter it.
>
> This line should redirect stdout to /dev/null, so that the pipe does not
> need to wait until process completion:
>
> eval "$QEMU_VXHS -d  $TEST_DIR > /dev/null &"
>
>

Will make this change in the next series. Thanks again!

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Thu, Feb 9, 2017 at 10:45 AM, ashish mittal <ashmit...@gmail.com> wrote:
> On Thu, Feb 9, 2017 at 10:08 AM, ashish mittal <ashmit...@gmail.com> wrote:
>> On Thu, Feb 9, 2017 at 8:50 AM, Jeff Cody <jc...@redhat.com> wrote:
>>> On Thu, Feb 09, 2017 at 08:14:38AM -0800, ashish mittal wrote:
>>>> On Thu, Feb 9, 2017 at 6:32 AM, Jeff Cody <jc...@redhat.com> wrote:
>>>> > On Thu, Feb 09, 2017 at 01:24:58AM -0800, ashish mittal wrote:
>>>> >> On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
>>>> >> > On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>>>> >> >> From: Ashish Mittal <ashish.mit...@veritas.com>
>>>> >> >>
>>>> >> >> Source code for the qnio library that this code loads can be 
>>>> >> >> downloaded from:
>>>> >> >> https://github.com/VeritasHyperScale/libqnio.git
>>>> >> >>
>>>> >> >> Sample command line using JSON syntax:
>>>> >> >> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
>>>> >> >> 0.0.0.0:0
>>>> >> >> -k en-us -vga cirrus -device 
>>>> >> >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>>>> >> >> -msg timestamp=on
>>>> >> >> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>>>> >> >> "server":{"host":"172.172.17.4","port":""}}'
>>>> >> >>
>>>> >> >> Sample command line using URI syntax:
>>>> >> >> qemu-img convert -f raw -O raw -n
>>>> >> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>>>> >> >> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>>> >> >>
>>>> >> >
>>>> >> > I don't know if I am using the qnio_server test server correctly or 
>>>> >> > not, but
>>>> >> > when I run qemu-io from the command line I get an i/o error.  When I 
>>>> >> > run the
>>>> >> > qemu-iotests, I get a segfault.
>>>> >> >
>>>> >> > Were you able to run qemu-iotests with these patches?
>>>> >> >
>>>> >> > Here is how I am invoking qnio_server:
>>>> >> >
>>>> >> > # qnio_server  -d 
>>>> >> > /home/jcody/work/redhat/upstream/qemu-kvm/tests/qemu-iotests/scratch 
>>>> >> > -v
>>>> >> >
>>>> >> >
>>>> >>
>>>> >> I ran full qemu-iotests and qemu-io manually with the test server on
>>>> >> version 7 patches. Ran qemu-io manually with the test server on
>>>> >> version 8, but the libvxhs code is undergoing a lot of checkins. Will
>>>> >> test again tomorrow and get back.
>>>> >>
>>>> >
>>>> > Does my invocation above look correct, for running the qemu-iotests?
>>>>
>>>> qemu-iotest starts the server internally. The server does not have to
>>>> be started explicitly before running qemu-iotest. The server must not
>>>> be running before you begin the test  because then the tests would
>>>> find port  busy.
>>>> Running make install on libvxhs copies the test server to the location
>>>> where the qemu-iotests expect to find it.
>>>
>>>
>>> OK, thanks.  I tried that too, and I also tried against the branch
>>> "ashish_securify_changes" for libqnio.  I still have qemu-iotests giving me
>>> a segfault.
>>>
>>>
>>
>> Qemu patch v7 and v8 do not have many changes. I did successfully run
>> iotests with v7 and libqnio master. I'm guessing there could be
>> something in the 'securify' library branch that's causing this. I will
>> rebuild and retest.
>
> Confirmed that there is a problem and qemu-iotest is segfaulting. Will
> get back with a fix.

Checked in some changes to libqnio. Could you please try it again with
the latest 'securify' branch?
Thanks!

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Thu, Feb 9, 2017 at 10:08 AM, ashish mittal <ashmit...@gmail.com> wrote:
> On Thu, Feb 9, 2017 at 8:50 AM, Jeff Cody <jc...@redhat.com> wrote:
>> On Thu, Feb 09, 2017 at 08:14:38AM -0800, ashish mittal wrote:
>>> On Thu, Feb 9, 2017 at 6:32 AM, Jeff Cody <jc...@redhat.com> wrote:
>>> > On Thu, Feb 09, 2017 at 01:24:58AM -0800, ashish mittal wrote:
>>> >> On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
>>> >> > On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>>> >> >> From: Ashish Mittal <ashish.mit...@veritas.com>
>>> >> >>
>>> >> >> Source code for the qnio library that this code loads can be 
>>> >> >> downloaded from:
>>> >> >> https://github.com/VeritasHyperScale/libqnio.git
>>> >> >>
>>> >> >> Sample command line using JSON syntax:
>>> >> >> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
>>> >> >> 0.0.0.0:0
>>> >> >> -k en-us -vga cirrus -device 
>>> >> >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>>> >> >> -msg timestamp=on
>>> >> >> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>>> >> >> "server":{"host":"172.172.17.4","port":""}}'
>>> >> >>
>>> >> >> Sample command line using URI syntax:
>>> >> >> qemu-img convert -f raw -O raw -n
>>> >> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>>> >> >> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>> >> >>
>>> >> >
>>> >> > I don't know if I am using the qnio_server test server correctly or 
>>> >> > not, but
>>> >> > when I run qemu-io from the command line I get an i/o error.  When I 
>>> >> > run the
>>> >> > qemu-iotests, I get a segfault.
>>> >> >
>>> >> > Were you able to run qemu-iotests with these patches?
>>> >> >
>>> >> > Here is how I am invoking qnio_server:
>>> >> >
>>> >> > # qnio_server  -d 
>>> >> > /home/jcody/work/redhat/upstream/qemu-kvm/tests/qemu-iotests/scratch -v
>>> >> >
>>> >> >
>>> >>
>>> >> I ran full qemu-iotests and qemu-io manually with the test server on
>>> >> version 7 patches. Ran qemu-io manually with the test server on
>>> >> version 8, but the libvxhs code is undergoing a lot of checkins. Will
>>> >> test again tomorrow and get back.
>>> >>
>>> >
>>> > Does my invocation above look correct, for running the qemu-iotests?
>>>
>>> qemu-iotest starts the server internally. The server does not have to
>>> be started explicitly before running qemu-iotest. The server must not
>>> be running before you begin the test  because then the tests would
>>> find port  busy.
>>> Running make install on libvxhs copies the test server to the location
>>> where the qemu-iotests expect to find it.
>>
>>
>> OK, thanks.  I tried that too, and I also tried against the branch
>> "ashish_securify_changes" for libqnio.  I still have qemu-iotests giving me
>> a segfault.
>>
>>
>
> Qemu patch v7 and v8 do not have many changes. I did successfully run
> iotests with v7 and libqnio master. I'm guessing there could be
> something in the 'securify' library branch that's causing this. I will
> rebuild and retest.

Confirmed that there is a problem and qemu-iotest is segfaulting. Will
get back with a fix.

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Thu, Feb 9, 2017 at 8:50 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Thu, Feb 09, 2017 at 08:14:38AM -0800, ashish mittal wrote:
>> On Thu, Feb 9, 2017 at 6:32 AM, Jeff Cody <jc...@redhat.com> wrote:
>> > On Thu, Feb 09, 2017 at 01:24:58AM -0800, ashish mittal wrote:
>> >> On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
>> >> > On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>> >> >> From: Ashish Mittal <ashish.mit...@veritas.com>
>> >> >>
>> >> >> Source code for the qnio library that this code loads can be 
>> >> >> downloaded from:
>> >> >> https://github.com/VeritasHyperScale/libqnio.git
>> >> >>
>> >> >> Sample command line using JSON syntax:
>> >> >> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
>> >> >> 0.0.0.0:0
>> >> >> -k en-us -vga cirrus -device 
>> >> >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> >> >> -msg timestamp=on
>> >> >> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> >> >> "server":{"host":"172.172.17.4","port":""}}'
>> >> >>
>> >> >> Sample command line using URI syntax:
>> >> >> qemu-img convert -f raw -O raw -n
>> >> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> >> >> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>> >> >>
>> >> >
>> >> > I don't know if I am using the qnio_server test server correctly or 
>> >> > not, but
>> >> > when I run qemu-io from the command line I get an i/o error.  When I 
>> >> > run the
>> >> > qemu-iotests, I get a segfault.
>> >> >
>> >> > Were you able to run qemu-iotests with these patches?
>> >> >
>> >> > Here is how I am invoking qnio_server:
>> >> >
>> >> > # qnio_server  -d 
>> >> > /home/jcody/work/redhat/upstream/qemu-kvm/tests/qemu-iotests/scratch -v
>> >> >
>> >> >
>> >>
>> >> I ran full qemu-iotests and qemu-io manually with the test server on
>> >> version 7 patches. Ran qemu-io manually with the test server on
>> >> version 8, but the libvxhs code is undergoing a lot of checkins. Will
>> >> test again tomorrow and get back.
>> >>
>> >
>> > Does my invocation above look correct, for running the qemu-iotests?
>>
>> qemu-iotest starts the server internally. The server does not have to
>> be started explicitly before running qemu-iotest. The server must not
>> be running before you begin the test  because then the tests would
>> find port  busy.
>> Running make install on libvxhs copies the test server to the location
>> where the qemu-iotests expect to find it.
>
>
> OK, thanks.  I tried that too, and I also tried against the branch
> "ashish_securify_changes" for libqnio.  I still have qemu-iotests giving me
> a segfault.
>
>

Qemu patch v7 and v8 do not have many changes. I did successfully run
iotests with v7 and libqnio master. I'm guessing there could be
something in the 'securify' library branch that's causing this. I will
rebuild and retest.

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Thu, Feb 9, 2017 at 6:32 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Thu, Feb 09, 2017 at 01:24:58AM -0800, ashish mittal wrote:
>> On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
>> > On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>> >> From: Ashish Mittal <ashish.mit...@veritas.com>
>> >>
>> >> Source code for the qnio library that this code loads can be downloaded 
>> >> from:
>> >> https://github.com/VeritasHyperScale/libqnio.git
>> >>
>> >> Sample command line using JSON syntax:
>> >> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
>> >> 0.0.0.0:0
>> >> -k en-us -vga cirrus -device 
>> >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> >> -msg timestamp=on
>> >> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> >> "server":{"host":"172.172.17.4","port":""}}'
>> >>
>> >> Sample command line using URI syntax:
>> >> qemu-img convert -f raw -O raw -n
>> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> >> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>> >>
>> >
>> > I don't know if I am using the qnio_server test server correctly or not, 
>> > but
>> > when I run qemu-io from the command line I get an i/o error.  When I run 
>> > the
>> > qemu-iotests, I get a segfault.
>> >
>> > Were you able to run qemu-iotests with these patches?
>> >
>> > Here is how I am invoking qnio_server:
>> >
>> > # qnio_server  -d 
>> > /home/jcody/work/redhat/upstream/qemu-kvm/tests/qemu-iotests/scratch -v
>> >
>> >
>>
>> I ran full qemu-iotests and qemu-io manually with the test server on
>> version 7 patches. Ran qemu-io manually with the test server on
>> version 8, but the libvxhs code is undergoing a lot of checkins. Will
>> test again tomorrow and get back.
>>
>
> Does my invocation above look correct, for running the qemu-iotests?

qemu-iotest starts the server internally. The server does not have to
be started explicitly before running qemu-iotest. The server must not
be running before you begin the test  because then the tests would
find port  busy.
Running make install on libvxhs copies the test server to the location
where the qemu-iotests expect to find it.

Re: [Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Feb 8, 2017 at 10:29 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Wed, Feb 08, 2017 at 09:23:33PM -0800, Ashish Mittal wrote:
>> From: Ashish Mittal <ashish.mit...@veritas.com>
>>
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/VeritasHyperScale/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
>> -k en-us -vga cirrus -device 
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>
> I don't know if I am using the qnio_server test server correctly or not, but
> when I run qemu-io from the command line I get an i/o error.  When I run the
> qemu-iotests, I get a segfault.
>
> Were you able to run qemu-iotests with these patches?
>
> Here is how I am invoking qnio_server:
>
> # qnio_server  -d 
> /home/jcody/work/redhat/upstream/qemu-kvm/tests/qemu-iotests/scratch -v
>
>

I ran full qemu-iotests and qemu-io manually with the test server on
version 7 patches. Ran qemu-io manually with the test server on
version 8, but the libvxhs code is undergoing a lot of checkins. Will
test again tomorrow and get back.

>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> TODO:
>> (1) valgrind report to follow soon.
>>
>> v8 changelog:
>> (1) Security implementation for libqnio present in branch 'securify'.
>> Please use 'securify' branch for building libqnio and testing
>> with this patch.
>> (2) Renamed libqnio to libvxhs.
>> (3) Pass instance ID to libvxhs for SSL authentication.
>>
>> v7 changelog:
>> (1) IO failover code has moved out to the libqnio library.
>> (2) Fixes for issues reported by Stefan on v6.
>> (3) Incorporated the QEMUBH patch provided by Stefan.
>> This is a replacement for the pipe mechanism used earlier.
>> (4) Fixes to the buffer overflows reported in libqnio.
>> (5) Input validations in vxhs.c to prevent any buffer overflows for
>> arguments passed to libqnio.
>>
>> v6 changelog:
>> (1) Added qemu-iotests for VxHS as a new patch in the series.
>> (2) Replaced release version from 2.8 to 2.9 in block-core.json.
>>
>> v5 changelog:
>> (1) Incorporated v4 review comments.
>>
>> v4 changelog:
>> (1) Incorporated v3 review comments on QAPI changes.
>> (2) Added refcounting for device open/close.
>> Free library resources on last device close.
>>
>> v3 changelog:
>> (1) Added QAPI schema for the VxHS driver.
>>
>> v2 changelog:
>> (1) Changes done in response to v1 comments.
>>
>>  block/Makefile.objs  |   2 +
>>  block/trace-events   |  16 ++
>>  block/vxhs.c | 499 
>> +++
>>  configure|  41 +
>>  qapi/block-core.json |  20 ++-
>>  5 files changed, 576 insertions(+), 2 deletions(-)
>>  create mode 100644 block/vxhs.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index c6bd14e..75675b4 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
>>  block-obj-$(CONFIG_CURL) += curl.o
>>  block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>> @@ -39,6 +40,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
>>  rbd.o-libs := $(RBD_LIBS)
>>  gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
>>  gluster.o-libs := $(GLUSTERFS_LIBS)
>> +vxhs.o-libs:= $(VXHS_LIBS)
>>  ssh.o-cflags   := $(LIBSSH2_CFLAGS)
>>  ssh.o-libs := $(LIBSSH2_LIBS)
>>  archipelago.o-libs := $(ARCHIPELAGO_LIBS)
>> diff --git a/block/trace-events b/block/trace-events
>> index 0bc5c0a..d5eca93 100644
>> --- a/block/trace-events
>> +++ b/block/trace-events
>> @@ -110,3 +110,19 @@ qed_aio_write_data(void *s, void *acb, int ret, 
>>

[Qemu-devel] [PATCH v8 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

From: Ashish Mittal <ashish.mit...@veritas.com>

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/VeritasHyperScale/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v8/v7 changelog:
(1) No changes.

v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index b6274be..318a81d 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -158,6 +158,7 @@ check options
 -nfstest nfs
 -archipelagotest archipelago
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -261,6 +262,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index f6384fb..c7a80c0 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 240ed06..a8a4d0e 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -123,6 +123,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 3213765..06a3164 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -89,6 +89,9 @@ else
 TEST_IMG=$TEST_DIR/t.$IMGFMT
 elif [ "$IMGPROTO" = "archipelago" ]; then
 TEST_IMG="archipelago:at.$IMGFMT"
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -175,6 +178,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
&"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -201,6 +210,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
1.8.3.1

[Qemu-devel] [PATCH v8 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

From: Ashish Mittal <ashish.mit...@veritas.com>

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/VeritasHyperScale/libqnio.git

Sample command line using JSON syntax:
./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
-k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
TODO:
(1) valgrind report to follow soon.

v8 changelog:
(1) Security implementation for libqnio present in branch 'securify'.
Please use 'securify' branch for building libqnio and testing
with this patch.
(2) Renamed libqnio to libvxhs.
(3) Pass instance ID to libvxhs for SSL authentication.

v7 changelog:
(1) IO failover code has moved out to the libqnio library.
(2) Fixes for issues reported by Stefan on v6.
(3) Incorporated the QEMUBH patch provided by Stefan.
This is a replacement for the pipe mechanism used earlier.
(4) Fixes to the buffer overflows reported in libqnio.
(5) Input validations in vxhs.c to prevent any buffer overflows for 
arguments passed to libqnio.

v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  16 ++
 block/vxhs.c | 499 +++
 configure|  41 +
 qapi/block-core.json |  20 ++-
 5 files changed, 576 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index c6bd14e..75675b4 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -39,6 +40,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 0bc5c0a..d5eca93 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -110,3 +110,19 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error) "ctx is NULL: error %d"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk-id from json %

Re: [Qemu-devel] [PATCH v7 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Feb 8, 2017 at 2:35 PM, Ketan Nilangekar
<ketan.nilange...@veritas.com> wrote:
>
>
> On 2/8/17, 2:21 PM, "Jeff Cody" <jc...@redhat.com> wrote:
>
> On Tue, Feb 07, 2017 at 08:18:13PM -0800, Ashish Mittal wrote:
> > From: Ashish Mittal <ashish.mit...@veritas.com>
> >
> > Source code for the qnio library that this code loads can be downloaded 
> from:
> > https://github.com/VeritasHyperScale/libqnio.git
> >
>
> What branch does this use for libqnio?  I was under the impression that v7
> of this series was going to work and test against the latest SSL branch
> upstream, but that doesn't appear to be the case - there are API 
> differences
> and this code can't compile against the 'securify' branch on
> https://github.com/VeritasHyperScale/libqnio.git
>
> So some questions and thoughts around this:
>
> 1. If the target libqnio library version is essentially what it is on the
>libqnio 'securify' branch, then obviously this patch series can't be 
> in a
>merge-ready state, since its API usage is (at the very least) stale;
>
> 2. It is getting very late (compared to feature freeze) to have patches 
> that
>can't be tested or compiled against the intended libqnio release.  Do 
> you
>know exactly what the API differences are and the impact this has on 
> this
>series?  If so, can you list what that is, and/or maybe post a quick 
> v8?
>
> [Ketan]
> There is only one API (iio_init) in libqnio that that was under review based 
> on comments from Stefan.
> Iio_init is now changed to pass an instance id. This instance Id will be used 
> to lookup local credentials for the instance. This was also supposed to be 
> used for extra level of authorization on VxHS server. But that seems 
> redundant based on Stefan's inputs.
> So we have decided retan the API change that is proposed in 'securify' branch 
> and have the QEMU driver use that.
> Change suggested by Stefan will be done without changing the API in the 
> 'securify' branch.
>
> Ashish, please post a v8 patch based on securify branch so Jeff can complete 
> the review.
>

Qemu changes to work with the libqnio securify branch are not very
many - only a few lines. The configure script has some changes to
account for the library name change (libvxhs now) and the new
iio_init() interface. I will post another patch.

> 3. Or, perhaps the API on the 'securify' libqnio branch is not stable.  If
>that is the case, I guess there is no choice but to wait for that to
>stabilize before posting a v8+ patch series that uses the API.
>
>
> I'll go ahead and review this series on its own merits against the libqnio
> master branch, but I wanted to point that out now in case you are able to 
> do
> anything in point 2.  The rest of my review will be in subsequent 
> email(s).
>
>
> -Jeff
>
> > Sample command line using JSON syntax:
> > ./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 
> 0.0.0.0:0
> > -k en-us -vga cirrus -device 
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
> > -msg timestamp=on
> > 
> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
> > "server":{"host":"172.172.17.4","port":""}}'
> >
> > Sample command line using URI syntax:
> > qemu-img convert -f raw -O raw -n
> > /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
> > vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
> >
> > Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
> > ---
> > TODO:
> > (1) valgrind report to follow soon.
> > (2) Security implementation for libqnio present in branch "securify".
> > QEMU patch will use this once it gets merged to libqnio master.
> > Please use master for building libqnio and testing with this patch.
> > (3) Rename libqnio to libvxhs. Changes ready in branch "securify".
> >
> > v7 changelog:
> > (1) IO failover code has moved out to the libqnio library.
> > (2) Fixes for issues reported by Stefan on v6.
> > (3) Incorporated the QEMUBH patch provided by Stefan.
> > This is a replacement for the pipe mechanism used earlier.
> > (4) Fixes to the buffer overflows reported in libqnio.
> > (5) Input validations in vxhs.c to prevent any buffer overflows for
> > arguments passed to lib

Re: [Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

We have retained the original syntax in v7 per the following
suggestion on libvirt thread -

>> That is correct. Above syntax would also work for us. I will pose this
>> suggestion to the qemu community and update with their response.
>>

It's not that important... I was looking for a simplification and
generation of only what's required. You can continue using the server
syntax - perhaps just leave a note/comment in the code indicating the
decision point and move on.


On Thu, Feb 2, 2017 at 2:08 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Jan 31, 2017 at 05:55:47PM -0800, ashish mittal wrote:
>> Hi,
>>
>> There has been some work going on on the VxHS libvirt patches and we
>> are making good progress.
>>
>> A suggestion has been made on the libvirt community to check if we can
>> change the qemu VxHS device specification syntax as follows.
>>
>> Replace
>>
>> (a) +file.server.host=192.168.0.1,file.server.port=
>>
>> with
>>
>> (b) +file.host=192.168.0.1,file.port=
>>
>> The reasoning being that since we have only one host (true as the
>> failover is now being handled completely/transparently) within the
>> libqnio library), the "server" part is redundant.
>
> Sounds good to me.
>
> Stefan

[Qemu-devel] [PATCH v7 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

From: Ashish Mittal <ashish.mit...@veritas.com>

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/VeritasHyperScale/libqnio.git

Sample command line using JSON syntax:
./x86_64-softmmu/qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0
-k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
TODO:
(1) valgrind report to follow soon.
(2) Security implementation for libqnio present in branch "securify".
QEMU patch will use this once it gets merged to libqnio master.
Please use master for building libqnio and testing with this patch.
(3) Rename libqnio to libvxhs. Changes ready in branch "securify".

v7 changelog:
(1) IO failover code has moved out to the libqnio library.
(2) Fixes for issues reported by Stefan on v6.
(3) Incorporated the QEMUBH patch provided by Stefan.
This is a replacement for the pipe mechanism used earlier.
(4) Fixes to the buffer overflows reported in libqnio.
(5) Input validations in vxhs.c to prevent any buffer overflows for 
arguments passed to libqnio.

v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  16 ++
 block/vxhs.c | 488 +++
 configure|  40 +
 qapi/block-core.json |  20 ++-
 5 files changed, 564 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index c6bd14e..75675b4 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -19,6 +19,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -39,6 +40,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 0bc5c0a..d5eca93 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -110,3 +110,19 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error) "ctx is NULL: error %d"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vd

[Qemu-devel] [PATCH v7 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

From: Ashish Mittal <ashish.mit...@veritas.com>

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/VeritasHyperScale/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v7 changelog:
(1) No changes.

v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index b6274be..318a81d 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -158,6 +158,7 @@ check options
 -nfstest nfs
 -archipelagotest archipelago
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -261,6 +262,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index f6384fb..c7a80c0 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 240ed06..a8a4d0e 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -123,6 +123,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 3213765..06a3164 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -89,6 +89,9 @@ else
 TEST_IMG=$TEST_DIR/t.$IMGFMT
 elif [ "$IMGPROTO" = "archipelago" ]; then
 TEST_IMG="archipelago:at.$IMGFMT"
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -175,6 +178,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
&"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -201,6 +210,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Stefan,

Patch V7, which I plan to submit soon,  will have fixes to all the
issues you pointed out in reply to -
[PATCH v6 1/2] block/vxhs.c: Add support for a new block device type
called "vxhs"


It will also have the QEMUBH patch you provided.

Thanks,
Ashish


On Tue, Nov 15, 2016 at 12:51 PM, ashish mittal <ashmit...@gmail.com> wrote:
> Thank you. Will work with this.
>
> On Tue, Nov 15, 2016 at 12:48 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
>> On Tue, Nov 15, 2016 at 11:02:34AM -0800, ashish mittal wrote:
>>> I had replied to the QEMUBH suggestion in the email below.
>>>
>>> Regards,
>>> Ashish
>>>
>>> On Tue, Aug 23, 2016 at 3:22 PM, ashish mittal <ashmit...@gmail.com> wrote:
>>> > Thanks Stefan, I will look at block/quorum.c.
>>> >
>>> > I have sent V4 of the patch with a reworked parsing logic for both
>>> > JSON and URI. Both of them are quite compact now.
>>> >
>>> > URI parsing now follows the suggestion given by Kevin.
>>> > //
>>> > However, you should use the proper interfaces to implement this, which
>>> > is .bdrv_parse_filename(). This is a function that gets a string and
>>> > converts it into a QDict, which is then passed to .bdrv_open(). So it
>>> > uses exactly the same code path in .bdrv_open() as if used directly with
>>> > QAPI.
>>> > //
>>> >
>>> > Additionally, I have fixed all the issues pointed out by you on V1 of
>>> > the patch. The only change I haven't done is to replace pipes with
>>> > QEMUBH. I am hoping this will not hold up the patch from being
>>> > accepted, and I can make this transition later with proper dev and
>>> > testing.
>>
>> Sorry, I forgot about this email.  I still think the QEMUBH approach
>> makes sense.  Please try the following patch.
>>
>> I have compiled but not run it.  You are welcome to squash it into your
>> next patch.
>>
>> The following is Signed-off-by: Stefan Hajnoczi <stefa...@redhat.com>.
>>
>> ---8<---
>>
>> diff --git a/block/vxhs.c b/block/vxhs.c
>> index 8913e8f..22fd989 100644
>> --- a/block/vxhs.c
>> +++ b/block/vxhs.c
>> @@ -60,9 +60,6 @@ typedef struct VXHSvDiskHostsInfo {
>>   * Structure per vDisk maintained for state
>>   */
>>  typedef struct BDRVVXHSState {
>> -int fds[2];
>> -int event_reader_pos;
>> -VXHSAIOCB *qnio_event_acb;
>>  VXHSvDiskHostsInfo vdisk_hostinfo; /* Per host info */
>>  char *vdisk_guid;
>>  } BDRVVXHSState;
>> @@ -73,12 +70,33 @@ static QNIOLibState qniolib;
>>  /* vdisk prefix to pass to qnio */
>>  static const char vdisk_prefix[] = "/dev/of/vdisk";
>>
>> +static void vxhs_complete_aio_bh(void *opaque)
>> +{
>> +VXHSAIOCB *acb = opaque;
>> +BlockCompletionFunc *cb = acb->common.cb;
>> +void *cb_opaque = acb->common.opaque;
>> +int ret = 0;
>> +
>> +if (acb->err != 0) {
>> +trace_vxhs_complete_aio(acb, acb->err);
>> +/*
>> + * We mask all the IO errors generically as EIO for upper layers
>> + * Right now our IO Manager uses non standard error codes. Instead
>> + * of confusing upper layers with incorrect interpretation we are
>> + * doing this workaround.
>> + */
>> +ret = (-EIO);
>> +}
>> +
>> +qemu_aio_unref(acb);
>> +cb(cb_opaque, ret);
>> +}
>> +
>> +/* Called from a libqnio thread */
>>  static void vxhs_iio_callback(int32_t rfd, uint32_t reason, void *ctx,
>>uint32_t error, uint32_t opcode)
>>  {
>>  VXHSAIOCB *acb = NULL;
>> -BDRVVXHSState *s = NULL;
>> -ssize_t ret;
>>
>>  switch (opcode) {
>>  case IRP_READ_REQUEST:
>> @@ -91,7 +109,6 @@ static void vxhs_iio_callback(int32_t rfd, uint32_t 
>> reason, void *ctx,
>>   */
>>  if (ctx) {
>>  acb = ctx;
>> -s = acb->common.bs->opaque;
>>  } else {
>>  trace_vxhs_iio_callback(error, reason);
>>  goto out;
>> @@ -104,8 +121,8 @@ static void vxhs_iio_callback(int32_t rfd, uint32_t 
>> reason, void *ctx,
>>  trace_vxhs_iio_callback(error, reason);
>>  }
>>
>> -ret = qemu_write_full(s->fds[VDISK_FD_WRITE], , sizeof(acb));
>>

Re: [Qemu-devel] [PATCH v6 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[ashish mittal]

I plan to submit v7 soon, therefore going over any unanswered emails
at this time.

On Tue, Nov 8, 2016 at 12:44 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Mon, Nov 07, 2016 at 04:59:45PM -0800, Ashish Mittal wrote:
>> These changes use a vxhs test server that is a part of the following
>> repository:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v6 changelog:
>> (1) Added iotests for VxHS block device.
>>
>>  tests/qemu-iotests/common|  6 ++
>>  tests/qemu-iotests/common.config | 13 +
>>  tests/qemu-iotests/common.filter |  1 +
>>  tests/qemu-iotests/common.rc | 19 +++
>>  4 files changed, 39 insertions(+)
>>
>> diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
>> index d60ea2c..41430d8 100644
>> --- a/tests/qemu-iotests/common
>> +++ b/tests/qemu-iotests/common
>
> When using raw format, I was able to run the test successfully for all
> supported test cases (26 of them).
>
> With qcow2, they fail - but not the fault of this patch, I think; but
> rather, the fault of the test server.  Can qnio_server be modified so that
> it does not work on just raw files?
>
>

VxHS supports and uses only the raw format.

>
>> @@ -158,6 +158,7 @@ check options
>>  -nfstest nfs
>>  -archipelagotest archipelago
>>  -luks   test luks
>> +-vxhs   test vxhs
>>  -xdiff  graphical mode diff
>>  -nocacheuse O_DIRECT on backing file
>>  -misalign   misalign memory allocations
>> @@ -261,6 +262,11 @@ testlist options
>>  xpand=false
>>  ;;
>>
>> +-vxhs)
>> +IMGPROTO=vxhs
>> +xpand=false
>> +;;
>> +
>>  -ssh)
>>  IMGPROTO=ssh
>>  xpand=false
>> diff --git a/tests/qemu-iotests/common.config 
>> b/tests/qemu-iotests/common.config
>> index f6384fb..c7a80c0 100644
>> --- a/tests/qemu-iotests/common.config
>> +++ b/tests/qemu-iotests/common.config
>> @@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
>>  export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
>>  fi
>>
>> +if [ -z "$QEMU_VXHS_PROG" ]; then
>> +export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
>> +fi
>> +
>>  _qemu_wrapper()
>>  {
>>  (
>> @@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
>>  )
>>  }
>>
>> +_qemu_vxhs_wrapper()
>> +{
>> +(
>> +echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
>> +exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
>> +)
>> +}
>> +
>>  export QEMU=_qemu_wrapper
>>  export QEMU_IMG=_qemu_img_wrapper
>>  export QEMU_IO=_qemu_io_wrapper
>>  export QEMU_NBD=_qemu_nbd_wrapper
>> +export QEMU_VXHS=_qemu_vxhs_wrapper
>>
>>  QEMU_IMG_EXTRA_ARGS=
>>  if [ "$IMGOPTSSYNTAX" = "true" ]; then
>> diff --git a/tests/qemu-iotests/common.filter 
>> b/tests/qemu-iotests/common.filter
>> index 240ed06..a8a4d0e 100644
>> --- a/tests/qemu-iotests/common.filter
>> +++ b/tests/qemu-iotests/common.filter
>> @@ -123,6 +123,7 @@ _filter_img_info()
>>  -e "s#$TEST_DIR#TEST_DIR#g" \
>>  -e "s#$IMGFMT#IMGFMT#g" \
>>  -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
>> +-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
>>  -e "/encrypted: yes/d" \
>>  -e "/cluster_size: [0-9]\\+/d" \
>>  -e "/table_size: [0-9]\\+/d" \
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 3213765..06a3164 100644
>> --- a/tests/qemu-iotests/common.rc
>> +++ b/tests/qemu-iotests/common.rc
>> @@ -89,6 +89,9 @@ else
>>  TEST_IMG=$TEST_DIR/t.$IMGFMT
>>  elif [ "$IMGPROTO" = "archipelago" ]; then
>>  TEST_IMG="archipelago:at.$IMGFMT"
>> +elif [ "$IMGPROTO" = "vxhs" ]; then
>> +TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
>> +TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
>>  else
>>  TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
>>  fi
>> @@ -175,6 +178,12 @@ _make_test_img()
>>  eval "$

Re: [Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Hi,

There has been some work going on on the VxHS libvirt patches and we
are making good progress.

A suggestion has been made on the libvirt community to check if we can
change the qemu VxHS device specification syntax as follows.

Replace

(a) +file.server.host=192.168.0.1,file.server.port=

with

(b) +file.host=192.168.0.1,file.port=

The reasoning being that since we have only one host (true as the
failover is now being handled completely/transparently) within the
libqnio library), the "server" part is redundant.

Excerpt from John Ferlan's email -
/==/
#2. Is the desire to ever support more than 1 host? If not, then is the
"server" syntax you've borrowed from the Gluster code necessary? Could
you just go with the single "host" like NBD and SSH. As it relates to
the qemu command line - I'm not quite as clear. From the example I see
in commit id '7b7da9e28', the gluster syntax would have:

+file.server.0.type=tcp,file.server.0.host=example.org,file.server.0.port=6000,\
+file.server.1.type=tcp,file.server.1.host=example.org,file.server.1.port=24007,\
+file.server.2.type=unix,file.server.2.socket=/path/to/sock,format=qcow2,\

whereas, the VxHS syntax is:
 +file.server.host=192.168.0.1,file.server.port=,format=raw,if=none,\

FWIW: I also note there is no ".type=tcp" in your output - so perhaps
the "default" is tcp unless otherwise specified, but I'm sure of the
qemu syntax requirements in this area. I assume that since there's only
1 server, the ".0, .1, .2" become unnecessary (something added by commit
id 'f1bbc7df4' for multiple gluster hosts).

I haven't closedly followed the qemu syntax discussion, but it would it
would be possible to use:

+file.host=192.168.0.1,file.port=

Similar to how NBD (see commit id 'a1674fd9') and SSH (see commit id
'bc225b1b5') are handled.
/==/

If this proposal looks OK to the community, then I will make this user
interface change in the next VxHS qemu patch.

Thanks,
Ashish

On Wed, Nov 16, 2016 at 9:05 AM, ashish mittal <ashmit...@gmail.com> wrote:
> On Wed, Nov 16, 2016 at 3:27 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
>> On Wed, Nov 16, 2016 at 9:49 AM, Fam Zheng <f...@redhat.com> wrote:
>>> On Wed, 11/16 10:04, Markus Armbruster wrote:
>>>> ashish mittal <ashmit...@gmail.com> writes:
>>>>
>>>> > Thanks for concluding on this.
>>>> >
>>>> > I will rearrange the qnio_api.h header accordingly as follows:
>>>> >
>>>> > +#include "qemu/osdep.h"
>>>>
>>>> Headers should not include osdep.h.
>>>
>>> This is about including "osdep.h" _and_ "qnio_api.h" in block/vxhs.c, so 
>>> what
>>> Ashish means looks good to me.
>>
>> Yes, I think "will rearrange the qnio_api.h header" was a typo and was
>> supposed to be block/vxhs.c.
>>
>> Stefan
>
> Thanks for the correction. Yes, i meant rearrange headers in block/vxhs.c.

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi,

I am requesting feedback on the following design proposal for libqnio.

This adds an access control mechanism between libqnio client and
server. It is not a full client-server authentication model, and it is
not intended to substitute an authentication mechanism.

We wanted to check if the following would be acceptable for the first
version of VxHS patch while we design/implement a proper
authentication mechanism (possibly on a libqnio side branch)?

1.   Client passes VM ID and vdisk ID to the server when it wants
to open a vdisk.
2.   Server verifies whether the client/VM has access to open the
disk and passes/fails the open request.
3.   Server returns unique token for every open vdisk.
4.   Client passes this token with every request to server.
5.   Server verifies this token for every request.
6.   Server invalidates token when a client closes the corresponding vdisk.

Thanks,
Ashish

On Wed, Nov 30, 2016 at 1:01 AM, Stefan Hajnoczi  wrote:
> On Mon, Nov 28, 2016 at 02:17:56PM +, Stefan Hajnoczi wrote:
>> Please take a look at vhost-user-scsi, which folks from Nutanix are
>> currently working on.  See "[PATCH v2 0/3] Introduce vhost-user-scsi and
>> sample application" on qemu-devel.  It is a true zero-copy local I/O tap
>> because it shares guest RAM.  This is more efficient than cross memory
>> attach's single memory copy.  It does not require running the server as
>> root.  This is the #1 thing you should evaluate for your final
>> architecture.
>>
>> vhost-user-scsi works on the virtio-scsi emulation level.  That means
>> the server must implement the virtio-scsi vring and device emulation.
>> It is not a block driver.  By hooking in at this level you can achieve
>> the best performance but you lose all QEMU block layer functionality and
>> need to implement your own SCSI target.  You also need to consider live
>> migration.
>
> To clarify why I think vhost-user-scsi is best suited to your
> requirements for performance:
>
> With vhost-user-scsi the qnio server would be notified by kvm.ko via
> eventfd when the VM submits new I/O requests to the virtio-scsi HBA.
> The QEMU process is completely bypassed for I/O request submission and
> the qnio server processes the SCSI command instead.  This avoids the
> context switch to QEMU and then to the qnio server.  With cross memory
> attach QEMU first needs to process the I/O request and hand it to
> libqnio before the qnio server can be scheduled.
>
> The vhost-user-scsi qnio server has shared memory access to guest RAM
> and is therefore able to do zero-copy I/O into guest buffers.  Cross
> memory attach always incurs a memory copy.
>
> Using this high-performance architecture requires significant changes
> though.  vhost-user-scsi hooks into the stack at a different layer so a
> QEMU block driver is not used at all.  QEMU also wouldn't use libqnio.
> Instead everything will live in your qnio server process (not part of
> QEMU).
>
> You'd have to rethink the resiliency strategy because you currently rely
> on the QEMU block driver connecting to a different qnio server if the
> local qnio server fails.  In the vhost-user-scsi world it's more like
> having a phyiscal SCSI adapter - redundancy and multipathing are used to
> achieve resiliency.
>
> For example, virtio-scsi HBA #1 would connect to the local qnio server
> process.  virtio-scsi HBA #2 would connect to another local process
> called the "proxy process" which forwards requests to a remote qnio
> server (using libqnio?).  If HBA #1 fails then I/O is sent to HBA #2
> instead.  The path can reset back to HBA #1 once that becomes
> operational again.
>
> If the qnio server is supposed to run in a VM instead of directly in the
> host environment then it's worth looking at the vhost-pci work that Wei
> Wang  is working on.  The email thread is called
> "[PATCH v2 0/4] *** vhost-user spec extension for vhost-pci ***".  The
> idea here is to allow inter-VM virtio device emulation so that instead
> of terminating the virtio-scsi device in the qnio server process on the
> host, you can terminate it inside another VM with good performance
> characteristics.
>
> Stefan

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

+ Rakesh from Veritas

On Mon, Nov 28, 2016 at 6:17 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Mon, Nov 28, 2016 at 10:23:41AM +, Ketan Nilangekar wrote:
>>
>>
>> On 11/25/16, 5:05 PM, "Stefan Hajnoczi" <stefa...@gmail.com> wrote:
>>
>> On Fri, Nov 25, 2016 at 08:27:26AM +, Ketan Nilangekar wrote:
>> > On 11/24/16, 9:38 PM, "Stefan Hajnoczi" <stefa...@gmail.com> wrote:
>> > On Thu, Nov 24, 2016 at 11:31:14AM +, Ketan Nilangekar wrote:
>> > > On 11/24/16, 4:41 PM, "Stefan Hajnoczi" <stefa...@gmail.com> 
>> wrote:
>> > > On Thu, Nov 24, 2016 at 05:44:37AM +, Ketan Nilangekar 
>> wrote:
>> > > > On 11/24/16, 4:07 AM, "Paolo Bonzini" 
>> <pbonz...@redhat.com> wrote:
>> > > > >On 23/11/2016 23:09, ashish mittal wrote:
>> > > > >> On the topic of protocol security -
>> > > > >>
>> > > > >> Would it be enough for the first patch to implement only
>> > > > >> authentication and not encryption?
>> > > > >
>> > > > >Yes, of course.  However, as we introduce more and more 
>> QEMU-specific
>> > > > >characteristics to a protocol that is already 
>> QEMU-specific (it doesn't
>> > > > >do failover, etc.), I am still not sure of the actual 
>> benefit of using
>> > > > >libqnio versus having an NBD server or FUSE driver.
>> > > > >
>> > > > >You have already mentioned performance, but the design 
>> has changed so
>> > > > >much that I think one of the two things has to change: 
>> either failover
>> > > > >moves back to QEMU and there is no (closed source) 
>> translator running on
>> > > > >the node, or the translator needs to speak a well-known 
>> and
>> > > > >already-supported protocol.
>> > > >
>> > > > IMO design has not changed. Implementation has changed 
>> significantly. I would propose that we keep resiliency/failover code out of 
>> QEMU driver and implement it entirely in libqnio as planned in a subsequent 
>> revision. The VxHS server does not need to understand/handle failover at all.
>> > > >
>> > > > Today libqnio gives us significantly better performance 
>> than any NBD/FUSE implementation. We know because we have prototyped with 
>> both. Significant improvements to libqnio are also in the pipeline which 
>> will use cross memory attach calls to further boost performance. Ofcourse a 
>> big reason for the performance is also the HyperScale storage backend but we 
>> believe this method of IO tapping/redirecting can be leveraged by other 
>> solutions as well.
>> > >
>> > > By "cross memory attach" do you mean
>> > > process_vm_readv(2)/process_vm_writev(2)?
>> > >
>> > > Ketan> Yes.
>> > >
>> > > That puts us back to square one in terms of security.  You 
>> have
>> > > (untrusted) QEMU + (untrusted) libqnio directly accessing 
>> the memory of
>> > > another process on the same machine.  That process is 
>> therefore also
>> > > untrusted and may only process data for one guest so that 
>> guests stay
>> > > isolated from each other.
>> > >
>> > > Ketan> Understood but this will be no worse than the current 
>> network based communication between qnio and vxhs server. And although we 
>> have questions around QEMU trust/vulnerability issues, we are looking to 
>> implement basic authentication scheme between libqnio and vxhs server.
>> >
>> > This is incorrect.
>> >
>> > Cross memory attach is equivalent to ptrace(2) (i.e. debugger) 
>> access.
>> > It means process A reads/writes directly from/to process B memory. 
>>  Both
>> > processes must have the same uid/gid.  There is no trust boundary
>> > between them.
>> >
>> > Ketan> Not if vxhs server is running as root and initi

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

On the topic of protocol security -

Would it be enough for the first patch to implement only
authentication and not encryption?

On Wed, Nov 23, 2016 at 12:25 AM, Ketan Nilangekar
 wrote:
> +Nitin Jerath from Veritas.
>
>
>
>
> On 11/18/16, 7:06 PM, "Daniel P. Berrange"  wrote:
>
>>On Fri, Nov 18, 2016 at 01:25:43PM +, Ketan Nilangekar wrote:
>>>
>>>
>>> > On Nov 18, 2016, at 5:25 PM, Daniel P. Berrange  
>>> > wrote:
>>> >
>>> >> On Fri, Nov 18, 2016 at 11:36:02AM +, Ketan Nilangekar wrote:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>> On 11/18/16, 3:32 PM, "Stefan Hajnoczi"  wrote:
>>> >>>
>>>  On Fri, Nov 18, 2016 at 02:26:21AM -0500, Jeff Cody wrote:
>>>  * Daniel pointed out that there is no authentication method for taking 
>>>  to a
>>>   remote server.  This seems a bit scary.  Maybe all that is needed 
>>>  here is
>>>   some clarification of the security scheme for authentication?  My
>>>   impression from above is that you are relying on the networks being
>>>   private to provide some sort of implicit authentication, though, and 
>>>  this
>>>   seems fragile (and doesn't protect against a compromised guest or 
>>>  other
>>>   process on the server, for one).
>>> >>>
>>> >>> Exactly, from the QEMU trust model you must assume that QEMU has been
>>> >>> compromised by the guest.  The escaped guest can connect to the VxHS
>>> >>> server since it controls the QEMU process.
>>> >>>
>>> >>> An escaped guest must not have access to other guests' volumes.
>>> >>> Therefore authentication is necessary.
>>> >>
>>> >> Just so I am clear on this, how will such an escaped guest get to know
>>> >> the other guest vdisk IDs?
>>> >
>>> > There can be a multiple approaches depending on the deployment scenario.
>>> > At the very simplest it could directly read the IDs out of the libvirt
>>> > XML files in /var/run/libvirt. Or it can rnu "ps" to list other running
>>> > QEMU processes and see the vdisk IDs in the command line args of those
>>> > processes. Or the mgmt app may be creating vdisk IDs based on some
>>> > particular scheme, and the attacker may have info about this which lets
>>> > them determine likely IDs.  Or the QEMU may have previously been
>>> > permitted to the use the disk and remembered the ID for use later
>>> > after access to the disk has been removed.
>>> >
>>>
>>> Are we talking about a compromised guest here or compromised hypervisor?
>>> How will a compromised guest read the xml file or list running qemu
>>> processes?
>>
>>Compromised QEMU process, aka hypervisor userspace
>>
>>
>>Regards,
>>Daniel
>>--
>>|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
>>|: http://libvirt.org  -o- http://virt-manager.org :|
>>|: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Wed, Nov 16, 2016 at 3:27 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Wed, Nov 16, 2016 at 9:49 AM, Fam Zheng <f...@redhat.com> wrote:
>> On Wed, 11/16 10:04, Markus Armbruster wrote:
>>> ashish mittal <ashmit...@gmail.com> writes:
>>>
>>> > Thanks for concluding on this.
>>> >
>>> > I will rearrange the qnio_api.h header accordingly as follows:
>>> >
>>> > +#include "qemu/osdep.h"
>>>
>>> Headers should not include osdep.h.
>>
>> This is about including "osdep.h" _and_ "qnio_api.h" in block/vxhs.c, so what
>> Ashish means looks good to me.
>
> Yes, I think "will rearrange the qnio_api.h header" was a typo and was
> supposed to be block/vxhs.c.
>
> Stefan

Thanks for the correction. Yes, i meant rearrange headers in block/vxhs.c.

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Stefan


On Wed, Sep 28, 2016 at 2:45 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>
> Review of .bdrv_open() and .bdrv_aio_writev() code paths.
>
> The big issues I see in this driver and libqnio:
>
> 1. Showstoppers like broken .bdrv_open() and leaking memory on every
>reply message.
> 2. Insecure due to missing input validation (network packets and
>configuration) and incorrect string handling.
> 3. Not fully asynchronous so QEMU and the guest may hang.
>
> Please think about the whole codebase and not just the lines I've
> pointed out in this review when fixing these sorts of issues.  There may
> be similar instances of these bugs elsewhere and it's important that
> they are fixed so that this can be merged.
>
>> +/*
>> + * Structure per vDisk maintained for state
>> + */
>> +typedef struct BDRVVXHSState {
>> +int fds[2];
>> +int64_t vdisk_size;
>> +int64_t vdisk_blocks;
>> +int64_t vdisk_flags;
>> +int vdisk_aio_count;
>> +int event_reader_pos;
>> +VXHSAIOCB   *qnio_event_acb;
>> +void*qnio_ctx;
>> +QemuSpinvdisk_lock; /* Lock to protect BDRVVXHSState */
>> +QemuSpinvdisk_acb_lock;  /* Protects ACB */
>
> These comments are insufficient for documenting locking.  Not all fields
> are actually protected by these locks.  Please order fields according to
> lock coverage:
>
> typedef struct VXHSAIOCB {
> ...
>
> /* Protected by BDRVVXHSState->vdisk_acb_lock */
> int segments;
> ...
> };
>
> typedef struct BDRVVXHSState {
> ...
>
> /* Protected by vdisk_lock */
> QemuSpinvdisk_lock;
> int vdisk_aio_count;
> QSIMPLEQ_HEAD(aio_retryq, VXHSAIOCB) vdisk_aio_retryq;
> ...
> }
>
>> +static void vxhs_qnio_iio_close(BDRVVXHSState *s, int idx)
>> +{
>> +/*
>> + * Close vDisk device
>> + */
>> +if (s->vdisk_hostinfo[idx].vdisk_rfd >= 0) {
>> +iio_devclose(s->qnio_ctx, 0, s->vdisk_hostinfo[idx].vdisk_rfd);
>
> libqnio comment:
> Why does iio_devclose() take an unused cfd argument?  Perhaps it can be
> dropped.
>
>> +s->vdisk_hostinfo[idx].vdisk_rfd = -1;
>> +}
>> +
>> +/*
>> + * Close QNIO channel against cached channel-fd
>> + */
>> +if (s->vdisk_hostinfo[idx].qnio_cfd >= 0) {
>> +iio_close(s->qnio_ctx, s->vdisk_hostinfo[idx].qnio_cfd);
>
> libqnio comment:
> Why does iio_devclose() take an int32_t cfd argument but iio_close()
> takes a uint32_t cfd argument?
>
>> +s->vdisk_hostinfo[idx].qnio_cfd = -1;
>> +}
>> +}
>> +
>> +static int vxhs_qnio_iio_open(int *cfd, const char *of_vsa_addr,
>> +  int *rfd, const char *file_name)
>> +{
>> +/*
>> + * Open qnio channel to storage agent if not opened before.
>> + */
>> +if (*cfd < 0) {
>> +*cfd = iio_open(global_qnio_ctx, of_vsa_addr, 0);
>
> libqnio comments:
>
> 1.
> There is a buffer overflow in qnio_create_channel().  strncpy() is used
> incorrectly so long hostname or port (both can be 99 characters long)
> will overflow channel->name[] (64 characters) or channel->port[] (8
> characters).
>
> strncpy(channel->name, hostname, strlen(hostname) + 1);
> strncpy(channel->port, port, strlen(port) + 1);
>
> The third argument must be the size of the *destination* buffer, not the
> source buffer.  Also note that strncpy() doesn't NUL-terminate the
> destination string so you must do that manually to ensure there is a NUL
> byte at the end of the buffer.
>
> 2.
> channel is leaked in the "Failed to open single connection" error case
> in qnio_create_channel().
>
> 3.
> If host is longer the 63 characters then the ioapi_ctx->channels and
> qnio_ctx->channels maps will use different keys due to string truncation
> in qnio_create_channel().  This means "Channel already exists" in
> qnio_create_channel() and possibly other things will not work as
> expected.
>
>> +if (*cfd < 0) {
>> +trace_vxhs_qnio_iio_open(of_vsa_addr);
>> +return -ENODEV;
>> +}
>> +}
>> +
>> +/*
>> + * Open vdisk device
>> + */
>> +*

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Thank you. Will work with this.

On Tue, Nov 15, 2016 at 12:48 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Nov 15, 2016 at 11:02:34AM -0800, ashish mittal wrote:
>> I had replied to the QEMUBH suggestion in the email below.
>>
>> Regards,
>> Ashish
>>
>> On Tue, Aug 23, 2016 at 3:22 PM, ashish mittal <ashmit...@gmail.com> wrote:
>> > Thanks Stefan, I will look at block/quorum.c.
>> >
>> > I have sent V4 of the patch with a reworked parsing logic for both
>> > JSON and URI. Both of them are quite compact now.
>> >
>> > URI parsing now follows the suggestion given by Kevin.
>> > //
>> > However, you should use the proper interfaces to implement this, which
>> > is .bdrv_parse_filename(). This is a function that gets a string and
>> > converts it into a QDict, which is then passed to .bdrv_open(). So it
>> > uses exactly the same code path in .bdrv_open() as if used directly with
>> > QAPI.
>> > //
>> >
>> > Additionally, I have fixed all the issues pointed out by you on V1 of
>> > the patch. The only change I haven't done is to replace pipes with
>> > QEMUBH. I am hoping this will not hold up the patch from being
>> > accepted, and I can make this transition later with proper dev and
>> > testing.
>
> Sorry, I forgot about this email.  I still think the QEMUBH approach
> makes sense.  Please try the following patch.
>
> I have compiled but not run it.  You are welcome to squash it into your
> next patch.
>
> The following is Signed-off-by: Stefan Hajnoczi <stefa...@redhat.com>.
>
> ---8<---
>
> diff --git a/block/vxhs.c b/block/vxhs.c
> index 8913e8f..22fd989 100644
> --- a/block/vxhs.c
> +++ b/block/vxhs.c
> @@ -60,9 +60,6 @@ typedef struct VXHSvDiskHostsInfo {
>   * Structure per vDisk maintained for state
>   */
>  typedef struct BDRVVXHSState {
> -int fds[2];
> -int event_reader_pos;
> -VXHSAIOCB *qnio_event_acb;
>  VXHSvDiskHostsInfo vdisk_hostinfo; /* Per host info */
>  char *vdisk_guid;
>  } BDRVVXHSState;
> @@ -73,12 +70,33 @@ static QNIOLibState qniolib;
>  /* vdisk prefix to pass to qnio */
>  static const char vdisk_prefix[] = "/dev/of/vdisk";
>
> +static void vxhs_complete_aio_bh(void *opaque)
> +{
> +VXHSAIOCB *acb = opaque;
> +BlockCompletionFunc *cb = acb->common.cb;
> +void *cb_opaque = acb->common.opaque;
> +int ret = 0;
> +
> +if (acb->err != 0) {
> +trace_vxhs_complete_aio(acb, acb->err);
> +/*
> + * We mask all the IO errors generically as EIO for upper layers
> + * Right now our IO Manager uses non standard error codes. Instead
> + * of confusing upper layers with incorrect interpretation we are
> + * doing this workaround.
> + */
> +ret = (-EIO);
> +}
> +
> +qemu_aio_unref(acb);
> +cb(cb_opaque, ret);
> +}
> +
> +/* Called from a libqnio thread */
>  static void vxhs_iio_callback(int32_t rfd, uint32_t reason, void *ctx,
>uint32_t error, uint32_t opcode)
>  {
>  VXHSAIOCB *acb = NULL;
> -BDRVVXHSState *s = NULL;
> -ssize_t ret;
>
>  switch (opcode) {
>  case IRP_READ_REQUEST:
> @@ -91,7 +109,6 @@ static void vxhs_iio_callback(int32_t rfd, uint32_t 
> reason, void *ctx,
>   */
>  if (ctx) {
>  acb = ctx;
> -s = acb->common.bs->opaque;
>  } else {
>  trace_vxhs_iio_callback(error, reason);
>  goto out;
> @@ -104,8 +121,8 @@ static void vxhs_iio_callback(int32_t rfd, uint32_t 
> reason, void *ctx,
>  trace_vxhs_iio_callback(error, reason);
>  }
>
> -ret = qemu_write_full(s->fds[VDISK_FD_WRITE], , sizeof(acb));
> -g_assert(ret == sizeof(acb));
> +aio_bh_schedule_oneshot(bdrv_get_aio_context(acb->common.bs),
> +vxhs_complete_aio_bh, acb);
>  break;
>
>  default:
> @@ -223,53 +240,6 @@ static void vxhs_qnio_iio_close(BDRVVXHSState *s)
>  vxhs_qnio_close();
>  }
>
> -static void vxhs_complete_aio(VXHSAIOCB *acb, BDRVVXHSState *s)
> -{
> -BlockCompletionFunc *cb = acb->common.cb;
> -void *opaque = acb->common.opaque;
> -int ret = 0;
> -
> -if (acb->err != 0) {
> -trace_vxhs_complete_aio(acb, acb->err);
> -/*
> - * We mask all the IO errors generically as EIO for upper layers
> - * Right now our IO Manager uses non s

Re: [Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

On Mon, Nov 14, 2016 at 7:07 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Mon, Nov 07, 2016 at 04:59:44PM -0800, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using the JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
>> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using the URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v6 changelog:
>> (1) Added qemu-iotests for VxHS as a new patch in the series.
>> (2) Replaced release version from 2.8 to 2.9 in block-core.json.
>>
>> v5 changelog:
>> (1) Incorporated v4 review comments.
>>
>> v4 changelog:
>> (1) Incorporated v3 review comments on QAPI changes.
>> (2) Added refcounting for device open/close.
>> Free library resources on last device close.
>>
>> v3 changelog:
>> (1) Added QAPI schema for the VxHS driver.
>>
>> v2 changelog:
>> (1) Changes done in response to v1 comments.
>>
>>  block/Makefile.objs  |   2 +
>>  block/trace-events   |  21 ++
>>  block/vxhs.c | 689 
>> +++
>>  configure|  41 +++
>>  qapi/block-core.json |  21 +-
>>  5 files changed, 772 insertions(+), 2 deletions(-)
>>  create mode 100644 block/vxhs.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 67a036a..58313a2 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
>>  block-obj-$(CONFIG_CURL) += curl.o
>>  block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>> @@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
>>  rbd.o-libs := $(RBD_LIBS)
>>  gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
>>  gluster.o-libs := $(GLUSTERFS_LIBS)
>> +vxhs.o-libs:= $(VXHS_LIBS)
>>  ssh.o-cflags   := $(LIBSSH2_CFLAGS)
>>  ssh.o-libs := $(LIBSSH2_LIBS)
>>  archipelago.o-libs := $(ARCHIPELAGO_LIBS)
>> diff --git a/block/trace-events b/block/trace-events
>> index 882c903..efdd5ef 100644
>> --- a/block/trace-events
>> +++ b/block/trace-events
>> @@ -112,3 +112,24 @@ qed_aio_write_data(void *s, void *acb, int ret, 
>> uint64_t offset, size_t len) "s
>>  qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t 
>> len) "s %p acb %p ret %d offset %"PRIu64" len %zu"
>> +
>> +# block/vxhs.c
>> +vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
>> +vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
>> +vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o 
>> %d, %d"
>> +vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno 
>> %d"
>> +vxhs_open_fail(int ret) "Could not open the device. Error = %d"
>> +vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
>> Error=%d"
>> +vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
>> +vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
>> *acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = 
>> %lu offset = %lu ACB = %p. Error = %d, errno = %d"
>> +vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat 
>> ioctl failed, 

Re: [Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[ashish mittal]

Thanks for concluding on this.

I will rearrange the qnio_api.h header accordingly as follows:

+#include "qemu/osdep.h"
+#include<=== after osdep.h
+#include "block/block_int.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "trace.h"
+#include "qemu/uri.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"  < remove


On Tue, Nov 15, 2016 at 11:20 AM, Stefan Hajnoczi  wrote:
> On Tue, Nov 15, 2016 at 11:00:16PM +0800, Fam Zheng wrote:
>> On Tue, 11/15 14:45, Stefan Hajnoczi wrote:
>> > On Tue, Nov 15, 2016 at 08:44:17PM +0800, Fam Zheng wrote:
>> > > On Tue, 11/15 10:18, Stefan Hajnoczi wrote:
>> > > > Fam, Markus: Unfortunately neither the clean-includes script nor its
>> > > > patch series cover letter explains *why* osdep.h should be included
>> > > > before system headers.
>> > >
>> > > I don't know Peter's exact intention either, but AFAICT it is about the 
>> > > few
>> > > quirks in osdep.h:
>> > >
>> > >
>> > > /* Older versions of C++ don't get definitions of various macros from
>> > >  * stdlib.h unless we define these macros before first inclusion of
>> > >  * that system header.
>> > >  */
>> > > #ifndef __STDC_CONSTANT_MACROS
>> > > #define __STDC_CONSTANT_MACROS
>> > > #endif
>> > > #ifndef __STDC_LIMIT_MACROS
>> > > #define __STDC_LIMIT_MACROS
>> > > #endif
>> > > #ifndef __STDC_FORMAT_MACROS
>> > > #define __STDC_FORMAT_MACROS
>> > > #endif
>> > >
>> > > /* The following block of code temporarily renames the daemon() function 
>> > > so the
>> > >  * compiler does not see the warning associated with it in stdlib.h on 
>> > > OSX
>> > >  */
>> > > #ifdef __APPLE__
>> > > #define daemon qemu_fake_daemon_function
>> > > #include 
>> > > #undef daemon
>> > > extern int daemon(int, int);
>> > > #endif
>> > >
>> > > <...>
>> > >
>> > >
>> > > /* Mac OSX has a  bug that incorrectly defines SIZE_MAX with
>> > >  * the wrong type. Our replacement isn't usable in preprocessor
>> > >  * expressions, but it is sufficient for our needs. */
>> > > #if defined(HAVE_BROKEN_SIZE_MAX) && HAVE_BROKEN_SIZE_MAX
>> > > #undef SIZE_MAX
>> > > #define SIZE_MAX ((size_t)-1)
>> > > #endif
>> >
>> > This is exactly the kind of stuff we should not be doing to the libqnio
>> > header file!
>> >
>> > Those redefinitions are useful for QEMU code.  They should not be done
>> > to third-party system headers though.
>>
>> But we still want to include osdep.h before libqnio pulls in stdint.h. If we
>> don't make osdep.h the very first, that cannot be guaranteed.
>
> Thanks, that makes sense!
>
> I'll add a comment to clean-includes with this explanation of why
> osdep.h must go before system headers.
>
> Stefan

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Stefan,

I had replied to the QEMUBH suggestion in the email below.

Regards,
Ashish

On Tue, Aug 23, 2016 at 3:22 PM, ashish mittal <ashmit...@gmail.com> wrote:
> Thanks Stefan, I will look at block/quorum.c.
>
> I have sent V4 of the patch with a reworked parsing logic for both
> JSON and URI. Both of them are quite compact now.
>
> URI parsing now follows the suggestion given by Kevin.
> //
> However, you should use the proper interfaces to implement this, which
> is .bdrv_parse_filename(). This is a function that gets a string and
> converts it into a QDict, which is then passed to .bdrv_open(). So it
> uses exactly the same code path in .bdrv_open() as if used directly with
> QAPI.
> //
>
> Additionally, I have fixed all the issues pointed out by you on V1 of
> the patch. The only change I haven't done is to replace pipes with
> QEMUBH. I am hoping this will not hold up the patch from being
> accepted, and I can make this transition later with proper dev and
> testing.
>
> Regards,
> Ashish
>
>
> On Tue, Aug 23, 2016 at 2:58 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
>> On Sat, Aug 20, 2016 at 11:42:22AM -0700, ashish mittal wrote:
>>> I'm trying to understand how I can parse a json command line having
>>> multiple server list (InetSocketAddressList) without the QemuOpts ->
>>> QDict -> QAPI conversion that I am currently doing.
>>>
>>> block/nbd.c has been suggested, but it only parses a single host
>>> entry, which is pretty straightforward, but not very helpful.
>>>
>>> Could somebody please suggest a sample implementation (other than
>>> gluster.c) that parses a list of server like options in the json
>>> format?
>>
>> Another block driver that takes an array of objects is block/quorum.c.
>> See quorum_open() and how it picks out "children." objects.
>>
>> I haven't read the full discussion in this thread but maybe that's what
>> you're looking for.
>>
>> Stefan

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Will look into these ASAP.

On Mon, Nov 14, 2016 at 7:05 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Wed, Sep 28, 2016 at 10:45 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
>> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>>
>> Review of .bdrv_open() and .bdrv_aio_writev() code paths.
>>
>> The big issues I see in this driver and libqnio:
>>
>> 1. Showstoppers like broken .bdrv_open() and leaking memory on every
>>reply message.
>> 2. Insecure due to missing input validation (network packets and
>>configuration) and incorrect string handling.
>> 3. Not fully asynchronous so QEMU and the guest may hang.
>>
>> Please think about the whole codebase and not just the lines I've
>> pointed out in this review when fixing these sorts of issues.  There may
>> be similar instances of these bugs elsewhere and it's important that
>> they are fixed so that this can be merged.
>
> Ping?
>
> You didn't respond to the comments I raised.  The libqnio buffer
> overflows and other issues from this email are still present.
>
> I put a lot of time into reviewing this patch series and libqnio.  If
> you want to get reviews please address feedback before sending a new
> patch series.
>
>>
>>> +/*
>>> + * Structure per vDisk maintained for state
>>> + */
>>> +typedef struct BDRVVXHSState {
>>> +int fds[2];
>>> +int64_t vdisk_size;
>>> +int64_t vdisk_blocks;
>>> +int64_t vdisk_flags;
>>> +int vdisk_aio_count;
>>> +int event_reader_pos;
>>> +VXHSAIOCB   *qnio_event_acb;
>>> +void*qnio_ctx;
>>> +QemuSpinvdisk_lock; /* Lock to protect BDRVVXHSState */
>>> +QemuSpinvdisk_acb_lock;  /* Protects ACB */
>>
>> These comments are insufficient for documenting locking.  Not all fields
>> are actually protected by these locks.  Please order fields according to
>> lock coverage:
>>
>> typedef struct VXHSAIOCB {
>> ...
>>
>> /* Protected by BDRVVXHSState->vdisk_acb_lock */
>> int segments;
>> ...
>> };
>>
>> typedef struct BDRVVXHSState {
>> ...
>>
>> /* Protected by vdisk_lock */
>> QemuSpinvdisk_lock;
>> int vdisk_aio_count;
>> QSIMPLEQ_HEAD(aio_retryq, VXHSAIOCB) vdisk_aio_retryq;
>> ...
>> }
>>
>>> +static void vxhs_qnio_iio_close(BDRVVXHSState *s, int idx)
>>> +{
>>> +/*
>>> + * Close vDisk device
>>> + */
>>> +if (s->vdisk_hostinfo[idx].vdisk_rfd >= 0) {
>>> +iio_devclose(s->qnio_ctx, 0, s->vdisk_hostinfo[idx].vdisk_rfd);
>>
>> libqnio comment:
>> Why does iio_devclose() take an unused cfd argument?  Perhaps it can be
>> dropped.
>>
>>> +s->vdisk_hostinfo[idx].vdisk_rfd = -1;
>>> +}
>>> +
>>> +/*
>>> + * Close QNIO channel against cached channel-fd
>>> + */
>>> +if (s->vdisk_hostinfo[idx].qnio_cfd >= 0) {
>>> +iio_close(s->qnio_ctx, s->vdisk_hostinfo[idx].qnio_cfd);
>>
>> libqnio comment:
>> Why does iio_devclose() take an int32_t cfd argument but iio_close()
>> takes a uint32_t cfd argument?
>>
>>> +s->vdisk_hostinfo[idx].qnio_cfd = -1;
>>> +}
>>> +}
>>> +
>>> +static int vxhs_qnio_iio_open(int *cfd, const char *of_vsa_addr,
>>> +  int *rfd, const char *file_name)
>>> +{
>>> +/*
>>> + * Open qnio channel to storage agent if not opened before.
>>> + */
>>> +if (*cfd < 0) {
>>> +*cfd = iio_open(global_qnio_ctx, of_vsa_addr, 0);
>>
>> libqnio comments:
>>
>> 1.
>> There is a buffer overflow in qnio_create_channel().  strncpy() is used
>> incorrectly so long hostname or port (both can be 99 characters long)
>> will overflow channel->name[] (64 characters) or channel->port[] (8
>> characters).
>>
>> strncpy(channel->name, hostname, strlen(hostname) + 1);
>> strncpy(channel->port, port, strlen(port) + 1);
>>
>> The third argument must be the size of the *destination* buffer, not the
>> source buffer.  Also note that strncpy() doesn't NUL-ter

Re: [Qemu-devel] [PATCH v4] This patch adds support for a new block device type called "vxhs".

[ashish mittal]

On Mon, Nov 7, 2016 at 5:52 PM, Fam Zheng <f...@redhat.com> wrote:
> On Mon, 11/07 17:39, ashish mittal wrote:
>> I guess the email subject of the individual patches is OK to
>> change?
>
> Yes, that is fine, as long as it doesn't burden incremental reviewing
> unnecessarily.
>

OK. Thanks!

Re: [Qemu-devel] [PATCH v4] This patch adds support for a new block device type called "vxhs".

[ashish mittal]

On Fri, Nov 4, 2016 at 6:04 AM, Stefan Hajnoczi  wrote:
> Please keep using "block/vxhs: Add Veritas HyperScale VxHS block device
> support" as the cover letter email subject.  This way tools are able to
> automatically mark old versions of this patch series as obsolete.

Sent out the new set of patches as a series. Hope I got it right!
Will keep the same email subject on the cover letter in subsequent
versions. I guess the email subject of the individual patches is OK to
change?

[Qemu-devel] [PATCH v6 2/2] block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

[Ashish Mittal]

These changes use a vxhs test server that is a part of the following
repository:
https://github.com/MittalAshish/libqnio.git

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v6 changelog:
(1) Added iotests for VxHS block device.

 tests/qemu-iotests/common|  6 ++
 tests/qemu-iotests/common.config | 13 +
 tests/qemu-iotests/common.filter |  1 +
 tests/qemu-iotests/common.rc | 19 +++
 4 files changed, 39 insertions(+)

diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
index d60ea2c..41430d8 100644
--- a/tests/qemu-iotests/common
+++ b/tests/qemu-iotests/common
@@ -158,6 +158,7 @@ check options
 -nfstest nfs
 -archipelagotest archipelago
 -luks   test luks
+-vxhs   test vxhs
 -xdiff  graphical mode diff
 -nocacheuse O_DIRECT on backing file
 -misalign   misalign memory allocations
@@ -261,6 +262,11 @@ testlist options
 xpand=false
 ;;
 
+-vxhs)
+IMGPROTO=vxhs
+xpand=false
+;;
+
 -ssh)
 IMGPROTO=ssh
 xpand=false
diff --git a/tests/qemu-iotests/common.config b/tests/qemu-iotests/common.config
index f6384fb..c7a80c0 100644
--- a/tests/qemu-iotests/common.config
+++ b/tests/qemu-iotests/common.config
@@ -105,6 +105,10 @@ if [ -z "$QEMU_NBD_PROG" ]; then
 export QEMU_NBD_PROG="`set_prog_path qemu-nbd`"
 fi
 
+if [ -z "$QEMU_VXHS_PROG" ]; then
+export QEMU_VXHS_PROG="`set_prog_path qnio_server /usr/local/bin`"
+fi
+
 _qemu_wrapper()
 {
 (
@@ -156,10 +160,19 @@ _qemu_nbd_wrapper()
 )
 }
 
+_qemu_vxhs_wrapper()
+{
+(
+echo $BASHPID > "${TEST_DIR}/qemu-vxhs.pid"
+exec "$QEMU_VXHS_PROG" $QEMU_VXHS_OPTIONS "$@"
+)
+}
+
 export QEMU=_qemu_wrapper
 export QEMU_IMG=_qemu_img_wrapper
 export QEMU_IO=_qemu_io_wrapper
 export QEMU_NBD=_qemu_nbd_wrapper
+export QEMU_VXHS=_qemu_vxhs_wrapper
 
 QEMU_IMG_EXTRA_ARGS=
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 240ed06..a8a4d0e 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -123,6 +123,7 @@ _filter_img_info()
 -e "s#$TEST_DIR#TEST_DIR#g" \
 -e "s#$IMGFMT#IMGFMT#g" \
 -e 's#nbd://127.0.0.1:10810$#TEST_DIR/t.IMGFMT#g' \
+-e 's#json.*vdisk-id.*vxhs"}}#TEST_DIR/t.IMGFMT#' \
 -e "/encrypted: yes/d" \
 -e "/cluster_size: [0-9]\\+/d" \
 -e "/table_size: [0-9]\\+/d" \
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 3213765..06a3164 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -89,6 +89,9 @@ else
 TEST_IMG=$TEST_DIR/t.$IMGFMT
 elif [ "$IMGPROTO" = "archipelago" ]; then
 TEST_IMG="archipelago:at.$IMGFMT"
+elif [ "$IMGPROTO" = "vxhs" ]; then
+TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT
+TEST_IMG="vxhs://127.0.0.1:/t.$IMGFMT"
 else
 TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT
 fi
@@ -175,6 +178,12 @@ _make_test_img()
 eval "$QEMU_NBD -v -t -b 127.0.0.1 -p 10810 -f $IMGFMT  $TEST_IMG_FILE 
&"
 sleep 1 # FIXME: qemu-nbd needs to be listening before we continue
 fi
+
+# Start QNIO server on image directory for vxhs protocol
+if [ $IMGPROTO = "vxhs" ]; then
+eval "$QEMU_VXHS -d  $TEST_DIR &"
+sleep 1 # Wait for server to come up.
+fi
 }
 
 _rm_test_img()
@@ -201,6 +210,16 @@ _cleanup_test_img()
 fi
 rm -f "$TEST_IMG_FILE"
 ;;
+vxhs)
+if [ -f "${TEST_DIR}/qemu-vxhs.pid" ]; then
+local QEMU_VXHS_PID
+read QEMU_VXHS_PID < "${TEST_DIR}/qemu-vxhs.pid"
+kill ${QEMU_VXHS_PID} >/dev/null 2>&1
+rm -f "${TEST_DIR}/qemu-vxhs.pid"
+fi
+rm -f "$TEST_IMG_FILE"
+;;
+
 file)
 _rm_test_img "$TEST_DIR/t.$IMGFMT"
 _rm_test_img "$TEST_DIR/t.$IMGFMT.orig"
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 1/2] block/vxhs.c: Add support for a new block device type called "vxhs"

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using the JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
-vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using the URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v6 changelog:
(1) Added qemu-iotests for VxHS as a new patch in the series.
(2) Replaced release version from 2.8 to 2.9 in block-core.json.

v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs  |   2 +
 block/trace-events   |  21 ++
 block/vxhs.c | 689 +++
 configure|  41 +++
 qapi/block-core.json |  21 +-
 5 files changed, 772 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 882c903..efdd5ef 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -112,3 +112,24 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk-id from json %s"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..8913

[Qemu-devel] [PATCH v6 0/2] block/vxhs: Add Veritas HyperScale VxHS block device support

[Ashish Mittal]

- Veritas HyperScale block driver in QEMU is designed to provide an accelerated
  IO path from KVM virtual machines to Veritas HyperScale storage service.

- A network IO transfer library that translates block IO from HyperScale block
  driver to a network IO format to send it to Veritas HyperScale storage
  service. This library (libqnio) has been open sourced and is available on
  github here: https://github.com/MittalAshish/libqnio

Ashish Mittal (2):
  block/vxhs.c: Add support for a new block device type called "vxhs"
  block/vxhs.c: Add qemu-iotests for new block device type "vxhs"

 block/Makefile.objs  |   2 +
 block/trace-events   |  21 ++
 block/vxhs.c | 689 +++
 configure|  41 +++
 qapi/block-core.json |  21 +-
 tests/qemu-iotests/common|   6 +
 tests/qemu-iotests/common.config |  13 +
 tests/qemu-iotests/common.filter |   1 +
 tests/qemu-iotests/common.rc |  19 ++
 9 files changed, 811 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v3] block/vxhs: Add Veritas HyperScale VxHS block device support

[ashish mittal]

On Wed, Nov 2, 2016 at 7:13 AM, Eric Blake  wrote:
> On 11/02/2016 04:57 AM, Kevin Wolf wrote:
 IMHO it should allow use of UNIX sockets, as its possible to have
 SSH setup a tunnel to a IP server, and expose the endpoint via a
 UNIX socket. So even if your reference server only supports IPv4,
 users can conceivably connect with any sockets protocol.

>>>
>>> This is not a use-case that we have in mind right now, but a very fair
>>> point! Kind of like accessing the audio/video streams remotely from my
>>> Raspberry Pi over ssh. Is it OK if we target this in a future patch
>>> after proper review/testing?
>>
>> No, going from InetSocketAddress to SocketAddress changes the API in an
>> incompatible way (previously working blockdev-add commands would stop
>> working), so we must decide now before the API is introduced.
>
> Going from InetSocketAddress to SocketAddress is a pain because
> SocketAddress is a 'simple union' (which really means extra nesting on
> the wire).  I'd really rather that blockdev-add move towards the
> GlusterServer style of a 'flat union', which CAN be done in a
> backwards-compatible manner.  Or, with examples:
>
> InetServer alone:
> { "host": "foo", "port": "1000" }
>
> SocketAddress:
> { "type": "inet", "data": { "host": "foo", "port": "1000" } }
>
> GlusterServer:
> { "type": "tcp", "host": "foo", "port": "1000" }
>
> For comparison, we wrote NFSServer with an eye towards possible
> expansion into a flat version (more like GlusterServer than SocketAddress).
>
> NFSServer:
> { "type": "inet", "host": "foo" }
>

So do I still need to add the UNIX sockets (go from InetSocketAddress
to SocketAddress)?
If yes, would it be OK to implement just the interface right now (not
the backend) and issue a message like "UNIX sockets are not yet
supported"?

>
> [side note: Uggh - why did we let gluster go into 2.7 with 'tcp' instead
> of 'inet' :( I'm wondering if we want gluster in 2.8 to recognize 'inet'
> as a synonym for 'tcp'; I could propose a patch if we think it is
> worthwhile]
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-devel] [PATCH v5] block/vxhs.c Add support for a new block device type called "vxhs"

[ashish mittal]

Done!

All builds are fine, but I have no way of knowing if that fixed the
issue! Please let me know if you still see a problem.

[root@rhel72ga-build-upstream qemu] 2016-11-01 09:46:06$ make
  CC  block/vxhs.o
  LINKqemu-nbd
  LINKqemu-img
  LINKqemu-io
  LINKaarch64-softmmu/qemu-system-aarch64
  LINKalpha-softmmu/qemu-system-alpha
  LINKarm-softmmu/qemu-system-arm
  LINKcris-softmmu/qemu-system-cris
  LINKi386-softmmu/qemu-system-i386
  LINKlm32-softmmu/qemu-system-lm32
  LINKm68k-softmmu/qemu-system-m68k
  LINKmicroblazeel-softmmu/qemu-system-microblazeel
  LINKmicroblaze-softmmu/qemu-system-microblaze
  LINKmips64el-softmmu/qemu-system-mips64el
  LINKmips64-softmmu/qemu-system-mips64
  LINKmipsel-softmmu/qemu-system-mipsel
  LINKmips-softmmu/qemu-system-mips
  LINKmoxie-softmmu/qemu-system-moxie
  LINKor32-softmmu/qemu-system-or32
  LINKppc64-softmmu/qemu-system-ppc64
  LINKppcemb-softmmu/qemu-system-ppcemb
  LINKppc-softmmu/qemu-system-ppc
  LINKs390x-softmmu/qemu-system-s390x
  LINKsh4eb-softmmu/qemu-system-sh4eb
  LINKsh4-softmmu/qemu-system-sh4
  LINKsparc64-softmmu/qemu-system-sparc64
  LINKsparc-softmmu/qemu-system-sparc
  LINKtricore-softmmu/qemu-system-tricore
  LINKunicore32-softmmu/qemu-system-unicore32
  LINKx86_64-softmmu/qemu-system-x86_64
  LINKxtensaeb-softmmu/qemu-system-xtensaeb
  LINKxtensa-softmmu/qemu-system-xtensa
[root@rhel72ga-build-upstream qemu] 2016-11-01 09:48:13$

Thanks!

On Tue, Nov 1, 2016 at 9:27 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Tue, Nov 01, 2016 at 09:09:31AM -0700, ashish mittal wrote:
>> Hi Daniel,
>>
>> Thanks for pointing that out. I had done a fresh configure and make
>> before submitting the patch. I am somehow not able to reproduce that
>> error!
>>
>> Checked out fresh source just now and tried again, still no luck.
>>
>> [root@rhel72ga-build-upstream qemu] 2016-11-01 08:51:15$ git apply
>> ~/qemu/31Oct2016/0001-block-vxhs.c-Add-support-for-a-new-block-device-type.patch
>> [root@rhel72ga-build-upstream qemu] 2016-11-01 08:51:31$ ./configure
>> --enable-vxhs
>> Install prefix/usr/local
>> BIOS directory/usr/local/share/qemu
>> binary directory  /usr/local/bin
>> library directory /usr/local/lib
>> module directory  /usr/local/lib/qemu
>> libexec directory /usr/local/libexec
>> include directory /usr/local/include
>> config directory  /usr/local/etc
>> local state directory   /usr/local/var
>> Manual directory  /usr/local/share/man
>> ELF interp prefix /usr/gnemul/qemu-%M
>> Source path   /root/qemu_second_buildarea/qemu
>> C compilercc
>> Host C compiler   cc
>> C++ compiler  c++
>> Objective-C compiler cc
>> ARFLAGS   rv
>> CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
>> QEMU_CFLAGS   -I/usr/include/pixman-1-Werror
>> -DHAS_LIBSSH2_SFTP_FSYNC -pthread -I/usr/include/glib-2.0
>> -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16
>> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings
>> -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv
>> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs
>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers
>> -Wold-style-declaration -Wold-style-definition -Wtype-limits
>> -fstack-protector-strong   -I/usr/include/p11-kit-1
>> -I/usr/include/libpng15   -I/usr/include/spice-server
>> -I/usr/include/cacard -I/usr/include/glib-2.0
>> -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1
>> -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1
>> -I/usr/include/cacard -I/usr/include/nss3 -I/usr/include/nspr4
>> -I/usr/include/libusb-1.0
>> LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
>> make  make
>> install   install
>> pythonpython -B
>> smbd  /usr/sbin/smbd
>> module supportno
>> host CPU  x86_64
>> host big endian   no
>> target listaarch64-softmmu alpha-softmmu arm-softmmu
>> cris-softmmu i386-softmmu lm32-softmmu m68k-softmmu
>> microblazeel-softmmu microblaze-softmmu mips64el-softmmu
>> mips64-softmmu mipsel-softmmu mips-softmmu moxie-softmmu or32-softmmu
>> ppc64-softmmu ppcemb-softmmu ppc-softmmu s390x-softmmu sh4eb-softmmu
>> sh4-softmmu sparc64-softmmu sparc-softmmu tricore-softmmu
>> unicore32-softmmu x86_64-softmmu xtensaeb-softmmu xtensa-softmmu
>> aarch64-linux-user alpha-linux-user armeb-linux-user arm-linux-user
&

Re: [Qemu-devel] [PATCH v5] block/vxhs.c Add support for a new block device type called "vxhs"

[ashish mittal]

snappy supportyes
bzip2 support no
NUMA host support yes
tcmalloc support  no
jemalloc support  no
avx2 optimization yes
replication support yes
VxHS block device yes
[root@rhel72ga-build-upstream qemu] 2016-11-01 08:52:22$ echo $?
0

Am I missing some step here?
Should I just add #include  to qnio_api.h ?
Doing a build on all platforms right now.

Regards,
Ashish

On Tue, Nov 1, 2016 at 2:07 AM, Daniel P. Berrange <berra...@redhat.com> wrote:
> On Mon, Oct 31, 2016 at 11:34:58PM -0700, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded
>> from:
>> https://github.com/MittalAshish/libqnio.git
>
> Your patch is unable to compile against this, because the qnio_api.h
> header is using a "struct iovec" type in its APIs, but not including
> any header to define that type. This causes configure to fail
>
> cc -Werror -DHAS_LIBSSH2_SFTP_FSYNC -pthread -I/usr/include/glib-2.0 
> -I/usr/lib64/glib-2.0/include -D_GNU_SOURCE -fPIE -DPIE -m64 -mcx16 
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
> -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
> -fno-strict-aliasing -fno-common -fwrapv 
> -I/home/berrange/src/external/libqnio/src -Wendif-labels 
> -Wno-shift-negative-value -Wmissing-include-dirs -Wempty-body 
> -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self 
> -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition 
> -Wtype-limits -fstack-protector-strong -I/usr/include/p11-kit-1 
> -I/usr/include/libpng16 -I/usr/include/cacard -I/usr/include/nss3 
> -I/usr/include/nspr4 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include 
> -I/usr/include/libusb-1.0 -o config-temp/qemu-conf.exe 
> config-temp/qemu-conf.c -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
> -L/home/berrange/src/external/libqnio/src -lqnio
> In file included from config-temp/qemu-conf.c:2:0:
> /home/berrange/src/external/libqnio/src/qnio/qnio_api.h:186:54: error: 
> ‘struct iovec’ declared inside parameter list will not be visible outside of 
> this definition or declaration [-Werror]
>  int32_t iio_writev(void *apictx, int32_t rfd, struct iovec *iov, int iovcnt,
>   ^
> /home/berrange/src/external/libqnio/src/qnio/qnio_api.h:189:53: error: 
> ‘struct iovec’ declared inside parameter list will not be visible outside of 
> this definition or declaration [-Werror]
>  int32_t iio_readv(void *apictx, int32_t rfd, struct iovec *iov, int iovcnt,
>  ^
> cc1: all warnings being treated as errors
>
>
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

Re: [Qemu-devel] [PATCH v4] This patch adds support for a new block device type called "vxhs".

[ashish mittal]

Thanks Jeff! All of the review comments have been addressed in v5.

On Mon, Oct 31, 2016 at 2:55 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Fri, Oct 28, 2016 at 11:47:11PM -0700, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using the JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
>> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk_id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using the URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v4 changelog:
>> (1) Incorporated v3 review comments on QAPI changes.
>> (2) Added refcounting for device open/close.
>> Free library resources on last device close.
>>
>> v3 changelog:
>> (1) Added QAPI schema for the VxHS driver.
>>
>> v2 changelog:
>> (1) Changes done in response to v1 comments.
>>
>> TODO:
>> (1) Add qemu-iotest
>>
>>  block/Makefile.objs  |   2 +
>>  block/trace-events   |  21 ++
>>  block/vxhs.c | 703 
>> +++
>>  configure|  41 +++
>>  qapi/block-core.json |  20 +-
>>  5 files changed, 785 insertions(+), 2 deletions(-)
>>  create mode 100644 block/vxhs.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 67a036a..58313a2 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
>>  block-obj-$(CONFIG_CURL) += curl.o
>>  block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>> @@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
>>  rbd.o-libs := $(RBD_LIBS)
>>  gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
>>  gluster.o-libs := $(GLUSTERFS_LIBS)
>> +vxhs.o-libs:= $(VXHS_LIBS)
>>  ssh.o-cflags   := $(LIBSSH2_CFLAGS)
>>  ssh.o-libs := $(LIBSSH2_LIBS)
>>  archipelago.o-libs := $(ARCHIPELAGO_LIBS)
>> diff --git a/block/trace-events b/block/trace-events
>> index aff8a96..5cb8089 100644
>> --- a/block/trace-events
>> +++ b/block/trace-events
>> @@ -113,3 +113,24 @@ qed_aio_write_data(void *s, void *acb, int ret, 
>> uint64_t offset, size_t len) "s
>>  qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t 
>> len) "s %p acb %p ret %d offset %"PRIu64" len %zu"
>> +
>> +# block/vxhs.c
>> +vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
>> +vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
>> +vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o 
>> %d, %d"
>> +vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno 
>> %d"
>> +vxhs_open_fail(int ret) "Could not open the device. Error = %d"
>> +vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
>> Error=%d"
>> +vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
>> +vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
>> *acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = 
>> %lu offset = %lu ACB = %p. Error = %d, errno = %d"
>> +vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat 
>> ioctl failed, ret = %d, errno = %d"
>> +vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
>> ioctl returned size %lu&

[Qemu-devel] [PATCH v5] block/vxhs.c Add support for a new block device type called "vxhs"

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded
from:
https://github.com/MittalAshish/libqnio.git

Sample command line using the JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
-vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk-id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using the URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v5 changelog:
(1) Incorporated v4 review comments.

v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

TODO:
(1) Add qemu-iotest

 block/Makefile.objs  |   2 +
 block/trace-events   |  21 ++
 block/vxhs.c | 689 +++
 configure|  41 +++
 qapi/block-core.json |  22 +-
 5 files changed, 772 insertions(+), 3 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index aff8a96..5cb8089 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -113,3 +113,24 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk-id from json %s"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..8913e8f
--- /dev/null
+++ b/block/vxhs.c
@@ -0,0 +1,689 @@
+/*
+ * QEMU Block driver for Veritas HyperScale (VxHS)
+ *

Re: [Qemu-devel] [PATCH v3] block/vxhs: Add Veritas HyperScale VxHS block device support

[ashish mittal]

>> >> +##
>> >> +{ 'struct': 'BlockdevOptionsVxHS',
>> >> +  'data': { 'vdisk_id': 'str',
>> >> +'server': 'InetSocketAddress' } }
>> >
>> > Is there any way to use a Unix socket, or is this server ONLY accessible
>> > over IPv4/IPv6?
>> >
>>
>> Right now we support IPv4 only.
>
> IMHO it should allow use of UNIX sockets, as its possible to have
> SSH setup a tunnel to a IP server, and expose the endpoint via a
> UNIX socket. So even if your reference server only supports IPv4,
> users can conceivably connect with any sockets protocol.
>

This is not a use-case that we have in mind right now, but a very fair
point! Kind of like accessing the audio/video streams remotely from my
Raspberry Pi over ssh. Is it OK if we target this in a future patch
after proper review/testing?

-Ashish

Re: [Qemu-devel] [PATCH v4] This patch adds support for a new block device type called "vxhs".

[ashish mittal]

Thanks! Will fix these in the next patch!

On Mon, Oct 31, 2016 at 12:05 PM, Eric Blake <ebl...@redhat.com> wrote:
> On 10/29/2016 01:47 AM, Ashish Mittal wrote:
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>
> For now, just an interface review:
>
>>
>> Sample command line using the JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
>> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk_id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
>
> This doesn't match the code, which spelled it "vdisk-id".
>
>> "server":{"host":"172.172.17.4","port":""}}'
>>
>> Sample command line using the URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>
>> +++ b/qapi/block-core.json
>> @@ -1704,12 +1704,13 @@
>>  # @host_device, @host_cdrom: Since 2.1
>>  # @gluster: Since 2.7
>>  # @nbd: Since 2.8
>> +# @vxhs: Since 2.8
>
> Please rebase on top of the latest block pull request:
> https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg08089.html
>
> In particular, your patch should look something like:
>
> -# @nbd, @nfs, @replication, @ssh: Since 2.8
> +# @nbd, @nfs, @replication, @ssh, @vxhs: Since 2.8
>
>>  #
>>  # Since: 2.0
>>  ##
>>  { 'enum': 'BlockdevDriver',
>>'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
>> -'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom',
>> +'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'vxhs',
>>  'host_device', 'http', 'https', 'luks', 'nbd', 'null-aio',
>>  'null-co', 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw',
>>  'replication', 'tftp', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
>
> Please keep this enum sorted alphabetically (your entry should be after
> vvfat, not in the random middle).
>
>> @@ -2238,6 +2239,20 @@
>>  '*export': 'str',
>>  '*tls-creds': 'str' } }
>>
>> +# @BlockdevOptionsVxHS
>
> Missing a leading ## marker.
>
>> +#
>> +# Driver specific block device options for VxHS
>> +#
>> +# @vdisk-id:UUID of VxHS volume
>> +#
>> +# @server:  vxhs server IP, port
>> +#
>> +# Since: 2.8
>> +##
>> +{ 'struct': 'BlockdevOptionsVxHS',
>> +  'data': { 'vdisk-id': 'str',
>> +'server': 'InetSocketAddress' } }
>> +
>>  ##
>>  # @BlockdevOptions
>>  #
>> @@ -2302,7 +2317,8 @@
>>'vhdx':   'BlockdevOptionsGenericFormat',
>>'vmdk':   'BlockdevOptionsGenericCOWFormat',
>>'vpc':'BlockdevOptionsGenericFormat',
>> -  'vvfat':  'BlockdevOptionsVVFAT'
>> +  'vvfat':  'BlockdevOptionsVVFAT',
>> +  'vxhs':   'BlockdevOptionsVxHS'
>>} }
>>
>>  ##
>>
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

[Qemu-devel] [PATCH v4] This patch adds support for a new block device type called "vxhs".

[Ashish Mittal]

Source code for the qnio library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using the JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
-vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk_id":"c3e9095a-a5ee-4dce-afeb-2a59fb387410",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using the URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/c6718f6b-0401-441d-a8c3-1f0064d75ee0

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v4 changelog:
(1) Incorporated v3 review comments on QAPI changes.
(2) Added refcounting for device open/close.
Free library resources on last device close.

v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

TODO:
(1) Add qemu-iotest

 block/Makefile.objs  |   2 +
 block/trace-events   |  21 ++
 block/vxhs.c | 703 +++
 configure|  41 +++
 qapi/block-core.json |  20 +-
 5 files changed, 785 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index aff8a96..5cb8089 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -113,3 +113,24 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk-id from json %s"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..d95be11
--- /dev/null
+++ b/block/vxhs.c
@@ -0,0 +1,703 @@
+/*
+ * QEMU Block driver for Veritas HyperScale (VxHS)
+ *
+ * This work is licensed under the terms of 

Re: [Qemu-devel] [PATCH v3] block/vxhs: Add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Eric,

On Fri, Oct 28, 2016 at 7:44 AM, Eric Blake <ebl...@redhat.com> wrote:
> On 10/28/2016 02:44 AM, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the qnio library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using the JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
>> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>> -msg timestamp=on
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}",
>> "server":{"host":"172.172.17.4","port":""}}'
>
> Are the {} really part of vdisk_id?  That's not a usual portion of a
> UUID.  And it should probably be spelled vdisk-id.
>

The {} are not required for qemu/libqnio or the test server. It is
just some artifact that our openstack components need. Then test
server does not need {} at all. I will remove it from the sample
command syntax in v4.

> It's a shame that port is a string and not an int, but it matches
> existing practice, so that part is correct.
>
>>
>> Sample command line using the URI syntax:
>> qemu-img convert -f raw -O raw -n
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>
> The URI syntax is potentially harder to use when compared to structured
> usage (Kevin's work to add -blockdev, which depends on the QAPI definition).
>

We decided to keep both after discussion with other reviewers. Do
qemu-img and qemu-io binaries support json syntax similar to
qemu-system-x86_64?

>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v3 changelog:
>> (1) Added QAPI schema for the VxHS driver.
>>
>
> For this email, I'm focusing just on the QAPI.
>
>> +++ b/qapi/block-core.json
>> @@ -1708,7 +1708,7 @@
>>  ##
>>  { 'enum': 'BlockdevDriver',
>>'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
>> -'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom',
>> +'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'vxhs',
>>  'host_device', 'http', 'https', 'luks', 'null-aio', 'null-co',
>
> Missing mention of the new enum value in the comments above.
>

Will fix in v4.

>>  'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw',
>>   'replication', 'tftp', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
>> @@ -2220,6 +2220,21 @@
>>'data': { 'filename': 'str' } }
>>
>>  ##
>> +# @BlockdevOptionsVxHS
>> +#
>> +# Driver specific block device options for VxHS
>> +#
>> +# @vdisk_id:UUID of VxHS volume
>
> Again, this should be vdisk-id.
>
Will fix in v4.

>> +#
>> +# @server:  vxhs server IP, port
>> +#
>> +# Since: 2.7
>
> You missed 2.7; this should be 2.8 (if you make soft freeze) or even 2.9.
>
Will fix in v4.

>> +##
>> +{ 'struct': 'BlockdevOptionsVxHS',
>> +  'data': { 'vdisk_id': 'str',
>> +'server': 'InetSocketAddress' } }
>
> Is there any way to use a Unix socket, or is this server ONLY accessible
> over IPv4/IPv6?
>

Right now we support IPv4 only.

>
>> +
>> +##
>>  # @BlockdevOptions
>>  #
>>  # Options for creating a block device.  Many options are available for all
>> @@ -2283,7 +2298,8 @@
>>'vhdx':   'BlockdevOptionsGenericFormat',
>>'vmdk':   'BlockdevOptionsGenericCOWFormat',
>>'vpc':'BlockdevOptionsGenericFormat',
>> -  'vvfat':  'BlockdevOptionsVVFAT'
>> +  'vvfat':  'BlockdevOptionsVVFAT',
>> +  'vxhs':   'BlockdevOptionsVxHS'
>>} }
>>
>>  ##
>>
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-devel] [PATCH v2] block/vxhs: Add Veritas HyperScale VxHS block device support

[ashish mittal]

I have submitted v3 with the QAPI schema changes.

Regards,
Ashish

On Thu, Oct 27, 2016 at 2:38 AM, ashish mittal <ashmit...@gmail.com> wrote:
> Hi Daniel,
>
>>> >I think this version still does not address Daniel's concerns regarding a
>>> >QAPI schema for vxhs.
>>>
>>> We are working on QAPI schema changes and will submit them in separate
>>> patch.
>>
>> They should be part of the main patch, not a separate patch as they are
>> a pre-requisite for any new block driver.
>>
>
> I guess what Madhav meant was that this patch (v2) is missing the QAPI
> changes, and we will provide those in the next version ASAP.
>
> Regards,
> Ashish
>
> On Thu, Oct 27, 2016 at 12:29 AM, Daniel P. Berrange
> <berra...@redhat.com> wrote:
>> On Wed, Oct 26, 2016 at 03:23:05PM -0700, Ashish Mittal wrote:
>>> This patch adds support for a new block device type called "vxhs".
>>> Source code for the library that this code loads can be downloaded from:
>>> https://github.com/MittalAshish/libqnio.git
>>>
>>> Sample command line using JSON syntax:
>>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us 
>>> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>>> timestamp=on 
>>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""}]}'
>>>
>>> Sample command line using URI syntax:
>>> qemu-img convert -f raw -O raw -n 
>>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>>
>>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>>> ---
>>> v2 changelog:
>>> (1) Changes done in response to v1 comments.
>>
>> Yet again you've ignored the request to include the QAPI schema changes
>>
>>>  block/Makefile.objs |   2 +
>>>  block/trace-events  |  22 ++
>>>  block/vxhs.c| 682 
>>> 
>>>  configure   |  41 
>>>  4 files changed, 747 insertions(+)
>>>  create mode 100644 block/vxhs.c
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org  -o- http://virt-manager.org :|
>> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

[Qemu-devel] [PATCH v3] block/vxhs: Add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the qnio library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using the JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us
-vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}",
"server":{"host":"172.172.17.4","port":""}}'

Sample command line using the URI syntax:
qemu-img convert -f raw -O raw -n
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v3 changelog:
(1) Added QAPI schema for the VxHS driver.

v2 changelog:
(1) Changes done in response to v1 comments.

TODO:
(1) Add qemu-iotest
(2) We need to be able to free all resources once we close the last vxhs drive.

 block/Makefile.objs  |   2 +
 block/trace-events   |  21 ++
 block/vxhs.c | 669 +++
 configure|  41 
 qapi/block-core.json |  20 +-
 5 files changed, 751 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..94249ee 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,24 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk_id from json %s"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..08ad681
--- /dev/null
+++ b/block/vxhs.c
@@ -0,0 +1,669 @@
+/*
+ * QEMU Block driver for Veritas HyperScale (VxHS)
+ *
+ * This work is licensed under the terms of th

Re: [Qemu-devel] [PATCH v2] block/vxhs: Add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Daniel,

>> >I think this version still does not address Daniel's concerns regarding a
>> >QAPI schema for vxhs.
>>
>> We are working on QAPI schema changes and will submit them in separate
>> patch.
>
> They should be part of the main patch, not a separate patch as they are
> a pre-requisite for any new block driver.
>

I guess what Madhav meant was that this patch (v2) is missing the QAPI
changes, and we will provide those in the next version ASAP.

Regards,
Ashish

On Thu, Oct 27, 2016 at 12:29 AM, Daniel P. Berrange
<berra...@redhat.com> wrote:
> On Wed, Oct 26, 2016 at 03:23:05PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
>> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>> timestamp=on 
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""}]}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n 
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v2 changelog:
>> (1) Changes done in response to v1 comments.
>
> Yet again you've ignored the request to include the QAPI schema changes
>
>>  block/Makefile.objs |   2 +
>>  block/trace-events  |  22 ++
>>  block/vxhs.c| 682 
>> 
>>  configure   |  41 
>>  4 files changed, 747 insertions(+)
>>  create mode 100644 block/vxhs.c
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://entangle-photo.org   -o-http://search.cpan.org/~danberr/ :|

[Qemu-devel] [PATCH v2] block/vxhs: Add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v2 changelog:
(1) Changes done in response to v1 comments.

 block/Makefile.objs |   2 +
 block/trace-events  |  22 ++
 block/vxhs.c| 682 
 configure   |  41 
 4 files changed, 747 insertions(+)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..aea97cb 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,25 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk_id from json %s"
+vxhs_qemu_init_numservers(int num_servers) "Number of servers passed = %d"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..0facb33
--- /dev/null
+++ b/block/vxhs.c
@@ -0,0 +1,682 @@
+/*
+ * QEMU Block driver for Veritas HyperScale (VxHS)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#incl

[Qemu-devel] [PATCH v1] block/vxhs: Add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
 block/Makefile.objs |   2 +
 block/trace-events  |  22 ++
 block/vxhs.c| 736 
 configure   |  41 +++
 4 files changed, 801 insertions(+)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..58313a2 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..aea97cb 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,25 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. 
Error=%d"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu 
offset = %lu ACB = %p. Error = %d, errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl 
failed, ret = %d, errno = %d"
+vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat 
ioctl returned size %lu"
+vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on 
host-ip %s"
+vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s"
+vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld"
+vxhs_parse_uri_filename(const char *filename) "URI passed via 
bdrv_parse_filename %s"
+vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk_id from json %s"
+vxhs_qemu_init_numservers(int num_servers) "Number of servers passed = %d"
+vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port 
%d"
+vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to 
BDRVVXHSState"
+vxhs_qemu_init_filename(const char *filename) "Filename passed as %s"
+vxhs_close(char *vdisk_guid) "Closing vdisk %s"
diff --git a/block/vxhs.c b/block/vxhs.c
new file mode 100644
index 000..97fb804
--- /dev/null
+++ b/block/vxhs.c
@@ -0,0 +1,736 @@
+/*
+ * QEMU Block driver for Veritas HyperScale (VxHS)
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include 
+

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Checked in a test server to libqnio that allows to open, read, write
to a vxhs vdisk using the qemu-io binary.

Steps to run the test server:
(1) Touch a file in /tmp with the base-name of the vdisk ID to be opened.
touch /tmp/\{98f48eef-b62f-46ee-b6e3-ad48ffd9ad0a\}
(2) Start the test server. It listens on port  by default.
/path/to/qnio_server
(3) In another terminal, start the qemu-io binary and open the vdisk
qemu-io> open vxhs://127.0.0.1:/%7B98f48eef-b62f-46ee-b6e3-ad48ffd9ad0a%7D
(4) Now you can write and read data from the vdisk that is backed by a file.
qemu-io> writev -P 81 0 1k
qemu-io> read -v 0 1k

Following change would be needed to my last patch to allow opening of the vdisk:
$ git diff
diff --git a/block/vxhs.c b/block/vxhs.c
index 90a4343..d849a9b 100644
--- a/block/vxhs.c
+++ b/block/vxhs.c
@@ -1215,7 +1215,7 @@ static int vxhs_qemu_init(QDict *options,
BDRVVXHSState *s,
 }

 ret = vxhs_qnio_iio_open(cfd, of_vsa_addr, rfd, file_name);
-if (!ret) {
+if (ret) {
 error_setg(_err, "Failed qnio_iio_open");
 ret = -EIO;
 }

Will work on the qemu-iotests test suite next.

Regards,
Ashish

On Tue, Oct 4, 2016 at 9:02 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Wed, Sep 28, 2016 at 12:13:32PM +0100, Stefan Hajnoczi wrote:
>> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>
> [...]
>
>> > +/*
>> > + * This is called by QEMU when a flush gets triggered from within
>> > + * a guest at the block layer, either for IDE or SCSI disks.
>> > + */
>> > +static int vxhs_co_flush(BlockDriverState *bs)
>>
>> This is called from coroutine context, please add the coroutine_fn
>> function attribute to document this.
>>
>> > +{
>> > +BDRVVXHSState *s = bs->opaque;
>> > +int64_t size = 0;
>> > +int ret = 0;
>> > +
>> > +/*
>> > + * VDISK_AIO_FLUSH ioctl is a no-op at present and will
>> > + * always return success. This could change in the future.
>> > + */
>> > +ret = vxhs_qnio_iio_ioctl(s->qnio_ctx,
>> > +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd,
>> > +VDISK_AIO_FLUSH, , NULL, IIO_FLAG_SYNC);
>>
>> This function is not allowed to block.  It cannot do a synchronous
>> flush.  This line is misleading because the constant is called
>> VDISK_AIO_FLUSH, but looking at the library code I see it's actually a
>> synchronous call that ends up in a loop that sleeps (!) waiting for the
>> response.
>>
>> Please do an async flush and qemu_coroutine_yield() to return
>> control to QEMU's event loop.  When the flush completes you can
>> qemu_coroutine_enter() again to return from this function.
>>
>> > +
>> > +if (ret < 0) {
>> > +trace_vxhs_co_flush(s->vdisk_guid, ret, errno);
>> > +vxhs_close(bs);
>>
>> This looks unsafe.  Won't it cause double close() calls for s->fds[]
>> when bdrv_close() is called later?
>>
>
> Calling the close on a failed flush is a good idea, in my opinion.  However,
> to make it safe, bs->drv MUST be set to NULL after the call to vxhs_close().
> That will prevent double free / close calls,  and also fail out new I/O.
> (This is actually what the gluster driver does, for instance).
>
> Jeff
>
>
>

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Stefan, others,

Thank you for all the review comments.

On Wed, Sep 28, 2016 at 4:13 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>> +vxhs_bdrv_init(const char c) "Registering VxHS AIO driver%c"
>
> Why do several trace events have a %c format specifier at the end and it
> always takes a '.' value?
>
>> +#define QNIO_CONNECT_TIMOUT_SECS120
>
> This isn't used and there is a typo (s/TIMOUT/TIMEOUT/).  Can it be
> dropped?
>
>> +static int32_t
>> +vxhs_qnio_iio_ioctl(void *apictx, uint32_t rfd, uint32_t opcode, int64_t 
>> *in,
>> +void *ctx, uint32_t flags)
>> +{
>> +int   ret = 0;
>> +
>> +switch (opcode) {
>> +case VDISK_STAT:
>
> It seems unnecessary to abstract the iio_ioctl() constants and then have
> a switch statement to translate to the actual library constants.  It
> makes little sense since the flags argument already uses the library
> constants.  Just use the library's constants.
>
>> +ret = iio_ioctl(apictx, rfd, IOR_VDISK_STAT,
>> + in, ctx, flags);
>> +break;
>> +
>> +case VDISK_AIO_FLUSH:
>> +ret = iio_ioctl(apictx, rfd, IOR_VDISK_FLUSH,
>> + in, ctx, flags);
>> +break;
>> +
>> +case VDISK_CHECK_IO_FAILOVER_READY:
>> +ret = iio_ioctl(apictx, rfd, IOR_VDISK_CHECK_IO_FAILOVER_READY,
>> + in, ctx, flags);
>> +break;
>> +
>> +default:
>> +ret = -ENOTSUP;
>> +break;
>> +}
>> +
>> +if (ret) {
>> +*in = 0;
>
> Some callers pass in = NULL so this will crash.
>
> The naming seems wrong: this is an output argument, not an input
> argument.  Please call it "out_val" or similar.
>
>> +res = vxhs_reopen_vdisk(s, s->vdisk_ask_failover_idx);
>> +if (res == 0) {
>> +res = vxhs_qnio_iio_ioctl(s->qnio_ctx,
>> +  s->vdisk_hostinfo[s->vdisk_ask_failover_idx].vdisk_rfd,
>> +  VDISK_CHECK_IO_FAILOVER_READY, NULL, s, flags);
>
> Looking at iio_ioctl(), I'm not sure how this can ever work.  The fourth
> argument is NULL and iio_ioctl() will attempt *vdisk_size = 0 so this
> will crash.
>
> Do you have tests that exercise this code path?
>

You are right. This bug crept in to the FAILOVER path when I moved the
qnio shim code to qemu. Earlier code in libqnio did *in = 0 on a
per-case basis and skipped it for VDISK_CHECK_IO_FAILOVER_READY. I
will fix this.

We do thoroughly test these code paths, but the problem is that the
existing tests do not fully work with the new changes I am doing. I do
not yet have test case to test failover with the latest code. I do
frequently test using qemu-io (open a vdisk, read, write and re-read
to check written data) and also try to bring up an existing guest VM
using latest qemu-system-x86_64 binary to make sure I don't regress
main functionality. I did not however run these tests for v7 patch,
therefore some of the v7 changes do break the code. This patch had
some major code reconfiguration over v6, therefore my intention was to
just get a feel for whether the main code structure looks good.

A lot of changes have been proposed. I will discuss these with the
team and get back with inputs. I guess having a test framework is
really important at this time.

Regards,
Ashish

On Fri, Sep 30, 2016 at 1:36 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>
> The QEMU block driver should deal with BlockDriver<->libqnio integration
> and libqnio should deal with vxhs logic (network protocol, failover,
> etc).  Right now the vxhs logic is spread between both components.  If
> responsibilities aren't cleanly separated between QEMU and libqnio then
> I see no point in having libqnio.
>
> Failover code should move into libqnio so that programs using libqnio
> avoid duplicating the failover code.
>
> Similarly IIO_IO_BUF_SIZE/segments should be handled internally by
> libqnio so programs using libqnio do not duplicate this code.
>
> libqnio itself can be simplified significantly:
>
> The multi-threading is not necessary and adds complexity.  Right now
> there seem to be two reasons for multi-threading: shared contexts and
> the epoll thread.  Both can be elimina

Re: [Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

That makes perfect sense. I will try and follow this method now onwards. Thanks!

On Wed, Sep 28, 2016 at 7:18 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Tue, Sep 27, 2016 at 09:09:49PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
>> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>> timestamp=on 
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n 
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>
> Hi Ashish,
>
> You've received a lot of feedback to digest on your patch -- creating a
> whole new block driver can be difficult!
>
> If I may make a suggestion: it is usually more productive to address the
> feedback via email _before_ you code up and send out the next patch version,
> unless the comments are straightforward and don't need any discussion (many
> comments often don't).
>
> For instance, I appreciate your reply to the feedback on v6, but by the time
> it hit my inbox v7 was already there, so it more or less kills the
> discussion and starts the review cycle afresh.  Not a big deal, but overall
> it would probably be more productive if that reply was sent first, and we
> could discuss things like sleep, coroutines, caching, global ctx instances,
> etc... :) Those discussion might then help shape the next patch even more,
> and result in fewer iterations.
>
>
> Thanks, and happy hacking!
> -Jeff

Re: [Qemu-devel] [PATCH v6 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

On Tue, Sep 20, 2016 at 10:53 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Tue, Sep 20, 2016 at 06:07:45PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
>> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>> timestamp=on 
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n 
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>
> I could not get the configure to work without massaging it a bit, nor could
> I get QEMU to compile with the (presumably) latest version of libqnio from
> github.  I'm assuming there are some commits to libqnio that have not been
> pushed yet, between v5 and v6 of this RFC series?  (It's an RFC series, so
> that is fine, I am just curious because I was able to compile v5).
>
>

Yes some commits were pending. I checked them in the next day after
some cleanup. Latest patch v7 should build fine.

>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v6 changelog:
>> (1) Removed cJSON dependency out of the libqnioshim layer.
>> (2) Merged libqnioshim code into qemu vxhs driver proper.
>> Now qemu-vxhs code only links with libqnio.so.
>> (3) Replaced use of custom spinlocks with qemu_spin_lock.
>>
>> v5 changelog:
>> (1) Removed unused functions.
>> (2) Changed all qemu_ prefix for functions defined in libqnio and vxhs.c.
>> (3) Fixed memory leaks in vxhs_qemu_init() and on the close of vxhs device.
>> (4) Added upper bounds check on num_servers.
>> (5) Close channel fds whereever necessary.
>> (6) Changed vdisk_size to int64_t for 32-bit compilations.
>> (7) Added message to configure file to indicate if vxhs is enabled or not.
>>
>> v4 changelog:
>> (1) Reworked QAPI/JSON parsing.
>> (2) Reworked URI parsing as suggested by Kevin.
>> (3) Fixes per review comments from Stefan on v1.
>> (4) Fixes per review comments from Daniel on v3.
>>
>> v3 changelog:
>> (1) Implemented QAPI interface for passing VxHS block device parameters.
>>
>> v2 changelog:
>> (1) Removed code to dlopen library. We now check if libqnio is installed 
>> during
>> configure, and directly link with it.
>> (2) Changed file headers to mention GPLv2-or-later license.
>> (3) Removed unnecessary type casts and inlines.
>> (4) Removed custom tokenize function and modified code to use g_strsplit.
>> (5) Replaced malloc/free with g_new/g_free and removed code that checks for
>> memory allocation failure conditions.
>> (6) Removed some block ops implementations that were place-holders only.
>> (7) Removed all custom debug messages. Added new messages in 
>> block/trace-events
>> (8) Other miscellaneous corrections.
>>
>> v1 changelog:
>> (1) First patch submission for review comments.
>>
>>  block/Makefile.objs |2 +
>>  block/trace-events  |   47 ++
>>  block/vxhs.c| 1602 
>> +++
>>  block/vxhs.h|  221 +++
>>  configure   |   41 ++
>>  5 files changed, 1913 insertions(+)
>>  create mode 100644 block/vxhs.c
>>  create mode 100644 block/vxhs.h
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 55da626..bafb7c9 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
>>  block-obj-$(CONFIG_CURL) += curl.o
>>  block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>> @@ -37,6 +38,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
>>  rbd.o-libs := $(RBD_LIBS)
>>  gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
>>  gluster.o-libs := $(GLUSTERFS_LIBS)
>> +vxhs.o-

Re: [Qemu-devel] [PATCH v6 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

On Wed, Sep 21, 2016 at 8:03 AM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>
>
> On 21/09/2016 03:07, Ashish Mittal wrote:
>> +int32_t vxhs_qnio_iio_writev(void *qnio_ctx, uint32_t rfd, struct iovec 
>> *iov,
>> +int iovcnt, uint64_t offset,
>> +void *ctx, uint32_t flags);
>> +int32_t vxhs_qnio_iio_readv(void *qnio_ctx, uint32_t rfd, struct iovec *iov,
>> +int iovcnt, uint64_t offset,
>> +void *ctx, uint32_t flags);
>> +int32_t vxhs_qnio_iio_ioctl(void *apictx, uint32_t rfd, uint32_t opcode,
>> +int64_t *in, void *ctx,
>> +uint32_t flags);
>
> Since you have wrappers for this, please use less verbose arguments, such as
>
> - BDRVVXHSState *s instead of the void * (qnio_ctx = s->qnio_ctx)
>
> - int idx instead of uint32_t rfd (rfd = s->vdisk_hostinfo[idx].vdisk_rfd)
>
> - the QEMUIOVector * instead of the iov/iovcnt pair
>

I have hopefully got all of these. Please let me know if any others
need changed.

> Likewise I suggest adding a wrapper
>
> void vxhs_qnio_iio_close(BDRVVXHSState *s, int idx)
> {
> if (s->vdisk_hostinfo[idx].vdisk_rfd >= 0) {
> iio_devclose(s->qnio_ctx, 0, s->vdisk_hostinfo[i].vdisk_rfd);
> s->vdisk_hostinfo[i].vdisk_rfd = -1;
> }
>
>
> if (s->vdisk_hostinfo[i].qnio_cfd >= 0) {
> iio_close(s->qnio_ctx, s->vdisk_hostinfo[i].qnio_cfd);
> s->vdisk_hostinfo[i].qnio_cfd = -1;
> }
> }
>

Done.

> (Likewise, iio_open/iio_devopen always happen in pairs and always build the
> openflame URI, so that's another candidate for a wrapper function).
>

Done.

> Also on the topic of closing:
>
> - there's no loop that initializes vdisk_rfd's and qnio_cfd's to -1.
>

vxhs_close() calling vxhs_qnio_iio_close() in a loop should initialize
rfd and cfd to -1 for all VXHS_MAX_HOSTS. Please let me know if I
missed this somewhere else.

> - here you are closing a vdisk_rfd twice:
>
> +if (s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd >= 0) {
> +iio_devclose(s->qnio_ctx, 0,
> +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd);
> +}
>
> because later you have another call to iio_devclose within
> "for (i = 0; i < VXHS_MAX_HOSTS; i++) {".  (It's also the only place
> that calls iio_devclose and not iio_close).
>

Fixed. Thanks!

>>
>> +if (s->vdisk_hostinfo[index].qnio_cfd < 0) {
>> +s->vdisk_hostinfo[index].qnio_cfd =
>> +iio_open(global_qnio_ctx, of_vsa_addr, 0);
>
> s->qnio_ctx seems to be always equal to global_qnio_ctx.  If that's
> how the API works that's fine, however please use s->qnio_ctx consistently.
> Initialize it early.
>

Yes, they are the same. I am now using global_qnio_ctx in
vxhs_qnio_iio_open() to avoid passing an additional arg. Let me know
if I should change this to get rid of the global variable.

>> + * Return Value:
>> + *  On Success : return VXHS_VECTOR_ALIGNED
>> + *  On Failure : return VXHS_VECTOR_NOT_ALIGNED.
>> + */
>> +int vxhs_is_iovector_read_aligned(struct iovec *iov, int niov, size_t 
>> sector)
>
> Pass a QEMUIOVector here too.
>

Changed in all places to pass QEMUIOVector * instead of iiv/niov pair.

>> +{
>> +int i;
>> +
>> +if (!iov || niov == 0) {
>> +return VXHS_VECTOR_ALIGNED;
>> +}
>
> Unnecessary "if".  The loop below never rolls if niov == 0, and you should
> never have "!iov && niov > 0".
>

Fixed.

>> +for (i = 0; i < niov; i++) {
>> +if (iov[i].iov_len % sector != 0) {
>> +return VXHS_VECTOR_NOT_ALIGNED;
>> +}
>> +}
>> +return VXHS_VECTOR_ALIGNED;
>> +}
>
> Please return just true or false.
>

Fixed.

>> +void *vxhs_convert_iovector_to_buffer(struct iovec *iov, int niov,
>> +  size_t sector)
>> +{
>> +void *buf = NULL;
>> +size_t size = 0;
>> +
>> +if (!iov || niov == 0) {
>> +return buf;
>> +}
>> +
>> +size = vxhs_calculate_iovec_size(iov, niov);
>
> If you have the QEMUIOVector, vxhs_calculate_iovec_size is just qiov->size.
>

Fixed. Thanks!

>> +buf = qemu_memalign(sector, size);
>> +if (!buf) {
>> +trace_vxhs_convert_iovector_to_buffer(size);
>> + 

[Qemu-devel] [PATCH v7 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v7 changelog:
(1) Got rid of the header file and most of function forward-declarations.
(2) Added wrappers for vxhs_qnio_iio_open() and vxhs_qnio_iio_close()
(3) Fixed a double close attempt of vdisk_rfd.
(4) Changed to pass QEMUIOVector * in a couple of functions instead of
individual structure members.
(5) Got rid of VXHS_VECTOR_ALIGNED/NOT_ALIGNED.
(6) Got rid of vxhs_calculate_iovec_size().
(7) Changed to use qemu_try_memalign().
(8) Got rid of unnecessary "else" conditions in a couple of places.
(9) Limited the filename case to pass a single URI in vxhs_parse_uri().
Users will have to use the host/port/vdisk_id syntax to specify
multiple host information.
(10) Inlined couple of macros including the ones for qemu_spin_unlock.
(11) Other miscellaneous changes.

v6 changelog:
(1) Removed cJSON dependency out of the libqnioshim layer.
(2) Merged libqnioshim code into qemu vxhs driver proper.
Now qemu-vxhs code only links with libqnio.so.
(3) Replaced use of custom spinlocks with qemu_spin_lock.

v5 changelog:
(1) Removed unused functions.
(2) Changed all qemu_ prefix for functions defined in libqnio and vxhs.c.
(3) Fixed memory leaks in vxhs_qemu_init() and on the close of vxhs device.
(4) Added upper bounds check on num_servers.
(5) Close channel fds whereever necessary.
(6) Changed vdisk_size to int64_t for 32-bit compilations.
(7) Added message to configure file to indicate if vxhs is enabled or not.

v4 changelog:
(1) Reworked QAPI/JSON parsing.
(2) Reworked URI parsing as suggested by Kevin.
(3) Fixes per review comments from Stefan on v1.
(4) Fixes per review comments from Daniel on v3.

v3 changelog:
(1) Implemented QAPI interface for passing VxHS block device parameters.

v2 changelog:
(1) Removed code to dlopen library. We now check if libqnio is installed during
configure, and directly link with it.
(2) Changed file headers to mention GPLv2-or-later license.
(3) Removed unnecessary type casts and inlines.
(4) Removed custom tokenize function and modified code to use g_strsplit.
(5) Replaced malloc/free with g_new/g_free and removed code that checks for
memory allocation failure conditions.
(6) Removed some block ops implementations that were place-holders only.
(7) Removed all custom debug messages. Added new messages in block/trace-events
(8) Other miscellaneous corrections.

v1 changelog:
(1) First patch submission for review comments.

 block/Makefile.objs |2 +
 block/trace-events  |   47 ++
 block/vxhs.c| 1645 +++
 configure   |   41 ++
 4 files changed, 1735 insertions(+)
 create mode 100644 block/vxhs.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 7d4031d..1861bb9 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -38,6 +39,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..44de452 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,50 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offse

Re: [Qemu-devel] [PATCH v6 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

libqnio source has been updated. qemu changes should now build properly.
https://github.com/MittalAshish/libqnio.git

I will work on the other suggestions and get back.

Regards,
Ashish


On Wed, Sep 21, 2016 at 8:03 AM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>
>
> On 21/09/2016 03:07, Ashish Mittal wrote:
>> +int32_t vxhs_qnio_iio_writev(void *qnio_ctx, uint32_t rfd, struct iovec 
>> *iov,
>> +int iovcnt, uint64_t offset,
>> +void *ctx, uint32_t flags);
>> +int32_t vxhs_qnio_iio_readv(void *qnio_ctx, uint32_t rfd, struct iovec *iov,
>> +int iovcnt, uint64_t offset,
>> +void *ctx, uint32_t flags);
>> +int32_t vxhs_qnio_iio_ioctl(void *apictx, uint32_t rfd, uint32_t opcode,
>> +int64_t *in, void *ctx,
>> +uint32_t flags);
>
> Since you have wrappers for this, please use less verbose arguments, such as
>
> - BDRVVXHSState *s instead of the void * (qnio_ctx = s->qnio_ctx)
>
> - int idx instead of uint32_t rfd (rfd = s->vdisk_hostinfo[idx].vdisk_rfd)
>
> - the QEMUIOVector * instead of the iov/iovcnt pair
>
> Likewise I suggest adding a wrapper
>
> void vxhs_qnio_iio_close(BDRVVXHSState *s, int idx)
> {
> if (s->vdisk_hostinfo[idx].vdisk_rfd >= 0) {
> iio_devclose(s->qnio_ctx, 0, s->vdisk_hostinfo[i].vdisk_rfd);
> s->vdisk_hostinfo[i].vdisk_rfd = -1;
> }
>
>
> if (s->vdisk_hostinfo[i].qnio_cfd >= 0) {
> iio_close(s->qnio_ctx, s->vdisk_hostinfo[i].qnio_cfd);
> s->vdisk_hostinfo[i].qnio_cfd = -1;
> }
> }
>
> (Likewise, iio_open/iio_devopen always happen in pairs and always build the
> openflame URI, so that's another candidate for a wrapper function).
>
> Also on the topic of closing:
>
> - there's no loop that initializes vdisk_rfd's and qnio_cfd's to -1.
>
> - here you are closing a vdisk_rfd twice:
>
> +if (s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd >= 0) {
> +iio_devclose(s->qnio_ctx, 0,
> +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd);
> +}
>
> because later you have another call to iio_devclose within
> "for (i = 0; i < VXHS_MAX_HOSTS; i++) {".  (It's also the only place
> that calls iio_devclose and not iio_close).
>
>>
>> +if (s->vdisk_hostinfo[index].qnio_cfd < 0) {
>> +s->vdisk_hostinfo[index].qnio_cfd =
>> +iio_open(global_qnio_ctx, of_vsa_addr, 0);
>
> s->qnio_ctx seems to be always equal to global_qnio_ctx.  If that's
> how the API works that's fine, however please use s->qnio_ctx consistently.
> Initialize it early.
>
>> + * Return Value:
>> + *  On Success : return VXHS_VECTOR_ALIGNED
>> + *  On Failure : return VXHS_VECTOR_NOT_ALIGNED.
>> + */
>> +int vxhs_is_iovector_read_aligned(struct iovec *iov, int niov, size_t 
>> sector)
>
> Pass a QEMUIOVector here too.
>
>> +{
>> +int i;
>> +
>> +if (!iov || niov == 0) {
>> +return VXHS_VECTOR_ALIGNED;
>> +}
>
> Unnecessary "if".  The loop below never rolls if niov == 0, and you should
> never have "!iov && niov > 0".
>
>> +for (i = 0; i < niov; i++) {
>> +if (iov[i].iov_len % sector != 0) {
>> +return VXHS_VECTOR_NOT_ALIGNED;
>> +}
>> +}
>> +return VXHS_VECTOR_ALIGNED;
>> +}
>
> Please return just true or false.
>
>> +void *vxhs_convert_iovector_to_buffer(struct iovec *iov, int niov,
>> +  size_t sector)
>> +{
>> +void *buf = NULL;
>> +size_t size = 0;
>> +
>> +if (!iov || niov == 0) {
>> +return buf;
>> +}
>> +
>> +size = vxhs_calculate_iovec_size(iov, niov);
>
> If you have the QEMUIOVector, vxhs_calculate_iovec_size is just qiov->size.
>
>> +buf = qemu_memalign(sector, size);
>> +if (!buf) {
>> +trace_vxhs_convert_iovector_to_buffer(size);
>> +errno = -ENOMEM;
>> +return NULL;
>> +}
>> +return buf;
>> +}
>> +
>
> This function should use qemu_try_memalign, not qemu_memalign.  But it is
> obviously not very well tested, because the !iov || niov == 0 case doesn't
> set errno and returns NULL.
>
> You should just use qemu_try_memalign(qiov->size, BDRV_S

Re: [Qemu-devel] [PATCH v6 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

On Tue, Sep 20, 2016 at 10:53 PM, Jeff Cody <jc...@redhat.com> wrote:
> On Tue, Sep 20, 2016 at 06:07:45PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
>> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>> timestamp=on 
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n 
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>
> I could not get the configure to work without massaging it a bit, nor could
> I get QEMU to compile with the (presumably) latest version of libqnio from
> github.  I'm assuming there are some commits to libqnio that have not been
> pushed yet, between v5 and v6 of this RFC series?  (It's an RFC series, so
> that is fine, I am just curious because I was able to compile v5).
>

That is correct. I will push the changes to libqnio tomorrow after
cleaning up the header files a bit. Will go over other comments also.

Thanks,
Ashish

>
>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v6 changelog:
>> (1) Removed cJSON dependency out of the libqnioshim layer.
>> (2) Merged libqnioshim code into qemu vxhs driver proper.
>> Now qemu-vxhs code only links with libqnio.so.
>> (3) Replaced use of custom spinlocks with qemu_spin_lock.
>>
>> v5 changelog:
>> (1) Removed unused functions.
>> (2) Changed all qemu_ prefix for functions defined in libqnio and vxhs.c.
>> (3) Fixed memory leaks in vxhs_qemu_init() and on the close of vxhs device.
>> (4) Added upper bounds check on num_servers.
>> (5) Close channel fds whereever necessary.
>> (6) Changed vdisk_size to int64_t for 32-bit compilations.
>> (7) Added message to configure file to indicate if vxhs is enabled or not.
>>
>> v4 changelog:
>> (1) Reworked QAPI/JSON parsing.
>> (2) Reworked URI parsing as suggested by Kevin.
>> (3) Fixes per review comments from Stefan on v1.
>> (4) Fixes per review comments from Daniel on v3.
>>
>> v3 changelog:
>> (1) Implemented QAPI interface for passing VxHS block device parameters.
>>
>> v2 changelog:
>> (1) Removed code to dlopen library. We now check if libqnio is installed 
>> during
>> configure, and directly link with it.
>> (2) Changed file headers to mention GPLv2-or-later license.
>> (3) Removed unnecessary type casts and inlines.
>> (4) Removed custom tokenize function and modified code to use g_strsplit.
>> (5) Replaced malloc/free with g_new/g_free and removed code that checks for
>> memory allocation failure conditions.
>> (6) Removed some block ops implementations that were place-holders only.
>> (7) Removed all custom debug messages. Added new messages in 
>> block/trace-events
>> (8) Other miscellaneous corrections.
>>
>> v1 changelog:
>> (1) First patch submission for review comments.
>>
>>  block/Makefile.objs |2 +
>>  block/trace-events  |   47 ++
>>  block/vxhs.c| 1602 
>> +++
>>  block/vxhs.h|  221 +++
>>  configure   |   41 ++
>>  5 files changed, 1913 insertions(+)
>>  create mode 100644 block/vxhs.c
>>  create mode 100644 block/vxhs.h
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 55da626..bafb7c9 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
>>  block-obj-$(CONFIG_CURL) += curl.o
>>  block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>> @@ -37,6 +38,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
>>  rbd.o-libs := $(RBD_LIBS)
>>  gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
>>  gluster.o-libs := $

Re: [Qemu-devel] [PATCH v4 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi,

I have submitted a new patch (V6) after fixing the problem of
libqnioshim referencing symbols from inside qemu vxhs.c code.

(1) libqnioshim code has now been brought inside vxhs.c
(2) vxhs.c now needs to link only with libqnio.so.
(3) libqnio.so does not have any unreferenced symbols coming from qemu
code. This can also be witnessed from the code in configure that
checks for the presence of libqnio. It does not have to provide any
empty function definitions (as before) for the test code to compile.

Have also fixed a couple of other points raised during review. Will
reply to pending questions soon.

Regards,
Ashish

On Thu, Sep 8, 2016 at 4:15 PM, ashish mittal <ashmit...@gmail.com> wrote:
>>> Yes, qemu_iio_ioctl(VDISK_AIO_FLUSH) is only a place-holder at present
>>> in case we later want to add some functionality to it. I have now
>>> added a comment to this affect to avoid any confusion.
>>>
>>
>> The problem is you don't know which version of the qnio library any given
>> QEMU binary will be using, since it is a shared library.  Future versions
>> may implement the flush ioctl as expressed above, in which case we may hide
>> a valid error.
>>
>> Am I correct in assuming that this call suppresses errors because an error
>> is returned for an unknown ioctl operation of VDISK_AIO_FLUSH?  If so, and
>> you want a placeholder here for flushing, you should go all the way and stub
>> out the underlying ioctl call to return success.  Then QEMU can at least
>> rely on the error return from the flush operation.
>>
>>
>
> I agree. Will change accordingly. I also completely agree on the
> discussion of libqnio calling a qemu function. Will check on the best
> way to resolve that and get back.
>
> Thanks,
> Ashish
>
>
> On Thu, Sep 8, 2016 at 7:00 AM, Jeff Cody <jc...@redhat.com> wrote:
>> On Wed, Sep 07, 2016 at 03:32:47PM -0700, ashish mittal wrote:
>>> Hi Jeff,
>>>
>>> Thank you for the comments and corrections.
>>>
>>> I have submitted a v5 patch after incorporating most of your review
>>> comments. Please find details in-line.
>>>
>>> Thanks,
>>> Ashish
>>>
>>> On Tue, Aug 30, 2016 at 10:35 AM, Jeff Cody <jc...@redhat.com> wrote:
>>> >
>>> > First-pass-over-the-code review:
>>> >
>>> > On Mon, Aug 22, 2016 at 11:56:30PM -0700, Ashish Mittal wrote:
>>> >> This patch adds support for a new block device type called "vxhs".
>>> >> Source code for the library that this code loads can be downloaded from:
>>> >> https://github.com/MittalAshish/libqnio.git
>>> >>
>>> >> Sample command line using JSON syntax:
>>> >> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us 
>>> >> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 
>>> >> -msg timestamp=on 
>>> >> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>>> >>
>>> >> Sample command line using URI syntax:
>>> >> qemu-img convert -f raw -O raw -n 
>>> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>>> >> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>> >>
>>> >
>>> > Is there a reference specification for vxhs available?  If so, the commit
>>> > message would be a good place to drop a link.
>>> >
>>>
>>> We don't have a reference specification doc yet. Although, I will
>>> shortly be adding a test program to libqnio that can be used to
>>> test/demo basic libqnio IO transfer.
>>>
>>> >> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>>> >> ---
>>> >> v3 changelog:
>>> >> =
>>> >> (1) Reworked QAPI/JSON parsing.
>>> >> (2) Reworked URI parsing as suggested by Kevin.
>>> >> (3) Fixes per review comments from Stefan on v1.
>>> >> (4) Fixes per review comments from Daniel on v3.
>>> >>
>>> >>  block/Makefile.objs |2 +
>>> >>  block/trace-events  |   41 ++
>>> >>  block/vxhs.c| 1304 
>>> >> +++
>>> >>  block

[Qemu-devel] [PATCH v6 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v6 changelog:
(1) Removed cJSON dependency out of the libqnioshim layer.
(2) Merged libqnioshim code into qemu vxhs driver proper.
Now qemu-vxhs code only links with libqnio.so.
(3) Replaced use of custom spinlocks with qemu_spin_lock.

v5 changelog:
(1) Removed unused functions.
(2) Changed all qemu_ prefix for functions defined in libqnio and vxhs.c.
(3) Fixed memory leaks in vxhs_qemu_init() and on the close of vxhs device.
(4) Added upper bounds check on num_servers.
(5) Close channel fds whereever necessary.
(6) Changed vdisk_size to int64_t for 32-bit compilations.
(7) Added message to configure file to indicate if vxhs is enabled or not.

v4 changelog:
(1) Reworked QAPI/JSON parsing.
(2) Reworked URI parsing as suggested by Kevin.
(3) Fixes per review comments from Stefan on v1.
(4) Fixes per review comments from Daniel on v3.

v3 changelog:
(1) Implemented QAPI interface for passing VxHS block device parameters.

v2 changelog:
(1) Removed code to dlopen library. We now check if libqnio is installed during
configure, and directly link with it.
(2) Changed file headers to mention GPLv2-or-later license.
(3) Removed unnecessary type casts and inlines.
(4) Removed custom tokenize function and modified code to use g_strsplit.
(5) Replaced malloc/free with g_new/g_free and removed code that checks for
memory allocation failure conditions.
(6) Removed some block ops implementations that were place-holders only.
(7) Removed all custom debug messages. Added new messages in block/trace-events
(8) Other miscellaneous corrections.

v1 changelog:
(1) First patch submission for review comments.

 block/Makefile.objs |2 +
 block/trace-events  |   47 ++
 block/vxhs.c| 1602 +++
 block/vxhs.h|  221 +++
 configure   |   41 ++
 5 files changed, 1913 insertions(+)
 create mode 100644 block/vxhs.c
 create mode 100644 block/vxhs.h

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 55da626..bafb7c9 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
@@ -37,6 +38,7 @@ rbd.o-cflags   := $(RBD_CFLAGS)
 rbd.o-libs := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs := $(GLUSTERFS_LIBS)
+vxhs.o-libs:= $(VXHS_LIBS)
 ssh.o-cflags   := $(LIBSSH2_CFLAGS)
 ssh.o-libs := $(LIBSSH2_LIBS)
 archipelago.o-libs := $(ARCHIPELAGO_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..b0098a7 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,50 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_bdrv_init(const char c) "Registering VxHS AIO driver%c"
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
Bailing out%c"
+vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) "Read/Write 
failed: error %d, reason %d, acb %p, segment %d"
+vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %

Re: [Qemu-devel] [PATCH v4 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

>> Yes, qemu_iio_ioctl(VDISK_AIO_FLUSH) is only a place-holder at present
>> in case we later want to add some functionality to it. I have now
>> added a comment to this affect to avoid any confusion.
>>
>
> The problem is you don't know which version of the qnio library any given
> QEMU binary will be using, since it is a shared library.  Future versions
> may implement the flush ioctl as expressed above, in which case we may hide
> a valid error.
>
> Am I correct in assuming that this call suppresses errors because an error
> is returned for an unknown ioctl operation of VDISK_AIO_FLUSH?  If so, and
> you want a placeholder here for flushing, you should go all the way and stub
> out the underlying ioctl call to return success.  Then QEMU can at least
> rely on the error return from the flush operation.
>
>

I agree. Will change accordingly. I also completely agree on the
discussion of libqnio calling a qemu function. Will check on the best
way to resolve that and get back.

Thanks,
Ashish


On Thu, Sep 8, 2016 at 7:00 AM, Jeff Cody <jc...@redhat.com> wrote:
> On Wed, Sep 07, 2016 at 03:32:47PM -0700, ashish mittal wrote:
>> Hi Jeff,
>>
>> Thank you for the comments and corrections.
>>
>> I have submitted a v5 patch after incorporating most of your review
>> comments. Please find details in-line.
>>
>> Thanks,
>> Ashish
>>
>> On Tue, Aug 30, 2016 at 10:35 AM, Jeff Cody <jc...@redhat.com> wrote:
>> >
>> > First-pass-over-the-code review:
>> >
>> > On Mon, Aug 22, 2016 at 11:56:30PM -0700, Ashish Mittal wrote:
>> >> This patch adds support for a new block device type called "vxhs".
>> >> Source code for the library that this code loads can be downloaded from:
>> >> https://github.com/MittalAshish/libqnio.git
>> >>
>> >> Sample command line using JSON syntax:
>> >> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us 
>> >> -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 
>> >> -msg timestamp=on 
>> >> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>> >>
>> >> Sample command line using URI syntax:
>> >> qemu-img convert -f raw -O raw -n 
>> >> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> >> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>> >>
>> >
>> > Is there a reference specification for vxhs available?  If so, the commit
>> > message would be a good place to drop a link.
>> >
>>
>> We don't have a reference specification doc yet. Although, I will
>> shortly be adding a test program to libqnio that can be used to
>> test/demo basic libqnio IO transfer.
>>
>> >> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> >> ---
>> >> v3 changelog:
>> >> =
>> >> (1) Reworked QAPI/JSON parsing.
>> >> (2) Reworked URI parsing as suggested by Kevin.
>> >> (3) Fixes per review comments from Stefan on v1.
>> >> (4) Fixes per review comments from Daniel on v3.
>> >>
>> >>  block/Makefile.objs |2 +
>> >>  block/trace-events  |   41 ++
>> >>  block/vxhs.c| 1304 
>> >> +++
>> >>  block/vxhs.h|  237 ++
>> >>  configure   |   50 ++
>> >>  5 files changed, 1634 insertions(+)
>> >>  create mode 100644 block/vxhs.c
>> >>  create mode 100644 block/vxhs.h
>> >>
>> >> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> >> index 2593a2f..5f83305 100644
>> >> --- a/block/Makefile.objs
>> >> +++ b/block/Makefile.objs
>> >> @@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
>> >>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>> >>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>> >>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>> >> +block-obj-$(CONFIG_VXHS) += vxhs.o
>> >>  block-obj-y += accounting.o dirty-bitmap.o
>> >>  block-obj-y += write-threshold.o
>> >>
>> >> @@ -43,3 +44,4 @@ block-obj-m+= dmg.o
>> >>  dmg.o-libs := $(BZIP2_LIBS)
>> >>  qcow.

Re: [Qemu-devel] vxhs caching behaviour (was: [PATCH v4 RFC] block/vxhs: Initial commit to add) Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Kevin,

By design, our writeback cache is on non-volatile SSD device. We do
async writes to this cache and also maintain a persistent index map of
the data written. This gives us the capability to recover write-back
cache if needed.

Thanks,
Ashish

On Thu, Sep 8, 2016 at 7:20 AM, Kevin Wolf  wrote:
> Am 08.09.2016 um 16:00 hat Jeff Cody geschrieben:
>> > >> +/*
>> > >> + * This is called by QEMU when a flush gets triggered from within
>> > >> + * a guest at the block layer, either for IDE or SCSI disks.
>> > >> + */
>> > >> +int vxhs_co_flush(BlockDriverState *bs)
>> > >> +{
>> > >> +BDRVVXHSState *s = bs->opaque;
>> > >> +uint64_t size = 0;
>> > >> +int ret = 0;
>> > >> +
>> > >> +ret = qemu_iio_ioctl(s->qnio_ctx,
>> > >> +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd,
>> > >> +VDISK_AIO_FLUSH, , NULL, IIO_FLAG_SYNC);
>> > >> +
>> > >> +if (ret < 0) {
>> > >> +/*
>> > >> + * Currently not handling the flush ioctl
>> > >> + * failure because of network connection
>> > >> + * disconnect. Since all the writes are
>> > >> + * commited into persistent storage hence
>> > >> + * this flush call is noop and we can safely
>> > >> + * return success status to the caller.
>> > >
>> > > I'm not sure I understand here.  Are you saying the qemu_iio_ioctl() call
>> > > above is a noop?
>> > >
>> >
>> > Yes, qemu_iio_ioctl(VDISK_AIO_FLUSH) is only a place-holder at present
>> > in case we later want to add some functionality to it. I have now
>> > added a comment to this affect to avoid any confusion.
>> >
>>
>> The problem is you don't know which version of the qnio library any given
>> QEMU binary will be using, since it is a shared library.  Future versions
>> may implement the flush ioctl as expressed above, in which case we may hide
>> a valid error.
>>
>> Am I correct in assuming that this call suppresses errors because an error
>> is returned for an unknown ioctl operation of VDISK_AIO_FLUSH?  If so, and
>> you want a placeholder here for flushing, you should go all the way and stub
>> out the underlying ioctl call to return success.  Then QEMU can at least
>> rely on the error return from the flush operation.
>
> So what's the story behind the missing flush command?
>
> Does the server always use something like O_SYNC, i.e. all potential
> write caches in the stack operate in a writethrough mode? So each write
> request is only completed successfully if it is ensured that the data is
> safe on disk rather than in a volatile writeback cache?
>
> As soon as any writeback cache can be involved (e.g. the kernel page
> cache or a volatile disk cache) and there is no flush command (a real
> one, not just stubbed), the driver is not operating correctly and
> therefore not ready for inclusion.
>
> So Ashish, can you tell us something about caching behaviour across the
> storage stack when vxhs is involved?
>
> Kevin

Re: [Qemu-devel] [PATCH v4 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Jeff,

Thank you for the comments and corrections.

I have submitted a v5 patch after incorporating most of your review
comments. Please find details in-line.

Thanks,
Ashish

On Tue, Aug 30, 2016 at 10:35 AM, Jeff Cody <jc...@redhat.com> wrote:
>
> First-pass-over-the-code review:
>
> On Mon, Aug 22, 2016 at 11:56:30PM -0700, Ashish Mittal wrote:
>> This patch adds support for a new block device type called "vxhs".
>> Source code for the library that this code loads can be downloaded from:
>> https://github.com/MittalAshish/libqnio.git
>>
>> Sample command line using JSON syntax:
>> ./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
>> cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
>> timestamp=on 
>> 'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
>>
>> Sample command line using URI syntax:
>> qemu-img convert -f raw -O raw -n 
>> /var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
>> vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D
>>
>
> Is there a reference specification for vxhs available?  If so, the commit
> message would be a good place to drop a link.
>

We don't have a reference specification doc yet. Although, I will
shortly be adding a test program to libqnio that can be used to
test/demo basic libqnio IO transfer.

>> Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
>> ---
>> v3 changelog:
>> =
>> (1) Reworked QAPI/JSON parsing.
>> (2) Reworked URI parsing as suggested by Kevin.
>> (3) Fixes per review comments from Stefan on v1.
>> (4) Fixes per review comments from Daniel on v3.
>>
>>  block/Makefile.objs |2 +
>>  block/trace-events  |   41 ++
>>  block/vxhs.c| 1304 
>> +++
>>  block/vxhs.h|  237 ++
>>  configure   |   50 ++
>>  5 files changed, 1634 insertions(+)
>>  create mode 100644 block/vxhs.c
>>  create mode 100644 block/vxhs.h
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 2593a2f..5f83305 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
>>  block-obj-$(CONFIG_GLUSTERFS) += gluster.o
>>  block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
>>  block-obj-$(CONFIG_LIBSSH2) += ssh.o
>> +block-obj-$(CONFIG_VXHS) += vxhs.o
>>  block-obj-y += accounting.o dirty-bitmap.o
>>  block-obj-y += write-threshold.o
>>
>> @@ -43,3 +44,4 @@ block-obj-m+= dmg.o
>>  dmg.o-libs := $(BZIP2_LIBS)
>>  qcow.o-libs:= -lz
>>  linux-aio.o-libs   := -laio
>> +vxhs.o-libs:= $(VXHS_LIBS)
>> diff --git a/block/trace-events b/block/trace-events
>> index 05fa13c..06c6d8c 100644
>> --- a/block/trace-events
>> +++ b/block/trace-events
>> @@ -114,3 +114,44 @@ qed_aio_write_data(void *s, void *acb, int ret, 
>> uint64_t offset, size_t len) "s
>>  qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
>> uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
>>  qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t 
>> len) "s %p acb %p ret %d offset %"PRIu64" len %zu"
>> +
>> +# block/vxhs.c
>> +vxhs_bdrv_init(const char c) "Registering VxHS AIO driver%c"
>> +vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
>> +vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
>> +vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
>> Bailing out%c"
>> +vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) 
>> "Read/Write failed: error %d, reason %d, acb %p, segment %d"
>> +vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %s, added acb %p to 
>> retry queue (5)"
>> +vxhs_iio_callback_chnlfail(int error) "QNIO channel failed, no i/o (%d)"
>> +vxhs_iio_callback_fail(int r, void *acb, int seg, uint64_t size, int err) " 
>> ALERT: reason = %d , acb = %p, acb->segments = %

[Qemu-devel] [PATCH v5 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---

v5 changelog:
(1) Removed unused functions.
(2) Changed all qemu_ prefix for functions defined in libqnio and vxhs.c.
(3) Fixed memory leaks in vxhs_qemu_init() and on the close of vxhs device.
(4) Added upper bounds check on num_servers.
(5) Close channel fds whereever necessary.
(6) Changed vdisk_size to int64_t for 32-bit compilations.
(7) Added message to configure file to indicate if vxhs is enabled or not.

v4 changelog:
(1) Reworked QAPI/JSON parsing.
(2) Reworked URI parsing as suggested by Kevin.
(3) Fixes per review comments from Stefan on v1.
(4) Fixes per review comments from Daniel on v3.

v3 changelog:
(1) Implemented QAPI interface for passing VxHS block device parameters.

v2 changelog:
(1) Removed code to dlopen library. We now check if libqnio is installed during
configure, and directly link with it.
(2) Changed file headers to mention GPLv2-or-later license.
(3) Removed unnecessary type casts and inlines.
(4) Removed custom tokenize function and modified code to use g_strsplit.
(5) Replaced malloc/free with g_new/g_free and removed code that checks for
memory allocation failure conditions.
(6) Removed some block ops implementations that were place-holders only.
(7) Removed all custom debug messages. Added new messages in block/trace-events
(8) Other miscellaneous corrections.

v1 changelog:
(1) First patch submission for review comments.

 block/Makefile.objs |2 +
 block/trace-events  |   42 ++
 block/vxhs.c| 1341 +++
 block/vxhs.h|  236 +
 configure   |   51 ++
 5 files changed, 1672 insertions(+)
 create mode 100644 block/vxhs.c
 create mode 100644 block/vxhs.h

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 2593a2f..5f83305 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 
@@ -43,3 +44,4 @@ block-obj-m+= dmg.o
 dmg.o-libs := $(BZIP2_LIBS)
 qcow.o-libs:= -lz
 linux-aio.o-libs   := -laio
+vxhs.o-libs:= $(VXHS_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..1e892cf 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,45 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_bdrv_init(const char c) "Registering VxHS AIO driver%c"
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
Bailing out%c"
+vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) "Read/Write 
failed: error %d, reason %d, acb %p, segment %d"
+vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %s, added acb %p to 
retry queue (5)"
+vxhs_iio_callback_chnlfail(int error) "QNIO channel failed, no i/o (%d)"
+vxhs_iio_callback_fail(int r, void *acb, int seg, uint64_t size, int err) " 
ALERT: reason = %d , acb = %p, acb->segments = %d, acb->size = %lu Error = %d"
+vxhs_fail_aio(char * guid, void *acb) "vDisk %s, failing acb %p"
+vxhs_iio_callback_ready(char *vd, int err) "async vxhs_iio

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Kevin,

Changed per your suggestion in V4 patch. Please review when you get some time.

Thanks,
Ashish

On Mon, Aug 15, 2016 at 3:47 AM, Kevin Wolf <kw...@redhat.com> wrote:
> Am 15.08.2016 um 12:20 hat Daniel P. Berrange geschrieben:
>> On Sat, Aug 13, 2016 at 09:35:12PM -0700, Ashish Mittal wrote:
>> > +/*
>> > + * vxhs_parse_uri: Parse the incoming URI and populate *conf with the
>> > + * vdisk_id, and all the host(s) information. Host at index 0 is local 
>> > storage
>> > + * agent and the rest, reflection target storage agents. The local storage
>> > + * agent ip is the efficient internal address in the uri, e.g. 
>> > 192.168.0.2.
>> > + * The local storage agent address is stored at index 0. The reflection 
>> > target
>> > + * ips, are the E-W data network addresses of the reflection node agents, 
>> > also
>> > + * extracted from the uri.
>> > + */
>> > +static int vxhs_parse_uri(BlockdevOptionsVxHS *conf,
>> > +   const char *filename)
>>
>> Delete this method entirely. We should not be adding URI syntax for any new
>> block driver. The QAPI schema syntax is all we need.
>
> I disagree. URI syntax is nice for human users.
>
> However, you should use the proper interfaces to implement this, which
> is .bdrv_parse_filename(). This is a function that gets a string and
> converts it into a QDict, which is then passed to .bdrv_open(). So it
> uses exactly the same code path in .bdrv_open() as if used directly with
> QAPI.
>
> Kevin

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Daniel,

In V4 of the patch:

On Mon, Aug 15, 2016 at 3:20 AM, Daniel P. Berrange  wrote:

>> + * Convert the json formatted command line into qapi.
>> +*/
>> +
>> +static int vxhs_parse_json(BlockdevOptionsVxHS *conf,
>> +  QDict *options, Error **errp)
>> +{

>> +}
>
> Ewww this is really horrible code. It is open-coding a special purpose
> conversion of QemuOpts -> QDict -> QAPI scheme. We should really put
> my qdict_crumple() API impl as a pre-requisite of this, so you can then
> use qdict_crumple + qmp_input_visitor to do this conversion in a generic
> manner removing all this code.
>
>   https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03118.html

Changed to not do the manual conversion anymore.

>
>> +/*
>> + * vxhs_parse_uri: Parse the incoming URI and populate *conf with the
>> + * vdisk_id, and all the host(s) information. Host at index 0 is local 
>> storage
>> + * agent and the rest, reflection target storage agents. The local storage
>> + * agent ip is the efficient internal address in the uri, e.g. 192.168.0.2.
>> + * The local storage agent address is stored at index 0. The reflection 
>> target
>> + * ips, are the E-W data network addresses of the reflection node agents, 
>> also
>> + * extracted from the uri.
>> + */
>> +static int vxhs_parse_uri(BlockdevOptionsVxHS *conf,
>> +   const char *filename)
>
> Delete this method entirely. We should not be adding URI syntax for any new
> block driver. The QAPI schema syntax is all we need.
>

Changed per suggestion from Kevin.


>> +of_vsa_addr = g_new0(char, OF_MAX_SERVER_ADDR);
>> +file_name = g_new0(char, OF_MAX_FILE_LEN);
>> +snprintf(file_name, OF_MAX_FILE_LEN, "%s%s", vdisk_prefix, 
>> s->vdisk_guid);
>> +snprintf(of_vsa_addr, OF_MAX_SERVER_ADDR, "of://%s:%d",
>> + s->vdisk_hostinfo[s->vdisk_cur_host_idx].hostip,
>> + s->vdisk_hostinfo[s->vdisk_cur_host_idx].port);
>
> *Never* use  g_new + snprintf, particularly not with a fixed length
> buffer. g_strdup_printf() is the right way.
>

Fixed all such places.


>> +out:
>> +if (file_name != NULL) {
>> +g_free(file_name);
>> +}
>> +if (of_vsa_addr != NULL) {
>> +g_free(of_vsa_addr);
>> +}
>
> Useless if-before-free - g_free happily accepts NULL so don't check
> for it yourself.
>

Removed all if-before-free - g_free.

>
>> +
>> +/*
>> + * This is called by QEMU when a flush gets triggered from within
>> + * a guest at the block layer, either for IDE or SCSI disks.
>> + */
>> +int vxhs_co_flush(BlockDriverState *bs)
>> +{
>> +BDRVVXHSState *s = bs->opaque;
>> +uint64_t size = 0;
>> +int ret = 0;
>> +uint32_t iocount = 0;
>> +
>> +ret = qemu_iio_ioctl(s->qnio_ctx,
>> +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd,
>> +VDISK_AIO_FLUSH, , NULL, IIO_FLAG_SYNC);
>> +
>> +if (ret < 0) {
>> +/*
>> + * Currently not handling the flush ioctl
>> + * failure because of network connection
>> + * disconnect. Since all the writes are
>> + * commited into persistent storage hence
>> + * this flush call is noop and we can safely
>> + * return success status to the caller.
>> + *
>> + * If any write failure occurs for inflight
>> + * write AIO because of network disconnect
>> + * then anyway IO failover will be triggered.
>> + */
>> +trace_vxhs_co_flush(s->vdisk_guid, ret, errno);
>> +ret = 0;
>> +}
>> +
>> +iocount = vxhs_get_vdisk_iocount(s);
>> +if (iocount > 0) {
>> +trace_vxhs_co_flush_iocnt(iocount);
>> +}
>
> You're not doing anything with the iocount value here. Either
> delete this, or do something useful with it.
>

Deleted.


>> +unsigned long vxhs_get_vdisk_stat(BDRVVXHSState *s)
>> +{
>> +void *ctx = NULL;
>> +int flags = 0;
>> +unsigned long vdisk_size = 0;
>
> Is this meansured in bytes ? If so 'unsigned long' is a rather
> limited choice for 32-bit builds. long long  / int64_t
> would be better.
>

You are right. This will need change in the qnio library as well. Hope
I can fix this later!

>> +int ret = 0;
>> +
>> +ret = qemu_iio_ioctl(s->qnio_ctx,
>> +s->vdisk_hostinfo[s->vdisk_cur_host_idx].vdisk_rfd,
>> +VDISK_STAT, _size, ctx, flags);
>> +
>> +if (ret < 0) {
>> +trace_vxhs_get_vdisk_stat_err(s->vdisk_guid, ret, errno);
>> +}
>> +
>> +trace_vxhs_get_vdisk_stat(s->vdisk_guid, vdisk_size);
>> +return vdisk_size;
>
> Presumably vdisk_size is garbage when ret < 0, so you really
> need to propagate the error better.
>

Fixed.


>> +/*
>> + * build storage agent address and vdisk device name strings
>> + */
>> +of_vsa_addr = g_new0(char, OF_MAX_SERVER_ADDR);
>> +file_name = g_new0(char, OF_MAX_FILE_LEN);
>> +snprintf(file_name, OF_MAX_FILE_LEN, "%s%s", 

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Thanks Stefan, I will look at block/quorum.c.

I have sent V4 of the patch with a reworked parsing logic for both
JSON and URI. Both of them are quite compact now.

URI parsing now follows the suggestion given by Kevin.
//
However, you should use the proper interfaces to implement this, which
is .bdrv_parse_filename(). This is a function that gets a string and
converts it into a QDict, which is then passed to .bdrv_open(). So it
uses exactly the same code path in .bdrv_open() as if used directly with
QAPI.
//

Additionally, I have fixed all the issues pointed out by you on V1 of
the patch. The only change I haven't done is to replace pipes with
QEMUBH. I am hoping this will not hold up the patch from being
accepted, and I can make this transition later with proper dev and
testing.

Regards,
Ashish


On Tue, Aug 23, 2016 at 2:58 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote:
> On Sat, Aug 20, 2016 at 11:42:22AM -0700, ashish mittal wrote:
>> I'm trying to understand how I can parse a json command line having
>> multiple server list (InetSocketAddressList) without the QemuOpts ->
>> QDict -> QAPI conversion that I am currently doing.
>>
>> block/nbd.c has been suggested, but it only parses a single host
>> entry, which is pretty straightforward, but not very helpful.
>>
>> Could somebody please suggest a sample implementation (other than
>> gluster.c) that parses a list of server like options in the json
>> format?
>
> Another block driver that takes an array of objects is block/quorum.c.
> See quorum_open() and how it picks out "children." objects.
>
> I haven't read the full discussion in this thread but maybe that's what
> you're looking for.
>
> Stefan

[Qemu-devel] [PATCH v4 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line using JSON syntax:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'

Sample command line using URI syntax:
qemu-img convert -f raw -O raw -n 
/var/lib/nova/instances/_base/0c5eacd5ebea5ed914b6a3e7b18f1ce734c386ad 
vxhs://192.168.0.1:/%7Bc6718f6b-0401-441d-a8c3-1f0064d75ee0%7D

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v3 changelog:
=
(1) Reworked QAPI/JSON parsing.
(2) Reworked URI parsing as suggested by Kevin.
(3) Fixes per review comments from Stefan on v1.
(4) Fixes per review comments from Daniel on v3.

 block/Makefile.objs |2 +
 block/trace-events  |   41 ++
 block/vxhs.c| 1304 +++
 block/vxhs.h|  237 ++
 configure   |   50 ++
 5 files changed, 1634 insertions(+)
 create mode 100644 block/vxhs.c
 create mode 100644 block/vxhs.h

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 2593a2f..5f83305 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 
@@ -43,3 +44,4 @@ block-obj-m+= dmg.o
 dmg.o-libs := $(BZIP2_LIBS)
 qcow.o-libs:= -lz
 linux-aio.o-libs   := -laio
+vxhs.o-libs:= $(VXHS_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 05fa13c..06c6d8c 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,44 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_bdrv_init(const char c) "Registering VxHS AIO driver%c"
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
Bailing out%c"
+vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) "Read/Write 
failed: error %d, reason %d, acb %p, segment %d"
+vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %s, added acb %p to 
retry queue (5)"
+vxhs_iio_callback_chnlfail(int error) "QNIO channel failed, no i/o (%d)"
+vxhs_iio_callback_fail(int r, void *acb, int seg, uint64_t size, int err) " 
ALERT: reason = %d , acb = %p, acb->segments = %d, acb->size = %lu Error = %d"
+vxhs_fail_aio(char * guid, void *acb) "vDisk %s, failing acb %p"
+vxhs_iio_callback_ready(char *vd, int err) "async vxhs_iio_callback: 
IRP_VDISK_CHECK_IO_FAILOVER_READY completed for vdisk %s with error %d"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(char c) "Could not create a pipe for device. Bailing out%c"
+vxhs_aio_rw(char *guid, int iodir, uint64_t size, uint64_t offset) "vDisk %s, 
vDisk device is in failed state iodir = %d size = %lu offset = %lu"
+vxhs_aio_rw_retry(char *guid, void *acb, int queue) "vDisk %s, added acb %p to 
retry queue(%d)"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int seg, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d 
size = %lu offset = %lu ACB = %p Segments = %d. Error = %d, errno = %d"
+vxhs_co_flush(char *guid, int ret, int err) "vDisk (%s) Flush ioctl failed ret 
= %d errno = %d"
+vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) 

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Need some help!

I'm trying to understand how I can parse a json command line having
multiple server list (InetSocketAddressList) without the QemuOpts ->
QDict -> QAPI conversion that I am currently doing.

block/nbd.c has been suggested, but it only parses a single host
entry, which is pretty straightforward, but not very helpful.

Could somebody please suggest a sample implementation (other than
gluster.c) that parses a list of server like options in the json
format?

Thanks,
Ashish


On Wed, Aug 17, 2016 at 2:58 PM, ashish mittal <ashmit...@gmail.com> wrote:
> Thanks Paolo, and everyone else for the corrections :)
>
> I will try to fix it in a patch this week or the next. I referred to
> gluster.c implementation as it was closer to what we wanted to achieve
> i.e. passing multiple servers for the block device. I picked up the
> idea of referring to gluster.c from the following email that I
> received on libvirt mailing list. I did test my implementation to make
> sure that it worked fine, but did not realize that it was not clean. I
> will fix it.
>
> //
> Hmm, unless I'm mis-reading this, there is no separator between
> the two URIs you're providing - is there a missing ',' or something
> similar.
>
> Slightly off topic for the libvirt list, but I would be pretty surprised
> if the QEMU developers accepted patches using this URI syntax for providing
> multiple servers.
>
> A similar approach was originally proposed for the glusterfs driver and
> the submitter was told to write it a different way, not using the legacy
> URI syntax at all, but instead a QAPI based syntax:
>
>   https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg04126.html
>
> So the sooner you can submit your QEMU patches to the qemu-devel mailing
> list the better, as we'll need to get clarity on the accepted QEMU syntax
> in order to proceed further with libvirt patch review.
>
> Broadly speaking I think the patch you've proposed looks pretty much
> fine now. I'd have a slight preference for it to be done as two patches.
> One patch adds the XML schema, parser changes, etc. The second patch
> just does the QEMU driver integration & unit tests.
> ...
> //
>
> Regards,
> Ashish
>
>
> On Wed, Aug 17, 2016 at 4:22 AM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>>
>>
>> On 15/08/2016 18:29, ashish mittal wrote:
>>>>> +/*
>>>>> + * Convert the json formatted command line into qapi.
>>>>> +*/
>>>>> +
>>>>> +static int vxhs_parse_json(BlockdevOptionsVxHS *conf,
>>>>> +  QDict *options, Error **errp)
>>>>> +{
>>> ...
>>>>> +errno = EINVAL;
>>>>> +return -errno;
>>>>> +}
>>>>
>>>> Ewww this is really horrible code. It is open-coding a special purpose
>>>> conversion of QemuOpts -> QDict -> QAPI scheme. We should really put
>>>> my qdict_crumple() API impl as a pre-requisite of this, so you can then
>>>> use qdict_crumple + qmp_input_visitor to do this conversion in a generic
>>>> manner removing all this code.
>>>>
>>>>   https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03118.html
>>>>
>>>
>>> Thanks for the suggestions. I will try to incorporate them in the next
>>> version. Actually, I used block/gluster.c for reference (as advised).
>>> This is exactly how it parses json CLI options. It also implements the
>>> *parse_uri() in a similar way.
>>
>> I think I pointed you to block/nbd.c instead, which works as Kevin
>> suggested.
>>
>> Paolo

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Thanks Paolo, and everyone else for the corrections :)

I will try to fix it in a patch this week or the next. I referred to
gluster.c implementation as it was closer to what we wanted to achieve
i.e. passing multiple servers for the block device. I picked up the
idea of referring to gluster.c from the following email that I
received on libvirt mailing list. I did test my implementation to make
sure that it worked fine, but did not realize that it was not clean. I
will fix it.

//
Hmm, unless I'm mis-reading this, there is no separator between
the two URIs you're providing - is there a missing ',' or something
similar.

Slightly off topic for the libvirt list, but I would be pretty surprised
if the QEMU developers accepted patches using this URI syntax for providing
multiple servers.

A similar approach was originally proposed for the glusterfs driver and
the submitter was told to write it a different way, not using the legacy
URI syntax at all, but instead a QAPI based syntax:

  https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg04126.html

So the sooner you can submit your QEMU patches to the qemu-devel mailing
list the better, as we'll need to get clarity on the accepted QEMU syntax
in order to proceed further with libvirt patch review.

Broadly speaking I think the patch you've proposed looks pretty much
fine now. I'd have a slight preference for it to be done as two patches.
One patch adds the XML schema, parser changes, etc. The second patch
just does the QEMU driver integration & unit tests.
...
//

Regards,
Ashish


On Wed, Aug 17, 2016 at 4:22 AM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>
>
> On 15/08/2016 18:29, ashish mittal wrote:
>>>> +/*
>>>> + * Convert the json formatted command line into qapi.
>>>> +*/
>>>> +
>>>> +static int vxhs_parse_json(BlockdevOptionsVxHS *conf,
>>>> +  QDict *options, Error **errp)
>>>> +{
>> ...
>>>> +errno = EINVAL;
>>>> +return -errno;
>>>> +}
>>>
>>> Ewww this is really horrible code. It is open-coding a special purpose
>>> conversion of QemuOpts -> QDict -> QAPI scheme. We should really put
>>> my qdict_crumple() API impl as a pre-requisite of this, so you can then
>>> use qdict_crumple + qmp_input_visitor to do this conversion in a generic
>>> manner removing all this code.
>>>
>>>   https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03118.html
>>>
>>
>> Thanks for the suggestions. I will try to incorporate them in the next
>> version. Actually, I used block/gluster.c for reference (as advised).
>> This is exactly how it parses json CLI options. It also implements the
>> *parse_uri() in a similar way.
>
> I think I pointed you to block/nbd.c instead, which works as Kevin
> suggested.
>
> Paolo

Re: [Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

>> +/*
>> + * Convert the json formatted command line into qapi.
>> +*/
>> +
>> +static int vxhs_parse_json(BlockdevOptionsVxHS *conf,
>> +  QDict *options, Error **errp)
>> +{
...
>> +errno = EINVAL;
>> +return -errno;
>> +}
>
> Ewww this is really horrible code. It is open-coding a special purpose
> conversion of QemuOpts -> QDict -> QAPI scheme. We should really put
> my qdict_crumple() API impl as a pre-requisite of this, so you can then
> use qdict_crumple + qmp_input_visitor to do this conversion in a generic
> manner removing all this code.
>
>   https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03118.html
>

Thanks for the suggestions. I will try to incorporate them in the next
version. Actually, I used block/gluster.c for reference (as advised).
This is exactly how it parses json CLI options. It also implements the
*parse_uri() in a similar way.


>> +/*
>> + * vxhs_parse_uri: Parse the incoming URI and populate *conf with the
>> + * vdisk_id, and all the host(s) information. Host at index 0 is local 
>> storage
>> + * agent and the rest, reflection target storage agents. The local storage
>> + * agent ip is the efficient internal address in the uri, e.g. 192.168.0.2.
>> + * The local storage agent address is stored at index 0. The reflection 
>> target
>> + * ips, are the E-W data network addresses of the reflection node agents, 
>> also
>> + * extracted from the uri.
>> + */
>> +static int vxhs_parse_uri(BlockdevOptionsVxHS *conf,
>> +   const char *filename)
>
> Delete this method entirely. We should not be adding URI syntax for any new
> block driver. The QAPI schema syntax is all we need.
>

Regards,
Ashish

[Qemu-devel] [PATCH v3 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Sample command line with a vxhs block device specification:
./qemu-system-x86_64 -name instance-0008 -S -vnc 0.0.0.0:0 -k en-us -vga 
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg 
timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'

Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
v3 changelog:
=
(1) Implemented QAPI interface for passing VxHS block device parameters.

Sample trace o/p after filtering out libqnio debug messages:

./qemu-system-x86_64 -trace enable=vxhs* -name instance-0008 -S -vnc 
0.0.0.0:0 -k en-us -vga cirrus -device 
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on 
'json:{"driver":"vxhs","vdisk_id":"/{c3e9095a-a5ee-4dce-afeb-2a59fb387410}","server":[{"host":"172.172.17.4","port":""},{"host":"172.172.17.2","port":""}]}'
2149@1471122960.293340:vxhs_parse_json vdisk_id from json 
/{c3e9095a-a5ee-4dce-afeb-2a59fb387410}
2149@1471122960.293359:vxhs_parse_json_numservers Number of servers passed = 2
2149@1471122960.293369:vxhs_parse_json_hostinfo Host 1: IP 172.172.17.4, Port 

2149@1471122960.293379:vxhs_parse_json_hostinfo Host 2: IP 172.172.17.2, Port 

2149@1471122960.293382:vxhs_open vxhs_vxhs_open: came in to open file = (null)
2149@1471122960.301444:vxhs_setup_qnio Context to HyperScale IO manager = 
0x7f0996fd3920
2149@1471122960.301499:vxhs_open_device Adding host 172.172.17.4: to 
BDRVVXHSState
2149@1471122960.301512:vxhs_open_device Adding host 172.172.17.2: to 
BDRVVXHSState
2149@1471122960.305062:vxhs_get_vdisk_stat vDisk 
/{c3e9095a-a5ee-4dce-afeb-2a59fb387410} stat ioctl returned size 429
...

TODO: Fixes to issues raised by review comments from Stefan and Kevin.

 block/Makefile.objs  |2 +
 block/trace-events   |   46 ++
 block/vxhs.c | 1424 ++
 block/vxhs.h |  293 +++
 configure|   50 ++
 qapi/block-core.json |   22 +-
 6 files changed, 1835 insertions(+), 2 deletions(-)
 create mode 100644 block/vxhs.c
 create mode 100644 block/vxhs.h

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 2593a2f..5f83305 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 
@@ -43,3 +44,4 @@ block-obj-m+= dmg.o
 dmg.o-libs := $(BZIP2_LIBS)
 qcow.o-libs:= -lz
 linux-aio.o-libs   := -laio
+vxhs.o-libs:= $(VXHS_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 978ef4f..d3decb0 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,49 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_bdrv_init(int version) "Registering VxHS %d AIO driver"
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
Bailing out%c"
+vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) "Read/Write 
failed: error %d, reason %d, acb %p, segment %d"
+vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %s, added acb %p to 
retry queue (5)"
+vxhs_iio_callback_chnlfail(int error) "QNIO channel failed, no i/o (%d)"
+vxhs_iio_callback_fail(int r, void *acb, int seg, uint64_t size, int err) " 
ALERT: reason = %d , acb = %p, acb->segments = %d, acb->size = %lu Error = %d"
+vxhs_fail_aio(char * guid, void *acb) "vDisk %s, failing acb %p"
+vxhs_iio_callback_ready(char *vd, int err) "async vxhs_iio_ca

Re: [Qemu-devel] [PATCH RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[ashish mittal]

Hi Christopher,

We are working on a sample server implementation, and will add it to
libqnio soon.

Thanks,
Ashish

On Mon, Aug 8, 2016 at 12:46 AM, Christoph Hellwig <h...@lst.de> wrote:

> On Sun, Jul 31, 2016 at 04:52:16PM -0700, Ashish Mittal wrote:
> > This patch adds support for a new block device type called "vxhs".
> > Source code for the library that this code loads can be downloaded from:
> > https://github.com/MittalAshish/libqnio.git
>
> Do you also have a pointer to the server implementation?
>

[Qemu-devel] [PATCH v2 RFC] block/vxhs: Initial commit to add Veritas HyperScale VxHS block device support

[Ashish Mittal]

This patch adds support for a new block device type called "vxhs".
Source code for the library that this code loads can be downloaded from:
https://github.com/MittalAshish/libqnio.git

Version 2 patch submission fixes the following issues:
(1) Removed code to dlopen library. We now check if libqnio is installed during
configure, and directly link with it.
(2) Changed file headers to mention GPLv2-or-later license.
(3) Removed unnecessary type casts and inlines.
(4) Removed custom tokenize function and modified code to use g_strsplit.
(5) Replaced malloc/free with g_new/g_free and removed code that checks for
memory allocation failure conditions.
(6) Removed some block ops implementations that were place-holders only.
(7) Removed all custom debug messages. Added new messages in block/trace-events
(8) Other miscellaneous corrections.

TODO: QAPI changes and (fixes to) review comments from Stefan.
Signed-off-by: Ashish Mittal <ashish.mit...@veritas.com>
---
 block/Makefile.objs |2 +
 block/trace-events  |   40 ++
 block/vxhs.c| 1199 +++
 block/vxhs.h|  294 +
 configure   |   50 +++
 5 files changed, 1585 insertions(+)
 create mode 100644 block/vxhs.c
 create mode 100644 block/vxhs.h

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 2593a2f..5f83305 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -20,6 +20,7 @@ block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_VXHS) += vxhs.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 
@@ -43,3 +44,4 @@ block-obj-m+= dmg.o
 dmg.o-libs := $(BZIP2_LIBS)
 qcow.o-libs:= -lz
 linux-aio.o-libs   := -laio
+vxhs.o-libs:= $(VXHS_LIBS)
diff --git a/block/trace-events b/block/trace-events
index 978ef4f..8f02725 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -114,3 +114,43 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t 
offset, size_t len) "s
 qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t 
offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, 
uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64
 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) 
"s %p acb %p ret %d offset %"PRIu64" len %zu"
+
+# block/vxhs.c
+vxhs_bdrv_init(int version) "Registering VxHS %d AIO driver"
+vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d"
+vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p"
+vxhs_setup_qnio_nwerror(char c) "Could not initialize the network channel. 
Bailing out%c"
+vxhs_open_device_cmdline(const char *vxhs_uri) "Original command line : %s"
+vxhs_iio_callback_iofail(int err, int reason, void *acb, int seg) "Read/Write 
failed: error %d, reason %d, acb %p, segment %d"
+vxhs_iio_callback_retry(char *guid, void *acb) "vDisk %s, added acb %p to 
retry queue (5)"
+vxhs_iio_callback_chnlfail(int error) "QNIO channel failed, no i/o (%d)"
+vxhs_iio_callback_fail(int r, void *acb, int seg, uint64_t size, int err) " 
ALERT: reason = %d , acb = %p, acb->segments = %d, acb->size = %lu Error = %d"
+vxhs_fail_aio(char * guid, void *acb) "vDisk %s, failing acb %p"
+vxhs_iio_callback_ready(char *vd, int err) "async vxhs_iio_callback: 
IRP_VDISK_CHECK_IO_FAILOVER_READY completed for vdisk %s with error %d"
+vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, 
%d"
+vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d"
+vxhs_open_device_qnio(char *of_vsa_addr) "Could not open an QNIO connection 
to: %s"
+vxhs_create(const char *filename) "vxhs_create: came in to open file = %s"
+vxhs_open(const char *filename) "vxhs_vxhs_open: came in to open file = %s"
+vxhs_open_fail(int ret) "Could not open the device. Error = %d"
+vxhs_open_epipe(char c) "Could not create a pipe for device. bailing out%c"
+vxhs_aio_rw(char *guid, int iodir, uint64_t size, uint64_t offset) "vDisk %s, 
vDisk device is in failed state iodir = %d size = %lu offset = %lu"
+vxhs_aio_rw_retry(char *guid, void *acb, int queue) "vDisk %s, added acb %p to 
retry queue(%d)"
+vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d"
+vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void 
*acb, int seg, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d 
size = %lu offset = %lu ACB = %p Segments = %d. Error = %d, errno = %d"
+v