[Qemu-devel] [PATCH 0/4] Allow blockdev-add for SSH

[Ashijeet Acharya]

This series adds blockdev-add support for SSH block driver.

Patch 1 prepares the code for the addition of a new option prefix,
which is "server.". This is accomplished by adding a
ssh_has_filename_options_conflict() function which helps to iterate
over the various options and check for conflict.

Patch 2 first adds InetSocketAddress compatibility to SSH block driver
and then makes it accept a InetSocketAddress under the "server" option.
The old options "host" and "port" are supported as legacy options and
then translated to the respective InetSocketAddress representation.

Patch 3 drops the usage of "host" and "port" outside of
ssh_has_filename_options_conflict() and
ssh_process_legacy_socket_options() functions in order to make them
legacy options completely.

Patch 4 helps to allow blockdev-add support for the SSH block driver
by making the SSH option available.


*** This series depends on the following patch: ***
"qdict: implement a qdict_crumple method for un-flattening a dict"
from Daniel's "QAPI/QOM work for non-scalar object properties"
series.

Ashijeet Acharya (4):
  block/ssh: Add ssh_has_filename_options_conflict()
  block/ssh: Add InetSocketAddress and accept it
  block/ssh: Use InetSocketAddress options
  qapi: allow blockdev-add for ssh

 block/ssh.c  | 121 +++
 qapi/block-core.json |  24 +-
 2 files changed, 125 insertions(+), 20 deletions(-)

--
2.6.2

Re: [Qemu-devel] Deprecating old machine-types (was Re: [PATCH v4 7/8] intel_iommu: keep buggy EIM enabled in 2.7 machine type)

[Paolo Bonzini]

On 10/10/2016 19:46, Eduardo Habkost wrote:
> I don't think we have a plan, but I would support deprecating and
> removing very old machine-types. The question is: how old is too
> old?
> 
> For reference, the commits and dates when each machine-type was
> added are below:
> 
> machine   commitcommit date  release  release date
> pc-0.10   e8b2a1c6  Jul 8 2009   v0.11.0  Sep 22 2009
> pc-0.13   95747581  Jul 22 2009  v0.12.0  Dec 19 2009
> pc-0.12   2cae6f5e  Jan 8 2010   v0.13.0  Oct 14 2010
> pc-0.13   d76fa62d  Feb 15 2010  v0.13.0  Oct 14 2010
> pc-0.14   b903a0f7  Nov 11 2010  v0.14.0  Feb 16 2011
> pc-0.15   ce01a508  Dec 18 2011  v1.1.0   Jun 1 2012
> pc-1.019857e62  Nov 7 2011   v1.0 Dec 1 2011
> pc-1.1382b3a68  Feb 21 2012  v1.1.0   Jun 1 2012
> pc-1.2f1dacf1c  Jun 11 2012  v1.2.0   Sep 5 2012
> pc-1.3f4306941  Sep 13 2012  v1.3.0   Dec 3 2012

Anything before pc-1.3 has issues with migration due to the introduction
of the memory API.  Basically, 0xf-0xf is not migrated
correctly, and the result is that rebooting after migration causes the
guest to crash.  So that could be a reasonable place to draw the line at.

Paolo

[Qemu-devel] [PATCH V2] colo-proxy: fix memory leak

[Zhang Chen]

Fix memory leak in colo-compare.c and filter-rewriter.c
Report by Coverity.

v2:
  - use traces instead of fprintf in colo-compare.c

v1:
  - initial patch

Signed-off-by: Zhang Chen 
---
 net/colo-compare.c| 11 +++
 net/filter-rewriter.c | 17 +
 trace-events  |  1 +
 3 files changed, 9 insertions(+), 20 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index 22b1da1..bbb6ca6 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -188,7 +188,6 @@ static int colo_packet_compare_tcp(Packet *spkt, Packet 
*ppkt)
 {
 struct tcphdr *ptcp, *stcp;
 int res;
-char *sdebug, *ddebug;
 
 trace_colo_compare_main("compare tcp");
 if (ppkt->size != spkt->size) {
@@ -219,11 +218,9 @@ static int colo_packet_compare_tcp(Packet *spkt, Packet 
*ppkt)
 (spkt->size - ETH_HLEN));
 
 if (res != 0 && trace_event_get_state(TRACE_COLO_COMPARE_MISCOMPARE)) {
-sdebug = strdup(inet_ntoa(ppkt->ip->ip_src));
-ddebug = strdup(inet_ntoa(ppkt->ip->ip_dst));
-fprintf(stderr, "%s: src/dst: %s/%s p: seq/ack=%u/%u"
-" s: seq/ack=%u/%u res=%d flags=%x/%x\n",
-__func__, sdebug, ddebug,
+trace_colo_compare_pkt_info(__func__,
+inet_ntoa(ppkt->ip->ip_src),
+inet_ntoa(ppkt->ip->ip_dst),
 (unsigned int)ntohl(ptcp->th_seq),
 (unsigned int)ntohl(ptcp->th_ack),
 (unsigned int)ntohl(stcp->th_seq),
@@ -235,8 +232,6 @@ static int colo_packet_compare_tcp(Packet *spkt, Packet 
*ppkt)
 fprintf(stderr, "Secondary len = %d\n", spkt->size);
 qemu_hexdump((char *)spkt->data, stderr, "colo-compare", spkt->size);
 
-g_free(sdebug);
-g_free(ddebug);
 }
 
 return res;
diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c
index 89abe72..c4ab91c 100644
--- a/net/filter-rewriter.c
+++ b/net/filter-rewriter.c
@@ -68,15 +68,11 @@ static int handle_primary_tcp_pkt(NetFilterState *nf,
 
 tcp_pkt = (struct tcphdr *)pkt->transport_header;
 if (trace_event_get_state(TRACE_COLO_FILTER_REWRITER_DEBUG)) {
-char *sdebug, *ddebug;
-sdebug = strdup(inet_ntoa(pkt->ip->ip_src));
-ddebug = strdup(inet_ntoa(pkt->ip->ip_dst));
-trace_colo_filter_rewriter_pkt_info(__func__, sdebug, ddebug,
+trace_colo_filter_rewriter_pkt_info(__func__,
+inet_ntoa(pkt->ip->ip_src), inet_ntoa(pkt->ip->ip_dst),
 ntohl(tcp_pkt->th_seq), ntohl(tcp_pkt->th_ack),
 tcp_pkt->th_flags);
 trace_colo_filter_rewriter_conn_offset(conn->offset);
-g_free(sdebug);
-g_free(ddebug);
 }
 
 if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_SYN)) {
@@ -116,15 +112,11 @@ static int handle_secondary_tcp_pkt(NetFilterState *nf,
 tcp_pkt = (struct tcphdr *)pkt->transport_header;
 
 if (trace_event_get_state(TRACE_COLO_FILTER_REWRITER_DEBUG)) {
-char *sdebug, *ddebug;
-sdebug = strdup(inet_ntoa(pkt->ip->ip_src));
-ddebug = strdup(inet_ntoa(pkt->ip->ip_dst));
-trace_colo_filter_rewriter_pkt_info(__func__, sdebug, ddebug,
+trace_colo_filter_rewriter_pkt_info(__func__,
+inet_ntoa(pkt->ip->ip_src), inet_ntoa(pkt->ip->ip_dst),
 ntohl(tcp_pkt->th_seq), ntohl(tcp_pkt->th_ack),
 tcp_pkt->th_flags);
 trace_colo_filter_rewriter_conn_offset(conn->offset);
-g_free(sdebug);
-g_free(ddebug);
 }
 
 if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == (TH_ACK | TH_SYN))) {
@@ -162,6 +154,7 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf,
 
 iov_to_buf(iov, iovcnt, 0, buf, size);
 pkt = packet_new(buf, size);
+g_free(buf);
 
 /*
  * if we get tcp packet
diff --git a/trace-events b/trace-events
index 1cb9d37..8115430 100644
--- a/trace-events
+++ b/trace-events
@@ -149,6 +149,7 @@ colo_compare_icmp_miscompare(const char *sta, int size) ": 
%s = %d"
 colo_compare_ip_info(int psize, const char *sta, const char *stb, int ssize, 
const char *stc, const char *std) "ppkt size = %d, ip_src = %s, ip_dst = %s, 
spkt size = %d, ip_src = %s, ip_dst = %s"
 colo_old_packet_check_found(int64_t old_time) "%" PRId64
 colo_compare_miscompare(void) ""
+colo_compare_pkt_info(const char *func, const char *src, const char *dst, 
uint32_t pseq, uint32_t pack, uint32_t sseq, uint32_t sack, int res, uint32_t 
pflag, uint32_t sflag) "%s: src/dst: %s/%s p: seq/ack=%u/%u   s: seq/ack=%u/%u 
res=%d flags=%x/%x\n"
 
 # net/filter-rewriter.c
 colo_filter_rewriter_debug(void) ""
-- 
2.7.4

Re: [Qemu-devel] [Qemu-ppc] [QEMU PATCH v5 4/6] migration: migrate QTAILQ

[Paolo Bonzini]

On 10/10/2016 23:29, Jianjun Duan wrote:
> You are right. What we have in VMSTATE now cannot handle a recursive (or
> cyclic as you call it) structure. The idea was to use VMS_LINKED to
> indicate a recursive structure.

Sure, but it's unnecessary.  If you didn't have VMS_LINKED, no one would
notice the difference.

Yes, existing VMStateInfos didn't need the new arguments, the new one
does.  So you add the new arguments.  There's no need to add the flag.

Paolo

> In this patch is used on a queue. It can
> also be used on list or trees.
> In this regard, VMS_LINKED does represent something.

Re: [Qemu-devel] [PATCH v4] build: Work around SIZE_MAX bug in OSX headers

[Markus Armbruster]

Eric Blake  writes:

> C99 requires SIZE_MAX to be declared with the same type as the
> integral promotion of size_t, but OSX mistakenly defines it as
> an 'unsigned long long' expression even though size_t is only
> 'unsigned long'.  Rather than futzing around with whether size_t
> is 32- or 64-bits wide (which would be needed if we cared about
> using SIZE_T in a #if expression), let the compiler get the right
> type for us by virtue of integer promotion - if we later need it

By virtue of usual arithmetic conversions, actually.

> during the preprocessor, the build will break on Mac until we
> improve our replacement.
>
> See also https://patchwork.ozlabs.org/patch/542327/ for an
> instance where the wrong type trips us up if we don't fix it
> for good in osdep.h.
>
> Some versions of glibc make a similar mistake with SSIZE_MAX; the
> goal is that the approach of this patch could be copied to work
> around that problem if it ever becomes important to us.
>
> Signed-off-by: Eric Blake 
>
> ---
> v1 was here:
> https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg02520.html
>
> The topic recently came up again, and I noticed this patch sitting
> on one of my older branches, so I've taken another shot at it.
> https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg00950.html
>
> v2: rewrite into a configure check (not sure if directly adding a
> -D to QEMU_CFLAGS is the best, so advice welcome)
>
> v3: Use config-host.mak rather than direct -D
>
> v4: placate -Wunused builds
>
> I lack easy access to a Mac box, so this is untested as to whether
> it actually solves the issue...
> ---
>  include/qemu/osdep.h |  8 
>  configure| 16 
>  2 files changed, 24 insertions(+)
>
> diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
> index 9e9fa61..e06ac47 100644
> --- a/include/qemu/osdep.h
> +++ b/include/qemu/osdep.h
> @@ -141,6 +141,14 @@ extern int daemon(int, int);
>  # error Unknown pointer size
>  #endif
>
> +/* Mac OSX has a  bug that incorrectly defines SIZE_MAX with
> + * the wrong type. Our replacement isn't usable in preprocessor
> + * expressions, but it is sufficient for our needs. */
> +#if defined(HAVE_BROKEN_SIZE_MAX) && HAVE_BROKEN_SIZE_MAX
> +#undef SIZE_MAX
> +#define SIZE_MAX ((sizeof(char)) * -1)

All right, let's see how this works.

* -1 is of type int (C99 §6.4.4.1).

* sizeof(char) is a clever way to say (size_t)1.

* Integer promotion (§6.3.1.1) is a no-op for int, i.e. the left operand
  stays int.

* Integer promotion is a no-op for size_t, since size_t is surely at
  least as wide as unsigned int.  The right operand stays size_t.

* The type of the multiplication is determined by the usual arithmetic
  conversions (§6.3.1.6).  Since size_t's rank is surely greater or
  equal to int's rank, the int operand is converted to size_t, and the
  result is size_t.

* The value of the expression is therefore (size_t)-1 * (size_t)1,
  yielding (size_t)-1.

Cute, but what's wrong with straightforward

#define SIZE_MAX ((size_t)-1)

?

> +#endif
> +
>  #ifndef MIN
>  #define MIN(a, b) (((a) < (b)) ? (a) : (b))
>  #endif
> diff --git a/configure b/configure
> index 5751d8e..dd9e679 100755
> --- a/configure
> +++ b/configure
> @@ -1725,6 +1725,19 @@ if test "$cocoa" = "yes"; then
>  sdl=no
>  fi
>
> +# Some versions of Mac OS X incorrectly define SIZE_MAX
> +cat > $TMPC << EOF
> +#include 
> +#include 
> +int main(int argc, char *argv[]) {
> +return printf("%zu", SIZE_MAX);
> +}
> +EOF
> +have_broken_size_max=no
> +if ! compile_object -Werror ; then
> +have_broken_size_max=yes
> +fi
> +
>  ##
>  # L2TPV3 probe
>
> @@ -5245,6 +5258,9 @@ fi
>  if test "$have_ifaddrs_h" = "yes" ; then
>  echo "HAVE_IFADDRS_H=y" >> $config_host_mak
>  fi
> +if test "$have_broken_size_max" = "yes" ; then
> +echo "HAVE_BROKEN_SIZE_MAX=y" >> $config_host_mak
> +fi
>
>  # Work around a system header bug with some kernel/XFS header
>  # versions where they both try to define 'struct fsxattr':

Re: [Qemu-devel] 答复： Re: [PATCH v3 2/3] qapi: auto generate enum value strings

[Markus Armbruster]

Eric Blake  writes:

> On 10/10/2016 10:09 AM, Lin Ma wrote:
>> 
>> 
> Eric Blake  2016/9/27 星期二 上午 4:17 >>>
>>> On 09/26/2016 05:38 AM, Daniel P. Berrange wrote:
 On Mon, Sep 26, 2016 at 06:16:26PM +0800, Lin Ma wrote:
> Automatically generate enum value strings that containing the acceptable 
> values.
> (Borrowwed Daniel's code.)
>>>
>>> s/Borrowwed/Borrowed/
>> Sorry for the late reply, I was on vacation.
>> Thanks for the review.
>>>
>
> Signed-off-by: Lin Ma 
> ---
>  scripts/qapi-types.py | 2 ++
>  scripts/qapi.py   | 9 +
>  2 files changed, 11 insertions(+)

 This will need some test case coverage in tests/ somewhere, but I'm
 not sure exactly which place is best - Eric/Markus can probably advise
>>>
>>> tests/test-qmp-commands.c is the first one that comes to mind, for
>>> adding another test case to an existing program.

Yes, that's the closest we got.

>> I'm not familiar with how to write qapi generator code and related test
>> code at all. I'll start to dig, Any guidance is appreciated.
>> For adding test case, Only this tests/test-qmp-commands.c needs to be
>> modified, right? 
>
> Yes, I think the easiest approach is to add a new line in the main()
> file that calls out to a new function, and the new function tests that
> an existing QAPI enum (from tests/qapi-schema/qapi-schema-test.json) has
> a sane conversion to a string listing all its members.  Markus may have
> better ideas on where to place a new test, though.

I think tests/test-qmp-commands.c should be split.  See
Message-ID: <8760p7yv8n@dusky.pond.sub.org>
http://lists.gnu.org/archive/html/qemu-devel/2016-10/msg00664.html

However, splitting it out of scope of Lin Ma's work.  Go ahead and add
to tests/test-qmp-commands.c.

Re: [Qemu-devel] [PATCH 1/2] vhost: enable any layout feature

[Yuanhan Liu]

On Mon, Oct 10, 2016 at 07:39:59AM +0300, Michael S. Tsirkin wrote:
> > > > > > 1. why is this memset expensive?
> > > > >
> > > > > I don't have the exact answer, but just some rough thoughts:
> > > > >
> > > > > It's an external clib function: there is a call stack and the
> > > > > IP register will bounch back and forth.
> > > >
> > > > for memset 0?  gcc 5.3.1 on fedora happily inlines it.
> > > 
> > > Good to know!
> > > 
> > > > > overkill to use that for resetting 14 bytes structure.
> > > > >
> > > > > Some trick like
> > > > > *(struct virtio_net_hdr *)hdr = {0, };
> > > > >
> > > > > Or even
> > > > > hdr->xxx = 0;
> > > > > hdr->yyy = 0;
> > > > >
> > > > > should behaviour better.
> > > > >
> > > > > There was an example: the vhost enqueue optmization patchset from
> > > > > Zhihong [0] uses memset, and it introduces more than 15% drop (IIRC)
> > > > > on my Ivybridge server: it has no such issue on his server though.
> > > > >
> > > > > [0]: http://dpdk.org/ml/archives/dev/2016-August/045272.html
> > > > >
> > > > >   --yliu
> > > >
> > > > I'd say that's weird. what's your config? any chance you
> > > > are using an old compiler?
> > > 
> > > Not really, it's gcc 5.3.1. Maybe Zhihong could explain more. IIRC,
> > > he said the memset is not well optimized for Ivybridge server.
> > 
> > The dst is remote in that case. It's fine on Haswell but has complication
> > in Ivy Bridge which (wasn't supposed to but) causes serious frontend issue.
> > 
> > I don't think gcc inlined it there. I'm using fc24 gcc 6.1.1.
> 
> 
> So try something like this then:

Yes, I saw memset is inlined when this diff is applied.

So, mind to send a formal patch? You might want to try build at least:
it doesn't build.

 
> Generally pointer chasing in vq->hw->vtnet_hdr_size can't be good
> for performance. Move fields used on data path into vq
> and use from there to avoid indirections?

Good suggestion!

--yliu

Re: [Qemu-devel] [PATCH v2 0/4] generic docker run patches

[Alex Bennée]

Alex Bennée  writes:

> Hi Fam,
>
> I've re-based the series for the generic run target. The aim being to
> allow a developer to run tests against any generic docker target even
> if it is not in the list of approved targets:

Ping?

>
>   make docker-run TEST=test-quick IMAGE=debian:arm64 \
> EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1
>
> The existing auto targets still work but have been converted to use
> the generic runner once they have done the validation.
>
> Since last post:
>   - review comments on the main patch
>   - added a travis docker file
>   - minor tweak to the test-mingw script
>
> Alex Bennée (4):
>   tests/docker: add travis dockerfile
>   tests/docker: test-build script
>   tests/docker: make test-mingw honour TARGET_LIST
>   tests/docker/Makefile.include: add a generic docker-run target
>
>  tests/docker/Makefile.include  | 89 
> +-
>  tests/docker/dockerfiles/travis.docker |  6 +++
>  tests/docker/test-build| 18 +++
>  tests/docker/test-mingw|  4 +-
>  4 files changed, 93 insertions(+), 24 deletions(-)
>  create mode 100644 tests/docker/dockerfiles/travis.docker
>  create mode 100755 tests/docker/test-build


--
Alex Bennée

Re: [Qemu-devel] [PATCH 1/2] vhost: enable any layout feature

[Yuanhan Liu]

On Tue, Oct 11, 2016 at 08:39:54AM +0200, Maxime Coquelin wrote:
> 
> 
> On 10/11/2016 08:04 AM, Yuanhan Liu wrote:
> >On Mon, Oct 10, 2016 at 04:54:39PM +0200, Maxime Coquelin wrote:
> >>
> >>
> >>On 10/10/2016 04:42 PM, Yuanhan Liu wrote:
> >>>On Mon, Oct 10, 2016 at 02:40:44PM +0200, Maxime Coquelin wrote:
> >>>At that time, a packet always use 2 descs. Since indirect desc is
> >>>enabled (by default) now, the assumption is not true then. What's
> >>>worse, it might even slow things a bit down. That should also be
> >>>part of the reason why performance is slightly worse than before.
> >>>
> >>>   --yliu
> >>
> >>I'm not sure I get what you are saying
> >>
> >>>commit 1d41d77cf81c448c1b09e1e859bfd300e2054a98
> >>>Author: Yuanhan Liu 
> >>>Date:   Mon May 2 17:46:17 2016 -0700
> >>>
> >>>  vhost: optimize dequeue for small packets
> >>>
> >>>  A virtio driver normally uses at least 2 desc buffers for Tx: the
> >>>  first for storing the header, and the others for storing the data.
> >>>
> >>>  Therefore, we could fetch the first data desc buf before the main
> >>>  loop, and do the copy first before the check of "are we done yet?".
> >>>  This could save one check for small packets that just have one data
> >>>  desc buffer and need one mbuf to store it.
> >>>
> >>>  Signed-off-by: Yuanhan Liu 
> >>>  Acked-by: Huawei Xie 
> >>>  Tested-by: Rich Lane 
> >>
> >>This fast-paths the 2-descriptors format but it's not active
> >>for indirect descriptors. Is this what you mean?
> >
> >Yes. It's also not active when ANY_LAYOUT is actually turned on.
> >>Should be a simple matter to apply this optimization for indirect.
> >
> >Might be.
> 
> If I understand the code correctly, indirect descs also benefit from this
> optimization, or am I missing something?
> >>>
> >>>Aha..., you are right!
> >>
> >>The interesting thing is that the patch I send on Thursday that removes
> >>header access when no offload has been negotiated[0] seems to reduce
> >>almost to zero the performance seen with indirect descriptors enabled.
> >
> >Didn't follow that.
> >
> >>I see this with 64 bytes packets using testpmd on both ends.
> >>
> >>When I did the patch, I would have expected the same gain with both
> >>modes, whereas I measured +1% for direct and +4% for indirect.
> >
> >IIRC, I did a test before (remove those offload code piece), and the
> >performance was basically the same before and after that. Well, there
> >might be some small difference, say 1% as you said. But the result has
> >never been steady.
> >
> >Anyway, I think your patch is good to have: I just didn't see v2.
> 
> I waited to gather some comments/feedback before sending the v2.
> I'll send it today or tomorrow.

Interesting, I saw a deadlock then: I haven't looked at the code
carefully once you said there is a v2, thus I'm waiting for it.
However, you are waitting for my review. :)

Anyway, I will take time to look at it shortly.

--yliu

Re: [Qemu-devel] [PATCH v5 13/35] tcg: Add atomic helpers

[Alex Bennée]

Richard Henderson  writes:

> Add all of cmpxchg, op_fetch, fetch_op, and xchg.
> Handle both endian-ness, and sizes up to 8.
> Handle expanding non-atomically, when emulating in serial.
>
> Signed-off-by: Richard Henderson 

This still seems to be suffering expansion problems on Travis:

  https://travis-ci.org/stsquad/qemu/builds/166466729

I'll see if I can come up with a reproducer with the docker targets.

> ---
>  Makefile.objs |   2 +-
>  Makefile.target   |   1 +
>  atomic_template.h | 173 ++
>  cputlb.c  | 112 -
>  include/qemu/atomic.h |  19 ++-
>  tcg-runtime.c |  49 ++--
>  tcg/tcg-op.c  | 328 
> ++
>  tcg/tcg-op.h  |  44 +++
>  tcg/tcg-runtime.h |  75 
>  tcg/tcg.h |  53 
>  10 files changed, 836 insertions(+), 20 deletions(-)
>  create mode 100644 atomic_template.h
>
> diff --git a/Makefile.objs b/Makefile.objs
> index 02fb8e7..99d1f6d 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -89,7 +89,7 @@ endif
>
>  ###
>  # Target-independent parts used in system and user emulation
> -common-obj-y += tcg-runtime.o cpus-common.o
> +common-obj-y += cpus-common.o
>  common-obj-y += hw/
>  common-obj-y += qom/
>  common-obj-y += disas/
> diff --git a/Makefile.target b/Makefile.target
> index 9968871..91b6fbd 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -94,6 +94,7 @@ obj-$(CONFIG_TCG_INTERPRETER) += disas/tci.o
>  obj-y += fpu/softfloat.o
>  obj-y += target-$(TARGET_BASE_ARCH)/
>  obj-y += disas.o
> +obj-y += tcg-runtime.o
>  obj-$(call notempty,$(TARGET_XML_FILES)) += gdbstub-xml.o
>  obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
>
> diff --git a/atomic_template.h b/atomic_template.h
> new file mode 100644
> index 000..d2c8a08
> --- /dev/null
> +++ b/atomic_template.h
> @@ -0,0 +1,173 @@
> +/*
> + * Atomic helper templates
> + * Included from tcg-runtime.c and cputlb.c.
> + *
> + * Copyright (c) 2016 Red Hat, Inc
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see 
> .
> + */
> +
> +#if DATA_SIZE == 8
> +# define SUFFIX q
> +# define DATA_TYPE  uint64_t
> +# define BSWAP  bswap64
> +#elif DATA_SIZE == 4
> +# define SUFFIX l
> +# define DATA_TYPE  uint32_t
> +# define BSWAP  bswap32
> +#elif DATA_SIZE == 2
> +# define SUFFIX w
> +# define DATA_TYPE  uint16_t
> +# define BSWAP  bswap16
> +#elif DATA_SIZE == 1
> +# define SUFFIX b
> +# define DATA_TYPE  uint8_t
> +# define BSWAP
> +#else
> +# error unsupported data size
> +#endif
> +
> +#if DATA_SIZE >= 4
> +# define ABI_TYPE  DATA_TYPE
> +#else
> +# define ABI_TYPE  uint32_t
> +#endif
> +
> +#if DATA_SIZE == 1
> +# define END
> +#elif defined(HOST_WORDS_BIGENDIAN)
> +# define END  _be
> +#else
> +# define END  _le
> +#endif
> +
> +ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
> +  ABI_TYPE cmpv, ABI_TYPE newv EXTRA_ARGS)
> +{
> +DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP;
> +return atomic_cmpxchg__nocheck(haddr, cmpv, newv);
> +}
> +
> +ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
> +   ABI_TYPE val EXTRA_ARGS)
> +{
> +DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP;
> +return atomic_xchg__nocheck(haddr, val);
> +}
> +
> +#define GEN_ATOMIC_HELPER(X)\
> +ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
> + ABI_TYPE val EXTRA_ARGS)   \
> +{   \
> +DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP;   \
> +return atomic_##X(haddr, val);  \
> +}   \
> +
> +GEN_ATOMIC_HELPER(fetch_add)
> +GEN_ATOMIC_HELPER(fetch_and)
> +GEN_ATOMIC_HELPER(fetch_or)
> +GEN_ATOMIC_HELPER(fetch_xor)
> +GEN_ATOMIC_HELPER(add_fetch)
> +GEN_ATOMIC_HELPER(and_fetch)
> +GEN_ATOMIC_HELPER(or_fetch)
> +GEN_ATOMIC_HELPER(xor_fetch)
> +
> +#undef GEN_ATOMIC_HELPER
> +#undef END
> +
> +#if DATA_SIZE > 1
> +
> +#ifdef HOST_WORDS_BIGENDIAN
> +# 

Re: [Qemu-devel] [PATCH 1/2] vhost: enable any layout feature

[Maxime Coquelin]

On 10/11/2016 08:04 AM, Yuanhan Liu wrote:

On Mon, Oct 10, 2016 at 04:54:39PM +0200, Maxime Coquelin wrote:



On 10/10/2016 04:42 PM, Yuanhan Liu wrote:

On Mon, Oct 10, 2016 at 02:40:44PM +0200, Maxime Coquelin wrote:

At that time, a packet always use 2 descs. Since indirect desc is
enabled (by default) now, the assumption is not true then. What's
worse, it might even slow things a bit down. That should also be
part of the reason why performance is slightly worse than before.

--yliu


I'm not sure I get what you are saying


commit 1d41d77cf81c448c1b09e1e859bfd300e2054a98
Author: Yuanhan Liu 
Date:   Mon May 2 17:46:17 2016 -0700

  vhost: optimize dequeue for small packets

  A virtio driver normally uses at least 2 desc buffers for Tx: the
  first for storing the header, and the others for storing the data.

  Therefore, we could fetch the first data desc buf before the main
  loop, and do the copy first before the check of "are we done yet?".
  This could save one check for small packets that just have one data
  desc buffer and need one mbuf to store it.

  Signed-off-by: Yuanhan Liu 
  Acked-by: Huawei Xie 
  Tested-by: Rich Lane 


This fast-paths the 2-descriptors format but it's not active
for indirect descriptors. Is this what you mean?


Yes. It's also not active when ANY_LAYOUT is actually turned on.

Should be a simple matter to apply this optimization for indirect.


Might be.


If I understand the code correctly, indirect descs also benefit from this
optimization, or am I missing something?


Aha..., you are right!


The interesting thing is that the patch I send on Thursday that removes
header access when no offload has been negotiated[0] seems to reduce
almost to zero the performance seen with indirect descriptors enabled.


Didn't follow that.


I see this with 64 bytes packets using testpmd on both ends.

When I did the patch, I would have expected the same gain with both
modes, whereas I measured +1% for direct and +4% for indirect.


IIRC, I did a test before (remove those offload code piece), and the
performance was basically the same before and after that. Well, there
might be some small difference, say 1% as you said. But the result has
never been steady.

Anyway, I think your patch is good to have: I just didn't see v2.


I waited to gather some comments/feedback before sending the v2.
I'll send it today or tomorrow.

Thanks,
Maxime

Re: [Qemu-devel] [RFC QEMU PATCH 8/8] qmp: add a qmp command 'query-nvdimms' to get plugged NVDIMM devices

[Haozhong Zhang]

On 10/10/16 14:16, Eric Blake wrote:
> On 10/09/2016 07:34 PM, Haozhong Zhang wrote:
> > Xen uses this command to get the backend resource, guest SPA and size of
> > NVDIMM devices so as to map them to guest.
> > 
> > Signed-off-by: Haozhong Zhang 
> > ---
> > Cc: Markus Armbruster 
> 
> > +++ b/docs/qmp-commands.txt
> > @@ -3800,3 +3800,39 @@ Example for pc machine type started with
> >  "props": {"core-id": 0, "socket-id": 0, "thread-id": 0}
> >   }
> > ]}
> > +
> > +EQMP
> > +
> > +{
> > +.name   = "query-nvdimms",
> > +.args_type  = "",
> > +.mhandler.cmd_new = qmp_marshal_query_nvdimms,
> 
> Needs rebasing - we no longer need SQMP/EQMP sections or callouts to the
> initializers, now that commit bd6092e4 has automated the mapping of QAPI
> to command registration.
>

good to know this, will change in next version

> > +},
> > +
> > +SQMP
> > +Show plugged NVDIMM devices
> > +---
> > +
> > +Arguments: None.
> > +
> > +Example for pc machine type started with
> > +-object memory-backend-file,id=mem1,mem-path=/path/to/nvm1,size=4G
> > +-device nvdimm,id=nvdimm1,memdev=mem1
> > +-object memory-backend-file,id=mem2,mem-path=/path/to/nvm2,size=8G
> > +-device nvdimm,id=nvdimm2,memdev=mem2:
> > +
> > +-> { "execute": "query-nvdimms" }
> > +<- { "returns": [
> > +  {
> > + "mem-path": "/path/to/nvm1",
> > +"slot": 0,
> 
> TAB damage; please fix.
>

will fix

> > +"spa": 17179869184,
> > +"length": 4294967296
> > +  },
> > +  {
> > + "mem-path": "/path/to/nvm2",
> > +"slot": 1,
> > +"spa": 21474836480,
> > +"length": 8589934592
> > +  }
> > +   ]}
> 
> > +++ b/qapi-schema.json
> > @@ -4646,3 +4646,32 @@
> >  # Since: 2.7
> >  ##
> >  { 'command': 'query-hotpluggable-cpus', 'returns': ['HotpluggableCPU'] }
> > +
> > +##
> > +# @NvdimmInfo
> > +#
> > +# Information about an NVDIMM device.
> > +#
> > +# @mem-path: the backend file of the NVDIMM device
> > +#
> > +# @slot: the slot index of the NVDIMM device
> > +#
> > +# @spa: the 64-bit SPA base address of the NVDIMM device
> > +#
> > +# @length: the 64-bit size in bytes of the NVDIMM device
> > +#
> > +# Since 2.8
> > +##
> > +{ 'struct': 'NvdimmInfo',
> > +  'data': {'mem-path' : 'str', 'slot': 'int', 'spa': 'int', 'length': 
> > 'int'} }
> > +
> > +##
> > +# @query-nvdimms:
> > +#
> > +# Returns information about each NVDIMM device
> > +#
> > +# Returns: a list of @NvdimmInfo for each device
> > +#
> > +# Since: 2.8
> > +##
> > +{ 'command': 'query-nvdimms', 'returns': ['NvdimmInfo'] }
> > 
> 
> Is this something that can be added to the existing query-memdev or
> query-memory-devices command, instead of needing a new command?
>

Both query-memdev and query-memory-devices do not tell whether a
device is NVDIMM. If adding an optional argument to existing a qmp
command does not break backwards compatibility, I propose to add an
argument "nvdimm" to query-memory-devices which controls whether this
command to return all memory devices or only NVDIMM devices.

Thanks,
Haozhong

Re: [Qemu-devel] [PATCH v1 2/2] block/replication: Clarify 'top-id' parameter usage

[Fam Zheng]

On Tue, 10/11 13:39, Changlong Xie wrote:
> Replication driver only support 'top-id' parameter in secondary side,
> and it must not be supplied in primary side
> 
> Signed-off-by: Changlong Xie 
> ---
>  block/replication.c  | 5 +
>  qapi/block-core.json | 3 ++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/block/replication.c b/block/replication.c
> index 5b432d9..1e8284b 100644
> --- a/block/replication.c
> +++ b/block/replication.c
> @@ -101,6 +101,11 @@ static int replication_open(BlockDriverState *bs, QDict 
> *options,
>  
>  if (!strcmp(mode, "primary")) {
>  s->mode = REPLICATION_MODE_PRIMARY;
> +top_id = qemu_opt_get(opts, REPLICATION_TOP_ID);
> +if (top_id) {
> +error_setg(_err, "The primary side do not support option 
> top-id");

s/do not/does not/

> +goto fail;
> +}
>  } else if (!strcmp(mode, "secondary")) {
>  s->mode = REPLICATION_MODE_SECONDARY;
>  top_id = qemu_opt_get(opts, REPLICATION_TOP_ID);
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 4badb97..ec92df4 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2197,7 +2197,8 @@
>  # @mode: the replication mode
>  #
>  # @top-id: #optional In secondary mode, node name or device ID of the root
> -#  node who owns the replication node chain. Ignored in primary mode.
> +#  node who owns the replication node chain. Must not be given in
> +#  primary mode.
>  #
>  # Since: 2.8
>  ##
> -- 
> 1.9.3
> 
> 
> 
> 

Re: [Qemu-devel] [PATCH 1/2] vhost: enable any layout feature

[Yuanhan Liu]

On Mon, Oct 10, 2016 at 04:54:39PM +0200, Maxime Coquelin wrote:
> 
> 
> On 10/10/2016 04:42 PM, Yuanhan Liu wrote:
> >On Mon, Oct 10, 2016 at 02:40:44PM +0200, Maxime Coquelin wrote:
> >At that time, a packet always use 2 descs. Since indirect desc is
> >enabled (by default) now, the assumption is not true then. What's
> >worse, it might even slow things a bit down. That should also be
> >part of the reason why performance is slightly worse than before.
> >
> > --yliu
> 
> I'm not sure I get what you are saying
> 
> >commit 1d41d77cf81c448c1b09e1e859bfd300e2054a98
> >Author: Yuanhan Liu 
> >Date:   Mon May 2 17:46:17 2016 -0700
> >
> >   vhost: optimize dequeue for small packets
> >
> >   A virtio driver normally uses at least 2 desc buffers for Tx: the
> >   first for storing the header, and the others for storing the data.
> >
> >   Therefore, we could fetch the first data desc buf before the main
> >   loop, and do the copy first before the check of "are we done yet?".
> >   This could save one check for small packets that just have one data
> >   desc buffer and need one mbuf to store it.
> >
> >   Signed-off-by: Yuanhan Liu 
> >   Acked-by: Huawei Xie 
> >   Tested-by: Rich Lane 
> 
> This fast-paths the 2-descriptors format but it's not active
> for indirect descriptors. Is this what you mean?
> >>>
> >>>Yes. It's also not active when ANY_LAYOUT is actually turned on.
> Should be a simple matter to apply this optimization for indirect.
> >>>
> >>>Might be.
> >>
> >>If I understand the code correctly, indirect descs also benefit from this
> >>optimization, or am I missing something?
> >
> >Aha..., you are right!
> 
> The interesting thing is that the patch I send on Thursday that removes
> header access when no offload has been negotiated[0] seems to reduce
> almost to zero the performance seen with indirect descriptors enabled.

Didn't follow that.

> I see this with 64 bytes packets using testpmd on both ends.
> 
> When I did the patch, I would have expected the same gain with both
> modes, whereas I measured +1% for direct and +4% for indirect.

IIRC, I did a test before (remove those offload code piece), and the
performance was basically the same before and after that. Well, there
might be some small difference, say 1% as you said. But the result has
never been steady.

Anyway, I think your patch is good to have: I just didn't see v2.

--yliu