Re: [PATCH 2/3] vdpa: load vlan configuration at NIC startup

2022-09-14 Thread Jason Wang
On Wed, Sep 14, 2022 at 11:44 PM Si-Wei Liu  wrote:
>
>
>
> On 9/14/2022 2:57 PM, Eugenio Perez Martin wrote:
> > On Wed, Sep 14, 2022 at 1:33 PM Si-Wei Liu  wrote:
> >>
> >>
> >> On 9/14/2022 3:20 AM, Jason Wang wrote:
> >>> On Fri, Sep 9, 2022 at 4:02 PM Eugenio Perez Martin  
> >>> wrote:
>  On Fri, Sep 9, 2022 at 8:40 AM Jason Wang  wrote:
> > On Fri, Sep 9, 2022 at 2:38 PM Jason Wang  wrote:
> >> On Wed, Sep 7, 2022 at 12:36 AM Eugenio Pérez  
> >> wrote:
> >>> To have enabled vlans at device startup may happen in the destination 
> >>> of
> >>> a live migration, so this configuration must be restored.
> >>>
> >>> At this moment the code is not accessible, since SVQ refuses to start 
> >>> if
> >>> vlan feature is exposed by the device.
> >>>
> >>> Signed-off-by: Eugenio Pérez 
> >>> ---
> >>>net/vhost-vdpa.c | 46 
> >>> --
> >>>1 file changed, 44 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
> >>> index 4bc3fd01a8..ecbfd08eb9 100644
> >>> --- a/net/vhost-vdpa.c
> >>> +++ b/net/vhost-vdpa.c
> >>> @@ -100,6 +100,8 @@ static const uint64_t vdpa_svq_device_features =
> >>>BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
> >>>BIT_ULL(VIRTIO_NET_F_STANDBY);
> >>>
> >>> +#define MAX_VLAN(1 << 12)   /* Per 802.1Q definition */
> >>> +
> >>>VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
> >>>{
> >>>VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
> >>> @@ -423,6 +425,47 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState 
> >>> *s,
> >>>return *s->status != VIRTIO_NET_OK;
> >>>}
> >>>
> >>> +static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s,
> >>> +   const VirtIONet *n,
> >>> +   uint16_t vid)
> >>> +{
> >>> +ssize_t dev_written = vhost_vdpa_net_load_cmd(s, 
> >>> VIRTIO_NET_CTRL_VLAN,
> >>> +  
> >>> VIRTIO_NET_CTRL_VLAN_ADD,
> >>> +  , sizeof(vid));
> >>> +if (unlikely(dev_written < 0)) {
> >>> +return dev_written;
> >>> +}
> >>> +
> >>> +if (unlikely(*s->status != VIRTIO_NET_OK)) {
> >>> +return -EINVAL;
> >>> +}
> >>> +
> >>> +return 0;
> >>> +}
> >>> +
> >>> +static int vhost_vdpa_net_load_vlan(VhostVDPAState *s,
> >>> +const VirtIONet *n)
> >>> +{
> >>> +uint64_t features = n->parent_obj.guest_features;
> >>> +
> >>> +if (!(features & BIT_ULL(VIRTIO_NET_F_CTRL_VLAN))) {
> >>> +return 0;
> >>> +}
> >>> +
> >>> +for (int i = 0; i < MAX_VLAN >> 5; i++) {
> >>> +for (int j = 0; n->vlans[i] && j <= 0x1f; j++) {
> >>> +if (n->vlans[i] & (1U << j)) {
> >>> +int r = vhost_vdpa_net_load_single_vlan(s, n, (i << 
> >>> 5) + j);
> >> This seems to cause a lot of latency if the driver has a lot of vlans.
> >>
> >> I wonder if it's simply to let all vlan traffic go by disabling
> >> CTRL_VLAN feature at vDPA layer.
>  The guest will not be able to recover that vlan configuration.
> >>> Spec said it's best effort and we actually don't do this for
> >>> vhost-net/user for years and manage to survive.
> >> I thought there's a border line between best effort and do nothing. ;-)
> >>
> > I also think it is worth at least trying to migrate it for sure. For
> > MAC there may be too many entries, but VLAN should be totally
> > manageable (and the information is already there, the bitmap is
> > actually being migrated).
> The vlan bitmap is migrated, though you'd still need to add vlan filter
> one by one through ctrl vq commands, correct? AFAIK you can add one
> filter for one single vlan ID at a time via VIRTIO_NET_CTRL_VLAN_ADD.
>
> >
> >> I think that the reason this could survive is really software
> >> implementation specific - say if the backend doesn't start with VLAN
> >> filtering disabled (meaning allow all VLAN traffic to pass) then it
> >> would become a problem. This won't be a problem for almost PF NICs but
> >> may be problematic for vDPA e.g. VF backed VDPAs.
> > I'd say the driver would expect all vlan filters to be cleared after a
> > reset, isn't it?
> If I understand the intended behavior (from QEMU implementation)
> correctly, when VIRTIO_NET_F_CTRL_VLAN is negotiated it would start with
> all VLANs filtered (meaning only untagged traffic can be received and
> traffic with VLAN tag would get dropped). However, if
> VIRTIO_NET_F_CTRL_VLAN is not negotiated, all traffic including untagged
> and tagged can be received.
>
> >   The spec doesn't 

Re: [PATCH 2/3] vdpa: load vlan configuration at NIC startup

2022-09-14 Thread Jason Wang
On Wed, Sep 14, 2022 at 7:33 PM Si-Wei Liu  wrote:
>
>
>
> On 9/14/2022 3:20 AM, Jason Wang wrote:
> > On Fri, Sep 9, 2022 at 4:02 PM Eugenio Perez Martin  
> > wrote:
> >> On Fri, Sep 9, 2022 at 8:40 AM Jason Wang  wrote:
> >>> On Fri, Sep 9, 2022 at 2:38 PM Jason Wang  wrote:
>  On Wed, Sep 7, 2022 at 12:36 AM Eugenio Pérez  
>  wrote:
> > To have enabled vlans at device startup may happen in the destination of
> > a live migration, so this configuration must be restored.
> >
> > At this moment the code is not accessible, since SVQ refuses to start if
> > vlan feature is exposed by the device.
> >
> > Signed-off-by: Eugenio Pérez 
> > ---
> >   net/vhost-vdpa.c | 46 --
> >   1 file changed, 44 insertions(+), 2 deletions(-)
> >
> > diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
> > index 4bc3fd01a8..ecbfd08eb9 100644
> > --- a/net/vhost-vdpa.c
> > +++ b/net/vhost-vdpa.c
> > @@ -100,6 +100,8 @@ static const uint64_t vdpa_svq_device_features =
> >   BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
> >   BIT_ULL(VIRTIO_NET_F_STANDBY);
> >
> > +#define MAX_VLAN(1 << 12)   /* Per 802.1Q definition */
> > +
> >   VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
> >   {
> >   VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
> > @@ -423,6 +425,47 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState 
> > *s,
> >   return *s->status != VIRTIO_NET_OK;
> >   }
> >
> > +static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s,
> > +   const VirtIONet *n,
> > +   uint16_t vid)
> > +{
> > +ssize_t dev_written = vhost_vdpa_net_load_cmd(s, 
> > VIRTIO_NET_CTRL_VLAN,
> > +  
> > VIRTIO_NET_CTRL_VLAN_ADD,
> > +  , sizeof(vid));
> > +if (unlikely(dev_written < 0)) {
> > +return dev_written;
> > +}
> > +
> > +if (unlikely(*s->status != VIRTIO_NET_OK)) {
> > +return -EINVAL;
> > +}
> > +
> > +return 0;
> > +}
> > +
> > +static int vhost_vdpa_net_load_vlan(VhostVDPAState *s,
> > +const VirtIONet *n)
> > +{
> > +uint64_t features = n->parent_obj.guest_features;
> > +
> > +if (!(features & BIT_ULL(VIRTIO_NET_F_CTRL_VLAN))) {
> > +return 0;
> > +}
> > +
> > +for (int i = 0; i < MAX_VLAN >> 5; i++) {
> > +for (int j = 0; n->vlans[i] && j <= 0x1f; j++) {
> > +if (n->vlans[i] & (1U << j)) {
> > +int r = vhost_vdpa_net_load_single_vlan(s, n, (i << 5) 
> > + j);
>  This seems to cause a lot of latency if the driver has a lot of vlans.
> 
>  I wonder if it's simply to let all vlan traffic go by disabling
>  CTRL_VLAN feature at vDPA layer.
> >> The guest will not be able to recover that vlan configuration.
> > Spec said it's best effort and we actually don't do this for
> > vhost-net/user for years and manage to survive.
> I thought there's a border line between best effort and do nothing. ;-)
>
> I think that the reason this could survive is really software
> implementation specific - say if the backend doesn't start with VLAN
> filtering disabled (meaning allow all VLAN traffic to pass) then it
> would become a problem. This won't be a problem for almost PF NICs but
> may be problematic for vDPA e.g. VF backed VDPAs.

So it looks like an issue of the implementation. If CTRL_VLAN is not
negotiated, the device should disable vlan filters.


> >
> >>> Another idea is to extend the spec to allow us to accept a bitmap of
> >>> the vlan ids via a single command, then we will be fine.
> >>>
> >> I'm not sure if adding more ways to configure something is the answer,
> >> but I'd be ok to implement it.
> >>
> >> Another idea is to allow the sending of many CVQ commands in parallel.
> >> It shouldn't be very hard to achieve using exposed buffers as ring
> >> buffers, and it will short down the start of the devices with many
> >> features.
> > In the extreme case, what if guests decide to filter all of the vlans?
> > It means we need MAX_VLAN commands which may exceeds the size of the
> > control virtqueue.
> Indeed, that's a case where a single flat device state blob would be
> more efficient for hardware drivers to apply than individual control
> command messages do.

Right, so we can optimize the spec for this.

Thanks

>
> -Siwei
> >
> > Thanks
> >
> >> Thanks!
> >>
> >>> Thanks
> >>>
>  Thanks
> 
> > +if (unlikely(r != 0)) {
> > +return r;
> > +}
> > +}
> > +}
> > +}
> > +
> > +

Re: [PATCH v2 3/6] vhost-net: vhost-user: update vhost_net_virtqueue_reset()

2022-09-14 Thread Jason Wang
On Wed, Sep 14, 2022 at 2:21 PM Xuan Zhuo  wrote:
>
> On Wed, 14 Sep 2022 11:13:29 +0800, Jason Wang  wrote:
> >
> > 在 2022/9/12 11:10, Kangjie Xu 写道:
> > > Update vhost_net_virtqueue_reset() for vhost-user scenario.
> > >
> > > In order to reuse some functions, we process the idx for
> > > vhost-user scenario because vhost_get_vq_index behave
> > > differently for vhost-user.
> > >
> > > Signed-off-by: Kangjie Xu 
> > > Signed-off-by: Xuan Zhuo 
> > > ---
> > >   hw/net/vhost_net.c | 3 +++
> > >   1 file changed, 3 insertions(+)
> > >
> > > diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> > > index ea896ea75b..25e5665489 100644
> > > --- a/hw/net/vhost_net.c
> > > +++ b/hw/net/vhost_net.c
> > > @@ -545,6 +545,9 @@ void vhost_net_virtqueue_reset(VirtIODevice *vdev, 
> > > NetClientState *nc,
> > >   assert(vhost_ops);
> > >
> > >   idx = vhost_ops->vhost_get_vq_index(>dev, vq_index);
> > > +if (net->nc->info->type == NET_CLIENT_DRIVER_VHOST_USER) {
> > > +idx -= net->dev.vq_index;
> > > +}
> >
> >
> > This looks tricky. Any reason we can't simply use vq_index for both
> > vhost-user and vhost-net?
>
>
> static int vhost_user_get_vq_index(struct vhost_dev *dev, int idx)
> {
> assert(idx >= dev->vq_index && idx < dev->vq_index + dev->nvqs);
>
> return idx;
> }
>
>
> static int vhost_kernel_get_vq_index(struct vhost_dev *dev, int idx)
> {
> assert(idx >= dev->vq_index && idx < dev->vq_index + dev->nvqs);
>
> return idx - dev->vq_index;
> }
>
> The implementation of these two callbacks is different. The structure of the 
> two
> scenarios is different. We may need to do some optimizations in the future.

Yes, but what I meant is, you do

idx -= net->dev.vq_index;

and then

net->dev.vq_index + idx

This is a hint that we should have a better organization of the code.

Thanks

>
> Thanks.
>
>
> >
> > Thanks
> >
> >
> > >
> > >   if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) {
> > >   file.index = idx;
> >
>




Re:Re: [PATCH] bugfix:migrate with block-dirty-bitmap (disk size is big enough) can't be finished

2022-09-14 Thread liuhaiwei9699
Hi ,Vladimir
sometimes ,post-copy mode is not the best choice. For instance, Supposing 
migrate process will take ten minutes,but network may be interruptted In this 
process .
If it does happenthe , memory data of VM will be splitted into two parts, and 
will not be rollback.This is a bad situation


so  migrate-start-postcopy will not be setted in conservative scenario. In this 
case, the migration with block dirty bitmap may not be finished.




I think  migration of block dirty bitmap should not dependent on post-copy or 
pre-copy mode.




Best regards
At 2022-09-10 18:18:04, "Vladimir Sementsov-Ogievskiy" 
 wrote:
>On 9/10/22 09:35, liuhaiwei wrote:
>> From: liuhaiwei 
>> 
>> bug description as  https://gitlab.com/qemu-project/qemu/-/issues/1203
>> Usually,we use the precopy or postcopy mode to migrate block dirty bitmap.
>> but if block-dirty-bitmap size more than threshold size,we cannot entry the 
>> migration_completion in migration_iteration_run function
>> To solve this problem, we can setting  the pending size to a fake 
>> value(threshold-1 or 0) to tell  migration_iteration_run function to entry 
>> the migration_completion,if pending size > threshold size
>> 
>
>
>Actually, bitmaps migrate in postcopy. So, you should start postcopy for it to 
>work (qmp command migrate-start-postcopy). This command simply set the boolean 
>variable, so that in migration_iteration_run() we'll move to postcopy when 
>needed. So, you can start this command immediately after migrate command, or 
>even before it, but after setting the "dirty-bitmaps" capability.
>
>Fake pending is a wrong thing to do, it means that you will make downtime to 
>be larger than expected.
>
>-- 
>Best regards,
>Vladimir


Re: [PATCH 1/2] target/i386: fix cmpxchg with 32-bit register destination

2022-09-14 Thread Paolo Bonzini

On 9/12/22 09:55, Richard Henderson wrote:


  * Compute...

Why bother passing NULL, and fixing it up at each use...

+static void gen_op_mov_reg_v(DisasContext *s, MemOp ot, int reg, TCGv 
t0)

+{
+    gen_op_deposit_reg_v(s, ot, reg, NULL, t0);
  }


... when you can just as easily pass in the register here?


Because dest can be fixed up to either cpu_regs[reg - 4] for high-byte 
registers, or cpu_regs[reg] for everything else.



We have an outstanding bug report that suggests that the move to eax must use 
the deposit in both cases:

https://gitlab.com/qemu-project/qemu/-/issues/508


Ok, so that's two bugs.  But both of them can indeed be fixed with 
gen_op_deposit_reg_v.


Paolo




Re: [PATCH 27/37] target/i386: Use tcg gvec ops for pmovmskb

2022-09-14 Thread Paolo Bonzini
On Tue, Sep 13, 2022 at 10:17 AM Richard Henderson
 wrote:
>
> On 9/12/22 00:04, Paolo Bonzini wrote:
> > +while (vec_len > 8) {
> > +vec_len -= 8;
> > +tcg_gen_shli_tl(s->T0, s->T0, 8);
> > +tcg_gen_ld8u_tl(t, cpu_env, offsetof(CPUX86State, 
> > xmm_t0.ZMM_B(vec_len - 1)));
> > +tcg_gen_or_tl(s->T0, s->T0, t);
> >   }
>
> The shl + or is deposit, for those hosts that have it,
> and will be re-expanded to shl + or for those that don't:
>
>  tcg_gen_ld8u_tl(t, ...);
>  tcg_gen_deposit_tl(s->T0, t, s->T0, 8, TARGET_LONG_BITS - 8);

What you get from that is an shl(t, 56) followed by extract2 (i.e.
SHRD). Yeah there are targets with a native deposit (x86 itself could
add PDEP/PEXT support I guess) but I find it hard to believe that it
outperforms a simple shl + or.

If we want to get clever, I should instead load ZMM_B(vec_len - 1)
directly into the *high* byte of t, using ZMM_L or ZMM_Q, and then
issue the extract2 myself.

Paolo




Re: [PATCH 30/37] target/i386: reimplement 0x0f 0x10-0x17, add AVX

2022-09-14 Thread Paolo Bonzini
On Tue, Sep 13, 2022 at 12:14 PM Richard Henderson
 wrote:
> > +static void gen_VMOVLPx(DisasContext *s, CPUX86State *env, X86DecodedInsn 
> > *decode)
> > +{
> > +int vec_len = sse_vec_len(s, decode);
> > +
> > +tcg_gen_ld_i64(s->tmp1_i64, cpu_env, decode->op[2].offset + 
> > offsetof(XMMReg, XMM_Q(0)));
> > +tcg_gen_gvec_mov(MO_64, decode->op[0].offset, decode->op[1].offset, 
> > vec_len, vec_len);
> > +tcg_gen_st_i64(s->tmp1_i64, cpu_env, decode->op[0].offset + 
> > offsetof(XMMReg, XMM_Q(0)));
> > +}
>
> You've just been moving i64 pieces in the other functions, why is this one 
> different using
> a gvec move in the middle?  I do wonder if a generic helper moving 
> offset->offset, with
> the comparison wouldn't be helpful within these functions, even when you know 
> off1 !=
> off2, due to Q(0) vs Q(1).

Because this one is the only one that has a VEX.256 version (the
operand is type "x"
rather than "dq" as in MOVHLPS, MOVLHPS, MOVHPx).

Paolo


Paolo




Re: [PATCH v2 04/30] tests/avocado: add explicit timeout for s390 TCG tests

2022-09-14 Thread Thomas Huth

On 14/09/2022 16.59, Alex Bennée wrote:

We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée 
---
  tests/avocado/boot_linux.py | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index 67a24fe51c..4f07c27ac6 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -130,6 +130,8 @@ class BootLinuxS390X(LinuxTest):
  :avocado: tags=arch:s390x
  """
  
+timeout = 240

+
  @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
  def test_s390_ccw_virtio_tcg(self):
  """


Reviewed-by: Thomas Huth 




Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Nicolin Chen
On Wed, Sep 14, 2022 at 02:03:25PM -0600, Alex Williamson wrote:

> > > > +container->pgsizes = 4096;
> > >
> > > This might be a separate question/issue: I wonder if we should use
> > > "sysconf(_SC_PAGE_SIZE)" here instead of 4096.
> > >
> > > With a kernel using a larger page size, e.g. CONFIG_ARM64_64K_PAGES,
> > > the IO page size is likely to be 64K too. If the ioctl fails, this
> > > default 4K setup won't work.
> >
> > Perhaps, but IIRC this solution came about because we originally forgot
> > to expose the IOMMU_INFO flag to indicate the pgsize field was valid.
> > At the time we only supported 4K systems, so it made sense to provide
> > this default, though it is indeed dated.
> >
> > TBH, I don't really see why we should try to continue if the ioctl
> > itself fails, so maybe this should be:

OK. Makes sense to me.

> > diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> > index ace9562a9ba1..ad188b7649e6 100644
> > --- a/hw/vfio/common.c
> > +++ b/hw/vfio/common.c
> > @@ -2111,29 +2111,31 @@ static int vfio_connect_container(VFIOGroup *group, 
> > AddressSpace *as,
> >  {
> >  struct vfio_iommu_type1_info *info;
> >
> > -/*
> > - * FIXME: This assumes that a Type1 IOMMU can map any 64-bit
> > - * IOVA whatsoever.  That's not actually true, but the current
> > - * kernel interface doesn't tell us what it can map, and the
> > - * existing Type1 IOMMUs generally support any IOVA we're
> > - * going to actually try in practice.
> > - */
> >  ret = vfio_get_iommu_info(container, );
> > +if (ret) {
> 
> Clearly untested,
> 
>ret = -errno;
> 
> > +error_setg_errno(errp, -ret, "Failed to get VFIO IOMMU info");
> > +goto enable_discards_exit:;

There is a ":" in-between :)



Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Alex Williamson
On Wed, 14 Sep 2022 13:53:39 -0600
Alex Williamson  wrote:

> On Wed, 14 Sep 2022 12:02:56 -0700
> Nicolin Chen  wrote:
> 
> > Hi Alex,
> > 
> > On Wed, Sep 14, 2022 at 12:10:29PM -0600, Alex Williamson wrote:
> >   
> > > > > > Its caller vfio_connect_container() assigns a default value
> > > > > > to info->iova_pgsizes, even if vfio_get_iommu_info() fails.
> > > > > > This would result in a "Segmentation fault" error, when the
> > > > > > VFIO_IOMMU_GET_INFO ioctl errors out.
> > > > > >
> > > > > > Since the caller has g_free already, drop the g_free in its
> > > > > > rollback routine and add a line of comments to highlight it.
> > > > >
> > > > > There's basically two ways to fix this:
> > > > >
> > > > > - return *info in any case, even on error
> > > > > - free *info on error, and make sure that the caller doesn't try to
> > > > >   access *info if the function returned !0
> > > > >
> > > > > The problem with the first option is that the caller will access 
> > > > > invalid
> > > > > information if it neglects to check the return code, and that might 
> > > > > lead
> > > > > to not-that-obvious errors; in the second case, a broken caller would 
> > > > > at
> > > > > least fail quickly with a segfault. The current code is easier to fix
> > > > > with the first option.
> > > > >
> > > > > I think I'd prefer the second option; but obviously maintainer's 
> > > > > choice.
> > > >
> > > > The caller does check rc all the time. So I made a smaller fix
> > > > (the first option). Attaching the git-diff for the second one.
> > > >
> > > > Alex, please let me know which one you prefer. Thanks!
> >   
> > > I think we can do better than that, I don't think we need to maintain
> > > the existing grouping, and that FIXME comment is outdated and has
> > > drifted from the relevant line of code.  What about:
> > 
> > Your patch looks good and clean to me (some nits inline).
> > 
> > Would you please send and merge it, replacing mine?
> >   
> > > +   /*
> > > + * Setup defaults for container pgsizes and dma_max_mappings if 
> > > not
> > > + * provided by kernel below.
> > >   */
> > 
> > Indentation is misaligned at the first line.  
> 
> Oops, will run through checkpatch.
> 
> >   
> > > +container->pgsizes = 4096;
> > 
> > This might be a separate question/issue: I wonder if we should use
> > "sysconf(_SC_PAGE_SIZE)" here instead of 4096.
> > 
> > With a kernel using a larger page size, e.g. CONFIG_ARM64_64K_PAGES,
> > the IO page size is likely to be 64K too. If the ioctl fails, this
> > default 4K setup won't work.  
> 
> Perhaps, but IIRC this solution came about because we originally forgot
> to expose the IOMMU_INFO flag to indicate the pgsize field was valid.
> At the time we only supported 4K systems, so it made sense to provide
> this default, though it is indeed dated.
> 
> TBH, I don't really see why we should try to continue if the ioctl
> itself fails, so maybe this should be:
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index ace9562a9ba1..ad188b7649e6 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -2111,29 +2111,31 @@ static int vfio_connect_container(VFIOGroup *group, 
> AddressSpace *as,
>  {
>  struct vfio_iommu_type1_info *info;
>  
> -/*
> - * FIXME: This assumes that a Type1 IOMMU can map any 64-bit
> - * IOVA whatsoever.  That's not actually true, but the current
> - * kernel interface doesn't tell us what it can map, and the
> - * existing Type1 IOMMUs generally support any IOVA we're
> - * going to actually try in practice.
> - */
>  ret = vfio_get_iommu_info(container, );
> +if (ret) {

Clearly untested,

   ret = -errno;

> +error_setg_errno(errp, -ret, "Failed to get VFIO IOMMU info");
> +goto enable_discards_exit:;
> +}
>  
> -if (ret || !(info->flags & VFIO_IOMMU_INFO_PGSIZES)) {
> -/* Assume 4k IOVA page size */
> -info->iova_pgsizes = 4096;
> +if (info->flags & VFIO_IOMMU_INFO_PGSIZES) {
> +container->pgsizes = info->iova_pgsizes;
> +} else {
> +container->pgsizes = qemu_real_host_page_size();
>  }
> -vfio_host_win_add(container, 0, (hwaddr)-1, info->iova_pgsizes);
> -container->pgsizes = info->iova_pgsizes;
>  
> -/* The default in the kernel ("dma_entry_limit") is 65535. */
> -container->dma_max_mappings = 65535;
> -if (!ret) {
> -vfio_get_info_dma_avail(info, >dma_max_mappings);
> -vfio_get_iommu_info_migration(container, info);
> +if (!vfio_get_info_dma_avail(info, >dma_max_mappings)) {
> +container->dma_max_mappings = 65535;
>  }
> +vfio_get_iommu_info_migration(container, info);
>  g_free(info);
> +
> +/*
> + * FIXME: We should parse 

Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Alex Williamson
On Wed, 14 Sep 2022 12:02:56 -0700
Nicolin Chen  wrote:

> Hi Alex,
> 
> On Wed, Sep 14, 2022 at 12:10:29PM -0600, Alex Williamson wrote:
> 
> > > > > Its caller vfio_connect_container() assigns a default value
> > > > > to info->iova_pgsizes, even if vfio_get_iommu_info() fails.
> > > > > This would result in a "Segmentation fault" error, when the
> > > > > VFIO_IOMMU_GET_INFO ioctl errors out.
> > > > >
> > > > > Since the caller has g_free already, drop the g_free in its
> > > > > rollback routine and add a line of comments to highlight it.  
> > > >
> > > > There's basically two ways to fix this:
> > > >
> > > > - return *info in any case, even on error
> > > > - free *info on error, and make sure that the caller doesn't try to
> > > >   access *info if the function returned !0
> > > >
> > > > The problem with the first option is that the caller will access invalid
> > > > information if it neglects to check the return code, and that might lead
> > > > to not-that-obvious errors; in the second case, a broken caller would at
> > > > least fail quickly with a segfault. The current code is easier to fix
> > > > with the first option.
> > > >
> > > > I think I'd prefer the second option; but obviously maintainer's 
> > > > choice.  
> > >
> > > The caller does check rc all the time. So I made a smaller fix
> > > (the first option). Attaching the git-diff for the second one.
> > >
> > > Alex, please let me know which one you prefer. Thanks!  
> 
> > I think we can do better than that, I don't think we need to maintain
> > the existing grouping, and that FIXME comment is outdated and has
> > drifted from the relevant line of code.  What about:  
> 
> Your patch looks good and clean to me (some nits inline).
> 
> Would you please send and merge it, replacing mine?
> 
> > +   /*
> > + * Setup defaults for container pgsizes and dma_max_mappings if not
> > + * provided by kernel below.
> >   */  
> 
> Indentation is misaligned at the first line.

Oops, will run through checkpatch.

> 
> > +container->pgsizes = 4096;  
> 
> This might be a separate question/issue: I wonder if we should use
> "sysconf(_SC_PAGE_SIZE)" here instead of 4096.
> 
> With a kernel using a larger page size, e.g. CONFIG_ARM64_64K_PAGES,
> the IO page size is likely to be 64K too. If the ioctl fails, this
> default 4K setup won't work.

Perhaps, but IIRC this solution came about because we originally forgot
to expose the IOMMU_INFO flag to indicate the pgsize field was valid.
At the time we only supported 4K systems, so it made sense to provide
this default, though it is indeed dated.

TBH, I don't really see why we should try to continue if the ioctl
itself fails, so maybe this should be:

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index ace9562a9ba1..ad188b7649e6 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -2111,29 +2111,31 @@ static int vfio_connect_container(VFIOGroup *group, 
AddressSpace *as,
 {
 struct vfio_iommu_type1_info *info;
 
-/*
- * FIXME: This assumes that a Type1 IOMMU can map any 64-bit
- * IOVA whatsoever.  That's not actually true, but the current
- * kernel interface doesn't tell us what it can map, and the
- * existing Type1 IOMMUs generally support any IOVA we're
- * going to actually try in practice.
- */
 ret = vfio_get_iommu_info(container, );
+if (ret) {
+error_setg_errno(errp, -ret, "Failed to get VFIO IOMMU info");
+goto enable_discards_exit:;
+}
 
-if (ret || !(info->flags & VFIO_IOMMU_INFO_PGSIZES)) {
-/* Assume 4k IOVA page size */
-info->iova_pgsizes = 4096;
+if (info->flags & VFIO_IOMMU_INFO_PGSIZES) {
+container->pgsizes = info->iova_pgsizes;
+} else {
+container->pgsizes = qemu_real_host_page_size();
 }
-vfio_host_win_add(container, 0, (hwaddr)-1, info->iova_pgsizes);
-container->pgsizes = info->iova_pgsizes;
 
-/* The default in the kernel ("dma_entry_limit") is 65535. */
-container->dma_max_mappings = 65535;
-if (!ret) {
-vfio_get_info_dma_avail(info, >dma_max_mappings);
-vfio_get_iommu_info_migration(container, info);
+if (!vfio_get_info_dma_avail(info, >dma_max_mappings)) {
+container->dma_max_mappings = 65535;
 }
+vfio_get_iommu_info_migration(container, info);
 g_free(info);
+
+/*
+ * FIXME: We should parse VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE
+ * information to get the actual window extent rather than assume
+ * a 64-bit IOVA address space.
+ */
+vfio_host_win_add(container, 0, (hwaddr)-1, container->pgsizes);
+
 break;
 }
 case VFIO_SPAPR_TCE_v2_IOMMU:




Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Nicolin Chen
Hi Alex,

On Wed, Sep 14, 2022 at 12:10:29PM -0600, Alex Williamson wrote:

> > > > Its caller vfio_connect_container() assigns a default value
> > > > to info->iova_pgsizes, even if vfio_get_iommu_info() fails.
> > > > This would result in a "Segmentation fault" error, when the
> > > > VFIO_IOMMU_GET_INFO ioctl errors out.
> > > >
> > > > Since the caller has g_free already, drop the g_free in its
> > > > rollback routine and add a line of comments to highlight it.
> > >
> > > There's basically two ways to fix this:
> > >
> > > - return *info in any case, even on error
> > > - free *info on error, and make sure that the caller doesn't try to
> > >   access *info if the function returned !0
> > >
> > > The problem with the first option is that the caller will access invalid
> > > information if it neglects to check the return code, and that might lead
> > > to not-that-obvious errors; in the second case, a broken caller would at
> > > least fail quickly with a segfault. The current code is easier to fix
> > > with the first option.
> > >
> > > I think I'd prefer the second option; but obviously maintainer's choice.
> >
> > The caller does check rc all the time. So I made a smaller fix
> > (the first option). Attaching the git-diff for the second one.
> >
> > Alex, please let me know which one you prefer. Thanks!

> I think we can do better than that, I don't think we need to maintain
> the existing grouping, and that FIXME comment is outdated and has
> drifted from the relevant line of code.  What about:

Your patch looks good and clean to me (some nits inline).

Would you please send and merge it, replacing mine?

> +   /*
> + * Setup defaults for container pgsizes and dma_max_mappings if not
> + * provided by kernel below.
>   */

Indentation is misaligned at the first line.

> +container->pgsizes = 4096;

This might be a separate question/issue: I wonder if we should use
"sysconf(_SC_PAGE_SIZE)" here instead of 4096.

With a kernel using a larger page size, e.g. CONFIG_ARM64_64K_PAGES,
the IO page size is likely to be 64K too. If the ioctl fails, this
default 4K setup won't work.

Thanks!
Nic



Re: [PATCH v8 11/14] qemu-sockets: move and rename SocketAddress_to_str()

2022-09-14 Thread Dr. David Alan Gilbert
* Laurent Vivier (lviv...@redhat.com) wrote:
> Rename SocketAddress_to_str() to socket_uri() and move it to
> util/qemu-sockets.c close to socket_parse().
> 
> socket_uri() generates a string from a SocketAddress while
> socket_parse() generates a SocketAddress from a string.
> 
> Signed-off-by: Laurent Vivier 

Reviewed-by: Dr. David Alan Gilbert 

> ---
>  include/qemu/sockets.h |  2 +-
>  monitor/hmp-cmds.c | 23 +--
>  util/qemu-sockets.c| 20 
>  3 files changed, 22 insertions(+), 23 deletions(-)
> 
> diff --git a/include/qemu/sockets.h b/include/qemu/sockets.h
> index 47194b9732f8..e5a06d2e3729 100644
> --- a/include/qemu/sockets.h
> +++ b/include/qemu/sockets.h
> @@ -40,6 +40,7 @@ NetworkAddressFamily inet_netfamily(int family);
>  int unix_listen(const char *path, Error **errp);
>  int unix_connect(const char *path, Error **errp);
>  
> +char *socket_uri(SocketAddress *addr);
>  SocketAddress *socket_parse(const char *str, Error **errp);
>  int socket_connect(SocketAddress *addr, Error **errp);
>  int socket_listen(SocketAddress *addr, int num, Error **errp);
> @@ -123,5 +124,4 @@ SocketAddress *socket_address_flatten(SocketAddressLegacy 
> *addr);
>   * Return 0 on success.
>   */
>  int socket_address_parse_named_fd(SocketAddress *addr, Error **errp);
> -
>  #endif /* QEMU_SOCKETS_H */
> diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
> index c6cd6f91dde6..cb35059c2d45 100644
> --- a/monitor/hmp-cmds.c
> +++ b/monitor/hmp-cmds.c
> @@ -197,27 +197,6 @@ void hmp_info_mice(Monitor *mon, const QDict *qdict)
>  qapi_free_MouseInfoList(mice_list);
>  }
>  
> -static char *SocketAddress_to_str(SocketAddress *addr)
> -{
> -switch (addr->type) {
> -case SOCKET_ADDRESS_TYPE_INET:
> -return g_strdup_printf("tcp:%s:%s",
> -   addr->u.inet.host,
> -   addr->u.inet.port);
> -case SOCKET_ADDRESS_TYPE_UNIX:
> -return g_strdup_printf("unix:%s",
> -   addr->u.q_unix.path);
> -case SOCKET_ADDRESS_TYPE_FD:
> -return g_strdup_printf("fd:%s", addr->u.fd.str);
> -case SOCKET_ADDRESS_TYPE_VSOCK:
> -return g_strdup_printf("tcp:%s:%s",
> -   addr->u.vsock.cid,
> -   addr->u.vsock.port);
> -default:
> -return g_strdup("unknown address type");
> -}
> -}
> -
>  void hmp_info_migrate(Monitor *mon, const QDict *qdict)
>  {
>  MigrationInfo *info;
> @@ -380,7 +359,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
>  monitor_printf(mon, "socket address: [\n");
>  
>  for (addr = info->socket_address; addr; addr = addr->next) {
> -char *s = SocketAddress_to_str(addr->value);
> +char *s = socket_uri(addr->value);
>  monitor_printf(mon, "\t%s\n", s);
>  g_free(s);
>  }
> diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
> index 83f4bd6fd211..9f6f655fd526 100644
> --- a/util/qemu-sockets.c
> +++ b/util/qemu-sockets.c
> @@ -1077,6 +1077,26 @@ int unix_connect(const char *path, Error **errp)
>  return sock;
>  }
>  
> +char *socket_uri(SocketAddress *addr)
> +{
> +switch (addr->type) {
> +case SOCKET_ADDRESS_TYPE_INET:
> +return g_strdup_printf("tcp:%s:%s",
> +   addr->u.inet.host,
> +   addr->u.inet.port);
> +case SOCKET_ADDRESS_TYPE_UNIX:
> +return g_strdup_printf("unix:%s",
> +   addr->u.q_unix.path);
> +case SOCKET_ADDRESS_TYPE_FD:
> +return g_strdup_printf("fd:%s", addr->u.fd.str);
> +case SOCKET_ADDRESS_TYPE_VSOCK:
> +return g_strdup_printf("tcp:%s:%s",
> +   addr->u.vsock.cid,
> +   addr->u.vsock.port);
> +default:
> +return g_strdup("unknown address type");
> +}
> +}
>  
>  SocketAddress *socket_parse(const char *str, Error **errp)
>  {
> -- 
> 2.37.3
> 
-- 
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK




Re: [PATCH v8 12/14] qemu-sockets: update socket_uri() and socket_parse() to be consistent

2022-09-14 Thread Dr. David Alan Gilbert
* Laurent Vivier (lviv...@redhat.com) wrote:
> To be consistent with socket_uri(), add 'tcp:' prefix for inet type in
> socket_parse(), by default socket_parse() use tcp when no prefix is
> provided (format is host:port).
> 
> In socket_uri(), use 'vsock:' prefix for vsock type rather than 'tcp:'
> because it makes a vsock address look like an inet address with CID
> misinterpreted as host.
> Goes back to commit 9aca82ba31 "migration: Create socket-address parameter"
> 
> Signed-off-by: Laurent Vivier 

Yeh I think that's safer than the last version for migrate URIs

Reviewed-by: Dr. David Alan Gilbert 

> ---
>  util/qemu-sockets.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
> index 9f6f655fd526..a9926af714c4 100644
> --- a/util/qemu-sockets.c
> +++ b/util/qemu-sockets.c
> @@ -1090,7 +1090,7 @@ char *socket_uri(SocketAddress *addr)
>  case SOCKET_ADDRESS_TYPE_FD:
>  return g_strdup_printf("fd:%s", addr->u.fd.str);
>  case SOCKET_ADDRESS_TYPE_VSOCK:
> -return g_strdup_printf("tcp:%s:%s",
> +return g_strdup_printf("vsock:%s:%s",
> addr->u.vsock.cid,
> addr->u.vsock.port);
>  default:
> @@ -1124,6 +1124,11 @@ SocketAddress *socket_parse(const char *str, Error 
> **errp)
>  if (vsock_parse(>u.vsock, str + strlen("vsock:"), errp)) {
>  goto fail;
>  }
> +} else if (strstart(str, "tcp:", NULL)) {
> +addr->type = SOCKET_ADDRESS_TYPE_INET;
> +if (inet_parse(>u.inet, str + strlen("tcp:"), errp)) {
> +goto fail;
> +}
>  } else {
>  addr->type = SOCKET_ADDRESS_TYPE_INET;
>  if (inet_parse(>u.inet, str, errp)) {
> -- 
> 2.37.3
> 
-- 
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK




[PATCH v2] tcg/ppc: Optimize 26-bit jumps

2022-09-14 Thread Leandro Lupori
PowerPC64 processors handle direct branches better than indirect
ones, resulting in less stalled cycles and branch misses.

However, PPC's tb_target_set_jmp_target() was only using direct
branches for 16-bit jumps, while PowerPC64's unconditional branch
instructions are able to handle displacements of up to 26 bits.
To take advantage of this, now jumps whose displacements fit in
between 17 and 26 bits are also converted to direct branches.

Signed-off-by: Leandro Lupori 
---
v2: use stq to replace all instructions atomically

 tcg/ppc/tcg-target.c.inc | 84 +++-
 1 file changed, 66 insertions(+), 18 deletions(-)

diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc
index 1cbd047ab3..484014cb1a 100644
--- a/tcg/ppc/tcg-target.c.inc
+++ b/tcg/ppc/tcg-target.c.inc
@@ -1847,38 +1847,86 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0)
 tcg_out32(s, insn);
 }
 
+#if TCG_TARGET_REG_BITS == 64
+
+static inline uint64_t make_pair(tcg_insn_unit i1, tcg_insn_unit i2)
+{
+#if HOST_BIG_ENDIAN
+return (uint64_t)i1 << 32 | i2;
+#else
+return (uint64_t)i2 << 32 | i1;
+#endif
+}
+
+static inline void ppc64_stq(uintptr_t dst, uint64_t src0, uint64_t src1)
+{
+asm volatile (
+"mr  %%r6, %0\n\t"
+"mr  %%r7, %1\n\t"
+"stq %%r6, 0(%2)"
+: : "r" (src0), "r" (src1), "r" (dst) : "r6", "r7", "memory");
+}
+
+#endif
+
+static inline void ppc64_replace_insns(uintptr_t rx, uintptr_t rw,
+tcg_insn_unit *insn)
+{
+#if TCG_TARGET_REG_BITS == 64
+uint64_t pair[2];
+
+pair[0] = make_pair(insn[0], insn[1]);
+if (have_isa_2_07) {
+pair[1] = make_pair(insn[2], insn[3]);
+#if HOST_BIG_ENDIAN
+ppc64_stq(rw, pair[0], pair[1]);
+#else
+ppc64_stq(rw, pair[1], pair[0]);
+#endif
+flush_idcache_range(rx, rw, 16);
+} else {
+qatomic_set((uint64_t *)rw, pair[0]);
+flush_idcache_range(rx, rw, 8);
+}
+#endif
+}
+
 void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx,
   uintptr_t jmp_rw, uintptr_t addr)
 {
 if (TCG_TARGET_REG_BITS == 64) {
-tcg_insn_unit i1, i2;
+tcg_insn_unit i[4];
 intptr_t tb_diff = addr - tc_ptr;
 intptr_t br_diff = addr - (jmp_rx + 4);
-uint64_t pair;
 
-/* This does not exercise the range of the branch, but we do
-   still need to be able to load the new value of TCG_REG_TB.
-   But this does still happen quite often.  */
+/*
+ * This does not exercise the range of the branch, but we do
+ * still need to be able to load the new value of TCG_REG_TB.
+ * But this does still happen quite often.
+ */
 if (tb_diff == (int16_t)tb_diff) {
-i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
-i2 = B | (br_diff & 0x3fc);
+i[0] = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff);
+i[1] = B | (br_diff & 0x3fc);
+i[2] = MTSPR | RS(TCG_REG_TB) | CTR;
+
 } else {
 intptr_t lo = (int16_t)tb_diff;
 intptr_t hi = (int32_t)(tb_diff - lo);
 assert(tb_diff == hi + lo);
-i1 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
-i2 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
+i[0] = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16);
+i[1] = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo);
+
+br_diff -= 4;
+if (have_isa_2_07 && in_range_b(br_diff)) {
+i[2] = B | (br_diff & 0x3fc);
+} else {
+i[2] = MTSPR | RS(TCG_REG_TB) | CTR;
+}
 }
-#if HOST_BIG_ENDIAN
-pair = (uint64_t)i1 << 32 | i2;
-#else
-pair = (uint64_t)i2 << 32 | i1;
-#endif
 
-/* As per the enclosing if, this is ppc64.  Avoid the _Static_assert
-   within qatomic_set that would fail to build a ppc32 host.  */
-qatomic_set__nocheck((uint64_t *)jmp_rw, pair);
-flush_idcache_range(jmp_rx, jmp_rw, 8);
+i[3] = BCCTR | BO_ALWAYS;
+ppc64_replace_insns(jmp_rx, jmp_rw, i);
+
 } else {
 intptr_t diff = addr - jmp_rx;
 tcg_debug_assert(in_range_b(diff));
-- 
2.25.1




Re: [PATCH v3 03/20] ppc4xx_sdram: Get rid of the init RAM hack

2022-09-14 Thread BALATON Zoltan

On Wed, 14 Sep 2022, BALATON Zoltan wrote:

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/14/22 13:44, BALATON Zoltan wrote:

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/13/22 21:52, BALATON Zoltan wrote:

The do_init parameter of ppc4xx_sdram_init() is used to map memory
regions that is normally done by the firmware by programming the SDRAM
controller. This is needed when booting a kernel directly from -kernel
without a firmware. Do this from board code accesing normal SDRAM


accessing


Fixed, also two ofhers in another patch you haven't noticed.


controller registers the same way as firmware would do, so we can get
rid of this hack.

Signed-off-by: BALATON Zoltan 
---
v2: Fix ref405ep boot with -kernel and U-Boot

  hw/ppc/ppc405.h |  1 -
  hw/ppc/ppc405_boards.c  | 12 ++--
  hw/ppc/ppc405_uc.c  |  4 +---
  hw/ppc/ppc440_bamboo.c  |  8 +++-
  hw/ppc/ppc440_uc.c  |  2 --
  hw/ppc/ppc4xx_devs.c    | 11 +--
  include/hw/ppc/ppc4xx.h |  8 ++--
  7 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/hw/ppc/ppc405.h b/hw/ppc/ppc405.h
index 1e558c7831..756865621b 100644
--- a/hw/ppc/ppc405.h
+++ b/hw/ppc/ppc405.h
@@ -169,7 +169,6 @@ struct Ppc405SoCState {
  /* Public */
  MemoryRegion ram_banks[2];
  hwaddr ram_bases[2], ram_sizes[2];
-    bool do_dram_init;
    MemoryRegion *dram_mr;
  hwaddr ram_size;
diff --git a/hw/ppc/ppc405_boards.c b/hw/ppc/ppc405_boards.c
index 083f12b23e..bf02a71c6d 100644
--- a/hw/ppc/ppc405_boards.c
+++ b/hw/ppc/ppc405_boards.c
@@ -274,6 +274,7 @@ static void ppc405_init(MachineState *machine)
  MachineClass *mc = MACHINE_GET_CLASS(machine);
  const char *kernel_filename = machine->kernel_filename;
  MemoryRegion *sysmem = get_system_memory();
+    CPUPPCState *env;
    if (machine->ram_size != mc->default_ram_size) {
  char *sz = size_to_str(mc->default_ram_size);
@@ -288,12 +289,19 @@ static void ppc405_init(MachineState *machine)
   machine->ram_size, _fatal);
  object_property_set_link(OBJECT(>soc), "dram",
   OBJECT(machine->ram), _abort);
-    object_property_set_bool(OBJECT(>soc), "dram-init",
- kernel_filename != NULL, _abort);
  object_property_set_uint(OBJECT(>soc), "sys-clk", 
,

   _abort);
  qdev_realize(DEVICE(>soc), NULL, _fatal);
  +    /* Enable SDRAM memory regions */
+    /* FIXME This shouldn't be needed with firmware but we lack SPD 
data */


what do you mean ?


U-Boot detects the available RAM by reading the SPD info of the RAM 
modules but that probably also needs i2c emulation. See sam460ex.



+    env = >soc.cpu.env;
+    if (ppc_dcr_write(env->dcr_env, SDRAM0_CFGADDR, 0x20) ||
+    ppc_dcr_write(env->dcr_env, SDRAM0_CFGDATA, 0x8000)) {



I am not in favor of these ppc_drc_write calls and this is still a hack.


It's not. Normally this is done by firmware to enable memory controller 
but the board code has to do it if not using firmware (e.g. booting with 
-kernel) the same way it provides bootinfo or device tree mods the 
firmware would normally do or in this case maybe the emulation is 
incomplete so the part of firmware that configures the SDRAM controller 
does not run.


Exactly, and what the above proposal does is mimicking execution of CPU
instructions before the CPU is even fully initiated. Reset has not been
called at that stage.


I don't get this. We're not calling any CPU instructions, ppc_dcr_write just 
calls the write callback the device has registered for the dcr so it just 
does the same as the hack did at the end just doing it the same way the 
firmware should do.


The "dram-init" property is a cleaner solution. It takes care of doing 
the

pre-mapping of RAM banks in the realize routine of the sdram model (when
available).


I disagree, the hardware does not have such feature, it proviesd DCRs as 
the way to configure it. Adding a special property for it deviates from 
hardware and clutters qtree. 



In this machine, running QEMU with -kernel deviates from HW. That's


In all machines booting with -kernel likely deviates and all machines 
probably have additinal code in this case to do some things normally done by 
the firmware. Look at pegasos2_machine_reset() for example. All that is not 
needed when we boot with firmware as then the firmware will do all that and 
provide the device tree, etc. bur we need to do these when booting without 
firmware. In thes case QEMU also emulates the firmware and has to do thinigs 
like enabling the memory controller.



the whole purpose of this option. It assumes that the SDRAM device
is pre-initialized (and much more should be done) by the QEMU machine
and the simplest way to acheive this goal is to inform the SDRAM model
to take care of the pre-initialization.


In my opinion the SDRAM controller model should model the hardware and if the 
board 

Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Alex Williamson
On Wed, 14 Sep 2022 12:10:29 -0600
Alex Williamson  wrote:

> On Mon, 12 Sep 2022 13:51:30 -0700
> Nicolin Chen  wrote:
> 
> > On Mon, Sep 12, 2022 at 02:38:52PM +0200, Cornelia Huck wrote:  
> > > External email: Use caution opening links or attachments
> > > 
> > > 
> > > On Fri, Sep 09 2022, Nicolin Chen  wrote:
> > > 
> > > > Its caller vfio_connect_container() assigns a default value
> > > > to info->iova_pgsizes, even if vfio_get_iommu_info() fails.
> > > > This would result in a "Segmentation fault" error, when the
> > > > VFIO_IOMMU_GET_INFO ioctl errors out.
> > > >
> > > > Since the caller has g_free already, drop the g_free in its
> > > > rollback routine and add a line of comments to highlight it.
> > > 
> > > There's basically two ways to fix this:
> > > 
> > > - return *info in any case, even on error
> > > - free *info on error, and make sure that the caller doesn't try to
> > >   access *info if the function returned !0
> > > 
> > > The problem with the first option is that the caller will access invalid
> > > information if it neglects to check the return code, and that might lead
> > > to not-that-obvious errors; in the second case, a broken caller would at
> > > least fail quickly with a segfault. The current code is easier to fix
> > > with the first option.
> > > 
> > > I think I'd prefer the second option; but obviously maintainer's choice.  
> > >   
> > 
> > The caller does check rc all the time. So I made a smaller fix
> > (the first option). Attaching the git-diff for the second one.
> > 
> > Alex, please let me know which one you prefer. Thanks!
> > 
> > diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> > index 51b2e05c76..74431411ab 100644
> > --- a/hw/vfio/common.c
> > +++ b/hw/vfio/common.c  
> [snip]
> 
> I think we can do better than that, I don't think we need to maintain
> the existing grouping, and that FIXME comment is outdated and has
> drifted from the relevant line of code.  What about:
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index ace9562a9ba1..8d8c54d59083 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -2111,29 +2111,31 @@ static int vfio_connect_container(VFIOGroup *group, 
> AddressSpace *as,
>  {
>  struct vfio_iommu_type1_info *info;
>  
> -/*
> - * FIXME: This assumes that a Type1 IOMMU can map any 64-bit
> - * IOVA whatsoever.  That's not actually true, but the current
> - * kernel interface doesn't tell us what it can map, and the
> - * existing Type1 IOMMUs generally support any IOVA we're
> - * going to actually try in practice.
> + /*
> + * Setup defaults for container pgsizes and dma_max_mappings if not
> + * provided by kernel below.
>   */
> -ret = vfio_get_iommu_info(container, );
> -
> -if (ret || !(info->flags & VFIO_IOMMU_INFO_PGSIZES)) {
> -/* Assume 4k IOVA page size */
> -info->iova_pgsizes = 4096;
> -}
> -vfio_host_win_add(container, 0, (hwaddr)-1, info->iova_pgsizes);
> -container->pgsizes = info->iova_pgsizes;
> -
> -/* The default in the kernel ("dma_entry_limit") is 65535. */
> +container->pgsizes = 4096;
>  container->dma_max_mappings = 65535;
> +
> +ret = vfio_get_iommu_info(container, );
>  if (!ret) {

Or better still:

if (!vfio_get_iommu_info(container, )) {

> +if (info->flags & VFIO_IOMMU_INFO_PGSIZES) {
> +container->pgsizes = info->iova_pgsizes;
> +}
> +
>  vfio_get_info_dma_avail(info, >dma_max_mappings);
>  vfio_get_iommu_info_migration(container, info);
> +g_free(info);
>  }
> -g_free(info);
> +
> +/*
> + * FIXME: We should parse VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE
> + * information to get the actual window extent rather than assume
> + * a 64-bit IOVA address space.
> + */
> +vfio_host_win_add(container, 0, (hwaddr)-1, container->pgsizes);
> +
>  break;
>  }
>  case VFIO_SPAPR_TCE_v2_IOMMU:
> 




Re: [PATCH v2 4/4] scripts: add script to compare compatible properties

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 9/14/22 13:55, Maksim Davydov wrote:

This script run QEMU to obtain compat_props of machines and default
values of different types and produce appropriate table. This table
can be used to compare machine types to choose the most suitable
machine. Also table in json or csv format should be used to check that
new machine doesn't affect previous ones via comparisin tables with and
without new machine.
Default values of properties are needed to fill "holes" in the table (one
machine has these properties and another not).

Notes:
* some init values from the devices can't be available like properties
from virtio-9p when configure has --disable-virtfs. This situations will
be seen in the table as "unavailable driver".
* Default values can be get can be obtained in an unobvious way, like
x86 features. If the script doesn't know how to get property default value
to compare one machine with another it fills "holes" with "unavailable
method". This is done because script uses whitelist model to get default
values of different types. It means that the method that can't be applied
to a new type that can crash this script. It is better to get an
"unavailable driver" when creating a new machine with new compatible
properties than to break this script. So it turns out a more stable and
generic script.
* If the default value can't be obtained because this property doesn't
exist or because this property can't have default value, appropriate
"hole" will be filled by "unknown property" or "no default value"
* If the property is applied to the abstract class, the script collects
default values from all child classes (set of default values)

Example:

./scripts/compare_mt.py --mt pc-q35-3.1 pc-q35-2.12

╒╤═══╤═══╕
││  pc-q35-2.12  │  pc-q35-3.1   │
╞╪═══╪═══╡
│EPYC-IBPB-x86_64-cpu-xlevel │  0x800a   │  2147483678   │
├┼───┼───┤
│   EPYC-x86_64-cpu-xlevel   │  0x800a   │  2147483678   │
├┼───┼───┤
│ Skylake-Server-IBRS-x86_64-cpu-pku │ False │ True  │
├┼───┼───┤
│   Skylake-Server-x86_64-cpu-pku│ False │ True  │
├┼───┼───┤
│ VGA-global-vmstate │ True  │ False │
├┼───┼───┤
│ cirrus-vga-global-vmstate  │ True  │ False │
├┼───┼───┤
│hda-audio-use-timer │ False │ True  │
├┼───┼───┤
│  migration-decompress-error-check  │ False │ True  │
├┼───┼───┤
│   qxl-vga-global-vmstate   │ True  │ False │
├┼───┼───┤
│ vmware-svga-global-vmstate │ True  │ False │
├┼───┼───┤
│  x86_64-cpu-legacy-cache   │ True  │ [True, False] │
├┼───┼───┤
│ x86_64-cpu-topoext │ False │ [True, False] │
├┼───┼───┤
│   x86_64-cpu-x-hv-synic-kvm-only   │ True  │ False │
╘╧═══╧═══╛


Looks great!

Some nitpicking:

1. I'd use left-align, at least for the first column as more readable.

2. Seems '-' is bad separator for driver-property, as both drivers and 
properties sometimes has own '-' inside. Let's use something else. Maybe colon, 
like driver:property. Or, maybe, just add one more column, to keep driver and 
property names in different columns.

Hmm, and what means [True, False] ?



Signed-off-by: Maksim Davydov 
---
  scripts/compare_mt.py | 374 ++
  1 file changed, 374 insertions(+)
  create mode 100755 scripts/compare_mt.py

diff --git a/scripts/compare_mt.py b/scripts/compare_mt.py
new file mode 100755
index 00..1df5e6c4f4
--- /dev/null
+++ b/scripts/compare_mt.py
@@ -0,0 +1,374 @@
+#!/usr/bin/env python3
+#
+# Script to compare machine type compatible properties (include/hw/boards.h).
+# compat_props are applied to the driver during initialization to change
+# default values, for instance, to maintain compatibility.
+# This script constructs table with machines and values of their compat_props
+# to compare and to find places for improvements or places with bugs. If
+# during the comparision, some machine type doesn't have a 

Re: [PATCH v3 03/20] ppc4xx_sdram: Get rid of the init RAM hack

2022-09-14 Thread BALATON Zoltan

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/14/22 13:44, BALATON Zoltan wrote:

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/13/22 21:52, BALATON Zoltan wrote:

The do_init parameter of ppc4xx_sdram_init() is used to map memory
regions that is normally done by the firmware by programming the SDRAM
controller. This is needed when booting a kernel directly from -kernel
without a firmware. Do this from board code accesing normal SDRAM


accessing


Fixed, also two ofhers in another patch you haven't noticed.


controller registers the same way as firmware would do, so we can get
rid of this hack.

Signed-off-by: BALATON Zoltan 
---
v2: Fix ref405ep boot with -kernel and U-Boot

  hw/ppc/ppc405.h |  1 -
  hw/ppc/ppc405_boards.c  | 12 ++--
  hw/ppc/ppc405_uc.c  |  4 +---
  hw/ppc/ppc440_bamboo.c  |  8 +++-
  hw/ppc/ppc440_uc.c  |  2 --
  hw/ppc/ppc4xx_devs.c    | 11 +--
  include/hw/ppc/ppc4xx.h |  8 ++--
  7 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/hw/ppc/ppc405.h b/hw/ppc/ppc405.h
index 1e558c7831..756865621b 100644
--- a/hw/ppc/ppc405.h
+++ b/hw/ppc/ppc405.h
@@ -169,7 +169,6 @@ struct Ppc405SoCState {
  /* Public */
  MemoryRegion ram_banks[2];
  hwaddr ram_bases[2], ram_sizes[2];
-    bool do_dram_init;
    MemoryRegion *dram_mr;
  hwaddr ram_size;
diff --git a/hw/ppc/ppc405_boards.c b/hw/ppc/ppc405_boards.c
index 083f12b23e..bf02a71c6d 100644
--- a/hw/ppc/ppc405_boards.c
+++ b/hw/ppc/ppc405_boards.c
@@ -274,6 +274,7 @@ static void ppc405_init(MachineState *machine)
  MachineClass *mc = MACHINE_GET_CLASS(machine);
  const char *kernel_filename = machine->kernel_filename;
  MemoryRegion *sysmem = get_system_memory();
+    CPUPPCState *env;
    if (machine->ram_size != mc->default_ram_size) {
  char *sz = size_to_str(mc->default_ram_size);
@@ -288,12 +289,19 @@ static void ppc405_init(MachineState *machine)
   machine->ram_size, _fatal);
  object_property_set_link(OBJECT(>soc), "dram",
   OBJECT(machine->ram), _abort);
-    object_property_set_bool(OBJECT(>soc), "dram-init",
- kernel_filename != NULL, _abort);
  object_property_set_uint(OBJECT(>soc), "sys-clk", ,
   _abort);
  qdev_realize(DEVICE(>soc), NULL, _fatal);
  +    /* Enable SDRAM memory regions */
+    /* FIXME This shouldn't be needed with firmware but we lack SPD data 
*/


what do you mean ?


U-Boot detects the available RAM by reading the SPD info of the RAM modules 
but that probably also needs i2c emulation. See sam460ex.



+    env = >soc.cpu.env;
+    if (ppc_dcr_write(env->dcr_env, SDRAM0_CFGADDR, 0x20) ||
+    ppc_dcr_write(env->dcr_env, SDRAM0_CFGDATA, 0x8000)) {



I am not in favor of these ppc_drc_write calls and this is still a hack.


It's not. Normally this is done by firmware to enable memory controller but 
the board code has to do it if not using firmware (e.g. booting with 
-kernel) the same way it provides bootinfo or device tree mods the firmware 
would normally do or in this case maybe the emulation is incomplete so the 
part of firmware that configures the SDRAM controller does not run.


Exactly, and what the above proposal does is mimicking execution of CPU
instructions before the CPU is even fully initiated. Reset has not been
called at that stage.


I don't get this. We're not calling any CPU instructions, ppc_dcr_write 
just calls the write callback the device has registered for the dcr so it 
just does the same as the hack did at the end just doing it the same way 
the firmware should do.



The "dram-init" property is a cleaner solution. It takes care of doing the
pre-mapping of RAM banks in the realize routine of the sdram model (when
available).


I disagree, the hardware does not have such feature, it proviesd DCRs as 
the way to configure it. Adding a special property for it deviates from 
hardware and clutters qtree. 



In this machine, running QEMU with -kernel deviates from HW. That's


In all machines booting with -kernel likely deviates and all machines 
probably have additinal code in this case to do some things normally done 
by the firmware. Look at pegasos2_machine_reset() for example. All that is 
not needed when we boot with firmware as then the firmware will do all 
that and provide the device tree, etc. bur we need to do these when 
booting without firmware. In thes case QEMU also emulates the firmware 
and has to do thinigs like enabling the memory controller.



the whole purpose of this option. It assumes that the SDRAM device
is pre-initialized (and much more should be done) by the QEMU machine
and the simplest way to acheive this goal is to inform the SDRAM model
to take care of the pre-initialization.


In my opinion the SDRAM controller model should model the hardware and if 
the board uses it differently then it should take care of 

Re: [PATCH] vfio/common: Do not g_free in vfio_get_iommu_info

2022-09-14 Thread Alex Williamson
On Mon, 12 Sep 2022 13:51:30 -0700
Nicolin Chen  wrote:

> On Mon, Sep 12, 2022 at 02:38:52PM +0200, Cornelia Huck wrote:
> > External email: Use caution opening links or attachments
> > 
> > 
> > On Fri, Sep 09 2022, Nicolin Chen  wrote:
> >   
> > > Its caller vfio_connect_container() assigns a default value
> > > to info->iova_pgsizes, even if vfio_get_iommu_info() fails.
> > > This would result in a "Segmentation fault" error, when the
> > > VFIO_IOMMU_GET_INFO ioctl errors out.
> > >
> > > Since the caller has g_free already, drop the g_free in its
> > > rollback routine and add a line of comments to highlight it.  
> > 
> > There's basically two ways to fix this:
> > 
> > - return *info in any case, even on error
> > - free *info on error, and make sure that the caller doesn't try to
> >   access *info if the function returned !0
> > 
> > The problem with the first option is that the caller will access invalid
> > information if it neglects to check the return code, and that might lead
> > to not-that-obvious errors; in the second case, a broken caller would at
> > least fail quickly with a segfault. The current code is easier to fix
> > with the first option.
> > 
> > I think I'd prefer the second option; but obviously maintainer's choice.  
> 
> The caller does check rc all the time. So I made a smaller fix
> (the first option). Attaching the git-diff for the second one.
> 
> Alex, please let me know which one you prefer. Thanks!
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 51b2e05c76..74431411ab 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
[snip]

I think we can do better than that, I don't think we need to maintain
the existing grouping, and that FIXME comment is outdated and has
drifted from the relevant line of code.  What about:

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index ace9562a9ba1..8d8c54d59083 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -2111,29 +2111,31 @@ static int vfio_connect_container(VFIOGroup *group, 
AddressSpace *as,
 {
 struct vfio_iommu_type1_info *info;
 
-/*
- * FIXME: This assumes that a Type1 IOMMU can map any 64-bit
- * IOVA whatsoever.  That's not actually true, but the current
- * kernel interface doesn't tell us what it can map, and the
- * existing Type1 IOMMUs generally support any IOVA we're
- * going to actually try in practice.
+   /*
+ * Setup defaults for container pgsizes and dma_max_mappings if not
+ * provided by kernel below.
  */
-ret = vfio_get_iommu_info(container, );
-
-if (ret || !(info->flags & VFIO_IOMMU_INFO_PGSIZES)) {
-/* Assume 4k IOVA page size */
-info->iova_pgsizes = 4096;
-}
-vfio_host_win_add(container, 0, (hwaddr)-1, info->iova_pgsizes);
-container->pgsizes = info->iova_pgsizes;
-
-/* The default in the kernel ("dma_entry_limit") is 65535. */
+container->pgsizes = 4096;
 container->dma_max_mappings = 65535;
+
+ret = vfio_get_iommu_info(container, );
 if (!ret) {
+if (info->flags & VFIO_IOMMU_INFO_PGSIZES) {
+container->pgsizes = info->iova_pgsizes;
+}
+
 vfio_get_info_dma_avail(info, >dma_max_mappings);
 vfio_get_iommu_info_migration(container, info);
+g_free(info);
 }
-g_free(info);
+
+/*
+ * FIXME: We should parse VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE
+ * information to get the actual window extent rather than assume
+ * a 64-bit IOVA address space.
+ */
+vfio_host_win_add(container, 0, (hwaddr)-1, container->pgsizes);
+
 break;
 }
 case VFIO_SPAPR_TCE_v2_IOMMU:




Re: [PATCH 0/2] target/m68k: fix two writes to %sr

2022-09-14 Thread Howard Spoelstra
On Tue, Sep 13, 2022 at 6:29 PM Mark Cave-Ayland <
mark.cave-ayl...@ilande.co.uk> wrote:

> On 13/09/2022 15:28, Richard Henderson wrote:
>
> > The second was described by Mark in the lobby of KVM Forum.
> > The first was found by inspection of other uses of gen_helper_set_sr.
> >
> > r~
> >
> > Richard Henderson (2):
> >target/m68k: Fix MACSR to CCR
> >target/m68k: Perform writback before modifying SR
> >
> >   target/m68k/translate.c | 14 +-
> >   1 file changed, 9 insertions(+), 5 deletions(-)
>
> I've applied these on top of my MacOS virtual memory branch at
> https://github.com/mcayland/qemu/commits/q800.upstream2-vm and I can
> confirm that
> MacOS 8.1 now boots here with virtual memory enabled :)
>
> Possibly it might be worth including a tidied-up version of the "WIP:
> target/m68k:
> always exit_tb when changing sr with andi/ori/eori" commit from that
> branch which is
> also related to switching between supervisor and user modes under MacOS.
> Shall I tidy
> it up and send it to the list?
>
>
> ATB,
>
> Mark.
>
>
I've compiled the branch mentioned above with a fully updated MSYS2 on
windows. The executable hangs when running Mac OS 8 with Virtual Memory
enabled. On a fast machine I see Error 7 as before, on a slower machine,
the boot screen just hangs with no error shown. A Linux build does work,
also on the slower machine.

Best,
Howard


Re: [PATCH 0/2] target/m68k: fix two writes to %sr

2022-09-14 Thread Howard Spoelstra
On Wed, Sep 14, 2022 at 6:48 PM Howard Spoelstra  wrote:

>
>
> On Tue, Sep 13, 2022 at 6:29 PM Mark Cave-Ayland <
> mark.cave-ayl...@ilande.co.uk> wrote:
>
>> On 13/09/2022 15:28, Richard Henderson wrote:
>>
>> > The second was described by Mark in the lobby of KVM Forum.
>> > The first was found by inspection of other uses of gen_helper_set_sr.
>> >
>> > r~
>> >
>> > Richard Henderson (2):
>> >target/m68k: Fix MACSR to CCR
>> >target/m68k: Perform writback before modifying SR
>> >
>> >   target/m68k/translate.c | 14 +-
>> >   1 file changed, 9 insertions(+), 5 deletions(-)
>>
>> I've applied these on top of my MacOS virtual memory branch at
>> https://github.com/mcayland/qemu/commits/q800.upstream2-vm and I can
>> confirm that
>> MacOS 8.1 now boots here with virtual memory enabled :)
>>
>> Possibly it might be worth including a tidied-up version of the "WIP:
>> target/m68k:
>> always exit_tb when changing sr with andi/ori/eori" commit from that
>> branch which is
>> also related to switching between supervisor and user modes under MacOS.
>> Shall I tidy
>> it up and send it to the list?
>>
>>
>> ATB,
>>
>> Mark.
>>
>>
> I've compiled the branch mentioned above with a fully updated MSYS2 on
> windows. The executable hangs when running Mac OS 8 with Virtual Memory
> enabled. On a fast machine I see Error 7 as before, on a slower machine,
> the boot screen just hangs with no error shown. A Linux build does work,
> also on the slower machine.
>
> Best,
> Howard
>

ps: a debug enabled non-stripped build will run with Virtual Memory enabled
on Windows.


Re: [PATCH 28/37] target/i386: reimplement 0x0f 0x38, add AVX

2022-09-14 Thread Paolo Bonzini
On Tue, Sep 13, 2022 at 11:31 AM Richard Henderson
 wrote:
> > +void glue(helper_vpmaskmovq_st, SUFFIX)(CPUX86State *env,
> > +Reg *v, Reg *s, target_ulong a0)
> > +{
> > +int i;
> > +
> > +for (i = 0; i < (1 << SHIFT); i++) {
> > +if (v->Q(i) >> 63) {
> > +cpu_stq_data_ra(env, a0 + i * 8, s->Q(i), GETPC());
> > +}
> > +}
> > +}
>
> Any idea if hw will write incomplete data if the pieces cross page 
> boundaries, and the
> second page is invalid?  We're not good at that for any other vector sized 
> write, though,
> so not critical.

No, I will check.

> > +void glue(helper_vpgatherdd, SUFFIX)(CPUX86State *env,
> > +Reg *d, Reg *v, Reg *s, target_ulong a0, unsigned scale)
> > +{
> > +int i;
> > +for (i = 0; i < (2 << SHIFT); i++) {
> > +if (v->L(i) >> 31) {
> > +target_ulong addr = a0
> > ++ ((target_ulong)(int32_t)s->L(i) << scale);
> > +d->L(i) = cpu_ldl_data_ra(env, addr, GETPC());
> > +}
> > +v->L(i) = 0;
> > +}
> > +}
>
> Better to not modify registers until all potential #GP are raised.

This is actually intentional: elements of v are zeroes whenever an
element is read successfully, so that values are not reread when the
instruction restarts. The manual says "If a fault is triggered by an
element and delivered, all elements closer to the LSB of the
destination zmm will be completed".

Paolo




Re: [PATCH] target/m68k: Implement atomic test-and-set

2022-09-14 Thread Laurent Vivier

Le 29/08/2022 à 07:17, Richard Henderson a écrit :

This is slightly more complicated than cas,
because tas is allowed on data registers.

Signed-off-by: Richard Henderson 
---
  target/m68k/translate.c | 40 ++--
  1 file changed, 30 insertions(+), 10 deletions(-)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 8f3c298ad0..0aef827b38 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -2825,19 +2825,39 @@ DISAS_INSN(illegal)
  gen_exception(s, s->base.pc_next, EXCP_ILLEGAL);
  }
  
-/* ??? This should be atomic.  */

  DISAS_INSN(tas)
  {
-TCGv dest;
-TCGv src1;
-TCGv addr;
+int mode = extract32(insn, 3, 3);
+int reg0 = REG(insn, 0);
  
-dest = tcg_temp_new();

-SRC_EA(env, src1, OS_BYTE, 1, );
-gen_logic_cc(s, src1, OS_BYTE);
-tcg_gen_ori_i32(dest, src1, 0x80);
-DEST_EA(env, insn, OS_BYTE, dest, );
-tcg_temp_free(dest);
+if (mode == 0) {
+/* data register direct */
+TCGv dest = cpu_dregs[reg0];
+gen_logic_cc(s, dest, OS_BYTE);
+tcg_gen_ori_tl(dest, dest, 0x80);
+} else {
+TCGv src1, addr;
+
+addr = gen_lea_mode(env, s, mode, reg0, OS_BYTE);
+if (IS_NULL_QREG(addr)) {
+gen_addr_fault(s);
+return;
+}
+src1 = tcg_temp_new();
+tcg_gen_atomic_fetch_or_tl(src1, addr, tcg_constant_tl(0x80),
+   IS_USER(s), MO_SB);
+gen_logic_cc(s, src1, OS_BYTE);
+tcg_temp_free(src1);
+
+switch (mode) {
+case 3: /* Indirect postincrement.  */
+tcg_gen_addi_i32(AREG(insn, 0), addr, 1);
+break;
+case 4: /* Indirect predecrememnt.  */
+tcg_gen_mov_i32(AREG(insn, 0), addr);
+break;
+}
+}
  }
  
  DISAS_INSN(mull)


Reviewed-by: Laurent Vivier 



[RFC PATCH 2/4] qtest: make read/write operation appear to be from CPU

2022-09-14 Thread Alex Bennée
The point of qtest is to simulate how running code might interact with
the system. However because it's not a real system we have places in
the code which especially handle check qtest_enabled() before
referencing current_cpu. Now we can encode these details in the
MemTxAttrs lets do that so we can start removing them.

Signed-off-by: Alex Bennée 
---
 softmmu/qtest.c | 31 +++
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/softmmu/qtest.c b/softmmu/qtest.c
index f8acef2628..c086bd34b7 100644
--- a/softmmu/qtest.c
+++ b/softmmu/qtest.c
@@ -362,6 +362,13 @@ static void qtest_clock_warp(int64_t dest)
 qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
 }
 
+/*
+ * QTest memory accesses are treated as though they come from the
+ * first (non-existent) CPU. We need to expose this via MemTxAttrs for
+ * those bits of HW which care which core is accessing them.
+ */
+#define MEMTXATTRS_QTEST ((MemTxAttrs) { .requester_cpu = 1 })
+
 static void qtest_process_command(CharBackend *chr, gchar **words)
 {
 const gchar *command;
@@ -525,17 +532,17 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 } else if (words[0][5] == 'w') {
 uint16_t data = value;
 tswap16s();
-address_space_write(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_write(first_cpu->as, addr, MEMTXATTRS_QTEST,
 , 2);
 } else if (words[0][5] == 'l') {
 uint32_t data = value;
 tswap32s();
-address_space_write(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_write(first_cpu->as, addr, MEMTXATTRS_QTEST,
 , 4);
 } else if (words[0][5] == 'q') {
 uint64_t data = value;
 tswap64s();
-address_space_write(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_write(first_cpu->as, addr, MEMTXATTRS_QTEST,
 , 8);
 }
 qtest_send_prefix(chr);
@@ -554,21 +561,21 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 
 if (words[0][4] == 'b') {
 uint8_t data;
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST,
, 1);
 value = data;
 } else if (words[0][4] == 'w') {
 uint16_t data;
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST,
, 2);
 value = tswap16(data);
 } else if (words[0][4] == 'l') {
 uint32_t data;
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST,
, 4);
 value = tswap32(data);
 } else if (words[0][4] == 'q') {
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST,
, 8);
 tswap64s();
 }
@@ -589,7 +596,7 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 g_assert(len);
 
 data = g_malloc(len);
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED, data,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST, data,
len);
 
 enc = g_malloc(2 * len + 1);
@@ -615,7 +622,7 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 g_assert(ret == 0);
 
 data = g_malloc(len);
-address_space_read(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED, data,
+address_space_read(first_cpu->as, addr, MEMTXATTRS_QTEST, data,
len);
 b64_data = g_base64_encode(data, len);
 qtest_send_prefix(chr);
@@ -650,7 +657,7 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 data[i] = 0;
 }
 }
-address_space_write(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED, data,
+address_space_write(first_cpu->as, addr, MEMTXATTRS_QTEST, data,
 len);
 g_free(data);
 
@@ -673,7 +680,7 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 if (len) {
 data = g_malloc(len);
 memset(data, pattern, len);
-address_space_write(first_cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+address_space_write(first_cpu->as, addr, MEMTXATTRS_QTEST,
 data, len);
 g_free(data);
 }
@@ -707,7 +714,7 @@ static void qtest_process_command(CharBackend *chr, gchar 
**words)
 out_len = MIN(out_len, len);
   

Re: [PATCH 23/37] target/i386: reimplement 0x0f 0x78-0x7f, add AVX

2022-09-14 Thread Paolo Bonzini
On Mon, Sep 12, 2022 at 3:56 PM Richard Henderson
 wrote:

> > +static void gen_SSE4a_I(DisasContext *s, CPUX86State *env, X86DecodedInsn 
> > *decode)
> > +{
> > +TCGv_i32 length = tcg_const_i32(decode->immediate & 255);
> > +TCGv_i32 index = tcg_const_i32(decode->immediate >> 8);
> > +
> > +if (s->prefix & PREFIX_DATA) {
> > +gen_helper_extrq_i(cpu_env, s->ptr0, index, length);
> > +} else {
> > +gen_helper_insertq_i(cpu_env, s->ptr0, index, length);
> > +}
> > +tcg_temp_free_i32(length);
> > +tcg_temp_free_i32(index);
>
> Again, why the choice of delayed decode?  I guess it doesn't matter, but it's 
> odd.

Mostly because I wasn't sure of which would be preferable so I tried
different things. I think now I have a better picture.

I will mostly switch to decode_by_prefix.

Paolo




[PATCH v2 23/30] tests/docker: update and flatten debian-all-test-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We also need to ensure we install clang as it is
used for those builds as well.

It would be nice to port this to lcitool but for now this will do.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-18-alex.ben...@linaro.org>

---
v2
  - move ccache/clang/git/ninja-build to main insall stanza
  - minor comment tweaks
---
 .gitlab-ci.d/container-cross.yml   |  1 -
 tests/docker/Makefile.include  |  1 -
 .../dockerfiles/debian-all-test-cross.docker   | 18 +-
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 3a8bd75473..091c0d8fcb 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -14,7 +14,6 @@ amd64-debian-cross-container:
 amd64-debian-user-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-all-test-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 24cd44e667..ddcc502049 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -129,7 +129,6 @@ docker-image-debian-nios2-cross: 
$(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(call debian-toolchain, $@)
 
 # Specialist build images, sometimes very limited tools
-docker-image-debian-all-test-cross: docker-image-debian10
 docker-image-debian-loongarch-cross: docker-image-debian11
 docker-image-debian-microblaze-cross: docker-image-debian10
 docker-image-debian-nios2-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker 
b/tests/docker/dockerfiles/debian-all-test-cross.docker
index dedcea58b4..2beb077fb4 100644
--- a/tests/docker/dockerfiles/debian-all-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
@@ -6,16 +6,24 @@
 # basic compilers for as many targets as possible. We shall use this
 # to build and run linux-user tests on GitLab
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-# What we need to build QEMU itself
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> 
/etc/apt/sources.list
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
 apt build-dep -yy qemu
 
-# Add the foreign architecture we want and install dependencies
+# Add extra build tools and as many cross compilers as we can for testing
 RUN DEBIAN_FRONTEND=noninteractive eatmydata \
 apt install -y --no-install-recommends \
+ccache \
+clang  \
+git \
+ninja-build \
 gcc-aarch64-linux-gnu \
 libc6-dev-arm64-cross \
 gcc-alpha-linux-gnu \
-- 
2.34.1




[RFC PATCH 0/4] use MemTxAttrs to signal cpu index

2022-09-14 Thread Alex Bennée
Hi,

This ostensibly fixes a bug with gdbstub when accessing the GIC state
(via system registers). It also proposes a pattern for removing
current_cpu/qtest_enabled() hacks in hw/ by passing the cpu index via
MemTxAttrs. For the ARM timer code we also assert that those accesses
do come from a CPU (as opposed to HW DMA attempting to overwrite
stuff).

We do need to audit helpers in target/ which use the address_space API
to read and write data and ensure they properly form their MxTxAttrs
so this works cleanly.

What do people think?

Alex Bennée (4):
  hw: encode accessing CPU index in MemTxAttrs
  qtest: make read/write operation appear to be from CPU
  hw/intc/gic: use MxTxAttrs to divine accessing CPU
  hw/timer: convert mptimer access to attrs to derive cpu index

 include/exec/memattrs.h |  4 +++-
 accel/tcg/cputlb.c  | 22 --
 hw/core/cpu-sysemu.c| 17 +
 hw/intc/arm_gic.c   | 39 ++-
 hw/timer/arm_mptimer.c  | 25 ++---
 softmmu/qtest.c | 31 +++
 6 files changed, 87 insertions(+), 51 deletions(-)

-- 
2.34.1




[PATCH v2 18/30] configure: explicitly set cflags for --disable-pie

2022-09-14 Thread Alex Bennée
This is working around current limitation of Meson's handling of
--disable-pie.

Signed-off-by: Alex Bennée 
Cc: Paolo Bonzini 
---
 configure | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/configure b/configure
index 575dde1c1f..6c169b23b5 100755
--- a/configure
+++ b/configure
@@ -1394,6 +1394,12 @@ else
   pie="no"
 fi
 
+# Meson currently only handles pie as a boolean for now so if we have
+# explicitly disabled PIE we need to extend our cflags because it wont.
+if test "$pie" = "no"; then
+QEMU_CFLAGS="-fno-pie -no-pie $QEMU_CFLAGS"
+fi
+
 # Detect support for PT_GNU_RELRO + DT_BIND_NOW.
 # The combination is known as "full relro", because .got.plt is read-only too.
 if compile_prog "" "-Wl,-z,relro -Wl,-z,now" ; then
-- 
2.34.1




[RFC PATCH 4/4] hw/timer: convert mptimer access to attrs to derive cpu index

2022-09-14 Thread Alex Bennée
This removes the hacks to deal with empty current_cpu.

Signed-off-by: Alex Bennée 
---
 hw/timer/arm_mptimer.c | 25 ++---
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/hw/timer/arm_mptimer.c b/hw/timer/arm_mptimer.c
index cdfca3000b..a7fe6ddc0d 100644
--- a/hw/timer/arm_mptimer.c
+++ b/hw/timer/arm_mptimer.c
@@ -41,9 +41,10 @@
  * which is used in both the ARM11MPCore and Cortex-A9MP.
  */
 
-static inline int get_current_cpu(ARMMPTimerState *s)
+static inline int get_current_cpu(ARMMPTimerState *s, MemTxAttrs attrs)
 {
-int cpu_id = current_cpu ? current_cpu->cpu_index : 0;
+int cpu_id = attrs.requester_id;
+g_assert(attrs.requester_cpu == 1);
 
 if (cpu_id >= s->num_cpu) {
 hw_error("arm_mptimer: num-cpu %d but this cpu is %d!\n",
@@ -178,25 +179,27 @@ static void timerblock_write(void *opaque, hwaddr addr,
 /* Wrapper functions to implement the "read timer/watchdog for
  * the current CPU" memory regions.
  */
-static uint64_t arm_thistimer_read(void *opaque, hwaddr addr,
-   unsigned size)
+static MemTxResult arm_thistimer_read(void *opaque, hwaddr addr, uint64_t 
*data,
+  unsigned size, MemTxAttrs attrs)
 {
 ARMMPTimerState *s = (ARMMPTimerState *)opaque;
-int id = get_current_cpu(s);
-return timerblock_read(>timerblock[id], addr, size);
+int id = get_current_cpu(s, attrs);
+*data = timerblock_read(>timerblock[id], addr, size);
+return MEMTX_OK;
 }
 
-static void arm_thistimer_write(void *opaque, hwaddr addr,
-uint64_t value, unsigned size)
+static MemTxResult arm_thistimer_write(void *opaque, hwaddr addr,
+uint64_t value, unsigned size, MemTxAttrs 
attrs)
 {
 ARMMPTimerState *s = (ARMMPTimerState *)opaque;
-int id = get_current_cpu(s);
+int id = get_current_cpu(s, attrs);
 timerblock_write(>timerblock[id], addr, value, size);
+return MEMTX_OK;
 }
 
 static const MemoryRegionOps arm_thistimer_ops = {
-.read = arm_thistimer_read,
-.write = arm_thistimer_write,
+.read_with_attrs = arm_thistimer_read,
+.write_with_attrs = arm_thistimer_write,
 .valid = {
 .min_access_size = 4,
 .max_access_size = 4,
-- 
2.34.1




[PATCH v2 22/30] tests/docker: flatten debian-riscv64-test-cross

2022-09-14 Thread Alex Bennée
Flatten into a single dockerfile and update to match the rest of the
test cross compile dockerfiles.

Signed-off-by: Alex Bennée 
Message-Id: <20220826172128.353798-17-alex.ben...@linaro.org>

---
v2
  - minor reword of commit msg
---
 .gitlab-ci.d/container-cross.yml   |  1 -
 tests/docker/Makefile.include  |  1 -
 .../dockerfiles/debian-riscv64-test-cross.docker   | 10 ++
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 95d57e1c5d..3a8bd75473 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -122,7 +122,6 @@ riscv64-debian-cross-container:
 riscv64-debian-test-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian11-container']
   variables:
 NAME: debian-riscv64-test-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index b1bf56434f..24cd44e667 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -133,7 +133,6 @@ docker-image-debian-all-test-cross: docker-image-debian10
 docker-image-debian-loongarch-cross: docker-image-debian11
 docker-image-debian-microblaze-cross: docker-image-debian10
 docker-image-debian-nios2-cross: docker-image-debian10
-docker-image-debian-riscv64-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
 DOCKER_PARTIAL_IMAGES += debian-alpha-cross
diff --git a/tests/docker/dockerfiles/debian-riscv64-test-cross.docker 
b/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
index 1d90901298..e5f83a5aeb 100644
--- a/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-test-cross.docker
@@ -3,10 +3,12 @@
 #
 # This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian11
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-riscv64-linux-gnu \
 libc6-dev-riscv64-cross
-- 
2.34.1




[RFC PATCH 1/4] hw: encode accessing CPU index in MemTxAttrs

2022-09-14 Thread Alex Bennée
We currently have hacks across the hw/ to reference current_cpu to
work out what the current accessing CPU is. This breaks in some cases
including using gdbstub to access HW state. As we have MemTxAttrs to
describe details about the access lets extend it to mention if this is
a CPU access and which one it is.

There are a number of places we need to fix up including:

  CPU helpers directly calling address_space_*() fns
  models in hw/ fishing the data out of current_cpu

I'll start addressing some of these in following patches.

Signed-off-by: Alex Bennée 
---
 include/exec/memattrs.h |  4 +++-
 accel/tcg/cputlb.c  | 22 --
 hw/core/cpu-sysemu.c| 17 +
 3 files changed, 32 insertions(+), 11 deletions(-)

diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h
index 9fb98bc1ef..3bccd5d291 100644
--- a/include/exec/memattrs.h
+++ b/include/exec/memattrs.h
@@ -43,7 +43,9 @@ typedef struct MemTxAttrs {
  * (see MEMTX_ACCESS_ERROR).
  */
 unsigned int memory:1;
-/* Requester ID (for MSI for example) */
+/* Requester is CPU (or as CPU, e.g. debug) */
+unsigned int requester_cpu:1;
+/* Requester ID (for MSI for example) or cpu_index */
 unsigned int requester_id:16;
 /* Invert endianness for this page */
 unsigned int byte_swap:1;
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 8fad2d9b83..68dc7dc646 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1340,8 +1340,13 @@ static uint64_t io_readx(CPUArchState *env, 
CPUIOTLBEntry *iotlbentry,
 uint64_t val;
 bool locked = false;
 MemTxResult r;
+MemTxAttrs attrs = iotlbentry->attrs;
 
-section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
+/* encode the accessing CPU */
+attrs.requester_cpu = 1;
+attrs.requester_id = cpu->cpu_index;
+
+section = iotlb_to_section(cpu, iotlbentry->addr, attrs);
 mr = section->mr;
 mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 cpu->mem_io_pc = retaddr;
@@ -1353,14 +1358,14 @@ static uint64_t io_readx(CPUArchState *env, 
CPUIOTLBEntry *iotlbentry,
 qemu_mutex_lock_iothread();
 locked = true;
 }
-r = memory_region_dispatch_read(mr, mr_offset, , op, 
iotlbentry->attrs);
+r = memory_region_dispatch_read(mr, mr_offset, , op, attrs);
 if (r != MEMTX_OK) {
 hwaddr physaddr = mr_offset +
 section->offset_within_address_space -
 section->offset_within_region;
 
 cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), 
access_type,
-   mmu_idx, iotlbentry->attrs, r, retaddr);
+   mmu_idx, attrs, r, retaddr);
 }
 if (locked) {
 qemu_mutex_unlock_iothread();
@@ -1395,8 +1400,13 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry 
*iotlbentry,
 MemoryRegion *mr;
 bool locked = false;
 MemTxResult r;
+MemTxAttrs attrs = iotlbentry->attrs;
+
+/* encode the accessing CPU */
+attrs.requester_cpu = 1;
+attrs.requester_id = cpu->cpu_index;
 
-section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs);
+section = iotlb_to_section(cpu, iotlbentry->addr, attrs);
 mr = section->mr;
 mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr;
 if (!cpu->can_do_io) {
@@ -1414,14 +1424,14 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry 
*iotlbentry,
 qemu_mutex_lock_iothread();
 locked = true;
 }
-r = memory_region_dispatch_write(mr, mr_offset, val, op, 
iotlbentry->attrs);
+r = memory_region_dispatch_write(mr, mr_offset, val, op, attrs);
 if (r != MEMTX_OK) {
 hwaddr physaddr = mr_offset +
 section->offset_within_address_space -
 section->offset_within_region;
 
 cpu_transaction_failed(cpu, physaddr, addr, memop_size(op),
-   MMU_DATA_STORE, mmu_idx, iotlbentry->attrs, r,
+   MMU_DATA_STORE, mmu_idx, attrs, r,
retaddr);
 }
 if (locked) {
diff --git a/hw/core/cpu-sysemu.c b/hw/core/cpu-sysemu.c
index 00253f8929..bd7ae983ed 100644
--- a/hw/core/cpu-sysemu.c
+++ b/hw/core/cpu-sysemu.c
@@ -51,13 +51,22 @@ hwaddr cpu_get_phys_page_attrs_debug(CPUState *cpu, vaddr 
addr,
  MemTxAttrs *attrs)
 {
 CPUClass *cc = CPU_GET_CLASS(cpu);
+MemTxAttrs local = { };
+hwaddr res;
 
 if (cc->sysemu_ops->get_phys_page_attrs_debug) {
-return cc->sysemu_ops->get_phys_page_attrs_debug(cpu, addr, attrs);
+res = cc->sysemu_ops->get_phys_page_attrs_debug(cpu, addr, );
+} else {
+/* Fallback for CPUs which don't implement the _attrs_ hook */
+local = MEMTXATTRS_UNSPECIFIED;
+res = cc->sysemu_ops->get_phys_page_debug(cpu, addr);
 }
-/* Fallback for CPUs which don't implement the _attrs_ hook */
-*attrs = 

[PATCH v2 11/30] tests/docker: update and flatten debian-mips64-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-7-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml|  1 -
 tests/docker/Makefile.include   |  1 -
 tests/docker/dockerfiles/debian-mips64-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 15a5270f6d..a3bfa483bf 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -77,7 +77,6 @@ m68k-debian-cross-container:
 mips64-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-mips64-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 95790e974e..6c2ee3b175 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -89,7 +89,6 @@ DOCKER_PARTIAL_IMAGES += fedora
 endif
 
 docker-image-debian-mips-cross: docker-image-debian10
-docker-image-debian-mips64-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
 
diff --git a/tests/docker/dockerfiles/debian-mips64-cross.docker 
b/tests/docker/dockerfiles/debian-mips64-cross.docker
index 09c2ba584e..afcff9726f 100644
--- a/tests/docker/dockerfiles/debian-mips64-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-mips64-linux-gnuabi64 \
 libc6-dev-mips64-cross
-- 
2.34.1




[RFC PATCH 3/4] hw/intc/gic: use MxTxAttrs to divine accessing CPU

2022-09-14 Thread Alex Bennée
Now that MxTxAttrs encodes a CPU we should use that to figure it out.
This solves edge cases like accessing via gdbstub or qtest.

Signed-off-by: Alex Bennée 
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/124
---
 hw/intc/arm_gic.c | 39 ++-
 1 file changed, 22 insertions(+), 17 deletions(-)

diff --git a/hw/intc/arm_gic.c b/hw/intc/arm_gic.c
index 492b2421ab..7feedac735 100644
--- a/hw/intc/arm_gic.c
+++ b/hw/intc/arm_gic.c
@@ -56,17 +56,22 @@ static const uint8_t gic_id_gicv2[] = {
 0x04, 0x00, 0x00, 0x00, 0x90, 0xb4, 0x2b, 0x00, 0x0d, 0xf0, 0x05, 0xb1
 };
 
-static inline int gic_get_current_cpu(GICState *s)
+static inline int gic_get_current_cpu(GICState *s, MemTxAttrs attrs)
 {
-if (!qtest_enabled() && s->num_cpu > 1) {
-return current_cpu->cpu_index;
-}
-return 0;
+/*
+ * Something other than a CPU accessing the GIC would be a bug as
+ * would a CPU index higher than the GICState expects to be
+ * handling
+ */
+g_assert(attrs.requester_cpu == 1);
+g_assert(attrs.requester_id < s->num_cpu);
+
+return attrs.requester_id;
 }
 
-static inline int gic_get_current_vcpu(GICState *s)
+static inline int gic_get_current_vcpu(GICState *s, MemTxAttrs attrs)
 {
-return gic_get_current_cpu(s) + GIC_NCPU;
+return gic_get_current_cpu(s, attrs) + GIC_NCPU;
 }
 
 /* Return true if this GIC config has interrupt groups, which is
@@ -951,7 +956,7 @@ static uint32_t gic_dist_readb(void *opaque, hwaddr offset, 
MemTxAttrs attrs)
 int cm;
 int mask;
 
-cpu = gic_get_current_cpu(s);
+cpu = gic_get_current_cpu(s, attrs);
 cm = 1 << cpu;
 if (offset < 0x100) {
 if (offset == 0) {  /* GICD_CTLR */
@@ -1182,7 +1187,7 @@ static void gic_dist_writeb(void *opaque, hwaddr offset,
 int i;
 int cpu;
 
-cpu = gic_get_current_cpu(s);
+cpu = gic_get_current_cpu(s, attrs);
 if (offset < 0x100) {
 if (offset == 0) {
 if (s->security_extn && !attrs.secure) {
@@ -1476,7 +1481,7 @@ static void gic_dist_writel(void *opaque, hwaddr offset,
 int mask;
 int target_cpu;
 
-cpu = gic_get_current_cpu(s);
+cpu = gic_get_current_cpu(s, attrs);
 irq = value & 0xf;
 switch ((value >> 24) & 3) {
 case 0:
@@ -1780,7 +1785,7 @@ static MemTxResult gic_thiscpu_read(void *opaque, hwaddr 
addr, uint64_t *data,
 unsigned size, MemTxAttrs attrs)
 {
 GICState *s = (GICState *)opaque;
-return gic_cpu_read(s, gic_get_current_cpu(s), addr, data, attrs);
+return gic_cpu_read(s, gic_get_current_cpu(s, attrs), addr, data, attrs);
 }
 
 static MemTxResult gic_thiscpu_write(void *opaque, hwaddr addr,
@@ -1788,7 +1793,7 @@ static MemTxResult gic_thiscpu_write(void *opaque, hwaddr 
addr,
  MemTxAttrs attrs)
 {
 GICState *s = (GICState *)opaque;
-return gic_cpu_write(s, gic_get_current_cpu(s), addr, value, attrs);
+return gic_cpu_write(s, gic_get_current_cpu(s, attrs), addr, value, attrs);
 }
 
 /* Wrappers to read/write the GIC CPU interface for a specific CPU.
@@ -1818,7 +1823,7 @@ static MemTxResult gic_thisvcpu_read(void *opaque, hwaddr 
addr, uint64_t *data,
 {
 GICState *s = (GICState *)opaque;
 
-return gic_cpu_read(s, gic_get_current_vcpu(s), addr, data, attrs);
+return gic_cpu_read(s, gic_get_current_vcpu(s, attrs), addr, data, attrs);
 }
 
 static MemTxResult gic_thisvcpu_write(void *opaque, hwaddr addr,
@@ -1827,7 +1832,7 @@ static MemTxResult gic_thisvcpu_write(void *opaque, 
hwaddr addr,
 {
 GICState *s = (GICState *)opaque;
 
-return gic_cpu_write(s, gic_get_current_vcpu(s), addr, value, attrs);
+return gic_cpu_write(s, gic_get_current_vcpu(s, attrs), addr, value, 
attrs);
 }
 
 static uint32_t gic_compute_eisr(GICState *s, int cpu, int lr_start)
@@ -1860,7 +1865,7 @@ static uint32_t gic_compute_elrsr(GICState *s, int cpu, 
int lr_start)
 
 static void gic_vmcr_write(GICState *s, uint32_t value, MemTxAttrs attrs)
 {
-int vcpu = gic_get_current_vcpu(s);
+int vcpu = gic_get_current_vcpu(s, attrs);
 uint32_t ctlr;
 uint32_t abpr;
 uint32_t bpr;
@@ -1995,7 +2000,7 @@ static MemTxResult gic_thiscpu_hyp_read(void *opaque, 
hwaddr addr, uint64_t *dat
 {
 GICState *s = (GICState *)opaque;
 
-return gic_hyp_read(s, gic_get_current_cpu(s), addr, data, attrs);
+return gic_hyp_read(s, gic_get_current_cpu(s, attrs), addr, data, attrs);
 }
 
 static MemTxResult gic_thiscpu_hyp_write(void *opaque, hwaddr addr,
@@ -2004,7 +2009,7 @@ static MemTxResult gic_thiscpu_hyp_write(void *opaque, 
hwaddr addr,
 {
 GICState *s = (GICState *)opaque;
 
-return gic_hyp_write(s, gic_get_current_cpu(s), addr, value, attrs);
+return gic_hyp_write(s, gic_get_current_cpu(s, attrs), addr, value, attrs);
 }
 
 static MemTxResult gic_do_hyp_read(void *opaque, hwaddr addr, uint64_t *data,
-- 
2.34.1

[PATCH v2 30/30] tests/docker: remove the Debian base images

2022-09-14 Thread Alex Bennée
We no longer use these in any of our images. Clean-up the remaining
comments and documentation that reference them and remove from the
build.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-25-alex.ben...@linaro.org>
---
 docs/devel/testing.rst   |  2 +-
 .gitlab-ci.d/container-core.yml  |  5 
 .gitlab-ci.d/containers.yml  |  5 
 tests/docker/Makefile.include|  8 +
 tests/docker/dockerfiles/debian10.docker | 38 
 tests/docker/dockerfiles/debian11.docker | 18 ---
 6 files changed, 2 insertions(+), 74 deletions(-)
 delete mode 100644 tests/docker/dockerfiles/debian10.docker
 delete mode 100644 tests/docker/dockerfiles/debian11.docker

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index f35f117d95..aea5b42356 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -375,7 +375,7 @@ locally by using the ``NOCACHE`` build option:
 
 .. code::
 
-   make docker-image-debian10 NOCACHE=1
+   make docker-image-debian-arm64-cross NOCACHE=1
 
 Images
 ~~
diff --git a/.gitlab-ci.d/container-core.yml b/.gitlab-ci.d/container-core.yml
index e8dd1f476a..08f8450fa1 100644
--- a/.gitlab-ci.d/container-core.yml
+++ b/.gitlab-ci.d/container-core.yml
@@ -10,8 +10,3 @@ amd64-fedora-container:
   extends: .container_job_template
   variables:
 NAME: fedora
-
-amd64-debian10-container:
-  extends: .container_job_template
-  variables:
-NAME: debian10
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index be34cbc7ba..96d2a3b58b 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -7,11 +7,6 @@ amd64-alpine-container:
   variables:
 NAME: alpine
 
-amd64-debian11-container:
-  extends: .container_job_template
-  variables:
-NAME: debian11
-
 amd64-debian-container:
   extends: .container_job_template
   stage: containers
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a3174346da..270e99786e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -69,9 +69,7 @@ docker-binfmt-image-debian-%: 
$(DOCKER_FILES_DIR)/debian-bootstrap.docker
{ echo "You will need to build $(EXECUTABLE)"; exit 
1;},\
"CHECK", "debian-$* exists"))
 
-# Enforce dependencies for composite images
-# we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian10 debian11
+# Special case cross-compiling x86_64 on non-x86_64 systems.
 ifeq ($(HOST_ARCH),x86_64)
 DOCKER_PARTIAL_IMAGES += debian-amd64-cross
 else
@@ -89,10 +87,6 @@ endif
 # The native build should never use the registry
 docker-image-debian-native: DOCKER_REGISTRY=
 
-# base images should not add a local user
-docker-image-debian10: NOUSER=1
-docker-image-debian11: NOUSER=1
-
 # alpine has no adduser
 docker-image-alpine: NOUSER=1
 
diff --git a/tests/docker/dockerfiles/debian10.docker 
b/tests/docker/dockerfiles/debian10.docker
deleted file mode 100644
index 03be923066..00
--- a/tests/docker/dockerfiles/debian10.docker
+++ /dev/null
@@ -1,38 +0,0 @@
-#
-# Docker multiarch cross-compiler target
-#
-# This docker target is builds on Debian cross compiler targets to build distro
-# with a selection of cross compilers for building test binaries.
-#
-# On its own you can't build much but the docker-foo-cross targets
-# build on top of the base debian image.
-#
-FROM docker.io/library/debian:buster-slim
-
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> 
/etc/apt/sources.list
-
-# Install common build utilities
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
-bc \
-build-essential \
-ca-certificates \
-ccache \
-clang \
-dbus \
-gdb-multiarch \
-gettext \
-git \
-libffi-dev \
-libncurses5-dev \
-ninja-build \
-pkg-config \
-psmisc \
-python3 \
-python3-sphinx \
-python3-sphinx-rtd-theme \
-python3-venv \
-$(apt-get -s build-dep --arch-only qemu | egrep ^Inst | fgrep '[all]' 
| cut -d\  -f2)
diff --git a/tests/docker/dockerfiles/debian11.docker 
b/tests/docker/dockerfiles/debian11.docker
deleted file mode 100644
index febf884f8f..00
--- a/tests/docker/dockerfiles/debian11.docker
+++ /dev/null
@@ -1,18 +0,0 @@
-#
-# Docker multiarch cross-compiler target
-#
-# This docker target uses the current development version of Debian as
-# a base for cross compilers for building test binaries. We won't
-# attempt to build QEMU on it yet given it is still in development.
-#
-# On its own you can't build much but the docker-foo-cross targets
-# build on top of the base debian image.
-#
-FROM 

[PATCH v2 24/30] tests/lcitool: bump to latest version

2022-09-14 Thread Alex Bennée
We need this to be able to cleanly build the x86 cross images. There
are a few minor updates triggered by lcitool-refresh including adding
"libslirp" to the freebsd vars and opensuse-leap which will help when
we finally drop the slirp submodule from QEMU.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-19-alex.ben...@linaro.org>

---
v2
  - expand commit re:libslirp
---
 .gitlab-ci.d/cirrus/freebsd-12.vars   | 2 +-
 .gitlab-ci.d/cirrus/freebsd-13.vars   | 2 +-
 tests/docker/dockerfiles/opensuse-leap.docker | 3 ++-
 tests/docker/dockerfiles/ubuntu2004.docker| 2 +-
 tests/lcitool/libvirt-ci  | 2 +-
 5 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.d/cirrus/freebsd-12.vars 
b/.gitlab-ci.d/cirrus/freebsd-12.vars
index 8fa5a320e9..1a5959810f 100644
--- a/.gitlab-ci.d/cirrus/freebsd-12.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-12.vars
@@ -11,6 +11,6 @@ MAKE='/usr/local/bin/gmake'
 NINJA='/usr/local/bin/ninja'
 PACKAGING_COMMAND='pkg'
 PIP3='/usr/local/bin/pip-3.8'
-PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage 
cmocka ctags curl cyrus-sasl dbus diffutils dtc fusefs-libs3 gettext git glib 
gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs 
libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja opencv 
perl5 pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx 
py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy 
spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage 
cmocka ctags curl cyrus-sasl dbus diffutils dtc fusefs-libs3 gettext git glib 
gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs 
libslirp libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja 
opencv perl5 pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx 
py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy 
spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
 PYPI_PKGS=''
 PYTHON='/usr/local/bin/python3'
diff --git a/.gitlab-ci.d/cirrus/freebsd-13.vars 
b/.gitlab-ci.d/cirrus/freebsd-13.vars
index 8ed7e33a77..5e5aafd7e5 100644
--- a/.gitlab-ci.d/cirrus/freebsd-13.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-13.vars
@@ -11,6 +11,6 @@ MAKE='/usr/local/bin/gmake'
 NINJA='/usr/local/bin/ninja'
 PACKAGING_COMMAND='pkg'
 PIP3='/usr/local/bin/pip-3.8'
-PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage 
cmocka ctags curl cyrus-sasl dbus diffutils dtc fusefs-libs3 gettext git glib 
gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs 
libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja opencv 
perl5 pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx 
py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy 
spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
+PKGS='alsa-lib bash bzip2 ca_root_nss capstone4 ccache cdrkit-genisoimage 
cmocka ctags curl cyrus-sasl dbus diffutils dtc fusefs-libs3 gettext git glib 
gmake gnutls gsed gtk3 json-c libepoxy libffi libgcrypt libjpeg-turbo libnfs 
libslirp libspice-server libssh libtasn1 llvm lzo2 meson ncurses nettle ninja 
opencv perl5 pixman pkgconf png py39-numpy py39-pillow py39-pip py39-sphinx 
py39-sphinx_rtd_theme py39-yaml python3 rpm2cpio sdl2 sdl2_image snappy 
spice-protocol tesseract texinfo usbredir virglrenderer vte3 zstd'
 PYPI_PKGS=''
 PYTHON='/usr/local/bin/python3'
diff --git a/tests/docker/dockerfiles/opensuse-leap.docker 
b/tests/docker/dockerfiles/opensuse-leap.docker
index 047a435ab5..041cf9c1ff 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -66,6 +66,7 @@ RUN zypper update -y && \
librbd-devel \
libseccomp-devel \
libselinux-devel \
+   libslirp-devel \
libspice-server-devel \
libssh-devel \
libtasn1-devel \
@@ -127,7 +128,7 @@ RUN zypper update -y && \
 ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
 ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-RUN pip3 install meson==0.56.0
+RUN /usr/bin/pip3 install meson==0.56.0
 
 ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker 
b/tests/docker/dockerfiles/ubuntu2004.docker
index 99803b343b..e1f4ed7c80 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -137,7 +137,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
 ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
 ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
 
-RUN pip3 install meson==0.56.0
+RUN /usr/bin/pip3 install meson==0.56.0
 
 ENV LANG "en_US.UTF-8"
 ENV MAKE "/usr/bin/make"
diff --git 

[PATCH v2 20/30] gitlab-ci: update aarch32/aarch64 custom runner jobs

2022-09-14 Thread Alex Bennée
The custom runner is now using 22.04 so we can drop our hacks to deal
with broken libssh and glusterfs. The provisioning scripts will be
updated in a separate commit.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-15-alex.ben...@linaro.org>

---
v2
  - fix extra-cflags merge conflict
  - update MAINTAINERS
---
 .gitlab-ci.d/custom-runners.yml   |  4 +--
 ...4-aarch32.yml => ubuntu-22.04-aarch32.yml} |  6 ++--
 ...4-aarch64.yml => ubuntu-22.04-aarch64.yml} | 36 +--
 MAINTAINERS   |  3 +-
 4 files changed, 24 insertions(+), 25 deletions(-)
 rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-aarch32.yml => 
ubuntu-22.04-aarch32.yml} (86%)
 rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-aarch64.yml => 
ubuntu-22.04-aarch64.yml} (82%)

diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 15aaccc481..97f99e29c2 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -15,6 +15,6 @@ variables:
 
 include:
   - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml'
-  - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml'
-  - local: '/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml'
+  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml'
+  - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml'
   - local: '/.gitlab-ci.d/custom-runners/centos-stream-8-x86_64.yml'
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml 
b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
similarity index 86%
rename from .gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml
rename to .gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
index cbfa9cc164..1a2f9b8dbe 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch32.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch32.yml
@@ -1,12 +1,12 @@
-# All ubuntu-20.04 jobs should run successfully in an environment
+# All ubuntu-22.04 jobs should run successfully in an environment
 # setup by the scripts/ci/setup/qemu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 20.04"
 
-ubuntu-20.04-aarch32-all:
+ubuntu-22.04-aarch32-all:
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - aarch32
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ 
/^staging/'
diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml 
b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
similarity index 82%
rename from .gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
rename to .gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
index 3f4dee4f86..ce0b18af6f 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-22.04-aarch64.yml
@@ -2,23 +2,21 @@
 # setup by the scripts/ci/setup/qemu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 20.04"
 
-ubuntu-20.04-aarch64-all-linux-static:
+ubuntu-22.04-aarch64-all-linux-static:
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - aarch64
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ 
/^staging/'
  - if: "$AARCH64_RUNNER_AVAILABLE"
  script:
- # --disable-libssh is needed because of 
https://bugs.launchpad.net/qemu/+bug/1838763
- # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
  - mkdir build
  - cd build
  # Disable -static-pie due to build error with system libc:
  # https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1987438
- - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh --disable-pie
+ - ../configure --enable-debug --static --disable-system --disable-pie
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
  - make --output-sync -j`nproc --ignore=40` check V=1
@@ -26,11 +24,11 @@ ubuntu-20.04-aarch64-all-linux-static:
  - make --output-sync -j`nproc --ignore=40` check-tcg V=1
|| { cat meson-logs/testlog.txt; exit 1; } ;
 
-ubuntu-20.04-aarch64-all:
+ubuntu-22.04-aarch64-all:
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - aarch64
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ 
/^staging/'
@@ -42,17 +40,17 @@ ubuntu-20.04-aarch64-all:
  script:
  - mkdir build
  - cd build
- - ../configure --disable-libssh
+ - ../configure
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
  - make --output-sync -j`nproc --ignore=40` check V=1
|| { cat meson-logs/testlog.txt; exit 1; } ;
 
-ubuntu-20.04-aarch64-alldbg:
+ubuntu-22.04-aarch64-alldbg:
  needs: []
  stage: build
  tags:
- - ubuntu_20.04
+ - ubuntu_22.04
  - aarch64
  rules:
  - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ 
/^staging/'
@@ -60,18 +58,18 @@ 

[PATCH v2 26/30] tests/docker: update and flatten debian-loongarch-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from the
QEMU base image just to compile test images. In this case it is a
binary distribution of the toolchain anyway.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-21-alex.ben...@linaro.org>

---
v2
  - use debian:11-slim for consistency with the others
---
 tests/docker/Makefile.include  | 1 -
 tests/docker/dockerfiles/debian-loongarch-cross.docker | 5 -
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a0f5109628..1d5a6f1fb4 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -128,7 +128,6 @@ docker-image-debian-nios2-cross: 
$(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(call debian-toolchain, $@)
 
 # Specialist build images, sometimes very limited tools
-docker-image-debian-loongarch-cross: docker-image-debian11
 docker-image-debian-microblaze-cross: docker-image-debian10
 docker-image-debian-nios2-cross: docker-image-debian10
 
diff --git a/tests/docker/dockerfiles/debian-loongarch-cross.docker 
b/tests/docker/dockerfiles/debian-loongarch-cross.docker
index ca2469d2a8..df578da40d 100644
--- a/tests/docker/dockerfiles/debian-loongarch-cross.docker
+++ b/tests/docker/dockerfiles/debian-loongarch-cross.docker
@@ -5,7 +5,10 @@
 # using a prebuilt toolchains for LoongArch64 from:
 # https://github.com/loongson/build-tools/releases
 #
-FROM qemu/debian11
+FROM docker.io/library/debian:11-slim
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> 
/etc/apt/sources.list
 
 RUN apt-get update && \
 DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
-- 
2.34.1




[PATCH v2 27/30] tests/docker: update and flatten debian-hexagon-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile as we do not some of the extraneous packages from
the base image to build the toolchain.

Signed-off-by: Alex Bennée 
Message-Id: <20220826172128.353798-22-alex.ben...@linaro.org>

---
v2
  - use debian11 consistently
  - remove "stand-alone" comments which will soon be irrelevant
  - reword commit as we do need some extra packages
---
 .../dockerfiles/debian-hexagon-cross.docker   | 19 ---
 .../dockerfiles/debian-loongarch-cross.docker |  3 +--
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker 
b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index d5dc299dc1..8d219bb81d 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -2,12 +2,10 @@
 # Docker Hexagon cross-compiler target
 #
 # This docker target is used for building hexagon tests. As it also
-# needs to be able to build QEMU itself in CI we include it's
-# build-deps. It is also a "stand-alone" image so as not to be
-# triggered by re-builds on other base images given it takes a long
-# time to build.
+# needs to be able to build QEMU itself in CI we include its
+# build-deps.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
 # Install common build utilities
 RUN apt update && \
@@ -15,11 +13,18 @@ RUN apt update && \
 DEBIAN_FRONTEND=noninteractive eatmydata \
 apt install -y --no-install-recommends \
 bison \
+ca-certificates \
+clang \
 cmake \
 flex \
+gcc \
 lld \
+make \
+ninja-build \
+python3 \
 rsync \
-wget
+wget \
+xz-utils
 
 ENV TOOLCHAIN_INSTALL /usr/local
 ENV ROOTFS /usr/local
@@ -32,7 +37,7 @@ ADD build-toolchain.sh 
/root/hexagon-toolchain/build-toolchain.sh
 
 RUN cd /root/hexagon-toolchain && ./build-toolchain.sh
 
-FROM debian:buster-slim
+FROM docker.io/library/debian:11-slim
 # Duplicate deb line as deb-src
 RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> 
/etc/apt/sources.list
 # Install QEMU build deps for use in CI
diff --git a/tests/docker/dockerfiles/debian-loongarch-cross.docker 
b/tests/docker/dockerfiles/debian-loongarch-cross.docker
index df578da40d..a8e8e98909 100644
--- a/tests/docker/dockerfiles/debian-loongarch-cross.docker
+++ b/tests/docker/dockerfiles/debian-loongarch-cross.docker
@@ -1,8 +1,7 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian11 base image,
-# using a prebuilt toolchains for LoongArch64 from:
+# This docker target uses prebuilt toolchains for LoongArch64 from:
 # https://github.com/loongson/build-tools/releases
 #
 FROM docker.io/library/debian:11-slim
-- 
2.34.1




[PATCH v2 25/30] tests/docker: update and flatten debian-amd64-cross

2022-09-14 Thread Alex Bennée
Now lcitool has support for building a x86_64 cross image we can use
it for this.

Signed-off-by: Alex Bennée 
Acked-by: Thomas Huth 
Message-Id: <20220826172128.353798-20-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml  |   1 -
 tests/docker/Makefile.include |   1 -
 .../dockerfiles/debian-amd64-cross.docker | 178 --
 tests/lcitool/refresh |   7 +
 4 files changed, 169 insertions(+), 18 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 091c0d8fcb..2d560e9764 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -7,7 +7,6 @@ alpha-debian-cross-container:
 amd64-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-amd64-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ddcc502049..a0f5109628 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -75,7 +75,6 @@ DOCKER_PARTIAL_IMAGES := debian10 debian11
 ifeq ($(HOST_ARCH),x86_64)
 DOCKER_PARTIAL_IMAGES += debian-amd64-cross
 else
-docker-image-debian-amd64-cross: docker-image-debian10
 DOCKER_PARTIAL_IMAGES += debian-amd64
 endif
 
diff --git a/tests/docker/dockerfiles/debian-amd64-cross.docker 
b/tests/docker/dockerfiles/debian-amd64-cross.docker
index 870109ef6a..7d2feb7bf7 100644
--- a/tests/docker/dockerfiles/debian-amd64-cross.docker
+++ b/tests/docker/dockerfiles/debian-amd64-cross.docker
@@ -1,22 +1,168 @@
+# THIS FILE WAS AUTO-GENERATED
 #
-# Docker x86_64 cross target
+#  $ lcitool dockerfile --layers all --cross x86_64 debian-11 qemu
 #
-# This docker target is used on non-x86_64 machines which need the
-# x86_64 cross compilers installed.
-#
-FROM qemu/debian10
-MAINTAINER Alex Bennée 
+# https://gitlab.com/libvirt/libvirt-ci
+
+FROM docker.io/library/debian:11-slim
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
+bash \
+bc \
+bsdextrautils \
+bzip2 \
+ca-certificates \
+ccache \
+dbus \
+debianutils \
+diffutils \
+exuberant-ctags \
+findutils \
+gcovr \
+genisoimage \
+gettext \
+git \
+hostname \
+libpcre2-dev \
+libspice-protocol-dev \
+llvm \
+locales \
+make \
+meson \
+ncat \
+ninja-build \
+openssh-client \
+perl-base \
+pkgconf \
+python3 \
+python3-numpy \
+python3-opencv \
+python3-pillow \
+python3-pip \
+python3-sphinx \
+python3-sphinx-rtd-theme \
+python3-venv \
+python3-yaml \
+rpm2cpio \
+sed \
+sparse \
+tar \
+tesseract-ocr \
+tesseract-ocr-eng \
+texinfo && \
+eatmydata apt-get autoremove -y && \
+eatmydata apt-get autoclean -y && \
+sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+dpkg-reconfigure locales
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
 
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture amd64
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
-crossbuild-essential-amd64
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt build-dep -yy -a amd64 --arch-only qemu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+dpkg --add-architecture amd64 && \
+eatmydata apt-get update && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+eatmydata apt-get install --no-install-recommends -y \
+g++-x86-64-linux-gnu \
+gcc-x86-64-linux-gnu \
+libaio-dev:amd64 \
+libasan5:amd64 \
+libasound2-dev:amd64 \
+libattr1-dev:amd64 \
+libbpf-dev:amd64 \
+libbrlapi-dev:amd64 \
+libbz2-dev:amd64 \
+libc6-dev:amd64 \
+libcacard-dev:amd64 \
+libcap-ng-dev:amd64 \
+libcapstone-dev:amd64 \
+libcmocka-dev:amd64 \
+libcurl4-gnutls-dev:amd64 \
+libdaxctl-dev:amd64 \
+libdrm-dev:amd64 \
+libepoxy-dev:amd64 \
+libfdt-dev:amd64 \
+libffi-dev:amd64 \
+ 

[PATCH v2 28/30] tests/docker: update and flatten debian-toolchain

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile as we do not need anything from the base image to
build the toolchain. This is used to build both the nios and
microblaze toolchains.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-23-alex.ben...@linaro.org>
---
 tests/docker/Makefile.include| 4 
 tests/docker/dockerfiles/debian-toolchain.docker | 5 +++--
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 1d5a6f1fb4..a3174346da 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -127,10 +127,6 @@ docker-image-debian-nios2-cross: 
$(DOCKER_FILES_DIR)/debian-toolchain.docker \
 $(DOCKER_FILES_DIR)/debian-nios2-cross.d/build-toolchain.sh
$(call debian-toolchain, $@)
 
-# Specialist build images, sometimes very limited tools
-docker-image-debian-microblaze-cross: docker-image-debian10
-docker-image-debian-nios2-cross: docker-image-debian10
-
 # These images may be good enough for building tests but not for test builds
 DOCKER_PARTIAL_IMAGES += debian-alpha-cross
 DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
diff --git a/tests/docker/dockerfiles/debian-toolchain.docker 
b/tests/docker/dockerfiles/debian-toolchain.docker
index 738d808aa6..c723377495 100644
--- a/tests/docker/dockerfiles/debian-toolchain.docker
+++ b/tests/docker/dockerfiles/debian-toolchain.docker
@@ -4,7 +4,7 @@
 # This dockerfile is used for building a cross-compiler toolchain.
 # The script for building the toolchain is supplied via extra-files.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:bullseye-slim
 
 # Install build utilities for building gcc and glibc.
 # ??? The build-dep isn't working, missing a number of
@@ -15,6 +15,7 @@ RUN apt update && \
 DEBIAN_FRONTEND=noninteractive eatmydata \
 apt install -y --no-install-recommends \
 bison \
+ca-certificates \
 flex \
 gawk \
 libmpc-dev \
@@ -32,5 +33,5 @@ RUN cd /root && ./build-toolchain.sh
 # Throw away the extra toolchain build deps, the downloaded source,
 # and the build trees by restoring the original debian10 image,
 # then copying the built toolchain from stage 0.
-FROM qemu/debian10
+FROM docker.io/library/debian:bullseye-slim
 COPY --from=0 /usr/local /usr/local
-- 
2.34.1




[PATCH v2 19/30] gitlab-ci/custom-runners: Disable -static-pie for ubuntu-20.04-aarch64

2022-09-14 Thread Alex Bennée
From: Richard Henderson 

The project has reached the magic size at which we see

/usr/aarch64-linux-gnu/lib/libc.a(init-first.o): in function 
`__libc_init_first':
(.text+0x10): relocation truncated to fit: R_AARCH64_LD64_GOTPAGE_LO15 against \
symbol `__environ' defined in .bss section in 
/usr/aarch64-linux-gnu/lib/libc.a(environ.o)
/usr/bin/ld: (.text+0x10): warning: too many GOT entries for -fpic, please 
recompile with -fPIC

The bug has been reported upstream, but in the meantime there is
nothing we can do except build a non-pie executable.

Signed-off-by: Richard Henderson 
Signed-off-by: Alex Bennée 
Message-Id: <20220823210329.1969895-1-richard.hender...@linaro.org>
Message-Id: <20220826172128.353798-14-alex.ben...@linaro.org>

---
v2
  - rm --extra-cflags workaround (now in configure)
---
 .gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml 
b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
index 3d878914e7..3f4dee4f86 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-20.04-aarch64.yml
@@ -16,7 +16,9 @@ ubuntu-20.04-aarch64-all-linux-static:
  # --disable-glusterfs is needed because there's no static version of those 
libs in distro supplied packages
  - mkdir build
  - cd build
- - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh
+ # Disable -static-pie due to build error with system libc:
+ # https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1987438
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs 
--disable-libssh --disable-pie
|| { cat config.log meson-logs/meson-log.txt; exit 1; }
  - make --output-sync -j`nproc --ignore=40`
  - make --output-sync -j`nproc --ignore=40` check V=1
-- 
2.34.1




[PATCH v2 29/30] tests/docker: remove FROM qemu/ support from docker.py

2022-09-14 Thread Alex Bennée
We want to migrate from docker.py to building our images directly with
docker/podman. Before we get there we need to make sure we don't
re-introduce our layered builds so bug out if we see FROM qemu/ in a
Dockerfile.

Signed-off-by: Alex Bennée 
Acked-by: Thomas Huth 
Message-Id: <20220826172128.353798-24-alex.ben...@linaro.org>
---
 tests/docker/docker.py | 38 ++
 1 file changed, 10 insertions(+), 28 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index d0af2861b8..3a1ed7cb18 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -205,22 +205,17 @@ def _read_qemu_dockerfile(img_name):
 return _read_dockerfile(df)
 
 
-def _dockerfile_preprocess(df):
-out = ""
+def _dockerfile_verify_flat(df):
+"Verify we do not include other qemu/ layers"
 for l in df.splitlines():
 if len(l.strip()) == 0 or l.startswith("#"):
 continue
 from_pref = "FROM qemu/"
 if l.startswith(from_pref):
-# TODO: Alternatively we could replace this line with "FROM $ID"
-# where $ID is the image's hex id obtained with
-#$ docker images $IMAGE --format="{{.Id}}"
-# but unfortunately that's not supported by RHEL 7.
-inlining = _read_qemu_dockerfile(l[len(from_pref):])
-out += _dockerfile_preprocess(inlining)
-continue
-out += l + "\n"
-return out
+print("We no longer support multiple QEMU layers.")
+print("Dockerfiles should be flat, ideally created by lcitool")
+return False
+return True
 
 
 class Docker(object):
@@ -309,23 +304,10 @@ def build_image(self, tag, docker_dir, dockerfile,
 if argv is None:
 argv = []
 
-# pre-calculate the docker checksum before any
-# substitutions we make for caching
-checksum = _text_checksum(_dockerfile_preprocess(dockerfile))
+if not _dockerfile_verify_flat(dockerfile):
+return -1
 
-if registry is not None:
-sources = re.findall("FROM qemu\/(.*)", dockerfile)
-# Fetch any cache layers we can, may fail
-for s in sources:
-pull_args = ["pull", "%s/qemu/%s" % (registry, s)]
-if self._do(pull_args, quiet=quiet) != 0:
-registry = None
-break
-# Make substitutions
-if registry is not None:
-dockerfile = dockerfile.replace("FROM qemu/",
-"FROM %s/qemu/" %
-(registry))
+checksum = _text_checksum(dockerfile)
 
 tmp_df = tempfile.NamedTemporaryFile(mode="w+t",
  encoding='utf-8',
@@ -371,7 +353,7 @@ def image_matches_dockerfile(self, tag, dockerfile):
 checksum = self.get_image_dockerfile_checksum(tag)
 except Exception:
 return False
-return checksum == _text_checksum(_dockerfile_preprocess(dockerfile))
+return checksum == _text_checksum(dockerfile)
 
 def run(self, cmd, keep, quiet, as_user=False):
 label = uuid.uuid4().hex
-- 
2.34.1




[PATCH v2 21/30] Deprecate 32 bit big-endian MIPS

2022-09-14 Thread Alex Bennée
It's becoming harder to maintain a cross-compiler to test this host
architecture as the old stable Debian 10 ("Buster") moved into LTS
which supports fewer architectures. For now:

  - mark it's deprecation in the docs
  - downgrade the containers to build TCG tests only
  - drop the cross builds from our CI

Users with an appropriate toolchain and user-space can still take
their chances building it.

Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Huacai Chen 
Message-Id: <20220826172128.353798-16-alex.ben...@linaro.org>

---
v2
  - explicit little endian instead of LE
  - s/A/As/
  - restore mips to dockerfile
---
 docs/about/build-platforms.rst|  2 +-
 docs/about/deprecated.rst | 13 +++
 .gitlab-ci.d/container-cross.yml  |  1 -
 .gitlab-ci.d/crossbuilds.yml  | 14 ---
 tests/docker/Makefile.include |  5 +--
 .../dockerfiles/debian-mips-cross.docker  | 38 +--
 6 files changed, 26 insertions(+), 47 deletions(-)

diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
index a2fee53248..1c1e7b9e11 100644
--- a/docs/about/build-platforms.rst
+++ b/docs/about/build-platforms.rst
@@ -41,7 +41,7 @@ Those hosts are officially supported, with various 
accelerators:
  - Accelerators
* - Arm
  - kvm (64 bit only), tcg, xen
-   * - MIPS
+   * - MIPS (little endian only)
  - kvm, tcg
* - PPC
  - kvm, tcg
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index c75a25daad..0d1fd4469b 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -213,6 +213,19 @@ MIPS ``Trap-and-Emul`` KVM support (since 6.0)
 The MIPS ``Trap-and-Emul`` KVM host and guest support has been removed
 from Linux upstream kernel, declare it deprecated.
 
+Host Architectures
+--
+
+BE MIPS (since 7.2)
+'''
+
+As Debian 10 ("Buster") moved into LTS the big endian 32 bit version of
+MIPS moved out of support making it hard to maintain our
+cross-compilation CI tests of the architecture. As we no longer have
+CI coverage support may bitrot away before the deprecation process
+completes. The little endian variants of MIPS (both 32 and 64 bit) are
+still a supported host architecture.
+
 QEMU API (QAPI) events
 --
 
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 611c6c0b39..95d57e1c5d 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -89,7 +89,6 @@ mips64el-debian-cross-container:
 mips-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-mips-cross
 
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 4a5fb6ea2a..c4cd96433d 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -70,20 +70,6 @@ cross-i386-tci:
 EXTRA_CONFIGURE_OPTS: 
--target-list=i386-softmmu,i386-linux-user,aarch64-softmmu,aarch64-linux-user,ppc-softmmu,ppc-linux-user
 MAKE_CHECK_ARGS: check check-tcg
 
-cross-mips-system:
-  extends: .cross_system_build_job
-  needs:
-job: mips-debian-cross-container
-  variables:
-IMAGE: debian-mips-cross
-
-cross-mips-user:
-  extends: .cross_user_build_job
-  needs:
-job: mips-debian-cross-container
-  variables:
-IMAGE: debian-mips-cross
-
 cross-mipsel-system:
   extends: .cross_system_build_job
   needs:
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index c3375f89c5..b1bf56434f 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -81,14 +81,12 @@ endif
 
 # For non-x86 hosts not all cross-compilers have been packaged
 ifneq ($(HOST_ARCH),x86_64)
-DOCKER_PARTIAL_IMAGES += debian-mips-cross debian-mipsel-cross 
debian-mips64el-cross
+DOCKER_PARTIAL_IMAGES += debian-mipsel-cross debian-mips64el-cross
 DOCKER_PARTIAL_IMAGES += debian-ppc64el-cross
 DOCKER_PARTIAL_IMAGES += debian-s390x-cross
 DOCKER_PARTIAL_IMAGES += fedora
 endif
 
-docker-image-debian-mips-cross: docker-image-debian10
-
 # The native build should never use the registry
 docker-image-debian-native: DOCKER_REGISTRY=
 
@@ -144,6 +142,7 @@ DOCKER_PARTIAL_IMAGES += debian-hppa-cross
 DOCKER_PARTIAL_IMAGES += debian-loongarch-cross
 DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
 DOCKER_PARTIAL_IMAGES += debian-microblaze-cross
+DOCKER_PARTIAL_IMAGES += debian-mips-cross
 DOCKER_PARTIAL_IMAGES += debian-nios2-cross
 DOCKER_PARTIAL_IMAGES += debian-riscv64-test-cross
 DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
diff --git a/tests/docker/dockerfiles/debian-mips-cross.docker 
b/tests/docker/dockerfiles/debian-mips-cross.docker
index 26c154014d..7b55f0f3b2 100644
--- a/tests/docker/dockerfiles/debian-mips-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips-cross.docker
@@ -1,32 

[PATCH v2 17/30] tests/vm: Remove obsolete Fedora VM test

2022-09-14 Thread Alex Bennée
From: Thomas Huth 

It's still based on Fedora 30 - which is not supported anymore by QEMU
since years. Seems like nobody is using (and refreshing) this, and it's
easier to test this via a container anyway, so let's remove this now.

Signed-off-by: Thomas Huth 
Message-Id: <20220822175317.190551-1-th...@redhat.com>
Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20220826172128.353798-13-alex.ben...@linaro.org>
---
 tests/vm/Makefile.include |   3 +-
 tests/vm/fedora   | 190 --
 2 files changed, 1 insertion(+), 192 deletions(-)
 delete mode 100755 tests/vm/fedora

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 8d2a164552..2cc2203d09 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -15,7 +15,7 @@ endif
 
 EFI_AARCH64 = $(wildcard $(BUILD_DIR)/pc-bios/edk2-aarch64-code.fd)
 
-X86_IMAGES := freebsd netbsd openbsd fedora haiku.x86_64
+X86_IMAGES := freebsd netbsd openbsd haiku.x86_64
 ifneq ($(GENISOIMAGE),)
 X86_IMAGES += centos
 ifneq ($(EFI_AARCH64),)
@@ -45,7 +45,6 @@ vm-help vm-test:
@echo "  vm-build-freebsd- Build QEMU in FreeBSD VM"
@echo "  vm-build-netbsd - Build QEMU in NetBSD VM"
@echo "  vm-build-openbsd- Build QEMU in OpenBSD VM"
-   @echo "  vm-build-fedora - Build QEMU in Fedora VM"
 ifneq ($(GENISOIMAGE),)
@echo "  vm-build-centos - Build QEMU in CentOS VM, 
with Docker"
 ifneq ($(EFI_AARCH64),)
diff --git a/tests/vm/fedora b/tests/vm/fedora
deleted file mode 100755
index 12eca919a0..00
--- a/tests/vm/fedora
+++ /dev/null
@@ -1,190 +0,0 @@
-#!/usr/bin/env python3
-#
-# Fedora VM image
-#
-# Copyright 2019 Red Hat Inc.
-#
-# Authors:
-#  Gerd Hoffmann 
-#
-# This code is licensed under the GPL version 2 or later.  See
-# the COPYING file in the top-level directory.
-#
-
-import os
-import re
-import sys
-import time
-import socket
-import subprocess
-import basevm
-
-class FedoraVM(basevm.BaseVM):
-name = "fedora"
-arch = "x86_64"
-
-base = 
"https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/;
-link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
-repo = base + "Server/x86_64/os/"
-full = base + "Everything/x86_64/os/"
-csum = "5e4eac4566d8c572bfb3bcf54b7d6c82006ec3c6c882a2c9235c6d3494d7b100"
-size = "20G"
-pkgs = [
-# tools
-'git-core',
-'gcc', 'binutils', 'make', 'ninja-build',
-
-# perl
-'perl',
-
-# libs: usb
-'"pkgconfig(libusb-1.0)"',
-'"pkgconfig(libusbredirparser-0.5)"',
-
-# libs: crypto
-'"pkgconfig(gnutls)"',
-
-# libs: ui
-'"pkgconfig(sdl2)"',
-'"pkgconfig(gtk+-3.0)"',
-'"pkgconfig(ncursesw)"',
-
-# libs: audio
-'"pkgconfig(libpulse)"',
-'"pkgconfig(alsa)"',
-
-# libs: migration
-'"pkgconfig(libzstd)"',
-]
-
-BUILD_SCRIPT = """
-set -e;
-rm -rf /home/qemu/qemu-test.*
-cd $(mktemp -d /home/qemu/qemu-test.XX);
-mkdir src build; cd src;
-tar -xf /dev/vdb;
-cd ../build
-../src/configure --python=python3 {configure_opts};
-gmake --output-sync -j{jobs} {target} {verbose};
-"""
-
-def build_image(self, img):
-self.print_step("Downloading install iso")
-cimg = self._download_with_cache(self.link, sha256sum=self.csum)
-img_tmp = img + ".tmp"
-iso = img + ".install.iso"
-
-self.print_step("Preparing iso and disk image")
-subprocess.check_call(["cp", "-f", cimg, iso])
-self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
-self.print_step("Booting installer")
-self.boot(img_tmp, extra_args = [
-"-machine", "graphics=off",
-"-device", "VGA",
-"-cdrom", iso
-])
-self.console_init(300)
-self.console_wait("installation process.")
-time.sleep(0.3)
-self.console_send("\t")
-time.sleep(0.3)
-self.console_send(" console=ttyS0")
-proxy = os.environ.get("http_proxy")
-if not proxy is None:
-self.console_send(" proxy=%s" % proxy)
-self.console_send(" inst.proxy=%s" % proxy)
-self.console_send(" inst.repo=%s" % self.repo)
-self.console_send("\n")
-
-self.console_wait_send("2) Use text mode", "2\n")
-
-self.console_wait_send("5) [!] Installation Dest", "5\n")
-self.console_wait_send("1) [x]",   "c\n")
-self.console_wait_send("2) [ ] Use All Space", "2\n")
-self.console_wait_send("2) [x] Use All Space", "c\n")
-self.console_wait_send("1) [ ] Standard Part", "1\n")
-self.console_wait_send("1) [x] Standard Part", "c\n")
-
-

[PATCH v2 15/30] tests/docker: remove tricore qemu/debian10 dependency

2022-09-14 Thread Alex Bennée
We missed removing this dependency when we flattened the build.

Fixes: 39ce923732 (gitlab: enable a very minimal build with the tricore 
container)
Signed-off-by: Alex Bennée 
Fixes: 39ce923732 ("gitlab: enable a very minimal build with the tricore  
container")
Message-Id: <20220826172128.353798-11-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml | 1 -
 tests/docker/Makefile.include| 1 -
 2 files changed, 2 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 67bbf19a27..611c6c0b39 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -148,7 +148,6 @@ sparc64-debian-cross-container:
 tricore-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-tricore-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e034eca3af..5c9398bbc9 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -132,7 +132,6 @@ docker-image-debian-nios2-cross: 
$(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(call debian-toolchain, $@)
 
 # Specialist build images, sometimes very limited tools
-docker-image-debian-tricore-cross: docker-image-debian10
 docker-image-debian-all-test-cross: docker-image-debian10
 docker-image-debian-loongarch-cross: docker-image-debian11
 docker-image-debian-microblaze-cross: docker-image-debian10
-- 
2.34.1




[PATCH v2 14/30] tests/docker: flatten debian-powerpc-test-cross

2022-09-14 Thread Alex Bennée
Flatten into a single dockerfile. We really don't need the rest of the
stuff from the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-10-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml |  1 -
 tests/docker/Makefile.include|  1 -
 .../dockerfiles/debian-powerpc-test-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index db0ea15d0d..67bbf19a27 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -102,7 +102,6 @@ mipsel-debian-cross-container:
 powerpc-test-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian11-container']
   variables:
 NAME: debian-powerpc-test-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 8828b6b8fa..e034eca3af 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -137,7 +137,6 @@ docker-image-debian-all-test-cross: docker-image-debian10
 docker-image-debian-loongarch-cross: docker-image-debian11
 docker-image-debian-microblaze-cross: docker-image-debian10
 docker-image-debian-nios2-cross: docker-image-debian10
-docker-image-debian-powerpc-test-cross: docker-image-debian11
 docker-image-debian-riscv64-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
diff --git a/tests/docker/dockerfiles/debian-powerpc-test-cross.docker 
b/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
index 36b336f709..d6b2909cc4 100644
--- a/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-powerpc-test-cross.docker
@@ -1,13 +1,15 @@
 #
 # Docker powerpc/ppc64/ppc64le cross-compiler target
 #
-# This docker target builds on the debian Bullseye base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian11
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-powerpc-linux-gnu \
 libc6-dev-powerpc-cross \
 gcc-10-powerpc64-linux-gnu \
-- 
2.34.1




[PATCH v2 13/30] tests/docker: update and flatten debian-sparc64-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-9-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml |  1 -
 tests/docker/Makefile.include|  1 -
 tests/docker/dockerfiles/debian-sparc64-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 8a611fc824..db0ea15d0d 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -143,7 +143,6 @@ sh4-debian-cross-container:
 sparc64-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-sparc64-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 37c4ea913f..8828b6b8fa 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -89,7 +89,6 @@ DOCKER_PARTIAL_IMAGES += fedora
 endif
 
 docker-image-debian-mips-cross: docker-image-debian10
-docker-image-debian-sparc64-cross: docker-image-debian10
 
 # The native build should never use the registry
 docker-image-debian-native: DOCKER_REGISTRY=
diff --git a/tests/docker/dockerfiles/debian-sparc64-cross.docker 
b/tests/docker/dockerfiles/debian-sparc64-cross.docker
index f4bb9b561c..8d3d306bc1 100644
--- a/tests/docker/dockerfiles/debian-sparc64-cross.docker
+++ b/tests/docker/dockerfiles/debian-sparc64-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-sparc64-linux-gnu \
 libc6-dev-sparc64-cross
-- 
2.34.1




[PATCH v2 05/30] tests/avocado: add explicit timeout for ppc64le TCG tests

2022-09-14 Thread Alex Bennée
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée 
---
 tests/avocado/boot_linux.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index 4f07c27ac6..b7522ad3a1 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -115,6 +115,8 @@ class BootLinuxPPC64(LinuxTest):
 :avocado: tags=arch:ppc64
 """
 
+timeout = 180
+
 def test_pseries_tcg(self):
 """
 :avocado: tags=machine:pseries
-- 
2.34.1




[PATCH v2 12/30] tests/docker: update and flatten debian-sh4-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-8-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml |  1 -
 tests/docker/Makefile.include|  1 -
 tests/docker/dockerfiles/debian-sh4-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index a3bfa483bf..8a611fc824 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -137,7 +137,6 @@ s390x-debian-cross-container:
 sh4-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-sh4-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 6c2ee3b175..37c4ea913f 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -89,7 +89,6 @@ DOCKER_PARTIAL_IMAGES += fedora
 endif
 
 docker-image-debian-mips-cross: docker-image-debian10
-docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
 
 # The native build should never use the registry
diff --git a/tests/docker/dockerfiles/debian-sh4-cross.docker 
b/tests/docker/dockerfiles/debian-sh4-cross.docker
index fd3af89575..d48ed9065f 100644
--- a/tests/docker/dockerfiles/debian-sh4-cross.docker
+++ b/tests/docker/dockerfiles/debian-sh4-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-sh4-linux-gnu \
 libc6-dev-sh4-cross
-- 
2.34.1




[PATCH v2 08/30] tests/docker: update and flatten debian-alpha-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-4-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml   |  1 -
 tests/docker/Makefile.include  |  1 -
 tests/docker/dockerfiles/debian-alpha-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 505b267542..802e332205 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -1,7 +1,6 @@
 alpha-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-alpha-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 9a45e8890b..c565aa5e7b 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -88,7 +88,6 @@ DOCKER_PARTIAL_IMAGES += debian-s390x-cross
 DOCKER_PARTIAL_IMAGES += fedora
 endif
 
-docker-image-debian-alpha-cross: docker-image-debian10
 docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-alpha-cross.docker 
b/tests/docker/dockerfiles/debian-alpha-cross.docker
index 10fe30df0d..4eeb43c78a 100644
--- a/tests/docker/dockerfiles/debian-alpha-cross.docker
+++ b/tests/docker/dockerfiles/debian-alpha-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-alpha-linux-gnu \
 libc6.1-dev-alpha-cross
-- 
2.34.1




[PATCH v2 16/30] tests/docker: remove amd64 qemu/debian10 dependency

2022-09-14 Thread Alex Bennée
We missed removing this dependency when we flattened the build.

Fixes 9e19fd7d4a (tests/docker: update debian-amd64 with lcitool)

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-12-alex.ben...@linaro.org>
---
 tests/docker/Makefile.include | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 5c9398bbc9..c3375f89c5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -73,7 +73,6 @@ docker-binfmt-image-debian-%: 
$(DOCKER_FILES_DIR)/debian-bootstrap.docker
 # we don't run tests on intermediate images (used as base by another image)
 DOCKER_PARTIAL_IMAGES := debian10 debian11
 ifeq ($(HOST_ARCH),x86_64)
-docker-image-debian-amd64: docker-image-debian10
 DOCKER_PARTIAL_IMAGES += debian-amd64-cross
 else
 docker-image-debian-amd64-cross: docker-image-debian10
-- 
2.34.1




[PATCH v2 07/30] tests/avocado: reduce the default timeout to 120s

2022-09-14 Thread Alex Bennée
We should be aiming to keep our tests under 2 minutes so lets reduce
the default timeout to that. Tests that we know take longer should
explicitly set a longer timeout.

Signed-off-by: Alex Bennée 
Message-Id: <20220826172128.353798-3-alex.ben...@linaro.org>
---
 tests/avocado/avocado_qemu/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py 
b/tests/avocado/avocado_qemu/__init__.py
index 5f1d3e551c..910f3ba1ea 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -229,7 +229,7 @@ def exec_command_and_wait_for_pattern(test, command,
 class QemuBaseTest(avocado.Test):
 
 # default timeout for all tests, can be overridden
-timeout = 900
+timeout = 120
 
 def _get_unique_tag_val(self, tag_name):
 """
-- 
2.34.1




[PATCH v2 10/30] tests/docker: update and flatten debian-m68k-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-6-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml  |  1 -
 tests/docker/Makefile.include |  1 -
 tests/docker/dockerfiles/debian-m68k-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 6c1d765463..15a5270f6d 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -71,7 +71,6 @@ hppa-debian-cross-container:
 m68k-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-m68k-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e39597d35c..95790e974e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -88,7 +88,6 @@ DOCKER_PARTIAL_IMAGES += debian-s390x-cross
 DOCKER_PARTIAL_IMAGES += fedora
 endif
 
-docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
 docker-image-debian-mips64-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-m68k-cross.docker 
b/tests/docker/dockerfiles/debian-m68k-cross.docker
index fcb10e3534..dded71c5d2 100644
--- a/tests/docker/dockerfiles/debian-m68k-cross.docker
+++ b/tests/docker/dockerfiles/debian-m68k-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-m68k-linux-gnu \
 libc6-dev-m68k-cross
-- 
2.34.1




[PATCH v2 03/30] tests/avocado: add explicit timeout for Aarch64 TCG tests

2022-09-14 Thread Alex Bennée
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée 
---
 tests/avocado/boot_linux.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index ee584d2fdf..67a24fe51c 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -63,6 +63,7 @@ class BootLinuxAarch64(LinuxTest):
 :avocado: tags=machine:virt
 :avocado: tags=machine:gic-version=2
 """
+timeout = 240
 
 def add_common_args(self):
 self.vm.add_args('-bios',
-- 
2.34.1




[PATCH v2 09/30] tests/docker: update and flatten debian-hppa-cross

2022-09-14 Thread Alex Bennée
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-5-alex.ben...@linaro.org>
---
 .gitlab-ci.d/container-cross.yml  |  1 -
 tests/docker/Makefile.include |  1 -
 tests/docker/dockerfiles/debian-hppa-cross.docker | 12 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 802e332205..6c1d765463 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -65,7 +65,6 @@ hexagon-cross-container:
 hppa-debian-cross-container:
   extends: .container_job_template
   stage: containers
-  needs: ['amd64-debian10-container']
   variables:
 NAME: debian-hppa-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index c565aa5e7b..e39597d35c 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -88,7 +88,6 @@ DOCKER_PARTIAL_IMAGES += debian-s390x-cross
 DOCKER_PARTIAL_IMAGES += fedora
 endif
 
-docker-image-debian-hppa-cross: docker-image-debian10
 docker-image-debian-m68k-cross: docker-image-debian10
 docker-image-debian-mips-cross: docker-image-debian10
 docker-image-debian-mips64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-hppa-cross.docker 
b/tests/docker/dockerfiles/debian-hppa-cross.docker
index 3d6c65a3ef..af1c8403d8 100644
--- a/tests/docker/dockerfiles/debian-hppa-cross.docker
+++ b/tests/docker/dockerfiles/debian-hppa-cross.docker
@@ -1,12 +1,14 @@
 #
 # Docker cross-compiler target
 #
-# This docker target builds on the debian Buster base image.
+# This docker target builds on the Debian Bullseye base image.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
 
-RUN apt update && \
-DEBIAN_FRONTEND=noninteractive eatmydata \
-apt install -y --no-install-recommends \
+RUN export DEBIAN_FRONTEND=noninteractive && \
+apt-get update && \
+apt-get install -y eatmydata && \
+eatmydata apt-get dist-upgrade -y && \
+eatmydata apt-get install --no-install-recommends -y \
 gcc-hppa-linux-gnu \
 libc6-dev-hppa-cross
-- 
2.34.1




[PATCH v2 04/30] tests/avocado: add explicit timeout for s390 TCG tests

2022-09-14 Thread Alex Bennée
We don't want to rely on the soon to be reduced default time. These
tests are still slow for something we want to run in CI though.

Signed-off-by: Alex Bennée 
---
 tests/avocado/boot_linux.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index 67a24fe51c..4f07c27ac6 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -130,6 +130,8 @@ class BootLinuxS390X(LinuxTest):
 :avocado: tags=arch:s390x
 """
 
+timeout = 240
+
 @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
 def test_s390_ccw_virtio_tcg(self):
 """
-- 
2.34.1




[PATCH v2 06/30] tests/avocado: split the AST2x00Machine classes

2022-09-14 Thread Alex Bennée
The SDK tests take a lot longer to run and hence need a longer
timeout. As they run well over the 60 second maximum for CI lets also
disable them for CI as well.

I suspect they also suffer from the inability to detect the login
prompt due to no newlines being processed.

Signed-off-by: Alex Bennée 
Reviewed-by: Thomas Huth 
Message-Id: <20220826172128.353798-2-alex.ben...@linaro.org>
---
 tests/avocado/machine_aspeed.py | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/tests/avocado/machine_aspeed.py b/tests/avocado/machine_aspeed.py
index 65d38f4efa..0f64eb636c 100644
--- a/tests/avocado/machine_aspeed.py
+++ b/tests/avocado/machine_aspeed.py
@@ -6,12 +6,14 @@
 # later.  See the COPYING file in the top-level directory.
 
 import time
+import os
 
 from avocado_qemu import QemuSystemTest
 from avocado_qemu import wait_for_console_pattern
 from avocado_qemu import exec_command
 from avocado_qemu import exec_command_and_wait_for_pattern
 from avocado.utils import archive
+from avocado import skipIf
 
 
 class AST1030Machine(QemuSystemTest):
@@ -176,6 +178,20 @@ def test_arm_ast2600_evb_builroot(self):
 self.do_test_arm_aspeed_buidroot_poweroff()
 
 
+class AST2x00MachineSDK(QemuSystemTest):
+
+# FIXME: Although these tests boot a whole distro they are still
+# slower than comparable machine models. There may be some
+# optimisations which bring down the runtime. In the meantime they
+# have generous timeouts and are disable for CI which aims for all
+# tests to run in less than 60 seconds.
+timeout = 240
+
+def wait_for_console_pattern(self, success_message, vm=None):
+wait_for_console_pattern(self, success_message,
+ failure_message='Kernel panic - not syncing',
+ vm=vm)
+
 def do_test_arm_aspeed_sdk_start(self, image, cpu_id):
 self.vm.set_console()
 self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw',
@@ -187,6 +203,7 @@ def do_test_arm_aspeed_sdk_start(self, image, cpu_id):
 self.wait_for_console_pattern('Starting kernel ...')
 self.wait_for_console_pattern('Booting Linux on physical CPU ' + 
cpu_id)
 
+@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
 def test_arm_ast2500_evb_sdk(self):
 """
 :avocado: tags=arch:arm
@@ -204,6 +221,7 @@ def test_arm_ast2500_evb_sdk(self):
 self.workdir + '/ast2500-default/image-bmc', '0x0')
 self.wait_for_console_pattern('ast2500-default login:')
 
+@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
 def test_arm_ast2600_evb_sdk(self):
 """
 :avocado: tags=arch:arm
-- 
2.34.1




[PATCH v2 00/30] testing/next pre-PR (testing update and mips deprecation)

2022-09-14 Thread Alex Bennée
Hi,

This is the current state of the testing tree. It was slightly delayed
from the last posting as I was waiting for the libvirt-ci update to
get merged. Changes since last post:

  - added some more avocado timeouts
  - moved cflags hack in the --disable-pie patch into configure
  - usual addressing of review comments (see --- in patches)

I'd like to roll the PR next week so please review the remaining
un-reviewed patches:

- tests/docker: remove FROM qemu/ support from docker.py
- tests/docker: update and flatten debian-hexagon-cross
- tests/docker: update and flatten debian-amd64-cross
- tests/docker: flatten debian-riscv64-test-cross
- configure: explicitly set cflags for --disable-pie
- tests/docker: remove tricore qemu/debian10 dependency
- tests/avocado: reduce the default timeout to 120s
- tests/avocado: add explicit timeout for ppc64le TCG tests
- tests/avocado: add explicit timeout for s390 TCG tests
- tests/avocado: add explicit timeout for Aarch64 TCG tests
- gitlab: reduce targets in cross_user_build_job

Alex Bennée (27):
  gitlab: reduce targets in cross_user_build_job
  tests/avocado: add explicit timeout for Aarch64 TCG tests
  tests/avocado: add explicit timeout for s390 TCG tests
  tests/avocado: add explicit timeout for ppc64le TCG tests
  tests/avocado: split the AST2x00Machine classes
  tests/avocado: reduce the default timeout to 120s
  tests/docker: update and flatten debian-alpha-cross
  tests/docker: update and flatten debian-hppa-cross
  tests/docker: update and flatten debian-m68k-cross
  tests/docker: update and flatten debian-mips64-cross
  tests/docker: update and flatten debian-sh4-cross
  tests/docker: update and flatten debian-sparc64-cross
  tests/docker: flatten debian-powerpc-test-cross
  tests/docker: remove tricore qemu/debian10 dependency
  tests/docker: remove amd64 qemu/debian10 dependency
  configure: explicitly set cflags for --disable-pie
  gitlab-ci: update aarch32/aarch64 custom runner jobs
  Deprecate 32 bit big-endian MIPS
  tests/docker: flatten debian-riscv64-test-cross
  tests/docker: update and flatten debian-all-test-cross
  tests/lcitool: bump to latest version
  tests/docker: update and flatten debian-amd64-cross
  tests/docker: update and flatten debian-loongarch-cross
  tests/docker: update and flatten debian-hexagon-cross
  tests/docker: update and flatten debian-toolchain
  tests/docker: remove FROM qemu/ support from docker.py
  tests/docker: remove the Debian base images

Richard Henderson (1):
  gitlab-ci/custom-runners: Disable -static-pie for ubuntu-20.04-aarch64

Thomas Huth (2):
  tests/avocado/boot_linux_console: Fix the
test_aarch64_xlnx_versal_virt test
  tests/vm: Remove obsolete Fedora VM test

 docs/about/build-platforms.rst|   2 +-
 docs/about/deprecated.rst |  13 ++
 docs/devel/testing.rst|   2 +-
 configure |   6 +
 .gitlab-ci.d/cirrus/freebsd-12.vars   |   2 +-
 .gitlab-ci.d/cirrus/freebsd-13.vars   |   2 +-
 .gitlab-ci.d/container-core.yml   |   5 -
 .gitlab-ci.d/container-cross.yml  |  12 --
 .gitlab-ci.d/containers.yml   |   5 -
 .gitlab-ci.d/crossbuild-template.yml  |   5 +-
 .gitlab-ci.d/crossbuilds.yml  |  14 --
 .gitlab-ci.d/custom-runners.yml   |   4 +-
 ...4-aarch32.yml => ubuntu-22.04-aarch32.yml} |   6 +-
 ...4-aarch64.yml => ubuntu-22.04-aarch64.yml} |  38 ++--
 MAINTAINERS   |   3 +-
 tests/avocado/avocado_qemu/__init__.py|   2 +-
 tests/avocado/boot_linux.py   |   5 +
 tests/avocado/boot_linux_console.py   |   6 +-
 tests/avocado/machine_aspeed.py   |  18 ++
 tests/docker/Makefile.include |  30 +--
 tests/docker/docker.py|  38 +---
 .../dockerfiles/debian-all-test-cross.docker  |  18 +-
 .../dockerfiles/debian-alpha-cross.docker |  12 +-
 .../dockerfiles/debian-amd64-cross.docker | 178 ++--
 .../dockerfiles/debian-hexagon-cross.docker   |  19 +-
 .../dockerfiles/debian-hppa-cross.docker  |  12 +-
 .../dockerfiles/debian-loongarch-cross.docker |   8 +-
 .../dockerfiles/debian-m68k-cross.docker  |  12 +-
 .../dockerfiles/debian-mips-cross.docker  |  38 +---
 .../dockerfiles/debian-mips64-cross.docker|  12 +-
 .../debian-powerpc-test-cross.docker  |  12 +-
 .../debian-riscv64-test-cross.docker  |  10 +-
 .../dockerfiles/debian-sh4-cross.docker   |  12 +-
 .../dockerfiles/debian-sparc64-cross.docker   |  12 +-
 .../dockerfiles/debian-toolchain.docker   |   5 +-
 tests/docker/dockerfiles/debian10.docker  |  38 
 tests/docker/dockerfiles/debian11.docker  |  18 --
 tests/docker/dockerfiles/opensuse-leap.docker |   3 +-
 tests/docker/dockerfiles/ubuntu2004.docker|   2 +-
 tests/lcitool/libvirt-ci  |   2 +-
 

[PATCH v2 02/30] tests/avocado/boot_linux_console: Fix the test_aarch64_xlnx_versal_virt test

2022-09-14 Thread Alex Bennée
From: Thomas Huth 

The assets that this test tries to download have been removed from the
server. Update to a newer version to get it working again.

Signed-off-by: Thomas Huth 
Reviewed-by: Alistair Francis 
Message-Id: <20220829080940.110831-1-th...@redhat.com>
Signed-off-by: Alex Bennée 
---
 tests/avocado/boot_linux_console.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/avocado/boot_linux_console.py 
b/tests/avocado/boot_linux_console.py
index 6b1533c17c..f26e036ab5 100644
--- a/tests/avocado/boot_linux_console.py
+++ b/tests/avocado/boot_linux_console.py
@@ -335,13 +335,13 @@ def test_aarch64_xlnx_versal_virt(self):
 """
 images_url = ('http://ports.ubuntu.com/ubuntu-ports/dists/'
   'bionic-updates/main/installer-arm64/'
-  '20101020ubuntu543.15/images/')
+  '20101020ubuntu543.19/images/')
 kernel_url = images_url + 'netboot/ubuntu-installer/arm64/linux'
-kernel_hash = '5bfc54cf7ed8157d93f6e5b0241e727b6dc22c50'
+kernel_hash = 'e167757620640eb26de0972f578741924abb3a82'
 kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
 initrd_url = images_url + 'netboot/ubuntu-installer/arm64/initrd.gz'
-initrd_hash = 'd385d3e88d53e2004c5d43cbe668b458a094f772'
+initrd_hash = 'cab5cb3fcefca8408aa5aae57f24574bfce8bdb9'
 initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
 
 self.vm.set_console()
-- 
2.34.1




[PATCH v2 01/30] gitlab: reduce targets in cross_user_build_job

2022-09-14 Thread Alex Bennée
We already limit the scope of the cross system build to reduce the
cross build times. With the recent addition of more targets we are
also running into timeout issues for some of the cross user builds.

I've selected a few of those linux-user targets which are less likely
to be in common use as distros don't have pre-built rootfs for them.
I've also added the same CROSS_SKIP_TARGETS variable as is
occasionally used to further limit cross system builds.

Signed-off-by: Alex Bennée 
---
 .gitlab-ci.d/crossbuild-template.yml | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml 
b/.gitlab-ci.d/crossbuild-template.yml
index 28b2142ec2..5e8892fd49 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -46,5 +46,8 @@
 - cd build
 - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
   ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
---disable-system
+--disable-system --target-list-exclude="aarch64_be-linux-user
+  alpha-linux-user cris-linux-user m68k-linux-user 
microblazeel-linux-user
+  nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
+  xtensa-linux-user $CROSS_SKIP_TARGETS"
 - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
-- 
2.34.1




Re: [PATCH 2/3] vdpa: load vlan configuration at NIC startup

2022-09-14 Thread Si-Wei Liu




On 9/14/2022 2:57 PM, Eugenio Perez Martin wrote:

On Wed, Sep 14, 2022 at 1:33 PM Si-Wei Liu  wrote:



On 9/14/2022 3:20 AM, Jason Wang wrote:

On Fri, Sep 9, 2022 at 4:02 PM Eugenio Perez Martin  wrote:

On Fri, Sep 9, 2022 at 8:40 AM Jason Wang  wrote:

On Fri, Sep 9, 2022 at 2:38 PM Jason Wang  wrote:

On Wed, Sep 7, 2022 at 12:36 AM Eugenio Pérez  wrote:

To have enabled vlans at device startup may happen in the destination of
a live migration, so this configuration must be restored.

At this moment the code is not accessible, since SVQ refuses to start if
vlan feature is exposed by the device.

Signed-off-by: Eugenio Pérez 
---
   net/vhost-vdpa.c | 46 --
   1 file changed, 44 insertions(+), 2 deletions(-)

diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
index 4bc3fd01a8..ecbfd08eb9 100644
--- a/net/vhost-vdpa.c
+++ b/net/vhost-vdpa.c
@@ -100,6 +100,8 @@ static const uint64_t vdpa_svq_device_features =
   BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
   BIT_ULL(VIRTIO_NET_F_STANDBY);

+#define MAX_VLAN(1 << 12)   /* Per 802.1Q definition */
+
   VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
   {
   VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
@@ -423,6 +425,47 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s,
   return *s->status != VIRTIO_NET_OK;
   }

+static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s,
+   const VirtIONet *n,
+   uint16_t vid)
+{
+ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_VLAN,
+  VIRTIO_NET_CTRL_VLAN_ADD,
+  , sizeof(vid));
+if (unlikely(dev_written < 0)) {
+return dev_written;
+}
+
+if (unlikely(*s->status != VIRTIO_NET_OK)) {
+return -EINVAL;
+}
+
+return 0;
+}
+
+static int vhost_vdpa_net_load_vlan(VhostVDPAState *s,
+const VirtIONet *n)
+{
+uint64_t features = n->parent_obj.guest_features;
+
+if (!(features & BIT_ULL(VIRTIO_NET_F_CTRL_VLAN))) {
+return 0;
+}
+
+for (int i = 0; i < MAX_VLAN >> 5; i++) {
+for (int j = 0; n->vlans[i] && j <= 0x1f; j++) {
+if (n->vlans[i] & (1U << j)) {
+int r = vhost_vdpa_net_load_single_vlan(s, n, (i << 5) + j);

This seems to cause a lot of latency if the driver has a lot of vlans.

I wonder if it's simply to let all vlan traffic go by disabling
CTRL_VLAN feature at vDPA layer.

The guest will not be able to recover that vlan configuration.

Spec said it's best effort and we actually don't do this for
vhost-net/user for years and manage to survive.

I thought there's a border line between best effort and do nothing. ;-)


I also think it is worth at least trying to migrate it for sure. For
MAC there may be too many entries, but VLAN should be totally
manageable (and the information is already there, the bitmap is
actually being migrated).
The vlan bitmap is migrated, though you'd still need to add vlan filter 
one by one through ctrl vq commands, correct? AFAIK you can add one 
filter for one single vlan ID at a time via VIRTIO_NET_CTRL_VLAN_ADD.





I think that the reason this could survive is really software
implementation specific - say if the backend doesn't start with VLAN
filtering disabled (meaning allow all VLAN traffic to pass) then it
would become a problem. This won't be a problem for almost PF NICs but
may be problematic for vDPA e.g. VF backed VDPAs.

I'd say the driver would expect all vlan filters to be cleared after a
reset, isn't it?
If I understand the intended behavior (from QEMU implementation) 
correctly, when VIRTIO_NET_F_CTRL_VLAN is negotiated it would start with 
all VLANs filtered (meaning only untagged traffic can be received and 
traffic with VLAN tag would get dropped). However, if 
VIRTIO_NET_F_CTRL_VLAN is not negotiated, all traffic including untagged 
and tagged can be received.



  The spec doesn't explicitly say anything about that
as far as I see.
Here the spec is totally ruled by the (software artifact of) 
implementation rather than what a real device is expected to work with 
VLAN rx filters. Are we sure we'd stick to this flawed device 
implementation? The guest driver seems to be agnostic with this broken 
spec behavior so far, and I am afraid it's an overkill to add another 
feature bit or ctrl command to VLAN filter in clean way.





Another idea is to extend the spec to allow us to accept a bitmap of
the vlan ids via a single command, then we will be fine.


I'm not sure if adding more ways to configure something is the answer,
but I'd be ok to implement it.

Another idea is to allow the sending of many CVQ commands in parallel.
It shouldn't be very hard to achieve using exposed buffers as ring
buffers, and it will short down the start of the devices with many
features.


Re: [PATCH v11 20/21] blockjob: remove unused functions

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:21, Emanuele Giuseppe Esposito wrote:

These public functions are not used anywhere, thus can be dropped.

Signed-off-by: Emanuele Giuseppe Esposito
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Kevin Wolf


Reviewed-by: Vladimir Sementsov-Ogievskiy 

--
Best regards,
Vladimir



Re: [PATCH v11 21/21] job: remove unused functions

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:21, Emanuele Giuseppe Esposito wrote:

These public functions are not used anywhere, thus can be dropped.
Also, since this is the final job API that doesn't use AioContext
lock and replaces it with job_lock, adjust all remaining function
documentation to clearly specify if the job lock is taken or not.

Also document the locking requirements for a few functions
where the second version is not removed.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Stefan Hajnoczi 
Reviewed-by: Kevin Wolf 
---
  include/qemu/job.h |  96 ++
  job.c  | 101 ++---
  2 files changed, 34 insertions(+), 163 deletions(-)

diff --git a/include/qemu/job.h b/include/qemu/job.h
index ad8b32b4ba..762d6a98a7 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -384,6 +384,8 @@ JobTxn *job_txn_new(void);
  /**
   * Release a reference that was previously acquired with job_txn_add_job or


[..]


+/**
+ * Returns true if the job is user-paused.
+ * Called with job lock held.
+ */
  bool job_user_paused_locked(Job *job);
  
  /**

   * Resume the specified @job.
   * Must be paired with a preceding job_user_pause.


in comment: job_user_puase_locked


Please also fix other removed function mentioning in comments, for example I 
see in comments mentioning of removed job_ref, job_unref, job_user_pause...


- */
-void job_user_resume(Job *job, Error **errp);
-
-/*
- * Same as job_user_resume(), but called with job lock held.
- * Might release the lock temporarily.
+ * Called with job lock held, but might release it temporarily.
   */
  void job_user_resume_locked(Job *job, Error **errp);
  
@@ -624,6 +611,7 @@ void job_user_resume_locked(Job *job, Error **errp);

   * first one if @job is %NULL.


[..]


  /**
@@ -725,9 +703,6 @@ void job_cancel_sync_all(void);
   * Returns the return value from the job.
   * Called with job_lock *not* held.


in comment: with lock held.


   */
-int job_complete_sync(Job *job, Error **errp);
-
-/* Same as job_complete_sync, but called with job lock held. */
  int job_complete_sync_locked(Job *job, Error **errp);
  




--
Best regards,
Vladimir



Re: [PATCH v3 03/20] ppc4xx_sdram: Get rid of the init RAM hack

2022-09-14 Thread Cédric Le Goater

On 9/14/22 13:44, BALATON Zoltan wrote:

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/13/22 21:52, BALATON Zoltan wrote:

The do_init parameter of ppc4xx_sdram_init() is used to map memory
regions that is normally done by the firmware by programming the SDRAM
controller. This is needed when booting a kernel directly from -kernel
without a firmware. Do this from board code accesing normal SDRAM


accessing


Fixed, also two ofhers in another patch you haven't noticed.


controller registers the same way as firmware would do, so we can get
rid of this hack.

Signed-off-by: BALATON Zoltan 
---
v2: Fix ref405ep boot with -kernel and U-Boot

  hw/ppc/ppc405.h |  1 -
  hw/ppc/ppc405_boards.c  | 12 ++--
  hw/ppc/ppc405_uc.c  |  4 +---
  hw/ppc/ppc440_bamboo.c  |  8 +++-
  hw/ppc/ppc440_uc.c  |  2 --
  hw/ppc/ppc4xx_devs.c    | 11 +--
  include/hw/ppc/ppc4xx.h |  8 ++--
  7 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/hw/ppc/ppc405.h b/hw/ppc/ppc405.h
index 1e558c7831..756865621b 100644
--- a/hw/ppc/ppc405.h
+++ b/hw/ppc/ppc405.h
@@ -169,7 +169,6 @@ struct Ppc405SoCState {
  /* Public */
  MemoryRegion ram_banks[2];
  hwaddr ram_bases[2], ram_sizes[2];
-    bool do_dram_init;
    MemoryRegion *dram_mr;
  hwaddr ram_size;
diff --git a/hw/ppc/ppc405_boards.c b/hw/ppc/ppc405_boards.c
index 083f12b23e..bf02a71c6d 100644
--- a/hw/ppc/ppc405_boards.c
+++ b/hw/ppc/ppc405_boards.c
@@ -274,6 +274,7 @@ static void ppc405_init(MachineState *machine)
  MachineClass *mc = MACHINE_GET_CLASS(machine);
  const char *kernel_filename = machine->kernel_filename;
  MemoryRegion *sysmem = get_system_memory();
+    CPUPPCState *env;
    if (machine->ram_size != mc->default_ram_size) {
  char *sz = size_to_str(mc->default_ram_size);
@@ -288,12 +289,19 @@ static void ppc405_init(MachineState *machine)
   machine->ram_size, _fatal);
  object_property_set_link(OBJECT(>soc), "dram",
   OBJECT(machine->ram), _abort);
-    object_property_set_bool(OBJECT(>soc), "dram-init",
- kernel_filename != NULL, _abort);
  object_property_set_uint(OBJECT(>soc), "sys-clk", ,
   _abort);
  qdev_realize(DEVICE(>soc), NULL, _fatal);
  +    /* Enable SDRAM memory regions */
+    /* FIXME This shouldn't be needed with firmware but we lack SPD data */


what do you mean ?


U-Boot detects the available RAM by reading the SPD info of the RAM modules but 
that probably also needs i2c emulation. See sam460ex.


+    env = >soc.cpu.env;
+    if (ppc_dcr_write(env->dcr_env, SDRAM0_CFGADDR, 0x20) ||
+    ppc_dcr_write(env->dcr_env, SDRAM0_CFGDATA, 0x8000)) {



I am not in favor of these ppc_drc_write calls and this is still a hack.


It's not. Normally this is done by firmware to enable memory controller but the 
board code has to do it if not using firmware (e.g. booting with -kernel) the 
same way it provides bootinfo or device tree mods the firmware would normally 
do or in this case maybe the emulation is incomplete so the part of firmware 
that configures the SDRAM controller does not run.


Exactly, and what the above proposal does is mimicking execution of CPU
instructions before the CPU is even fully initiated. Reset has not been
called at that stage.




The "dram-init" property is a cleaner solution. It takes care of doing the
pre-mapping of RAM banks in the realize routine of the sdram model (when
available).


I disagree, the hardware does not have such feature, it proviesd DCRs as the way to configure it. Adding a special property for it deviates from hardware and clutters qtree. 



In this machine, running QEMU with -kernel deviates from HW. That's
the whole purpose of this option. It assumes that the SDRAM device
is pre-initialized (and much more should be done) by the QEMU machine
and the simplest way to acheive this goal is to inform the SDRAM model
to take care of the pre-initialization.

Another way would be to change the default reset values of the SDRAM
registers (in the realize method using some property) and perform
some actions (mapping the banks) in the reset handler of the SDRAM
device model. That would be a deferred initialization but a property
is still needed to change the default behavior of the SDRAM model.

Anyhow, this should be isolated under the SDRAM device model and
not in the machine init by using the CPU state. That's clearly ugly.

Thanks,

C.






Doing it like this patch is cleaner IMO.

Regards,
BALATON Zoltan



C.


+    error_report("Could not enable memory regions");
+    exit(1);
+    }
+
  /* allocate and load BIOS */
  if (machine->firmware) {
  MemoryRegion *bios = g_new(MemoryRegion, 1);
diff --git a/hw/ppc/ppc405_uc.c b/hw/ppc/ppc405_uc.c
index 2ca42fdef6..1e02347e57 100644
--- a/hw/ppc/ppc405_uc.c
+++ b/hw/ppc/ppc405_uc.c
@@ -1081,8 +1081,7 

[PULL 11/11] target/riscv: Honour -semihosting-config userspace=on and enable=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

The riscv target incorrectly enabled semihosting always, whether the
user asked for it or not.  Call semihosting_enabled() passing the
correct value to the is_userspace argument, which fixes this and also
handles the userspace=on argument.  Because we do this at translate
time, we no longer need to check the privilege level in
riscv_cpu_do_interrupt().

Note that this is a behaviour change: we used to default to
semihosting being enabled, and now the user must pass
"-semihosting-config enable=on" if they want it.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Reviewed-by: Alistair Francis 
Message-Id: <20220822141230.3658237-8-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/riscv/cpu_helper.c  | 9 +++--
 target/riscv/translate.c   | 1 +
 target/riscv/insn_trans/trans_privileged.c.inc | 3 ++-
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c
index 67e4c0efd2..278d163803 100644
--- a/target/riscv/cpu_helper.c
+++ b/target/riscv/cpu_helper.c
@@ -1589,12 +1589,9 @@ void riscv_cpu_do_interrupt(CPUState *cs)
 target_ulong mtval2 = 0;
 
 if  (cause == RISCV_EXCP_SEMIHOST) {
-if (env->priv >= PRV_S) {
-do_common_semihosting(cs);
-env->pc += 4;
-return;
-}
-cause = RISCV_EXCP_BREAKPOINT;
+do_common_semihosting(cs);
+env->pc += 4;
+return;
 }
 
 if (!async) {
diff --git a/target/riscv/translate.c b/target/riscv/translate.c
index 8925a44c6e..db123da5ec 100644
--- a/target/riscv/translate.c
+++ b/target/riscv/translate.c
@@ -28,6 +28,7 @@
 
 #include "exec/translator.h"
 #include "exec/log.h"
+#include "semihosting/semihost.h"
 
 #include "instmap.h"
 #include "internals.h"
diff --git a/target/riscv/insn_trans/trans_privileged.c.inc 
b/target/riscv/insn_trans/trans_privileged.c.inc
index 46f96ad74d..3281408a87 100644
--- a/target/riscv/insn_trans/trans_privileged.c.inc
+++ b/target/riscv/insn_trans/trans_privileged.c.inc
@@ -52,7 +52,8 @@ static bool trans_ebreak(DisasContext *ctx, arg_ebreak *a)
  * that no exception will be raised when fetching them.
  */
 
-if ((pre_addr & TARGET_PAGE_MASK) == (post_addr & TARGET_PAGE_MASK)) {
+if (semihosting_enabled(ctx->mem_idx < PRV_S) &&
+(pre_addr & TARGET_PAGE_MASK) == (post_addr & TARGET_PAGE_MASK)) {
 pre= opcode_at(>base, pre_addr);
 ebreak = opcode_at(>base, ebreak_addr);
 post   = opcode_at(>base, post_addr);
-- 
2.34.1




Re: [PATCH v11 13/21] jobs: protect job.aio_context with BQL and job_mutex

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:20, Emanuele Giuseppe Esposito wrote:

In order to make it thread safe, implement a "fake rwlock",
where we allow reads under BQL *or* job_mutex held, but
writes only under BQL *and* job_mutex.

The only write we have is in child_job_set_aio_ctx, which always
happens under drain (so the job is paused).
For this reason, introduce job_set_aio_context and make sure that
the context is set under BQL, job_mutex and drain.
Also make sure all other places where the aiocontext is read
are protected.

The reads in commit.c and mirror.c are actually safe, because always
done under BQL.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Suggested-by: Paolo Bonzini 
Signed-off-by: Emanuele Giuseppe Esposito 
---
  block/replication.c |  7 +--
  blockjob.c  |  3 ++-
  include/qemu/job.h  | 23 ---
  job.c   | 12 
  4 files changed, 39 insertions(+), 6 deletions(-)

diff --git a/block/replication.c b/block/replication.c
index 55c8f894aa..6e02d98126 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -142,14 +142,17 @@ static void replication_close(BlockDriverState *bs)
  {
  BDRVReplicationState *s = bs->opaque;
  Job *commit_job;
+GLOBAL_STATE_CODE();
  
  if (s->stage == BLOCK_REPLICATION_RUNNING) {

  replication_stop(s->rs, false, NULL);
  }
  if (s->stage == BLOCK_REPLICATION_FAILOVER) {
  commit_job = >commit_job->job;
-assert(commit_job->aio_context == qemu_get_current_aio_context());
-job_cancel_sync(commit_job, false);
+WITH_JOB_LOCK_GUARD() {
+assert(commit_job->aio_context == qemu_get_current_aio_context());
+job_cancel_sync_locked(commit_job, false);
+}


As Kevin said, this hunk seems not needed.. Why to add locking for reading 
aio_context, when we have GLOBAL_STATE_CODE()?


  }
  
  if (s->mode == REPLICATION_MODE_SECONDARY) {

diff --git a/blockjob.c b/blockjob.c
index 96fb9d9f73..c8919cef9b 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -162,12 +162,13 @@ static void child_job_set_aio_ctx(BdrvChild *c, 
AioContext *ctx,
  bdrv_set_aio_context_ignore(sibling->bs, ctx, ignore);
  }
  
-job->job.aio_context = ctx;

+job_set_aio_context(>job, ctx);
  }
  
  static AioContext *child_job_get_parent_aio_context(BdrvChild *c)

  {
  BlockJob *job = c->opaque;
+GLOBAL_STATE_CODE();
  
  return job->job.aio_context;

  }
diff --git a/include/qemu/job.h b/include/qemu/job.h
index 5709e8d4a8..cede227e67 100644
--- a/include/qemu/job.h
+++ b/include/qemu/job.h
@@ -74,11 +74,17 @@ typedef struct Job {
  /* ProgressMeter API is thread-safe */
  ProgressMeter progress;
  
+/**

+ * AioContext to run the job coroutine in.
+ * The job Aiocontext can be read when holding *either*
+ * the BQL (so we are in the main loop) or the job_mutex.
+ * It can only be written when we hold *both* BQL
+ * and the job_mutex.
+ */
+AioContext *aio_context;
  
-/** Protected by AioContext lock */
  
-/** AioContext to run the job coroutine in */

-AioContext *aio_context;
+/** Protected by AioContext lock */
  
  /** Reference count of the block job */

  int refcnt;
@@ -741,4 +747,15 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error 
**errp),
  int job_finish_sync_locked(Job *job, void (*finish)(Job *, Error **errp),
 Error **errp);
  
+/**

+ * Sets the @job->aio_context.
+ * Called with job_mutex *not* held.
+ *
+ * This function must run in the main thread to protect against
+ * concurrent read in job_finish_sync_locked(), takes the job_mutex
+ * lock to protect against the read in job_do_yield_locked(), and must
+ * be called when the coroutine is quiescent.


May be "job is quiscent" or "job is doing nothing", "no in-flight io operations in 
job".

For example, backup has several running coroutines in contest of block_copy 
process, and main coroutine of the job
is almost always "quescent"..


+ */
+void job_set_aio_context(Job *job, AioContext *ctx);
+
  #endif
diff --git a/job.c b/job.c
index 85ae843f03..9f2fb2e73b 100644
--- a/job.c
+++ b/job.c
@@ -396,6 +396,17 @@ Job *job_get(const char *id)
  return job_get_locked(id);
  }
  
+void job_set_aio_context(Job *job, AioContext *ctx)

+{
+/* protect against read in job_finish_sync_locked and job_start */
+GLOBAL_STATE_CODE();
+/* protect against read in job_do_yield_locked */
+JOB_LOCK_GUARD();
+/* ensure the coroutine is quiescent while the AioContext is changed */


same not here.


+assert(job->paused || job_is_completed_locked(job));
+job->aio_context = ctx;
+}
+
  /* Called with job_mutex *not* held. */
  static void job_sleep_timer_cb(void *opaque)
  {
@@ -1379,6 +1390,7 @@ int job_finish_sync_locked(Job *job,
  {
  Error *local_err = NULL;
  int ret;
+GLOBAL_STATE_CODE();
  
  

[PULL 06/11] target/arm: Honour -semihosting-config userspace=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Honour the commandline -semihosting-config userspace=on option,
instead of never permitting userspace semihosting calls in system
emulation mode, by passing the correct value to the is_userspace
argument of semihosting_enabled(), instead of manually checking and
always forbidding semihosting if the guest is in userspace and this
isn't the linux-user build.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-3-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/arm/translate-a64.c | 12 +---
 target/arm/translate.c | 16 
 2 files changed, 5 insertions(+), 23 deletions(-)

diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 3decc8da57..9bed336b47 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -2219,17 +2219,7 @@ static void disas_exc(DisasContext *s, uint32_t insn)
  * it is required for halting debug disabled: it will UNDEF.
  * Secondly, "HLT 0xf000" is the A64 semihosting syscall instruction.
  */
-if (semihosting_enabled(false) && imm16 == 0xf000) {
-#ifndef CONFIG_USER_ONLY
-/* In system mode, don't allow userspace access to semihosting,
- * to provide some semblance of security (and for consistency
- * with our 32-bit semihosting).
- */
-if (s->current_el == 0) {
-unallocated_encoding(s);
-break;
-}
-#endif
+if (semihosting_enabled(s->current_el == 0) && imm16 == 0xf000) {
 gen_exception_internal_insn(s, s->pc_curr, EXCP_SEMIHOST);
 } else {
 unallocated_encoding(s);
diff --git a/target/arm/translate.c b/target/arm/translate.c
index b1e013270d..5aaccbbf71 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -1169,10 +1169,7 @@ static inline void gen_hlt(DisasContext *s, int imm)
  * semihosting, to provide some semblance of security
  * (and for consistency with our 32-bit semihosting).
  */
-if (semihosting_enabled(false) &&
-#ifndef CONFIG_USER_ONLY
-s->current_el != 0 &&
-#endif
+if (semihosting_enabled(s->current_el != 0) &&
 (imm == (s->thumb ? 0x3c : 0xf000))) {
 gen_exception_internal_insn(s, s->pc_curr, EXCP_SEMIHOST);
 return;
@@ -6556,10 +6553,7 @@ static bool trans_BKPT(DisasContext *s, arg_BKPT *a)
 /* BKPT is OK with ECI set and leaves it untouched */
 s->eci_handled = true;
 if (arm_dc_feature(s, ARM_FEATURE_M) &&
-semihosting_enabled(false) &&
-#ifndef CONFIG_USER_ONLY
-!IS_USER(s) &&
-#endif
+semihosting_enabled(s->current_el == 0) &&
 (a->imm == 0xab)) {
 gen_exception_internal_insn(s, s->pc_curr, EXCP_SEMIHOST);
 } else {
@@ -8764,10 +8758,8 @@ static bool trans_SVC(DisasContext *s, arg_SVC *a)
 {
 const uint32_t semihost_imm = s->thumb ? 0xab : 0x123456;
 
-if (!arm_dc_feature(s, ARM_FEATURE_M) && semihosting_enabled(false) &&
-#ifndef CONFIG_USER_ONLY
-!IS_USER(s) &&
-#endif
+if (!arm_dc_feature(s, ARM_FEATURE_M) &&
+semihosting_enabled(s->current_el == 0) &&
 (a->imm == semihost_imm)) {
 gen_exception_internal_insn(s, s->pc_curr, EXCP_SEMIHOST);
 } else {
-- 
2.34.1




[PULL 03/11] target/m68k: Use semihosting/syscalls.h

2022-09-14 Thread Richard Henderson
This separates guest file descriptors from host file descriptors,
and utilizes shared infrastructure for integration with gdbstub.

Acked-by: Laurent Vivier 
Signed-off-by: Richard Henderson 
---
 target/m68k/m68k-semi.c | 281 +++-
 1 file changed, 49 insertions(+), 232 deletions(-)

diff --git a/target/m68k/m68k-semi.c b/target/m68k/m68k-semi.c
index d0697ddbd1..586a801034 100644
--- a/target/m68k/m68k-semi.c
+++ b/target/m68k/m68k-semi.c
@@ -21,6 +21,7 @@
 
 #include "cpu.h"
 #include "exec/gdbstub.h"
+#include "semihosting/syscalls.h"
 #include "semihosting/softmmu-uaccess.h"
 #include "hw/boards.h"
 #include "qemu/log.h"
@@ -40,56 +41,6 @@
 #define HOSTED_ISATTY 12
 #define HOSTED_SYSTEM 13
 
-static int translate_openflags(int flags)
-{
-int hf;
-
-if (flags & GDB_O_WRONLY)
-hf = O_WRONLY;
-else if (flags & GDB_O_RDWR)
-hf = O_RDWR;
-else
-hf = O_RDONLY;
-
-if (flags & GDB_O_APPEND) hf |= O_APPEND;
-if (flags & GDB_O_CREAT) hf |= O_CREAT;
-if (flags & GDB_O_TRUNC) hf |= O_TRUNC;
-if (flags & GDB_O_EXCL) hf |= O_EXCL;
-
-return hf;
-}
-
-static void translate_stat(CPUM68KState *env, target_ulong addr, struct stat 
*s)
-{
-struct gdb_stat *p;
-
-p = lock_user(VERIFY_WRITE, addr, sizeof(struct gdb_stat), 0);
-if (!p) {
-/* FIXME - should this return an error code? */
-return;
-}
-p->gdb_st_dev = cpu_to_be32(s->st_dev);
-p->gdb_st_ino = cpu_to_be32(s->st_ino);
-p->gdb_st_mode = cpu_to_be32(s->st_mode);
-p->gdb_st_nlink = cpu_to_be32(s->st_nlink);
-p->gdb_st_uid = cpu_to_be32(s->st_uid);
-p->gdb_st_gid = cpu_to_be32(s->st_gid);
-p->gdb_st_rdev = cpu_to_be32(s->st_rdev);
-p->gdb_st_size = cpu_to_be64(s->st_size);
-#ifdef _WIN32
-/* Windows stat is missing some fields.  */
-p->gdb_st_blksize = 0;
-p->gdb_st_blocks = 0;
-#else
-p->gdb_st_blksize = cpu_to_be64(s->st_blksize);
-p->gdb_st_blocks = cpu_to_be64(s->st_blocks);
-#endif
-p->gdb_st_atime = cpu_to_be32(s->st_atime);
-p->gdb_st_mtime = cpu_to_be32(s->st_mtime);
-p->gdb_st_ctime = cpu_to_be32(s->st_ctime);
-unlock_user(p, addr, sizeof(struct gdb_stat));
-}
-
 static void m68k_semi_u32_cb(CPUState *cs, uint64_t ret, int err)
 {
 M68kCPU *cpu = M68K_CPU(cs);
@@ -129,248 +80,109 @@ static void m68k_semi_u64_cb(CPUState *cs, uint64_t ret, 
int err)
  */
 #define GET_ARG(n) do { \
 if (get_user_ual(arg ## n, args + (n) * 4)) {   \
-result = -1;\
-errno = EFAULT; \
 goto failed;\
 }   \
 } while (0)
 
+#define GET_ARG64(n) do {   \
+if (get_user_ual(arg ## n, args + (n) * 4)) {   \
+goto failed64;  \
+}   \
+} while (0)
+
+
 void do_m68k_semihosting(CPUM68KState *env, int nr)
 {
 CPUState *cs = env_cpu(env);
 uint32_t args;
 target_ulong arg0, arg1, arg2, arg3;
-void *p;
-void *q;
-uint32_t len;
-uint32_t result;
 
 args = env->dregs[1];
 switch (nr) {
 case HOSTED_EXIT:
 gdb_exit(env->dregs[0]);
 exit(env->dregs[0]);
+
 case HOSTED_OPEN:
 GET_ARG(0);
 GET_ARG(1);
 GET_ARG(2);
 GET_ARG(3);
-if (use_gdb_syscalls()) {
-gdb_do_syscall(m68k_semi_u32_cb, "open,%s,%x,%x", arg0, (int)arg1,
-   arg2, arg3);
-return;
-} else {
-p = lock_user_string(arg0);
-if (!p) {
-/* FIXME - check error code? */
-result = -1;
-} else {
-result = open(p, translate_openflags(arg2), arg3);
-unlock_user(p, arg0, 0);
-}
-}
+semihost_sys_open(cs, m68k_semi_u32_cb, arg0, arg1, arg2, arg3);
 break;
+
 case HOSTED_CLOSE:
-{
-/* Ignore attempts to close stdin/out/err.  */
-GET_ARG(0);
-int fd = arg0;
-if (fd > 2) {
-if (use_gdb_syscalls()) {
-gdb_do_syscall(m68k_semi_u32_cb, "close,%x", arg0);
-return;
-} else {
-result = close(fd);
-}
-} else {
-result = 0;
-}
-break;
-}
+GET_ARG(0);
+semihost_sys_close(cs, m68k_semi_u32_cb, arg0);
+break;
+
 case HOSTED_READ:
 GET_ARG(0);
 GET_ARG(1);
 GET_ARG(2);
-len = arg2;
-if (use_gdb_syscalls()) {
-gdb_do_syscall(m68k_semi_u32_cb, "read,%x,%x,%x",
-   arg0, arg1, len);
-return;
-} 

[PULL 09/11] target/nios2: Honour -semihosting-config userspace=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Honour the commandline -semihosting-config userspace=on option,
instead of always permitting userspace semihosting calls in system
emulation mode, by passing the correct value to the is_userspace
argument of semihosting_enabled().

Note that this is a behaviour change: if the user wants to
do semihosting calls from userspace they must now specifically
enable them on the command line.

nios2 semihosting is not implemented for linux-user builds.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-6-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/nios2/translate.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/nios2/translate.c b/target/nios2/translate.c
index ff631a42f6..8dc0a32c6c 100644
--- a/target/nios2/translate.c
+++ b/target/nios2/translate.c
@@ -817,8 +817,9 @@ static void gen_break(DisasContext *dc, uint32_t code, 
uint32_t flags)
 {
 #ifndef CONFIG_USER_ONLY
 /* The semihosting instruction is "break 1".  */
+bool is_user = FIELD_EX32(dc->tb_flags, TBFLAGS, U);
 R_TYPE(instr, code);
-if (semihosting_enabled(false) && instr.imm5 == 1) {
+if (semihosting_enabled(is_user) && instr.imm5 == 1) {
 t_gen_helper_raise_exception(dc, EXCP_SEMIHOST);
 return;
 }
-- 
2.34.1




[PULL 00/11] semihosting patch queue

2022-09-14 Thread Richard Henderson
The following changes since commit 79dfa177ae348bb5ab5f97c0915359b13d6186e2:

  Merge tag 'pull-qapi-2022-09-07' of git://repo.or.cz/qemu/armbru into staging 
(2022-09-07 13:13:30 -0400)

are available in the Git repository at:

  https://gitlab.com/rth7680/qemu.git tags/pull-semi-20220914

for you to fetch changes up to 7d7fb11615809839ff858328134c6a0abad27ea4:

  target/riscv: Honour -semihosting-config userspace=on and enable=on 
(2022-09-13 17:18:21 +0100)


Convert m68k to semihosting/syscalls.h.
Convert nios2 to semihosting/syscalls.h.
Allow optional use of semihosting from userspace.


Peter Maydell (7):
  semihosting: Allow optional use of semihosting from userspace
  target/arm: Honour -semihosting-config userspace=on
  target/m68k: Honour -semihosting-config userspace=on
  target/mips: Honour -semihosting-config userspace=on
  target/nios2: Honour -semihosting-config userspace=on
  target/xtensa: Honour -semihosting-config userspace=on
  target/riscv: Honour -semihosting-config userspace=on and enable=on

Richard Henderson (4):
  target/nios2: Use semihosting/syscalls.h
  target/nios2: Convert semihosting errno to gdb remote errno
  target/m68k: Use semihosting/syscalls.h
  target/m68k: Convert semihosting errno to gdb remote errno

 include/semihosting/semihost.h |  10 +-
 semihosting/config.c   |  10 +-
 softmmu/vl.c   |   2 +-
 stubs/semihost.c   |   2 +-
 target/arm/translate-a64.c |  12 +-
 target/arm/translate.c |  16 +-
 target/m68k/m68k-semi.c| 306 ++-
 target/m68k/op_helper.c|   3 +-
 target/mips/tcg/translate.c|   9 +-
 target/nios2/nios2-semi.c  | 321 ++---
 target/nios2/translate.c   |   3 +-
 target/riscv/cpu_helper.c  |   9 +-
 target/riscv/translate.c   |   1 +
 target/xtensa/translate.c  |   7 +-
 target/mips/tcg/micromips_translate.c.inc  |   6 +-
 target/mips/tcg/mips16e_translate.c.inc|   2 +-
 target/mips/tcg/nanomips_translate.c.inc   |   4 +-
 target/riscv/insn_trans/trans_privileged.c.inc |   3 +-
 qemu-options.hx|  11 +-
 19 files changed, 209 insertions(+), 528 deletions(-)



[PULL 02/11] target/nios2: Convert semihosting errno to gdb remote errno

2022-09-14 Thread Richard Henderson
The semihosting abi used by nios2 uses the gdb remote
protocol filesys errnos.

Signed-off-by: Richard Henderson 
---
 target/nios2/nios2-semi.c | 33 +++--
 1 file changed, 31 insertions(+), 2 deletions(-)

diff --git a/target/nios2/nios2-semi.c b/target/nios2/nios2-semi.c
index 614fd76695..f76e8588c5 100644
--- a/target/nios2/nios2-semi.c
+++ b/target/nios2/nios2-semi.c
@@ -43,6 +43,35 @@
 #define HOSTED_ISATTY 12
 #define HOSTED_SYSTEM 13
 
+static int host_to_gdb_errno(int err)
+{
+#define E(X)  case E##X: return GDB_E##X
+switch (err) {
+E(PERM);
+E(NOENT);
+E(INTR);
+E(BADF);
+E(ACCES);
+E(FAULT);
+E(BUSY);
+E(EXIST);
+E(NODEV);
+E(NOTDIR);
+E(ISDIR);
+E(INVAL);
+E(NFILE);
+E(MFILE);
+E(FBIG);
+E(NOSPC);
+E(SPIPE);
+E(ROFS);
+E(NAMETOOLONG);
+default:
+return GDB_EUNKNOWN;
+}
+#undef E
+}
+
 static void nios2_semi_u32_cb(CPUState *cs, uint64_t ret, int err)
 {
 Nios2CPU *cpu = NIOS2_CPU(cs);
@@ -50,7 +79,7 @@ static void nios2_semi_u32_cb(CPUState *cs, uint64_t ret, int 
err)
 target_ulong args = env->regs[R_ARG1];
 
 if (put_user_u32(ret, args) ||
-put_user_u32(err, args + 4)) {
+put_user_u32(host_to_gdb_errno(err), args + 4)) {
 /*
  * The nios2 semihosting ABI does not provide any way to report this
  * error to the guest, so the best we can do is log it in qemu.
@@ -69,7 +98,7 @@ static void nios2_semi_u64_cb(CPUState *cs, uint64_t ret, int 
err)
 
 if (put_user_u32(ret >> 32, args) ||
 put_user_u32(ret, args + 4) ||
-put_user_u32(err, args + 8)) {
+put_user_u32(host_to_gdb_errno(err), args + 8)) {
 /* No way to report this via nios2 semihosting ABI; just log it */
 qemu_log_mask(LOG_GUEST_ERROR, "nios2-semihosting: return value "
   "discarded because argument block not writable\n");
-- 
2.34.1




Re: [PATCH v11 17/21] job.h: categorize JobDriver callbacks that need the AioContext lock

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:21, Emanuele Giuseppe Esposito wrote:

Some callbacks implementation use bdrv_* APIs that assume the
AioContext lock is held. Make sure this invariant is documented.

Signed-off-by: Emanuele Giuseppe Esposito



Reviewed-by: Vladimir Sementsov-Ogievskiy 

--
Best regards,
Vladimir



[PATCH v2 4/4] tests/docker: run script use realpath instead of readlink

2022-09-14 Thread Lucas Mateus Castro(alqotel)
From: "Lucas Mateus Castro (alqotel)" 

The alpine docker image only comes with busybox, which doesn't have the
'-e' option on its readlink, so change it to 'realpath' to avoid that
problem.

Suggested-by: Daniel P. Berrangé 
Signed-off-by: Lucas Mateus Castro (alqotel) 
---
 tests/docker/run | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/run b/tests/docker/run
index 421393046b..9eb96129da 100755
--- a/tests/docker/run
+++ b/tests/docker/run
@@ -15,7 +15,7 @@ if test -n "$V"; then
 set -x
 fi
 
-BASE="$(dirname $(readlink -e $0))"
+BASE="$(dirname $(realpath $0))"
 
 # Prepare the environment
 export PATH=/usr/lib/ccache:/usr/lib64/ccache:$PATH
-- 
2.31.1




Re: [PATCH v11 16/21] blockjob: protect iostatus field in BlockJob struct

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:20, Emanuele Giuseppe Esposito wrote:

iostatus is the only field (together with .job) that needs
protection using the job mutex.

It is set in the main loop (GLOBAL_STATE functions) but read
in I/O code (block_job_error_action).

In order to protect it, change block_job_iostatus_set_err
to block_job_iostatus_set_err_locked(), always called under
job lock.

Signed-off-by: Emanuele Giuseppe Esposito
Reviewed-by: Kevin Wolf


Reviewed-by: Vladimir Sementsov-Ogievskiy 

--
Best regards,
Vladimir



Re: [PATCH 2/3] vdpa: load vlan configuration at NIC startup

2022-09-14 Thread Eugenio Perez Martin
On Wed, Sep 14, 2022 at 1:33 PM Si-Wei Liu  wrote:
>
>
>
> On 9/14/2022 3:20 AM, Jason Wang wrote:
> > On Fri, Sep 9, 2022 at 4:02 PM Eugenio Perez Martin  
> > wrote:
> >> On Fri, Sep 9, 2022 at 8:40 AM Jason Wang  wrote:
> >>> On Fri, Sep 9, 2022 at 2:38 PM Jason Wang  wrote:
>  On Wed, Sep 7, 2022 at 12:36 AM Eugenio Pérez  
>  wrote:
> > To have enabled vlans at device startup may happen in the destination of
> > a live migration, so this configuration must be restored.
> >
> > At this moment the code is not accessible, since SVQ refuses to start if
> > vlan feature is exposed by the device.
> >
> > Signed-off-by: Eugenio Pérez 
> > ---
> >   net/vhost-vdpa.c | 46 --
> >   1 file changed, 44 insertions(+), 2 deletions(-)
> >
> > diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
> > index 4bc3fd01a8..ecbfd08eb9 100644
> > --- a/net/vhost-vdpa.c
> > +++ b/net/vhost-vdpa.c
> > @@ -100,6 +100,8 @@ static const uint64_t vdpa_svq_device_features =
> >   BIT_ULL(VIRTIO_NET_F_RSC_EXT) |
> >   BIT_ULL(VIRTIO_NET_F_STANDBY);
> >
> > +#define MAX_VLAN(1 << 12)   /* Per 802.1Q definition */
> > +
> >   VHostNetState *vhost_vdpa_get_vhost_net(NetClientState *nc)
> >   {
> >   VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc);
> > @@ -423,6 +425,47 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState 
> > *s,
> >   return *s->status != VIRTIO_NET_OK;
> >   }
> >
> > +static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s,
> > +   const VirtIONet *n,
> > +   uint16_t vid)
> > +{
> > +ssize_t dev_written = vhost_vdpa_net_load_cmd(s, 
> > VIRTIO_NET_CTRL_VLAN,
> > +  
> > VIRTIO_NET_CTRL_VLAN_ADD,
> > +  , sizeof(vid));
> > +if (unlikely(dev_written < 0)) {
> > +return dev_written;
> > +}
> > +
> > +if (unlikely(*s->status != VIRTIO_NET_OK)) {
> > +return -EINVAL;
> > +}
> > +
> > +return 0;
> > +}
> > +
> > +static int vhost_vdpa_net_load_vlan(VhostVDPAState *s,
> > +const VirtIONet *n)
> > +{
> > +uint64_t features = n->parent_obj.guest_features;
> > +
> > +if (!(features & BIT_ULL(VIRTIO_NET_F_CTRL_VLAN))) {
> > +return 0;
> > +}
> > +
> > +for (int i = 0; i < MAX_VLAN >> 5; i++) {
> > +for (int j = 0; n->vlans[i] && j <= 0x1f; j++) {
> > +if (n->vlans[i] & (1U << j)) {
> > +int r = vhost_vdpa_net_load_single_vlan(s, n, (i << 5) 
> > + j);
>  This seems to cause a lot of latency if the driver has a lot of vlans.
> 
>  I wonder if it's simply to let all vlan traffic go by disabling
>  CTRL_VLAN feature at vDPA layer.
> >> The guest will not be able to recover that vlan configuration.
> > Spec said it's best effort and we actually don't do this for
> > vhost-net/user for years and manage to survive.
> I thought there's a border line between best effort and do nothing. ;-)
>

I also think it is worth at least trying to migrate it for sure. For
MAC there may be too many entries, but VLAN should be totally
manageable (and the information is already there, the bitmap is
actually being migrated).

> I think that the reason this could survive is really software
> implementation specific - say if the backend doesn't start with VLAN
> filtering disabled (meaning allow all VLAN traffic to pass) then it
> would become a problem. This won't be a problem for almost PF NICs but
> may be problematic for vDPA e.g. VF backed VDPAs.

I'd say the driver would expect all vlan filters to be cleared after a
reset, isn't it? The spec doesn't explicitly say anything about that
as far as I see.

> >
> >>> Another idea is to extend the spec to allow us to accept a bitmap of
> >>> the vlan ids via a single command, then we will be fine.
> >>>
> >> I'm not sure if adding more ways to configure something is the answer,
> >> but I'd be ok to implement it.
> >>
> >> Another idea is to allow the sending of many CVQ commands in parallel.
> >> It shouldn't be very hard to achieve using exposed buffers as ring
> >> buffers, and it will short down the start of the devices with many
> >> features.
> > In the extreme case, what if guests decide to filter all of the vlans?
> > It means we need MAX_VLAN commands which may exceeds the size of the
> > control virtqueue.
> Indeed, that's a case where a single flat device state blob would be
> more efficient for hardware drivers to apply than individual control
> command messages do.
>

If we're going that route, being able to set a bitmap for vlan
filtering 

Re: [PATCH v11 11/21] jobs: group together API calls under the same job lock

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:20, Emanuele Giuseppe Esposito wrote:

Now that the API offers also _locked() functions, take advantage
of it and give also the caller control to take the lock and call
_locked functions.

This makes sense especially when we have for loops, because it
makes no sense to have:

for(job = job_next(); ...)

where each job_next() takes the lock internally.
Instead we want

JOB_LOCK_GUARD();
for(job = job_next_locked(); ...)

In addition, protect also direct field accesses, by either creating a
new critical section or widening the existing ones.

Note: at this stage, job_{lock/unlock} and job lock guard macros
are *nop*.

Signed-off-by: Emanuele Giuseppe Esposito 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
  block.c| 17 ++---
  blockdev.c | 14 ++
  blockjob.c | 35 ++-
  job-qmp.c  |  9 ++---
  job.c  |  7 +--
  monitor/qmp-cmds.c |  7 +--
  qemu-img.c | 17 +++--
  7 files changed, 69 insertions(+), 37 deletions(-)



[..]


diff --git a/job.c b/job.c
index dcd561fd46..e336af0c1c 100644
--- a/job.c
+++ b/job.c


job.c hunks are unrelated in this patch, they should go to patch 05.


@@ -672,7 +672,7 @@ void coroutine_fn job_pause_point(Job *job)
  job_pause_point_locked(job);
  }
  
-void job_yield_locked(Job *job)

+static void job_yield_locked(Job *job)
  {
  assert(job->busy);
  
@@ -1046,11 +1046,14 @@ static void job_completed_txn_abort_locked(Job *job)

  /* Called with job_mutex held, but releases it temporarily */
  static int job_prepare_locked(Job *job)
  {
+int ret;
+
  GLOBAL_STATE_CODE();
  if (job->ret == 0 && job->driver->prepare) {
  job_unlock();
-job->ret = job->driver->prepare(job);
+ret = job->driver->prepare(job);
  job_lock();
+job->ret = ret;
  job_update_rc_locked(job);
  }
  return job->ret;
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 7314cd813d..81c8fdadf8 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -135,8 +135,11 @@ void qmp_cont(Error **errp)
  blk_iostatus_reset(blk);
  }
  
-for (job = block_job_next(NULL); job; job = block_job_next(job)) {

-block_job_iostatus_reset(job);
+WITH_JOB_LOCK_GUARD() {
+for (job = block_job_next_locked(NULL); job;
+ job = block_job_next_locked(job)) {
+block_job_iostatus_reset_locked(job);
+}
  }
  
  /* Continuing after completed migration. Images have been inactivated to

diff --git a/qemu-img.c b/qemu-img.c
index 7d4b33b3da..c8ae704166 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -912,25 +912,30 @@ static void run_block_job(BlockJob *job, Error **errp)
  int ret = 0;
  
  aio_context_acquire(aio_context);

-job_ref(>job);
+job_lock();
+job_ref_locked(>job);
  do {
  float progress = 0.0f;
+job_unlock();
  aio_poll(aio_context, true);
  
  progress_get_snapshot(>job.progress, _current,

-  _total);
+_total);


broken indentation


  if (progress_total) {
  progress = (float)progress_current / progress_total * 100.f;
  }
  qemu_progress_print(progress, 0);
-} while (!job_is_ready(>job) && !job_is_completed(>job));
+job_lock();
+} while (!job_is_ready_locked(>job) &&
+!job_is_completed_locked(>job));


and here

  
-if (!job_is_completed(>job)) {

-ret = job_complete_sync(>job, errp);
+if (!job_is_completed_locked(>job)) {
+ret = job_complete_sync_locked(>job, errp);
  } else {
  ret = job->job.ret;
  }
-job_unref(>job);
+job_unref_locked(>job);
+job_unlock();
  aio_context_release(aio_context);
  
  /* publish completion progress only when success */



--
Best regards,
Vladimir



[PULL 10/11] target/xtensa: Honour -semihosting-config userspace=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Honour the commandline -semihosting-config userspace=on option,
instead of always permitting userspace semihosting calls in system
emulation mode, by passing the correct value to the is_userspace
argument of semihosting_enabled().

Note that this is a behaviour change: if the user wants to
do semihosting calls from userspace they must now specifically
enable them on the command line.

xtensa semihosting is not implemented for linux-user builds.

Signed-off-by: Peter Maydell 
Acked-by: Max Filippov 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-7-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/xtensa/translate.c | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index afae8a1bea..bdd4690a5c 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -2362,13 +2362,14 @@ static uint32_t test_exceptions_simcall(DisasContext 
*dc,
 const OpcodeArg arg[],
 const uint32_t par[])
 {
+bool is_semi = semihosting_enabled(dc->cring != 0);
 #ifdef CONFIG_USER_ONLY
 bool ill = true;
 #else
 /* Between RE.2 and RE.3 simcall opcode's become nop for the hardware. */
-bool ill = dc->config->hw_version <= 250002 && !semihosting_enabled(false);
+bool ill = dc->config->hw_version <= 250002 && !is_semi;
 #endif
-if (ill || !semihosting_enabled(false)) {
+if (ill || !is_semi) {
 qemu_log_mask(LOG_GUEST_ERROR, "SIMCALL but semihosting is 
disabled\n");
 }
 return ill ? XTENSA_OP_ILL : 0;
@@ -2378,7 +2379,7 @@ static void translate_simcall(DisasContext *dc, const 
OpcodeArg arg[],
   const uint32_t par[])
 {
 #ifndef CONFIG_USER_ONLY
-if (semihosting_enabled(false)) {
+if (semihosting_enabled(dc->cring != 0)) {
 gen_helper_simcall(cpu_env);
 }
 #endif
-- 
2.34.1




[PATCH v2 2/4] scripts/ci/setup: Fix libxen requirements

2022-09-14 Thread Lucas Mateus Castro(alqotel)
From: "Lucas Mateus Castro (alqotel)" 

XEN hypervisor is only available in ARM and x86, but the yaml only
checked if the architecture is different from s390x, changed it to
a more accurate test.
Tested this change on a Ubuntu 20.04 ppc64le.

Signed-off-by: Lucas Mateus Castro (alqotel) 
Reviewed-by: Alex Bennée 
---
 scripts/ci/setup/build-environment.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/setup/build-environment.yml 
b/scripts/ci/setup/build-environment.yml
index 6df3e61d94..7535228685 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -97,7 +97,7 @@
 state: present
   when:
 - ansible_facts['distribution'] == 'Ubuntu'
-- ansible_facts['architecture'] != 's390x'
+- ansible_facts['architecture'] == 'aarch64' or 
ansible_facts['architecture'] == 'x86_64'
 
 - name: Install basic packages to build QEMU on Ubuntu 20.04
   package:
-- 
2.31.1




Proposal for a fixed ram migration stream format for file-based migrations

2022-09-14 Thread Nikolay Borisov

Hello,

Based on several discussions I've had in the past 2 days and time spent
looking at the migration stream code I came up with the following
proposal for changes to the stream format. Let me recap what we have
right now:


...()

Where  is put only when the current page we are writing
to belongs to a different block than the last written page. My proposal is to
change this format to:

...

Each page belonging to a ramblock would be at a fixed offset in the file. Which 
is going to be
 + page_offset. The size of the region for a particular 
ramblock would be
ramblock::used_length but not the whole range is going to be written, only 
those pages which
have been dirtied. If we assume a 1tb ramblock then we'd make the the region 1 
tb but it will
be sparsely populated. This means that we can have the following layout for the 
memory range:


|---p---ppppp---p|
01tb

Each 'p' signifies an allocated page and '-' is a page which hasn't been 
dirtied ergo its
index in the stream is not touched. This of course would result in having a lot 
of holes, so when
the incoming migration starts parsing the stream it might end up in a situation
where page at offset 0 is read, but then it has to jump some at arbitrary
location for the next page. To avoid resorting to calling into the filesystem 
and dealing with
fiemap's format I also intend to extend the ramblock header format by adding a 
dirty bitmap in
which every bit would signify whether a page has been written or not, the bit's 
position would be
used to index into the allocated space for pages. Simple maths shows that for 
4k pages, 1tb can be
indexed with just 32kbytes of memory. The way the indexing would work is

page = ramblock->offset (this is the offset in the file, not the in-memory 
::ofset field) +
bit's position * target_page_size

That way we can still handle sparsely dirtied memory and have direct mapping in 
the file, this
dirty bitmap would be populated during the course of ram migration and once the 
final page is written,
those in-memory bitmaps would be written into their respective positions in the 
file, needless to say
this would be supported only on seekable channels (for now only file-based 
channels).

Another important thing worth mentioning is I don't intend on incrementing the
format version but rather introducing a new migration capability.

Any thoughts/comments/suggestions, would be highly appreciated.



[PATCH v2 1/4] scripts/ci/setup: ninja missing from build-environment

2022-09-14 Thread Lucas Mateus Castro(alqotel)
From: "Lucas Mateus Castro (alqotel)" 

ninja-build is missing from the RHEL environment, so a system prepared
with that script would still fail to compile QEMU.
Tested on a Fedora 36

Signed-off-by: Lucas Mateus Castro (alqotel) 
---
 scripts/ci/setup/build-environment.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/ci/setup/build-environment.yml 
b/scripts/ci/setup/build-environment.yml
index 232525b91d..6df3e61d94 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -150,6 +150,7 @@
   - libepoxy-devel
   - libgcrypt-devel
   - lzo-devel
+  - ninja-build
   - make
   - mesa-libEGL-devel
   - nettle-devel
-- 
2.31.1




[PULL 07/11] target/m68k: Honour -semihosting-config userspace=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Honour the commandline -semihosting-config userspace=on option,
instead of never permitting userspace semihosting calls in system
emulation mode, by passing the correct value to the is_userspace
argument of semihosting_enabled(), instead of manually checking and
always forbidding semihosting if the guest is in userspace.

(Note that target/m68k doesn't support semihosting at all
in the linux-user build.)

Signed-off-by: Peter Maydell 
Reviewed-by: Laurent Vivier 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-4-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/m68k/op_helper.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/m68k/op_helper.c b/target/m68k/op_helper.c
index 4b3dfec130..a96a034050 100644
--- a/target/m68k/op_helper.c
+++ b/target/m68k/op_helper.c
@@ -203,8 +203,7 @@ static void cf_interrupt_all(CPUM68KState *env, int is_hw)
 cf_rte(env);
 return;
 case EXCP_HALT_INSN:
-if (semihosting_enabled(false)
-&& (env->sr & SR_S) != 0
+if (semihosting_enabled((env->sr & SR_S) == 0)
 && (env->pc & 3) == 0
 && cpu_lduw_code(env, env->pc - 4) == 0x4e71
 && cpu_ldl_code(env, env->pc) == 0x4e7bf000) {
-- 
2.34.1




[PULL 08/11] target/mips: Honour -semihosting-config userspace=on

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Honour the commandline -semihosting-config userspace=on option,
instead of always permitting userspace semihosting calls in system
emulation mode, by passing the correct value to the is_userspace
argument of semihosting_enabled().

Note that this is a behaviour change: if the user wants to
do semihosting calls from userspace they must now specifically
enable them on the command line.

MIPS semihosting is not implemented for linux-user builds.

Signed-off-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-5-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/mips/tcg/translate.c   | 9 +
 target/mips/tcg/micromips_translate.c.inc | 6 +++---
 target/mips/tcg/mips16e_translate.c.inc   | 2 +-
 target/mips/tcg/nanomips_translate.c.inc  | 4 ++--
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/target/mips/tcg/translate.c b/target/mips/tcg/translate.c
index 0d936e2648..c3f92ea652 100644
--- a/target/mips/tcg/translate.c
+++ b/target/mips/tcg/translate.c
@@ -12082,12 +12082,13 @@ static void gen_cache_operation(DisasContext *ctx, 
uint32_t op, int base,
 tcg_temp_free_i32(t0);
 }
 
-static inline bool is_uhi(int sdbbp_code)
+static inline bool is_uhi(DisasContext *ctx, int sdbbp_code)
 {
 #ifdef CONFIG_USER_ONLY
 return false;
 #else
-return semihosting_enabled() && sdbbp_code == 1;
+bool is_user = (ctx->hflags & MIPS_HFLAG_KSU) == MIPS_HFLAG_UM;
+return semihosting_enabled(is_user) && sdbbp_code == 1;
 #endif
 }
 
@@ -13898,7 +13899,7 @@ static void decode_opc_special_r6(CPUMIPSState *env, 
DisasContext *ctx)
 }
 break;
 case R6_OPC_SDBBP:
-if (is_uhi(extract32(ctx->opcode, 6, 20))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 6, 20))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 if (ctx->hflags & MIPS_HFLAG_SBRI) {
@@ -14310,7 +14311,7 @@ static void decode_opc_special2_legacy(CPUMIPSState 
*env, DisasContext *ctx)
 gen_cl(ctx, op1, rd, rs);
 break;
 case OPC_SDBBP:
-if (is_uhi(extract32(ctx->opcode, 6, 20))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 6, 20))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 /*
diff --git a/target/mips/tcg/micromips_translate.c.inc 
b/target/mips/tcg/micromips_translate.c.inc
index b2c696f891..632895cc9e 100644
--- a/target/mips/tcg/micromips_translate.c.inc
+++ b/target/mips/tcg/micromips_translate.c.inc
@@ -825,7 +825,7 @@ static void gen_pool16c_insn(DisasContext *ctx)
 generate_exception_break(ctx, extract32(ctx->opcode, 0, 4));
 break;
 case SDBBP16:
-if (is_uhi(extract32(ctx->opcode, 0, 4))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 0, 4))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 /*
@@ -941,7 +941,7 @@ static void gen_pool16c_r6_insn(DisasContext *ctx)
 break;
 case R6_SDBBP16:
 /* SDBBP16 */
-if (is_uhi(extract32(ctx->opcode, 6, 4))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 6, 4))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 if (ctx->hflags & MIPS_HFLAG_SBRI) {
@@ -1310,7 +1310,7 @@ static void gen_pool32axf(CPUMIPSState *env, DisasContext 
*ctx, int rt, int rs)
 generate_exception_end(ctx, EXCP_SYSCALL);
 break;
 case SDBBP:
-if (is_uhi(extract32(ctx->opcode, 16, 10))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 16, 10))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 check_insn(ctx, ISA_MIPS_R1);
diff --git a/target/mips/tcg/mips16e_translate.c.inc 
b/target/mips/tcg/mips16e_translate.c.inc
index 7568933e23..918b15d55c 100644
--- a/target/mips/tcg/mips16e_translate.c.inc
+++ b/target/mips/tcg/mips16e_translate.c.inc
@@ -951,7 +951,7 @@ static int decode_ase_mips16e(CPUMIPSState *env, 
DisasContext *ctx)
 }
 break;
 case RR_SDBBP:
-if (is_uhi(extract32(ctx->opcode, 5, 6))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 5, 6))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;
 } else {
 /*
diff --git a/target/mips/tcg/nanomips_translate.c.inc 
b/target/mips/tcg/nanomips_translate.c.inc
index b3aff22c18..812c111e3c 100644
--- a/target/mips/tcg/nanomips_translate.c.inc
+++ b/target/mips/tcg/nanomips_translate.c.inc
@@ -3694,7 +3694,7 @@ static int decode_nanomips_32_48_opc(CPUMIPSState *env, 
DisasContext *ctx)
 generate_exception_end(ctx, EXCP_BREAK);
 break;
 case NM_SDBBP:
-if (is_uhi(extract32(ctx->opcode, 0, 19))) {
+if (is_uhi(ctx, extract32(ctx->opcode, 0, 19))) {
 ctx->base.is_jmp = DISAS_SEMIHOST;

[PULL 05/11] semihosting: Allow optional use of semihosting from userspace

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Currently our semihosting implementations generally prohibit use of
semihosting calls in system emulation from the guest userspace.  This
is a very long standing behaviour justified originally "to provide
some semblance of security" (since code with access to the
semihosting ABI can do things like read and write arbitrary files on
the host system).  However, it is sometimes useful to be able to run
trusted guest code which performs semihosting calls from guest
userspace, notably for test code.  Add a command line suboption to
the existing semihosting-config option group so that you can
explicitly opt in to semihosting from guest userspace with
 -semihosting-config userspace=on

(There is no equivalent option for the user-mode emulator, because
there by definition all code runs in userspace and has access to
semihosting already.)

This commit adds the infrastructure for the command line option and
adds a bool 'is_user' parameter to the function
semihosting_userspace_enabled() that target code can use to check
whether it should be permitting the semihosting call for userspace.
It mechanically makes all the callsites pass 'false', so they
continue checking "is semihosting enabled in general".  Subsequent
commits will make each target that implements semihosting honour the
userspace=on option by passing the correct value and removing
whatever "don't do this for userspace" checking they were doing by
hand.

Signed-off-by: Peter Maydell 
Acked-by: Alex Bennée 
Reviewed-by: Alistair Francis 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 
Message-Id: <20220822141230.3658237-2-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 include/semihosting/semihost.h | 10 --
 semihosting/config.c   | 10 --
 softmmu/vl.c   |  2 +-
 stubs/semihost.c   |  2 +-
 target/arm/translate-a64.c |  2 +-
 target/arm/translate.c |  6 +++---
 target/m68k/op_helper.c|  2 +-
 target/nios2/translate.c   |  2 +-
 target/xtensa/translate.c  |  6 +++---
 qemu-options.hx| 11 +--
 10 files changed, 36 insertions(+), 17 deletions(-)

diff --git a/include/semihosting/semihost.h b/include/semihosting/semihost.h
index 93a3c21b44..efd2efa25a 100644
--- a/include/semihosting/semihost.h
+++ b/include/semihosting/semihost.h
@@ -27,7 +27,7 @@ typedef enum SemihostingTarget {
 } SemihostingTarget;
 
 #ifdef CONFIG_USER_ONLY
-static inline bool semihosting_enabled(void)
+static inline bool semihosting_enabled(bool is_user)
 {
 return true;
 }
@@ -52,7 +52,13 @@ static inline const char *semihosting_get_cmdline(void)
 return NULL;
 }
 #else /* !CONFIG_USER_ONLY */
-bool semihosting_enabled(void);
+/**
+ * semihosting_enabled:
+ * @is_user: true if guest code is in usermode (i.e. not privileged)
+ *
+ * Return true if guest code is allowed to make semihosting calls.
+ */
+bool semihosting_enabled(bool is_user);
 SemihostingTarget semihosting_get_target(void);
 const char *semihosting_get_arg(int i);
 int semihosting_get_argc(void);
diff --git a/semihosting/config.c b/semihosting/config.c
index e171d4d6bc..89a1759687 100644
--- a/semihosting/config.c
+++ b/semihosting/config.c
@@ -34,6 +34,9 @@ QemuOptsList qemu_semihosting_config_opts = {
 {
 .name = "enable",
 .type = QEMU_OPT_BOOL,
+}, {
+.name = "userspace",
+.type = QEMU_OPT_BOOL,
 }, {
 .name = "target",
 .type = QEMU_OPT_STRING,
@@ -50,6 +53,7 @@ QemuOptsList qemu_semihosting_config_opts = {
 
 typedef struct SemihostingConfig {
 bool enabled;
+bool userspace_enabled;
 SemihostingTarget target;
 char **argv;
 int argc;
@@ -59,9 +63,9 @@ typedef struct SemihostingConfig {
 static SemihostingConfig semihosting;
 static const char *semihost_chardev;
 
-bool semihosting_enabled(void)
+bool semihosting_enabled(bool is_user)
 {
-return semihosting.enabled;
+return semihosting.enabled && (!is_user || semihosting.userspace_enabled);
 }
 
 SemihostingTarget semihosting_get_target(void)
@@ -137,6 +141,8 @@ int qemu_semihosting_config_options(const char *optarg)
 if (opts != NULL) {
 semihosting.enabled = qemu_opt_get_bool(opts, "enable",
 true);
+semihosting.userspace_enabled = qemu_opt_get_bool(opts, "userspace",
+  false);
 const char *target = qemu_opt_get(opts, "target");
 /* setup of chardev is deferred until they are initialised */
 semihost_chardev = qemu_opt_get(opts, "chardev");
diff --git a/softmmu/vl.c b/softmmu/vl.c
index dea4005e47..263f029a8e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1822,7 +1822,7 @@ static void qemu_apply_machine_options(QDict *qdict)
 {
 object_set_properties_from_keyval(OBJECT(current_machine), qdict, false, 
_fatal);
 
-if 

[PATCH v2 0/4] Patch series to set up a ppc64le CI

2022-09-14 Thread Lucas Mateus Castro(alqotel)
From: "Lucas Mateus Castro (alqotel)" 

This patch series aim to make easier to set up a compilation and CI
environment on PPC64 and PPC64LE machines.

v2:
This patch series are only patches 2-4 of v1 and an alternative to patch 1
suggested by Daniel.

Lucas Mateus Castro (alqotel) (4):
  scripts/ci/setup: ninja missing from build-environment
  scripts/ci/setup: Fix libxen requirements
  scripts/ci/setup: spice-server only on x86 aarch64
  tests/docker: run script use realpath instead of readlink

 scripts/ci/setup/build-environment.yml | 15 +--
 tests/docker/run   |  2 +-
 2 files changed, 14 insertions(+), 3 deletions(-)

-- 
2.31.1




[PULL 04/11] target/m68k: Convert semihosting errno to gdb remote errno

2022-09-14 Thread Richard Henderson
The semihosting abi used by m68k uses the gdb remote
protocol filesys errnos.

Acked-by: Laurent Vivier 
Signed-off-by: Richard Henderson 
---
 target/m68k/m68k-semi.c | 33 +++--
 1 file changed, 31 insertions(+), 2 deletions(-)

diff --git a/target/m68k/m68k-semi.c b/target/m68k/m68k-semi.c
index 586a801034..87b1314925 100644
--- a/target/m68k/m68k-semi.c
+++ b/target/m68k/m68k-semi.c
@@ -41,6 +41,35 @@
 #define HOSTED_ISATTY 12
 #define HOSTED_SYSTEM 13
 
+static int host_to_gdb_errno(int err)
+{
+#define E(X)  case E##X: return GDB_E##X
+switch (err) {
+E(PERM);
+E(NOENT);
+E(INTR);
+E(BADF);
+E(ACCES);
+E(FAULT);
+E(BUSY);
+E(EXIST);
+E(NODEV);
+E(NOTDIR);
+E(ISDIR);
+E(INVAL);
+E(NFILE);
+E(MFILE);
+E(FBIG);
+E(NOSPC);
+E(SPIPE);
+E(ROFS);
+E(NAMETOOLONG);
+default:
+return GDB_EUNKNOWN;
+}
+#undef E
+}
+
 static void m68k_semi_u32_cb(CPUState *cs, uint64_t ret, int err)
 {
 M68kCPU *cpu = M68K_CPU(cs);
@@ -48,7 +77,7 @@ static void m68k_semi_u32_cb(CPUState *cs, uint64_t ret, int 
err)
 
 target_ulong args = env->dregs[1];
 if (put_user_u32(ret, args) ||
-put_user_u32(err, args + 4)) {
+put_user_u32(host_to_gdb_errno(err), args + 4)) {
 /*
  * The m68k semihosting ABI does not provide any way to report this
  * error to the guest, so the best we can do is log it in qemu.
@@ -67,7 +96,7 @@ static void m68k_semi_u64_cb(CPUState *cs, uint64_t ret, int 
err)
 target_ulong args = env->dregs[1];
 if (put_user_u32(ret >> 32, args) ||
 put_user_u32(ret, args + 4) ||
-put_user_u32(err, args + 8)) {
+put_user_u32(host_to_gdb_errno(err), args + 8)) {
 /* No way to report this via m68k semihosting ABI; just log it */
 qemu_log_mask(LOG_GUEST_ERROR, "m68k-semihosting: return value "
   "discarded because argument block not writable\n");
-- 
2.34.1




[PULL 01/11] target/nios2: Use semihosting/syscalls.h

2022-09-14 Thread Richard Henderson
This separates guest file descriptors from host file descriptors,
and utilizes shared infrastructure for integration with gdbstub.

Signed-off-by: Richard Henderson 
---
 target/nios2/nios2-semi.c | 296 +++---
 1 file changed, 50 insertions(+), 246 deletions(-)

diff --git a/target/nios2/nios2-semi.c b/target/nios2/nios2-semi.c
index 55061bb2dc..614fd76695 100644
--- a/target/nios2/nios2-semi.c
+++ b/target/nios2/nios2-semi.c
@@ -24,6 +24,7 @@
 #include "qemu/osdep.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
+#include "semihosting/syscalls.h"
 #include "semihosting/softmmu-uaccess.h"
 #include "qemu/log.h"
 
@@ -42,67 +43,6 @@
 #define HOSTED_ISATTY 12
 #define HOSTED_SYSTEM 13
 
-static int translate_openflags(int flags)
-{
-int hf;
-
-if (flags & GDB_O_WRONLY) {
-hf = O_WRONLY;
-} else if (flags & GDB_O_RDWR) {
-hf = O_RDWR;
-} else {
-hf = O_RDONLY;
-}
-
-if (flags & GDB_O_APPEND) {
-hf |= O_APPEND;
-}
-if (flags & GDB_O_CREAT) {
-hf |= O_CREAT;
-}
-if (flags & GDB_O_TRUNC) {
-hf |= O_TRUNC;
-}
-if (flags & GDB_O_EXCL) {
-hf |= O_EXCL;
-}
-
-return hf;
-}
-
-static bool translate_stat(CPUNios2State *env, target_ulong addr,
-   struct stat *s)
-{
-struct gdb_stat *p;
-
-p = lock_user(VERIFY_WRITE, addr, sizeof(struct gdb_stat), 0);
-
-if (!p) {
-return false;
-}
-p->gdb_st_dev = cpu_to_be32(s->st_dev);
-p->gdb_st_ino = cpu_to_be32(s->st_ino);
-p->gdb_st_mode = cpu_to_be32(s->st_mode);
-p->gdb_st_nlink = cpu_to_be32(s->st_nlink);
-p->gdb_st_uid = cpu_to_be32(s->st_uid);
-p->gdb_st_gid = cpu_to_be32(s->st_gid);
-p->gdb_st_rdev = cpu_to_be32(s->st_rdev);
-p->gdb_st_size = cpu_to_be64(s->st_size);
-#ifdef _WIN32
-/* Windows stat is missing some fields.  */
-p->gdb_st_blksize = 0;
-p->gdb_st_blocks = 0;
-#else
-p->gdb_st_blksize = cpu_to_be64(s->st_blksize);
-p->gdb_st_blocks = cpu_to_be64(s->st_blocks);
-#endif
-p->gdb_st_atime = cpu_to_be32(s->st_atime);
-p->gdb_st_mtime = cpu_to_be32(s->st_mtime);
-p->gdb_st_ctime = cpu_to_be32(s->st_ctime);
-unlock_user(p, addr, sizeof(struct gdb_stat));
-return true;
-}
-
 static void nios2_semi_u32_cb(CPUState *cs, uint64_t ret, int err)
 {
 Nios2CPU *cpu = NIOS2_CPU(cs);
@@ -142,22 +82,22 @@ static void nios2_semi_u64_cb(CPUState *cs, uint64_t ret, 
int err)
  */
 #define GET_ARG(n) do { \
 if (get_user_ual(arg ## n, args + (n) * 4)) {   \
-result = -1;\
-errno = EFAULT; \
 goto failed;\
 }   \
 } while (0)
 
+#define GET_ARG64(n) do {   \
+if (get_user_ual(arg ## n, args + (n) * 4)) {   \
+goto failed64;  \
+}   \
+} while (0)
+
 void do_nios2_semihosting(CPUNios2State *env)
 {
 CPUState *cs = env_cpu(env);
 int nr;
 uint32_t args;
 target_ulong arg0, arg1, arg2, arg3;
-void *p;
-void *q;
-uint32_t len;
-uint32_t result;
 
 nr = env->regs[R_ARG0];
 args = env->regs[R_ARG1];
@@ -165,234 +105,98 @@ void do_nios2_semihosting(CPUNios2State *env)
 case HOSTED_EXIT:
 gdb_exit(env->regs[R_ARG0]);
 exit(env->regs[R_ARG0]);
+
 case HOSTED_OPEN:
 GET_ARG(0);
 GET_ARG(1);
 GET_ARG(2);
 GET_ARG(3);
-if (use_gdb_syscalls()) {
-gdb_do_syscall(nios2_semi_u32_cb, "open,%s,%x,%x", arg0, (int)arg1,
-   arg2, arg3);
-return;
-} else {
-p = lock_user_string(arg0);
-if (!p) {
-result = -1;
-errno = EFAULT;
-} else {
-result = open(p, translate_openflags(arg2), arg3);
-unlock_user(p, arg0, 0);
-}
-}
+semihost_sys_open(cs, nios2_semi_u32_cb, arg0, arg1, arg2, arg3);
 break;
+
 case HOSTED_CLOSE:
-{
-/* Ignore attempts to close stdin/out/err.  */
-GET_ARG(0);
-int fd = arg0;
-if (fd > 2) {
-if (use_gdb_syscalls()) {
-gdb_do_syscall(nios2_semi_u32_cb, "close,%x", arg0);
-return;
-} else {
-result = close(fd);
-}
-} else {
-result = 0;
-}
-break;
-}
+GET_ARG(0);
+semihost_sys_close(cs, nios2_semi_u32_cb, arg0);
+break;
+
 case HOSTED_READ:
 GET_ARG(0);
 GET_ARG(1);
 GET_ARG(2);
-len = arg2;
-if 

Re: [PATCH] target/arm: Fix alignment for VLD4.32

2022-09-14 Thread Richard Henderson

On 9/14/22 11:50, Clément Chigot wrote:

When requested, the alignment for VLD4.32 is 8 and not 16.

See ARM documentation about VLD4 encoding:
 ebytes = 1 << UInt(size);
 if size == '10' then
 alignment = if a == '0' then 1 else 8;
 else
 alignment = if a == '0' then 1 else 4*ebytes;

Signed-off-by: Clément Chigot 


Reviewed-by: Richard Henderson 

r~


---
  target/arm/translate-neon.c | 6 +-
  1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c
index 321c17e2c7..4016339d46 100644
--- a/target/arm/translate-neon.c
+++ b/target/arm/translate-neon.c
@@ -584,7 +584,11 @@ static bool trans_VLD_all_lanes(DisasContext *s, 
arg_VLD_all_lanes *a)
  case 3:
  return false;
  case 4:
-align = pow2_align(size + 2);
+if (size == 2) {
+align = pow2_align(3);
+} else {
+align = pow2_align(size + 2);
+}
  break;
  default:
  g_assert_not_reached();





[PULL 20/20] target/arm: Make boards pass base address to armv7m_load_kernel()

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Currently armv7m_load_kernel() takes the size of the block of memory
where it should load the initial guest image, but assumes that it
should always load it at address 0.  This happens to be true of all
our M-profile boards at the moment, but it isn't guaranteed to always
be so: M-profile CPUs can be configured (via init-svtor and
init-nsvtor, which match equivalent hardware configuration signals)
to have the initial vector table at any address, not just zero.  (For
instance the Teeny board has the boot ROM at address 0x0200_.)

Add a base address argument to armv7m_load_kernel(), so that
callers now pass in both base address and size. All the current
callers pass 0, so this is not a behaviour change.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20220823160417.3858216-3-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 include/hw/arm/boot.h | 5 -
 hw/arm/armv7m.c   | 5 +++--
 hw/arm/aspeed.c   | 1 +
 hw/arm/microbit.c | 2 +-
 hw/arm/mps2-tz.c  | 2 +-
 hw/arm/mps2.c | 2 +-
 hw/arm/msf2-som.c | 2 +-
 hw/arm/musca.c| 3 ++-
 hw/arm/netduino2.c| 2 +-
 hw/arm/netduinoplus2.c| 2 +-
 hw/arm/stellaris.c| 2 +-
 hw/arm/stm32vldiscovery.c | 2 +-
 12 files changed, 18 insertions(+), 12 deletions(-)

diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
index c7ebae156e..f18cc3064f 100644
--- a/include/hw/arm/boot.h
+++ b/include/hw/arm/boot.h
@@ -25,13 +25,16 @@ typedef enum {
  * armv7m_load_kernel:
  * @cpu: CPU
  * @kernel_filename: file to load
+ * @mem_base: base address to load image at (should be where the
+ *CPU expects to find its vector table on reset)
  * @mem_size: mem_size: maximum image size to load
  *
  * Load the guest image for an ARMv7M system. This must be called by
  * any ARMv7M board. (This is necessary to ensure that the CPU resets
  * correctly on system reset, as well as for kernel loading.)
  */
-void armv7m_load_kernel(ARMCPU *cpu, const char *kernel_filename, int 
mem_size);
+void armv7m_load_kernel(ARMCPU *cpu, const char *kernel_filename,
+hwaddr mem_base, int mem_size);
 
 /* arm_boot.c */
 struct arm_boot_info {
diff --git a/hw/arm/armv7m.c b/hw/arm/armv7m.c
index fa4c2c735d..50a9507c0b 100644
--- a/hw/arm/armv7m.c
+++ b/hw/arm/armv7m.c
@@ -568,7 +568,8 @@ static void armv7m_reset(void *opaque)
 cpu_reset(CPU(cpu));
 }
 
-void armv7m_load_kernel(ARMCPU *cpu, const char *kernel_filename, int mem_size)
+void armv7m_load_kernel(ARMCPU *cpu, const char *kernel_filename,
+hwaddr mem_base, int mem_size)
 {
 ssize_t image_size;
 uint64_t entry;
@@ -588,7 +589,7 @@ void armv7m_load_kernel(ARMCPU *cpu, const char 
*kernel_filename, int mem_size)
  , NULL, NULL,
  NULL, 0, EM_ARM, 1, 0, as);
 if (image_size < 0) {
-image_size = load_image_targphys_as(kernel_filename, 0,
+image_size = load_image_targphys_as(kernel_filename, mem_base,
 mem_size, as);
 }
 if (image_size < 0) {
diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index b3bbe06f8f..bc3ecdb619 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -1430,6 +1430,7 @@ static void aspeed_minibmc_machine_init(MachineState 
*machine)
 
 armv7m_load_kernel(ARM_CPU(first_cpu),
machine->kernel_filename,
+   0,
AST1030_INTERNAL_FLASH_SIZE);
 }
 
diff --git a/hw/arm/microbit.c b/hw/arm/microbit.c
index e9494334ce..50df362088 100644
--- a/hw/arm/microbit.c
+++ b/hw/arm/microbit.c
@@ -57,7 +57,7 @@ static void microbit_init(MachineState *machine)
 mr, -1);
 
 armv7m_load_kernel(ARM_CPU(first_cpu), machine->kernel_filename,
-   s->nrf51.flash_size);
+   0, s->nrf51.flash_size);
 }
 
 static void microbit_machine_class_init(ObjectClass *oc, void *data)
diff --git a/hw/arm/mps2-tz.c b/hw/arm/mps2-tz.c
index 4017392bf5..394192b9b2 100644
--- a/hw/arm/mps2-tz.c
+++ b/hw/arm/mps2-tz.c
@@ -1197,7 +1197,7 @@ static void mps2tz_common_init(MachineState *machine)
 }
 
 armv7m_load_kernel(ARM_CPU(first_cpu), machine->kernel_filename,
-   boot_ram_size(mms));
+   0, boot_ram_size(mms));
 }
 
 static void mps2_tz_idau_check(IDAUInterface *ii, uint32_t address,
diff --git a/hw/arm/mps2.c b/hw/arm/mps2.c
index bb76fa6889..a86a994dba 100644
--- a/hw/arm/mps2.c
+++ b/hw/arm/mps2.c
@@ -450,7 +450,7 @@ static void mps2_common_init(MachineState *machine)
   mmc->fpga_type == FPGA_AN511 ? 47 : 13));
 
 armv7m_load_kernel(ARM_CPU(first_cpu), machine->kernel_filename,
-   0x40);
+

Re: [PATCH v11 10/21] block/mirror.c: use of job helpers in drivers

2022-09-14 Thread Vladimir Sementsov-Ogievskiy

On 8/26/22 16:20, Emanuele Giuseppe Esposito wrote:

Once job lock is used and aiocontext is removed, mirror has
to perform job operations under the same critical section,
Note: at this stage, job_{lock/unlock} and job lock guard macros
are*nop*.

Signed-off-by: Emanuele Giuseppe Esposito


Reviewed-by: Vladimir Sementsov-Ogievskiy 

--
Best regards,
Vladimir



[PULL 15/20] target/arm: Rename pmu_8_n feature test functions

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Our feature test functions that check the PMU version are named
isar_feature_{aa32,aa64,any}_pmu_8_{1,4}.  This doesn't match the
current Arm ARM official feature names, which are FEAT_PMUv3p1 and
FEAT_PMUv3p4.  Rename these functions to _pmuv3p1 and _pmuv3p4.

This commit was created with:
  sed -i -e 's/pmu_8_/pmuv3p/g' target/arm/*.[ch]

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822132358.3524971-8-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/arm/cpu.h| 16 
 target/arm/helper.c | 18 +-
 2 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index fa24ce9f96..d86e51992a 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3712,14 +3712,14 @@ static inline bool isar_feature_aa32_ats1e1(const 
ARMISARegisters *id)
 return FIELD_EX32(id->id_mmfr3, ID_MMFR3, PAN) >= 2;
 }
 
-static inline bool isar_feature_aa32_pmu_8_1(const ARMISARegisters *id)
+static inline bool isar_feature_aa32_pmuv3p1(const ARMISARegisters *id)
 {
 /* 0xf means "non-standard IMPDEF PMU" */
 return FIELD_EX32(id->id_dfr0, ID_DFR0, PERFMON) >= 4 &&
 FIELD_EX32(id->id_dfr0, ID_DFR0, PERFMON) != 0xf;
 }
 
-static inline bool isar_feature_aa32_pmu_8_4(const ARMISARegisters *id)
+static inline bool isar_feature_aa32_pmuv3p4(const ARMISARegisters *id)
 {
 /* 0xf means "non-standard IMPDEF PMU" */
 return FIELD_EX32(id->id_dfr0, ID_DFR0, PERFMON) >= 5 &&
@@ -4038,13 +4038,13 @@ static inline bool isar_feature_aa64_sme(const 
ARMISARegisters *id)
 return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SME) != 0;
 }
 
-static inline bool isar_feature_aa64_pmu_8_1(const ARMISARegisters *id)
+static inline bool isar_feature_aa64_pmuv3p1(const ARMISARegisters *id)
 {
 return FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) >= 4 &&
 FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) != 0xf;
 }
 
-static inline bool isar_feature_aa64_pmu_8_4(const ARMISARegisters *id)
+static inline bool isar_feature_aa64_pmuv3p4(const ARMISARegisters *id)
 {
 return FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) >= 5 &&
 FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) != 0xf;
@@ -4213,14 +4213,14 @@ static inline bool isar_feature_any_predinv(const 
ARMISARegisters *id)
 return isar_feature_aa64_predinv(id) || isar_feature_aa32_predinv(id);
 }
 
-static inline bool isar_feature_any_pmu_8_1(const ARMISARegisters *id)
+static inline bool isar_feature_any_pmuv3p1(const ARMISARegisters *id)
 {
-return isar_feature_aa64_pmu_8_1(id) || isar_feature_aa32_pmu_8_1(id);
+return isar_feature_aa64_pmuv3p1(id) || isar_feature_aa32_pmuv3p1(id);
 }
 
-static inline bool isar_feature_any_pmu_8_4(const ARMISARegisters *id)
+static inline bool isar_feature_any_pmuv3p4(const ARMISARegisters *id)
 {
-return isar_feature_aa64_pmu_8_4(id) || isar_feature_aa32_pmu_8_4(id);
+return isar_feature_aa64_pmuv3p4(id) || isar_feature_aa32_pmuv3p4(id);
 }
 
 static inline bool isar_feature_any_ccidx(const ARMISARegisters *id)
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 11a7a57710..987ac19fe8 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -879,16 +879,16 @@ static int64_t instructions_ns_per(uint64_t icount)
 }
 #endif
 
-static bool pmu_8_1_events_supported(CPUARMState *env)
+static bool pmuv3p1_events_supported(CPUARMState *env)
 {
 /* For events which are supported in any v8.1 PMU */
-return cpu_isar_feature(any_pmu_8_1, env_archcpu(env));
+return cpu_isar_feature(any_pmuv3p1, env_archcpu(env));
 }
 
-static bool pmu_8_4_events_supported(CPUARMState *env)
+static bool pmuv3p4_events_supported(CPUARMState *env)
 {
 /* For events which are supported in any v8.1 PMU */
-return cpu_isar_feature(any_pmu_8_4, env_archcpu(env));
+return cpu_isar_feature(any_pmuv3p4, env_archcpu(env));
 }
 
 static uint64_t zero_event_get_count(CPUARMState *env)
@@ -922,17 +922,17 @@ static const pm_event pm_events[] = {
 },
 #endif
 { .number = 0x023, /* STALL_FRONTEND */
-  .supported = pmu_8_1_events_supported,
+  .supported = pmuv3p1_events_supported,
   .get_count = zero_event_get_count,
   .ns_per_count = zero_event_ns_per,
 },
 { .number = 0x024, /* STALL_BACKEND */
-  .supported = pmu_8_1_events_supported,
+  .supported = pmuv3p1_events_supported,
   .get_count = zero_event_get_count,
   .ns_per_count = zero_event_ns_per,
 },
 { .number = 0x03c, /* STALL */
-  .supported = pmu_8_4_events_supported,
+  .supported = pmuv3p4_events_supported,
   .get_count = zero_event_get_count,
   .ns_per_count = zero_event_ns_per,
 },
@@ -6400,7 +6400,7 @@ static void define_pmu_regs(ARMCPU *cpu)
 g_free(pmevtyper_name);
 g_free(pmevtyper_el0_name);
 }
-if (cpu_isar_feature(aa32_pmu_8_1, cpu)) {
+if (cpu_isar_feature(aa32_pmuv3p1, 

[PATCH v2] e1000e: set RX desc status with DD flag in a separate operation

2022-09-14 Thread Ding Hui
Like commit 034d00d48581 ("e1000: set RX descriptor status in
a separate operation"), there is also same issue in e1000e, which
would cause lost packets or stop sending packets to VM with DPDK.

Do similar fix in e1000e.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/402
Signed-off-by: Ding Hui 
---
 hw/net/e1000e_core.c | 53 +++-
 1 file changed, 52 insertions(+), 1 deletion(-)

---
v2: use uint8_t/uint32_t directly instead of typeof

diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c
index 208e3e0d79..a570b366b2 100644
--- a/hw/net/e1000e_core.c
+++ b/hw/net/e1000e_core.c
@@ -1364,6 +1364,57 @@ struct NetRxPkt *pkt, const E1000E_RSSInfo *rss_info,
 }
 }
 
+static inline void
+e1000e_pci_dma_write_rx_desc(E1000ECore *core, dma_addr_t addr,
+ uint8_t *desc, dma_addr_t len)
+{
+PCIDevice *dev = core->owner;
+
+if (e1000e_rx_use_legacy_descriptor(core)) {
+struct e1000_rx_desc *d = (struct e1000_rx_desc *) desc;
+size_t offset = offsetof(struct e1000_rx_desc, status);
+uint8_t status = d->status;
+
+d->status &= ~E1000_RXD_STAT_DD;
+pci_dma_write(dev, addr, desc, len);
+
+if (status & E1000_RXD_STAT_DD) {
+d->status = status;
+pci_dma_write(dev, addr + offset, , sizeof(status));
+}
+} else {
+if (core->mac[RCTL] & E1000_RCTL_DTYP_PS) {
+union e1000_rx_desc_packet_split *d =
+(union e1000_rx_desc_packet_split *) desc;
+size_t offset = offsetof(union e1000_rx_desc_packet_split,
+wb.middle.status_error);
+uint32_t status = d->wb.middle.status_error;
+
+d->wb.middle.status_error &= ~E1000_RXD_STAT_DD;
+pci_dma_write(dev, addr, desc, len);
+
+if (status & E1000_RXD_STAT_DD) {
+d->wb.middle.status_error = status;
+pci_dma_write(dev, addr + offset, , sizeof(status));
+}
+} else {
+union e1000_rx_desc_extended *d =
+(union e1000_rx_desc_extended *) desc;
+size_t offset = offsetof(union e1000_rx_desc_extended,
+wb.upper.status_error);
+uint32_t status = d->wb.upper.status_error;
+
+d->wb.upper.status_error &= ~E1000_RXD_STAT_DD;
+pci_dma_write(dev, addr, desc, len);
+
+if (status & E1000_RXD_STAT_DD) {
+d->wb.upper.status_error = status;
+pci_dma_write(dev, addr + offset, , sizeof(status));
+}
+}
+}
+}
+
 typedef struct e1000e_ba_state_st {
 uint16_t written[MAX_PS_BUFFERS];
 uint8_t cur_idx;
@@ -1600,7 +1651,7 @@ e1000e_write_packet_to_guest(E1000ECore *core, struct 
NetRxPkt *pkt,
 
 e1000e_write_rx_descr(core, desc, is_last ? core->rx_pkt : NULL,
rss_info, do_ps ? ps_hdr_len : 0, );
-pci_dma_write(d, base, , core->rx_desc_len);
+e1000e_pci_dma_write_rx_desc(core, base, desc, core->rx_desc_len);
 
 e1000e_ring_advance(core, rxi,
 core->rx_desc_len / E1000_MIN_RX_DESC_LEN);
-- 
2.17.1




[PATCH v2 3/4] scripts/ci/setup: spice-server only on x86 aarch64

2022-09-14 Thread Lucas Mateus Castro(alqotel)
From: "Lucas Mateus Castro (alqotel)" 

Changed build-environment.yml to only install spice-server on x86_64 and
aarch64 as this package is only available on those architectures.

Signed-off-by: Lucas Mateus Castro (alqotel) 
---
 scripts/ci/setup/build-environment.yml | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/scripts/ci/setup/build-environment.yml 
b/scripts/ci/setup/build-environment.yml
index 7535228685..43cf8c759f 100644
--- a/scripts/ci/setup/build-environment.yml
+++ b/scripts/ci/setup/build-environment.yml
@@ -160,7 +160,6 @@
   - python36
   - rdma-core-devel
   - spice-glib-devel
-  - spice-server
   - systemtap-sdt-devel
   - tar
   - zlib-devel
@@ -168,3 +167,14 @@
   when:
 - ansible_facts['distribution_file_variety'] == 'RedHat'
 - ansible_facts['distribution_version'] == '8'
+
+- name: Install packages only available on x86 and aarch64
+  dnf:
+# Spice server not available in ppc64le
+name:
+  - spice-server
+state: present
+  when:
+- ansible_facts['distribution_file_variety'] == 'RedHat'
+- ansible_facts['distribution_version'] == '8'
+- ansible_facts['architecture'] == 'aarch64' or 
ansible_facts['architecture'] == 'x86_64'
-- 
2.31.1




[PATCH] target/arm: Do alignment check when translation disabled

2022-09-14 Thread Richard Henderson
If translation is disabled, the default memory type is Device,
which requires alignment checking.  Document, but defer, the
more general case of per-page alignment checking.

Reported-by: Idan Horowitz 
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1204
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 38 --
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index d7bc467a2a..79609443aa 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -10713,6 +10713,39 @@ ARMMMUIdx arm_mmu_idx(CPUARMState *env)
 return arm_mmu_idx_el(env, arm_current_el(env));
 }
 
+/*
+ * Return true if memory alignment should be enforced.
+ */
+static bool aprofile_require_alignment(CPUARMState *env, int el, uint64_t 
sctlr)
+{
+/* Check the alignment enable bit. */
+if (sctlr & SCTLR_A) {
+return true;
+}
+
+/*
+ * If translation is disabled, then the default memory type
+ * may be Device(-nGnRnE) instead of Normal, which requires that
+ * alignment be enforced.
+ *
+ * TODO: The more general case is translation enabled, with a per-page
+ * check of the memory type as assigned via MAIR_ELx and the PTE.
+ * We could arrange for a bit in MemTxAttrs to enforce alignment
+ * via forced use of the softmmu slow path.  Given that such pages
+ * are intended for MMIO, where the slow path is required anyhow,
+ * this should not result in extra overhead.
+ */
+if (sctlr & SCTLR_M) {
+/* Translation enabled: memory type in PTE via MAIR_ELx. */
+return false;
+}
+if (el < 2 && (arm_hcr_el2_eff(env) & (HCR_DC | HCR_VM))) {
+/* Stage 2 translation enabled: memory type in PTE. */
+return false;
+}
+return true;
+}
+
 static CPUARMTBFlags rebuild_hflags_common(CPUARMState *env, int fp_el,
ARMMMUIdx mmu_idx,
CPUARMTBFlags flags)
@@ -10777,8 +10810,9 @@ static CPUARMTBFlags rebuild_hflags_a32(CPUARMState 
*env, int fp_el,
 {
 CPUARMTBFlags flags = {};
 int el = arm_current_el(env);
+uint64_t sctlr = arm_sctlr(env, el);
 
-if (arm_sctlr(env, el) & SCTLR_A) {
+if (aprofile_require_alignment(env, el, sctlr)) {
 DP_TBFLAG_ANY(flags, ALIGN_MEM, 1);
 }
 
@@ -10871,7 +10905,7 @@ static CPUARMTBFlags rebuild_hflags_a64(CPUARMState 
*env, int el, int fp_el,
 
 sctlr = regime_sctlr(env, stage1);
 
-if (sctlr & SCTLR_A) {
+if (aprofile_require_alignment(env, el, sctlr)) {
 DP_TBFLAG_ANY(flags, ALIGN_MEM, 1);
 }
 
-- 
2.34.1




[PULL 14/20] target/arm: Detect overflow when calculating next PMU interrupt

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

In pmccntr_op_finish() and pmevcntr_op_finish() we calculate the next
point at which we will get an overflow and need to fire the PMU
interrupt or set the overflow flag.  We do this by calculating the
number of nanoseconds to the overflow event and then adding it to
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL).  However, we don't check
whether that signed addition overflows, which can happen if the next
PMU interrupt would happen massively far in the future (250 years or
more).

Since QEMU assumes that "when the QEMU_CLOCK_VIRTUAL rolls over" is
"never", the sensible behaviour in this situation is simply to not
try to set the timer if it would be beyond that point.  Detect the
overflow, and skip setting the timer in that case.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822132358.3524971-7-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index b792694df0..11a7a57710 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1227,10 +1227,13 @@ static void pmccntr_op_finish(CPUARMState *env)
 int64_t overflow_in = cycles_ns_per(remaining_cycles);
 
 if (overflow_in > 0) {
-int64_t overflow_at = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-overflow_in;
-ARMCPU *cpu = env_archcpu(env);
-timer_mod_anticipate_ns(cpu->pmu_timer, overflow_at);
+int64_t overflow_at;
+
+if (!sadd64_overflow(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL),
+ overflow_in, _at)) {
+ARMCPU *cpu = env_archcpu(env);
+timer_mod_anticipate_ns(cpu->pmu_timer, overflow_at);
+}
 }
 #endif
 
@@ -1275,10 +1278,13 @@ static void pmevcntr_op_finish(CPUARMState *env, 
uint8_t counter)
 int64_t overflow_in = pm_events[event_idx].ns_per_count(delta);
 
 if (overflow_in > 0) {
-int64_t overflow_at = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-overflow_in;
-ARMCPU *cpu = env_archcpu(env);
-timer_mod_anticipate_ns(cpu->pmu_timer, overflow_at);
+int64_t overflow_at;
+
+if (!sadd64_overflow(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL),
+ overflow_in, _at)) {
+ARMCPU *cpu = env_archcpu(env);
+timer_mod_anticipate_ns(cpu->pmu_timer, overflow_at);
+}
 }
 #endif
 
-- 
2.34.1




[PULL 19/20] target/arm: Remove useless TARGET_BIG_ENDIAN check in armv7m_load_kernel()

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Arm system emulation targets always have TARGET_BIG_ENDIAN clear, so
there is no need to have handling in armv7m_load_kernel() for the
case when it is defined.  Remove the unnecessary code.

Side notes:
 * our M-profile implementation is always little-endian (that is, it
   makes the IMPDEF choice that the read-only AIRCR.ENDIANNESS is 0)
 * if we did want to handle big-endian ELF files here we should do it
   the way that hw/arm/boot.c:arm_load_elf() does, by looking at the
   ELF header to see what endianness the file itself is

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20220823160417.3858216-2-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 hw/arm/armv7m.c | 9 +
 1 file changed, 1 insertion(+), 8 deletions(-)

diff --git a/hw/arm/armv7m.c b/hw/arm/armv7m.c
index 990861ee5e..fa4c2c735d 100644
--- a/hw/arm/armv7m.c
+++ b/hw/arm/armv7m.c
@@ -572,17 +572,10 @@ void armv7m_load_kernel(ARMCPU *cpu, const char 
*kernel_filename, int mem_size)
 {
 ssize_t image_size;
 uint64_t entry;
-int big_endian;
 AddressSpace *as;
 int asidx;
 CPUState *cs = CPU(cpu);
 
-#if TARGET_BIG_ENDIAN
-big_endian = 1;
-#else
-big_endian = 0;
-#endif
-
 if (arm_feature(>env, ARM_FEATURE_EL3)) {
 asidx = ARMASIdx_S;
 } else {
@@ -593,7 +586,7 @@ void armv7m_load_kernel(ARMCPU *cpu, const char 
*kernel_filename, int mem_size)
 if (kernel_filename) {
 image_size = load_elf_as(kernel_filename, NULL, NULL, NULL,
  , NULL, NULL,
- NULL, big_endian, EM_ARM, 1, 0, as);
+ NULL, 0, EM_ARM, 1, 0, as);
 if (image_size < 0) {
 image_size = load_image_targphys_as(kernel_filename, 0,
 mem_size, as);
-- 
2.34.1




[PULL 12/20] target/arm: Ignore PMCR.D when PMCR.LC is set

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

The architecture requires that if PMCR.LC is set (for a 64-bit cycle
counter) then PMCR.D (which enables the clock divider so the counter
ticks every 64 cycles rather than every cycle) should be ignored.  We
were always honouring PMCR.D; fix the bug so we correctly ignore it
in this situation.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822132358.3524971-5-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index a348c7407d..f1b20de16d 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -1172,6 +1172,17 @@ static void pmu_update_irq(CPUARMState *env)
 (env->cp15.c9_pminten & env->cp15.c9_pmovsr));
 }
 
+static bool pmccntr_clockdiv_enabled(CPUARMState *env)
+{
+/*
+ * Return true if the clock divider is enabled and the cycle counter
+ * is supposed to tick only once every 64 clock cycles. This is
+ * controlled by PMCR.D, but if PMCR.LC is set to enable the long
+ * (64-bit) cycle counter PMCR.D has no effect.
+ */
+return (env->cp15.c9_pmcr & (PMCRD | PMCRLC)) == PMCRD;
+}
+
 /*
  * Ensure c15_ccnt is the guest-visible count so that operations such as
  * enabling/disabling the counter or filtering, modifying the count itself,
@@ -1184,8 +1195,7 @@ static void pmccntr_op_start(CPUARMState *env)
 
 if (pmu_counter_enabled(env, 31)) {
 uint64_t eff_cycles = cycles;
-if (env->cp15.c9_pmcr & PMCRD) {
-/* Increment once every 64 processor clock cycles */
+if (pmccntr_clockdiv_enabled(env)) {
 eff_cycles /= 64;
 }
 
@@ -1228,8 +1238,7 @@ static void pmccntr_op_finish(CPUARMState *env)
 #endif
 
 uint64_t prev_cycles = env->cp15.c15_ccnt_delta;
-if (env->cp15.c9_pmcr & PMCRD) {
-/* Increment once every 64 processor clock cycles */
+if (pmccntr_clockdiv_enabled(env)) {
 prev_cycles /= 64;
 }
 env->cp15.c15_ccnt_delta = prev_cycles - env->cp15.c15_ccnt;
-- 
2.34.1




[PULL 18/20] target/arm: Report FEAT_PMUv3p5 for TCG '-cpu max'

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Update the ID registers for TCG's '-cpu max' to report a FEAT_PMUv3p5
compliant PMU.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220822132358.3524971-11-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 docs/system/arm/emulation.rst | 1 +
 target/arm/cpu64.c| 2 +-
 target/arm/cpu_tcg.c  | 2 +-
 3 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst
index 811358fd0a..be7bbffe59 100644
--- a/docs/system/arm/emulation.rst
+++ b/docs/system/arm/emulation.rst
@@ -53,6 +53,7 @@ the following architecture extensions:
 - FEAT_PMULL (PMULL, PMULL2 instructions)
 - FEAT_PMUv3p1 (PMU Extensions v3.1)
 - FEAT_PMUv3p4 (PMU Extensions v3.4)
+- FEAT_PMUv3p5 (PMU Extensions v3.5)
 - FEAT_RAS (Reliability, availability, and serviceability)
 - FEAT_RASv1p1 (RAS Extension v1.1)
 - FEAT_RDM (Advanced SIMD rounding double multiply accumulate instructions)
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 3ac5e197a7..e6314e86d2 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -1152,7 +1152,7 @@ static void aarch64_max_initfn(Object *obj)
 
 t = cpu->isar.id_aa64dfr0;
 t = FIELD_DP64(t, ID_AA64DFR0, DEBUGVER, 9);  /* FEAT_Debugv8p4 */
-t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 5);/* FEAT_PMUv3p4 */
+t = FIELD_DP64(t, ID_AA64DFR0, PMUVER, 6);/* FEAT_PMUv3p5 */
 cpu->isar.id_aa64dfr0 = t;
 
 t = cpu->isar.id_aa64smfr0;
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index b714c61d94..98b5ba2160 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -85,7 +85,7 @@ void aa32_max_features(ARMCPU *cpu)
 t = cpu->isar.id_dfr0;
 t = FIELD_DP32(t, ID_DFR0, COPDBG, 9);/* FEAT_Debugv8p4 */
 t = FIELD_DP32(t, ID_DFR0, COPSDBG, 9);   /* FEAT_Debugv8p4 */
-t = FIELD_DP32(t, ID_DFR0, PERFMON, 5);   /* FEAT_PMUv3p4 */
+t = FIELD_DP32(t, ID_DFR0, PERFMON, 6);   /* FEAT_PMUv3p5 */
 cpu->isar.id_dfr0 = t;
 }
 
-- 
2.34.1




Re: [PATCH v3 00/20] ppc4xx_sdram QOMify and clean ups

2022-09-14 Thread BALATON Zoltan

On Wed, 14 Sep 2022, Cédric Le Goater wrote:

On 9/13/22 21:52, BALATON Zoltan wrote:

This is the end of the QOMify series started by Cédric. This series
handles the SDRAM controller models to clean them up, QOMify and unify
them and at least partially clean up the mess that has accumulated
around these in the past. This includes the not yet merged patches
from the last series and new ones that change the DDR2 version used by
sam460ex.



I made comments on the first ~10 patches. Let's try to agree on these
first. We will see the remaining ones in a second patchset.


Patch 10 does not make much sense without 11 and the final unificatoin of 
the two controllers is the real goal here so please try to review further 
patches too.


Regards,
BALATON Zoltan


Thanks,

C.




v3: Fix patches that got squashed during rebase
v2: address some review comments and try to avoid compile problem with
gcc 12.2 (untested)

BALATON Zoltan (20):
   ppc440_bamboo: Remove unnecessary memsets
   ppc4xx: Introduce Ppc4xxSdramBank struct
   ppc4xx_sdram: Get rid of the init RAM hack
   ppc4xx: Use Ppc4xxSdramBank in ppc4xx_sdram_banks()
   ppc440_bamboo: Add missing 4 MiB valid memory size
   ppc4xx_sdram: Move size check to ppc4xx_sdram_init()
   ppc4xx_sdram: QOM'ify
   ppc4xx_sdram: Drop extra zeros for readability
   ppc440_sdram: Split off map/unmap of sdram banks for later reuse
   ppc440_sdram: Implement enable bit in the DDR2 SDRAM
   ppc440_sdram: Get rid of the init RAM hack
   ppc440_sdram: Rename local variable for readibility
   ppc4xx_sdram: Rename functions to prevent name clashes
   ppc440_sdram: Move RAM size check to ppc440_sdram_init
   ppc440_sdram: QOM'ify
   ppc4xx_sdram: Move ppc4xx DDR and DDR2 SDRAM controller models
 together
   ppc4xx_sdram: Use hwaddr for memory bank size
   ppc4xx_sdram: Rename local state variable for brevity
   ppc4xx_sdram: Generalise bank setup
   ppc4xx_sdram: Convert DDR SDRAM controller to new bank handling

  hw/ppc/meson.build  |   3 +-
  hw/ppc/ppc405.h |   8 +-
  hw/ppc/ppc405_boards.c  |  22 +-
  hw/ppc/ppc405_uc.c  |  33 +-
  hw/ppc/ppc440.h |   4 -
  hw/ppc/ppc440_bamboo.c  |  29 +-
  hw/ppc/ppc440_uc.c  | 267 +--
  hw/ppc/ppc4xx_devs.c| 413 ---
  hw/ppc/ppc4xx_sdram.c   | 723 
  hw/ppc/sam460ex.c   |  48 +--
  hw/ppc/trace-events |   1 +
  include/hw/ppc/ppc4xx.h |  66 +++-
  12 files changed, 847 insertions(+), 770 deletions(-)
  create mode 100644 hw/ppc/ppc4xx_sdram.c






[PULL 08/20] target/arm: Add missing space in comment

2022-09-14 Thread Richard Henderson
From: Peter Maydell 

Fix a missing space before a comment terminator.

Signed-off-by: Peter Maydell 
Reviewed-by: Richard Henderson 
Message-Id: <20220819110052.2942289-7-peter.mayd...@linaro.org>
Signed-off-by: Richard Henderson 
---
 target/arm/cpu_tcg.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index f63f8cdd95..b714c61d94 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -64,7 +64,7 @@ void aa32_max_features(ARMCPU *cpu)
 t = FIELD_DP32(t, ID_MMFR4, HPDS, 1); /* FEAT_AA32HPD */
 t = FIELD_DP32(t, ID_MMFR4, AC2, 1);  /* ACTLR2, HACTLR2 */
 t = FIELD_DP32(t, ID_MMFR4, CNP, 1);  /* FEAT_TTCNP */
-t = FIELD_DP32(t, ID_MMFR4, XNX, 1);  /* FEAT_XNX*/
+t = FIELD_DP32(t, ID_MMFR4, XNX, 1);  /* FEAT_XNX */
 cpu->isar.id_mmfr4 = t;
 
 t = cpu->isar.id_mmfr5;
-- 
2.34.1




  1   2   >