Re: [PATCH v3 01/16] MAINTAINERS: Update Roman Bolshakov email address

[Roman Bolshakov]

24.06.2023 20:41, Philippe Mathieu-Daudé пишет:
> r.bolsha...@yadro.com is bouncing: Update Roman's email address
> using one found somewhere on the Internet; this way he can Ack-by.
>
> (Reorder Taylor's line to keep the section sorted alphabetically).
>
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>   MAINTAINERS | 4 ++--
>   .mailmap| 3 ++-
>   2 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 7f323cd2eb..1da135b0c8 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -497,14 +497,14 @@ F: target/arm/hvf/
>   
>   X86 HVF CPUs
>   M: Cameron Esfahani 
> -M: Roman Bolshakov 
> +M: Roman Bolshakov 
>   W: https://wiki.qemu.org/Features/HVF
>   S: Maintained
>   F: target/i386/hvf/
>   
>   HVF
>   M: Cameron Esfahani 
> -M: Roman Bolshakov 
> +M: Roman Bolshakov 
>   W: https://wiki.qemu.org/Features/HVF
>   S: Maintained
>   F: accel/hvf/
> diff --git a/.mailmap b/.mailmap
> index b57da4827e..64ef9f4de6 100644
> --- a/.mailmap
> +++ b/.mailmap
> @@ -76,9 +76,10 @@ Paul Burton  
>   Philippe Mathieu-Daudé  
>   Philippe Mathieu-Daudé  
>   Philippe Mathieu-Daudé  
> +Roman Bolshakov  
>   Stefan Brankovic  
> 
> -Yongbok Kim  
>   Taylor Simpson  
> +Yongbok Kim  
>   
>   # Also list preferred name forms where people have changed their
>   # git author config, or had utf8/latin1 encoding issues.

Hi Philippe,

Reviewed-by: Roman Bolshakov 

Thanks for updating the email.

Re: [PATCH v13 2/7] net/vmnet: add vmnet backends to qapi/net

[Roman Bolshakov]

On Tue, Jan 25, 2022 at 01:14:27PM +0900, Akihiko Odaki wrote:
> On Tue, Jan 25, 2022 at 8:00 AM Roman Bolshakov  wrote:
> >
> > On Mon, Jan 24, 2022 at 08:14:31PM +, Peter Maydell wrote:
> > > On Mon, 24 Jan 2022 at 17:49, Roman Bolshakov  wrote:
> > > > I'm not sure why blocks are Objective-C specific. All the data I have
> > > > shows the opposite [3][4][5]. They're just extensively used in Apple 
> > > > APIs.
> > >
> > > This is true, but for the purposes of our build machinery it is
> > > simpler to have three types of source files that it deals
> > > with (C, C++, ObjC) rather than four (C, C++, ObjC, C-that-uses-blocks).
> > > So unless there's a clear benefit from adding the extra category
> > > I think we should do the simple thing and keep these files named
> > > with a ".m" extension.
> > >
> >
> > Fine by me as long as majority finds it's simpler :) Perhaps it's just a
> > matter of personal preference.
> >
> > I've used to the fact that platform-specific code uses platform-specific
> > extensions or some sort of weird "GCC attributes". Therefore C with an
> > extension is easier to reason for me than Objective-C with ARC and other
> > kinds of implicit behaviour without an actual Objective-C code.
> >
> 
> Being technically pedantic, actually this vmnet implementation uses
> Objective-C and there is a file with .c which uses blocks.
> If a file is named .m, dispatch_retain(o) will be redefined as [o
> retain], and effectively makes it Objective-C code. Therefore, vmnet
> involves Objective-C as long as its files are named .m. It will be C
> with blocks if they are named .c.
> Speaking of use of blocks, actually audio/coreaudio.c involves blocks
> in header files; Core Audio has functions which accept blocks.
> 

Right, dispatch_retain()/dispatch_release() is just one example of the
implicit behaviour I'm talking about.

> I'm neutral about the decision.

> I think QEMU should avoid using Objective-C code except for
> interactions with Apple's APIs, and .c is superior in terms of that as
> it would prevent accidental introduction of Objective-C code.

That was exactly my point :)

> On the other hand, naming them .m will allow the
> introduction of Automatic Reference Counting to manage dispatch queue
> objects.

As of now ARC doesn't work automatically for .m files in QEMU. It
happens because QEMU doesn't enable it via -fobjc-arc.

If you try to enable it, Cocoa UI won't compile at all because of many
errors like this one and similar ones:

../ui/cocoa.m:1186:12: error: ARC forbids explicit message send of
'dealloc'
[super dealloc];
 ~ ^

> In fact, I have found a few memory leaks in vmnet in the last
> review and ui/cocoa.m has a suspicious construction of the object
> management (Particularly it has asynchronous dispatches wrapped with
> NSAutoreleasePool, which does not make sense).

> Introduction of Automatic Reference Counting would greatly help
> addressing those issues, but that would require significant rewriting
> of ui/cocoa.m.

Agreed.

Thanks,
Roman

P.S. I still think that given the mentioned facts and implicitness
introduced by Objective-C it would be more natural to have C code in
macOS-related device backends like vmnet and coreaudio unless
Objective-C is essential and required (like in UI code).

> Personally I'm concerned with ui/cocoa.m and do want to do that
> rewriting, but I'm being busy so it would not happen anytime soon.
> 
> Regards,
> Akihiko Odaki

Re: [PATCH v13 2/7] net/vmnet: add vmnet backends to qapi/net

[Roman Bolshakov]

On Mon, Jan 24, 2022 at 08:14:31PM +, Peter Maydell wrote:
> On Mon, 24 Jan 2022 at 17:49, Roman Bolshakov  wrote:
> > I'm not sure why blocks are Objective-C specific. All the data I have
> > shows the opposite [3][4][5]. They're just extensively used in Apple APIs.
> 
> This is true, but for the purposes of our build machinery it is
> simpler to have three types of source files that it deals
> with (C, C++, ObjC) rather than four (C, C++, ObjC, C-that-uses-blocks).
> So unless there's a clear benefit from adding the extra category
> I think we should do the simple thing and keep these files named
> with a ".m" extension.
> 

Fine by me as long as majority finds it's simpler :) Perhaps it's just a
matter of personal preference.

I've used to the fact that platform-specific code uses platform-specific
extensions or some sort of weird "GCC attributes". Therefore C with an
extension is easier to reason for me than Objective-C with ARC and other
kinds of implicit behaviour without an actual Objective-C code.

Thanks,
Roman

Re: [PATCH v13 2/7] net/vmnet: add vmnet backends to qapi/net

[Roman Bolshakov]

On Mon, Jan 24, 2022 at 12:27:40PM +0100, Christian Schoenebeck wrote:
> On Montag, 24. Januar 2022 10:56:00 CET Roman Bolshakov wrote:
> > On Thu, Jan 13, 2022 at 08:22:14PM +0300, Vladislav Yaroshchuk wrote:
> > >  net/vmnet-bridged.m |  25 +
> > >  net/vmnet-common.m  |  20 +++
> > 
> > It seems the last two files should have .c extension rather than .m.
> 
> I would not do that. Mind cross-compilers, please.
> 

Hi Christian,

Cross-compilers for Apple platforms can be constructed using à la carte
approach where toolchain comes from the source, SDK from Apple and a
port of cctools from GitHub (mind all library dependencies of QEMU).
That's quite an effort!

I very much doubt this is a relevant and typical case for QEMU on macOS.
And if cross-compiler is constructed properly it'll pass required flags
that enable blocks and will link block runtime in its default build
recipe like all cross-compilers do for the platform of interest.

Gladly, there's osxcross [1] and crossbuild image with Darwin support [2].
They can deal with blocks just fine:

  # CROSS_TRIPLE=i386-apple-darwin
  $ cc block.c && file a.out
  a.out: Mach-O i386 executable, 
flags:

  # CROSS_TRIPLE=x86_64-apple-darwin
  $ cc block.c && file a.out
  $ file a.out
  a.out: Mach-O 64-bit x86_64 executable, flags:

> > Unlike Cocoa UI code, the files do not contain Objective-C classes. They are
> > just C code with blocks (which is supported by compilers shipped with Xcode
> > SDK), e.g this program can be compiled without extra compiler flags:
> > 
> > $ cat block.c
> > int main() {
> > int (^x)(void) = ^{
> > return 0;
> > };
> > 
> > return x();
> > }
> > $ cc block.c && ./a.out
> > $
> > 
> 
> Such blocks are still Objective-C language specific, they are not C and 
> therefore won't work with GCC.
> 

I'm not sure why blocks are Objective-C specific. All the data I have
shows the opposite [3][4][5]. They're just extensively used in Apple APIs.

> $ gcc block.c
> 
> block.c: In function ‘main’:
> block.c:2:14: error: expected identifier or ‘(’ before ‘^’ token
>  int (^x)(void) = ^{
>   ^
> block.c:6:16: warning: implicit declaration of function ‘x’ [-Wimplicit-
> function-declaration]
>  return x();
> ^

You might do this on Linux and it'll work:

$ clang -g -fblocks -lBlocksRuntime block.c && ./a.out

However, vmnet code won't be compiled on non-Apple platforms because the
compilation happens only if vmnet is available which happens only if
appleframeworks dependency is available, that is not available on
non-OSX hosts [6]:

  "These dependencies can never be found for non-OSX hosts."

1. https://github.com/tpoechtrager/osxcross
2. https://github.com/multiarch/crossbuild
3. http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1370.pdf
4. https://clang.llvm.org/docs/BlockLanguageSpec.html
5. https://clang.llvm.org/docs/Block-ABI-Apple.html
6. https://mesonbuild.com/Dependencies.html#appleframeworks

Regards,
Roman

Re: [PATCH v13 2/7] net/vmnet: add vmnet backends to qapi/net

[Roman Bolshakov]

On Thu, Jan 13, 2022 at 08:22:14PM +0300, Vladislav Yaroshchuk wrote:
> Create separate netdevs for each vmnet operating mode:
> - vmnet-host
> - vmnet-shared
> - vmnet-bridged
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  net/clients.h   |  11 
>  net/meson.build |   7 +++
>  net/net.c   |  10 
>  net/vmnet-bridged.m |  25 +
>  net/vmnet-common.m  |  20 +++

Hi Vladislav,

It seems the last two files should have .c extension rather than .m.

Unlike Cocoa UI code, the files do not contain Objective-C classes. They are
just C code with blocks (which is supported by compilers shipped with Xcode
SDK), e.g this program can be compiled without extra compiler flags:

$ cat block.c
int main() {
int (^x)(void) = ^{
return 0;
};

return x();
}
$ cc block.c && ./a.out
$

Regards,
Roman

>  net/vmnet-host.c|  24 
>  net/vmnet-shared.c  |  25 +
>  net/vmnet_int.h |  25 +
>  qapi/net.json   | 133 +++-
>  9 files changed, 278 insertions(+), 2 deletions(-)
>  create mode 100644 net/vmnet-bridged.m
>  create mode 100644 net/vmnet-common.m
>  create mode 100644 net/vmnet-host.c
>  create mode 100644 net/vmnet-shared.c
>  create mode 100644 net/vmnet_int.h
> 
> diff --git a/net/clients.h b/net/clients.h
> index 92f9b59aed..c9157789f2 100644
> --- a/net/clients.h
> +++ b/net/clients.h
> @@ -63,4 +63,15 @@ int net_init_vhost_user(const Netdev *netdev, const char 
> *name,
>  
>  int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
>  NetClientState *peer, Error **errp);
> +#ifdef CONFIG_VMNET
> +int net_init_vmnet_host(const Netdev *netdev, const char *name,
> +  NetClientState *peer, Error **errp);
> +
> +int net_init_vmnet_shared(const Netdev *netdev, const char *name,
> +  NetClientState *peer, Error **errp);
> +
> +int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
> +  NetClientState *peer, Error **errp);
> +#endif /* CONFIG_VMNET */
> +
>  #endif /* QEMU_NET_CLIENTS_H */
> diff --git a/net/meson.build b/net/meson.build
> index 847bc2ac85..00a88c4951 100644
> --- a/net/meson.build
> +++ b/net/meson.build
> @@ -42,4 +42,11 @@ softmmu_ss.add(when: 'CONFIG_POSIX', if_true: 
> files(tap_posix))
>  softmmu_ss.add(when: 'CONFIG_WIN32', if_true: files('tap-win32.c'))
>  softmmu_ss.add(when: 'CONFIG_VHOST_NET_VDPA', if_true: files('vhost-vdpa.c'))
>  
> +vmnet_files = files(
> +  'vmnet-common.m',
> +  'vmnet-bridged.m',
> +  'vmnet-host.c',
> +  'vmnet-shared.c'
> +)
> +softmmu_ss.add(when: vmnet, if_true: vmnet_files)
>  subdir('can')
> diff --git a/net/net.c b/net/net.c
> index f0d14dbfc1..1dbb64b935 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -1021,6 +1021,11 @@ static int (* const 
> net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
>  #ifdef CONFIG_L2TPV3
>  [NET_CLIENT_DRIVER_L2TPV3]= net_init_l2tpv3,
>  #endif
> +#ifdef CONFIG_VMNET
> +[NET_CLIENT_DRIVER_VMNET_HOST] = net_init_vmnet_host,
> +[NET_CLIENT_DRIVER_VMNET_SHARED] = net_init_vmnet_shared,
> +[NET_CLIENT_DRIVER_VMNET_BRIDGED] = net_init_vmnet_bridged,
> +#endif /* CONFIG_VMNET */
>  };
>  
>  
> @@ -1106,6 +,11 @@ void show_netdevs(void)
>  #endif
>  #ifdef CONFIG_VHOST_VDPA
>  "vhost-vdpa",
> +#endif
> +#ifdef CONFIG_VMNET
> +"vmnet-host",
> +"vmnet-shared",
> +"vmnet-bridged",
>  #endif
>  };
>  
> diff --git a/net/vmnet-bridged.m b/net/vmnet-bridged.m
> new file mode 100644
> index 00..4e42a90391
> --- /dev/null
> +++ b/net/vmnet-bridged.m
> @@ -0,0 +1,25 @@
> +/*
> + * vmnet-bridged.m
> + *
> + * Copyright(c) 2021 Vladislav Yaroshchuk 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/qapi-types-net.h"
> +#include "vmnet_int.h"
> +#include "clients.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> +
> +#include 
> +
> +int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
> +   NetClientState *peer, Error **errp)
> +{
> +  error_setg(errp, "vmnet-bridged is not implemented yet");
> +  return -1;
> +}
> diff --git a/net/vmnet-common.m b/net/vmnet-common.m
> new file mode 100644
> index 00..532d152840
> --- /dev/null
> +++ b/net/vmnet-common.m
> @@ -0,0 +1,20 @@
> +/*
> + * vmnet-common.m - network client wrapper for Apple vmnet.framework
> + *
> + * Copyright(c) 2021 Vladislav Yaroshchuk 
> + * Copyright(c) 2021 Phillip Tennen 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/qapi-types-net.h"
> +#include "vmnet_int.h"
> +#include 

Re: [PATCH v13 2/7] net/vmnet: add vmnet backends to qapi/net

[Roman Bolshakov]

On Thu, Jan 13, 2022 at 08:22:14PM +0300, Vladislav Yaroshchuk wrote:
> Create separate netdevs for each vmnet operating mode:
> - vmnet-host
> - vmnet-shared
> - vmnet-bridged
> 

Sure I'm late to the party but what if we add only one backend - vmnet
with default mode set to shared and all parameters are added there?

The CLI would look more reasonable for the most typical use case:
 -netdev vmnet,id=if1 -device virtio-net,netdev=if1

That would remove duplication of options in QAPI schema (e.g. isolated
is available in all backends now, altough I'm not sure if it makes sense
for bridged mode):

 -netdev vmnet,id=if1,isolated=yes

start-address, end-address and subnet-mask are also used by both shared
and host modes.

Bridged netdev would lool like:

 -netdev vmnet,id=if1,mode=bridged,ifname=en1

Checksum offloading also seems to be available for all backends from
Monterey.

The approach might simplify integration of the changes to libvirt and
discovery of upcoming vmnet features via qapi.

Thanks,
Roman

> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  net/clients.h   |  11 
>  net/meson.build |   7 +++
>  net/net.c   |  10 
>  net/vmnet-bridged.m |  25 +
>  net/vmnet-common.m  |  20 +++
>  net/vmnet-host.c|  24 
>  net/vmnet-shared.c  |  25 +
>  net/vmnet_int.h |  25 +
>  qapi/net.json   | 133 +++-
>  9 files changed, 278 insertions(+), 2 deletions(-)
>  create mode 100644 net/vmnet-bridged.m
>  create mode 100644 net/vmnet-common.m
>  create mode 100644 net/vmnet-host.c
>  create mode 100644 net/vmnet-shared.c
>  create mode 100644 net/vmnet_int.h
> 
> diff --git a/net/net.c b/net/net.c
> index f0d14dbfc1..1dbb64b935 100644
> --- a/net/net.c
> +++ b/net/net.c
> @@ -1021,6 +1021,11 @@ static int (* const 
> net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
>  #ifdef CONFIG_L2TPV3
>  [NET_CLIENT_DRIVER_L2TPV3]= net_init_l2tpv3,
>  #endif
> +#ifdef CONFIG_VMNET
> +[NET_CLIENT_DRIVER_VMNET_HOST] = net_init_vmnet_host,
> +[NET_CLIENT_DRIVER_VMNET_SHARED] = net_init_vmnet_shared,
> +[NET_CLIENT_DRIVER_VMNET_BRIDGED] = net_init_vmnet_bridged,
> +#endif /* CONFIG_VMNET */
>  };
>  
>  
> @@ -1106,6 +,11 @@ void show_netdevs(void)
>  #endif
>  #ifdef CONFIG_VHOST_VDPA
>  "vhost-vdpa",
> +#endif
> +#ifdef CONFIG_VMNET
> +"vmnet-host",
> +"vmnet-shared",
> +"vmnet-bridged",
>  #endif
>  };
>  
> diff --git a/net/vmnet-bridged.m b/net/vmnet-bridged.m
> new file mode 100644
> index 00..4e42a90391
> --- /dev/null
> +++ b/net/vmnet-bridged.m
> @@ -0,0 +1,25 @@
> +/*
> + * vmnet-bridged.m
> + *
> + * Copyright(c) 2021 Vladislav Yaroshchuk 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/qapi-types-net.h"
> +#include "vmnet_int.h"
> +#include "clients.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> +
> +#include 
> +
> +int net_init_vmnet_bridged(const Netdev *netdev, const char *name,
> +   NetClientState *peer, Error **errp)
> +{
> +  error_setg(errp, "vmnet-bridged is not implemented yet");
> +  return -1;
> +}
> diff --git a/net/vmnet-common.m b/net/vmnet-common.m
> new file mode 100644
> index 00..532d152840
> --- /dev/null
> +++ b/net/vmnet-common.m
> @@ -0,0 +1,20 @@
> +/*
> + * vmnet-common.m - network client wrapper for Apple vmnet.framework
> + *
> + * Copyright(c) 2021 Vladislav Yaroshchuk 
> + * Copyright(c) 2021 Phillip Tennen 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/qapi-types-net.h"
> +#include "vmnet_int.h"
> +#include "clients.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> +
> +#include 
> +
> diff --git a/net/vmnet-host.c b/net/vmnet-host.c
> new file mode 100644
> index 00..4a5ef99dc7
> --- /dev/null
> +++ b/net/vmnet-host.c
> @@ -0,0 +1,24 @@
> +/*
> + * vmnet-host.c
> + *
> + * Copyright(c) 2021 Vladislav Yaroshchuk 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/qapi-types-net.h"
> +#include "vmnet_int.h"
> +#include "clients.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> +
> +#include 
> +
> +int net_init_vmnet_host(const Netdev *netdev, const char *name,
> +NetClientState *peer, Error **errp) {
> +  error_setg(errp, "vmnet-host is not implemented yet");
> +  return -1;
> +}
> diff --git a/net/vmnet-shared.c b/net/vmnet-shared.c
> new file mode 100644
> index 00..f8c4a4f3b8
> --- /dev/null
> +++ b/net/vmnet-shared.c
> 

Re: [PATCH v13 1/7] net/vmnet: add vmnet dependency and customizable option

[Roman Bolshakov]

On Thu, Jan 13, 2022 at 08:22:13PM +0300, Vladislav Yaroshchuk wrote:
> vmnet.framework dependency is added with 'vmnet' option
> to enable or disable it. Default value is 'auto'.
> 
> vmnet features to be used are available since macOS 11.0,

Hi Vladislav,

I'm not sure if the comment belongs here. Perhaps you mean that bridged
mode is available from 10.15:

VMNET_BRIDGED_MODE API_AVAILABLE(macos(10.15))  = 1002

This means vmnet.framework is supported on all macbooks starting from 2012.

With this fixed,
Tested-by: Roman Bolshakov 
Reviewed-by: Roman Bolshakov 

The other two modes - shared and host are supported on earlier versions
of macOS (from 10.10). But port forwarding is only available from macOS
10.15.

Theoretically it should possible to support the framework on the earlier
models from 2010 or 2007 on Yosemite up to High Sierra with less
features using MacPorts but I don't think it'd be reasonable to ask
that.

Thanks,
Roman

> corresponding probe is created into meson.build.
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  meson.build   | 16 +++-
>  meson_options.txt |  2 ++
>  scripts/meson-buildoptions.sh |  3 +++
>  3 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/meson.build b/meson.build
> index c1b1db1e28..285fb7bc41 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -496,6 +496,18 @@ if cocoa.found() and get_option('gtk').enabled()
>error('Cocoa and GTK+ cannot be enabled at the same time')
>  endif
>  
> +vmnet = dependency('appleframeworks', modules: 'vmnet', required: 
> get_option('vmnet'))
> +if vmnet.found() and not cc.has_header_symbol('vmnet/vmnet.h',
> +  'VMNET_BRIDGED_MODE',
> +  dependencies: vmnet)
> +  vmnet = not_found
> +  if get_option('vmnet').enabled()
> +error('vmnet.framework API is outdated')
> +  else
> +warning('vmnet.framework API is outdated, disabling')
> +  endif
> +endif
> +
>  seccomp = not_found
>  if not get_option('seccomp').auto() or have_system or have_tools
>seccomp = dependency('libseccomp', version: '>=2.3.0',
> @@ -1492,6 +1504,7 @@ config_host_data.set('CONFIG_SECCOMP', seccomp.found())
>  config_host_data.set('CONFIG_SNAPPY', snappy.found())
>  config_host_data.set('CONFIG_USB_LIBUSB', libusb.found())
>  config_host_data.set('CONFIG_VDE', vde.found())
> +config_host_data.set('CONFIG_VMNET', vmnet.found())
>  config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', 
> have_vhost_user_blk_server)
>  config_host_data.set('CONFIG_VNC', vnc.found())
>  config_host_data.set('CONFIG_VNC_JPEG', jpeg.found())
> @@ -3406,7 +3419,8 @@ summary(summary_info, bool_yn: true, section: 'Crypto')
>  # Libraries
>  summary_info = {}
>  if targetos == 'darwin'
> -  summary_info += {'Cocoa support':   cocoa}
> +  summary_info += {'Cocoa support':   cocoa}
> +  summary_info += {'vmnet.framework support': vmnet}
>  endif
>  summary_info += {'SDL support':   sdl}
>  summary_info += {'SDL image support': sdl_image}
> diff --git a/meson_options.txt b/meson_options.txt
> index 921967eddb..701e1381f9 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -151,6 +151,8 @@ option('netmap', type : 'feature', value : 'auto',
> description: 'netmap network backend support')
>  option('vde', type : 'feature', value : 'auto',
> description: 'vde network backend support')
> +option('vmnet', type : 'feature', value : 'auto',
> +   description: 'vmnet.framework network backend support')
>  option('virglrenderer', type : 'feature', value : 'auto',
> description: 'virgl rendering support')
>  option('vnc', type : 'feature', value : 'auto',
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index 50bd7bed4d..cdcece4b05 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -84,6 +84,7 @@ meson_options_help() {
>printf "%s\n" '  u2f U2F emulation support'
>printf "%s\n" '  usb-redir   libusbredir support'
>printf "%s\n" '  vde vde network backend support'
> +  printf "%s\n" '  vmnet   vmnet.framework network backend support'
>printf "%s\n" '  vhost-user-blk-server'
>printf "%s\n" '  build vhost-user-blk server'
>printf "%s\n" '  virglrenderer   virgl rendering support'
> @@ -248,6 +249,8 @@ _meson_option_parse() {
>  --disable-usb-redir) printf "%s" -Dusb_redir=disabled ;;
>  --enable-vde) printf "%s" -Dvde=enabled ;;
>  --disable-vde) printf "%s" -Dvde=disabled ;

Re: [PATCH v11 1/7] net/vmnet: add vmnet dependency and customizable option

[Roman Bolshakov]

On Wed, Jan 12, 2022 at 03:21:44PM +0300, Vladislav Yaroshchuk wrote:
> vmnet.framework dependency is added with 'vmnet' option
> to enable or disable it. Default value is 'auto'.
> 
> vmnet features to be used are available since macOS 11.0,
> corresponding probe is created into meson.build.
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  meson.build   | 23 ++-
>  meson_options.txt |  2 ++
>  scripts/meson-buildoptions.sh |  3 +++
>  3 files changed, 27 insertions(+), 1 deletion(-)
> 
> diff --git a/meson.build b/meson.build
> index c1b1db1e28..b912c9cb91 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -496,6 +496,24 @@ if cocoa.found() and get_option('gtk').enabled()
>error('Cocoa and GTK+ cannot be enabled at the same time')
>  endif
>  
> +vmnet = dependency('appleframeworks', modules: 'vmnet', required: 
> get_option('vmnet'))
> +vmnet_11_0_api = false
> +if vmnet.found() and not cc.has_header_symbol('vmnet/vmnet.h',
> +  'VMNET_BRIDGED_MODE',
> +  dependencies: vmnet)
> +  vmnet = not_found
> +  if get_option('vmnet').enabled()
> +error('vmnet.framework API is outdated')
> +  else
> +warning('vmnet.framework API is outdated, disabling')
> +  endif
> +endif
> +if vmnet.found() and cc.has_header_symbol('vmnet/vmnet.h',
> +  'VMNET_SHARING_SERVICE_BUSY',
> +  dependencies: vmnet)
> +  vmnet_11_0_api = true
> +endif
> +
>  seccomp = not_found
>  if not get_option('seccomp').auto() or have_system or have_tools
>seccomp = dependency('libseccomp', version: '>=2.3.0',
> @@ -1492,6 +1510,8 @@ config_host_data.set('CONFIG_SECCOMP', seccomp.found())
>  config_host_data.set('CONFIG_SNAPPY', snappy.found())
>  config_host_data.set('CONFIG_USB_LIBUSB', libusb.found())
>  config_host_data.set('CONFIG_VDE', vde.found())
> +config_host_data.set('CONFIG_VMNET', vmnet.found())
> +config_host_data.set('CONFIG_VMNET_11_0_API', vmnet_11_0_api)

Hi Vladislav,

There might be more functionality coming in the next macOS versions but
we likely don't want to add them as extra CONFIG defines. Instead we
wrap new symbols/functions/code that are avaialble above Big Sur in the
code as:

#if defined(MAC_OS_VERSION_11_0) && \
MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0

xpc_dictionary_set_bool(
if_desc,
vmnet_enable_isolation_key,
options->isolated
);

#endif

Please see similar thread here:
https://lists.gnu.org/archive/html/qemu-devel/2022-01/msg01915.html

Thanks,
Roman

>  config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', 
> have_vhost_user_blk_server)
>  config_host_data.set('CONFIG_VNC', vnc.found())
>  config_host_data.set('CONFIG_VNC_JPEG', jpeg.found())
> @@ -3406,7 +3426,8 @@ summary(summary_info, bool_yn: true, section: 'Crypto')
>  # Libraries
>  summary_info = {}
>  if targetos == 'darwin'
> -  summary_info += {'Cocoa support':   cocoa}
> +  summary_info += {'Cocoa support':   cocoa}
> +  summary_info += {'vmnet.framework support': vmnet}
>  endif
>  summary_info += {'SDL support':   sdl}
>  summary_info += {'SDL image support': sdl_image}
> diff --git a/meson_options.txt b/meson_options.txt
> index 921967eddb..701e1381f9 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -151,6 +151,8 @@ option('netmap', type : 'feature', value : 'auto',
> description: 'netmap network backend support')
>  option('vde', type : 'feature', value : 'auto',
> description: 'vde network backend support')
> +option('vmnet', type : 'feature', value : 'auto',
> +   description: 'vmnet.framework network backend support')
>  option('virglrenderer', type : 'feature', value : 'auto',
> description: 'virgl rendering support')
>  option('vnc', type : 'feature', value : 'auto',
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index 50bd7bed4d..cdcece4b05 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -84,6 +84,7 @@ meson_options_help() {
>printf "%s\n" '  u2f U2F emulation support'
>printf "%s\n" '  usb-redir   libusbredir support'
>printf "%s\n" '  vde vde network backend support'
> +  printf "%s\n" '  vmnet   vmnet.framework network backend support'
>printf "%s\n" '  vhost-user-blk-server'
>printf "%s\n" '  build vhost-user-blk server'
>printf "%s\n" '  virglrenderer   virgl rendering support'
> @@ -248,6 +249,8 @@ _meson_option_parse() {
>  --disable-usb-redir) printf "%s" -Dusb_redir=disabled ;;
>  --enable-vde) printf "%s" -Dvde=enabled ;;
>  --disable-vde) printf "%s" -Dvde=disabled ;;
> +--enable-vmnet) printf "%s" -Dvmnet=enabled ;;
> +--disable-vmnet) printf "%s" -Dvmnet=disabled ;;
>  --enable-vhost-user-blk-server) printf "%s" 
> 

Re: [PATCH v10 0/7] Add vmnet.framework based network backend

[Roman Bolshakov]

On Wed, Jan 12, 2022 at 04:23:30PM +0300, Vladislav Yaroshchuk wrote:
> ср, 12 янв. 2022 г. в 11:22, Roman Bolshakov :
> 
> > On Wed, Jan 12, 2022 at 10:50:04AM +0300, Roman Bolshakov wrote:
> > > On Wed, Jan 12, 2022 at 12:14:15AM +0300, Vladislav Yaroshchuk wrote:
> > > > v9 -> v10
> > > >  - Disable vmnet feature for macOS < 11.0: add
> > > >vmnet.framework API probe into meson.build.
> > > >This fixes QEMU building on macOS < 11.0:
> > > >
> > >
> > > Hi Vladislav,
> > >
> > > What symbols are missing on Catalina except VMNET_SHARING_BUSY?
> > >
> > > It'd be great to get the feature working there.
> > >
> > > Thanks,
> > > Roman
> > >
> >
> > Ok it turned out not that many symbols are needed for successfull
> > compilation on Catalina:
> >
> > vmnet_enable_isolation_key
> > vmnet_network_identifier_key
> > VMNET_SHARING_SERVICE_BUSY
> >
> > The compilation suceeds if they're wrappeed by ifdefs. I haven't tested
> > it yet though.
> >
> >
> New version with Catalina 10.15 support submitted as v11.
> 

Thanks!

I appreciate that.

Regards,
Roman

Re: [PATCH v10 0/7] Add vmnet.framework based network backend

[Roman Bolshakov]

On Wed, Jan 12, 2022 at 10:50:04AM +0300, Roman Bolshakov wrote:
> On Wed, Jan 12, 2022 at 12:14:15AM +0300, Vladislav Yaroshchuk wrote:
> > macOS provides networking API for VMs called 'vmnet.framework':
> > https://developer.apple.com/documentation/vmnet
> > 
> > We can provide its support as the new QEMU network backends which
> > represent three different vmnet.framework interface usage modes:
> > 
> >   * `vmnet-shared`:
> > allows the guest to communicate with other guests in shared mode and
> > also with external network (Internet) via NAT. Has (macOS-provided)
> > DHCP server; subnet mask and IP range can be configured;
> > 
> >   * `vmnet-host`:
> > allows the guest to communicate with other guests in host mode.
> > By default has enabled DHCP as `vmnet-shared`, but providing
> > network unique id (uuid) can make `vmnet-host` interfaces isolated
> > from each other and also disables DHCP.
> > 
> >   * `vmnet-bridged`:
> > bridges the guest with a physical network interface.
> > 
> > This backends cannot work on macOS Catalina 10.15 cause we use
> > vmnet.framework API provided only with macOS 11 and newer. Seems
> > that it is not a problem, because QEMU guarantees to work on two most
> > recent versions of macOS which now are Big Sur (11) and Monterey (12).
> > 
> > Also, we have one inconvenient restriction: vmnet.framework interfaces
> > can create only privileged user:
> > `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> > 
> > Attempt of `vmnet-*` netdev creation being unprivileged user fails with
> > vmnet's 'general failure'.
> > 
> > This happens because vmnet.framework requires `com.apple.vm.networking`
> > entitlement which is: "restricted to developers of virtualization software.
> > To request this entitlement, contact your Apple representative." as Apple
> > documentation says:
> > https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
> > 
> > One more note: we still have quite useful but not supported
> > 'vmnet.framework' features as creating port forwarding rules, IPv6
> > NAT prefix specifying and so on.
> > 
> > Nevertheless, new backends work fine and tested within `qemu-system-x86-64`
> > on macOS Bir Sur 11.5.2 host with such nic models:
> >   * e1000-82545em
> >   * virtio-net-pci
> >   * vmxnet3
> > 
> > The guests were:
> >   * macOS 10.15.7
> >   * Ubuntu Bionic (server cloudimg)
> > 
> > 
> > This series partially reuses patches by Phillip Tennen:
> > https://patchew.org/QEMU/20210218134947.1860-1-phillip.en...@gmail.com/
> > So I included them signed-off line into one of the commit messages and
> > also here.
> > 
> > v1 -> v2:
> >  Since v1 minor typos were fixed, patches rebased onto latest master,
> >  redundant changes removed (small commits squashed)
> > v2 -> v3:
> >  - QAPI style fixes
> >  - Typos fixes in comments
> >  - `#include`'s updated to be in sync with recent master
> > v3 -> v4:
> >  - Support vmnet interfaces isolation feature
> >  - Support vmnet-host network uuid setting feature
> >  - Refactored sources a bit
> > v4 -> v5:
> >  - Missed 6.2 boat, now 7.0 candidate
> >  - Fix qapi netdev descriptions and styles
> >(@subnetmask -> @subnet-mask)
> >  - Support vmnet-shared IPv6 prefix setting feature
> > v5 -> v6
> >  - provide detailed commit messages for commits of
> >many changes
> >  - rename properties @dhcpstart and @dhcpend to
> >@start-address and @end-address
> >  - improve qapi documentation about isolation
> >features (@isolated, @net-uuid)
> > v6 -> v7:
> >  - update MAINTAINERS list
> > v7 -> v8
> >  - QAPI code style fixes
> > v8 -> v9
> >  - Fix building on Linux: add missing qapi
> >`'if': 'CONFIG_VMNET'` statement to Netdev union
> > v9 -> v10
> >  - Disable vmnet feature for macOS < 11.0: add
> >vmnet.framework API probe into meson.build.
> >This fixes QEMU building on macOS < 11.0:
> >https://patchew.org/QEMU/20220110034000.20221-1-jasow...@redhat.com/
> > 
> 
> Hi Vladislav,
> 
> What symbols are missing on Catalina except VMNET_SHARING_BUSY?
> 
> It'd be great to get the feature working there.
> 
> Thanks,
> Roman
> 

Ok it turned out not that many symbols are needed for successfull
compilation on Catalina:

vmnet_enable_isolation_key
vmnet_network_identifier_key
VMNET_SHARING_SERVICE_B

Re: [PATCH v10 0/7] Add vmnet.framework based network backend

[Roman Bolshakov]

On Wed, Jan 12, 2022 at 12:14:15AM +0300, Vladislav Yaroshchuk wrote:
> macOS provides networking API for VMs called 'vmnet.framework':
> https://developer.apple.com/documentation/vmnet
> 
> We can provide its support as the new QEMU network backends which
> represent three different vmnet.framework interface usage modes:
> 
>   * `vmnet-shared`:
> allows the guest to communicate with other guests in shared mode and
> also with external network (Internet) via NAT. Has (macOS-provided)
> DHCP server; subnet mask and IP range can be configured;
> 
>   * `vmnet-host`:
> allows the guest to communicate with other guests in host mode.
> By default has enabled DHCP as `vmnet-shared`, but providing
> network unique id (uuid) can make `vmnet-host` interfaces isolated
> from each other and also disables DHCP.
> 
>   * `vmnet-bridged`:
> bridges the guest with a physical network interface.
> 
> This backends cannot work on macOS Catalina 10.15 cause we use
> vmnet.framework API provided only with macOS 11 and newer. Seems
> that it is not a problem, because QEMU guarantees to work on two most
> recent versions of macOS which now are Big Sur (11) and Monterey (12).
> 
> Also, we have one inconvenient restriction: vmnet.framework interfaces
> can create only privileged user:
> `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> 
> Attempt of `vmnet-*` netdev creation being unprivileged user fails with
> vmnet's 'general failure'.
> 
> This happens because vmnet.framework requires `com.apple.vm.networking`
> entitlement which is: "restricted to developers of virtualization software.
> To request this entitlement, contact your Apple representative." as Apple
> documentation says:
> https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_vm_networking
> 
> One more note: we still have quite useful but not supported
> 'vmnet.framework' features as creating port forwarding rules, IPv6
> NAT prefix specifying and so on.
> 
> Nevertheless, new backends work fine and tested within `qemu-system-x86-64`
> on macOS Bir Sur 11.5.2 host with such nic models:
>   * e1000-82545em
>   * virtio-net-pci
>   * vmxnet3
> 
> The guests were:
>   * macOS 10.15.7
>   * Ubuntu Bionic (server cloudimg)
> 
> 
> This series partially reuses patches by Phillip Tennen:
> https://patchew.org/QEMU/20210218134947.1860-1-phillip.en...@gmail.com/
> So I included them signed-off line into one of the commit messages and
> also here.
> 
> v1 -> v2:
>  Since v1 minor typos were fixed, patches rebased onto latest master,
>  redundant changes removed (small commits squashed)
> v2 -> v3:
>  - QAPI style fixes
>  - Typos fixes in comments
>  - `#include`'s updated to be in sync with recent master
> v3 -> v4:
>  - Support vmnet interfaces isolation feature
>  - Support vmnet-host network uuid setting feature
>  - Refactored sources a bit
> v4 -> v5:
>  - Missed 6.2 boat, now 7.0 candidate
>  - Fix qapi netdev descriptions and styles
>(@subnetmask -> @subnet-mask)
>  - Support vmnet-shared IPv6 prefix setting feature
> v5 -> v6
>  - provide detailed commit messages for commits of
>many changes
>  - rename properties @dhcpstart and @dhcpend to
>@start-address and @end-address
>  - improve qapi documentation about isolation
>features (@isolated, @net-uuid)
> v6 -> v7:
>  - update MAINTAINERS list
> v7 -> v8
>  - QAPI code style fixes
> v8 -> v9
>  - Fix building on Linux: add missing qapi
>`'if': 'CONFIG_VMNET'` statement to Netdev union
> v9 -> v10
>  - Disable vmnet feature for macOS < 11.0: add
>vmnet.framework API probe into meson.build.
>This fixes QEMU building on macOS < 11.0:
>https://patchew.org/QEMU/20220110034000.20221-1-jasow...@redhat.com/
> 

Hi Vladislav,

What symbols are missing on Catalina except VMNET_SHARING_BUSY?

It'd be great to get the feature working there.

Thanks,
Roman

> Vladislav Yaroshchuk (7):
>   net/vmnet: add vmnet dependency and customizable option
>   net/vmnet: add vmnet backends to qapi/net
>   net/vmnet: implement shared mode (vmnet-shared)
>   net/vmnet: implement host mode (vmnet-host)
>   net/vmnet: implement bridged mode (vmnet-bridged)
>   net/vmnet: update qemu-options.hx
>   net/vmnet: update MAINTAINERS list
> 
>  MAINTAINERS   |   5 +
>  meson.build   |  16 +-
>  meson_options.txt |   2 +
>  net/clients.h |  11 ++
>  net/meson.build   |   7 +
>  net/net.c |  10 ++
>  net/vmnet-bridged.m   | 111 
>  net/vmnet-common.m| 330 ++
>  net/vmnet-host.c  | 105 +++
>  net/vmnet-shared.c|  92 ++
>  net/vmnet_int.h   |  48 +
>  qapi/net.json | 132 +-
>  qemu-options.hx   |  25 +++
>  scripts/meson-buildoptions.sh |   3 +
>  14 files changed, 894 insertions(+), 3 deletions(-)
>  create mode 

Re: [PULL 00/13] Net patches

[Roman Bolshakov]

On Wed, Jan 12, 2022 at 01:39:28PM +0800, Jason Wang wrote:
> 
> 在 2022/1/12 上午6:02, Vladislav Yaroshchuk 写道:
> > 
> > 
> > вт, 11 янв. 2022 г., 5:10 AM Jason Wang :
> > 
> > On Tue, Jan 11, 2022 at 12:49 AM Peter Maydell
> >  wrote:
> > >
> > > On Mon, 10 Jan 2022 at 03:40, Jason Wang 
> > wrote:
> > > >
> > > > The following changes since commit
> > df722e33d5da26ea8604500ca8f509245a0ea524:
> > > >
> > > >   Merge tag 'bsd-user-arm-pull-request' of
> > gitlab.com:bsdimp/qemu into staging (2022-01-08 09:37:59 -0800)
> > > >
> > > > are available in the git repository at:
> > > >
> > > > https://github.com/jasowang/qemu.git tags/net-pull-request
> > > >
> > > > for you to fetch changes up to
> > 5136cc6d3b8b74f4fa572f0874656947a401330e:
> > > >
> > > >   net/vmnet: update MAINTAINERS list (2022-01-10 11:30:55 +0800)
> > > >
> > > > 
> > > >
> > > > 
> > >
> > > Fails to build on OSX Catalina:
> > >
> > > ../../net/vmnet-common.m:165:10: error: use of undeclared identifier
> > > 'VMNET_SHARING_SERVICE_BUSY'
> > >     case VMNET_SHARING_SERVICE_BUSY:
> > >          ^
> > >
> > > This constant only got added in macOS 11.0. I guess that technically
> > > our supported-platforms policy only requires us to support 11
> > (Big Sur)
> > > and 12 (Monterey) at this point, but it would be nice to still
> > be able
> > > to build on Catalina (10.15).
> > 
> > Yes, it was only supported by the vmnet framework starting from
> > Catalyst according to
> > https://developer.apple.com/documentation/vmnet?language=objc.
> > 
> > 
> > Yes, there are some symbols from macOS >= 11.0 new backend
> > uses, not only this one, ex. vmnet_enable_isolation_key:
> > https://developer.apple.com/documentation/vmnet/vmnet_enable_isolation_key
> > 
> > >
> > > (Personally I would like Catalina still to work at least for a
> > little
> > > while, because my x86 Mac is old enough that it is not supported by
> > > Big Sur. I'll have to dump it once Apple stops doing security
> > support
> > > for Catalina, but they haven't done that quite yet.)
> > 
> > 
> > Sure, broken builds on old macOSes are bad. For this case I think
> > it's enough to disable vmnet for macOS < 11.0 with a probe while
> > configure build step. Especially given that Apple supports ~three
> > latest macOS versions, support for Catalina is expected to end
> > in 2022, when QEMU releases 7.0.
> 
> 
> That should be fine.
> 

I agree with Peter on this,

There's a lot of hardware running with Catalina. I think it's useful to
support it a little longer.

Regards,
Roman

> 
> > 
> > If this workaround is not suitable and it's required to support vmnet
> > in Catalina 10.15 with a subset of available features, it can be done.
> > But I'll be ready to handle this in approximately two-three weeks only.
> > 
> > Sure, Vladislav please fix this and send a new version.
> > 
> > 
> > Quick fix as described above is available in v10:
> > https://patchew.org/QEMU/20220111211422.21789-1-yaroshchuk2...@gmail.com/
> 
> 
> Have you got chance to test that for macOS < 11.0?
> 
> Thanks
> 
> 
> > Thanks
> > 
> > >
> > > -- PMM
> > >
> > 
> > 
> > 
> > 
> > -- 
> > Best Regards,
> > 
> > Vladislav Yaroshchuk
> 
> 

Re: [RFC PATCH v3 5/7] audio/coreaudio: Remove a deprecation warning on macOS 12

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:59PM +0100, Philippe Mathieu-Daudé wrote:
> When building on macOS 12 we get:
> 
>   audio/coreaudio.c:50:5: error: 'kAudioObjectPropertyElementMaster' is 
> deprecated: first deprecated in macOS 12.0 [-Werror,-Wdeprecated-declarations]
>   kAudioObjectPropertyElementMaster
>   ^
>   kAudioObjectPropertyElementMain
>   
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreAudio.framework/Headers/AudioHardwareBase.h:208:5:
>  note: 'kAudioObjectPropertyElementMaster' has been explicitly marked 
> deprecated here
>   kAudioObjectPropertyElementMaster 
> API_DEPRECATED_WITH_REPLACEMENT("kAudioObjectPropertyElementMain", 
> macos(10.0, 12.0), ios(2.0, 15.0), watchos(1.0, 8.0), tvos(9.0, 15.0)) = 
> kAudioObjectPropertyElementMain
>   ^
> 
> Replace by kAudioObjectPropertyElementMain, redefining it to
> kAudioObjectPropertyElementMaster if not available, using
> Clang __is_identifier() feature (coreaudio is restricted to
> macOS).
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
> Checkpatch:
> 
>  WARNING: architecture specific defines should be avoided
>  #10: FILE: audio/coreaudio.c:47:
>  +#if !__is_identifier(kAudioObjectPropertyElementMain) /* macOS >= 12.0 */
> 
> Should we define __is_identifier() to 0 for GCC on macOS?
> ---
>  audio/coreaudio.c | 16 ++--
>  1 file changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/audio/coreaudio.c b/audio/coreaudio.c
> index d8a21d3e507..73cbfd479ac 100644
> --- a/audio/coreaudio.c
> +++ b/audio/coreaudio.c
> @@ -44,10 +44,14 @@ typedef struct coreaudioVoiceOut {
>  bool enabled;
>  } coreaudioVoiceOut;
>  
> +#if !__is_identifier(kAudioObjectPropertyElementMain) /* macOS >= 12.0 */
> +#define kAudioObjectPropertyElementMain kAudioObjectPropertyElementMaster
> +#endif

Christian and Akihiko are right you need to replace it with macOS version
wrappers:

diff --git a/audio/coreaudio.c b/audio/coreaudio.c
index 73cbfd479a..7367a2ffd4 100644
--- a/audio/coreaudio.c
+++ b/audio/coreaudio.c
@@ -44,7 +44,8 @@ typedef struct coreaudioVoiceOut {
 bool enabled;
 } coreaudioVoiceOut;

-#if !__is_identifier(kAudioObjectPropertyElementMain) /* macOS >= 12.0 */
+#if !defined(MAC_OS_VERSION_12_0) || \
+(MAC_OS_X_VERSION_MAX_ALLOWED < MAC_OS_VERSION_12_0)
 #define kAudioObjectPropertyElementMain kAudioObjectPropertyElementMaster
 #endif


And in the patch 6 you'd do likewise:

diff --git a/block/file-posix.c b/block/file-posix.c
index 1d0512026c..c0038629a1 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -3325,7 +3325,8 @@ BlockDriver bdrv_file = {
 static kern_return_t GetBSDPath(io_iterator_t mediaIterator, char *bsdPath,
 CFIndex maxPathSize, int flags);

-#if !__is_identifier(IOMainPort) /* macOS >= 12.0 */
+#if !defined(MAC_OS_VERSION_12_0) || \
+(MAC_OS_X_VERSION_MAX_ALLOWED < MAC_OS_VERSION_12_0)
 #define IOMainPort IOMasterPort
 #endif

This way it the build would work also on older macOS.


Two more issues are left:

1. Linker has corrupted paths to clang directory (happens on all macOS 
versions).

Monterey:

[732/737] Linking target qemu-system-mips-unsigned
ld: warning: directory not found for option 
'-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
[733/737] Linking target qemu-system-mips64-unsigned
ld: warning: directory not found for option 
'-Lns/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0'
[737/737] Generating qemu-system-mips64 with a custom command

Catalina:

ld: warning: directory not found for option 
'-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
[102/105] Linking target qemu-system-or1k-unsigned
ld: warning: directory not found for option 
'-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
[104/105] Linking target qemu-system-ppc-unsigned
ld: warning: directory not found for option 
'-Lveloper/CommandLineTools/usr/lib/clang/11.0.0'
[105/105] Generating qemu-system-ppc with a custom command

2. QEMU tests show FENV_ACCESS warning on Monterey:


[409/771] Compiling C object 
tests/fp/libtestfloat.a.p/berkeley-testfloat-3_source_test_az_f128_rx.c.o
../tests/fp/berkeley-testfloat-3/source/test_az_f128_rx.c:49:14: warning: 
'#pragma FENV_ACCESS' is not supported on this target - ignored 
[-Wignored-pragmas]
#pragma STDC FENV_ACCESS ON
 ^
1 warning generated.
[410/771] Compiling C object 
tests/fp/libtestfloat.a.p/berkeley-testfloat-3_source_test_abcz_f128.c.o
../tests/fp/berkeley-testfloat-3/source/test_abcz_f128.c:48:14: warning: 
'#pragma FENV_ACCESS' is not supported on this target - ignored 
[-Wignored-pragmas]
#pragma STDC FENV_ACCESS ON
 ^
1 warning generated.

Regards,
Roman

> +
>  static const AudioObjectPropertyAddress voice_addr = {
>  kAudioHardwarePropertyDefaultOutputDevice,
>  kAudioObjectPropertyScopeGlobal,
> -

Re: [RFC PATCH v3 5/7] audio/coreaudio: Remove a deprecation warning on macOS 12

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:59PM +0100, Philippe Mathieu-Daudé wrote:
> When building on macOS 12 we get:
> 
>   audio/coreaudio.c:50:5: error: 'kAudioObjectPropertyElementMaster' is 
> deprecated: first deprecated in macOS 12.0 [-Werror,-Wdeprecated-declarations]
>   kAudioObjectPropertyElementMaster
>   ^
>   kAudioObjectPropertyElementMain
>   
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreAudio.framework/Headers/AudioHardwareBase.h:208:5:
>  note: 'kAudioObjectPropertyElementMaster' has been explicitly marked 
> deprecated here
>   kAudioObjectPropertyElementMaster 
> API_DEPRECATED_WITH_REPLACEMENT("kAudioObjectPropertyElementMain", 
> macos(10.0, 12.0), ios(2.0, 15.0), watchos(1.0, 8.0), tvos(9.0, 15.0)) = 
> kAudioObjectPropertyElementMain
>   ^
> 
> Replace by kAudioObjectPropertyElementMain, redefining it to
> kAudioObjectPropertyElementMaster if not available, using
> Clang __is_identifier() feature (coreaudio is restricted to
> macOS).
> 

As of now it breaks the build on Catalina/10.15:

FAILED: libcommon.fa.p/audio_coreaudio.c.o
cc <...>
../audio/coreaudio.c:54:5: error: use of undeclared identifier 
'kAudioObjectPropertyElementMain'; did you mean 
'kAudioObjectPropertyElementName'?
kAudioObjectPropertyElementMain
^~~
kAudioObjectPropertyElementName

But __is_identifier itself works... Weird.

> Signed-off-by: Philippe Mathieu-Daudé 
> ---
> Checkpatch:
> 
>  WARNING: architecture specific defines should be avoided
>  #10: FILE: audio/coreaudio.c:47:
>  +#if !__is_identifier(kAudioObjectPropertyElementMain) /* macOS >= 12.0 */
> 
> Should we define __is_identifier() to 0 for GCC on macOS?

Clang documentation has this snippet:

#ifdef __is_identifier  // Compatibility with non-clang compilers.
  #if __is_identifier(__wchar_t)
typedef wchar_t __wchar_t;
  #endif
#endif

We can also add ifdef around just to be nice to GCC if it ever comes back on
macOS :)

Regards,
Roman

> ---
>  audio/coreaudio.c | 16 ++--
>  1 file changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/audio/coreaudio.c b/audio/coreaudio.c
> index d8a21d3e507..73cbfd479ac 100644
> --- a/audio/coreaudio.c
> +++ b/audio/coreaudio.c
> @@ -44,10 +44,14 @@ typedef struct coreaudioVoiceOut {
>  bool enabled;
>  } coreaudioVoiceOut;
>  
> +#if !__is_identifier(kAudioObjectPropertyElementMain) /* macOS >= 12.0 */
> +#define kAudioObjectPropertyElementMain kAudioObjectPropertyElementMaster
> +#endif
> +
>  static const AudioObjectPropertyAddress voice_addr = {
>  kAudioHardwarePropertyDefaultOutputDevice,
>  kAudioObjectPropertyScopeGlobal,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  static OSStatus coreaudio_get_voice(AudioDeviceID *id)
> @@ -69,7 +73,7 @@ static OSStatus coreaudio_get_framesizerange(AudioDeviceID 
> id,
>  AudioObjectPropertyAddress addr = {
>  kAudioDevicePropertyBufferFrameSizeRange,
>  kAudioDevicePropertyScopeOutput,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  return AudioObjectGetPropertyData(id,
> @@ -86,7 +90,7 @@ static OSStatus coreaudio_get_framesize(AudioDeviceID id, 
> UInt32 *framesize)
>  AudioObjectPropertyAddress addr = {
>  kAudioDevicePropertyBufferFrameSize,
>  kAudioDevicePropertyScopeOutput,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  return AudioObjectGetPropertyData(id,
> @@ -103,7 +107,7 @@ static OSStatus coreaudio_set_framesize(AudioDeviceID id, 
> UInt32 *framesize)
>  AudioObjectPropertyAddress addr = {
>  kAudioDevicePropertyBufferFrameSize,
>  kAudioDevicePropertyScopeOutput,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  return AudioObjectSetPropertyData(id,
> @@ -121,7 +125,7 @@ static OSStatus coreaudio_set_streamformat(AudioDeviceID 
> id,
>  AudioObjectPropertyAddress addr = {
>  kAudioDevicePropertyStreamFormat,
>  kAudioDevicePropertyScopeOutput,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  return AudioObjectSetPropertyData(id,
> @@ -138,7 +142,7 @@ static OSStatus coreaudio_get_isrunning(AudioDeviceID id, 
> UInt32 *result)
>  AudioObjectPropertyAddress addr = {
>  kAudioDevicePropertyDeviceIsRunning,
>  kAudioDevicePropertyScopeOutput,
> -kAudioObjectPropertyElementMaster
> +kAudioObjectPropertyElementMain
>  };
>  
>  return AudioObjectGetPropertyData(id,
> -- 
> 2.33.1

Re: [RFC PATCH v3 4/7] hvf: Remove deprecated hv_vcpu_flush() calls

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:58PM +0100, Philippe Mathieu-Daudé wrote:
> When building on macOS 12, we get:
> 
>   In file included from ../target/i386/hvf/hvf.c:59:
>   ../target/i386/hvf/vmx.h:174:5: error: 'hv_vcpu_flush' is deprecated: first 
> deprecated in macOS 11.0 - This API has no effect and always returns 
> HV_UNSUPPORTED [-Werror,-Wdeprecated-declarations]
>   hv_vcpu_flush(vcpu);
>   ^

This seems to be true even for older macOS (e.g. Catalina).

>   
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Hypervisor.framework/Headers/hv.h:364:20:
>  note: 'hv_vcpu_flush' has been explicitly marked deprecated here
>   extern hv_return_t hv_vcpu_flush(hv_vcpuid_t vcpu)
>  ^
> 
> Since this call "has no effect", simply remove it ¯\_(ツ)_/¯
> 
> Not very useful deprecation doc:
> https://developer.apple.com/documentation/hypervisor/1441386-hv_vcpu_flush
> 

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Thanks,
Roman

> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/i386/hvf/vmx.h  | 2 --
>  target/i386/hvf/x86_task.c | 1 -
>  target/i386/hvf/x86hvf.c   | 2 --
>  3 files changed, 5 deletions(-)
> 
> diff --git a/target/i386/hvf/vmx.h b/target/i386/hvf/vmx.h
> index 6df87116f62..094fb9b9dc9 100644
> --- a/target/i386/hvf/vmx.h
> +++ b/target/i386/hvf/vmx.h
> @@ -159,7 +159,6 @@ static inline void macvm_set_cr0(hv_vcpuid_t vcpu, 
> uint64_t cr0)
>  wvmcs(vcpu, VMCS_GUEST_CR0, cr0 | CR0_NE | CR0_ET);
>  
>  hv_vcpu_invalidate_tlb(vcpu);
> -hv_vcpu_flush(vcpu);
>  }
>  
>  static inline void macvm_set_cr4(hv_vcpuid_t vcpu, uint64_t cr4)
> @@ -171,7 +170,6 @@ static inline void macvm_set_cr4(hv_vcpuid_t vcpu, 
> uint64_t cr4)
>  wvmcs(vcpu, VMCS_CR4_MASK, CR4_VMXE);
>  
>  hv_vcpu_invalidate_tlb(vcpu);
> -hv_vcpu_flush(vcpu);
>  }
>  
>  static inline void macvm_set_rip(CPUState *cpu, uint64_t rip)
> diff --git a/target/i386/hvf/x86_task.c b/target/i386/hvf/x86_task.c
> index 422156128b7..c8dc3d48fa8 100644
> --- a/target/i386/hvf/x86_task.c
> +++ b/target/i386/hvf/x86_task.c
> @@ -181,5 +181,4 @@ void vmx_handle_task_switch(CPUState *cpu, 
> x68_segment_selector tss_sel, int rea
>  store_regs(cpu);
>  
>  hv_vcpu_invalidate_tlb(cpu->hvf->fd);
> -hv_vcpu_flush(cpu->hvf->fd);
>  }
> diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c
> index 907f09f1b43..bec9fc58146 100644
> --- a/target/i386/hvf/x86hvf.c
> +++ b/target/i386/hvf/x86hvf.c
> @@ -125,8 +125,6 @@ static void hvf_put_segments(CPUState *cpu_state)
>  
>  hvf_set_segment(cpu_state, , >ldt, false);
>  vmx_write_segment_descriptor(cpu_state, , R_LDTR);
> -
> -hv_vcpu_flush(cpu_state->hvf->fd);
>  }
>  
>  void hvf_put_msrs(CPUState *cpu_state)
> -- 
> 2.33.1

Re: [RFC PATCH v3 1/7] configure: Allow passing extra Objective C compiler flags

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:55PM +0100, Philippe Mathieu-Daudé wrote:
> We can pass C/CPP/LD flags via CFLAGS/CXXFLAGS/LDFLAGS environment
> variables, or via configure --extra-cflags / --extra-cxxflags /
> --extra-ldflags options. Provide similar behavior for Objective C:
> use existing flags from $OBJCFLAGS, or passed via --extra-objcflags.
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  configure   | 8 
>  meson.build | 5 +
>  2 files changed, 13 insertions(+)
> 
> diff --git a/configure b/configure
> index 0c57a063c66..adb42d8beb1 100755
> --- a/configure
> +++ b/configure
> @@ -288,6 +288,7 @@ done
>  
>  EXTRA_CFLAGS=""
>  EXTRA_CXXFLAGS=""
> +EXTRA_OBJCFLAGS=""
>  EXTRA_LDFLAGS=""
>  
>  xen_ctrl_version="$default_feature"
> @@ -400,9 +401,12 @@ for opt do
>--extra-cflags=*)
>  EXTRA_CFLAGS="$EXTRA_CFLAGS $optarg"
>  EXTRA_CXXFLAGS="$EXTRA_CXXFLAGS $optarg"
> +EXTRA_OBJCFLAGS="$EXTRA_OBJCFLAGS $optarg"
>  ;;
>--extra-cxxflags=*) EXTRA_CXXFLAGS="$EXTRA_CXXFLAGS $optarg"
>;;
> +  --extra-objcflags=*) EXTRA_OBJCFLAGS="$EXTRA_OBJCFLAGS $optarg"
> +  ;;
>--extra-ldflags=*) EXTRA_LDFLAGS="$EXTRA_LDFLAGS $optarg"
>;;
>--enable-debug-info) debug_info="yes"
> @@ -781,6 +785,8 @@ for opt do
>;;
>--extra-cxxflags=*)
>;;
> +  --extra-objcflags=*)
> +  ;;
>--extra-ldflags=*)
>;;
>--enable-debug-info)
> @@ -1318,6 +1324,7 @@ Advanced options (experts only):
>--objcc=OBJCCuse Objective-C compiler OBJCC [$objcc]
>--extra-cflags=CFLAGSappend extra C compiler flags CFLAGS
>--extra-cxxflags=CXXFLAGS append extra C++ compiler flags CXXFLAGS
> +  --extra-objcflags=OBJCFLAGS append extra Objective C compiler flags 
> OBJCFLAGS
>--extra-ldflags=LDFLAGS  append extra linker flags LDFLAGS
>--cross-cc-ARCH=CC   use compiler when building ARCH guest test cases
>--cross-cc-flags-ARCH=   use compiler flags when building ARCH guest tests
> @@ -3843,6 +3850,7 @@ if test "$skip_meson" = no; then
>echo "[built-in options]" >> $cross
>echo "c_args = [$(meson_quote $CFLAGS $EXTRA_CFLAGS)]" >> $cross
>echo "cpp_args = [$(meson_quote $CXXFLAGS $EXTRA_CXXFLAGS)]" >> $cross
> +  test -n "$objcc" && echo "objc_args = [$(meson_quote $OBJCFLAGS 
> $EXTRA_OBJCFLAGS)]" >> $cross
>echo "c_link_args = [$(meson_quote $CFLAGS $LDFLAGS $EXTRA_CFLAGS 
> $EXTRA_LDFLAGS)]" >> $cross
>echo "cpp_link_args = [$(meson_quote $CXXFLAGS $LDFLAGS $EXTRA_CXXFLAGS 
> $EXTRA_LDFLAGS)]" >> $cross
>echo "[binaries]" >> $cross
> diff --git a/meson.build b/meson.build
> index 0e52f54b100..a21305d62c1 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -3286,6 +3286,11 @@
> + ['-O' + 
> get_option('optimization')]
> + (get_option('debug') ? 
> ['-g'] : []))}
>  endif
> +if targetos == 'darwin'
> +  summary_info += {'OBJCFLAGS':   ' '.join(get_option('objc_args')
> +   + ['-O' + 
> get_option('optimization')]
> +   + (get_option('debug') ? 
> ['-g'] : []))}

Hi Philippe,

You need to add something like below to actually use the flags in build:

add_global_arguments(config_host['QEMU_OBJCFLAGS'].split(),
 native: false, language: 'objc')

Regards,
Roman

> +endif
>  link_args = get_option(link_language + '_link_args')
>  if link_args.length() > 0
>summary_info += {'LDFLAGS': ' '.join(link_args)}
> -- 
> 2.33.1

Re: [RFC PATCH v3 3/7] hvf: Make hvf_get_segments() / hvf_put_segments() local

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:57PM +0100, Philippe Mathieu-Daudé wrote:
> Both hvf_get_segments/hvf_put_segments() functions are only
> used within x86hvf.c: do not declare them as public API.
> 

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Thanks,
Roman

> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  target/i386/hvf/x86hvf.h | 2 --
>  target/i386/hvf/x86hvf.c | 4 ++--
>  2 files changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/target/i386/hvf/x86hvf.h b/target/i386/hvf/x86hvf.h
> index 99ed8d608dd..db6003d6bda 100644
> --- a/target/i386/hvf/x86hvf.h
> +++ b/target/i386/hvf/x86hvf.h
> @@ -26,11 +26,9 @@ void hvf_set_segment(struct CPUState *cpu, struct 
> vmx_segment *vmx_seg,
>   SegmentCache *qseg, bool is_tr);
>  void hvf_get_segment(SegmentCache *qseg, struct vmx_segment *vmx_seg);
>  void hvf_put_xsave(CPUState *cpu_state);
> -void hvf_put_segments(CPUState *cpu_state);
>  void hvf_put_msrs(CPUState *cpu_state);
>  void hvf_get_xsave(CPUState *cpu_state);
>  void hvf_get_msrs(CPUState *cpu_state);
>  void vmx_clear_int_window_exiting(CPUState *cpu);
> -void hvf_get_segments(CPUState *cpu_state);
>  void vmx_update_tpr(CPUState *cpu);
>  #endif
> diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c
> index 05ec1bddc4e..907f09f1b43 100644
> --- a/target/i386/hvf/x86hvf.c
> +++ b/target/i386/hvf/x86hvf.c
> @@ -83,7 +83,7 @@ void hvf_put_xsave(CPUState *cpu_state)
>  }
>  }
>  
> -void hvf_put_segments(CPUState *cpu_state)
> +static void hvf_put_segments(CPUState *cpu_state)
>  {
>  CPUX86State *env = _CPU(cpu_state)->env;
>  struct vmx_segment seg;
> @@ -166,7 +166,7 @@ void hvf_get_xsave(CPUState *cpu_state)
>  x86_cpu_xrstor_all_areas(X86_CPU(cpu_state), xsave, xsave_len);
>  }
>  
> -void hvf_get_segments(CPUState *cpu_state)
> +static void hvf_get_segments(CPUState *cpu_state)
>  {
>  CPUX86State *env = _CPU(cpu_state)->env;
>  
> -- 
> 2.33.1

Re: [RFC PATCH v3 2/7] ui/cocoa: Remove allowedFileTypes restriction in SavePanel

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:56PM +0100, Philippe Mathieu-Daudé wrote:
> setAllowedFileTypes is deprecated in macOS 12.
> 
> Per Akihiko Odaki [*]:
> 
>   An image file, which is being chosen by the panel, can be a
>   raw file and have a variety of file extensions and many are not
>   covered by the provided list (e.g. "udf"). Other platforms like
>   GTK can provide an option to open a file with an extension not
>   listed, but Cocoa can't.
>
> It forces the user to rename the file
>   to give an extension in the list. Moreover, Cocoa does not tell
>   which extensions are in the list so the user needs to read the
>   source code, which is pretty bad.
> 
> Since this code is harming the usability rather than improving it,
> simply remove the [NSSavePanel allowedFileTypes:] call, fixing:
> 

Yes, it is an issue for raw images with extensions outside of the
specified list or for images without any extension.

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Regards,
-Roman

>   [2789/6622] Compiling Objective-C object libcommon.fa.p/ui_cocoa.m.o
>   ui/cocoa.m:1411:16: error: 'setAllowedFileTypes:' is deprecated: first 
> deprecated in macOS 12.0 - Use -allowedContentTypes instead 
> [-Werror,-Wdeprecated-declarations]
>   [openPanel setAllowedFileTypes: supportedImageFileTypes];
>  ^
>   
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSSavePanel.h:215:49:
>  note: property 'allowedFileTypes' is declared deprecated here
>   @property (nullable, copy) NSArray *allowedFileTypes 
> API_DEPRECATED("Use -allowedContentTypes instead", macos(10.3,12.0));
>   ^
>   
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSSavePanel.h:215:49:
>  note: 'setAllowedFileTypes:' has been explicitly marked deprecated here
>   FAILED: libcommon.fa.p/ui_cocoa.m.o
> 
> [*] 
> https://lore.kernel.org/qemu-devel/4dde2e66-63cb-4390-9538-c032310db...@gmail.com/
> 
> Suggested-by: Akihiko Odaki 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  ui/cocoa.m | 6 --
>  1 file changed, 6 deletions(-)
> 
> diff --git a/ui/cocoa.m b/ui/cocoa.m
> index 69745c483b4..dec22968815 100644
> --- a/ui/cocoa.m
> +++ b/ui/cocoa.m
> @@ -100,7 +100,6 @@ static void cocoa_switch(DisplayChangeListener *dcl,
>  static char **gArgv;
>  static bool stretch_video;
>  static NSTextField *pauseLabel;
> -static NSArray * supportedImageFileTypes;
>  
>  static QemuSemaphore display_init_sem;
>  static QemuSemaphore app_started_sem;
> @@ -1162,10 +1161,6 @@ - (id) init
>  [pauseLabel setTextColor: [NSColor blackColor]];
>  [pauseLabel sizeToFit];
>  
> -// set the supported image file types that can be opened
> -supportedImageFileTypes = [NSArray arrayWithObjects: @"img", @"iso", 
> @"dmg",
> - @"qcow", @"qcow2", @"cloop", @"vmdk", 
> @"cdr",
> -  @"toast", nil];
>  [self make_about_window];
>  }
>  return self;
> @@ -1408,7 +1403,6 @@ - (void)changeDeviceMedia:(id)sender
>  openPanel = [NSOpenPanel openPanel];
>  [openPanel setCanChooseFiles: YES];
>  [openPanel setAllowsMultipleSelection: NO];
> -[openPanel setAllowedFileTypes: supportedImageFileTypes];
>  if([openPanel runModal] == NSModalResponseOK) {
>  NSString * file = [[[openPanel URLs] objectAtIndex: 0] path];
>  if(file == nil) {
> -- 
> 2.33.1
> 

Re: [RFC PATCH v3 0/7] host: Support macOS 12

[Roman Bolshakov]

On Mon, Jan 10, 2022 at 02:09:54PM +0100, Philippe Mathieu-Daudé wrote:
> Few patches to be able to build QEMU on macOS 12 (Monterey).
> 
> This basically consists of adapting deprecated APIs. I am not
> sure about these APIs, so tagging as RFC.
> 
> I couldn't succeed to adapt the Cocoa code.
> 
> CI job added to avoid bitrotting.
> 
> Since v2:
> - Addressed Akihiko Odaki comments:
>   . use __is_identifier(),
>   . remove cocoa setAllowedFileTypes()
> - Addressed Daniel Berrangé comment:
>   . rebased on testing/next, update libvirt-ci/lcitool
> 
> Based on Alex's testing/next

Hi Philippe,

Could you please share URI to the remote?
I want to apply the series on it.

Thanks,
Roman

> Based-on: <20220110124638.610145-1-f4...@amsat.org>
> 
> Philippe Mathieu-Daudé (7):
>   configure: Allow passing extra Objective C compiler flags
>   ui/cocoa: Remove allowedFileTypes restriction in SavePanel
>   hvf: Make hvf_get_segments() / hvf_put_segments() local
>   hvf: Remove deprecated hv_vcpu_flush() calls
>   audio/coreaudio: Remove a deprecation warning on macOS 12
>   block/file-posix: Remove a deprecation warning on macOS 12
>   gitlab-ci: Support macOS 12 via cirrus-run
> 
>  configure |  8 
>  meson.build   |  5 +
>  target/i386/hvf/vmx.h |  2 --
>  target/i386/hvf/x86hvf.h  |  2 --
>  audio/coreaudio.c | 16 ++--
>  block/file-posix.c| 13 +
>  target/i386/hvf/x86_task.c|  1 -
>  target/i386/hvf/x86hvf.c  |  6 ++
>  .gitlab-ci.d/cirrus.yml   | 15 +++
>  .gitlab-ci.d/cirrus/macos-12.vars | 16 
>  tests/lcitool/libvirt-ci  |  2 +-
>  tests/lcitool/refresh |  1 +
>  ui/cocoa.m|  6 --
>  13 files changed, 67 insertions(+), 26 deletions(-)
>  create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars
> 
> -- 
> 2.33.1
> 

Re: [PATCH 0/7] Add vmnet.framework based network backend

[Roman Bolshakov]

On Thu, Jun 17, 2021 at 05:32:39PM +0300, Vladislav Yaroshchuk wrote:
> macOS provides networking API for VMs called vmnet.framework.
> I tried to add it as a network backend. All three modes are supported:
> 
> -shared:
>   allows the guest to comminicate with other guests in shared mode and
>   also with external network (Internet) via NAT
> 
> -host:
>   allows the guest to communicate with other guests in host mode
> 
> -bridged:
>   bridges the guest with a physical network interface
> 
> Separate netdev for each vmnet mode was created because they use quite
> different settings, especially since macOS 11.0 when vmnet.framework
> gets a lot of updates.
> 
> Not sure that I use qemu_mutex_lock_iothread() and
> qemu_mutex_unlock_iothread() in correct way while sending packet
> from vmnet interface to QEMU. I'll be happy to receive
> recomendations how to make this thing better if I done sth wrong.
> 
> Also vmnet.framework requires com.apple.vm.networking entitlement to
> run without root priveledges. Ad-hoc signing does not fit there,
> so I didn't touch anything related to signing. As a result we should
> run qemu-system by a priviledged user:
> `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> otherwise vmnet fails with 'general failure'.
> 
> But in any way it seems working now,
> I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
> models:
> - e1000-82545em
> - virtio-net-pci
> 
> and having such guests:
> - macOS 10.15.7
> - Ubuntu Bionic (server cloudimg) 
> 

Hi Vladislav,

I appreciate the efforts and I'm sorry I didn't look into it yet, lack
of time :(

To all: earlier this year another series was sent by Phillip Tennen to
add vmnet.framework and some comments were provided:
https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg05874.html

I'm not sure how to proceed with arbitration which of the series is
preferred. FIFO or LIFO?

Regards,
Roman

> Vladislav Yaroshchuk (7):
>   net/vmnet: dependencies setup, initial preparations
>   net/vmnet: add new netdevs to qapi/net
>   net/vmnet: create common netdev state structure
>   net/vmnet: implement shared mode (vmnet-shared)
>   net/vmnet: implement host mode (vmnet-host)
>   net/vmnet: implement bridged mode (vmnet-bridged)
>   net/vmnet: update qemu-options.hx
> 
>  configure   |  31 +
>  meson.build |   5 +
>  net/clients.h   |  11 ++
>  net/meson.build |   7 ++
>  net/net.c   |  10 ++
>  net/vmnet-bridged.m | 123 ++
>  net/vmnet-common.m  | 294 
>  net/vmnet-host.c|  93 ++
>  net/vmnet-shared.c  |  94 ++
>  net/vmnet_int.h |  48 
>  qapi/net.json   |  99 ++-
>  qemu-options.hx |  17 +++
>  12 files changed, 830 insertions(+), 2 deletions(-)
>  create mode 100644 net/vmnet-bridged.m
>  create mode 100644 net/vmnet-common.m
>  create mode 100644 net/vmnet-host.c
>  create mode 100644 net/vmnet-shared.c
>  create mode 100644 net/vmnet_int.h
> 
> -- 
> 2.23.0
> 

[Bug 1914849] Re: mprotect fails after MacOS 11.2 on arm mac

[Roman Bolshakov]

** Changed in: qemu
   Status: Confirmed => Fix Committed

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1914849

Title:
  mprotect fails after MacOS 11.2 on arm mac

Status in QEMU:
  Fix Committed

Bug description:
  I got the following error when I ran qemu on arm mac(MacOS 11.2).

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

  I tested the same version of qemu on intel mac(MacOS 11.2), but it
  works fine.

  And my friend told me that they did not have this error with MacOS
  11.1.

  So, I think it is CPU architecture or an OS version dependent error.

  
  Environment:

  Qemu commit id: d0dddab40e472ba62b5f43f11cc7dba085dabe71
  OS: MacOS 11.2(20D64)
  Hardware: MacBook Air (M1, 2020)

  
  How to build:

  ```
  mkdir build/
  cd build/
  ../configure --target-list=aarch64-softmmu,x86_64-softmmu
  make
  ```

  
  How to reproduce:

  ```
  ./qemu-system-x86_64
  ```

  
  Error message:

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1914849/+subscriptions

Re: [PATCH v3 for-6.0 1/2] tcg: Do not set guard pages on the rx portion of code_gen_buffer

[Roman Bolshakov]

On Sat, Mar 20, 2021 at 10:57:19AM -0600, Richard Henderson wrote:
> The rw portion of the buffer is the only one in which overruns
> can be generated.  Allow the rx portion to be more completely
> covered by huge pages.
> 
> Signed-off-by: Richard Henderson 
> ---
>  tcg/tcg.c | 12 +---
>  1 file changed, 5 insertions(+), 7 deletions(-)
> 
> diff --git a/tcg/tcg.c b/tcg/tcg.c
> index de91bb6e9e..88c9e6f8a4 100644
> --- a/tcg/tcg.c
> +++ b/tcg/tcg.c
> @@ -828,7 +828,6 @@ void tcg_region_init(void)
>  size_t region_size;
>  size_t n_regions;
>  size_t i;
> -uintptr_t splitwx_diff;
>  
>  n_regions = tcg_n_regions();
>  
> @@ -858,8 +857,11 @@ void tcg_region_init(void)
>  /* account for that last guard page */
>  region.end -= page_size;
>  
> -/* set guard pages */
> -splitwx_diff = tcg_splitwx_diff;
> +/*
> + * Set guard pages in the rw buffer, as that's the one into which
> + * buffer overruns could occur.  Do not set guard pages in the rx
> + * buffer -- let that one use hugepages throughout.
> + */
>  for (i = 0; i < region.n; i++) {
>  void *start, *end;
>  int rc;
> @@ -867,10 +869,6 @@ void tcg_region_init(void)
>  tcg_region_bounds(i, , );
>  rc = qemu_mprotect_none(end, page_size);
>  g_assert(!rc);
> -if (splitwx_diff) {
> -rc = qemu_mprotect_none(end + splitwx_diff, page_size);
> -g_assert(!rc);
> -}
>  }
>  
>  tcg_region_trees_init();
> -- 
> 2.25.1
> 

Thanks for fixing the issue, Richard,

I have two questions:
 - Should we keep guards pages for rx on all platforms except darwin?
   (that would make it similar to what Philippe proposed in the comments
   to patch 2).
 - What does mean that rx might be covered by huge pages? (perhaps I'm
   missing some context)

Otherwise,

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

BR,
Roman

Re: [PATCH v3 for-6.0 2/2] tcg: Workaround macOS 11.2 mprotect bug

[Roman Bolshakov]

On Mon, Mar 22, 2021 at 11:03:05AM +0100, Philippe Mathieu-Daudé wrote:
> On 3/20/21 5:57 PM, Richard Henderson wrote:
> > There's a change in mprotect() behaviour [1] in the latest macOS
> > on M1 and it's not yet clear if it's going to be fixed by Apple.
> > 
> > As a short-term fix, ignore failures setting up the guard pages.
> > 
> > [1] https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f
> > 
> > Buglink: https://bugs.launchpad.net/qemu/+bug/1914849
> > Signed-off-by: Richard Henderson 
> > ---
> >  tcg/tcg.c | 10 +++---
> >  1 file changed, 7 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tcg/tcg.c b/tcg/tcg.c
> > index 88c9e6f8a4..1fbe0b686d 100644
> > --- a/tcg/tcg.c
> > +++ b/tcg/tcg.c
> > @@ -864,11 +864,15 @@ void tcg_region_init(void)
> >   */
> >  for (i = 0; i < region.n; i++) {
> >  void *start, *end;
> > -int rc;
> >  
> >  tcg_region_bounds(i, , );
> > -rc = qemu_mprotect_none(end, page_size);
> 
> What about:
> 
> #ifdef CONFIG_DARWIN
> 
>/* ... */
>(void)rc;
> #else
> 
> > -g_assert(!rc);
> 
> #endif
> 
> > +
> > +/*
> > + * macOS 11.2 has a bug (Apple Feedback FB8994773) in which 
> > mprotect
> > + * rejects a permission change from RWX -> NONE.  Guard pages are
> > + * nice for bug detection but are not essential; ignore any 
> > failure.
> > +     */
> > +(void)qemu_mprotect_none(end, page_size);
> >  }
> >  
> >  tcg_region_trees_init();
> > 
> 

I agree with Philippe, it's worth to keep the bug detection on non-buggy
platforms.

Otherwise:

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Thanks,
Roman

Re: [PATCH 2/2] cirrus.yml: Update the FreeBSD task to version 12.2

[Roman Bolshakov]

On Wed, Mar 17, 2021 at 01:44:05PM +0100, Thomas Huth wrote:
> On 17/03/2021 12.16, Peter Maydell wrote:
> > On Wed, 17 Mar 2021 at 11:09, Thomas Huth  wrote:
> > > 
> > > FreeBSD version 12.1 is out of service now, and the task in the
> > > Cirrus-CI is failing. Update to 12.2 to get it working again.
> > > Unfortunately, there is a bug in libtasn1 that triggers with the
> > > new version of Clang that is used there (see this thread for details:
> > > https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg00739.html ),
> > > so we have to disable gnutls for now to make it work again. We can
> > > enable it later again once libtasn1 has been fixed in FreeBSD.
> > > 
> > > Signed-off-by: Thomas Huth 
> > 
> > Is it feasible to make configure check for "is libtasn1 broken"?
> > I guess since it only shows up as "when you try to use it
> > it fails" that would be a runtime check rather than compile
> > time, which isn't really possible :-(
> 
> I don't really have a clue about this crypto stuff... Daniel, Stefan, Roman,
> Christian, Eric ... you debugged the original problem on macOS, do you think
> it's possible to add a check for this libtasn1 problem to our "configure"
> (or meson.build file)?
> 

Hi,

We need to define an ASN.1 object

https://gitlab.com/gnutls/libtasn1/-/blob/master/tests/Test_tree.c#L230

  {ACT_CREATE, "TEST_TREE.OidAndTimeTest", 0, 0, ASN1_SUCCESS, __LINE__},

The object is:
OidAndTimeTest ::= SEQUENCE{
   set SET OF INTEGER,
   oid OBJECT IDENTIFIER,
   time2   GeneralizedTime,
   bol BOOLEAN,
   oct OCTET STRING,
   bit BIT STRING OPTIONAL,
   bol2BOOLEAN DEFAULT TRUE,
   enumENUMERATED {v1(1),v2(2)} DEFAULT v1,
   any [1] ANY OPTIONAL,
   gen GeneralString OPTIONAL,
   time1   UTCTime
}

Create it with:

asn1_create_element (definitions, "TEST.OidAndTimeTest", _element);

and try to get it's DER length to mimic the part of the failing test:

https://gitlab.com/gnutls/libtasn1/-/blob/master/tests/Test_tree.c#L254

  result = asn1_der_coding (asn1_element, "", NULL, _len, errorDescription);

The result should be ASN1_MEM_ERROR.

Something like this should work as configure-time test.

Thanks,
Roman

Re: [PATCH v2 03/29] tcg: Re-order tcg_region_init vs tcg_prologue_init

[Roman Bolshakov]

On Sun, Mar 14, 2021 at 03:26:58PM -0600, Richard Henderson wrote:
> Instead of delaying tcg_region_init until after tcg_prologue_init
> is complete, do tcg_region_init first and let tcg_prologue_init
> shrink the first region by the size of the generated prologue.
> 
> Signed-off-by: Richard Henderson 
> ---
>  accel/tcg/tcg-all.c   | 11 -
>  accel/tcg/translate-all.c |  3 +++
>  bsd-user/main.c   |  1 -
>  linux-user/main.c |  1 -
>  tcg/tcg.c | 52 ++-
>  5 files changed, 22 insertions(+), 46 deletions(-)
> 
> diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
> index e378c2db73..f132033999 100644
> --- a/accel/tcg/tcg-all.c
> +++ b/accel/tcg/tcg-all.c
> @@ -111,17 +111,6 @@ static int tcg_init(MachineState *ms)
>  
>  tcg_exec_init(s->tb_size * 1024 * 1024, s->splitwx_enabled);
>  mttcg_enabled = s->mttcg_enabled;
> -
> -/*
> - * Initialize TCG regions only for softmmu.
> - *
> - * This needs to be done later for user mode, because the prologue
> - * generation needs to be delayed so that GUEST_BASE is already set.
> - */
> -#ifndef CONFIG_USER_ONLY
> -tcg_region_init();

Note that tcg_region_init() invokes tcg_n_regions() that depends on
qemu_tcg_mttcg_enabled() that evaluates mttcg_enabled. Likely you need
to move "mttcg_enabled = s->mttcg_enabled;" before tcg_exec_init() to
keep existing behaviour.

> -#endif /* !CONFIG_USER_ONLY */
> -
>  return 0;
>  }
>  
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index f32df8b240..b9057567f4 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -1339,6 +1339,9 @@ void tcg_exec_init(unsigned long tb_size, int splitwx)
> splitwx, _fatal);
>  assert(ok);
>  
> +/* TODO: allocating regions is hand-in-glove with code_gen_buffer. */
> +tcg_region_init();
> +
>  #if defined(CONFIG_SOFTMMU)
>  /* There's no guest base to take into account, so go ahead and
> initialize the prologue now.  */
> diff --git a/bsd-user/main.c b/bsd-user/main.c
> index 798aba512c..3669d2b89e 100644
> --- a/bsd-user/main.c
> +++ b/bsd-user/main.c
> @@ -994,7 +994,6 @@ int main(int argc, char **argv)
> generating the prologue until now so that the prologue can take
> the real value of GUEST_BASE into account.  */
>  tcg_prologue_init(tcg_ctx);
> -tcg_region_init();
>  
>  /* build Task State */
>  memset(ts, 0, sizeof(TaskState));
> diff --git a/linux-user/main.c b/linux-user/main.c
> index 4f4746dce8..1bc48ca954 100644
> --- a/linux-user/main.c
> +++ b/linux-user/main.c
> @@ -850,7 +850,6 @@ int main(int argc, char **argv, char **envp)
> generating the prologue until now so that the prologue can take
> the real value of GUEST_BASE into account.  */
>  tcg_prologue_init(tcg_ctx);
> -tcg_region_init();
>  
>  target_cpu_copy_regs(env, regs);
>  
> diff --git a/tcg/tcg.c b/tcg/tcg.c
> index 2991112829..0a2e5710de 100644
> --- a/tcg/tcg.c
> +++ b/tcg/tcg.c
> @@ -1204,32 +1204,18 @@ TranslationBlock *tcg_tb_alloc(TCGContext *s)
>  
>  void tcg_prologue_init(TCGContext *s)
>  {
> -size_t prologue_size, total_size;
> -void *buf0, *buf1;
> +size_t prologue_size;
>  
>  /* Put the prologue at the beginning of code_gen_buffer.  */
> -buf0 = s->code_gen_buffer;
> -total_size = s->code_gen_buffer_size;
> -s->code_ptr = buf0;
> -s->code_buf = buf0;
> +tcg_region_assign(s, 0);
> +s->code_ptr = s->code_gen_ptr;
> +s->code_buf = s->code_gen_ptr;

Pardon me for asking a naive question, what's the difference between
s->code_buf and s->code_gen_buf and, respectively, s->code_ptr and
s->code_gen_ptr?

Thanks,
Roman

>  s->data_gen_ptr = NULL;
>  
> -/*
> - * The region trees are not yet configured, but tcg_splitwx_to_rx
> - * needs the bounds for an assert.
> - */
> -region.start = buf0;
> -region.end = buf0 + total_size;
> -
>  #ifndef CONFIG_TCG_INTERPRETER
> -tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(buf0);
> +tcg_qemu_tb_exec = (tcg_prologue_fn *)tcg_splitwx_to_rx(s->code_ptr);
>  #endif
>  
> -/* Compute a high-water mark, at which we voluntarily flush the buffer
> -   and start over.  The size here is arbitrary, significantly larger
> -   than we expect the code generation for any one opcode to require.  */
> -s->code_gen_highwater = s->code_gen_buffer + (total_size - 
> TCG_HIGHWATER);
> -
>  #ifdef TCG_TARGET_NEED_POOL_LABELS
>  s->pool_labels = NULL;
>  #endif
> @@ -1246,32 +1232,32 @@ void tcg_prologue_init(TCGContext *s)
>  }
>  #endif
>  
> -buf1 = s->code_ptr;
> +prologue_size = tcg_current_code_size(s);
> +
>  #ifndef CONFIG_TCG_INTERPRETER
> -flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(buf0), (uintptr_t)buf0,
> -tcg_ptr_byte_diff(buf1, buf0));
> +

Re: [PATCH v2 01/29] meson: Split out tcg/meson.build

[Roman Bolshakov]

On Sun, Mar 14, 2021 at 03:26:56PM -0600, Richard Henderson wrote:
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Richard Henderson 
> ---

Reviewed-by: Roman Bolshakov 

Thanks,
Roman

>  meson.build |  9 ++---
>  tcg/meson.build | 13 +
>  2 files changed, 15 insertions(+), 7 deletions(-)
>  create mode 100644 tcg/meson.build
> 
> diff --git a/meson.build b/meson.build
> index a7d2dd429d..742f45c8d8 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1936,14 +1936,8 @@ specific_ss.add(files('cpu.c', 'disas.c', 
> 'gdbstub.c'), capstone)
>  specific_ss.add(files('exec-vary.c'))
>  specific_ss.add(when: 'CONFIG_TCG', if_true: files(
>'fpu/softfloat.c',
> -  'tcg/optimize.c',
> -  'tcg/tcg-common.c',
> -  'tcg/tcg-op-gvec.c',
> -  'tcg/tcg-op-vec.c',
> -  'tcg/tcg-op.c',
> -  'tcg/tcg.c',
>  ))
> -specific_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: 
> files('disas/tci.c', 'tcg/tci.c'))
> +specific_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: 
> files('disas/tci.c'))
>  
>  subdir('backends')
>  subdir('disas')
> @@ -1953,6 +1947,7 @@ subdir('net')
>  subdir('replay')
>  subdir('semihosting')
>  subdir('hw')
> +subdir('tcg')
>  subdir('accel')
>  subdir('plugins')
>  subdir('bsd-user')
> diff --git a/tcg/meson.build b/tcg/meson.build
> new file mode 100644
> index 00..84064a341e
> --- /dev/null
> +++ b/tcg/meson.build
> @@ -0,0 +1,13 @@
> +tcg_ss = ss.source_set()
> +
> +tcg_ss.add(files(
> +  'optimize.c',
> +  'tcg.c',
> +  'tcg-common.c',
> +  'tcg-op.c',
> +  'tcg-op-gvec.c',
> +  'tcg-op-vec.c',
> +))
> +tcg_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: files('tci.c'))
> +
> +specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
> -- 
> 2.25.1
> 

Re: [PATCH v2 02/29] meson: Split out fpu/meson.build

[Roman Bolshakov]

On Sun, Mar 14, 2021 at 03:26:57PM -0600, Richard Henderson wrote:
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Richard Henderson 
> ---

Reviewed-by: Roman Bolshakov 

Thanks,
Roman

>  meson.build | 4 +---
>  fpu/meson.build | 1 +
>  2 files changed, 2 insertions(+), 3 deletions(-)
>  create mode 100644 fpu/meson.build
> 
> diff --git a/meson.build b/meson.build
> index 742f45c8d8..bfa24b836e 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1934,9 +1934,6 @@ subdir('softmmu')
>  common_ss.add(capstone)
>  specific_ss.add(files('cpu.c', 'disas.c', 'gdbstub.c'), capstone)
>  specific_ss.add(files('exec-vary.c'))
> -specific_ss.add(when: 'CONFIG_TCG', if_true: files(
> -  'fpu/softfloat.c',
> -))
>  specific_ss.add(when: 'CONFIG_TCG_INTERPRETER', if_true: 
> files('disas/tci.c'))
>  
>  subdir('backends')
> @@ -1948,6 +1945,7 @@ subdir('replay')
>  subdir('semihosting')
>  subdir('hw')
>  subdir('tcg')
> +subdir('fpu')
>  subdir('accel')
>  subdir('plugins')
>  subdir('bsd-user')
> diff --git a/fpu/meson.build b/fpu/meson.build
> new file mode 100644
> index 00..1a9992ded5
> --- /dev/null
> +++ b/fpu/meson.build
> @@ -0,0 +1 @@
> +specific_ss.add(when: 'CONFIG_TCG', if_true: files('softfloat.c'))
> -- 
> 2.25.1
> 

Re: [PATCH v2 00/29] tcg: Workaround macOS 11.2 mprotect bug

[Roman Bolshakov]

On Sun, Mar 14, 2021 at 03:26:55PM -0600, Richard Henderson wrote:
> Changes for v2:
>   * Move tcg_init_ctx someplace more private (patch 29)
>   * Round result of tb_size based on qemu_get_host_physmem (patch 26)
> 
> Blurb for v1:
>   It took a few more patches than imagined to unify the two
>   places in which we manipulate the tcg code_gen buffer, but
>   the result is surely cleaner.
> 
>   There's a lot more that could be done to clean up this part
>   of tcg too.  I tried to not get too side-tracked, but didn't
>   wholly succeed.
> 
> 

Hi Richard,

Thanks for doing the changes!
I'm not sure if I'll find enough time for thorough review but the series
helps qemu on Big Sur 11.2.3, so:

Tested-by: Roman Bolshakov 

Regards,
Roman

> r~
> 
> 
> Richard Henderson (29):
>   meson: Split out tcg/meson.build
>   meson: Split out fpu/meson.build
>   tcg: Re-order tcg_region_init vs tcg_prologue_init
>   tcg: Remove error return from tcg_region_initial_alloc__locked
>   tcg: Split out tcg_region_initial_alloc
>   tcg: Split out tcg_region_prologue_set
>   tcg: Split out region.c
>   accel/tcg: Inline cpu_gen_init
>   accel/tcg: Move alloc_code_gen_buffer to tcg/region.c
>   accel/tcg: Rename tcg_init to tcg_init_machine
>   tcg: Create tcg_init
>   accel/tcg: Merge tcg_exec_init into tcg_init_machine
>   accel/tcg: Pass down max_cpus to tcg_init
>   tcg: Introduce tcg_max_ctxs
>   tcg: Move MAX_CODE_GEN_BUFFER_SIZE to tcg-target.h
>   tcg: Replace region.end with region.total_size
>   tcg: Rename region.start to region.after_prologue
>   tcg: Tidy tcg_n_regions
>   tcg: Tidy split_cross_256mb
>   tcg: Move in_code_gen_buffer and tests to region.c
>   tcg: Allocate code_gen_buffer into struct tcg_region_state
>   tcg: Return the map protection from alloc_code_gen_buffer
>   tcg: Sink qemu_madvise call to common code
>   tcg: Do not set guard pages in the rx buffer
>   util/osdep: Add qemu_mprotect_rw
>   tcg: Round the tb_size default from qemu_get_host_physmem
>   tcg: Merge buffer protection and guard page protection
>   tcg: When allocating for !splitwx, begin with PROT_NONE
>   tcg: Move tcg_init_ctx and tcg_ctx from accel/tcg/
> 
>  meson.build   |  13 +-
>  accel/tcg/internal.h  |   2 +
>  include/qemu/osdep.h  |   1 +
>  include/sysemu/tcg.h  |   2 -
>  include/tcg/tcg.h |  15 +-
>  tcg/aarch64/tcg-target.h  |   1 +
>  tcg/arm/tcg-target.h  |   1 +
>  tcg/i386/tcg-target.h |   2 +
>  tcg/internal.h|  40 ++
>  tcg/mips/tcg-target.h |   6 +
>  tcg/ppc/tcg-target.h  |   2 +
>  tcg/riscv/tcg-target.h|   1 +
>  tcg/s390/tcg-target.h |   3 +
>  tcg/sparc/tcg-target.h|   1 +
>  tcg/tci/tcg-target.h  |   1 +
>  accel/tcg/tcg-all.c   |  33 +-
>  accel/tcg/translate-all.c | 439 +
>  bsd-user/main.c   |   1 -
>  linux-user/main.c |   1 -
>  tcg/region.c  | 991 ++
>  tcg/tcg.c | 634 ++--
>  util/osdep.c  |   9 +
>  fpu/meson.build   |   1 +
>  tcg/meson.build   |  14 +
>  24 files changed, 1139 insertions(+), 1075 deletions(-)
>  create mode 100644 tcg/internal.h
>  create mode 100644 tcg/region.c
>  create mode 100644 fpu/meson.build
>  create mode 100644 tcg/meson.build
> 
> -- 
> 2.25.1
> 

Re: [PATCH] util/osdep: Avoid mprotect() RWX->NONE on Big Sur 11.2

[Roman Bolshakov]

On Sun, Mar 07, 2021 at 10:47:06PM -0800, Joelle van Dyne wrote:
> On Wed, Feb 10, 2021 at 2:55 AM Roman Bolshakov  wrote:
> >
> > There's a change in mprotect() behaviour [1] in the latest macOS on M1
> > and it's not yet clear if it's going to be fixed by Apple. For now we
> > can avoid unsupported mprotect() calls. QEMU and qtests work fine
> > without it.
> >
> > 1. https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f
> >
> > Buglink: https://bugs.launchpad.net/qemu/+bug/1914849
> > Apple-Feedback: FB8994773
> > Signed-off-by: Roman Bolshakov 
> 
> Reviewed-by: Joelle van Dyne 
> 

Thanks!

> FYI the "macOS 11.2, *" means it applies to all versions of iOS. I
> think it only broke in iOS 14.2 but making it return on other versions
> seems to be fine from my tests.
> 

Hm... do you know how to say "for macOS 11.2 and above only"?

Regards,
Roman

Re: [PATCH v2] FreeBSD: Upgrade to 12.2 release

[Roman Bolshakov]

> 8 марта 2021 г., в 18:41, Thomas Huth  написал(а):
> 
> On 08/03/2021 16.26, Warner Losh wrote:
>> On Mon, Mar 8, 2021 at 6:30 AM Thomas Huth > > wrote:
>>On 07/03/2021 16.56, Warner Losh wrote:
>> > FreeBSD 12.1 has reached end of life. Use 12.2 instead so that 
>> FreeBSD's
>> > project's packages will work.  Update which timezone to pick. Work
>>around a QEMU
>> > bug that incorrectly raises an exception on a CRC32 instruction with
>>the FPU
>> > disabled.  The qemu bug is described here:
>> > https://www.mail-archive.com/qemu-devel@nongnu.org/msg784158.html
>>
>> >
>> > Signed-off-by: Warner Losh mailto:i...@bsdimp.com>>
>> >
>> > ---
>> >   tests/vm/freebsd | 14 +-
>> >   1 file changed, 9 insertions(+), 5 deletions(-)
>>I gave this a try, but it's currently failing in the unit tests:
>>Running test test-crypto-tlscredsx509
>>** (tests/test-crypto-tlscredsx509:): CRITICAL **: 12:56:35.157: 
>> Failed
>>to sign certificate ASN1 parser: Value is not valid.
>>ERROR test-crypto-tlscredsx509 - Bail out! FATAL-CRITICAL: Failed to sign
>>certificate ASN1 parser: Value is not valid.
>>gmake: *** [Makefile.mtest:576: run-test-70] Error 1
>>gmake: *** Waiting for unfinished jobs
>>Running test test-crypto-tlssession
>>** (tests/test-crypto-tlssession:10002): CRITICAL **: 12:56:35.288: Failed
>>to sign certificate ASN1 parser: Value is not valid.
>>ERROR test-crypto-tlssession - Bail out! FATAL-CRITICAL: Failed to sign
>>certificate ASN1 parser: Value is not valid.
>> That's totally unrelated to my change. Was it failing before? What 
>> environment was it failing in because it all seemed to work for me...
> 
> It's been a while since I last ran "make vm-build-freebsd", so I can't really 
> tell whether the problem was already there before ... when I now try to run 
> it without your patch, it fails for me, too, but rather due to FreeBSD 12.1 
> being out of service instead.
> 
>>I guess it's the same problem as:
>>https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg06750.html
>>
>>... so this would require a bug fix in the libtasn of FreeBSD first? See:
>>https://gitlab.com/gnutls/libtasn1/-/merge_requests/71
>>
>>  Is this on the host that built qemu, or inside the VM or where exactly?
> 
> It's inside the VM ... I assume the libtasn there has the same bug as the one 
> on macOS?
> 

The gnutls failures on macOS and FreeBSD (with clang as main compiler) won’t 
happen only if libtasn1 from master is used. Otherwise libtasn1 has to be 
compiled with -O1/-O0.

Отправлено с iPhone

> Thomas
> 

Re: [PATCH v4] net/macos: implement vmnet-based netdev

[Roman Bolshakov]

On Thu, Feb 18, 2021 at 02:49:47PM +0100, phillip.en...@gmail.com wrote:
> From: Phillip Tennen 
> 
> This patch implements a new netdev device, reachable via -netdev
> vmnet-macos, that’s backed by macOS’s vmnet framework.
> 
> The vmnet framework provides native bridging support, and its usage in
> this patch is intended as a replacement for attempts to use a tap device
> via the tuntaposx kernel extension. Notably, the tap/tuntaposx approach
> never would have worked in the first place, as QEMU interacts with the
> tap device via poll(), and macOS does not support polling device files.
> 
> vmnet requires either a special entitlement, granted via a provisioning
> profile, or root access. Otherwise attempts to create the virtual
> interface will fail with a “generic error” status code. QEMU may not
> currently be signed with an entitlement granted in a provisioning
> profile, as this would necessitate pre-signed binary build distribution,
> rather than source-code distribution. As such, using this netdev
> currently requires that qemu be run with root access. I’ve opened a
> feedback report with Apple to allow the use of the relevant entitlement
> with this use case:
> https://openradar.appspot.com/radar?id=5007417364447232
> 
> vmnet offers three operating modes, all of which are supported by this
> patch via the “mode=host|shared|bridge” option:
> 
> * "Host" mode: Allows the vmnet interface to communicate with other
> * vmnet
> interfaces that are in host mode and also with the native host.
> * "Shared" mode: Allows traffic originating from the vmnet interface to
> reach the Internet through a NAT. The vmnet interface can also
> communicate with the native host.
> * "Bridged" mode: Bridges the vmnet interface with a physical network
> interface.
> 
> Each of these modes also provide some extra configuration that’s
> supported by this patch:
> 
> * "Bridged" mode: The user may specify the physical interface to bridge
> with. Defaults to en0.
> * "Host" mode / "Shared" mode: The user may specify the DHCP range and
> subnet. Allocated by vmnet if not provided.
> 
> vmnet also offers some extra configuration options that are not
> supported by this patch:
> 
> * Enable isolation from other VMs using vmnet
> * Port forwarding rules
> * Enabling TCP segmentation offload
> * Only applicable in "shared" mode: specifying the NAT IPv6 prefix
> * Only available in "host" mode: specifying the IP address for the VM
> within an isolated network
> 
> Note that this patch requires macOS 10.15 as a minimum, as this is when
> bridging support was implemented in vmnet.framework.
> 
> Signed-off-by: Phillip Tennen 
> ---
>  configure |   2 +-
>  net/clients.h |   6 +
>  net/meson.build   |   1 +
>  net/net.c |   3 +
>  net/vmnet-macos.c | 447 ++
>  qapi/net.json | 120 -
>  qemu-options.hx   |   9 +
>  7 files changed, 585 insertions(+), 3 deletions(-)
>  create mode 100644 net/vmnet-macos.c
> 

Hi Phillip,

Thanks for working on this!

Note that the patch doesn't apply to current master and there's a lot of
warnings wrt trailing whitespaces:

git am v4-net-macos-implement-vmnet-based-netdev.patch
Applying: net/macos: implement vmnet-based netdev
.git/rebase-apply/patch:462: trailing whitespace.
 * If QEMU is started with -nographic, no Cocoa event loop will be
.git/rebase-apply/patch:465: trailing whitespace.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH,
.git/rebase-apply/patch:466: trailing whitespace.
 0),
.git/rebase-apply/patch:532: trailing whitespace.
# @host: the guest may communicate with the host
.git/rebase-apply/patch:535: trailing whitespace.
# @shared: the guest may reach the Internet through a NAT,
error: patch failed: configure:778
error: configure: patch does not apply
Patch failed at 0001 net/macos: implement vmnet-based netdev
hint: Use 'git am --show-current-patch' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Also it would be helpful to provide a changelog under commit message
delimiter ("---")  for reach new version of the patch to provide an
overview of what has been changed between the versions.

> diff --git a/configure b/configure
> index 4afd22bdf5..f449198db1 100755
> --- a/configure
> +++ b/configure
> @@ -778,7 +778,7 @@ Darwin)
>fi
>audio_drv_list="coreaudio try-sdl"
>audio_possible_drivers="coreaudio sdl"
> -  QEMU_LDFLAGS="-framework CoreFoundation -framework IOKit $QEMU_LDFLAGS"
> +  QEMU_LDFLAGS="-framework CoreFoundation -framework IOKit -framework vmnet 
> $QEMU_LDFLAGS"

I'm not sure this is right approach. Instead, we need a new
configuration option for the feature + proper discovery. Something like
this should work:

Re: [PATCH v16 01/23] cpu: Introduce TCGCpuOperations struct

[Roman Bolshakov]

On Wed, Feb 10, 2021 at 07:32:55AM -0500, Eduardo Habkost wrote:
> On Wed, Feb 10, 2021 at 03:21:51PM +0300, Roman Bolshakov wrote:
> > On Thu, Feb 04, 2021 at 05:39:09PM +0100, Claudio Fontana wrote:
> > > From: Eduardo Habkost 
> > > 
> > > The TCG-specific CPU methods will be moved to a separate struct,
> > > to make it easier to move accel-specific code outside generic CPU
> > > code in the future.  Start by moving tcg_initialize().
> > > 
> > > The new CPUClass.tcg_opts field may eventually become a pointer,
> > > but keep it an embedded struct for now, to make code conversion
> > > easier.
> > > 
> > > Signed-off-by: Eduardo Habkost 
> > > 
> > > [claudio: move TCGCpuOperations inside include/hw/core/cpu.h]
> > > 
> > > Reviewed-by: Alex Bennée 
> > > ---
> [...]
> > > diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
> > > index b3fd6643e8..d66f0351a9 100644
> > > --- a/target/alpha/cpu.c
> > > +++ b/target/alpha/cpu.c
> > > @@ -231,7 +231,7 @@ static void alpha_cpu_class_init(ObjectClass *oc, 
> > > void *data)
> > >  dc->vmsd = _alpha_cpu;
> > >  #endif
> > >  cc->disas_set_info = alpha_cpu_disas_set_info;
> > > -cc->tcg_initialize = alpha_translate_init;
> > > +cc->tcg_ops.initialize = alpha_translate_init;
> > 
> > Hi,
> > 
> > Would it be cleaner if the file had:
> > 
> > static
> > TcgCpuOperations alpha_tcg_ops = {
> > .initialize = alpha_translate_init,
> > };
> > 
> > CPUClass definition would be:
> > struct CPUClass {
> >   ...
> >   TCGCpuOperations *tcg_ops;
> >   ...
> > }
> > 
> > And class init would be:
> > 
> > cc->tcg_ops = _tcg_ops;
> 
> That's exactly what's done by:
> 
>   Subject: [PATCH v16 15/23] cpu: tcg_ops: move to tcg-cpu-ops.h, keep a 
> pointer in CPUClass
>   https://lore.kernel.org/qemu-devel/20210204163931.7358-16-cfont...@suse.de/
> 

Great, thanks!

-Roman

> > 
> > And you would grow arch_tcg_ops as you convert them?
> > I'm sorry if I missed similar comment and it was already discussed.
> > 
> > Regards,
> > Roman
> > 
> 
> -- 
> Eduardo
> 

Re: [PATCH v16 02/23] target/riscv: remove CONFIG_TCG, as it is always TCG

[Roman Bolshakov]

On Thu, Feb 04, 2021 at 05:39:10PM +0100, Claudio Fontana wrote:
> for now only TCG is allowed as an accelerator for riscv,
> so remove the CONFIG_TCG use.
> 
> Signed-off-by: Claudio Fontana 
> Reviewed-by: Alistair Francis 
> Reviewed-by: Alex Bennée 
> ---
>  target/riscv/cpu.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> index 567f6790a9..60d0b43153 100644
> --- a/target/riscv/cpu.c
> +++ b/target/riscv/cpu.c
> @@ -617,10 +617,9 @@ static void riscv_cpu_class_init(ObjectClass *c, void 
> *data)
>  #endif
>  cc->gdb_arch_name = riscv_gdb_arch_name;
>  cc->gdb_get_dynamic_xml = riscv_gdb_get_dynamic_xml;
> -#ifdef CONFIG_TCG
>  cc->tcg_ops.initialize = riscv_translate_init;
>  cc->tlb_fill = riscv_cpu_tlb_fill;
> -#endif
> +
>  device_class_set_props(dc, riscv_cpu_properties);
>  }
>  

I'm not sure if it should be dropped because it might be readded soon
because of: https://lwn.net/Articles/830078/

Regards,
Roman

Re: [PATCH v16 01/23] cpu: Introduce TCGCpuOperations struct

[Roman Bolshakov]

On Thu, Feb 04, 2021 at 05:39:09PM +0100, Claudio Fontana wrote:
> From: Eduardo Habkost 
> 
> The TCG-specific CPU methods will be moved to a separate struct,
> to make it easier to move accel-specific code outside generic CPU
> code in the future.  Start by moving tcg_initialize().
> 
> The new CPUClass.tcg_opts field may eventually become a pointer,
> but keep it an embedded struct for now, to make code conversion
> easier.
> 
> Signed-off-by: Eduardo Habkost 
> 
> [claudio: move TCGCpuOperations inside include/hw/core/cpu.h]
> 
> Reviewed-by: Alex Bennée 
> ---
>  include/hw/core/cpu.h   | 16 +++-
>  cpu.c   |  6 +-
>  target/alpha/cpu.c  |  2 +-
>  target/arm/cpu.c|  2 +-
>  target/avr/cpu.c|  2 +-
>  target/cris/cpu.c   | 12 ++--
>  target/hppa/cpu.c   |  2 +-
>  target/i386/tcg/tcg-cpu.c   |  2 +-
>  target/lm32/cpu.c   |  2 +-
>  target/m68k/cpu.c   |  2 +-
>  target/microblaze/cpu.c |  2 +-
>  target/mips/cpu.c   |  2 +-
>  target/moxie/cpu.c  |  2 +-
>  target/nios2/cpu.c  |  2 +-
>  target/openrisc/cpu.c   |  2 +-
>  target/riscv/cpu.c  |  2 +-
>  target/rx/cpu.c |  2 +-
>  target/s390x/cpu.c  |  2 +-
>  target/sh4/cpu.c|  2 +-
>  target/sparc/cpu.c  |  2 +-
>  target/tilegx/cpu.c |  2 +-
>  target/tricore/cpu.c|  2 +-
>  target/unicore32/cpu.c  |  2 +-
>  target/xtensa/cpu.c |  2 +-
>  target/ppc/translate_init.c.inc |  2 +-
>  25 files changed, 48 insertions(+), 30 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index 140fa32a5e..26b89fd7a4 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -76,6 +76,19 @@ typedef struct CPUWatchpoint CPUWatchpoint;
>  
>  struct TranslationBlock;
>  
> +/**
> + * struct TcgCpuOperations: TCG operations specific to a CPU class
> + */
> +typedef struct TcgCpuOperations {
> +/**
> + * @initialize: Initalize TCG state
> + *
> + * Called when the first CPU is realized.
> + */
> +void (*initialize)(void);
> +
> +} TcgCpuOperations;
> +
>  /**
>   * CPUClass:
>   * @class_by_name: Callback to map -cpu command line model name to an
> @@ -222,12 +235,13 @@ struct CPUClass {
>  
>  void (*disas_set_info)(CPUState *cpu, disassemble_info *info);
>  vaddr (*adjust_watchpoint_address)(CPUState *cpu, vaddr addr, int len);
> -void (*tcg_initialize)(void);
>  
>  const char *deprecation_note;
>  /* Keep non-pointer data at the end to minimize holes.  */
>  int gdb_num_core_regs;
>  bool gdb_stop_before_watchpoint;
> +
> +TcgCpuOperations tcg_ops;
>  };
>  
>  /*
> diff --git a/cpu.c b/cpu.c
> index 0b245cda2e..79a2bf12b3 100644
> --- a/cpu.c
> +++ b/cpu.c
> @@ -159,14 +159,18 @@ void cpu_exec_initfn(CPUState *cpu)
>  void cpu_exec_realizefn(CPUState *cpu, Error **errp)
>  {
>  CPUClass *cc = CPU_GET_CLASS(cpu);
> +#ifdef CONFIG_TCG
>  static bool tcg_target_initialized;
> +#endif /* CONFIG_TCG */
>  
>  cpu_list_add(cpu);
>  
> +#ifdef CONFIG_TCG
>  if (tcg_enabled() && !tcg_target_initialized) {
>  tcg_target_initialized = true;
> -cc->tcg_initialize();
> +cc->tcg_ops.initialize();
>  }
> +#endif /* CONFIG_TCG */
>  tlb_init(cpu);
>  
>  qemu_plugin_vcpu_init_hook(cpu);
> diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
> index b3fd6643e8..d66f0351a9 100644
> --- a/target/alpha/cpu.c
> +++ b/target/alpha/cpu.c
> @@ -231,7 +231,7 @@ static void alpha_cpu_class_init(ObjectClass *oc, void 
> *data)
>  dc->vmsd = _alpha_cpu;
>  #endif
>  cc->disas_set_info = alpha_cpu_disas_set_info;
> -cc->tcg_initialize = alpha_translate_init;
> +cc->tcg_ops.initialize = alpha_translate_init;

Hi,

Would it be cleaner if the file had:

static
TcgCpuOperations alpha_tcg_ops = {
.initialize = alpha_translate_init,
};

CPUClass definition would be:
struct CPUClass {
  ...
  TCGCpuOperations *tcg_ops;
  ...
}

And class init would be:

cc->tcg_ops = _tcg_ops;

And you would grow arch_tcg_ops as you convert them?
I'm sorry if I missed similar comment and it was already discussed.

Regards,
Roman

>  
>  cc->gdb_num_core_regs = 67;
>  }
> diff --git a/target/arm/cpu.c b/target/arm/cpu.c
> index 40142ac141..fa4d4ba4eb 100644
> --- a/target/arm/cpu.c
> +++ b/target/arm/cpu.c
> @@ -2276,7 +2276,7 @@ static void arm_cpu_class_init(ObjectClass *oc, void 
> *data)
>  cc->gdb_stop_before_watchpoint = true;
>  cc->disas_set_info = arm_disas_set_info;
>  #ifdef CONFIG_TCG
> -cc->tcg_initialize = arm_translate_init;
> +cc->tcg_ops.initialize = arm_translate_init;
>  cc->tlb_fill = arm_cpu_tlb_fill;
>  cc->debug_excp_handler = arm_debug_excp_handler;
>  cc->debug_check_watchpoint = 

[Bug 1913505] Re: Windows XP slow on Apple M1

[Roman Bolshakov]

@John please build from master and apply the patch
https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03527.html

** Tags added: macos tcg

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1913505

Title:
  Windows XP slow on Apple M1

Status in QEMU:
  New

Bug description:
  Qemu installed by using brew install qemu -s on M1

  QEMU emulator version 5.2.0
  XP image from: https://archive.org/details/WinXPProSP3x86

  Commands run:
  $ qemu-img create -f qcow2 xpsp3.img 10G
  $ qemu-system-i386 -m 512 -hda xpsp3.img -cdrom 
WinXPProSP3x86/en_windows_xp_professional_with_service_pack_3_x86_cd_vl_x14-73974.iso
 -boot d

  It's taken 3 days now with qemu running at around 94% CPU and
  installation hasn't finished. The mouse pointer moves and occasionally
  changes between the pointer and hourglass so it doesn't seem to have
  frozen.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1913505/+subscriptions

[PATCH] util/osdep: Avoid mprotect() RWX->NONE on Big Sur 11.2

[Roman Bolshakov]

There's a change in mprotect() behaviour [1] in the latest macOS on M1
and it's not yet clear if it's going to be fixed by Apple. For now we
can avoid unsupported mprotect() calls. QEMU and qtests work fine
without it.

1. https://gist.github.com/hikalium/75ae822466ee4da13cbbe486498a191f

Buglink: https://bugs.launchpad.net/qemu/+bug/1914849
Apple-Feedback: FB8994773
Signed-off-by: Roman Bolshakov 
---
 util/osdep.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/util/osdep.c b/util/osdep.c
index 66d01b9160..1edd7b1caf 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -111,6 +111,12 @@ int qemu_mprotect_none(void *addr, size_t size)
 #ifdef _WIN32
 return qemu_mprotect__osdep(addr, size, PAGE_NOACCESS);
 #else
+# if defined(__APPLE__) && defined(__arm64__)
+if (__builtin_available(macOS 11.2, *)) {
+/* mprotect() in macOS 11.2 can't switch RWX to NONE */
+return 0;
+}
+# endif
 return qemu_mprotect__osdep(addr, size, PROT_NONE);
 #endif
 }
-- 
2.30.0

[PULL hvf 5/5] hvf: Fetch cr4 before evaluating CPUID(1)

[Roman Bolshakov]

From: Alexander Graf 

The CPUID function 1 has a bit called OSXSAVE which tells user space the
status of the CR4.OSXSAVE bit. Our generic CPUID function injects that bit
based on the status of CR4.

With Hypervisor.framework, we do not synchronize full CPU state often enough
for this function to see the CR4 update before guest user space asks for it.

To be on the save side, let's just always synchronize it when we receive a
CPUID(1) request. That way we can set the bit with real confidence.

Reported-by: Asad Ali 
Signed-off-by: Alexander Graf 
Message-Id: <20210123004129.6364-1-ag...@csgraf.de>
[RB: resolved conflict with another CPUID change]
Signed-off-by: Roman Bolshakov 
---
 target/i386/hvf/hvf.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index 5a8914564b..d2fb680058 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -828,6 +828,10 @@ int hvf_vcpu_exec(CPUState *cpu)
 uint32_t rcx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RCX);
 uint32_t rdx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RDX);
 
+if (rax == 1) {
+/* CPUID1.ecx.OSXSAVE needs to know CR4 */
+env->cr[4] = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR4);
+}
 hvf_cpu_x86_cpuid(env, rax, rcx, , , , );
 
 wreg(cpu->hvf_fd, HV_X86_RAX, rax);
-- 
2.30.0

[PULL hvf 4/5] target/i386/hvf: add rdmsr 35H MSR_CORE_THREAD_COUNT

[Roman Bolshakov]

From: Vladislav Yaroshchuk 

Some guests (ex. Darwin-XNU) can attemp to read this MSR to retrieve and
validate CPU topology comparing it to ACPI MADT content

MSR description from Intel Manual:
35H: MSR_CORE_THREAD_COUNT: Configured State of Enabled Processor Core
  Count and Logical Processor Count

Bits 15:0 THREAD_COUNT The number of logical processors that are
  currently enabled in the physical package

Bits 31:16 Core_COUNT The number of processor cores that are currently
  enabled in the physical package

Bits 63:32 Reserved

Signed-off-by: Vladislav Yaroshchuk 
Message-Id: <20210113205323.33310-1-yaroshchuk2...@gmail.com>
[RB: reordered MSR definition and dropped u suffix from shift offset]
Signed-off-by: Roman Bolshakov 
---
 target/i386/cpu.h | 1 +
 target/i386/hvf/x86_emu.c | 5 +
 2 files changed, 6 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index d23a5b340a..e2fe0689cc 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -366,6 +366,7 @@ typedef enum X86Seg {
 
 #define MSR_IA32_SMBASE 0x9e
 #define MSR_SMI_COUNT   0x34
+#define MSR_CORE_THREAD_COUNT   0x35
 #define MSR_MTRRcap 0xfe
 #define MSR_MTRRcap_VCNT8
 #define MSR_MTRRcap_FIXRANGE_SUPPORT(1 << 8)
diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c
index da570e352b..e52c39ddb1 100644
--- a/target/i386/hvf/x86_emu.c
+++ b/target/i386/hvf/x86_emu.c
@@ -668,6 +668,7 @@ void simulate_rdmsr(struct CPUState *cpu)
 {
 X86CPU *x86_cpu = X86_CPU(cpu);
 CPUX86State *env = _cpu->env;
+CPUState *cs = env_cpu(env);
 uint32_t msr = ECX(env);
 uint64_t val = 0;
 
@@ -745,6 +746,10 @@ void simulate_rdmsr(struct CPUState *cpu)
 case MSR_MTRRdefType:
 val = env->mtrr_deftype;
 break;
+case MSR_CORE_THREAD_COUNT:
+val = cs->nr_threads * cs->nr_cores; /* thread count, bits 15..0 */
+val |= ((uint32_t)cs->nr_cores << 16); /* core count, bits 31..16 */
+break;
 default:
 /* fprintf(stderr, "%s: unknown msr 0x%x\n", __func__, msr); */
 val = 0;
-- 
2.30.0

[PULL hvf 3/5] hvf: x86: Remove unused definitions

[Roman Bolshakov]

From: Alexander Graf 

The hvf i386 has a few struct and cpp definitions that are never
used. Remove them.

Suggested-by: Roman Bolshakov 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Alexander Graf 
Message-Id: <2021012022.71840-3-ag...@csgraf.de>
Signed-off-by: Roman Bolshakov 
---
 target/i386/hvf/hvf-i386.h | 16 
 1 file changed, 16 deletions(-)

diff --git a/target/i386/hvf/hvf-i386.h b/target/i386/hvf/hvf-i386.h
index e0edffd077..e31938e5ff 100644
--- a/target/i386/hvf/hvf-i386.h
+++ b/target/i386/hvf/hvf-i386.h
@@ -21,21 +21,6 @@
 #include "cpu.h"
 #include "x86.h"
 
-#define HVF_MAX_VCPU 0x10
-
-extern struct hvf_state hvf_global;
-
-struct hvf_vm {
-int id;
-struct hvf_vcpu_state *vcpus[HVF_MAX_VCPU];
-};
-
-struct hvf_state {
-uint32_t version;
-struct hvf_vm *vm;
-uint64_t mem_quota;
-};
-
 /* hvf_slot flags */
 #define HVF_SLOT_LOG (1 << 0)
 
@@ -75,7 +60,6 @@ hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t);
 
 /* Host specific functions */
 int hvf_inject_interrupt(CPUArchState *env, int vector);
-int hvf_vcpu_run(struct hvf_vcpu_state *vcpu);
 #endif
 
 #endif
-- 
2.30.0

[PULL hvf 1/5] hvf: Guard xgetbv call

[Roman Bolshakov]

From: Hill Ma 

This prevents illegal instruction on cpus that do not support xgetbv.

Buglink: https://bugs.launchpad.net/qemu/+bug/1758819
Reviewed-by: Cameron Esfahani 
Signed-off-by: Hill Ma 
Message-Id: 
Signed-off-by: Roman Bolshakov 
---
 target/i386/hvf/x86_cpuid.c | 34 ++
 1 file changed, 22 insertions(+), 12 deletions(-)

diff --git a/target/i386/hvf/x86_cpuid.c b/target/i386/hvf/x86_cpuid.c
index a6842912f5..32b0d131df 100644
--- a/target/i386/hvf/x86_cpuid.c
+++ b/target/i386/hvf/x86_cpuid.c
@@ -27,15 +27,22 @@
 #include "vmx.h"
 #include "sysemu/hvf.h"
 
-static uint64_t xgetbv(uint32_t xcr)
+static bool xgetbv(uint32_t cpuid_ecx, uint32_t idx, uint64_t *xcr)
 {
-uint32_t eax, edx;
+uint32_t xcrl, xcrh;
 
-__asm__ volatile ("xgetbv"
-  : "=a" (eax), "=d" (edx)
-  : "c" (xcr));
+if (cpuid_ecx & CPUID_EXT_OSXSAVE) {
+/*
+ * The xgetbv instruction is not available to older versions of
+ * the assembler, so we encode the instruction manually.
+ */
+asm(".byte 0x0f, 0x01, 0xd0" : "=a" (xcrl), "=d" (xcrh) : "c" (idx));
 
-return (((uint64_t)edx) << 32) | eax;
+*xcr = (((uint64_t)xcrh) << 32) | xcrl;
+return true;
+}
+
+return false;
 }
 
 uint32_t hvf_get_supported_cpuid(uint32_t func, uint32_t idx,
@@ -100,12 +107,15 @@ uint32_t hvf_get_supported_cpuid(uint32_t func, uint32_t 
idx,
 break;
 case 0xD:
 if (idx == 0) {
-uint64_t host_xcr0 = xgetbv(0);
-uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK | XSTATE_SSE_MASK 
|
-  XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK |
-  XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK |
-  XSTATE_ZMM_Hi256_MASK | 
XSTATE_Hi16_ZMM_MASK);
-eax &= supp_xcr0;
+uint64_t host_xcr0;
+if (xgetbv(ecx, 0, _xcr0)) {
+uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK |
+  XSTATE_SSE_MASK | XSTATE_YMM_MASK |
+  XSTATE_BNDREGS_MASK | XSTATE_BNDCSR_MASK |
+  XSTATE_OPMASK_MASK | XSTATE_ZMM_Hi256_MASK |
+  XSTATE_Hi16_ZMM_MASK);
+eax &= supp_xcr0;
+}
 } else if (idx == 1) {
 hv_vmx_read_capability(HV_VMX_CAP_PROCBASED2, );
 eax &= CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XGETBV1;
-- 
2.30.0

[PULL hvf 0/5] HVF updates for 2021-02-09

[Roman Bolshakov]

Hi Paolo,

Please apply the PR to i386 queue (not for master). It contains bug
fixes, cleanups and improvements for HVF accel:
 - Added support of older HW (Hill)
 - Fixed OSXSAVE reporting in CPUID (Alex)
 - Improved Darwin-XNU support (Vladislav)
 - dead code removed (Alex)

Test results: https://gitlab.com/roolebo/qemu/-/pipelines/253575182
The patches don't introduce regressions in kvm-unit-tests.

The following changes since commit d0dddab40e472ba62b5f43f11cc7dba085dabe71:

  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging 
(2021-02-05 15:27:02 +)

are available in the Git repository at:

  https://gitlab.com/roolebo/qemu.git tags/hvf-queue-20210209

for you to fetch changes up to db7884ccdde5425584bec758f72ed658b6549f8a:

  hvf: Fetch cr4 before evaluating CPUID(1) (2021-02-09 12:25:09 +0300)

Thanks,
Roman


Alexander Graf (2):
  hvf: x86: Remove unused definitions
  hvf: Fetch cr4 before evaluating CPUID(1)

Hill Ma (1):
  hvf: Guard xgetbv call

Vladislav Yaroshchuk (2):
  target/i386/hvf: add vmware-cpuid-freq cpu feature
  target/i386/hvf: add rdmsr 35H MSR_CORE_THREAD_COUNT

 target/i386/cpu.h   |   1 +
 target/i386/hvf/hvf-i386.h  |  16 ---
 target/i386/hvf/hvf.c   | 100 +++-
 target/i386/hvf/x86_cpuid.c |  34 +--
 target/i386/hvf/x86_emu.c   |   5 +++
 5 files changed, 127 insertions(+), 29 deletions(-)

-- 
2.30.0

[PULL hvf 2/5] target/i386/hvf: add vmware-cpuid-freq cpu feature

[Roman Bolshakov]

From: Vladislav Yaroshchuk 

For `-accel hvf` cpu_x86_cpuid() is wrapped with hvf_cpu_x86_cpuid() to
add paravirtualization cpuid leaf 0x4010
https://lkml.org/lkml/2008/10/1/246

Leaf 0x4010, Timing Information:
EAX: (Virtual) TSC frequency in kHz.
EBX: (Virtual) Bus (local apic timer) frequency in kHz.
ECX, EDX: RESERVED (Per above, reserved fields are set to zero).

On macOS TSC and APIC Bus frequencies can be readed by sysctl call with
names `machdep.tsc.frequency` and `hw.busfrequency`

This options is required for Darwin-XNU guest to be synchronized with
host

Leaf 0x4000 not exposes HVF leaving hypervisor signature empty

Signed-off-by: Vladislav Yaroshchuk 
Message-Id: <20210122150518.3551-1-yaroshchuk2...@gmail.com>
Signed-off-by: Roman Bolshakov 
---
 target/i386/hvf/hvf.c | 96 ++-
 1 file changed, 95 insertions(+), 1 deletion(-)

diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index ed9356565c..5a8914564b 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -65,6 +65,7 @@
 
 #include 
 #include 
+#include 
 
 #include "exec/address-spaces.h"
 #include "hw/i386/apic_internal.h"
@@ -456,6 +457,48 @@ static void dummy_signal(int sig)
 {
 }
 
+static void init_tsc_freq(CPUX86State *env)
+{
+size_t length;
+uint64_t tsc_freq;
+
+if (env->tsc_khz != 0) {
+return;
+}
+
+length = sizeof(uint64_t);
+if (sysctlbyname("machdep.tsc.frequency", _freq, , NULL, 0)) {
+return;
+}
+env->tsc_khz = tsc_freq / 1000;  /* Hz to KHz */
+}
+
+static void init_apic_bus_freq(CPUX86State *env)
+{
+size_t length;
+uint64_t bus_freq;
+
+if (env->apic_bus_freq != 0) {
+return;
+}
+
+length = sizeof(uint64_t);
+if (sysctlbyname("hw.busfrequency", _freq, , NULL, 0)) {
+return;
+}
+env->apic_bus_freq = bus_freq;
+}
+
+static inline bool tsc_is_known(CPUX86State *env)
+{
+return env->tsc_khz != 0;
+}
+
+static inline bool apic_bus_freq_is_known(CPUX86State *env)
+{
+return env->apic_bus_freq != 0;
+}
+
 int hvf_init_vcpu(CPUState *cpu)
 {
 
@@ -480,6 +523,15 @@ int hvf_init_vcpu(CPUState *cpu)
 hvf_state->hvf_caps = g_new0(struct hvf_vcpu_caps, 1);
 env->hvf_mmio_buf = g_new(char, 4096);
 
+if (x86cpu->vmware_cpuid_freq) {
+init_tsc_freq(env);
+init_apic_bus_freq(env);
+
+if (!tsc_is_known(env) || !apic_bus_freq_is_known(env)) {
+error_report("vmware-cpuid-freq: feature couldn't be enabled");
+}
+}
+
 r = hv_vcpu_create((hv_vcpuid_t *)>hvf_fd, HV_VCPU_DEFAULT);
 cpu->vcpu_dirty = 1;
 assert_hvf_ok(r);
@@ -597,6 +649,48 @@ static void hvf_store_events(CPUState *cpu, uint32_t 
ins_len, uint64_t idtvec_in
 }
 }
 
+static void hvf_cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
+  uint32_t *eax, uint32_t *ebx,
+  uint32_t *ecx, uint32_t *edx)
+{
+/*
+ * A wrapper extends cpu_x86_cpuid with 0x4000 and 0x4010 leafs,
+ * leafs 0x4001-0x400F are filled with zeros
+ * Provides vmware-cpuid-freq support to hvf
+ *
+ * Note: leaf 0x4000 not exposes HVF,
+ * leaving hypervisor signature empty
+ */
+
+if (index < 0x4000 || index > 0x4010 ||
+!tsc_is_known(env) || !apic_bus_freq_is_known(env)) {
+
+cpu_x86_cpuid(env, index, count, eax, ebx, ecx, edx);
+return;
+}
+
+switch (index) {
+case 0x4000:
+*eax = 0x4010;/* Max available cpuid leaf */
+*ebx = 0; /* Leave signature empty */
+*ecx = 0;
+*edx = 0;
+break;
+case 0x4010:
+*eax = env->tsc_khz;
+*ebx = env->apic_bus_freq / 1000; /* Hz to KHz */
+*ecx = 0;
+*edx = 0;
+break;
+default:
+*eax = 0;
+*ebx = 0;
+*ecx = 0;
+*edx = 0;
+break;
+}
+}
+
 int hvf_vcpu_exec(CPUState *cpu)
 {
 X86CPU *x86_cpu = X86_CPU(cpu);
@@ -734,7 +828,7 @@ int hvf_vcpu_exec(CPUState *cpu)
 uint32_t rcx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RCX);
 uint32_t rdx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RDX);
 
-cpu_x86_cpuid(env, rax, rcx, , , , );
+hvf_cpu_x86_cpuid(env, rax, rcx, , , , );
 
 wreg(cpu->hvf_fd, HV_X86_RAX, rax);
 wreg(cpu->hvf_fd, HV_X86_RBX, rbx);
-- 
2.30.0

Re: [PATCH] target/i386/hvf: add rdmsr 35H MSR_CORE_THREAD_COUNT

[Roman Bolshakov]

On Wed, Jan 13, 2021 at 11:53:23PM +0300, yaroshchuk2...@gmail.com wrote:
> From: Vladislav Yaroshchuk 
> 
> Some guests (ex. Darwin-XNU) can attemp to read this MSR to retrieve and
> validate CPU topology comparing it to ACPI MADT content
> 
> MSR description from Intel Manual:
> 35H: MSR_CORE_THREAD_COUNT: Configured State of Enabled Processor Core
>   Count and Logical Processor Count
> 
> Bits 15:0 THREAD_COUNT The number of logical processors that are
>   currently enabled in the physical package
> 
> Bits 31:16 Core_COUNT The number of processor cores that are currently
>   enabled in the physical package
> 
> Bits 63:32 Reserved
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  target/i386/cpu.h | 2 ++
>  target/i386/hvf/x86_emu.c | 5 +
>  2 files changed, 7 insertions(+)
> 

Queued, thanks!

-Roman

Re: [PATCH v3] target/i386/hvf: add vmware-cpuid-freq cpu feature

[Roman Bolshakov]

On Fri, Jan 22, 2021 at 06:05:18PM +0300, yaroshchuk2...@gmail.com wrote:
> From: Vladislav Yaroshchuk 
> 
> For `-accel hvf` cpu_x86_cpuid() is wrapped with hvf_cpu_x86_cpuid() to
> add paravirtualization cpuid leaf 0x4010
> https://lkml.org/lkml/2008/10/1/246
> 
> Leaf 0x4010, Timing Information:
> EAX: (Virtual) TSC frequency in kHz.
> EBX: (Virtual) Bus (local apic timer) frequency in kHz.
> ECX, EDX: RESERVED (Per above, reserved fields are set to zero).
> 
> On macOS TSC and APIC Bus frequencies can be readed by sysctl call with
> names `machdep.tsc.frequency` and `hw.busfrequency`
> 
> This options is required for Darwin-XNU guest to be synchronized with
> host
> 
> Leaf 0x4000 not exposes HVF leaving hypervisor signature empty
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  target/i386/hvf/hvf.c | 96 ++-
>  1 file changed, 95 insertions(+), 1 deletion(-)
> 

Queued, thanks!

-Roman

Re: [PATCH] hvf: Fetch cr4 before evaluating CPUID(1)

[Roman Bolshakov]

On Sat, Jan 23, 2021 at 01:41:29AM +0100, Alexander Graf wrote:
> The CPUID function 1 has a bit called OSXSAVE which tells user space the
> status of the CR4.OSXSAVE bit. Our generic CPUID function injects that bit
> based on the status of CR4.
> 
> With Hypervisor.framework, we do not synchronize full CPU state often enough
> for this function to see the CR4 update before guest user space asks for it.
> 
> To be on the save side, let's just always synchronize it when we receive a
> CPUID(1) request. That way we can set the bit with real confidence.
> 
> Reported-by: Asad Ali 
> Signed-off-by: Alexander Graf 
> ---
>  target/i386/hvf/hvf.c | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
> index 08b4adecd9..f660b829ac 100644
> --- a/target/i386/hvf/hvf.c
> +++ b/target/i386/hvf/hvf.c
> @@ -426,6 +426,10 @@ int hvf_vcpu_exec(CPUState *cpu)
>  uint32_t rcx = (uint32_t)rreg(cpu->hvf->fd, HV_X86_RCX);
>  uint32_t rdx = (uint32_t)rreg(cpu->hvf->fd, HV_X86_RDX);
>  
> +if (rax == 1) {
> +/* CPUID1.ecx.OSXSAVE needs to know CR4 */
> +env->cr[4] = rvmcs(cpu->hvf->fd, VMCS_GUEST_CR4);
> +}
>  cpu_x86_cpuid(env, rax, rcx, , , , );
>  
>  wreg(cpu->hvf->fd, HV_X86_RAX, rax);
> -- 
> 2.24.3 (Apple Git-128)
> 

The fix is based off hvf-arm patch series and doesn't build on
master branch because of "cpu->hvf->fd" has to be "cpu->hvf_fd".
I've corrected the issue and resolved conflicts with another patch in
hvf-queue. So, it's been queued.

Thanks,
Roman

Re: [PATCH v6 02/11] hvf: x86: Remove unused definitions

[Roman Bolshakov]

On Wed, Jan 20, 2021 at 11:44:35PM +0100, Alexander Graf wrote:
> The hvf i386 has a few struct and cpp definitions that are never
> used. Remove them.
> 
> Suggested-by: Roman Bolshakov 
> Signed-off-by: Alexander Graf 
> Reviewed-by: Roman Bolshakov 
> Tested-by: Roman Bolshakov 
> ---

Queued, thanks!

-Roman

[Bug 1914849] Re: mprotect fails after MacOS 11.2 on arm mac

[Roman Bolshakov]

The patch can be used as a workaround for now:
diff --git a/util/osdep.c b/util/osdep.c
index 66d01b9160..76be8c295b 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -110,6 +110,9 @@ int qemu_mprotect_none(void *addr, size_t size)
 {
 #ifdef _WIN32
 return qemu_mprotect__osdep(addr, size, PAGE_NOACCESS);
+#elif defined(__APPLE__) && defined(__arm64__)
+/* Workaround mprotect (RWX->NONE) issue on Big Sur 11.2 */
+return 0;
 #else
 return qemu_mprotect__osdep(addr, size, PROT_NONE);
 #endif

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1914849

Title:
  mprotect fails after MacOS 11.2 on arm mac

Status in QEMU:
  New

Bug description:
  I got the following error when I ran qemu on arm mac(MacOS 11.2).

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

  I tested the same version of qemu on intel mac(MacOS 11.2), but it
  works fine.

  And my friend told me that they did not have this error with MacOS
  11.1.

  So, I think it is CPU architecture or an OS version dependent error.

  
  Environment:

  Qemu commit id: d0dddab40e472ba62b5f43f11cc7dba085dabe71
  OS: MacOS 11.2(20D64)
  Hardware: MacBook Air (M1, 2020)

  
  How to build:

  ```
  mkdir build/
  cd build/
  ../configure --target-list=aarch64-softmmu,x86_64-softmmu
  make
  ```

  
  How to reproduce:

  ```
  ./qemu-system-x86_64
  ```

  
  Error message:

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1914849/+subscriptions

[Bug 1914849] Re: mprotect fails after MacOS 11.2 on arm mac

[Roman Bolshakov]

Thanks for submitting the ticket.
I've just stumbled upon it after updating to 11.2.

The question was already asked on apple developer forums:
https://developer.apple.com/forums/thread/672804

And there's a thread going on with regard to broken nodejs on 11.2:
https://github.com/nodejs/node/issues/37061#issuecomment-774175983

** Bug watch added: github.com/nodejs/node/issues #37061
   https://github.com/nodejs/node/issues/37061

** Tags added: macos tcg

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1914849

Title:
  mprotect fails after MacOS 11.2 on arm mac

Status in QEMU:
  New

Bug description:
  I got the following error when I ran qemu on arm mac(MacOS 11.2).

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

  I tested the same version of qemu on intel mac(MacOS 11.2), but it
  works fine.

  And my friend told me that they did not have this error with MacOS
  11.1.

  So, I think it is CPU architecture or an OS version dependent error.

  
  Environment:

  Qemu commit id: d0dddab40e472ba62b5f43f11cc7dba085dabe71
  OS: MacOS 11.2(20D64)
  Hardware: MacBook Air (M1, 2020)

  
  How to build:

  ```
  mkdir build/
  cd build/
  ../configure --target-list=aarch64-softmmu,x86_64-softmmu
  make
  ```

  
  How to reproduce:

  ```
  ./qemu-system-x86_64
  ```

  
  Error message:

  ```
  $ ./qemu-system-x86_64
  qemu-system-x86_64: qemu_mprotect__osdep: mprotect failed: Permission denied
  **
  ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  Bail out! ERROR:../tcg/tcg.c:844:tcg_region_init: assertion failed: (!rc)
  [1]34898 abort  ./qemu-system-x86_64
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1914849/+subscriptions

Re: [PATCH v3] target/i386/hvf: add vmware-cpuid-freq cpu feature

[Roman Bolshakov]

On Fri, Jan 22, 2021 at 06:05:18PM +0300, yaroshchuk2...@gmail.com wrote:
> From: Vladislav Yaroshchuk 
> 
> For `-accel hvf` cpu_x86_cpuid() is wrapped with hvf_cpu_x86_cpuid() to
> add paravirtualization cpuid leaf 0x4010
> https://lkml.org/lkml/2008/10/1/246
> 
> Leaf 0x4010, Timing Information:
> EAX: (Virtual) TSC frequency in kHz.
> EBX: (Virtual) Bus (local apic timer) frequency in kHz.
> ECX, EDX: RESERVED (Per above, reserved fields are set to zero).
> 
> On macOS TSC and APIC Bus frequencies can be readed by sysctl call with
> names `machdep.tsc.frequency` and `hw.busfrequency`
> 
> This options is required for Darwin-XNU guest to be synchronized with
> host
> 
> Leaf 0x4000 not exposes HVF leaving hypervisor signature empty
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  target/i386/hvf/hvf.c | 96 ++-
>  1 file changed, 95 insertions(+), 1 deletion(-)
> 

I'd prefer to have generic expose-accel option for CPU and
vmware-cpuid-freq=on would depend on expose-accel=on.

Regardless of that,

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Thanks,
Roman

Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Roman Bolshakov]

On Tue, Feb 02, 2021 at 08:50:24AM -0600, Eric Blake wrote:
> On 2/1/21 11:19 PM, Roman Bolshakov wrote:
> 
> > After a session of debugging I believe there's an issue with Clang 12.
> > Here's a test program (it reproduces unexpected ASN1_VALUE_NOT_VALID
> > from _asn1_time_der() in libtasn1):
> > 
> > #include 
> > 
> > static int func2(char *foo) {
> > fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
> > if (foo == NULL) {
> > fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
> > return 1;
> > }
> > return 0;
> > }
> > 
> > int func1(char *foo) {
> > int counter = 0;
> > if (fprintf(stderr, "IO\n") > 0)
> > counter += 10;
> > fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__, 
> > foo, counter);
> > if(!func2(foo + counter)) {
> 
> This line has unspecified behavior in the C standard.  Adding an integer
> to a pointer is only well-specified if the pointer is to an array and
> the integer is within the bounds or the slot just past the array.  But
> since you called func1(NULL), foo is NOT pointing to an array, and
> therefore foo+counter points to garbage, and the compiler is free to
> optimize it at will.

Hi Eric,

Thanks a lot for pointing out this. It was surprising to me but
interesting:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2012.htm#clarifying-the-c-memory-object-model-out-of-bounds-pointer-arithmetic
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n.htm#pointer-arithmetic

As far as I understand wording in the standard, null pointer doesn't
point to an object (6.3.2.3p3). Therefore pointer arithmetic exception
for non-array objects (6.5.6p7) doesn't apply to null pointers but it
does apply to valid object pointers:

"For the purposes of these operators, a pointer to an object that is not
an element of an array behaves the same as a pointer to the first
element of an array of length one with the type of the object as its
element type."

So I was curious how clang would behave if we pass NULL conditionally.
We could do that by changing main() in the example above to:

int main(int argc, char *argv[]) {
int ret;
char *foo;

if (argc > 1)
foo = malloc(90 * sizeof(char));
else
foo = NULL;

ret = func1(foo);

if (argc > 1)
free(foo);

return ret;
}

And it returns "good" for specified behaviour (if foo points to malloc'd
memory):
 $ ./a.out f
 IO
 func1:17 foo: 0x14be06790 counter 10
 func2:5 foo: 0x14be0679a
 good

The behaviour is different if foo is initialized to NULL.
 $ ./a.out
 IO
 func1:17 foo: 0x0 counter 10
 func2:5 foo: 0xa
 func2:7 foo: 0x0
 broken

> > 
> > So, immediate workaround would be to downgrade optimization level of 
> > libtasn1
> > to -O1 in homebrew.
> > 
> > I've submitted the issue to Apple bugtracker:
> > FB8986815
> 
> Yes, it's annoying that as compilers get smarter, it exposes the
> presence of unspecified code in weird ways.  But I don't see this as a
> bug in clang, but as a bug in libtasn1 for assuming undefined behavior
> produces a sane result.
> 

Yes, strictly speaking the compiler is compliant. Given the example
libtasn1 should likely introduce a second variable for an integer
offset instead of relying on null pointer arithmetic.

It'd also be good if clang would print an error or a warning for null
pointer arithmetic.

Thanks,
Roman

Re: [PATCH v3] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Fri, Jan 29, 2021 at 07:27:57PM -1000, Richard Henderson wrote:
> On 1/29/21 10:50 AM, Roman Bolshakov wrote:
> > 
> > I thought you already added MAP_JIT in 6f70ddee19e. It's getting enabled
> > on my M1 laptop. Was it intended or not?
> > 
> > /* Applicable to both iOS and macOS (Apple Silicon). */
> > if (!splitwx) {
> > flags |= MAP_JIT;
> > }
> > 
> > TCG from master branch of QEMU works fine on M1. I'm not sure why do we
> > need to duplicate it.
> 
> I thought there was something about abi/api build issues.  If there's nothing
> that needs doing, great!
> 

Hi Richard,

You're correct that older versions of OS X/macOS might not have MAP_JIT
definition, so a simple wrapping of the hunk with ifdef MAP_JIT might be
sufficient (or guard it for Big Sur and above):

  #if defined(MAC_OS_VERSION_11_0) && \
  MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0
  if (!splitwx && __builtin_available(macOS 11.0, *)) {
  flags |= MAP_JIT;
  }
  #endif

But I'm not sure if we want to support hosts older than 10.14.

Regards,
Roman

Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Roman Bolshakov]

On Fri, Jan 29, 2021 at 09:53:27AM +, Daniel P. Berrangé wrote:
> On Fri, Jan 29, 2021 at 11:43:32AM +0300, Roman Bolshakov wrote:
> > On Wed, Jan 27, 2021 at 06:59:17PM +, Daniel P. Berrangé wrote:
> > > On Wed, Jan 27, 2021 at 07:56:16PM +0100, Stefan Weil wrote:
> > > > Am 27.01.21 um 19:17 schrieb Daniel P. Berrangé:
> > > > 
> > > > > On Wed, Jan 27, 2021 at 06:05:08PM +0100, Stefan Weil wrote:
> > > > > > Am 27.01.21 um 17:53 schrieb Daniel P. Berrangé:
> > > > > > 
> > > > > > > In $QEMU.git/crypto/init.c can you uncomment the "#define 
> > > > > > > DEBUG_GNUTLS"
> > > > > > > line and then re-build and re-run the test case.
> > > > > > > 
> > > > > > > There's a bunch of debug logs in code paths from 
> > > > > > > gnutls_x509_crt_privkey_sign
> > > > > > > that might give us useful info.
> > > > > > > 
> > > > > > > Regards,
> > > > > > > Daniel
> > > > > > 
> > > > > > % LANG=C.UTF-8 tests/test-crypto-tlscredsx509
> > > > > > # random seed: R02S9b95072a368ad370cdd4c780b8074596
> > > > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > > > 2: signing structure using RSA-SHA256
> > > > > > 3: ASSERT: common.c[_gnutls_x509_der_encode]:855
> > > > > > 3: ASSERT: sign.c[_gnutls_x509_pkix_sign]:174
> > > > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_privkey_sign]:1834
> > > > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_sign2]:1152
> > > > > > Bail out! FATAL-CRITICAL: Failed to sign certificate ASN1 parser: 
> > > > > > Value is
> > > > > > not valid.
> > > > > So it shows its failing inside a asn1_der_coding call, but I can't see
> > > > > why it would fail, especially if the same test suite passes fine on
> > > > > macOS x86_64 hosts.
> > > > 
> > > > 
> > > > It returns ASN1_MEM_ERROR, so the input vector is too small.
> > > 
> > > Hmm, that's odd - "Value is not valid" corresponds to
> > > ASN1_VALUE_NOT_VALID error code.
> > > 
> > 
> > Hi Daniel, Stefan,
> > 
> > It's interesting that "make check" of libtasn1 fails with three tests
> > and two of them produce VALUE_NOT_VALID error.
> > 
> > The failing tests are:
> >   FAIL: Test_parser
> >   FAIL: Test_tree
> >   FAIL: copynode
> 
> That's interesting. Assuming 'make check' for libtasn1 succeeeds on
> x86_64 macOS, then I'm inclined to blame this whole problem on
> libtasn1 not QEMU.
> 

'make check' of libtasn1 doesn't succeed on x86_64 either.

After a session of debugging I believe there's an issue with Clang 12.
Here's a test program (it reproduces unexpected ASN1_VALUE_NOT_VALID
from _asn1_time_der() in libtasn1):

#include 

static int func2(char *foo) {
fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
if (foo == NULL) {
fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
return 1;
}
return 0;
}

int func1(char *foo) {
int counter = 0;
if (fprintf(stderr, "IO\n") > 0)
counter += 10;
fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__, foo, 
counter);
if(!func2(foo + counter)) {
fprintf(stderr, "good\n");
return 0;
} else {
fprintf(stderr, "broken\n");
return 1;
}
}

int main() {
char *foo = NULL;
return func1(foo);
}


What return value would you expect from the program?

If the program is compiled with -O0/O1 it returns zero exit code.
Here's the output:
IO
func1:16 foo: 0x0 counter 10
func2:4 foo: 0xa
good

If it is compiled with -O2 it returns 1:
IO
func1:16 foo: 0x0 counter 10
func2:4 foo: 0xa
func2:6 foo: 0x0
broken

That happens because clang uses register behind foo from func1 (it has zero
pointer) inside inlined func2 (it should have non zero pointer).

So, immediate workaround would be to downgrade optimization level of libtasn1
to -O1 in homebrew.

I've submitted the issue to Apple bugtracker:
FB8986815

Best regards,
Roman

Re: [PATCH v3] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Fri, Jan 29, 2021 at 10:18:58AM -1000, Richard Henderson wrote:
> On 1/21/21 8:34 AM, Richard Henderson wrote:
> > On 1/12/21 5:28 PM, Roman Bolshakov wrote:
> >> @@ -1083,6 +1083,12 @@ static bool alloc_code_gen_buffer_anon(size_t size, 
> >> int prot,
> >>  {
> >>  void *buf;
> >>  
> >> +#if defined(MAC_OS_VERSION_11_0) && \
> >> +MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0
> >> +if (__builtin_available(macOS 11.0, *)) {
> >> +flags |= MAP_JIT;
> >> +}
> >> +#endif
> > 
> > This hunk should be in alloc_code_gen_buffer, where we do the other flags
> > manipulation.
> > 
> > I'll drop this hunk and apply the rest, which is exclusively related to
> > toggling the jit bit.
> 
> Ping on this?
> 
Hi Richard,

> I would imagine that the patch would look something like
> 
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -1296,6 +1296,11 @@ static bool alloc_code_gen_buffer
>  #ifdef CONFIG_TCG_INTERPRETER
>  /* The tcg interpreter does not need execute permission. */
>  prot = PROT_READ | PROT_WRITE;
> +#elif defined(MAC_OS_VERSION_11_0) && \
> +MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0
> +if (__builtin_available(macOS 11.0, *)) {
> +flags |= MAP_JIT;
> +}
>  #elif defined(CONFIG_DARWIN)
>  /* Applicable to both iOS and macOS (Apple Silicon). */
>  if (!splitwx) {
> 
> But I don't know how CONFIG_DARWIN, iOS, and MAC_OS_VERSION interact, and I'm
> not able to even compile-test the patch.
> Certainly the final comment there looks suspicious, given the preceding MAC_OS
> stanza...
> 

I thought you already added MAP_JIT in 6f70ddee19e. It's getting enabled
on my M1 laptop. Was it intended or not?

/* Applicable to both iOS and macOS (Apple Silicon). */
if (!splitwx) {
flags |= MAP_JIT;
}

TCG from master branch of QEMU works fine on M1. I'm not sure why do we
need to duplicate it.

Thanks,
Roman

libtasn1 test suite fails on macOS Bug Sur with Apple Silicon

[Roman Bolshakov]

Hello,

I'm seeing the test failures on macOS and there's a concern that the
behaviour might affect GnuTLS and ultimately QEMU test suite.

Are there any ideas of what might be causing it? Any help is
appreciated.

Thanks,
Roman

===
   GNU Libtasn1 4.16.0: tests/test-suite.log
===

# TOTAL: 30
# PASS:  27
# SKIP:  0
# XFAIL: 0
# FAIL:  3
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: Test_parser
=

ERROR N. 1:
  Line 5 - TEST_PARSER2 { } DEFINITIONS IMPLICIT TAGS ::= BEGIN int1 ::= 
INTEGER END
  Error expected: SYNTAX_ERROR - Test_parser_ERROR.asn:6: Error: syntax error, 
unexpected IDENTIFIER, expecting $end near 'TEST_PARSER'
  Error detected: SYNTAX_ERROR - Test_parser_ERROR.asn:6: Error: syntax error, 
unexpected IDENTIFIER, expecting end of file near 'TEST_PARSER'

FAIL Test_parser (exit status: 1)

FAIL: Test_tree
===

./Test_tree.asn:121: Warning: VisibleString is a built-in ASN.1 type.
./Test_tree.asn:123: Warning: NumericString is a built-in ASN.1 type.
./Test_tree.asn:125: Warning: IA5String is a built-in ASN.1 type.
./Test_tree.asn:127: Warning: TeletexString is a built-in ASN.1 type.
./Test_tree.asn:129: Warning: PrintableString is a built-in ASN.1 type.
./Test_tree.asn:131: Warning: UniversalString is a built-in ASN.1 type.
./Test_tree.asn:134: Warning: BMPString is a built-in ASN.1 type.
./Test_tree.asn:138: Warning: UTF8String is a built-in ASN.1 type.
Error at line 707
ERROR in 254:
  Action 18 - 
  Error expected: MEM_ERROR - 79
  Error detected: VALUE_NOT_VALID - 0

FAIL Test_tree (exit status: 1)

FAIL: copynode
==

./pkix.asn:332: Warning: VisibleString is a built-in ASN.1 type.
./pkix.asn:334: Warning: NumericString is a built-in ASN.1 type.
./pkix.asn:336: Warning: IA5String is a built-in ASN.1 type.
./pkix.asn:338: Warning: TeletexString is a built-in ASN.1 type.
./pkix.asn:340: Warning: PrintableString is a built-in ASN.1 type.
./pkix.asn:342: Warning: UniversalString is a built-in ASN.1 type.
./pkix.asn:345: Warning: BMPString is a built-in ASN.1 type.
./pkix.asn:349: Warning: UTF8String is a built-in ASN.1 type.
LIBTASN1 ERROR: VALUE_NOT_VALID
Cannot copy node
FAIL copynode (exit status: 1)

Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Roman Bolshakov]

On Wed, Jan 27, 2021 at 06:59:17PM +, Daniel P. Berrangé wrote:
> On Wed, Jan 27, 2021 at 07:56:16PM +0100, Stefan Weil wrote:
> > Am 27.01.21 um 19:17 schrieb Daniel P. Berrangé:
> > 
> > > On Wed, Jan 27, 2021 at 06:05:08PM +0100, Stefan Weil wrote:
> > > > Am 27.01.21 um 17:53 schrieb Daniel P. Berrangé:
> > > > 
> > > > > In $QEMU.git/crypto/init.c can you uncomment the "#define 
> > > > > DEBUG_GNUTLS"
> > > > > line and then re-build and re-run the test case.
> > > > > 
> > > > > There's a bunch of debug logs in code paths from 
> > > > > gnutls_x509_crt_privkey_sign
> > > > > that might give us useful info.
> > > > > 
> > > > > Regards,
> > > > > Daniel
> > > > 
> > > > % LANG=C.UTF-8 tests/test-crypto-tlscredsx509
> > > > # random seed: R02S9b95072a368ad370cdd4c780b8074596
> > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > 3: ASSERT: mpi.c[wrap_nettle_mpi_print]:60
> > > > 2: signing structure using RSA-SHA256
> > > > 3: ASSERT: common.c[_gnutls_x509_der_encode]:855
> > > > 3: ASSERT: sign.c[_gnutls_x509_pkix_sign]:174
> > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_privkey_sign]:1834
> > > > 3: ASSERT: x509_write.c[gnutls_x509_crt_sign2]:1152
> > > > Bail out! FATAL-CRITICAL: Failed to sign certificate ASN1 parser: Value 
> > > > is
> > > > not valid.
> > > So it shows its failing inside a asn1_der_coding call, but I can't see
> > > why it would fail, especially if the same test suite passes fine on
> > > macOS x86_64 hosts.
> > 
> > 
> > It returns ASN1_MEM_ERROR, so the input vector is too small.
> 
> Hmm, that's odd - "Value is not valid" corresponds to
> ASN1_VALUE_NOT_VALID error code.
> 

Hi Daniel, Stefan,

It's interesting that "make check" of libtasn1 fails with three tests
and two of them produce VALUE_NOT_VALID error.

The failing tests are:
  FAIL: Test_parser
  FAIL: Test_tree
  FAIL: copynode

Full test log:
===
   GNU Libtasn1 4.16.0: tests/test-suite.log
===

# TOTAL: 30
# PASS:  27
# SKIP:  0
# XFAIL: 0
# FAIL:  3
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: Test_parser
=

ERROR N. 1:
  Line 5 - TEST_PARSER2 { } DEFINITIONS IMPLICIT TAGS ::= BEGIN int1 ::= 
INTEGER END
  Error expected: SYNTAX_ERROR - Test_parser_ERROR.asn:6: Error: syntax error, 
unexpected IDENTIFIER, expecting $end near 'TEST_PARSER'
  Error detected: SYNTAX_ERROR - Test_parser_ERROR.asn:6: Error: syntax error, 
unexpected IDENTIFIER, expecting end of file near 'TEST_PARSER'

FAIL Test_parser (exit status: 1)

FAIL: Test_tree
===

./Test_tree.asn:121: Warning: VisibleString is a built-in ASN.1 type.
./Test_tree.asn:123: Warning: NumericString is a built-in ASN.1 type.
./Test_tree.asn:125: Warning: IA5String is a built-in ASN.1 type.
./Test_tree.asn:127: Warning: TeletexString is a built-in ASN.1 type.
./Test_tree.asn:129: Warning: PrintableString is a built-in ASN.1 type.
./Test_tree.asn:131: Warning: UniversalString is a built-in ASN.1 type.
./Test_tree.asn:134: Warning: BMPString is a built-in ASN.1 type.
./Test_tree.asn:138: Warning: UTF8String is a built-in ASN.1 type.
Error at line 707
ERROR in 254:
  Action 18 - 
  Error expected: MEM_ERROR - 79
  Error detected: VALUE_NOT_VALID - 0

FAIL Test_tree (exit status: 1)

FAIL: copynode
==

./pkix.asn:332: Warning: VisibleString is a built-in ASN.1 type.
./pkix.asn:334: Warning: NumericString is a built-in ASN.1 type.
./pkix.asn:336: Warning: IA5String is a built-in ASN.1 type.
./pkix.asn:338: Warning: TeletexString is a built-in ASN.1 type.
./pkix.asn:340: Warning: PrintableString is a built-in ASN.1 type.
./pkix.asn:342: Warning: UniversalString is a built-in ASN.1 type.
./pkix.asn:345: Warning: BMPString is a built-in ASN.1 type.
./pkix.asn:349: Warning: UTF8String is a built-in ASN.1 type.
LIBTASN1 ERROR: VALUE_NOT_VALID
Cannot copy node
FAIL copynode (exit status: 1)

Regards,
Roman

Re: [PATCH v6 00/11] hvf: Implement Apple Silicon Support

[Roman Bolshakov]

On Thu, Jan 28, 2021 at 04:59:47PM +, Peter Maydell wrote:
> On Wed, 20 Jan 2021 at 22:44, Alexander Graf  wrote:
> >
> > Now that Apple Silicon is widely available, people are obviously excited
> > to try and run virtualized workloads on them, such as Linux and Windows.
> >
> > This patch set implements a fully functional version to get the ball
> > going on that. With this applied, I can successfully run both Linux and
> > Windows as guests. I am not aware of any limitations specific to
> > Hypervisor.framework apart from:
> >
> >   - Live migration / savevm
> >   - gdbstub debugging (SP register)
> >
> 
> I've gone through making code review comments.
> 
> Since patch 1 is also required for Big Sur support on x86 Macs,
> I'll take that via target-arm.next now (unless anybody would rather
> it went in via a different route).
> 

Hi Peter,

I wonder if patch 1 and patch 2 should go via Paolo's i386 or misc tree?
(IMO whatever the fastest way to master works).

Both are reviewed and nobody raised objections to them.

Thanks,
Roman

[Bug 1909256] Re: compile failure if gnutls headers not on default include path

[Roman Bolshakov]

The fix is committed in 3eacf70bb5a8.

** Changed in: qemu
   Status: New => Fix Committed

** Changed in: qemu
 Assignee: (unassigned) => Roman Bolshakov (roolebo)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1909256

Title:
  compile failure if gnutls headers not on default include path

Status in QEMU:
  Fix Committed

Bug description:
  If the gnutls headers are not on the default compiler include path,
  then configure correctly finds them and config-host.mak sets up the
  variables:

  GNUTLS_CFLAGS=-I/opt/homebrew/Cellar/gnutls/3.6.15/include 
-I/opt/homebrew/Cellar/nettle/3.6/include 
-I/opt/homebrew/Cellar/libtasn1/4.16.0/include 
-I/opt/homebrew/Cellar/libidn2/2.3.0/include 
-I/opt/homebrew/Cellar/p11-kit/0.23.22/include/p11-kit-1
  GNUTLS_LIBS=-L/opt/homebrew/Cellar/gnutls/3.6.15/lib -lgnutls

  but meson fails to put GNUTLS_CFLAGS in the compiler arguments and so
  you get compile failures like:

  [2/1865] Compiling C object qemu-nbd.p/qemu-nbd.c.o
  FAILED: qemu-nbd.p/qemu-nbd.c.o 
  cc -Iqemu-nbd.p -I. -I../.. -Iqapi -Itrace -Iui -Iui/shader 
-I/opt/homebrew/Cellar/glib/2.66.4/include 
-I/opt/homebrew/Cellar/glib/2.66.4/include/glib-2.0 
-I/opt/homebrew/Cellar/glib/2.66.4/lib/glib-2.0/include 
-I/opt/homebrew/opt/gettext/include -I/opt/homebrew/Cellar/pcre/8.44/include 
-Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -std=gnu99 -g 
-DOS_OBJECT_USE_OBJC=0 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
-Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings 
-Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv 
-Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self 
-Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels 
-Wexpansion-to-defined -Wno-initializer-overrides -Wno-missing-include-dirs 
-Wno-shift-negative-value -Wno-string-plus-int -Wno-typedef-redefinition 
-Wno-tautological-type-limit-compare -fstack-protector-strong -iquote 
/Users/pm215/qemu/tcg/aarch64 -iquote . -iquote /Users/pm215/qemu -iquote 
/Users/pm215/qemu/accel/tcg -iquote /Users/pm215/qemu/include -iquote 
/Users/pm215/qemu/disas/libvixl -MD -MQ qemu-nbd.p/qemu-nbd.c.o -MF 
qemu-nbd.p/qemu-nbd.c.o.d -o qemu-nbd.p/qemu-nbd.c.o -c ../../qemu-nbd.c
  In file included from ../../qemu-nbd.c:30:
  In file included from /Users/pm215/qemu/include/block/nbd.h:25:
  /Users/pm215/qemu/include/crypto/tlscreds.h:28:10: fatal error: 
'gnutls/gnutls.h' file not found
  #include 
   ^
  1 error generated.

  
  The compiler errors happen for any .c file that includes block/nbd.h and also 
for files in tests that include gnutls.h directly, and for files that directly 
or indirectly include crypto/tlssession.c.

  My meson-foo is insufficient to suggest the correct fix...

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1909256/+subscriptions

Re: [PULL v2 10/10] tcg: Restart code generation when we run out of temps

[Roman Bolshakov]

On Sun, Jan 24, 2021 at 08:11:22AM -1000, Richard Henderson wrote:
> Some large translation blocks can generate so many unique
> constants that we run out of temps to hold them.  In this
> case, longjmp back to the start of code generation and
> restart with a smaller translation block.
> 
> Buglink: https://bugs.launchpad.net/bugs/1912065
> Tested-by: BALATON Zoltan 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Richard Henderson 
> ---
>  include/tcg/tcg.h |  3 +++
>  accel/tcg/translate-all.c | 15 ++-
>  tcg/tcg.c | 11 ---
>  3 files changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
> index c5a9d65d5f..0f0695e90d 100644
> --- a/include/tcg/tcg.h
> +++ b/include/tcg/tcg.h
> @@ -680,6 +680,9 @@ struct TCGContext {
>  
>  uint16_t gen_insn_end_off[TCG_MAX_INSNS];
>  target_ulong gen_insn_data[TCG_MAX_INSNS][TARGET_INSN_START_WORDS];
> +
> +/* Exit to translator on overflow. */
> +sigjmp_buf jmp_trans;
>  };
>  
>  static inline bool temp_readonly(TCGTemp *ts)
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index d09c187e0f..81d4c83f22 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -1926,11 +1926,17 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
>  ti = profile_getclock();
>  #endif
>  
> +gen_code_size = sigsetjmp(tcg_ctx->jmp_trans, 0);
> +if (unlikely(gen_code_size != 0)) {
> +goto error_return;
> +}
> +
>  tcg_func_start(tcg_ctx);
>  
>  tcg_ctx->cpu = env_cpu(env);
>  gen_intermediate_code(cpu, tb, max_insns);
>  tcg_ctx->cpu = NULL;
> +max_insns = tb->icount;
>  
>  trace_translate_block(tb, tb->pc, tb->tc.ptr);
>  
> @@ -1955,6 +1961,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
>  
>  gen_code_size = tcg_gen_code(tcg_ctx, tb);
>  if (unlikely(gen_code_size < 0)) {
> + error_return:
>  switch (gen_code_size) {
>  case -1:
>  /*
> @@ -1966,6 +1973,9 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
>   * flush the TBs, allocate a new TB, re-initialize it per
>   * above, and re-do the actual code generation.
>   */
> +qemu_log_mask(CPU_LOG_TB_OP | CPU_LOG_TB_OP_OPT,
> +  "Restarting code generation for "
> +  "code_gen_buffer overflow\n");
>  goto buffer_overflow;
>  
>  case -2:
> @@ -1978,9 +1988,12 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
>   * Try again with half as many insns as we attempted this time.
>   * If a single insn overflows, there's a bug somewhere...
>   */
> -max_insns = tb->icount;
>  assert(max_insns > 1);
>  max_insns /= 2;
> +qemu_log_mask(CPU_LOG_TB_OP | CPU_LOG_TB_OP_OPT,
> +  "Restarting code generation with "
> +  "smaller translation block (max %d insns)\n",
> +  max_insns);
>  goto tb_overflow;
>  
>  default:
> diff --git a/tcg/tcg.c b/tcg/tcg.c
> index 67b08f708d..9e1b0d73c7 100644
> --- a/tcg/tcg.c
> +++ b/tcg/tcg.c
> @@ -1205,18 +1205,23 @@ void tcg_func_start(TCGContext *s)
>  QSIMPLEQ_INIT(>labels);
>  }
>  
> -static inline TCGTemp *tcg_temp_alloc(TCGContext *s)
> +static TCGTemp *tcg_temp_alloc(TCGContext *s)
>  {
>  int n = s->nb_temps++;
> -tcg_debug_assert(n < TCG_MAX_TEMPS);
> +
> +if (n >= TCG_MAX_TEMPS) {
> +/* Signal overflow, starting over with fewer guest insns. */
> +siglongjmp(s->jmp_trans, -2);
> +}
>  return memset(>temps[n], 0, sizeof(TCGTemp));
>  }
>  
> -static inline TCGTemp *tcg_global_alloc(TCGContext *s)
> +static TCGTemp *tcg_global_alloc(TCGContext *s)
>  {
>  TCGTemp *ts;
>  
>  tcg_debug_assert(s->nb_globals == s->nb_temps);
> +tcg_debug_assert(s->nb_globals < TCG_MAX_TEMPS);
>  s->nb_globals++;
>  ts = tcg_temp_alloc(s);
>  ts->kind = TEMP_GLOBAL;
> -- 
> 2.25.1
> 
> 

Hi Richard,

Thanks for providing the fix.

Tested-by: Roman Bolshakov 

Regards,
Roman

[Bug 1912065] Re: Segfaults in tcg/optimize.c:212 after commit 7c79721606be11b5bc556449e5bcbc331ef6867d

[Roman Bolshakov]

Richard, thanks for providing the workaround. It helps.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1912065

Title:
  Segfaults in tcg/optimize.c:212 after commit
  7c79721606be11b5bc556449e5bcbc331ef6867d

Status in QEMU:
  In Progress

Bug description:
  QEMU segfaults to NULL dereference in tcg/optimize.c:212 semi-randomly
  after commit 7c79721606be11b5bc556449e5bcbc331ef6867d

  Exception Type:EXC_BAD_ACCESS (SIGSEGV)
  Exception Codes:   KERN_INVALID_ADDRESS at 0x0020
  Exception Note:EXC_CORPSE_NOTIFY

  ...

  Thread 4 Crashed:
  0   qemu-system-ppc   0x000109cd26d2 tcg_opt_gen_mov + 
178 (optimize.c:212)
  1   qemu-system-ppc   0x000109ccf838 tcg_optimize + 5656
  2   qemu-system-ppc   0x000109c27600 tcg_gen_code + 64 
(tcg.c:4490)
  3   qemu-system-ppc   0x000109c17b6d tb_gen_code + 493 
(translate-all.c:1952)
  4   qemu-system-ppc   0x000109c16085 tb_find + 41 
(cpu-exec.c:454) [inlined]
  5   qemu-system-ppc   0x000109c16085 cpu_exec + 2117 
(cpu-exec.c:810)
  6   qemu-system-ppc   0x000109c09ac3 tcg_cpus_exec + 35 
(tcg-cpus.c:57)
  7   qemu-system-ppc   0x000109c75edd rr_cpu_thread_fn + 
445 (tcg-cpus-rr.c:217)
  8   qemu-system-ppc   0x000109e41fae qemu_thread_start + 
126 (qemu-thread-posix.c:521)
  9   libsystem_pthread.dylib   0x7fff2038e950 _pthread_start + 224
  10  libsystem_pthread.dylib   0x7fff2038a47b thread_start + 15

  Here the crash is in tcg/optimize.c line 212:

mask = si->mask;

  "si" is NULL. The NULL value arises from tcg/optimize.c line 198:

   si = ts_info(src_ts);

  I did not attempt to determine the root cause of this issue, however.
  It clearly is related to the "tcg/optimize" changes in this commit.
  The previous commit c0dd6654f207810b16a75b673258f5ce2ceffbf0 doesn't
  crash.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1912065/+subscriptions

Re: [PATCH v4] tcg: Toggle page execution for Apple Silicon

[Roman Bolshakov]

On Sat, Jan 23, 2021 at 07:33:37PM +0100, BALATON Zoltan wrote:
> On Sat, 23 Jan 2021, Roman Bolshakov wrote:
> > On Sat, Jan 23, 2021 at 02:53:49PM +0300, Roman Bolshakov wrote:
> > > On Thu, Jan 21, 2021 at 08:47:52AM -1000, Richard Henderson wrote:
> > > > From: Roman Bolshakov 
> > > > 
> > > > Pages can't be both write and executable at the same time on Apple
> > > > Silicon. macOS provides public API to switch write protection [1] for
> > > > JIT applications, like TCG.
> > > > 
> > > > 1. 
> > > > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> > > > 
> > > > Signed-off-by: Roman Bolshakov 
> > > > Message-Id: <20210113032806.18220-1-r.bolsha...@yadro.com>
> > > > [rth: Inline the qemu_thread_jit_* functions;
> > > >  drop the MAP_JIT change for a follow-on patch.]
> > > > Signed-off-by: Richard Henderson 
> > > > ---
> > > > 
> > > > Supercedes: <20210113032806.18220-1-r.bolsha...@yadro.com>
> > > > 
> > > > This is the version of Roman's patch that I'm queuing to tcg-next.
> > > > What's missing from the full "Fix execution" patch is setting MAP_JIT
> > > > for !splitwx in alloc_code_gen_buffer().
> > > > 
> > > 
> > > Richard, thanks for updating the patch. I have no objections against
> > > moving the functions and inlining them. However I'm seeing an issue that
> > > wasn't present in v3:
> > > 
> > > Process 37109 stopped 
> > >  * 
> > > thread #6, stop reason = EXC_BAD_ACCESS (code=1, 
> > > address=0xfd4f)
> > > frame #0: 0x0001002f1c90 
> > > qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 
> > > [opt]   2528 TCGOp 
> > > *tcg_emit_op(TCGOpcode opc)
> > >2529 { 
> > > 
> > > 2530 TCGOp *op = tcg_op_alloc(opc);
> > > -> 2531 QTAILQ_INSERT_TAIL(_ctx->ops, op, link);
> > >2532 return op;
> > >2533 }
> > >2534
> > > Target 0: (qemu-system-x86_64) stopped.
> > > (lldb) bt
> > > * thread #6, stop reason = EXC_BAD_ACCESS (code=1, 
> > > address=0xfd4f)
> > >   * frame #0: 0x0001002f1c90 
> > > qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 [opt]
> > > frame #1: 0x00010026f040 qemu-system-x86_64`tcg_gen_addi_i64 
> > > [inlined] tcg_gen_op3(opc=INDEX_op_add_i64, a1=4430334952, a2=4430333440,
> > > a3=4430361496) at tcg-op.c:60:17 [opt]
> > > frame #2: 0x00010026f038 qemu-system-x86_64`tcg_gen_addi_i64 
> > > [inlined] tcg_gen_op3_i64(opc=INDEX_op_add_i64, a1=, a2= > > ailable>, a3=) at tcg-op.h:94 [opt]
> > > frame #3: 0x00010026f030 qemu-system-x86_64`tcg_gen_addi_i64 
> > > [inlined] tcg_gen_add_i64(ret=, arg1=, arg2= > > vailable>) at tcg-op.h:618 [opt]
> > > frame #4: 0x00010026f030 
> > > qemu-system-x86_64`tcg_gen_addi_i64(ret=, 
> > > arg1=, arg2=) at tcg-op.c:123
> > > 5 [opt]
> > > frame #5: 0x00010021d1e0 
> > > qemu-system-x86_64`gen_lea_modrm_1(s=, a=(def_seg = 2, base 
> > > = 5, index = -1, scale = 0, disp = -6
> > > 89)) at translate.c:2101:9 [opt]
> > > frame #6: 0x00010020eeec qemu-system-x86_64`disas_insn [inlined] 
> > > gen_lea_modrm(env=0x000118610870, s=0x0001700b6b00, modrm= > > vailable>) at translate.c:2111:15 [opt]
> > > frame #7: 0x00010020eec0 
> > > qemu-system-x86_64`disas_insn(s=0x0001700b6b00, cpu=) at 
> > > translate.c:5509 [opt]
> > > frame #8: 0x00010020bb44 
> > > qemu-system-x86_64`i386_tr_translate_insn(dcbase=0x0001700b6b00, 
> > > cpu=) at translate.c:8573:15
> > >  [opt]
> > > frame #9: 0x0001002fbcf8 
> > > qemu-system-x86_64`translator_loop(ops=0x000100b209c8, 
> > > db=0x0001700b6b00, cpu=0x000118608000, tb=0
> > > x000120017200, max_insns=512) at translator.c:0 [opt]
> > > frame #10: 0x00010020b73c 
> > > qemu-system-x86_64`gen_intermediate_code(cpu=, 
> > >

Re: [PATCH v4] tcg: Toggle page execution for Apple Silicon

[Roman Bolshakov]

On Sat, Jan 23, 2021 at 02:53:49PM +0300, Roman Bolshakov wrote:
> On Thu, Jan 21, 2021 at 08:47:52AM -1000, Richard Henderson wrote:
> > From: Roman Bolshakov 
> > 
> > Pages can't be both write and executable at the same time on Apple
> > Silicon. macOS provides public API to switch write protection [1] for
> > JIT applications, like TCG.
> > 
> > 1. 
> > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> > 
> > Signed-off-by: Roman Bolshakov 
> > Message-Id: <20210113032806.18220-1-r.bolsha...@yadro.com>
> > [rth: Inline the qemu_thread_jit_* functions;
> >  drop the MAP_JIT change for a follow-on patch.]
> > Signed-off-by: Richard Henderson 
> > ---
> > 
> > Supercedes: <20210113032806.18220-1-r.bolsha...@yadro.com>
> > 
> > This is the version of Roman's patch that I'm queuing to tcg-next.
> > What's missing from the full "Fix execution" patch is setting MAP_JIT
> > for !splitwx in alloc_code_gen_buffer().
> > 
> 
> Richard, thanks for updating the patch. I have no objections against
> moving the functions and inlining them. However I'm seeing an issue that
> wasn't present in v3:
> 
> Process 37109 stopped 
>  * thread #6, 
> stop reason = EXC_BAD_ACCESS (code=1, address=0xfd4f)
> frame #0: 0x0001002f1c90 
> qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 [opt]
>2528 TCGOp *tcg_emit_op(TCGOpcode opc)
>2529 { 
> 2530 
> TCGOp *op = tcg_op_alloc(opc);
> -> 2531 QTAILQ_INSERT_TAIL(_ctx->ops, op, link);
>2532 return op;
>2533 }
>2534
> Target 0: (qemu-system-x86_64) stopped.
> (lldb) bt
> * thread #6, stop reason = EXC_BAD_ACCESS (code=1, address=0xfd4f)
>   * frame #0: 0x0001002f1c90 
> qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 [opt]
> frame #1: 0x00010026f040 qemu-system-x86_64`tcg_gen_addi_i64 
> [inlined] tcg_gen_op3(opc=INDEX_op_add_i64, a1=4430334952, a2=4430333440,
> a3=4430361496) at tcg-op.c:60:17 [opt]
> frame #2: 0x00010026f038 qemu-system-x86_64`tcg_gen_addi_i64 
> [inlined] tcg_gen_op3_i64(opc=INDEX_op_add_i64, a1=, a2= ailable>, a3=) at tcg-op.h:94 [opt]
> frame #3: 0x00010026f030 qemu-system-x86_64`tcg_gen_addi_i64 
> [inlined] tcg_gen_add_i64(ret=, arg1=, arg2= vailable>) at tcg-op.h:618 [opt]
> frame #4: 0x00010026f030 
> qemu-system-x86_64`tcg_gen_addi_i64(ret=, arg1=, 
> arg2=) at tcg-op.c:123
> 5 [opt]
> frame #5: 0x00010021d1e0 
> qemu-system-x86_64`gen_lea_modrm_1(s=, a=(def_seg = 2, base = 5, 
> index = -1, scale = 0, disp = -6
> 89)) at translate.c:2101:9 [opt]
> frame #6: 0x00010020eeec qemu-system-x86_64`disas_insn [inlined] 
> gen_lea_modrm(env=0x000118610870, s=0x0001700b6b00, modrm= vailable>) at translate.c:2111:15 [opt]
> frame #7: 0x00010020eec0 
> qemu-system-x86_64`disas_insn(s=0x0001700b6b00, cpu=) at 
> translate.c:5509 [opt]
> frame #8: 0x00010020bb44 
> qemu-system-x86_64`i386_tr_translate_insn(dcbase=0x0001700b6b00, 
> cpu=) at translate.c:8573:15
>  [opt]
> frame #9: 0x0001002fbcf8 
> qemu-system-x86_64`translator_loop(ops=0x000100b209c8, 
> db=0x0001700b6b00, cpu=0x000118608000, tb=0
> x000120017200, max_insns=512) at translator.c:0 [opt]
> frame #10: 0x00010020b73c 
> qemu-system-x86_64`gen_intermediate_code(cpu=, tb=, 
> max_insns=) at tra
> nslate.c:8635:5 [opt]
> frame #11: 0x000100257970 
> qemu-system-x86_64`tb_gen_code(cpu=0x000118608000, pc=, 
> cs_base=0, flags=4194483, cflags=-16
> 777216) at translate-all.c:1931:5 [opt]
> frame #12: 0x0001002deb90 qemu-system-x86_64`cpu_exec [inlined] 
> tb_find(cpu=0x000118608000, last_tb=0x, tb_exit= available>, cf_mask=0) at cpu-exec.c:456:14 [opt]
> frame #13: 0x0001002deb54 
> qemu-system-x86_64`cpu_exec(cpu=0x000118608000) at cpu-exec.c:812 [opt]
> frame #14: 0x0001002bc0d0 
> qemu-system-x86_64`tcg_cpus_exec(cpu=0x000118608000) at tcg-cpus.c:57:11 
> [opt]
> frame #15: 0x00010024c2cc 
> qemu-system-x86_64`rr_cpu_thread_fn(arg=) at 
> tcg-cpus-rr.c:217:21 [opt]
> frame #16: 0x0001004b00b4 
> qemu-system-x86_64`qemu_thread_start(args=) at 
> qemu-thread-posix.c:521:9 [opt]
> fr

Re: [PATCH v4] tcg: Toggle page execution for Apple Silicon

[Roman Bolshakov]

On Thu, Jan 21, 2021 at 08:47:52AM -1000, Richard Henderson wrote:
> From: Roman Bolshakov 
> 
> Pages can't be both write and executable at the same time on Apple
> Silicon. macOS provides public API to switch write protection [1] for
> JIT applications, like TCG.
> 
> 1. 
> https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> 
> Signed-off-by: Roman Bolshakov 
> Message-Id: <20210113032806.18220-1-r.bolsha...@yadro.com>
> [rth: Inline the qemu_thread_jit_* functions;
>  drop the MAP_JIT change for a follow-on patch.]
> Signed-off-by: Richard Henderson 
> ---
> 
> Supercedes: <20210113032806.18220-1-r.bolsha...@yadro.com>
> 
> This is the version of Roman's patch that I'm queuing to tcg-next.
> What's missing from the full "Fix execution" patch is setting MAP_JIT
> for !splitwx in alloc_code_gen_buffer().
> 

Richard, thanks for updating the patch. I have no objections against
moving the functions and inlining them. However I'm seeing an issue that
wasn't present in v3:

Process 37109 stopped   
   * thread #6, 
stop reason = EXC_BAD_ACCESS (code=1, address=0xfd4f)
frame #0: 0x0001002f1c90 
qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 [opt]  
 2528 TCGOp *tcg_emit_op(TCGOpcode opc)
   2529 {   
  2530 
TCGOp *op = tcg_op_alloc(opc);
-> 2531 QTAILQ_INSERT_TAIL(_ctx->ops, op, link);
   2532 return op;
   2533 }
   2534
Target 0: (qemu-system-x86_64) stopped.
(lldb) bt
* thread #6, stop reason = EXC_BAD_ACCESS (code=1, address=0xfd4f)
  * frame #0: 0x0001002f1c90 
qemu-system-x86_64`tcg_emit_op(opc=INDEX_op_add_i64) at tcg.c:2531:5 [opt]
frame #1: 0x00010026f040 qemu-system-x86_64`tcg_gen_addi_i64 [inlined] 
tcg_gen_op3(opc=INDEX_op_add_i64, a1=4430334952, a2=4430333440,
a3=4430361496) at tcg-op.c:60:17 [opt]
frame #2: 0x00010026f038 qemu-system-x86_64`tcg_gen_addi_i64 [inlined] 
tcg_gen_op3_i64(opc=INDEX_op_add_i64, a1=, a2=, a3=) at tcg-op.h:94 [opt]
frame #3: 0x00010026f030 qemu-system-x86_64`tcg_gen_addi_i64 [inlined] 
tcg_gen_add_i64(ret=, arg1=, arg2=) at tcg-op.h:618 [opt]
frame #4: 0x00010026f030 
qemu-system-x86_64`tcg_gen_addi_i64(ret=, arg1=, 
arg2=) at tcg-op.c:123
5 [opt]
frame #5: 0x00010021d1e0 
qemu-system-x86_64`gen_lea_modrm_1(s=, a=(def_seg = 2, base = 5, 
index = -1, scale = 0, disp = -6
89)) at translate.c:2101:9 [opt]
frame #6: 0x00010020eeec qemu-system-x86_64`disas_insn [inlined] 
gen_lea_modrm(env=0x000118610870, s=0x0001700b6b00, modrm=) at translate.c:2111:15 [opt]
frame #7: 0x00010020eec0 
qemu-system-x86_64`disas_insn(s=0x0001700b6b00, cpu=) at 
translate.c:5509 [opt]
frame #8: 0x00010020bb44 
qemu-system-x86_64`i386_tr_translate_insn(dcbase=0x0001700b6b00, 
cpu=) at translate.c:8573:15
 [opt]
frame #9: 0x0001002fbcf8 
qemu-system-x86_64`translator_loop(ops=0x000100b209c8, 
db=0x0001700b6b00, cpu=0x000118608000, tb=0
x000120017200, max_insns=512) at translator.c:0 [opt]
frame #10: 0x00010020b73c 
qemu-system-x86_64`gen_intermediate_code(cpu=, tb=, 
max_insns=) at tra
nslate.c:8635:5 [opt]
frame #11: 0x000100257970 
qemu-system-x86_64`tb_gen_code(cpu=0x000118608000, pc=, 
cs_base=0, flags=4194483, cflags=-16
777216) at translate-all.c:1931:5 [opt]
frame #12: 0x0001002deb90 qemu-system-x86_64`cpu_exec [inlined] 
tb_find(cpu=0x000118608000, last_tb=0x, tb_exit=, cf_mask=0) at cpu-exec.c:456:14 [opt]
frame #13: 0x0001002deb54 
qemu-system-x86_64`cpu_exec(cpu=0x000118608000) at cpu-exec.c:812 [opt]
frame #14: 0x0001002bc0d0 
qemu-system-x86_64`tcg_cpus_exec(cpu=0x000118608000) at tcg-cpus.c:57:11 
[opt]
frame #15: 0x00010024c2cc 
qemu-system-x86_64`rr_cpu_thread_fn(arg=) at tcg-cpus-rr.c:217:21 
[opt]
frame #16: 0x0001004b00b4 
qemu-system-x86_64`qemu_thread_start(args=) at 
qemu-thread-posix.c:521:9 [opt]
frame #17: 0x000191c4d06c libsystem_pthread.dylib`_pthread_start + 320

I'm looking into the issue but perhaps we'll need v5.

Best regards,
Roman

> 
> r~
> 
> ---
>  include/qemu/osdep.h  | 28 
>  accel/tcg/cpu-exec.c  |  2 ++
>  accel/tcg/translate-all.c |  3 +++
>  tcg/tcg.c |  1 +
>  4 files changed, 34 insertions(+)
> 
> diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
> index a434382c58..b6ffdc15bf 100644
> --- a/include/qemu/osdep.h
> +++ b/include/qemu/osdep.h
> @@ -119,6 +119,

Re: [PATCH v2] target/i386/hvf: add vmware-cpuid-freq cpu feature

[Roman Bolshakov]

On Thu, Jan 14, 2021 at 10:47:03PM +0300, yaroshchuk2...@gmail.com wrote:
> From: Vladislav Yaroshchuk 
> 
> For `-accel hvf` cpu_x86_cpuid() is wrapped with hvf_cpu_x86_cpuid() to
> add paravirtualization cpuid leaf 0x4010
> https://lkml.org/lkml/2008/10/1/246
> 
> Leaf 0x4010, Timing Information:
> EAX: (Virtual) TSC frequency in kHz.
> EBX: (Virtual) Bus (local apic timer) frequency in kHz.
> ECX, EDX: RESERVED (Per above, reserved fields are set to zero).
> 
> On macOS TSC and APIC Bus frequencies can be readed by sysctl call with
> names `machdep.tsc.frequency` and `hw.busfrequency`
> 
> This options is required for Darwin-XNU guest to be synchronized with
> host
> 
> Signed-off-by: Vladislav Yaroshchuk 
> ---
>  target/i386/hvf/hvf.c | 90 ++-
>  1 file changed, 89 insertions(+), 1 deletion(-)
> 
> diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
> index ed9356565c..a5daafe202 100644
> --- a/target/i386/hvf/hvf.c
> +++ b/target/i386/hvf/hvf.c
> @@ -65,6 +65,7 @@
>  
>  #include 
>  #include 
> +#include 
>  
>  #include "exec/address-spaces.h"
>  #include "hw/i386/apic_internal.h"
> @@ -456,6 +457,48 @@ static void dummy_signal(int sig)
>  {
>  }
>  
> +static void init_tsc_freq(CPUX86State *env)
> +{
> +size_t length;
> +uint64_t tsc_freq;
> +
> +if (env->tsc_khz != 0) {
> +return;
> +}
> +
> +length = sizeof(uint64_t);
> +if (sysctlbyname("machdep.tsc.frequency", _freq, , NULL, 0)) {
> +return;
> +}
> +env->tsc_khz = tsc_freq / 1000;  /* Hz to KHz */
> +}
> +
> +static void init_apic_bus_freq(CPUX86State *env)
> +{
> +size_t length;
> +uint64_t bus_freq;
> +
> +if (env->apic_bus_freq != 0) {
> +return;
> +}
> +
> +length = sizeof(uint64_t);
> +if (sysctlbyname("hw.busfrequency", _freq, , NULL, 0)) {
> +return;
> +}
> +env->apic_bus_freq = bus_freq;
> +}
> +
> +static inline bool tsc_is_known(CPUX86State *env)
> +{
> +return env->tsc_khz != 0;
> +}
> +
> +static inline bool apic_bus_freq_is_known(CPUX86State *env)
> +{
> +return env->apic_bus_freq != 0;
> +}
> +
>  int hvf_init_vcpu(CPUState *cpu)
>  {
>  
> @@ -480,6 +523,15 @@ int hvf_init_vcpu(CPUState *cpu)
>  hvf_state->hvf_caps = g_new0(struct hvf_vcpu_caps, 1);
>  env->hvf_mmio_buf = g_new(char, 4096);
>  
> +if (x86cpu->vmware_cpuid_freq) {
> +init_tsc_freq(env);
> +init_apic_bus_freq(env);
> +
> +if (!tsc_is_known(env) || !apic_bus_freq_is_known(env)) {
> +error_report("vmware-cpuid-freq: feature couldn't be enabled");
> +}
> +}
> +
>  r = hv_vcpu_create((hv_vcpuid_t *)>hvf_fd, HV_VCPU_DEFAULT);
>  cpu->vcpu_dirty = 1;
>  assert_hvf_ok(r);
> @@ -597,6 +649,42 @@ static void hvf_store_events(CPUState *cpu, uint32_t 
> ins_len, uint64_t idtvec_in
>  }
>  }
>  
> +static void hvf_cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t 
> count,
> +  uint32_t *eax, uint32_t *ebx,
> +  uint32_t *ecx, uint32_t *edx)
> +{
> +/*
> + * A wrapper extends cpu_x86_cpuid with 0x4000 and 0x4010 leafs
> + * Provides vmware-cpuid-freq support to hvf
> + */
> +
> +uint32_t signature[3];
> +
> +if (!tsc_is_known(env) || !apic_bus_freq_is_known(env)) {
> +cpu_x86_cpuid(env, index, count, eax, ebx, ecx, edx);
> +return;
> +}
> +
> +switch (index) {
> +case 0x4000:
> +memcpy(signature, "TCGTCGTCGTCG", 12); /* QEMU Signature */

Hi Vladislav,

TCG belongs to TCG accel identification, for HVF it should be
HVFHVFHVFHVF.

> +*eax = 0x4010; /* Max available cpuid leaf */
> +*ebx = signature[0];
> +*ecx = signature[1];
> +*edx = signature[2];

TCG and KVM don't report their identity unless kvm or tcg-cpuid
properties are set. I wonder if we need to guard it likewise?

But as of now QEMU is not consistent in that regard. Two parameters are
needed for KVM - kvm=on,vmware-cpuid-freq=on. vmware-cpuid-freq is
sufficient for WHPX but WHPX doesn't expose itself (ebx=ecx=edx=0). TCG
doesn't seem to support vmware-cpuid-freq but reports it's name only if
tcg-cpuid property is set.

> +break;

CPUID for not implemented hypervisor-specific leafs from 0x4001 up
to 0x400f should be all zeroes but cpu_x86_cpuid() only returns zero
values for 0x4001. Likely, you need to reset return values for the
leafs here or in cpu_x86_cpuid(). In the latter case you'll also fix a
similar bug in WHPX accel.

Otherwise, looks good.

Thanks,
Roman

> +case 0x4010:
> +*eax = env->tsc_khz;
> +*ebx = env->apic_bus_freq / 1000; /* Hz to KHz */
> +*ecx = 0;
> +*edx = 0;
> +break;
> +default:
> +cpu_x86_cpuid(env, index, count, eax, ebx, ecx, edx);
> +break;
> +}
> +}
> +
>  int 

Re: [PATCH v3] hvf: guard xgetbv call.

[Roman Bolshakov]

On Tue, Jan 12, 2021 at 10:07:35PM -0800, Hill Ma wrote:
> This prevents illegal instruction on cpus do not support xgetbv.
> 
> Buglink: https://bugs.launchpad.net/qemu/+bug/1758819
> Signed-off-by: Hill Ma 
> ---
> v3: addressed feedback.
> v2: xgetbv() modified based on feedback.
> 
>  target/i386/hvf/x86_cpuid.c | 34 ++
>  1 file changed, 22 insertions(+), 12 deletions(-)
> 

Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov 

Thanks,
Roman

> diff --git a/target/i386/hvf/x86_cpuid.c b/target/i386/hvf/x86_cpuid.c
> index a6842912f5..32b0d131df 100644
> --- a/target/i386/hvf/x86_cpuid.c
> +++ b/target/i386/hvf/x86_cpuid.c
> @@ -27,15 +27,22 @@
>  #include "vmx.h"
>  #include "sysemu/hvf.h"
>  
> -static uint64_t xgetbv(uint32_t xcr)
> +static bool xgetbv(uint32_t cpuid_ecx, uint32_t idx, uint64_t *xcr)
>  {
> -uint32_t eax, edx;
> +uint32_t xcrl, xcrh;
>  
> -__asm__ volatile ("xgetbv"
> -  : "=a" (eax), "=d" (edx)
> -  : "c" (xcr));
> +if (cpuid_ecx & CPUID_EXT_OSXSAVE) {
> +/*
> + * The xgetbv instruction is not available to older versions of
> + * the assembler, so we encode the instruction manually.
> + */
> +asm(".byte 0x0f, 0x01, 0xd0" : "=a" (xcrl), "=d" (xcrh) : "c" (idx));
>  
> -return (((uint64_t)edx) << 32) | eax;
> +*xcr = (((uint64_t)xcrh) << 32) | xcrl;
> +return true;
> +}
> +
> +return false;
>  }
>  
>  uint32_t hvf_get_supported_cpuid(uint32_t func, uint32_t idx,
> @@ -100,12 +107,15 @@ uint32_t hvf_get_supported_cpuid(uint32_t func, 
> uint32_t idx,
>  break;
>  case 0xD:
>  if (idx == 0) {
> -uint64_t host_xcr0 = xgetbv(0);
> -uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK | 
> XSTATE_SSE_MASK |
> -  XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK |
> -  XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK |
> -  XSTATE_ZMM_Hi256_MASK | 
> XSTATE_Hi16_ZMM_MASK);
> -eax &= supp_xcr0;
> +uint64_t host_xcr0;
> +if (xgetbv(ecx, 0, _xcr0)) {
> +uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK |
> +  XSTATE_SSE_MASK | XSTATE_YMM_MASK |
> +  XSTATE_BNDREGS_MASK | XSTATE_BNDCSR_MASK |
> +  XSTATE_OPMASK_MASK | XSTATE_ZMM_Hi256_MASK 
> |
> +  XSTATE_Hi16_ZMM_MASK);
> +eax &= supp_xcr0;
> +}
>  } else if (idx == 1) {
>  hv_vmx_read_capability(HV_VMX_CAP_PROCBASED2, );
>  eax &= CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XGETBV1;
> -- 
> 2.20.1 (Apple Git-117)
> 

[PATCH v3] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

Pages can't be both write and executable at the same time on Apple
Silicon. macOS provides public API to switch write protection [1] for
JIT applications, like TCG.

1. 
https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon

Signed-off-by: Roman Bolshakov 
---
v2: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00146.html
Changes since v2:
 - Wrapped pthread_jit_write_protect_np() with __builtin_available() [1]
   to allow build with modern SDK while targeting older macOS (Joelle)
 - Dropped redundant calls to pthread_jit_write_protect_supported_np()
   (Alex)

v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
Changes since v1:

 - Pruned not needed fiddling with W^X and dropped symmetry from write
   lock/unlock and renamed related functions.
   Similar approach is used in JavaScriptCore [2].

 - Moved jit helper functions to util/osdep

  As outlined 
in osdep.h, this matches to (2):
   * In an ideal world this header would contain only:
   *  (1) things which everybody needs
   *  (2) things without which code would work on most platforms but
   *  fail to compile or misbehave on a minority of host OSes

 - Fixed a checkpatch error

 - Limit new behaviour only to macOS 11.0 and above, because of the
   following declarations:

   __API_AVAILABLE(macos(11.0))
   __API_UNAVAILABLE(ios, tvos, watchos)
   void pthread_jit_write_protect_np(int enabled);

   __API_AVAILABLE(macos(11.0))
   __API_UNAVAILABLE(ios, tvos, watchos)
   int pthread_jit_write_protect_supported_np(void);

 1. https://developer.apple.com/videos/play/wwdc2017/411/
 2. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch

 accel/tcg/cpu-exec.c  |  2 ++
 accel/tcg/translate-all.c |  9 +
 include/qemu/osdep.h  |  7 +++
 tcg/tcg.c |  1 +
 util/osdep.c  | 20 
 5 files changed, 39 insertions(+)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index e0df9b6a1d..014810bf0a 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -185,6 +185,7 @@ cpu_tb_exec(CPUState *cpu, TranslationBlock *itb, int 
*tb_exit)
 }
 #endif /* DEBUG_DISAS */
 
+qemu_thread_jit_execute();
 ret = tcg_qemu_tb_exec(env, tb_ptr);
 cpu->can_do_io = 1;
 /*
@@ -405,6 +406,7 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
 {
 uintptr_t old;
 
+qemu_thread_jit_write();
 assert(n < ARRAY_SIZE(tb->jmp_list_next));
 qemu_spin_lock(_next->jmp_lock);
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index e9de6ff9dd..f5f4c7cc17 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1083,6 +1083,12 @@ static bool alloc_code_gen_buffer_anon(size_t size, int 
prot,
 {
 void *buf;
 
+#if defined(MAC_OS_VERSION_11_0) && \
+MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0
+if (__builtin_available(macOS 11.0, *)) {
+flags |= MAP_JIT;
+}
+#endif
 buf = mmap(NULL, size, prot, flags, -1, 0);
 if (buf == MAP_FAILED) {
 error_setg_errno(errp, errno,
@@ -1669,7 +1675,9 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, 
bool rm_from_page_list)
 
 static void tb_phys_invalidate__locked(TranslationBlock *tb)
 {
+qemu_thread_jit_write();
 do_tb_phys_invalidate(tb, true);
+qemu_thread_jit_execute();
 }
 
 /* invalidate one TB
@@ -1871,6 +1879,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 #endif
 
 assert_memory_lock();
+qemu_thread_jit_write();
 
 phys_pc = get_page_addr_code(env, pc);
 
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index f9ec8c84e9..929e970b0e 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -123,6 +123,10 @@ extern int daemon(int, int);
 #include "sysemu/os-posix.h"
 #endif
 
+#ifdef __APPLE__
+#include 
+#endif
+
 #include "glib-compat.h"
 #include "qemu/typedefs.h"
 
@@ -686,4 +690,7 @@ char *qemu_get_host_name(Error **errp);
  */
 size_t qemu_get_host_physmem(void);
 
+void qemu_thread_jit_write(void);
+void qemu_thread_jit_execute(void);
+
 #endif
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 472bf1755b..16b044eae7 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -1112,6 +1112,7 @@ void tcg_prologue_init(TCGContext *s)
 s->pool_labels = NULL;
 #endif
 
+qemu_thread_jit_write();
 /* Generate the prologue.  */
 tcg_target_qemu_prologue(s);
 
diff --git a/util/osdep.c b/util/osdep.c
index 66d01b9160..e211939a0c 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -606,3 +606,23 @@ writev(int fd, const struct iovec *iov, int iov_cnt)
 return readv_writev(fd, iov, iov_cnt, true);
 }
 #endif
+
+#if defined(MAC_OS_VERSION_11_0) && \
+MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0
+void qemu_

Re: [PATCH] hvf: guard xgetbv call.

[Roman Bolshakov]

On Mon, Jan 11, 2021 at 07:06:22AM -1000, Richard Henderson wrote:
> On 1/10/21 6:31 PM, Roman Bolshakov wrote:
> >> Also, if we're going to put this some place common, forcing the caller to 
> >> do
> >> the cpuid that feeds this, then we should probably make all of the startup
> >> cpuid stuff common as well.
> >>
> > 
> > I proposed the version because all callers of xgetbv instruction already
> > call cpuid before invoking inline xgetbv.
> > 
> >> Note that we'd probably have to use constructor priorities to get that 
> >> right
> >> for util/bufferiszero.c.
> >>
> > 
> > Please correct me if I read this wrong. What you're saying is we should
> > initialize cpuid in constructors and then use cached cpuid ecx in
> > xgetbv() (and drop one argument, respectively)?
> 
> I would have xgetbv and all of the cpuid values cached at startup, and all
> three places would read global variables.
> 

It makes sense, thanks for the clarification.

Do you think it should be part of the change Hill is doing or it might
be a follow-up cleanup patch?

-Roman

Re: [PATCH v2] hvf: guard xgetbv call.

[Roman Bolshakov]

On Mon, Jan 11, 2021 at 09:44:40PM -0800, Hill Ma wrote:
> On Sun, Jan 10, 2021 at 8:38 PM Roman Bolshakov  wrote:
> > I'm not sure if eax should be modified with mask because the mask has no
> > value per se. I.e. eax &= supp_xcr0 from below should be placed inside
> > the if. It'd express clearly that eax is not modified unless xgetbv is
> > supported.
> 
> Like this?
> 
> -uint64_t host_xcr0 = xgetbv(0);
> -uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK |
> XSTATE_SSE_MASK |
> +uint64_t host_xcr0;
> +if (xgetbv(ecx, 0, _xcr0)) {
> +uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK |
> XSTATE_SSE_MASK |
>XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK |
>XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK |
>XSTATE_ZMM_Hi256_MASK |
> XSTATE_Hi16_ZMM_MASK);
> -eax &= supp_xcr0;
> +eax &= supp_xcr0;
> +}

Yes.

Thanks,
Roman

Re: [PATCH v2] hvf: guard xgetbv call.

[Roman Bolshakov]

On Sun, Jan 10, 2021 at 01:08:54PM -0800, Hill Ma wrote:
> This prevents illegal instruction on cpus do not support xgetbv.
> 
> Buglink: https://bugs.launchpad.net/qemu/+bug/1758819
> Signed-off-by: Hill Ma 
> ---
>  v2: xgetbv() modified based on feedback.
> 
>  target/i386/hvf/x86_cpuid.c | 28 +++-
>  1 file changed, 19 insertions(+), 9 deletions(-)
> 
> diff --git a/target/i386/hvf/x86_cpuid.c b/target/i386/hvf/x86_cpuid.c
> index a6842912f5..edaa1b7da2 100644
> --- a/target/i386/hvf/x86_cpuid.c
> +++ b/target/i386/hvf/x86_cpuid.c
> @@ -27,15 +27,22 @@
>  #include "vmx.h"
>  #include "sysemu/hvf.h"
>  
> -static uint64_t xgetbv(uint32_t xcr)
> +static bool xgetbv(uint32_t cpuid_ecx, uint32_t idx, uint64_t *xcr)
>  {
> -uint32_t eax, edx;
> +uint32_t xcrl, xcrh;
>  
> -__asm__ volatile ("xgetbv"
> -  : "=a" (eax), "=d" (edx)
> -  : "c" (xcr));
> +if (cpuid_ecx & CPUID_EXT_OSXSAVE) {
> +/*
> + * The xgetbv instruction is not available to older versions of
> + * the assembler, so we encode the instruction manually.
> + */
> +asm(".byte 0x0f, 0x01, 0xd0" : "=a" (xcrl), "=d" (xcrh) : "c" (idx));
>  
> -return (((uint64_t)edx) << 32) | eax;
> +*xcr = (((uint64_t)xcrh) << 32) | xcrl;
> +return true;
> +}
> +
> +return false;
>  }
>  
>  uint32_t hvf_get_supported_cpuid(uint32_t func, uint32_t idx,
> @@ -100,11 +107,14 @@ uint32_t hvf_get_supported_cpuid(uint32_t func, 
> uint32_t idx,
>  break;
>  case 0xD:
>  if (idx == 0) {
> -uint64_t host_xcr0 = xgetbv(0);
> -uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK | 
> XSTATE_SSE_MASK |
> +uint64_t supp_xcr0 = XSTATE_FP_MASK | XSTATE_SSE_MASK |
>XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK |
>XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK |
> -  XSTATE_ZMM_Hi256_MASK | 
> XSTATE_Hi16_ZMM_MASK);
> +  XSTATE_ZMM_Hi256_MASK | 
> XSTATE_Hi16_ZMM_MASK;
> +uint64_t host_xcr0;
> +if (xgetbv(ecx, 0, _xcr0)) {
> +supp_xcr0 &= host_xcr0;

Hi Hill,

I'm not sure if eax should be modified with mask because the mask has no
value per se. I.e. eax &= supp_xcr0 from below should be placed inside
the if. It'd express clearly that eax is not modified unless xgetbv is
supported.

Thanks,
Roman

> +}
>  eax &= supp_xcr0;
>  } else if (idx == 1) {
>  hv_vmx_read_capability(HV_VMX_CAP_PROCBASED2, );
> -- 
> 2.20.1 (Apple Git-117)
> 

Re: [PATCH] hvf: guard xgetbv call.

[Roman Bolshakov]

On Sun, Jan 10, 2021 at 08:38:36AM -1000, Richard Henderson wrote:
> On 1/10/21 8:34 AM, Richard Henderson wrote:
> > On 1/9/21 3:46 PM, Roman Bolshakov wrote:
> >> +static int xgetbv(uint32_t cpuid_ecx, uint32_t idx, uint64_t *xcr)
> >>  {
> >> -uint32_t eax, edx;
> >> +uint32_t xcrl, xcrh;
> >>
> >> -__asm__ volatile ("xgetbv"
> >> -  : "=a" (eax), "=d" (edx)
> >> -  : "c" (xcr));
> >> +if (cpuid_ecx && CPUID_EXT_OSXSAVE) {
> >> +/* The xgetbv instruction is not available to older versions of
> >> + * the assembler, so we encode the instruction manually.
> >> + */
> >> +asm(".byte 0x0f, 0x01, 0xd0" : "=a" (xcrl), "=d" (xcrh) : "c" 
> >> (idx));
> >>
> >> -return (((uint64_t)edx) << 32) | eax;
> >> +*xcr = (((uint64_t)xcrh) << 32) | xcrl;
> >> +return 0;
> >> +}
> >> +
> >> +return 1;
> >>  }
> > 
> > Not to bikeshed too much, but this looks like it should return bool, and 
> > true
> > on success, not the other way around.
> 

I agree, it'd better to comprehend (and Hill has already sent v2 with
this).

> Also, if we're going to put this some place common, forcing the caller to do
> the cpuid that feeds this, then we should probably make all of the startup
> cpuid stuff common as well.
> 

I proposed the version because all callers of xgetbv instruction already
call cpuid before invoking inline xgetbv.

> Note that we'd probably have to use constructor priorities to get that right
> for util/bufferiszero.c.
> 

Please correct me if I read this wrong. What you're saying is we should
initialize cpuid in constructors and then use cached cpuid ecx in
xgetbv() (and drop one argument, respectively)?

Thanks,
Roman

Re: [PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Sun, Jan 10, 2021 at 02:13:48AM +0100, BALATON Zoltan wrote:
> On Sun, 10 Jan 2021, Roman Bolshakov wrote:
> > On Sat, Jan 09, 2021 at 12:13:36AM +0100, BALATON Zoltan wrote:
> > > On Sat, 9 Jan 2021, Roman Bolshakov wrote:
> > > > On Fri, Jan 08, 2021 at 03:00:07PM +, Peter Maydell wrote:
> > > > > On Fri, 8 Jan 2021 at 13:50, Peter Maydell  
> > > > > wrote:
> > > > > > 
> > > > > > On Sat, 2 Jan 2021 at 15:14, Roman Bolshakov 
> > > > > >  wrote:
> > > > > > > 
> > > > > > > ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first 
> > > > > > > deprecated in macOS 11.0 - Use -[NSWorkspace openURL:] instead.
> > > > > > >   [-Wdeprecated-declarations]
> > > > > > > if ([[NSWorkspace sharedWorkspace] openFile: 
> > > > > > > full_file_path] == YES) {
> > > > > > >^
> > > > > > > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
> > > > > > >  note:
> > > > > > >   'openFile:' has been explicitly marked deprecated here
> > > > > > > - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use 
> > > > > > > -[NSWorkspace openURL:] instead.", macos(10.0, 11.0));
> > > > > > > ^
> > > > > > > 
> > > > > > > Signed-off-by: Roman Bolshakov 
> > > > > > > ---
> > > > > > 
> > > > > > Reviewed-by: Peter Maydell 
> > > > > 
> > > > > 
> > > > > So I was just trying to test this patch, and I found that at least
> > > > > for me the osx menu bar has stopped working in QEMU -- keyboard
> > > > > shortcuts to it still work but none of the menu buttons respond
> > > > > to the mouse. Does that happen for anybody else?
> > > > > 
> > > > 
> > > > There's an old bug when QEMU menu bar is not responsive because it's not
> > > > properly activated. If you click off qemu and click on the qemu dock
> > > > icon then it "gets fixed" (cmd-tab works too). Do you hit the issue as
> > > > described in the article [1]? The code in the article does exactly the
> > > > same what I'm doing manually. I wanted to fix it but somehow it got
> > > > postponed for like a whole year :) I might try to make a fix this but
> > > > note, the issue is not related to the patch.
> > > 
> > > This does not sound like the best solution to the problem. There's some 
> > > info
> > > on this here (and blog post linked from it):
> > > 
> > > https://stackoverflow.com/questions/7460092/nswindow-makekeyandorderfront-makes-window-appear-but-not-key-or-front
> > > 
> > > Maybe we call makeKeyAndOrderFront: too early before the app is active and
> > > that's causing the problem? Would it work better if that's moved after
> > > [NSApp run]? (Maybe we also need canBecomeKey: somewhere but I don't see 
> > > why
> > > would that be needed for normal windows.)
> > > 
> > 
> > Hi Zoltan,
> > 
> > Thanks for the suggestions. I have tried to move it around but that
> > doesn't help. Note that minimal cococa app calls makeKeyAndOrderFront:
> > before [NSApp run] and doesn't experience the issue:
> > https://github.com/rgl/minimal-cocoa-app/blob/master/main.m
> 
> However this minimal app does call [NSApp activateIgnoringOtherApps:YES]
> before makeKeyAndOrderFront: and we don't seem to do that.

It's not really important. The minimal app is bundled and if you run it
outside of a bundle you'll get the very same issue we see with QEMU.

You can try it out if you apply the patch to the app (to avoid
references to the data in bundle's plist):
diff --git a/main.m b/main.m
index d5027a7..603c629 100644
--- a/main.m
+++ b/main.m
@@ -20,7 +20,7 @@ int main(void) {
 id appName = [bundleInfo objectForKey:@"CFBundleName"];
 id appVersion = [bundleInfo objectForKey:@"CFBundleVersion"];

-id quitMenuItemTitle = [@"Quit " stringByAppendingString:appName];
+id quitMenuItemTitle = @"Quit";
 id quitMenuItem = [[NSMenuItem alloc] autorelease];
 [quitMenuItem
 initWithTitle:quitMenuItemTitle
@@ -46,7 +46,7 @@ int main(void) {
 styleMask:NSTitledWindowMask
 backing:NSBackingStoreBuffered
   

Re: [PATCH] hvf: guard xgetbv call.

[Roman Bolshakov]

On Sat, Jan 09, 2021 at 11:42:18AM +, Peter Maydell wrote:
> On Sat, 9 Jan 2021 at 05:49, Roman Bolshakov  wrote:
> >
> > On Fri, Dec 18, 2020 at 06:13:47PM -0800, Hill Ma wrote:
> > > This prevents illegal instruction on cpus do not support xgetbv.
> > >
> > > Buglink: https://bugs.launchpad.net/qemu/+bug/1758819
> > > Signed-off-by: Hill Ma 
> > > ---
> > >  target/i386/hvf/x86_cpuid.c | 11 ---
> > >  1 file changed, 8 insertions(+), 3 deletions(-)
> > >
> >
> > Hi Hill,
> >
> > I'm sorry for delay with the review.
> 
> So, hvf added a third use of inline asm execution of "xgetbv" to
> the two we had already. Now we have:
>  * this in hvf
>  * a use in tcg_target_init() in tcg/i386/tcg-target.c.inc
>  * a use in init_cpuid_cache() in util/bufferiszero.c
> 
> Is it possible to abstract this out so we have one version
> of this, not three ? I note that the other two got the "avoid
> executing an illegal insn" tests right...

It surely is. If xgetbv() is extended like below and moved out of hvf,
we can reuse it in all other places and no duplication of #UD avoidance
will happen.

diff --git a/target/i386/hvf/x86_cpuid.c b/target/i386/hvf/x86_cpuid.c
index a6842912f5..7994f92d96 100644
--- a/target/i386/hvf/x86_cpuid.c
+++ b/target/i386/hvf/x86_cpuid.c
@@ -27,15 +27,21 @@
 #include "vmx.h"
 #include "sysemu/hvf.h"

-static uint64_t xgetbv(uint32_t xcr)
+static int xgetbv(uint32_t cpuid_ecx, uint32_t idx, uint64_t *xcr)
 {
-uint32_t eax, edx;
+uint32_t xcrl, xcrh;

-__asm__ volatile ("xgetbv"
-  : "=a" (eax), "=d" (edx)
-  : "c" (xcr));
+if (cpuid_ecx && CPUID_EXT_OSXSAVE) {
+/* The xgetbv instruction is not available to older versions of
+ * the assembler, so we encode the instruction manually.
+ */
+asm(".byte 0x0f, 0x01, 0xd0" : "=a" (xcrl), "=d" (xcrh) : "c" (idx));

-return (((uint64_t)edx) << 32) | eax;
+*xcr = (((uint64_t)xcrh) << 32) | xcrl;
+return 0;
+}
+
+return 1;
 }

 uint32_t hvf_get_supported_cpuid(uint32_t func, uint32_t idx,


Hill, feel free to update the three places Peter mentioned.  If it's
more convenient for you I can make complete patch.

Thanks,
Roman

> 
> thanks
> -- PMM

Re: [PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Sat, Jan 09, 2021 at 01:25:44PM +0100, Christian Schoenebeck via wrote:
> On Samstag, 9. Januar 2021 00:13:36 CET BALATON Zoltan wrote:
> > On Sat, 9 Jan 2021, Roman Bolshakov wrote:
> > > On Fri, Jan 08, 2021 at 03:00:07PM +, Peter Maydell wrote:
> > >> On Fri, 8 Jan 2021 at 13:50, Peter Maydell  
> wrote:
> > >>> On Sat, 2 Jan 2021 at 15:14, Roman Bolshakov  
> wrote:
> > >>>> ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first
> > >>>> deprecated in macOS 11.0 - Use -[NSWorkspace openURL:] instead.>>>> 
> > >>>>   [-Wdeprecated-declarations]
> > >>>>   
> > >>>> if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] ==
> > >>>> YES) {
> > >>>> 
> > >>>>^
> > >>>> 
> > >>>> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/
> Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1: note:
> > >>>>   'openFile:' has been explicitly marked deprecated here
> > >>>> 
> > >>>> - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace
> > >>>> openURL:] instead.", macos(10.0, 11.0)); ^
> > >>>> 
> > >>>> Signed-off-by: Roman Bolshakov 
> > >>>> ---
> > >>> 
> > >>> Reviewed-by: Peter Maydell 
> > >> 
> > >> So I was just trying to test this patch, and I found that at least
> > >> for me the osx menu bar has stopped working in QEMU -- keyboard
> > >> shortcuts to it still work but none of the menu buttons respond
> > >> to the mouse. Does that happen for anybody else?
> > > 
> > > There's an old bug when QEMU menu bar is not responsive because it's not
> > > properly activated. If you click off qemu and click on the qemu dock
> > > icon then it "gets fixed" (cmd-tab works too). Do you hit the issue as
> > > described in the article [1]? The code in the article does exactly the
> > > same what I'm doing manually. I wanted to fix it but somehow it got
> > > postponed for like a whole year :) I might try to make a fix this but
> > > note, the issue is not related to the patch.
> > 
> > This does not sound like the best solution to the problem. There's some
> > info on this here (and blog post linked from it):
> > 
> > https://stackoverflow.com/questions/7460092/nswindow-makekeyandorderfront-ma
> > kes-window-appear-but-not-key-or-front
> > 
> > Maybe we call makeKeyAndOrderFront: too early before the app is active and
> > that's causing the problem? Would it work better if that's moved after
> > [NSApp run]? (Maybe we also need canBecomeKey: somewhere but I don't see
> > why would that be needed for normal windows.)
> > 
> > Regards,
> > BALATON Zoltan
> 
> JFYI: I'm not sure whether that's related to this, but there was a general 
> event handling issue with Gtk3 on macOS which caused mouse events being 
> dropped:
> 
> https://gitlab.gnome.org/GNOME/gtk/-/issues/986

Hi Christian,

Thanks for the reference. I've looked at the patch and I'm not sure if
the Cocoa issues are related to GTK. It's likely something different.

After skimming over QT bug tracker I found a mathcing ticket that
confirms findings of earlier email:

  https://bugreports.qt.io/browse/QTBUG-89436

  Workaround is to build app as app bundle. Or manually deactivate and
  re-activate the app, like the JavaFX workaround does.

Regards,
Roman

> 
> According to the response, they seem to have fixed it meanwhile with a 
> different patch than suggested by me, but I haven't tested theirs.
> 
> Best regards,
> Christian Schoenebeck
> 
> 
> 

Re: [PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Sat, Jan 09, 2021 at 12:13:36AM +0100, BALATON Zoltan wrote:
> On Sat, 9 Jan 2021, Roman Bolshakov wrote:
> > On Fri, Jan 08, 2021 at 03:00:07PM +, Peter Maydell wrote:
> > > On Fri, 8 Jan 2021 at 13:50, Peter Maydell  
> > > wrote:
> > > > 
> > > > On Sat, 2 Jan 2021 at 15:14, Roman Bolshakov  
> > > > wrote:
> > > > > 
> > > > > ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first 
> > > > > deprecated in macOS 11.0 - Use -[NSWorkspace openURL:] instead.
> > > > >   [-Wdeprecated-declarations]
> > > > > if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] 
> > > > > == YES) {
> > > > >^
> > > > > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
> > > > >  note:
> > > > >   'openFile:' has been explicitly marked deprecated here
> > > > > - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use 
> > > > > -[NSWorkspace openURL:] instead.", macos(10.0, 11.0));
> > > > > ^
> > > > > 
> > > > > Signed-off-by: Roman Bolshakov 
> > > > > ---
> > > > 
> > > > Reviewed-by: Peter Maydell 
> > > 
> > > 
> > > So I was just trying to test this patch, and I found that at least
> > > for me the osx menu bar has stopped working in QEMU -- keyboard
> > > shortcuts to it still work but none of the menu buttons respond
> > > to the mouse. Does that happen for anybody else?
> > > 
> > 
> > There's an old bug when QEMU menu bar is not responsive because it's not
> > properly activated. If you click off qemu and click on the qemu dock
> > icon then it "gets fixed" (cmd-tab works too). Do you hit the issue as
> > described in the article [1]? The code in the article does exactly the
> > same what I'm doing manually. I wanted to fix it but somehow it got
> > postponed for like a whole year :) I might try to make a fix this but
> > note, the issue is not related to the patch.
> 
> This does not sound like the best solution to the problem. There's some info
> on this here (and blog post linked from it):
> 
> https://stackoverflow.com/questions/7460092/nswindow-makekeyandorderfront-makes-window-appear-but-not-key-or-front
> 
> Maybe we call makeKeyAndOrderFront: too early before the app is active and
> that's causing the problem? Would it work better if that's moved after
> [NSApp run]? (Maybe we also need canBecomeKey: somewhere but I don't see why
> would that be needed for normal windows.)
> 

Hi Zoltan,

Thanks for the suggestions. I have tried to move it around but that
doesn't help. Note that minimal cococa app calls makeKeyAndOrderFront:
before [NSApp run] and doesn't experience the issue:
https://github.com/rgl/minimal-cocoa-app/blob/master/main.m

The minimal program that experiences the issue of frozen menubar is:
/* cc -framework Cocoa menufreeze.m */
#import 

int main(void) {
[NSApplication sharedApplication];
[NSApp setActivationPolicy:NSApplicationActivationPolicyRegular];

dispatch_async(dispatch_get_main_queue(), ^{
[NSApp activateIgnoringOtherApps:YES];
});

[NSApp run];

return 0;
}

However if the program belongs to an app bundle it doesn't have the
issue. (Simply move a.out into
minimal-cocoa-app.app/Contents/MacOS/minimal-cocoa-app and use "open
minimal-cocoa-app.app" in shell)

Now if we apply the workaround mentioned in the article [1] that
switches focus to Dock and then back to the app we can resolve the issue
in QEMU:

diff --git a/ui/cocoa.m b/ui/cocoa.m
index f32adc3074..0986891ca0 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1114,6 +1114,15 @@ QemuCocoaView *cocoaView;
 allow_events = true;
 /* Tell cocoa_display_init to proceed */
 qemu_sem_post(_started_sem);
+
+/* Workaround unresponsive menu bar in macOS prior to Big Sur */
+NSArray *docks = [NSRunningApplication 
runningApplicationsWithBundleIdentifier: @"com.apple.dock"];
+if ([docks.firstObject activateWithOptions: 
NSApplicationActivateAllWindows]) {
+dispatch_after(dispatch_time(DISPATCH_TIME_NOW, 200 * NSEC_PER_MSEC),
+   dispatch_get_main_queue(), ^{
+[NSApp activateIgnoringOtherApps:YES];
+});
+}
 }

 - (void)applicationWillTerminate:(NSNotification *)aNotification

Peter, does it help you? And what version of macOS do you use?

BTW, similar workaround was applied to javafx:
https://github.com/openjdk/jfx/pull/361

Regards,
Roman

> > 
> > 1. 
> > https://ar.al/2018/09/17/workaround-for-unclickable-app-menu-bug-with-window.makekeyandorderfront-and-nsapp.activate-on-macos/
> > 

Re: [PATCH] ui/cocoa: Update path to docs in build tree

[Roman Bolshakov]

On Sat, Jan 09, 2021 at 12:38:15AM +0300, Roman Bolshakov wrote:
> QEMU documentation can't be opened if QEMU is run from build tree
> because executables are placed in the top of build tree after conversion
> to meson.
> 
> Signed-off-by: Roman Bolshakov 
> ---
>  ui/cocoa.m | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ui/cocoa.m b/ui/cocoa.m
> index ea3b845b53..13fba8103e 100644
> --- a/ui/cocoa.m
> +++ b/ui/cocoa.m
> @@ -1176,7 +1176,7 @@ QemuCocoaView *cocoaView;
>  - (void) openDocumentation: (NSString *) filename
>  {
>  /* Where to look for local files */
> -NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
> @"../docs/"};
> +NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
> @"docs/"};
>  NSString *full_file_path;
>  NSURL *full_file_url;
>  
> -- 
> 2.29.2
> 

Forgot to add:

Reported-by: Peter Maydell 

-Roman

Re: [PATCH] hvf: guard xgetbv call.

[Roman Bolshakov]

On Fri, Dec 18, 2020 at 06:13:47PM -0800, Hill Ma wrote:
> This prevents illegal instruction on cpus do not support xgetbv.
> 
> Buglink: https://bugs.launchpad.net/qemu/+bug/1758819
> Signed-off-by: Hill Ma 
> ---
>  target/i386/hvf/x86_cpuid.c | 11 ---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 

Hi Hill,

I'm sorry for delay with the review.

> diff --git a/target/i386/hvf/x86_cpuid.c b/target/i386/hvf/x86_cpuid.c
> index a6842912f5..b4b7111fc3 100644
> --- a/target/i386/hvf/x86_cpuid.c
> +++ b/target/i386/hvf/x86_cpuid.c
> @@ -100,11 +100,16 @@ uint32_t hvf_get_supported_cpuid(uint32_t func, 
> uint32_t idx,
>  break;
>  case 0xD:
>  if (idx == 0) {
> -uint64_t host_xcr0 = xgetbv(0);
> -uint64_t supp_xcr0 = host_xcr0 & (XSTATE_FP_MASK | 
> XSTATE_SSE_MASK |
> +uint64_t supp_xcr0 = XSTATE_FP_MASK | XSTATE_SSE_MASK |
>XSTATE_YMM_MASK | XSTATE_BNDREGS_MASK |
>XSTATE_BNDCSR_MASK | XSTATE_OPMASK_MASK |
> -  XSTATE_ZMM_Hi256_MASK | 
> XSTATE_Hi16_ZMM_MASK);
> +  XSTATE_ZMM_Hi256_MASK | 
> XSTATE_Hi16_ZMM_MASK;


> +if ((ecx & CPUID_EXT_AVX) &&
> +(ecx & CPUID_EXT_XSAVE) &&
> +(ecx & CPUID_EXT_OSXSAVE)) {

It's sufficient to check only CPUID_EXT_OSXSAVE to ensure xgetbv
presence (per SDM Vol. 1 13-5):

  Software operating with CPL > 0 may need to determine whether the
  XSAVE feature set and certain XSAVE-enabled features have been
  enabled. If CPL > 0, execution of the MOV from CR4 instruction causes
  a general-protection fault (#GP). The following alternative mechanisms
  allow software to discover the enabling of the XSAVE feature set
  regardless of CPL:

  * The value of CR4.OSXSAVE is returned in CPUID.1:ECX.OSXSAVE[bit 27].
If software determines that CPUID.1:ECX.OSXSAVE = 1, the processor
supports the XSAVE feature set and the feature set has been enabled in
CR4.

  * Executing the XGETBV instruction with ECX = 0 returns the value of
XCR0 in EDX:EAX. XGETBV can be executed if CR4.OSXSAVE = 1 (if
CPUID.1:ECX.OSXSAVE = 1), regardless of CPL.

> +uint64_t host_xcr0 = xgetbv(0);
> +supp_xcr0 &= host_xcr0;
> +}
>  eax &= supp_xcr0;

I think instead of the patch you can do:
-  if (idx == 0) {
+  if (idx == 0 && (ecx & CPUID_EXT_OSXSAVE)) {

That'd keep host values returned from CPUID on platforms that don't
support XSAVE.

Thanks,
Roman

>  } else if (idx == 1) {
>  hv_vmx_read_capability(HV_VMX_CAP_PROCBASED2, );
> -- 
> 2.20.1 (Apple Git-117)
> 

Re: [PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Fri, Jan 08, 2021 at 03:05:55PM +, Peter Maydell wrote:
> On Fri, 8 Jan 2021 at 15:00, Peter Maydell  wrote:
> >
> > On Fri, 8 Jan 2021 at 13:50, Peter Maydell  wrote:
> > >
> > > On Sat, 2 Jan 2021 at 15:14, Roman Bolshakov  
> > > wrote:
> > > >
> > > > ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first 
> > > > deprecated in macOS 11.0 - Use -[NSWorkspace openURL:] instead.
> > > >   [-Wdeprecated-declarations]
> > > > if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == 
> > > > YES) {
> > > >^
> > > > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
> > > >  note:
> > > >   'openFile:' has been explicitly marked deprecated here
> > > > - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace 
> > > > openURL:] instead.", macos(10.0, 11.0));
> > > > ^
> > > >
> > > > Signed-off-by: Roman Bolshakov 
> > > > ---
> > >
> > > Reviewed-by: Peter Maydell 
> >
> >
> > So I was just trying to test this patch, and I found that at least
> > for me the osx menu bar has stopped working in QEMU -- keyboard
> > shortcuts to it still work but none of the menu buttons respond
> > to the mouse. Does that happen for anybody else?
> 
> This menu bar breakage appears to be caused by this patch. I have
> no idea why, because the patch looks pretty harmless. Nonetheless,
> I'm going to have to drop it from my queue.
> 

I think the patch is valid per-se and doubt the patch would cause menu
bar breakage. I had unresponsive menu bar on Catalina even without the
patch.

And I've checked the pre-exesting menu bar issue is resolved in Big Sur
(I assume it was a bug in macOS). As a workaround you might use cmd-tab
or switch focus to another window using mouse and then return it back.

Thanks,
Roman

[PATCH] ui/cocoa: Update path to docs in build tree

[Roman Bolshakov]

QEMU documentation can't be opened if QEMU is run from build tree
because executables are placed in the top of build tree after conversion
to meson.

Signed-off-by: Roman Bolshakov 
---
 ui/cocoa.m | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index ea3b845b53..13fba8103e 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1176,7 +1176,7 @@ QemuCocoaView *cocoaView;
 - (void) openDocumentation: (NSString *) filename
 {
 /* Where to look for local files */
-NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"../docs/"};
+NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"docs/"};
 NSString *full_file_path;
 NSURL *full_file_url;
 
-- 
2.29.2

Re: [PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Fri, Jan 08, 2021 at 03:00:07PM +, Peter Maydell wrote:
> On Fri, 8 Jan 2021 at 13:50, Peter Maydell  wrote:
> >
> > On Sat, 2 Jan 2021 at 15:14, Roman Bolshakov  wrote:
> > >
> > > ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first deprecated 
> > > in macOS 11.0 - Use -[NSWorkspace openURL:] instead.
> > >   [-Wdeprecated-declarations]
> > > if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == 
> > > YES) {
> > >^
> > > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
> > >  note:
> > >   'openFile:' has been explicitly marked deprecated here
> > > - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace 
> > > openURL:] instead.", macos(10.0, 11.0));
> > > ^
> > >
> > > Signed-off-by: Roman Bolshakov 
> > > ---
> >
> > Reviewed-by: Peter Maydell 
> 
> 
> So I was just trying to test this patch, and I found that at least
> for me the osx menu bar has stopped working in QEMU -- keyboard
> shortcuts to it still work but none of the menu buttons respond
> to the mouse. Does that happen for anybody else?
> 

There's an old bug when QEMU menu bar is not responsive because it's not
properly activated. If you click off qemu and click on the qemu dock
icon then it "gets fixed" (cmd-tab works too). Do you hit the issue as
described in the article [1]? The code in the article does exactly the
same what I'm doing manually. I wanted to fix it but somehow it got
postponed for like a whole year :) I might try to make a fix this but
note, the issue is not related to the patch.


> Also, the "bring up the docs" help option (which is what this
> patch is changing) doesn't seem to work when QEMU is run from
> the source tree and the docs haven't been installed to the
> locations where it expects it might find them. Probably the
> code needs updating to work with qemu_find_file() or some
> variant on it.
> 

If I add:
diff --git a/ui/cocoa.m b/ui/cocoa.m
index ea3b845b53..4772b7f981 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1189,6 +1189,7 @@ - (void) openDocumentation: (NSString *) filename
   path_array[index], filename];
 full_file_url = [NSURL fileURLWithPath: full_file_path
isDirectory: false];
+NSLog(@"%@", full_file_url);
 if ([[NSWorkspace sharedWorkspace] openURL: full_file_url] == YES) {
 return;
 }

And click "Help"->"QEMU Documentation". I get the following logs:
2021-01-08 23:14:15.288 qemu-system-x86_64[46165:12969383] 
file:///Users/roolebo/dev/qemu/apple-silicon/build/../share/doc/qemu/index.html
2021-01-08 23:14:15.288 qemu-system-x86_64[46165:12969383] 
file:///Users/roolebo/dev/qemu/apple-silicon/build/../doc/qemu/index.html
2021-01-08 23:14:15.288 qemu-system-x86_64[46165:12969383] 
file:///Users/roolebo/dev/qemu/apple-silicon/build/../docs/index.html

In order to get documentation on macOS. sphinx-doc has to be installed
from homebrew. The package is keg-only so sphinx-build has to be added
to PATH.

Then you can build with --enable-docs. Generated documentation resides
in the build tree after the QEMU has been switched to meson:

find . -name index.html
./build/meson-private/temp/sphinx/out/index.html
./build/docs/devel/index.html
./build/docs/tools/index.html
./build/docs/index.html
./build/docs/specs/index.html
./build/docs/interop/index.html
./build/docs/user/index.html
./build/docs/system/index.html

The problem is that the paths above don't point to docs in build tree.
The patch only fixes a warning and doesn't break existing path
resolution. The fix for out-of-tree docs is trivial:
diff --git a/ui/cocoa.m b/ui/cocoa.m
index ea3b845b53..13fba8103e 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1176,7 +1176,7 @@ - (void)toggleFullScreen:(id)sender
 - (void) openDocumentation: (NSString *) filename
 {
 /* Where to look for local files */
-NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"../docs/"};
+NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"docs/"};
 NSString *full_file_path;
 NSURL *full_file_url;

I'll add it as a separate patch to v2.

1. 
https://ar.al/2018/09/17/workaround-for-unclickable-app-menu-bug-with-window.makekeyandorderfront-and-nsapp.activate-on-macos/

Regards,
Roman

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Thu, Jan 07, 2021 at 08:41:50PM +0100, Paolo Bonzini wrote:
> Il gio 7 gen 2021, 20:36 Roman Bolshakov  ha scritto:
> 
> > > No I think that Meson should simply explode link_whole libraries to their
> > > constituent objects.  This way duplicates are avoided.
> > >
> >
> > Ok. I've looked through related changes in meson and it flattens object
> > files implicitly for link_with/link_whole parameters of static_library:
> >
> >   https://github.com/mesonbuild/meson/pull/6030/files
> >
> > But qemu adds dependencies to source set and populates dependencies
> > parameter of static_library and declare_dependency and we get duplicate
> > symbols:
> >
> >   https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00411.html
> >
> > Perhaps it's a bug then.
> >
> 
> No, the same deduplication is not done for executables, because executables
> use libraries directly and not their object files.
> 

Paolo,

I tried to use extract_all_objects() to get all object files directly
but it doesn't work on dependency objects defined via
declare_dependency(). It works only on regular targets (libs and
executables). And as far as I understand the intention to have
declare_dependency() in QEMU was to specify public interface to avoid
some duplication. But meson doesn't have public/private notion for build
targets so if we drop declare_dependency we need to specify link_whole
in every user of a library that's had link_whole: declare_dependency()
and build files would become less lean. So I'm not sure how to proceed.

The proposed patch (in the subject) is the still the best we've got so
far that fixes macOS build immediately without much bigger wrestling
with meson.

-Roman

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Thu, Jan 07, 2021 at 07:22:06PM +0100, Paolo Bonzini wrote:
> On 07/01/21 19:18, Roman Bolshakov wrote:
> > 
> > > The real issue is that Meson's implementation of link_whole for
> > > library-in-library makes sense for one use case (convenience library that 
> > > is
> > > linked into another convenience library) but not for another (grouping 
> > > code
> > > for subsystems).  I cannot blame them for this because link_with is a more
> > > common case for the latter; OTOH QEMU is using link_whole a lot in order 
> > > to
> > > support the *_init() construct.
> > > 
> > > I really think the correct fix is for Meson to use objects instead of
> > > archives for link_whole, similar to how QEMU Makefiles used to do it. This
> > > would also remove the need for the special .fa suffix, so it would be an
> > > improvement all around.
> > > 
> > Does it mean that we need a kind of object target in meson? Do you think
> > if this interface would work?
> > 
> > crypto_objs = object_library(..., dependencies: public_deps + 
> > [aninternaldep])
> > crypto = declare_dependency(link_with: crypto_objs, dependencies: 
> > public_deps)
> 
> No I think that Meson should simply explode link_whole libraries to their
> constituent objects.  This way duplicates are avoided.
> 

Ok. I've looked through related changes in meson and it flattens object
files implicitly for link_with/link_whole parameters of static_library:

  https://github.com/mesonbuild/meson/pull/6030/files

But qemu adds dependencies to source set and populates dependencies
parameter of static_library and declare_dependency and we get duplicate
symbols:

  https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00411.html

Perhaps it's a bug then.

Regards,
Roman

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Thu, Jan 07, 2021 at 05:23:54PM +0100, Paolo Bonzini wrote:
> On 07/01/21 16:56, Roman Bolshakov wrote:
> > IMO duplication of dependencies shouldn't be needed for a build system.
> > Meta build system should allow private and public dependencies. Different
> > rules are applied to them. Private dependency is not propagated beyond a
> > target that uses it, public dependency is propagated.
> > 
> > Right now it seems that meson is missing the notion of public and
> > private dependencies and that's where the problem arises. The post [1] (and
> > the related issue) summarizes what I'm trying to say.
> 
> Meson doesn't have a concept of public dependencies because it separates the
> private (static library) and the public (declare_dependency) view. That is
> you'd have:
> 
> public_deps = [gnutls, anotherpublicdep]
> lib = static_library(..., dependencies: public_deps + [aninternaldep])
> dep = declare_dependency(link_with: lib, dependencies: public_deps)
> 

Thanks! This wasn't obvious to me. But what's not clear that CMake can
do both collection of objects (what I provided in the example) and
static libraries and they're different. I assume what you have shown
would look in CMake like (please note that STATIC is used instead of
OBJECT):

add_library(crypto STATIC crypto-file1.c ...)
target_link_libraries(crypto PRIVATE aninternaldep
  PUBLIC  gnutls
  anotherpublicdep)


That explains why attempt to use dependencies between link_whole static
libraries in meson causes symbol duplication. CMake on other hand can
just make collection of objects or even a chain of collection of
objects. They'll be linked in fully only in a final static library,
shared library or an executable.

> The real issue is that Meson's implementation of link_whole for
> library-in-library makes sense for one use case (convenience library that is
> linked into another convenience library) but not for another (grouping code
> for subsystems).  I cannot blame them for this because link_with is a more
> common case for the latter; OTOH QEMU is using link_whole a lot in order to
> support the *_init() construct.
> 
> I really think the correct fix is for Meson to use objects instead of
> archives for link_whole, similar to how QEMU Makefiles used to do it. This
> would also remove the need for the special .fa suffix, so it would be an
> improvement all around.
> 

Does it mean that we need a kind of object target in meson? Do you think
if this interface would work?

crypto_objs = object_library(..., dependencies: public_deps + [aninternaldep])
crypto = declare_dependency(link_with: crypto_objs, dependencies: public_deps)

Regards,
Roman

> Paolo
> 
> > If we resolve the issue, then we just specify gnutls as a public
> > dependency of crypto and all users of crypto would get gnutls headers.
> > 
> > Here's an example how clearly CMake approaches the issue [2][3]:
> > 
> > add_library(crypto OBJECT crypto-file1.c ...)
> > target_link_libraries(crypto PRIVATE aninternaldep
> >   PUBLIC  gnutls
> >   anotherpublicdep)
> > 
> > 1.https://github.com/mesonbuild/meson/issues/495#issuecomment-206178570
> > 2.https://cmake.org/cmake/help/latest/command/target_link_libraries.html#linking-object-libraries
> > 3.https://cmake.org/cmake/help/latest/command/target_link_libraries.html#libraries-for-a-target-and-or-its-dependents
> 

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Thu, Jan 07, 2021 at 12:41:40PM +0100, Paolo Bonzini wrote:
> On 05/01/21 15:37, Roman Bolshakov wrote:
> > Does it work if you do:
> > 
> > crypto_ss.add(authz, qom)
> > libcrypto = static_library('crypto', crypto_ss.sources() + genh,
> > dependencies: crypto_ss.dependencies(),
> > ...)
> > crypto = declare_dependency(link_whole: libcrypto,
> >  dependencies: crypto_ss.dependencies())
> 
> Ok, so the final attempt is a mix of the three :)  Keep the link_whole
> dependencies in the declare_dependency, and add the sourceset dependencies
> there too.

Hi Paolo,

Thanks for the patch but unfortunately it doesn't resolve the issue.
io and other libraries can't still find gnutls.

I've also tried your meson trans-deps branch and wonder if it's supposed
to fix the issue without any changes to qemu build files?
Do you need any help with meson changes?

IMO duplication of dependencies shouldn't be needed for a build system.
Meta build system should allow private and public dependencies. Different
rules are applied to them. Private dependency is not propagated beyond a
target that uses it, public dependency is propagated. There's also
declare_dependency that has to be always public because it serves no
purpose on it's own. declare_dependency is like INTERFACE library in
CMake.

If a project specifies a dependency that is public, it should be
transitively passed downstream. Build system shouldn't obscurely hide
flags a dependency provides on case-by-case basis.

Right now it seems that meson is missing the notion of public and
private dependencies and that's where the problem arises. The post [1] (and
the related issue) summarizes what I'm trying to say.

If we resolve the issue, then we just specify gnutls as a public
dependency of crypto and all users of crypto would get gnutls headers.

Here's an example how clearly CMake approaches the issue [2][3]:

add_library(crypto OBJECT crypto-file1.c ...)
target_link_libraries(crypto PRIVATE aninternaldep
 PUBLIC  gnutls
 anotherpublicdep)

1. https://github.com/mesonbuild/meson/issues/495#issuecomment-206178570
2. 
https://cmake.org/cmake/help/latest/command/target_link_libraries.html#linking-object-libraries
3. 
https://cmake.org/cmake/help/latest/command/target_link_libraries.html#libraries-for-a-target-and-or-its-dependents

Regards,
Roman

> 
> diff --git a/meson.build b/meson.build
> index e9bf290966..774df4db8e 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1904,7 +1904,8 @@ libqom = static_library('qom', qom_ss.sources() +
> genh,
>  dependencies: [qom_ss.dependencies()],
>  name_suffix: 'fa')
> 
> -qom = declare_dependency(link_whole: libqom)
> +qom = declare_dependency(link_whole: libqom,
> + dependencies: [qom_ss.dependencies()])
> 
>  authz_ss = authz_ss.apply(config_host, strict: false)
>  libauthz = static_library('authz', authz_ss.sources() + genh,
> @@ -1913,7 +1914,7 @@ libauthz = static_library('authz', authz_ss.sources()
> + genh,
>build_by_default: false)
> 
>  authz = declare_dependency(link_whole: libauthz,
> -   dependencies: qom)
> +  dependencies: [authz_ss.dependencies(), qom])
> 
>  crypto_ss = crypto_ss.apply(config_host, strict: false)
>  libcrypto = static_library('crypto', crypto_ss.sources() + genh,
> @@ -1922,7 +1923,7 @@ libcrypto = static_library('crypto',
> crypto_ss.sources() + genh,
> build_by_default: false)
> 
>  crypto = declare_dependency(link_whole: libcrypto,
> -dependencies: [authz, qom])
> +dependencies: [crypto_ss.dependencies(), authz,
> qom])
> 
>  io_ss = io_ss.apply(config_host, strict: false)
>  libio = static_library('io', io_ss.sources() + genh,
> @@ -1931,13 +1932,14 @@ libio = static_library('io', io_ss.sources() + genh,
> name_suffix: 'fa',
> build_by_default: false)
> 
> -io = declare_dependency(link_whole: libio, dependencies: [crypto, qom])
> +io = declare_dependency(link_whole: libio,
> +dependencies: [io_ss.dependencies(), crypto, qom])
> 
>  libmigration = static_library('migration', sources: migration_files + genh,
>name_suffix: 'fa',
>build_by_default: false)
>  migration = declare_dependency(link_with: libmigration,
> -   dependencies: [zlib, qom, io])
> +   dependencies: [qom, io])
>  softmmu_s

Re: [PATCH v5 00/11] hvf: Implement Apple Silicon Support

[Roman Bolshakov]

On Fri, Dec 11, 2020 at 04:12:49PM +0100, Alexander Graf wrote:
> Now that Apple Silicon is widely available, people are obviously excited
> to try and run virtualized workloads on them, such as Linux and Windows.
> 
> This patch set implements a fully functional version to get the ball
> going on that. With this applied, I can successfully run both Linux and
> Windows as guests. I am not aware of any limitations specific to
> Hypervisor.framework apart from:
> 
>   - Live migration / savevm
>   - gdbstub debugging (SP register)
> 
> 
> Enjoy!
> 

Hi Alex,

For the ARM part:
Tested-by: Roman Bolshakov 

Note, that it doesn't apply to current master. So I applied it over
tagged v5.2.0.

Are there any outstanding issues that blocking merge apart from the
rebase?

Thanks,
Roman

> Alex
> 
> v1 -> v2:
> 
>   - New patch: hvf: Actually set SIG_IPI mask
>   - New patch: hvf: Introduce hvf vcpu struct
>   - New patch: hvf: arm: Mark CPU as dirty on reset
>   - Removed patch: hw/arm/virt: Disable highmem when on hypervisor.framework
>   - Removed patch: arm: Synchronize CPU on PSCI on
>   - Fix build on 32bit arm
>   - Merge vcpu kick function patch into ARM enablement
>   - Implement WFI handling (allows vCPUs to sleep)
>   - Synchronize system registers (fixes OVMF crashes and reboot)
>   - Don't always call cpu_synchronize_state()
>   - Use more fine grained iothread locking
>   - Populate aa64mmfr0 from hardware
>   - Make safe to ctrl-C entitlement application
> 
> v2 -> v3:
> 
>   - Removed patch: hvf: Actually set SIG_IPI mask
>   - New patch: hvf: arm: Add support for GICv3
>   - New patch: hvf: arm: Implement -cpu host
>   - Advance PC on SMC
>   - Use cp list interface for sysreg syncs
>   - Do not set current_cpu
>   - Fix sysreg isread mask
>   - Move sysreg handling to functions
>   - Remove WFI logic again
>   - Revert to global iothread locking
> 
> v3 -> v4:
> 
>   - Removed patch: hvf: arm: Mark CPU as dirty on reset
>   - New patch: hvf: Simplify post reset/init/loadvm hooks
>   - Remove i386-softmmu target (meson.build for hvf target)
>   - Combine both if statements (PSCI)
>   - Use hv.h instead of Hypervisor.h for 10.15 compat
>   - Remove manual inclusion of Hypervisor.h in common .c files
>   - No longer include Hypervisor.h in arm hvf .c files
>   - Remove unused exe_full variable
>   - Reuse exe_name variable
> 
> v4 -> v5:
> 
>   - Use g_free() on destroy
> 
> Alexander Graf (10):
>   hvf: Add hypervisor entitlement to output binaries
>   hvf: x86: Remove unused definitions
>   hvf: Move common code out
>   hvf: Introduce hvf vcpu struct
>   arm: Set PSCI to 0.2 for HVF
>   hvf: Simplify post reset/init/loadvm hooks
>   hvf: Add Apple Silicon support
>   arm: Add Hypervisor.framework build target
>   hvf: arm: Add support for GICv3
>   hvf: arm: Implement -cpu host
> 
> Peter Collingbourne (1):
>   arm/hvf: Add a WFI handler
> 
>  MAINTAINERS  |  14 +-
>  accel/hvf/entitlements.plist |   8 +
>  accel/hvf/hvf-all.c  |  54 +++
>  accel/hvf/hvf-cpus.c | 466 +++
>  accel/hvf/meson.build|   7 +
>  accel/meson.build|   1 +
>  include/hw/core/cpu.h|   3 +-
>  include/sysemu/hvf.h |   2 +
>  include/sysemu/hvf_int.h |  66 +++
>  meson.build  |  40 +-
>  scripts/entitlement.sh   |  13 +
>  target/arm/cpu.c |  13 +-
>  target/arm/cpu.h |   2 +
>  target/arm/hvf/hvf.c | 856 +++
>  target/arm/hvf/meson.build   |   3 +
>  target/arm/kvm_arm.h |   2 -
>  target/arm/meson.build   |   2 +
>  target/i386/hvf/hvf-cpus.c   | 131 --
>  target/i386/hvf/hvf-cpus.h   |  25 -
>  target/i386/hvf/hvf-i386.h   |  49 +-
>  target/i386/hvf/hvf.c| 462 +++
>  target/i386/hvf/meson.build  |   1 -
>  target/i386/hvf/vmx.h|  24 +-
>  target/i386/hvf/x86.c|  28 +-
>  target/i386/hvf/x86_descr.c  |  26 +-
>  target/i386/hvf/x86_emu.c|  62 +--
>  target/i386/hvf/x86_mmu.c|   4 +-
>  target/i386/hvf/x86_task.c   |  12 +-
>  target/i386/hvf/x86hvf.c | 224 -
>  target/i386/hvf/x86hvf.h |   2 -
>  30 files changed, 1786 insertions(+), 816 deletions(-)
>  create mode 100644 accel/hvf/entitlements.plist
>  create mode 100644 accel/hvf/hvf-all.c
>  create mode 100644 accel/hvf/hvf-cpus.c
>  create mode 100644 accel/hvf/meson.build
>  create mode 100644 include/sysemu/hvf_int.h
>  create mode 100755 scripts/entitlement.sh
>  create mode 100644 target/arm/hvf/hvf.c
>  create mode 100644 target/arm/hvf/meson.build
>  delete mode 100644 target/i386/hvf/hvf-cpus.c
>  delete mode 100644 target/i386/hvf/hvf-cpus.h
> 
> -- 
> 2.24.3 (Apple Git-128)
> 

Re: [PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Mon, Jan 04, 2021 at 06:02:50PM -0800, Joelle van Dyne wrote:
> Tested-by: Joelle van Dyne 
> 
> It works for me. But one thing is that if you build it with the macOS
> 11.x SDK it won't run on < 11.x. This is why apple recommends
> something like:
> 
> if (__builtin_available(macOS 11, *)) {
> pthread_jit_write_protect_np();
> }
> 
> You still need a compile time check like MAC_OS_VERSION_11_0 to
> support linking with older SDKs.
> 

I'll address the issue in v3. Thanks for catching it.

Regards,
Roman

> On Sun, Jan 3, 2021 at 6:54 AM Roman Bolshakov  wrote:
> >
> > Pages can't be both write and executable at the same time on Apple
> > Silicon. macOS provides public API to switch write protection [1] for
> > JIT applications, like TCG.
> >
> > 1. 
> > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> >
> > Signed-off-by: Roman Bolshakov 
> > ---
> > v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
> > Changes since v1:
> >
> >  - Pruned not needed fiddling with W^X and dropped symmetry from write
> >lock/unlock and renamed related functions.
> >Similar approach is used in JavaScriptCore [1].
> >
> >  - Moved jit helper functions to util/osdep
> > 
> >   As 
> > outlined in osdep.h, this matches to (2):   
> > 
> > 
> >  * In an ideal 
> > world this header would contain only:   
> >  *  (1) things which 
> > everybody needs 
> >*  (2) things without which 
> > code would work on most platforms but   
> >*  fail to compile or misbehave 
> > on a minority of host OSes
> >
> >  - Fixed a checkpatch error
> >
> >  - Limit new behaviour only to macOS 11.0 and above, because of the
> >following declarations:
> >
> >__API_AVAILABLE(macos(11.0))
> >__API_UNAVAILABLE(ios, tvos, watchos)
> >void pthread_jit_write_protect_np(int enabled);
> >
> >__API_AVAILABLE(macos(11.0))
> >__API_UNAVAILABLE(ios, tvos, watchos)
> >int pthread_jit_write_protect_supported_np(void);
> >
> >  1. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch
> >
> >  accel/tcg/cpu-exec.c  |  2 ++
> >  accel/tcg/translate-all.c |  6 ++
> >  include/qemu/osdep.h  |  3 +++
> >  tcg/tcg.c |  1 +
> >  util/osdep.c  | 22 ++
> >  5 files changed, 34 insertions(+)
> >
> > diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> > index 8689c54499..374060eb45 100644
> > --- a/accel/tcg/cpu-exec.c
> > +++ b/accel/tcg/cpu-exec.c
> > @@ -175,6 +175,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState 
> > *cpu, TranslationBlock *itb)
> >  }
> >  #endif /* DEBUG_DISAS */
> >
> > +qemu_thread_jit_execute();
> >  ret = tcg_qemu_tb_exec(env, tb_ptr);
> >  cpu->can_do_io = 1;
> >  last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> > @@ -382,6 +383,7 @@ static inline void tb_add_jump(TranslationBlock *tb, 
> > int n,
> >  {
> >  uintptr_t old;
> >
> > +qemu_thread_jit_write();
> >  assert(n < ARRAY_SIZE(tb->jmp_list_next));
> >  qemu_spin_lock(_next->jmp_lock);
> >
> > diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> > index b7d50a73d4..88ae5d35ef 100644
> > --- a/accel/tcg/translate-all.c
> > +++ b/accel/tcg/translate-all.c
> > @@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
> >  size_t size = tcg_ctx->code_gen_buffer_size;
> >  void *buf;
> >
> > +#if defined(__APPLE__) && defined(MAC_OS_VERSION_11_0)
> > +flags |= MAP_JIT;
> > +#endif
> >  buf = mmap(NULL, size, prot, flags, -1, 0);
> >  if (buf == MAP_FAILED) {
> >  return NULL;
> > @@ -1485,7 +1488,9 @@ static void do_tb_phys_i

Re: [PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Mon, Jan 04, 2021 at 08:28:08PM +, Alex Bennée wrote:
> 
> Alexander Graf  writes:
> 
> > On 04.01.21 16:23, Alex Bennée wrote:
> >> Roman Bolshakov  writes:
> >>
> >>> Pages can't be both write and executable at the same time on Apple
> >>> Silicon. macOS provides public API to switch write protection [1] for
> >>> JIT applications, like TCG.
> >>>
> >>> 1. 
> >>> https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> >>>
> >>> Signed-off-by: Roman Bolshakov 
> >>> ---
> >>> v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
> >>> Changes since v1:
> >>>
> >>>   - Pruned not needed fiddling with W^X and dropped symmetry from write
> >>> lock/unlock and renamed related functions.
> >>> Similar approach is used in JavaScriptCore [1].
> >>>
> >>>   - Moved jit helper functions to util/osdep
> >>>   
> >>>  
> >>> As outlined in osdep.h, this matches to (2):  
> >>>   
> >>>   
> >>>   * In an 
> >>> ideal world this header would contain only:   
> >>>  *  (1) 
> >>> things which everybody needs  
> >>>   *  (2) 
> >>> things without which code would work on most platforms but
> >>>   *  fail 
> >>> to compile or misbehave on a minority of host OSes
> >>>
> >>>   - Fixed a checkpatch error
> >>>
> >>>   - Limit new behaviour only to macOS 11.0 and above, because of the
> >>> following declarations:
> >>>
> >>> __API_AVAILABLE(macos(11.0))
> >>> __API_UNAVAILABLE(ios, tvos, watchos)
> >>> void pthread_jit_write_protect_np(int enabled);
> >>>
> >>> __API_AVAILABLE(macos(11.0))
> >>> __API_UNAVAILABLE(ios, tvos, watchos)
> >>> int pthread_jit_write_protect_supported_np(void);
> >>>
> >>>   1. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch
> >>>
> >>>   accel/tcg/cpu-exec.c  |  2 ++
> >>>   accel/tcg/translate-all.c |  6 ++
> >>>   include/qemu/osdep.h  |  3 +++
> >>>   tcg/tcg.c |  1 +
> >>>   util/osdep.c  | 22 ++
> >>>   5 files changed, 34 insertions(+)
> >>>
> >>> diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> >>> index 8689c54499..374060eb45 100644
> >>> --- a/accel/tcg/cpu-exec.c
> >>> +++ b/accel/tcg/cpu-exec.c
> >>> @@ -175,6 +175,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState 
> >>> *cpu, TranslationBlock *itb)
> >>>   }
> >>>   #endif /* DEBUG_DISAS */
> >>>   
> >>> +qemu_thread_jit_execute();
> >>>   ret = tcg_qemu_tb_exec(env, tb_ptr);
> >>>   cpu->can_do_io = 1;
> >>>   last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> >>> @@ -382,6 +383,7 @@ static inline void tb_add_jump(TranslationBlock *tb, 
> >>> int n,
> >>>   {
> >>>   uintptr_t old;
> >>>   
> >>> +qemu_thread_jit_write();
> >>>   assert(n < ARRAY_SIZE(tb->jmp_list_next));
> >>>   qemu_spin_lock(_next->jmp_lock);
> >>>   
> >>> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> >>> index b7d50a73d4..88ae5d35ef 100644
> >>> --- a/accel/tcg/translate-all.c
> >>> +++ b/accel/tcg/translate-all.c
> >>> @@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
> >>>   size_t size = tcg_ctx->code_gen_buffer_size;
> >>>   void *buf;
> >>>   
> >>> +#if defined(__APPLE__) &&

Re: [PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Mon, Jan 04, 2021 at 07:39:13PM +0100, Alexander Graf wrote:
> 
> On 04.01.21 16:23, Alex Bennée wrote:
> > Roman Bolshakov  writes:
> > 
> > > Pages can't be both write and executable at the same time on Apple
> > > Silicon. macOS provides public API to switch write protection [1] for
> > > JIT applications, like TCG.
> > > 
> > > 1. 
> > > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> > > 
> > > Signed-off-by: Roman Bolshakov 
> > > ---
> > > v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
> > > Changes since v1:
> > > 
> > >   - Pruned not needed fiddling with W^X and dropped symmetry from write
> > > lock/unlock and renamed related functions.
> > > Similar approach is used in JavaScriptCore [1].
> > > 
> > >   - Moved jit helper functions to util/osdep
> > >   
> > >  
> > > As outlined in osdep.h, this matches to (2):  
> > >   
> > >   
> > >   * In an 
> > > ideal world this header would contain only:   
> > >  *  (1) 
> > > things which everybody needs  
> > >   *  (2) 
> > > things without which code would work on most platforms but
> > >   *  fail 
> > > to compile or misbehave on a minority of host OSes
> > > 
> > >   - Fixed a checkpatch error
> > > 
> > >   - Limit new behaviour only to macOS 11.0 and above, because of the
> > > following declarations:
> > > 
> > > __API_AVAILABLE(macos(11.0))
> > > __API_UNAVAILABLE(ios, tvos, watchos)
> > > void pthread_jit_write_protect_np(int enabled);
> > > 
> > > __API_AVAILABLE(macos(11.0))
> > > __API_UNAVAILABLE(ios, tvos, watchos)
> > > int pthread_jit_write_protect_supported_np(void);
> > > 
> > >   1. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch
> > > 
> > >   accel/tcg/cpu-exec.c  |  2 ++
> > >   accel/tcg/translate-all.c |  6 ++
> > >   include/qemu/osdep.h  |  3 +++
> > >   tcg/tcg.c |  1 +
> > >   util/osdep.c  | 22 ++
> > >   5 files changed, 34 insertions(+)
> > > 
> > > diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> > > index 8689c54499..374060eb45 100644
> > > --- a/accel/tcg/cpu-exec.c
> > > +++ b/accel/tcg/cpu-exec.c
> > > @@ -175,6 +175,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState 
> > > *cpu, TranslationBlock *itb)
> > >   }
> > >   #endif /* DEBUG_DISAS */
> > > +qemu_thread_jit_execute();
> > >   ret = tcg_qemu_tb_exec(env, tb_ptr);
> > >   cpu->can_do_io = 1;
> > >   last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> > > @@ -382,6 +383,7 @@ static inline void tb_add_jump(TranslationBlock *tb, 
> > > int n,
> > >   {
> > >   uintptr_t old;
> > > +qemu_thread_jit_write();
> > >   assert(n < ARRAY_SIZE(tb->jmp_list_next));
> > >   qemu_spin_lock(_next->jmp_lock);
> > > diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> > > index b7d50a73d4..88ae5d35ef 100644
> > > --- a/accel/tcg/translate-all.c
> > > +++ b/accel/tcg/translate-all.c
> > > @@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
> > >   size_t size = tcg_ctx->code_gen_buffer_size;
> > >   void *buf;
> > > +#if defined(__APPLE__) && defined(MAC_OS_VERSION_11_0)
> > > +flags |= MAP_JIT;
> > > +#endif
> > >   buf = mmap(NULL, size, prot, flags, -1, 0);
> > >   if (buf == MAP_FAILED) {
> > >   return NULL;
> > > @@ -1485,7 +1488,9 @@ static void do_tb_phys_invalidate(TranslationBlock 
> > > *tb, bool rm_fro

Re: [PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Mon, Jan 04, 2021 at 03:23:07PM +, Alex Bennée wrote:
> 
> Roman Bolshakov  writes:
> 
> > Pages can't be both write and executable at the same time on Apple
> > Silicon. macOS provides public API to switch write protection [1] for
> > JIT applications, like TCG.
> >
> > 1. 
> > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> >
> > Signed-off-by: Roman Bolshakov 
> > ---
> > v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
> > Changes since v1:
> >
> >  - Pruned not needed fiddling with W^X and dropped symmetry from write
> >lock/unlock and renamed related functions.
> >Similar approach is used in JavaScriptCore [1].
> >
> >  - Moved jit helper functions to util/osdep
> > 
> >   As 
> > outlined in osdep.h, this matches to (2):   
> > 
> > 
> >  * In an ideal 
> > world this header would contain only:   
> >  *  (1) things which 
> > everybody needs 
> >*  (2) things without which 
> > code would work on most platforms but   
> >*  fail to compile or misbehave 
> > on a minority of host OSes
> >
> >  - Fixed a checkpatch error
> >
> >  - Limit new behaviour only to macOS 11.0 and above, because of the
> >following declarations:
> >
> >__API_AVAILABLE(macos(11.0))
> >__API_UNAVAILABLE(ios, tvos, watchos)
> >void pthread_jit_write_protect_np(int enabled);
> >
> >__API_AVAILABLE(macos(11.0))
> >__API_UNAVAILABLE(ios, tvos, watchos)
> >int pthread_jit_write_protect_supported_np(void);
> >
> >  1. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch
> >
> >  accel/tcg/cpu-exec.c  |  2 ++
> >  accel/tcg/translate-all.c |  6 ++
> >  include/qemu/osdep.h  |  3 +++
> >  tcg/tcg.c |  1 +
> >  util/osdep.c  | 22 ++
> >  5 files changed, 34 insertions(+)
> >
> > diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> > index 8689c54499..374060eb45 100644
> > --- a/accel/tcg/cpu-exec.c
> > +++ b/accel/tcg/cpu-exec.c
> > @@ -175,6 +175,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState 
> > *cpu, TranslationBlock *itb)
> >  }
> >  #endif /* DEBUG_DISAS */
> >  
> > +qemu_thread_jit_execute();
> >  ret = tcg_qemu_tb_exec(env, tb_ptr);
> >  cpu->can_do_io = 1;
> >  last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> > @@ -382,6 +383,7 @@ static inline void tb_add_jump(TranslationBlock *tb, 
> > int n,
> >  {
> >  uintptr_t old;
> >  
> > +qemu_thread_jit_write();
> >  assert(n < ARRAY_SIZE(tb->jmp_list_next));
> >  qemu_spin_lock(_next->jmp_lock);
> >  
> > diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> > index b7d50a73d4..88ae5d35ef 100644
> > --- a/accel/tcg/translate-all.c
> > +++ b/accel/tcg/translate-all.c
> > @@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
> >  size_t size = tcg_ctx->code_gen_buffer_size;
> >  void *buf;
> >  
> > +#if defined(__APPLE__) && defined(MAC_OS_VERSION_11_0)
> > +flags |= MAP_JIT;
> > +#endif
> >  buf = mmap(NULL, size, prot, flags, -1, 0);
> >  if (buf == MAP_FAILED) {
> >  return NULL;
> > @@ -1485,7 +1488,9 @@ static void do_tb_phys_invalidate(TranslationBlock 
> > *tb, bool rm_from_page_list)
> >  
> >  static void tb_phys_invalidate__locked(TranslationBlock *tb)
> >  {
> > +qemu_thread_jit_write();
> >  do_tb_phys_invalidate(tb, true);
> > +qemu_thread_jit_execute();
> >  }
> >  
> >  /* invalidate one TB
> > @@ -1687,6 +1692,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
> >  #endif
> >  
> >  assert_memory_lock();
> > +qemu_thread_jit_write();
&

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Mon, Jan 04, 2021 at 09:50:32PM +0100, Paolo Bonzini wrote:
> On 04/01/21 18:24, Roman Bolshakov wrote:
> > Hi Paolo,
> > 
> > I'm sorry I didn't reply earlier. As I showed in an example to Peter
> > (https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00085.html):
> > https://github.com/mesonbuild/meson/commit/ff5dc65ef841857dd306694dff1fb1cd2bf801e4
> > 
> > The approach doesn't propogate dependencies of crypto beyond libcrypto.
> > i.e. if you specify crypto somewhere else as depedency, it won't pull
> > CFLAGS needed for gnutls.
> 
> Hi Roman,
> 
> After writing the meson patch in fact I noticed that get_dependencies() is
> used only for linker flags.  I got a very quick reply from the Meson
> maintainer (https://github.com/mesonbuild/meson/pull/8151):
> 

Thanks for providing a PR! I'll try if it works for QEMU with previous
proposal of fixing it (where we specify dependency in source set only
once and don't duplicate in declare_dependency).

I wonder if we should add a source set method like public_add to allow
the behavior we want and permit propogation of a dependency beyond
static_library without breaking all other users of meson out there?

>The fact that header flags are not passed transitively but libraries
>are (in some cases) is intentional. Otherwise compiler flag counts
>explode in deep hierarchies. Because of this include paths must be
>exported manually, typically by adding the appropriate bits to a
>declare_dependency.
> 
>Libs are a bit stupid, because you need to add direct dependencies
>if, for example, you link to a static library.
> 
> Does it work if you do:
> 
> crypto_ss.add(authz, qom)
> libcrypto = static_library('crypto', crypto_ss.sources() + genh,
>dependencies: crypto_ss.dependencies(),
>...)
> crypto = declare_dependency(link_whole: libcrypto,
> dependencies: crypto_ss.dependencies())
> 

I tried that approach before I sent the patch in the subject. It
produces duplicate symbols:

  duplicate symbol '_qauthz_pam_new' in:
  libcrypto.fa(authz_pamacct.c.o)
  libauthz.fa(authz_pamacct.c.o)
  [...]
  duplicate symbol '_object_property_set_qobject' in:
  libcrypto.fa(qom_qom-qobject.c.o) 


   libqom.fa(qom_qom-qobject.c.o)

My impression that it links in every static library that's mentioned in
dependencies of static_library, so they grow like a snow ball. Patch
below:

diff --git a/block/meson.build b/block/meson.build
index 7595d86c41..7eaf48c6dc 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -40,7 +40,7 @@ block_ss.add(files(
   'vmdk.c',
   'vpc.c',
   'write-threshold.c',
-), zstd, zlib)
+), crypto, zstd, zlib)
 
 softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
 
diff --git a/hw/nvram/meson.build b/hw/nvram/meson.build
index fd2951a860..1f2ed013b2 100644
--- a/hw/nvram/meson.build
+++ b/hw/nvram/meson.build
@@ -1,6 +1,3 @@
-# QOM interfaces must be available anytime QOM is used.
-qom_ss.add(files('fw_cfg-interface.c'))
-
 softmmu_ss.add(files('fw_cfg.c'))
 softmmu_ss.add(when: 'CONFIG_CHRP_NVRAM', if_true: files('chrp_nvram.c'))
 softmmu_ss.add(when: 'CONFIG_DS1225Y', if_true: files('ds1225y.c'))
diff --git a/io/meson.build b/io/meson.build
index bcd8b1e737..a844271b17 100644
--- a/io/meson.build
+++ b/io/meson.build
@@ -12,4 +12,4 @@ io_ss.add(files(
   'dns-resolver.c',
   'net-listener.c',
   'task.c',
-))
+), crypto)
diff --git a/meson.build b/meson.build
index 372576f82c..1a8c653067 100644
--- a/meson.build
+++ b/meson.build
@@ -1538,6 +1538,34 @@ libqemuutil = static_library('qemuutil',
 qemuutil = declare_dependency(link_with: libqemuutil,
   sources: genh + version_res)
 
+# QOM interfaces must be available anytime QOM is used.
+qom_ss.add(files('hw/nvram/fw_cfg-interface.c'))
+qom_ss = qom_ss.apply(config_host, strict: false)
+libqom = static_library('qom', qom_ss.sources() + genh,
+dependencies: [qom_ss.dependencies()],
+name_suffix: 'fa')
+
+qom = declare_dependency(link_whole: libqom)
+
+authz_ss = authz_ss.apply(config_host, strict: false)
+libauthz = static_library('authz', authz_ss.sources() + genh,
+  dependencies: [authz_ss.dependencies()],
+  name_suffix: 'fa',
+  build_by_default: false)
+
+authz = declare_dependency(link_whole: libauthz,
+   dependencies: qom)
+
+crypto_ss.add(authz)
+crypto_ss = crypto_ss.apply(config_host, strict: false)
+libcrypto = static_library('crypto', cry

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Sat, Jan 02, 2021 at 08:43:51PM +0100, Paolo Bonzini wrote:
> On 02/01/21 14:25, Peter Maydell wrote:
> > Question to Paolo -- it seems pretty fragile to have to explicitly
> > list "these source files need these extra CFLAGS" in half a dozen
> > meson.build files, because it's pretty non-obvious that adding
> > eg '#include "block/nbd.h"' to a .c file means that you also
> > need to update the meson.build file to say "and now it needs these
> > extra CFLAGS". Isn't there some way we can just have the CFLAGS
> > added more globally so that if we use gnutls.h directly or
> > indirectly from more .c files in future it Just Works ?
> > 
> > If the build failed for the common Linux case then it would be
> > at least more obvious that you needed to update the meson.build
> > files. I think it's better to avoid "you need to do this special
> > thing that you'll only notice you're missing if you happen to test
> > on a somewhat obscure host configuration" where we can.
> > 
> > (We don't want to link helper binaries etc against gnutls if
> > they don't need it, but that's LDFLAGS, not CFLAGS.)
> 
> The gnutls dependency will already propagate from
> 
> if 'CONFIG_GNUTLS' in config_host
>   crypto_ss.add(gnutls)
> endif
> 
> to
> 
> libcrypto = static_library('crypto', crypto_ss.sources() + genh,
>   dependencies: [crypto_ss.dependencies()], ...)
> crypto = declare_dependency(link_whole: libcrypto,
> dependencies: [authz, qom])
> 

Hi Paolo,

I'm sorry I didn't reply earlier. As I showed in an example to Peter
(https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00085.html):
https://github.com/mesonbuild/meson/commit/ff5dc65ef841857dd306694dff1fb1cd2bf801e4

The approach doesn't propogate dependencies of crypto beyond libcrypto.
i.e. if you specify crypto somewhere else as depedency, it won't pull
CFLAGS needed for gnutls.

> That is, Meson does know that everything that needs crypto needs gnutls (see
> get_dependencies in mesonbuild/build.py if you're curious).
> 

Thanks. I've been thinking to tinker with it (that's why I made the test case).
Sounds like meson has some issues with transitive dependencies.

> I think the issue is that dependencies are listed too late---in the
> declare_dependency rather than the static_library.  Take io/ for example:
> 
> libio = static_library('io', io_ss.sources() + genh,
>dependencies: [io_ss.dependencies()],
>link_with: libqemuutil,
>name_suffix: 'fa',
>build_by_default: false)
> io = declare_dependency(link_whole: libio, dependencies: [crypto, qom])
> 
> Listing "crypto" in io's declare_dependency is enough to propagate the
> gnutls LDFLAGS down to the executables, but it does not add the CFLAGS to
> io/ files itself.  So for the io/ files we aren't telling meson that they
> need crypto (and thus in turn gnutls on the include path).
> 
> The fix should be pretty simple and localized to the "Library dependencies"
> section of meson.build.  For the two libraries above, the fixed version
> would look like:
> 
> crypto_ss.add(authz, qom)
> libcrypto = ... # same as above
> crypto = declare_dependency(link_whole: libcrypto)
> 
> io_ss.add(crypto, qom)
> ...
> libio = ... # same as above
> io = declare_dependency(link_whole: libio)
> 
> (Roman, feel free to plunder the above if you want to turn it into a commit
> message, and if it's correct of course).
> 

Unfortunately it doesn't work, even if crypto is added to io_ss. I think
that's the same issue as in shown in test case above. The patch is
below:

diff --git a/hw/nvram/meson.build b/hw/nvram/meson.build
index fd2951a860..1f2ed013b2 100644
--- a/hw/nvram/meson.build
+++ b/hw/nvram/meson.build
@@ -1,6 +1,3 @@
-# QOM interfaces must be available anytime QOM is used.
-qom_ss.add(files('fw_cfg-interface.c'))
-
 softmmu_ss.add(files('fw_cfg.c'))
 softmmu_ss.add(when: 'CONFIG_CHRP_NVRAM', if_true: files('chrp_nvram.c'))
 softmmu_ss.add(when: 'CONFIG_DS1225Y', if_true: files('ds1225y.c'))
diff --git a/io/meson.build b/io/meson.build
index bcd8b1e737..a844271b17 100644
--- a/io/meson.build
+++ b/io/meson.build
@@ -12,4 +12,4 @@ io_ss.add(files(
   'dns-resolver.c',
   'net-listener.c',
   'task.c',
-))
+), crypto)
diff --git a/meson.build b/meson.build
index 372576f82c..c293ee39e4 100644
--- a/meson.build
+++ b/meson.build
@@ -1538,6 +1538,33 @@ libqemuutil = static_library('qemuutil',
 qemuutil = declare_dependency(link_with: libqemuutil,
   sources: genh + version_res)
 
+# QOM interfaces must be available anytime QOM is used.
+qom_ss.add(files('hw/nvram/fw_cfg-interface.c'))
+qom_ss = qom_ss.apply(config_host, strict: false)
+libqom = static_library('qom', qom_ss.sources() + genh,
+dependencies: [qom_ss.dependencies()],
+name_suffix: 'fa')
+
+qom = declare_dependency(link_whole: libqom)
+
+authz_ss = 

Re: [PATCH] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Sun, Jan 03, 2021 at 08:46:27AM -0800, Joelle van Dyne wrote:
> Can you test with a low memory (-m 512) and also with single threaded
> SMP (-smp 4)? Wondering if you're hitting all the edge cases because
> there's oddities with cache flushing (can be done both in code gen and
> code exec) and interrupt handling that caused issues for me.
> 

I tested XP with default memory (128m) and -m 512. I did run Ubuntu with with
-smp 1/2/4 and multiple variants of memory (2048,4096). I've just
installed Windows 10 and it somehow works noticably faster than Ubuntu
20.04 (what makes me wonder why Ubuntu 20.04 peforms worse).

But you know, I've noticed that MTTCG is disabled by default on arm
hosts, so -smp 4 has no effect (it should print a warning IMO that smp
is noop, or even quit from qemu to disallow single-threaded TCG and -smp
flag with a value beyond 1).

If I try to enable MTTCG, I get a warning from QEMU and only one CPU
inside VM (according to Windows Task Manager).

$ build/qemu-system-x86_64 -cpu Nehalem -accel tcg,thread=multi -smp 4 -m 4096 
-hda ~/vms/win10.qcow2

qemu-system-x86_64: -accel tcg,thread=multi: warning: Guest expects a stronger 
memory ordering than the host provides
This may cause strange/hard to debug errors

As far as I understand from the ticket below this is intentional:
https://bugs.launchpad.net/qemu/+bug/1824768

> There aren't many people overall who want to try to run emulation on
> anything other than x86 host.

Perhaps we could enable MTTCG by enabling TSO in M1 like it's done in
Rosetta to avoid performance overheads of implicit barriers?

BTW, I wonder if you tried my patch? Do you hit the mentioned issues?

With regards to do_tb_phys_invalidate(), the function doesn't care
about whether it was write or exec locked. It needs write permissions
at least for TB spin lock. And something after return from
tb_phys_invalidate() needs exec permssions. I can try to find "that
something" and move change of permissions to rx closer to the place that
needs exec permissions. And then, move change of permissions to rw
inside do_tb_phys_invalidate() just before TB spin lock is acquired.

Regards,
Roman

> -j
> 
> On Sun, Jan 3, 2021 at 6:20 AM Roman Bolshakov  wrote:
> >
> > On Sat, Jan 02, 2021 at 11:55:29AM -0800, Joelle van Dyne wrote:
> > > I see you didn't touch cpu_loop_exit() and I'm curious how async
> > > interrupts are handled. Have you tested this and it works i.e. booting
> > > Windows 7 or Ubuntu 20.04? Also I've seen do_tb_phys_invalidate()
> > > called both from code generation context (write unlocked) and
> > > execution context (write locked), how does this patch differentiate
> > > the two?
> > >
> >
> > Hi Joelle,
> >
> > I used the following rule of thumb when finding places where exec/write
> > protection has to be lifted. If it returns EXC_BAD_ACCESS under lldb and
> > stack backtrace is meaningful, then a write-protected region is
> > accessed. If the stack couldn't be unwinded and EXC_BAD_ACCESS is
> > returned then the region has exec restrictions.
> >
> > With the patch I wasn't able to see any EXC_BAD_ACCESS.
> >
> > I've tested x86_64 Ubuntu 20.04 Desktop. It boots but it's very slow
> > (but faster than TCG on x86). Windows XP is much faster and quite
> > responsive. I also tested Windows 95. I'll test Win 7/Win 10 a bit
> > later.
> >
> > I'm going to update v2 shortly and going to introduce assymetric changes of
> > permissions akin to Apple's JavaScriptCore. In v2, I'm not changing
> > permission back and force unless it's needed to avoid EXC_BAD_ACCESS.
> >
> > Regards,
> > Roman
> >
> > > -j
> > >
> > > On Sat, Jan 2, 2021 at 8:13 AM Roman Bolshakov  
> > > wrote:
> > > >
> > > > On Sat, Jan 02, 2021 at 03:21:02PM +0300, Roman Bolshakov wrote:
> > > > > Pages can't be both write and executable at the same time on Apple
> > > > > Silicon. macOS provides public API to switch write protection [1] for
> > > > > JIT applications, like TCG.
> > > > >
> > > > > 1. 
> > > > > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> > > > >
> > > > > Signed-off-by: Roman Bolshakov 
> > > > > ---
> > > > >
> > > > > Happy holidays, everyone.
> > > > >
> > > > > This is somewhat similar to 
> > > > > https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-...@getutm.app/
> > > > > but I couldn't apply the series so I started from scratch.
> > > &g

Re: [PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Sun, Jan 03, 2021 at 08:52:52AM -0800, Joelle van Dyne wrote:
> MAC_OS_VERSION_11_0 is always defined. You can see in
> usr/include/AvailabilityVersions.h
> 

It's not defined on my old MPB that has Catalina (10.15.7). The last
entries are:

#define MAC_OS_X_VERSION_10_15  101500
#define MAC_OS_X_VERSION_10_15_1101501

I was able to compile the patch on Catalina without any issues (and I've
checked Catalina SDK doesn't provide pthread_jit_write_protect).

> ...
> 
> #define MAC_OS_X_VERSION_10_15  101500
> #define MAC_OS_X_VERSION_10_15_1101501
> #define MAC_OS_X_VERSION_10_16  101600
> #define MAC_OS_VERSION_11_0 11
> 
> The proper way is to do an __builtin_available check but that assumes
> you have the symbol for pthread_jit_write_protect_np which you won't
> if building on 10.15, so you need a configure time check as well.

__builtin_available is a clang extension and I'm not sure if it's
available on GCC. But I can surely add a config-time check in v3 if you
find it more preferred for iOS host support.

> I have a newer version of my patch that I haven't submitted yet
> because I was waiting for some other patches to go in first, but I can
> decouple it from the iOS stuff and submit it as a separate patchset.
> 

I'm sorry I stepped in... I didn't want to bother anyone during NY
holidays and couldn't ask for new patch revision. So I hacked it for
myself because I recently got M1 laptop and some spare time off work. In
the patch I wanted to avoid conflicts with your iOS host support patches
by limiting the patch only to macOS.

Hopefully, qemu_thread_jit_write/execute provides the room to add
reverse-enginereed implementation of pthread_jit_write_protect_np for
iOS 13 in UTM app.

Thanks,
Roman

[PATCH v2] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

Pages can't be both write and executable at the same time on Apple
Silicon. macOS provides public API to switch write protection [1] for
JIT applications, like TCG.

1. 
https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon

Signed-off-by: Roman Bolshakov 
---
v1: https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg00073.html
Changes since v1:

 - Pruned not needed fiddling with W^X and dropped symmetry from write
   lock/unlock and renamed related functions.
   Similar approach is used in JavaScriptCore [1].

 - Moved jit helper functions to util/osdep

  As outlined 
in osdep.h, this matches to (2):


* In an ideal world this header would 
contain only:   
 *  (1) things which everybody needs

*  (2) things without which code would work on most platforms but   
   *  
fail to compile or misbehave on a minority of host OSes

 - Fixed a checkpatch error

 - Limit new behaviour only to macOS 11.0 and above, because of the
   following declarations:

   __API_AVAILABLE(macos(11.0))
   __API_UNAVAILABLE(ios, tvos, watchos)
   void pthread_jit_write_protect_np(int enabled);

   __API_AVAILABLE(macos(11.0))
   __API_UNAVAILABLE(ios, tvos, watchos)
   int pthread_jit_write_protect_supported_np(void);

 1. https://bugs.webkit.org/attachment.cgi?id=402515=prettypatch

 accel/tcg/cpu-exec.c  |  2 ++
 accel/tcg/translate-all.c |  6 ++
 include/qemu/osdep.h  |  3 +++
 tcg/tcg.c |  1 +
 util/osdep.c  | 22 ++
 5 files changed, 34 insertions(+)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 8689c54499..374060eb45 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -175,6 +175,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState *cpu, 
TranslationBlock *itb)
 }
 #endif /* DEBUG_DISAS */
 
+qemu_thread_jit_execute();
 ret = tcg_qemu_tb_exec(env, tb_ptr);
 cpu->can_do_io = 1;
 last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
@@ -382,6 +383,7 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
 {
 uintptr_t old;
 
+qemu_thread_jit_write();
 assert(n < ARRAY_SIZE(tb->jmp_list_next));
 qemu_spin_lock(_next->jmp_lock);
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index b7d50a73d4..88ae5d35ef 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
 size_t size = tcg_ctx->code_gen_buffer_size;
 void *buf;
 
+#if defined(__APPLE__) && defined(MAC_OS_VERSION_11_0)
+flags |= MAP_JIT;
+#endif
 buf = mmap(NULL, size, prot, flags, -1, 0);
 if (buf == MAP_FAILED) {
 return NULL;
@@ -1485,7 +1488,9 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, 
bool rm_from_page_list)
 
 static void tb_phys_invalidate__locked(TranslationBlock *tb)
 {
+qemu_thread_jit_write();
 do_tb_phys_invalidate(tb, true);
+qemu_thread_jit_execute();
 }
 
 /* invalidate one TB
@@ -1687,6 +1692,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 #endif
 
 assert_memory_lock();
+qemu_thread_jit_write();
 
 phys_pc = get_page_addr_code(env, pc);
 
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index f9ec8c84e9..89abebcf5d 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -686,4 +686,7 @@ char *qemu_get_host_name(Error **errp);
  */
 size_t qemu_get_host_physmem(void);
 
+void qemu_thread_jit_write(void);
+void qemu_thread_jit_execute(void);
+
 #endif
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 43c6cf8f52..ab8488f5d5 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -1065,6 +1065,7 @@ void tcg_prologue_init(TCGContext *s)
 s->pool_labels = NULL;
 #endif
 
+qemu_thread_jit_write();
 /* Generate the prologue.  */
 tcg_target_qemu_prologue(s);
 
diff --git a/util/osdep.c b/util/osdep.c
index 66d01b9160..80ec7185da 100644
--- a/util/osdep.c
+++ b/util/osdep.c
@@ -606,3 +606,25 @@ writev(int fd, const struct iovec *iov, int iov_cnt)
 return readv_writev(fd, iov, iov_cnt, true);
 }
 #endif
+
+#if defined(__APPLE__) && defined(MAC_OS_VERSION_11_0)
+static inline void qemu_thread_jit_write_protect(bool enabled)
+{
+if (pthread_jit_write_protect_supported_np()) {
+pthread_jit_write_

Re: [PATCH] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Sat, Jan 02, 2021 at 11:55:29AM -0800, Joelle van Dyne wrote:
> I see you didn't touch cpu_loop_exit() and I'm curious how async
> interrupts are handled. Have you tested this and it works i.e. booting
> Windows 7 or Ubuntu 20.04? Also I've seen do_tb_phys_invalidate()
> called both from code generation context (write unlocked) and
> execution context (write locked), how does this patch differentiate
> the two?
> 

Hi Joelle,

I used the following rule of thumb when finding places where exec/write
protection has to be lifted. If it returns EXC_BAD_ACCESS under lldb and
stack backtrace is meaningful, then a write-protected region is
accessed. If the stack couldn't be unwinded and EXC_BAD_ACCESS is
returned then the region has exec restrictions.

With the patch I wasn't able to see any EXC_BAD_ACCESS.

I've tested x86_64 Ubuntu 20.04 Desktop. It boots but it's very slow
(but faster than TCG on x86). Windows XP is much faster and quite
responsive. I also tested Windows 95. I'll test Win 7/Win 10 a bit
later.

I'm going to update v2 shortly and going to introduce assymetric changes of
permissions akin to Apple's JavaScriptCore. In v2, I'm not changing
permission back and force unless it's needed to avoid EXC_BAD_ACCESS.

Regards,
Roman

> -j
> 
> On Sat, Jan 2, 2021 at 8:13 AM Roman Bolshakov  wrote:
> >
> > On Sat, Jan 02, 2021 at 03:21:02PM +0300, Roman Bolshakov wrote:
> > > Pages can't be both write and executable at the same time on Apple
> > > Silicon. macOS provides public API to switch write protection [1] for
> > > JIT applications, like TCG.
> > >
> > > 1. 
> > > https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> > >
> > > Signed-off-by: Roman Bolshakov 
> > > ---
> > >
> > > Happy holidays, everyone.
> > >
> > > This is somewhat similar to 
> > > https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-...@getutm.app/
> > > but I couldn't apply the series so I started from scratch.
> > >
> > > The primary difference from the patch above is that public API is used.
> > > Other differences:
> > >   * TB pages are mostly kept write-locked except around tcg_qemu_tb_exec()
> > >   * x86_64 macOS doesn't use MAP_JIT and W^X switches
> > >
> > > Regards,
> > > Roman
> > >
> > >  accel/tcg/cpu-exec.c  | 10 ++
> > >  accel/tcg/translate-all.c | 26 ++
> > >  include/exec/exec-all.h   |  2 ++
> > >  tcg/tcg.c |  1 +
> > >  4 files changed, 39 insertions(+)
> > >
> > > diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> > > index 8689c54499..0042fc9f2b 100644
> > > --- a/accel/tcg/cpu-exec.c
> > > +++ b/accel/tcg/cpu-exec.c
> > > @@ -175,7 +175,9 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState 
> > > *cpu, TranslationBlock *itb)
> > >  }
> > >  #endif /* DEBUG_DISAS */
> > >
> > > +tb_write_lock();
> > >  ret = tcg_qemu_tb_exec(env, tb_ptr);
> > > +tb_write_unlock();
> > >  cpu->can_do_io = 1;
> > >  last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
> > >  tb_exit = ret & TB_EXIT_MASK;
> > > @@ -220,9 +222,11 @@ static void cpu_exec_nocache(CPUState *cpu, int 
> > > max_cycles,
> > >  cflags |= MIN(max_cycles, CF_COUNT_MASK);
> > >
> > >  mmap_lock();
> > > +tb_write_unlock();
> > >  tb = tb_gen_code(cpu, orig_tb->pc, orig_tb->cs_base,
> > >   orig_tb->flags, cflags);
> > >  tb->orig_tb = orig_tb;
> > > +tb_write_lock();
> > >  mmap_unlock();
> > >
> > >  /* execute the generated code */
> > > @@ -268,7 +272,9 @@ void cpu_exec_step_atomic(CPUState *cpu)
> > >  tb = tb_lookup__cpu_state(cpu, , _base, , cf_mask);
> > >  if (tb == NULL) {
> > >  mmap_lock();
> > > +tb_write_unlock();
> > >  tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
> > > +tb_write_lock();
> > >  mmap_unlock();
> > >  }
> > >
> > > @@ -428,7 +434,9 @@ static inline TranslationBlock *tb_find(CPUState *cpu,
> > >  tb = tb_lookup__cpu_state(cpu, , _base, , cf_mask);
> > >  if (tb == NULL) {
> > >  mmap_lock();
> > > +tb_write_unlock();
> > >  tb = tb_gen_code(cpu, pc, cs_base, flags, cf_ma

Re: [PATCH] tcg: Fix execution on Apple Silicon

[Roman Bolshakov]

On Sat, Jan 02, 2021 at 03:21:02PM +0300, Roman Bolshakov wrote:
> Pages can't be both write and executable at the same time on Apple
> Silicon. macOS provides public API to switch write protection [1] for
> JIT applications, like TCG.
> 
> 1. 
> https://developer.apple.com/documentation/apple_silicon/porting_just-in-time_compilers_to_apple_silicon
> 
> Signed-off-by: Roman Bolshakov 
> ---
> 
> Happy holidays, everyone.
> 
> This is somewhat similar to 
> https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-...@getutm.app/
> but I couldn't apply the series so I started from scratch.
> 
> The primary difference from the patch above is that public API is used.
> Other differences:
>   * TB pages are mostly kept write-locked except around tcg_qemu_tb_exec()
>   * x86_64 macOS doesn't use MAP_JIT and W^X switches
> 
> Regards,
> Roman
> 
>  accel/tcg/cpu-exec.c  | 10 ++
>  accel/tcg/translate-all.c | 26 ++
>  include/exec/exec-all.h   |  2 ++
>  tcg/tcg.c |  1 +
>  4 files changed, 39 insertions(+)
> 
> diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> index 8689c54499..0042fc9f2b 100644
> --- a/accel/tcg/cpu-exec.c
> +++ b/accel/tcg/cpu-exec.c
> @@ -175,7 +175,9 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState *cpu, 
> TranslationBlock *itb)
>  }
>  #endif /* DEBUG_DISAS */
>  
> +tb_write_lock();
>  ret = tcg_qemu_tb_exec(env, tb_ptr);
> +tb_write_unlock();
>  cpu->can_do_io = 1;
>  last_tb = (TranslationBlock *)(ret & ~TB_EXIT_MASK);
>  tb_exit = ret & TB_EXIT_MASK;
> @@ -220,9 +222,11 @@ static void cpu_exec_nocache(CPUState *cpu, int 
> max_cycles,
>  cflags |= MIN(max_cycles, CF_COUNT_MASK);
>  
>  mmap_lock();
> +tb_write_unlock();
>  tb = tb_gen_code(cpu, orig_tb->pc, orig_tb->cs_base,
>   orig_tb->flags, cflags);
>  tb->orig_tb = orig_tb;
> +tb_write_lock();
>  mmap_unlock();
>  
>  /* execute the generated code */
> @@ -268,7 +272,9 @@ void cpu_exec_step_atomic(CPUState *cpu)
>  tb = tb_lookup__cpu_state(cpu, , _base, , cf_mask);
>  if (tb == NULL) {
>  mmap_lock();
> +tb_write_unlock();
>  tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
> +tb_write_lock();
>  mmap_unlock();
>  }
>  
> @@ -428,7 +434,9 @@ static inline TranslationBlock *tb_find(CPUState *cpu,
>  tb = tb_lookup__cpu_state(cpu, , _base, , cf_mask);
>  if (tb == NULL) {
>  mmap_lock();
> +tb_write_unlock();
>  tb = tb_gen_code(cpu, pc, cs_base, flags, cf_mask);
> +tb_write_lock();
>  mmap_unlock();
>  /* We add the TB in the virtual pc hash table for the fast lookup */
>  qatomic_set(>tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
> @@ -444,7 +452,9 @@ static inline TranslationBlock *tb_find(CPUState *cpu,
>  #endif
>  /* See if we can patch the calling TB. */
>  if (last_tb) {
> +tb_write_unlock();
>  tb_add_jump(last_tb, tb_exit, tb);
> +tb_write_lock();
>  }
>  return tb;
>  }
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index b7d50a73d4..1562076ffb 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -1072,6 +1072,9 @@ static inline void *alloc_code_gen_buffer(void)
>  size_t size = tcg_ctx->code_gen_buffer_size;
>  void *buf;
>  
> +#if defined(__APPLE__) && defined(__aarch64__)
> +flags |= MAP_JIT;
> +#endif
>  buf = mmap(NULL, size, prot, flags, -1, 0);
>  if (buf == MAP_FAILED) {
>  return NULL;
> @@ -1485,7 +1488,9 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, 
> bool rm_from_page_list)
>  
>  static void tb_phys_invalidate__locked(TranslationBlock *tb)
>  {
> +tb_write_unlock();
>  do_tb_phys_invalidate(tb, true);
> +tb_write_lock();
>  }
>  
>  /* invalidate one TB
> @@ -2722,3 +2727,24 @@ void tcg_flush_softmmu_tlb(CPUState *cs)
>  tlb_flush(cs);
>  #endif
>  }
> +
> +#if defined(__APPLE__) && defined(__aarch64__)
> +static void tb_write_protect(bool locked)
> +{
> +if (pthread_jit_write_protect_supported_np()){
> +pthread_jit_write_protect_np(locked);
> +}
> +}
> +#else
> +static void tb_write_protect(bool locked) {}
> +#endif
> +
> +void tb_write_lock(void)
> +{
> +tb_write_protect(true);
> +}
> +
> +void tb_write_unlock(void)
> +{
> +tb_write_protect(false);
> +}
> diff --

[PATCH v2] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first deprecated in 
macOS 11.0 - Use -[NSWorkspace openURL:] instead.
  [-Wdeprecated-declarations]
if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == YES) {
   ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
 note:
  'openFile:' has been explicitly marked deprecated here
- (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace 
openURL:] instead.", macos(10.0, 11.0));
^

Signed-off-by: Roman Bolshakov 
---
Changes since v1:
 - Changed URLWithString: to fileURLWithPath:isDirectory: (Peter)

 ui/cocoa.m | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index f32adc3074..ea3b845b53 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1178,6 +1178,7 @@ QemuCocoaView *cocoaView;
 /* Where to look for local files */
 NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"../docs/"};
 NSString *full_file_path;
+NSURL *full_file_url;
 
 /* iterate thru the possible paths until the file is found */
 int index;
@@ -1186,7 +1187,9 @@ QemuCocoaView *cocoaView;
 full_file_path = [full_file_path stringByDeletingLastPathComponent];
 full_file_path = [NSString stringWithFormat: @"%@/%@%@", 
full_file_path,
   path_array[index], filename];
-if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == YES) {
+full_file_url = [NSURL fileURLWithPath: full_file_path
+   isDirectory: false];
+if ([[NSWorkspace sharedWorkspace] openURL: full_file_url] == YES) {
 return;
 }
 }
-- 
2.29.2

Re: [PATCH] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

On Sat, Jan 02, 2021 at 01:16:48PM +, Peter Maydell wrote:
> On Sat, 2 Jan 2021 at 12:52, Roman Bolshakov  wrote:
> >
> > ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first deprecated in 
> > macOS 11.0 - Use -[NSWorkspace openURL:] instead.
> >   [-Wdeprecated-declarations]
> > if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == 
> > YES) {
> >^
> > /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
> >  note:
> >   'openFile:' has been explicitly marked deprecated here
> > - (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace 
> > openURL:] instead.", macos(10.0, 11.0));
> > ^
> >
> > Signed-off-by: Roman Bolshakov 
> > ---
> >  ui/cocoa.m | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/ui/cocoa.m b/ui/cocoa.m
> > index f32adc3074..5909758a09 100644
> > --- a/ui/cocoa.m
> > +++ b/ui/cocoa.m
> > @@ -1178,6 +1178,7 @@ QemuCocoaView *cocoaView;
> >  /* Where to look for local files */
> >  NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
> > @"../docs/"};
> >  NSString *full_file_path;
> > +NSURL *full_file_url;
> >
> >  /* iterate thru the possible paths until the file is found */
> >  int index;
> > @@ -1186,7 +1187,8 @@ QemuCocoaView *cocoaView;
> >  full_file_path = [full_file_path 
> > stringByDeletingLastPathComponent];
> >  full_file_path = [NSString stringWithFormat: @"%@/%@%@", 
> > full_file_path,
> >path_array[index], filename];
> > -if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == 
> > YES) {
> > +full_file_url = [NSURL URLWithString: full_file_path];
> > +if ([[NSWorkspace sharedWorkspace] openURL: full_file_url] == YES) 
> > {
> >  return;
> >  }
> 
> The NSURL URLWithString method documentation:
> https://developer.apple.com/documentation/foundation/nsurl/1572047-urlwithstring
> says:
> # Important
> # To create NSURL objects for file system paths, use
> fileURLWithPath:isDirectory:
> # instead.
> 
> Should we be doing that instead ?
> 

Sure, Peter. I'll update it.

Thanks,
Roman

> thanks
> -- PMM

Re: [PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

On Sat, Jan 02, 2021 at 01:25:07PM +, Peter Maydell wrote:
> On Sat, 2 Jan 2021 at 12:54, Roman Bolshakov  wrote:
> >
> > crypto/tlscreds.h includes GnuTLS headers if CONFIG_GNUTLS is set, but
> > GNUTLS_CFLAGS, that describe include path, are not propagated
> > transitively to all users of crypto and build fails if GnuTLS headers
> > reside in non-standard directory (which is a case for homebrew on Apple
> > Silicon).
> >
> > Signed-off-by: Roman Bolshakov 
> 
> Ah, this is https://bugs.launchpad.net/qemu/+bug/1909256
> -- thanks for finding a fix.
> 

No problem :)

> > ---
> >  block/meson.build  | 2 +-
> >  io/meson.build | 2 +-
> >  meson.build| 5 +++--
> >  storage-daemon/meson.build | 2 +-
> >  tests/meson.build  | 6 +++---
> >  ui/meson.build | 2 +-
> >  6 files changed, 10 insertions(+), 9 deletions(-)
> 
> > diff --git a/ui/meson.build b/ui/meson.build
> > index 013258a01c..e6655c94a6 100644
> > --- a/ui/meson.build
> > +++ b/ui/meson.build
> > @@ -29,7 +29,7 @@ vnc_ss.add(files(
> >'vnc-ws.c',
> >'vnc-jobs.c',
> >  ))
> > -vnc_ss.add(zlib, png, jpeg)
> > +vnc_ss.add(zlib, png, jpeg, gnutls)
> >  vnc_ss.add(when: sasl, if_true: files('vnc-auth-sasl.c'))
> >  softmmu_ss.add_all(when: vnc, if_true: vnc_ss)
> >  softmmu_ss.add(when: vnc, if_false: files('vnc-stubs.c'))
> 
> Question to Paolo -- it seems pretty fragile to have to explicitly
> list "these source files need these extra CFLAGS" in half a dozen
> meson.build files, because it's pretty non-obvious that adding
> eg '#include "block/nbd.h"' to a .c file means that you also
> need to update the meson.build file to say "and now it needs these
> extra CFLAGS". Isn't there some way we can just have the CFLAGS
> added more globally so that if we use gnutls.h directly or
> indirectly from more .c files in future it Just Works ?
> 

Right. I converted a big C++ project to CMake 3 a few years ago and was
able to solve the problem in CMake because it properly supports
transitive dependencies.

In CMake I'd specify that crypto has public dependency on gnutls only
once and then all users of crypto (direct or indirect) would get
required CFLAGS, LDFLAGS and include directories.

I spent a few hours trying to figure out how to achieve the same in
meson (without code duplication and failed miserably). Here's a meson
project test that illustrates the problem of dependency duplication:

https://github.com/mesonbuild/meson/commit/ff5dc65ef841857dd306694dff1fb1cd2bf801e4

The project doesn't build because dependency on foo is not propagated
beyond foobar.

The only way to build it is to specify foo twice - in source set of
foobar and in declared_dependency (i.e. appending "dependencies: [foo]"
to declare_dependency helps).

Unfortunately, the approach doesn't work for meson/qemu because it
introduces duplicate symbols in different static libraries. That's why I
used much more uglier "specify headers where needed all over the code
base".

I'd be happy to hear what's the proper way to fix it.

Thanks,
Roman

> If the build failed for the common Linux case then it would be
> at least more obvious that you needed to update the meson.build
> files. I think it's better to avoid "you need to do this special
> thing that you'll only notice you're missing if you happen to test
> on a somewhat obscure host configuration" where we can.
> 
> (We don't want to link helper binaries etc against gnutls if
> they don't need it, but that's LDFLAGS, not CFLAGS.)
> 
> thanks
> -- PMM

[PATCH] meson: Propagate gnutls dependency

[Roman Bolshakov]

crypto/tlscreds.h includes GnuTLS headers if CONFIG_GNUTLS is set, but
GNUTLS_CFLAGS, that describe include path, are not propagated
transitively to all users of crypto and build fails if GnuTLS headers
reside in non-standard directory (which is a case for homebrew on Apple
Silicon).

Signed-off-by: Roman Bolshakov 
---
 block/meson.build  | 2 +-
 io/meson.build | 2 +-
 meson.build| 5 +++--
 storage-daemon/meson.build | 2 +-
 tests/meson.build  | 6 +++---
 ui/meson.build | 2 +-
 6 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/block/meson.build b/block/meson.build
index 5dcc1e5cce..61fc5e5955 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -39,7 +39,7 @@ block_ss.add(files(
   'vmdk.c',
   'vpc.c',
   'write-threshold.c',
-), zstd, zlib)
+), zstd, zlib, gnutls)
 
 softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
 
diff --git a/io/meson.build b/io/meson.build
index bcd8b1e737..bbcd3c53a4 100644
--- a/io/meson.build
+++ b/io/meson.build
@@ -12,4 +12,4 @@ io_ss.add(files(
   'dns-resolver.c',
   'net-listener.c',
   'task.c',
-))
+), gnutls)
diff --git a/meson.build b/meson.build
index 372576f82c..d39fc018f4 100644
--- a/meson.build
+++ b/meson.build
@@ -1567,7 +1567,7 @@ blockdev_ss.add(files(
   'blockdev-nbd.c',
   'iothread.c',
   'job-qmp.c',
-))
+), gnutls)
 
 # os-posix.c contains POSIX-specific functions used by qemu-storage-daemon,
 # os-win32.c does not
@@ -1723,6 +1723,7 @@ qmp = declare_dependency(link_whole: [libqmp])
 
 libchardev = static_library('chardev', chardev_ss.sources() + genh,
 name_suffix: 'fa',
+dependencies: [gnutls],
 build_by_default: false)
 
 chardev = declare_dependency(link_whole: libchardev)
@@ -1941,7 +1942,7 @@ if have_tools
   qemu_io = executable('qemu-io', files('qemu-io.c'),
  dependencies: [block, qemuutil], install: true)
   qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
-   dependencies: [blockdev, qemuutil], install: true)
+   dependencies: [blockdev, qemuutil, gnutls], install: true)
 
   subdir('storage-daemon')
   subdir('contrib/rdmacm-mux')
diff --git a/storage-daemon/meson.build b/storage-daemon/meson.build
index c5adce81c3..68852f3d25 100644
--- a/storage-daemon/meson.build
+++ b/storage-daemon/meson.build
@@ -1,6 +1,6 @@
 qsd_ss = ss.source_set()
 qsd_ss.add(files('qemu-storage-daemon.c'))
-qsd_ss.add(blockdev, chardev, qmp, qom, qemuutil)
+qsd_ss.add(blockdev, chardev, qmp, qom, qemuutil, gnutls)
 
 subdir('qapi')
 
diff --git a/tests/meson.build b/tests/meson.build
index 1fa068f27b..29ebaba48d 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -159,11 +159,11 @@ if have_block
  'CONFIG_POSIX' in config_host
 tests += {
   'test-crypto-tlscredsx509': ['crypto-tls-x509-helpers.c', 
'pkix_asn1_tab.c',
-   tasn1, crypto],
+   tasn1, crypto, gnutls],
   'test-crypto-tlssession': ['crypto-tls-x509-helpers.c', 
'pkix_asn1_tab.c', 'crypto-tls-psk-helpers.c',
- tasn1, crypto],
+ tasn1, crypto, gnutls],
   'test-io-channel-tls': ['io-channel-helpers.c', 
'crypto-tls-x509-helpers.c', 'pkix_asn1_tab.c',
-  tasn1, io, crypto]}
+  tasn1, io, crypto, gnutls]}
   endif
   if 'CONFIG_AUTH_PAM' in config_host
 tests += {'test-authz-pam': [authz]}
diff --git a/ui/meson.build b/ui/meson.build
index 013258a01c..e6655c94a6 100644
--- a/ui/meson.build
+++ b/ui/meson.build
@@ -29,7 +29,7 @@ vnc_ss.add(files(
   'vnc-ws.c',
   'vnc-jobs.c',
 ))
-vnc_ss.add(zlib, png, jpeg)
+vnc_ss.add(zlib, png, jpeg, gnutls)
 vnc_ss.add(when: sasl, if_true: files('vnc-auth-sasl.c'))
 softmmu_ss.add_all(when: vnc, if_true: vnc_ss)
 softmmu_ss.add(when: vnc, if_false: files('vnc-stubs.c'))
-- 
2.29.2

[PATCH] ui/cocoa: Fix openFile: deprecation on Big Sur

[Roman Bolshakov]

ui/cocoa.m:1188:44: warning: 'openFile:' is deprecated: first deprecated in 
macOS 11.0 - Use -[NSWorkspace openURL:] instead.
  [-Wdeprecated-declarations]
if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == YES) {
   ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/AppKit.framework/Headers/NSWorkspace.h:350:1:
 note:
  'openFile:' has been explicitly marked deprecated here
- (BOOL)openFile:(NSString *)fullPath API_DEPRECATED("Use -[NSWorkspace 
openURL:] instead.", macos(10.0, 11.0));
^

Signed-off-by: Roman Bolshakov 
---
 ui/cocoa.m | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ui/cocoa.m b/ui/cocoa.m
index f32adc3074..5909758a09 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1178,6 +1178,7 @@ QemuCocoaView *cocoaView;
 /* Where to look for local files */
 NSString *path_array[] = {@"../share/doc/qemu/", @"../doc/qemu/", 
@"../docs/"};
 NSString *full_file_path;
+NSURL *full_file_url;
 
 /* iterate thru the possible paths until the file is found */
 int index;
@@ -1186,7 +1187,8 @@ QemuCocoaView *cocoaView;
 full_file_path = [full_file_path stringByDeletingLastPathComponent];
 full_file_path = [NSString stringWithFormat: @"%@/%@%@", 
full_file_path,
   path_array[index], filename];
-if ([[NSWorkspace sharedWorkspace] openFile: full_file_path] == YES) {
+full_file_url = [NSURL URLWithString: full_file_path];
+if ([[NSWorkspace sharedWorkspace] openURL: full_file_url] == YES) {
 return;
 }
 }
-- 
2.29.2

[PATCH RESEND] configure: Don't warn about lack of PIE on macOS

[Roman Bolshakov]

ld64 is making PIE executables for 10.7 and above by default, as
documented in ld(1).

Reviewed-by: Cameron Esfahani 
Signed-off-by: Roman Bolshakov 
---
 configure | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/configure b/configure
index 881af4b6be..942e20dfe7 100755
--- a/configure
+++ b/configure
@@ -2157,6 +2157,8 @@ elif compile_prog "-Werror -fPIE -DPIE" "-pie"; then
   CONFIGURE_CFLAGS="-fPIE -DPIE $CONFIGURE_CFLAGS"
   CONFIGURE_LDFLAGS="-pie $CONFIGURE_LDFLAGS"
   pie="yes"
+elif test "$darwin" = "yes"; then
+  pie="yes"
 elif test "$pie" = "yes"; then
   error_exit "PIE not available due to missing toolchain support"
 else
-- 
2.29.2