RE: [PATCH] xen/arm: imx8qm: Re-license file to GPL-2.0-only

[Peng Fan]

> Subject: [PATCH] xen/arm: imx8qm: Re-license file to GPL-2.0-only
>
> New contributions are recommended to be under GPL-2.0-only [1], since this
> code piece originally came from the NXP tree the original license was 
> retained.
>
> However, as discussed both Peng [2] and I [3] are ok with GPL-2.0.-only as a
> license. Change the license.
>
> Cc: Peng Fan 
> Link:
> https://lore.ke/
> rnel.org%2Fxen-devel%2F084b9ed5-1585-4802-b504-
> 6ccd2f262542%40xen.org%2F=05%7C02%7Cpeng.fan%40nxp.com%7C
> 0620114718b349f6acaf08dc5e1a914c%7C686ea1d3bc2b4c6fa92cd99c5c30
> 1635%7C0%7C0%7C638488715401364850%7CUnknown%7CTWFpbGZsb3d
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C0%7C%7C%7C=kiZCfwbHB4ovpKwpcDSsC%2BqlHp2rHWOF9CP
> Z9vr9WQ0%3D=0 [1]
> Link:
> https://lore.ke/
> rnel.org%2Fxen-
> devel%2FDU0PR04MB9417A835B5D04517CC11500788082%40DU0PR04MB
> 9417.eurprd04.prod.outlook.com%2F=05%7C02%7Cpeng.fan%40nxp.c
> om%7C0620114718b349f6acaf08dc5e1a914c%7C686ea1d3bc2b4c6fa92cd9
> 9c5c301635%7C0%7C0%7C638488715401382481%7CUnknown%7CTWFpb
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> 6Mn0%3D%7C0%7C%7C%7C=SgqvF8j1qcbrRxWlibrZOt%2F5V2kcjJQI1
> pUSLcphjoo%3D=0 [2]
> Link:
> https://lore.ke/
> rnel.org%2Fxen-devel%2Fe3785d8a-9b16-4b74-9453-
> b0166bdbb171%40actia.se%2F=05%7C02%7Cpeng.fan%40nxp.com%7
> C0620114718b349f6acaf08dc5e1a914c%7C686ea1d3bc2b4c6fa92cd99c5c3
> 01635%7C0%7C0%7C638488715401397178%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> %3D%7C0%7C%7C%7C=INhtc4gq0P5UYCo9xLaM%2BA2Fuzzm2qIFb0s
> ec1oSGaU%3D=0 [3]
> Signed-off-by: John Ernberg 

Acked-by: Peng Fan 
> ---
>  xen/arch/arm/platforms/imx8qm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/xen/arch/arm/platforms/imx8qm.c
> b/xen/arch/arm/platforms/imx8qm.c index 3600a073e8..9dac6af425 100644
> --- a/xen/arch/arm/platforms/imx8qm.c
> +++ b/xen/arch/arm/platforms/imx8qm.c
> @@ -1,4 +1,4 @@
> -/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/* SPDX-License-Identifier: GPL-2.0-only */
>  /*
>   * xen/arch/arm/platforms/imx8qm.c
>   *
> --
> 2.44.0

RE: [PATCH v4 1/3] xen/arm: Add imx8q{m,x} platform glue

[Peng Fan]

Hi Julien, John,

> Subject: Re: [PATCH v4 1/3] xen/arm: Add imx8q{m,x} platform glue
>
> Hi John,
>
> On 15/04/2024 12:17, John Ernberg wrote:
> > Hi Julien,
> >
> > On 4/15/24 1:03 PM, Julien Grall wrote:
> >>
> >>
> >> On 15/04/2024 11:50, Andrew Cooper wrote:
> >>> On 15/04/2024 11:25 am, Julien Grall wrote:
>  Hi John,
> 
>  I saw this patch was committed. I have one question this may
>  require some adjustment.
> 
>  On 08/04/2024 17:11, John Ernberg wrote:
> > ---
> > xen/arch/arm/platforms/Makefile |   1 +
> > xen/arch/arm/platforms/imx8qm.c | 139
> > 
> > 2 files changed, 140 insertions(+)
> > create mode 100644 xen/arch/arm/platforms/imx8qm.c
> >
> > diff --git a/xen/arch/arm/platforms/Makefile
> > b/xen/arch/arm/platforms/Makefile index 8632f4115f..bec6e55d1f
> > 100644
> > --- a/xen/arch/arm/platforms/Makefile
> > +++ b/xen/arch/arm/platforms/Makefile
> > @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
> > obj-$(CONFIG_ALL64_PLAT) += thunderx.o
> > obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
> > obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
> > +obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
> > obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
> > obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o diff
> > --git a/xen/arch/arm/platforms/imx8qm.c
> > b/xen/arch/arm/platforms/imx8qm.c new file mode 100644 index
> > 00..3600a073e8
> > --- /dev/null
> > +++ b/xen/arch/arm/platforms/imx8qm.c
> > @@ -0,0 +1,139 @@
> > +/* SPDX-License-Identifier: GPL-2.0-or-later */
> 
>  The majority of Xen code is using GPL-2.0-only. In the early days
>  for Xen on Arm we started to use GPLv2+ which I consider it was a
> mistake.
>  Unfortunately this started to spread as people copied/pasted the
>  same copyright headers.
> 
>  So can you confirm whether you intended to use GPL-2.0+? If not
>  would you be able to send a patch to adjust it? (Better to it
>  before there are more modifications).
> >>>
> >>> Julien: I've called you out multiple times before.
> >>
> >> And there are multiple thread explaining why I am requesting if we
> >> can use GPLv2-only. In fact from CONTRIBUTING:
> >>
> >> The recommended license of a directory will depend on the COPYING file.
> >> If the new file is using a different license, this should be
> >> highlighted and discussed in the commit message or cover letter
> >> introducing the file.
> >>
> >
> > Since part of the code was not written by me, but by Peng, I think
> > both of us need to agree to a license change if one is to be made.
>
> Ah I didn't realize that Peng also contributed. Let's wait if he is happy 
> with the
> change.

I am ok with the change.

Thanks,
Peng.
>
> Also, offline, I was pointed out that I could explain a little bit more why I
> asked if this could be changed. From [1]:
>
> "IIRC from past discussion there are two broads concern with GPLv2+:
> - We are leaving the choice of which license applies to the person copying
> the code. So if a new version is released that is less favorable to the 
> initial
> contributor, then we have no leverage.
> - Some companies are rather cautious to contribute code that may be
> licensed under GPLv3 (would be allowed with GPLv2+).
>
> The later is particularly a problem because not many people realize that a 
> fair
> part of Xen on Arm is GPLv2+. I never really understood why we chose that
> (this was before my time) but this got spread as the existing copyright was
> added to a new file. Admittely, the contributor should be more cautious. But I
> would not say this is trivial to spot the difference."
>
> Cheers,
>
> [1]
> https://lore.ke/
> rnel.org%2Fxen-devel%2Ff235f6f8-d585-4e24-7fc8-
> 3f2df9240c9d%40xen.org%2F=05%7C02%7Cpeng.fan%40nxp.com%7C
> 07ac104b960d476c349f08dc5d420db6%7C686ea1d3bc2b4c6fa92cd99c5c3
> 01635%7C0%7C0%7C638487785488861385%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> %3D%7C0%7C%7C%7C=d5RQJ8nek5a8Xgi%2B%2F9g3GqDL8CZmd3c
> O9EN6w57S9qE%3D=0
>
> --
> Julien Grall

RE: [PATCH v4 3/3] MAINTAINERS: Become a reviewer of iMX8Q{M,XP} related patches

[Peng Fan]

> Subject: [PATCH v4 3/3] MAINTAINERS: Become a reviewer of iMX8Q{M,XP}
> related patches
> 
> I have experience with the IMX8QXP, and the supported parts of the IMX8QM
> are identical.
> 
> Help review patches touching these areas.
> 
> Signed-off-by: John Ernberg 
> Acked-by: Stefano Stabellini 
> 
> ---
> 
> v4:
>  - Properly sign the patch off (Stefano Stabellini)
>  - Pick up Stefano's ack
> 
> v3:
>  - New patch (Bertrand Marquis)
> ---
>  MAINTAINERS | 5 +
>  1 file changed, 5 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 1bd22fd75f..09982241b3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -337,6 +337,11 @@ F:   tools/misc/xenhypfs.c
>  F:   xen/common/hypfs.c
>  F:   xen/include/xen/hypfs.h
> 
> +IMX8QM/QXP SUPPORT
> +R:   John Ernberg 
> +F:   xen/arch/arm/platforms/imx8qm.c
> +F:   xen/drivers/char/imx-lpuart.c
> +
>  INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
>  R:   Lukasz Hawrylko 
>  R:   Daniel P. Smith 
> --
Acked-by: Peng Fan 

RE: [PATCH v4 1/3] xen/arm: Add imx8q{m,x} platform glue

[Peng Fan]

> Subject: [PATCH v4 1/3] xen/arm: Add imx8q{m,x} platform glue
> 
> When using Linux for dom0 there are a bunch of drivers that need to do SMC
> SIP calls into the firmware to enable certain hardware bits like the watchdog.
> 
> Provide a basic platform glue that implements the needed SMC forwarding.
> 
> The format of these calls are as follows:
>  - reg 0: function ID
>  - reg 1: subfunction ID (when there's a subfunction)  remaining regs: args
> 
> For now we only allow Dom0 to make these calls as they are all managing
> hardware. There is no specification for these SIP calls, the IDs and names 
> have
> been extracted from the upstream linux kernel and the vendor kernel.
> 
> Most of the SIP calls are only available for the iMX8M series of SoCs, so they
> are easy to reject and they need to be revisited when iMX8M series support is
> added.
> 
> From the other calls we can reject CPUFREQ because Dom0 cannot make an
> informed decision regarding CPU frequency scaling, WAKEUP_SRC is to wake
> up from suspend, which Xen doesn't support at this time.
> 
> This leaves the TIME SIP, OTP SIPs, BUILDINFO SIP and TEMP ALARM SIP,
> which for now are allowed to Dom0.
> 
> NOTE: This code is based on code found in NXP Xen tree located here:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2Fnxp-imx%2Fimx-xen%2Fblob%2Flf-
> 5.10.y_4.13%2Fxen%2Farch%2Farm%2Fplatforms%2Fimx8qm.c=05%7
> C02%7Cpeng.fan%40nxp.com%7C5602a2d54f074c7dcd9608dc57e69286%7
> C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6384818950076004
> 32%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=qSf9A7yE17P
> 2q2%2FqXINORXh2EfFAS1%2BilhiY%2FexjcbA%3D=0
> 
> Signed-off-by: Peng Fan 
> [jernberg: Add SIP call filtering]
> Signed-off-by: John Ernberg 
> 
> ---
> 
> v4:
>  - Fix coding style problems added in v3 (Michal Orzel)
>  - Remove all calls not currently used by Linux on imx8q{m,xp} (Michal Orzel)
>  - Fix {g,}printk inconsistencies, prefer gprintk (Michal Orzel)
> 
> v3:
>  - Adhere to style guidelines for line length and label indentation (Michal
> Orzel)
>  - Use smccc macros to build the SIP function identifier (Michal Orzel)
>  - Adjust platform name to be specific to QM and QXP variants (Michal Orzel)
>  - Pick up additional SIPs which may be used by other Linux versions (Michal
> Orzel)
>  - Changes to the commit message due to above
> 
> v2:
>  - Reword the commit message to be a bit clearer
>  - Include the link previously added as a context note to the commit message
> (Julien Grall)
>  - Add Pengs signed off (Julien Grall, Peng Fan)
>  - Add basic SIP call filter (Julien Grall)
>  - Expand the commit message a whole bunch because of the changes to the
> code
> ---
>  xen/arch/arm/platforms/Makefile |   1 +
>  xen/arch/arm/platforms/imx8qm.c | 139
> 
>  2 files changed, 140 insertions(+)
>  create mode 100644 xen/arch/arm/platforms/imx8qm.c
> 
> diff --git a/xen/arch/arm/platforms/Makefile
> b/xen/arch/arm/platforms/Makefile index 8632f4115f..bec6e55d1f 100644
> --- a/xen/arch/arm/platforms/Makefile
> +++ b/xen/arch/arm/platforms/Makefile
> @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
>  obj-$(CONFIG_ALL64_PLAT) += thunderx.o
>  obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
>  obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
> +obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
>  obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
>  obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o diff --git
> a/xen/arch/arm/platforms/imx8qm.c b/xen/arch/arm/platforms/imx8qm.c
> new file mode 100644 index 00..3600a073e8
> --- /dev/null
> +++ b/xen/arch/arm/platforms/imx8qm.c
> @@ -0,0 +1,139 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * xen/arch/arm/platforms/imx8qm.c
> + *
> + * i.MX 8QM setup
> + *
> + * Copyright (c) 2016 Freescale Inc.
> + * Copyright 2018-2019 NXP
> + *
> + *
> + * Peng Fan 
> + */
> +
> +#include 
> +#include 
> +#include 
> +
> +static const char * const imx8qm_dt_compat[] __initconst = {
> +"fsl,imx8qm",
> +"fsl,imx8qxp",
> +NULL
> +};
> +
> +#define IMX_SIP_FID(fid) \
> +ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
> +   ARM_SMCCC_CONV_64, \
> +   ARM_SMCCC_OWNER_SIP, \
> +   (fid))
> +
> +#define IMX_SIP_F_CPUFREQ0x1
> +#define IMX_SIP_F_TIME   0x2
> +#define IMX_SIP_F_WAKEUP_SRC 0x9
> +#define IMX_SIP_F_OTP_WRITE  0xB
> +
> +#define IMX_SIP_TIME_SF_RTC_SET_TIME 0x00
> +#define IMX_SIP_T

RE: [PATCH v4 2/3] xen/drivers: imx-lpuart: Replace iMX8QM compatible with iMX8QXP

[Peng Fan]

> Subject: [PATCH v4 2/3] xen/drivers: imx-lpuart: Replace iMX8QM compatible
> with iMX8QXP
> 
> Allow the uart to probe also with iMX8QXP. The ip-block is the same as in the
> QM.
> 
> Since the fsl,imx8qm-lpuart compatible in Linux exists in name only and is not
> used in the driver any iMX8QM device tree that can boot Linux must set
> fsl,imx8qxp-lpuart compatible as well as the QM one.
> 
> Thus we replace the compatible rather than adding just another one.
> 
> Signed-off-by: John Ernberg 
> Acked-by: Julien Grall 
> 
> ---
> 
> v4:
>  - no changes
> 
> v3:
>  - no changes
> 
> v2:
>  - Replace the compatible rather than adding to the list (Julien Grall)
>  - Reword commit message to reflect the above.
>  - Collect Julien's ack
> ---
>  xen/drivers/char/imx-lpuart.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
> index 7770d158bf..cf034d7ed0 100644
> --- a/xen/drivers/char/imx-lpuart.c
> +++ b/xen/drivers/char/imx-lpuart.c
> @@ -236,7 +236,7 @@ static int __init imx_lpuart_init(struct
> dt_device_node *dev,
> 
>  static const struct dt_device_match imx_lpuart_dt_compat[] __initconst =  {
> -DT_MATCH_COMPATIBLE("fsl,imx8qm-lpuart"),
> +DT_MATCH_COMPATIBLE("fsl,imx8qxp-lpuart"),
>  { /* sentinel */ },

Reviewed-by: Peng Fan 
>  };
> 
> --
> 2.44.0

RE: [PATCH 2/2] xen/arm: Add i.MX UART driver

[Peng Fan]

Hi Oleksandr,

> Subject: [PATCH 2/2] xen/arm: Add i.MX UART driver
> 
> From: Oleksandr Tyshchenko 
> 
> The i.MX UART Documentation:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> nxp.com%2Fwebapp%2FDownload%3FcolCode%3DIMX8MMRM=05%7
> C02%7Cpeng.fan%40nxp.com%7C6ada06c4133849667f3608dc530d5471%7
> C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6384765639197564
> 70%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=RmXgAMb7
> wFZ7epZgYgHJo4LH35rzQhD05yTXSkttXbc%3D=0
> Chapter 16.2 Universal Asynchronous Receiver/Transmitter (UART)
> 
> Tested on i.MX 8M Mini only, but I guess, it should be suitable for other
> i.MX8M* SoCs (those UART device tree nodes contain "fsl,imx6q-uart"
> compatible string).

Good to see people are interested in XEN on 8M.
I had an implementation back in 2015, you could take a look.

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#define imx_read(uart, off)   readl((uart)->regs + off)
#define imx_write(uart, off, val) writel((val), (uart)->regs + off)

#define CONFIG_IMX8_ZEBU
static struct imx_uart {
unsigned int baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
unsigned int irq;
char __iomem *regs;
struct irqaction irqaction;
struct vuart_info vuart;
} imx_com = {0};

static void imx_uart_interrupt(int irq, void *data, struct cpu_user_regs *regs)
{
struct serial_port *port = data;
struct imx_uart *uart = port->uart;
unsigned int sts, sts2;

sts = imx_read(uart, USR1);

if ((sts & USR1_RRDY) || (sts & USR1_AGTIM)) {
if (sts & USR1_AGTIM)
imx_write(uart, USR1, USR1_AGTIM);
serial_rx_interrupt(port, regs);
}

if ((sts & USR1_TRDY) && (imx_read(uart, UCR1) & UCR1_TXMPTYEN)) {
serial_tx_interrupt(port, regs);
}

if (sts & USR1_AWAKE) {
imx_write(uart, USR1, USR1_AWAKE);
}

if (sts & USR1_AIRINT) {
imx_write(uart, USR1, USR1_AIRINT);
}

sts2 = imx_read(uart, USR2);
if (sts2 & USR2_ORE) {
dprintk(XENLOG_ERR, "uart: rxfifo overrun\n");
imx_write(uart, USR2, sts2 | USR2_ORE);
}
}

static void __init imx_uart_init_preirq(struct serial_port *port)
{
struct imx_uart *uart = port->uart;
int val;

dprintk(XENLOG_ERR, "10\n");
imx_write(uart, USR1, USR1_RTSD);

dprintk(XENLOG_ERR, "12\n");
val = imx_read(uart, UCR1);
val |= UCR1_RRDYEN | UCR1_RTSDEN | UCR1_UARTEN;
imx_write(uart, UCR1, val);

dprintk(XENLOG_ERR, "13\n");
val = imx_read(uart, UCR2);
val |= UCR2_RXEN | UCR2_TXEN | UCR2_IRTS;
imx_write(uart, UCR2, val);

//#ifdef CONFIG_IMX8_ZEBU
#if 0
imx_write(uart, UFCR, (2U << 10) | (5U << 7) || (0U << 6) | (1U << 0));
#else
#define TXTL 2 /* reset default */
#define RXTL 1 /* For console port */
#define RXTL_UART 16 /* For uart */
val = imx_read(uart, UFCR) & (UFCR_RFDIV | UFCR_DCEDTE);
val |= TXTL << UFCR_TXTL_SHF | RXTL;
imx_write(uart, UFCR, val);
#endif
dprintk(XENLOG_ERR, "14\n");
dprintk(XENLOG_ERR, "===UFCR USR1 %x %x\n", imx_read(uart, 
UFCR), imx_read(uart, USR1));
}

static void __init imx_uart_init_postirq(struct serial_port *port)
{
struct imx_uart *uart = port->uart;

dprintk(XENLOG_ERR, "20\n");
uart->irqaction.handler = imx_uart_interrupt;
uart->irqaction.name = "imx_uart";
uart->irqaction.dev_id = port;

if ( setup_irq(uart->irq, 0, >irqaction) != 0 )
{
dprintk(XENLOG_ERR, "Failed to allocated imx_uart IRQ %d\n",
uart->irq);
return;
}

#ifdef CONFIG_IMX8_ZEBU
// Send autobaud character
imx_write(uart, UTXD, 0x55);
imx_write(uart, UTXD, 0x55);
imx_write(uart, UTXD, 0x0A);
#endif
/* Enable interrupts */
dprintk(XENLOG_ERR, "%s\n", __func__);
imx_write(uart, UCR1, UCR1_TXMPTYEN | UCR1_RRDYEN | UCR1_UARTEN);
//imx_write(uart, UCR1, UCR1_TXMPTYEN | UCR1_UARTEN);
}

static void imx_uart_suspend(struct serial_port *port)
{
BUG();
}

static void imx_uart_resume(struct serial_port *port)
{
BUG();
}

static int imx_uart_tx_ready(struct serial_port *port)
{
struct imx_uart *uart = port->uart;

return (imx_read(uart, USR2) & USR2_TXDC) ?  1 : 0;
}

static void imx_uart_putc(struct serial_port *port, char c)
{
struct imx_uart *uart = port->uart;

imx_write(uart, UTXD, (uint32_t)(unsigned char)c);
}

static int imx_uart_getc(struct serial_port *port, char *pc)
{
struct imx_uart *uart = port->uart;

if (!(imx_read(uart, USR2) & USR2_RDR))
return 0;

*pc = imx_read(uart, URXD) & 0xff;

return 1;
}

static int __init imx_uart_irq(struct serial_port *port)
{
struct imx_uart *uart = port->uart;

return ((uart->irq > 0) ? uart->irq : -1);
}

static const struct vuart_info *imx_vuart_info(struct serial_port *port)
{
struct imx_uart *uart = 

RE: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue

[Peng Fan]

> Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
> 
> On Wed, 20 Mar 2024, Julien Grall wrote:
> > Hi John,
> >
> > On 20/03/2024 16:24, John Ernberg wrote:
> > > Hi Bertrand,
> > >
> > > On 3/13/24 11:07, Bertrand Marquis wrote:
> > > > Hi,
> > > >
> > > > > On 8 Mar 2024, at 15:04, Julien Grall  wrote:
> > > > >
> > > > > Hi John,
> > > > >
> > > > > Thank you for the reply.
> > > > >
> > > > > On 08/03/2024 13:40, John Ernberg wrote:
> > > > > > On 3/7/24 00:07, Julien Grall wrote:
> > > > > > >> Ping on the watchdog discussion bits.
> > > > > > >
> > > > > > > Sorry for the late reply.
> > > > > > >
> > > > > > > On 06/03/2024 13:13, John Ernberg wrote:
> > > > > > > > On 2/9/24 14:14, John Ernberg wrote:
> > > > > > > > >
> > > > > > > > > >   * IMX_SIP_TIMER_*:  This seems to be related to
> > > > > > > > > > the watchdog.
> > > > > > > > > > Shouldn't dom0 rely on the watchdog provided by Xen
> instead?
> > > > > > > > > > So those
> > > > > > > > > > call will be used by Xen.
> > > > > > > > >
> > > > > > > > > That is indeed a watchdog SIP, and also for setting the
> > > > > > > > > SoC internal RTC if it is being used.
> > > > > > > > >
> > > > > > > > > I looked around if there was previous discussion and
> > > > > > > > > only really found [3].
> > > > > > > > > Is the xen/xen/include/watchdog.h header meant to be for
> > > > > > > > > this kind of watchdog support or is that more for the VM
> > > > > > > > > watchdog? Looking at the x86 ACPI NMI watchdog it seems
> > > > > > > > > like the former, but I have never worked with
> > > > > > > > > x86 nor ACPI.
> > > > > > >
> > > > > > > include/watchdog.h contains helper to configure the watchdog
> > > > > > > for Xen. We also have per-VM watchdog and this is configured
> > > > > > > by the hypercall SCHEDOP_watchdog.
> > > > > > >
> > > > > > > > >
> > > > > > > > > Currently forwarding it to Dom0 has not caused any
> > > > > > > > > watchdog resets with our watchdog timeout settings, our
> > > > > > > > > specific Dom0 setup and VM count.
> > > > > > >
> > > > > > > IIUC, the SMC API for the watchdog would be similar to the
> > > > > > > ACPI NMI watchdog. So I think it would make more sense if
> > > > > > > this is not exposed to
> > > > > > > dom0 (even if Xen is doing nothing with it).
> > > > > > >
> > > > > > > Can you try to hide the SMCs and check if dom0 still behave
> > > > > > > properly?
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > This SMC manages a hardware watchdog, if it's not pinged
> > > > > > within a specific interval the entire board resets.
> > > > >
> > > > > Do you know what's the default interval? Is it large enough so
> > > > > Xen +
> > > > > dom0 can boot (at least up to when the watchdog driver is 
> > > > > initialized)?
> > > > >
> > > > > > If I block the SMCs the watchdog driver in Dom0 will fail to
> > > > > > ping the watchdog, triggering a board reset because the system
> > > > > > looks to have become unresponsive. The reason this patch set
> > > > > > started is because we couldn't ping the watchdog when running with
> Xen.
> > > > > > In our specific system the bootloader enables the watchdog as
> > > > > > early as possible so that we can get watchdog protection for
> > > > > > as much of the boot as we possibly can.
> > > > > > So, if we are to block the SMC from Dom0, then Xen needs to
> > > > > > take over the pinging. It could be implemented similarly to
> > > > > > the NMI watchdog, except that the system will reset if the
> > > > > > ping is missed rather than backtrace.
> > > > > > It would also mean that Xen will get a whole watchdog
> > > > > > driver-category due to the watchdog being vendor and sometimes
> > > > > > even SoC specific when it comes to Arm.
> > > > > > My understanding of the domain watchdog code is that today the
> > > > > > domain needs to call SCHEDOP_watchdog at least once to start
> > > > > > the watchdog timer. Since watchdog protection through the
> > > > > > whole boot process is desirable we'd need some core changes,
> > > > > > such as an option to start the domain watchdog on init. > It's
> > > > > > quite a big change to make
> > > > >
> > > > > For clarification, above you seem to mention two changes:
> > > > >
> > > > > 1) Allow Xen to use the HW watchdog
> > > > > 2) Allow the domain to use the watchdog early
> > > > >
> > > > > I am assuming by big change, you are referring to 2?
> > > > >
> > > > > , while I am not against doing it if it
> > > > > > makes sense, I now wonder if Xen should manage hardware
> watchdogs.
> > > > > > Looking at the domain watchdog code it looks like if a domain
> > > > > > does not get enough execution time, the watchdog will not be
> > > > > > pinged enough and the guest will be reset. So either watchdog
> > > > > > approach requires Dom0 to get execution time. Dom0 also needs
> > > > > > to service all the PV backends it's responsible for. I'm not
> > > > > > sure it's valuable to add 

RE: question about virtio-vsock on xen

[Peng Fan]

> Subject: Re: question about virtio-vsock on xen
> 
> 
> 
> On 26.02.24 05:09, Peng Fan wrote:
> > Hi Oleksandr,
> 
> Hello Peng
> 
> 
> [snip]
> 
> >>
> >> ... Peng, we have vhost-vsock (and vhost-net) Xen PoC. Although
> >> it is non- upstreamable in its current shape (based on old Linux
> >> version, requires some rework and proper integration, most likely
> >> requires involving Qemu and protocol changes to pass an additional
> >> info to vhost), it works with Linux
> >> v5.10 + patched Qemu v7.0, so you can refer to the Yocto meta layer
> >> which contains kernel patches for the details [1].
> >
> > Thanks for the pointer, I am reading the code.
> >
> >>
> >> In a nutshell, before accessing the guest data the host module needs
> >> to map descriptors in virtio rings which contain either guest grant
> >> based DMA addresses (by using Xen grant mappings) or guest
> >> pseudo-physical addresses (by using Xen foreign mappings). After
> >> accessing the guest data the host module needs to unmap them.
> >
> > Ok, I thought  the current xen virtio code already map every ready.
> >
> 
> It does, as you said the virtio-blk-pci worked in your environment. But
> vhost(-vsock) is a special case, unlike for virtio-blk-pci where the whole
> backend resides in Qemu, here we have a split model. As I understand the
> Qemu performs only initial setup/configuration then offloads the I/O
> processing to a separate entity which is the Linux module in that particular
> case.

Thanks for sharing me the information. I need to learn more stuff:)

Thanks,
Peng.

RE: question about virtio-vsock on xen

[Peng Fan]

> Subject: RE: question about virtio-vsock on xen
> 
> On Mon, 26 Feb 2024, Peng Fan wrote:
> > Hi Stefano,
> >
> > > Subject: Re: question about virtio-vsock on xen
> > >
> > > Hi Peng,
> > >
> > > We haven't tried to setup virtio-vsock yet.
> > >
> > > In general, I am very supportive of using QEMU for virtio backends.
> > > We use QEMU to provide virtio-net, virtio-block, virtio-console and more.
> >
> > Would you mind share how to setup virtio-console using qemu + xen?
> 
> Vikram (CCed) has been working on it and should be able to share more
> details
> 
> 
> > > However, typically virtio-vsock comes into play for VM-to-VM
> > > communication, which is different. Going via QEMU in Dom0 just to
> > > have 1 VM communicate with another VM is not an ideal design: it
> > > adds latency and uses resources in Dom0 when actually we could do
> without it.
> > >
> > > A better model for VM-to-VM communication would be to have the VM
> > > talk to each other directly via grant table or pre-shared memory
> > > (see the static shared memory feature) or via Xen hypercalls (see
> > > Argo.)
> >
> > The goal is to make android trout VM run with XEN + i.MX95, so need vsock.
> 
> I am not familiar with the details of Android Trout... Where is vsock used?
> Just asking for my own understanding.

Here is the link
https://source.android.com/docs/automotive/virtualization/reference_platform

vsock is used for audio control hal, dumpstate HAL, vehicle HAL.

Regards,
Peng.

> 
> 
> > > For a good Xen design, I think the virtio-vsock backend would need
> > > to be in Xen itself (the hypervisor).
> > >
> > > Of course that is more work and it doesn't help you with the
> > > specific question you had below :-)
> > >
> > > For that, I don't have a pointer to help you but maybe others in CC have.
> > >
> > > Cheers,
> > >
> > > Stefano
> > >
> > >
> > > On Fri, 23 Feb 2024, Peng Fan wrote:
> > > > Hi All,
> > > >
> > > > Has anyone make virtio-vsock on xen work? My dm args as below:
> > > >
> > > > virtio = [
> > > >
> > > 'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type
> > > =qem
> > > u,grant_usage=true'
> > > > ]
> > > > device_model_args = [
> > > > '-D', '/home/root/qemu_log.txt',
> > > > '-d',
> > > > 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,tra
> > > > ce:* pci_route*,trace:*handle_ioreq*,trace:*xen*',
> > > > '-device',
> > > > 'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie
> > > > .0,a
> > > > ddr=5.0,guest-cid=3']
> > > >
> > > > During my test, it always return failure in dom0 kernel in below code:
> > > >
> > > > vhost_transport_do_send_pkt {
> > > > ...
> > > >nbytes = copy_to_iter(hdr, sizeof(*hdr), _iter);
> > > > if (nbytes != sizeof(*hdr)) {
> > > > vq_err(vq, "Faulted on copying pkt hdr %x
> > > > %x %x %px\n",
> > > nbytes, sizeof(*hdr),
> > > > __builtin_object_size(hdr, 0), _iter);
> > > > kfree_skb(skb);
> > > > break;
> > > > }
> > > > }
> > > >
> > > > I checked copy_to_iter, it is copy data to __user addr, but it
> > > > never pass, the copy to __user addr always return 0 bytes copied.
> > > >
> > > > The asm code "sttr x7, [x6]" will trigger data abort, the kernel
> > > > will run into do_page_fault, but lock_mm_and_find_vma report it is
> > > > VM_FAULT_BADMAP, that means the __user addr is not mapped, no
> vma
> > > has this addr.
> > > >
> > > > I am not sure what may cause this. Appreciate if any comments.
> > > >
> > > > BTW: I tested blk pci, it works, so the virtio pci should work on my 
> > > > setup.
> > > >
> > > > Thanks,
> > > > Peng.
> > > >
> >

RE: question about virtio-vsock on xen

[Peng Fan]

Hi Oleksandr,

> Subject: Re: question about virtio-vsock on xen
> 
> 
> 
> On 23.02.24 23:42, Stefano Stabellini wrote:
> > Hi Peng,
> 
> Hello Peng, Stefano
> 
> 
> >
> > We haven't tried to setup virtio-vsock yet.
> >
> > In general, I am very supportive of using QEMU for virtio backends. We
> > use QEMU to provide virtio-net, virtio-block, virtio-console and more.
> >
> > However, typically virtio-vsock comes into play for VM-to-VM
> > communication, which is different. Going via QEMU in Dom0 just to have
> > 1 VM communicate with another VM is not an ideal design: it adds
> > latency and uses resources in Dom0 when actually we could do without it.
> >
> > A better model for VM-to-VM communication would be to have the VM talk
> > to each other directly via grant table or pre-shared memory (see the
> > static shared memory feature) or via Xen hypercalls (see Argo.)
> >
> > For a good Xen design, I think the virtio-vsock backend would need to
> > be in Xen itself (the hypervisor).
> >
> > Of course that is more work and it doesn't help you with the specific
> > question you had below :-)
> >
> > For that, I don't have a pointer to help you but maybe others in CC
> > have.
> 
> 
> Yes, I will try to provide some info ...
> 
> 
> >
> > Cheers,
> >
> > Stefano
> >
> >
> > On Fri, 23 Feb 2024, Peng Fan wrote:
> >> Hi All,
> >>
> >> Has anyone make virtio-vsock on xen work? My dm args as below:
> >>
> >> virtio = [
> >>
> 'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type=qem
> u,grant_usage=true'
> >> ]
> >> device_model_args = [
> >> '-D', '/home/root/qemu_log.txt',
> >> '-d',
> >> 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:
> >> *pci_route*,trace:*handle_ioreq*,trace:*xen*',
> >> '-device',
> >> 'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie.0,
> >> addr=5.0,guest-cid=3']
> >>
> >> During my test, it always return failure in dom0 kernel in below code:
> >>
> >> vhost_transport_do_send_pkt {
> >> ...
> >> nbytes = copy_to_iter(hdr, sizeof(*hdr), _iter);
> >>  if (nbytes != sizeof(*hdr)) {
> >>  vq_err(vq, "Faulted on copying pkt hdr %x %x
> >> %x %px\n", nbytes, sizeof(*hdr), __builtin_object_size(hdr, 0), _iter);
> >>  kfree_skb(skb);
> >>  break;
> >>  }
> >> }
> >>
> >> I checked copy_to_iter, it is copy data to __user addr, but it never
> >> pass, the copy to __user addr always return 0 bytes copied.
> >>
> >> The asm code "sttr x7, [x6]" will trigger data abort, the kernel will
> >> run into do_page_fault, but lock_mm_and_find_vma report it is
> >> VM_FAULT_BADMAP, that means the __user addr is not mapped, no vma
> has this addr.
> >>
> >> I am not sure what may cause this. Appreciate if any comments.
> 
> 
>... Peng, we have vhost-vsock (and vhost-net) Xen PoC. Although it is non-
> upstreamable in its current shape (based on old Linux version, requires some
> rework and proper integration, most likely requires involving Qemu and
> protocol changes to pass an additional info to vhost), it works with Linux
> v5.10 + patched Qemu v7.0, so you can refer to the Yocto meta layer which
> contains kernel patches for the details [1].

Thanks for the pointer, I am reading the code.

> 
> In a nutshell, before accessing the guest data the host module needs to map
> descriptors in virtio rings which contain either guest grant based DMA
> addresses (by using Xen grant mappings) or guest pseudo-physical addresses
> (by using Xen foreign mappings). After accessing the guest data the host
> module needs to unmap them.

Ok, I thought  the current xen virtio code already map every ready.

> 
> Also note, in that PoC the target mapping scheme is controlled via module
> param and guest domain id is retrieved from the device-model specific part in
> the Xenstore (so Qemu/protocol are unmodified). But you might want to look
> at [2] as an example of vhost-user protocol changes how to pass that
> additional info.

Sure, thanks very much on the link. Giving a look.
> 
> Hope that helps.

Definitely.

Thanks,
Peng.

> 
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2Fxen-troops%2Fmeta-xt-
> vhost%2Fcommits%2Fmain%2F=05%7C02%7Cpeng.fan%40n

RE: question about virtio-vsock on xen

[Peng Fan]

Hi Stefano,

> Subject: Re: question about virtio-vsock on xen
> 
> Hi Peng,
> 
> We haven't tried to setup virtio-vsock yet.
> 
> In general, I am very supportive of using QEMU for virtio backends. We use
> QEMU to provide virtio-net, virtio-block, virtio-console and more.

Would you mind share how to setup virtio-console using qemu + xen?

> 
> However, typically virtio-vsock comes into play for VM-to-VM
> communication, which is different. Going via QEMU in Dom0 just to have 1
> VM communicate with another VM is not an ideal design: it adds latency and
> uses resources in Dom0 when actually we could do without it.
> 
> A better model for VM-to-VM communication would be to have the VM talk
> to each other directly via grant table or pre-shared memory (see the static
> shared memory feature) or via Xen hypercalls (see Argo.)

The goal is to make android trout VM run with XEN + i.MX95, so need vsock.

Thanks,
Peng.

> 
> For a good Xen design, I think the virtio-vsock backend would need to be in
> Xen itself (the hypervisor).
> 
> Of course that is more work and it doesn't help you with the specific question
> you had below :-)
> 
> For that, I don't have a pointer to help you but maybe others in CC have.
> 
> Cheers,
> 
> Stefano
> 
> 
> On Fri, 23 Feb 2024, Peng Fan wrote:
> > Hi All,
> >
> > Has anyone make virtio-vsock on xen work? My dm args as below:
> >
> > virtio = [
> >
> 'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type=qem
> u,grant_usage=true'
> > ]
> > device_model_args = [
> > '-D', '/home/root/qemu_log.txt',
> > '-d',
> > 'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:*
> > pci_route*,trace:*handle_ioreq*,trace:*xen*',
> > '-device',
> > 'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie.0,a
> > ddr=5.0,guest-cid=3']
> >
> > During my test, it always return failure in dom0 kernel in below code:
> >
> > vhost_transport_do_send_pkt {
> > ...
> >nbytes = copy_to_iter(hdr, sizeof(*hdr), _iter);
> > if (nbytes != sizeof(*hdr)) {
> > vq_err(vq, "Faulted on copying pkt hdr %x %x %x 
> > %px\n",
> nbytes, sizeof(*hdr),
> > __builtin_object_size(hdr, 0), _iter);
> > kfree_skb(skb);
> > break;
> > }
> > }
> >
> > I checked copy_to_iter, it is copy data to __user addr, but it never
> > pass, the copy to __user addr always return 0 bytes copied.
> >
> > The asm code "sttr x7, [x6]" will trigger data abort, the kernel will
> > run into do_page_fault, but lock_mm_and_find_vma report it is
> > VM_FAULT_BADMAP, that means the __user addr is not mapped, no vma
> has this addr.
> >
> > I am not sure what may cause this. Appreciate if any comments.
> >
> > BTW: I tested blk pci, it works, so the virtio pci should work on my setup.
> >
> > Thanks,
> > Peng.
> >

question about virtio-vsock on xen

[Peng Fan]

Hi All,

Has anyone make virtio-vsock on xen work? My dm args as below:

virtio = [
'backend=0,type=virtio,device,transport=pci,bdf=05:00.0,backend_type=qemu,grant_usage=true'
]
device_model_args = [
'-D', '/home/root/qemu_log.txt',
'-d', 
'trace:*vsock*,trace:*vhost*,trace:*virtio*,trace:*pci_update*,trace:*pci_route*,trace:*handle_ioreq*,trace:*xen*',
'-device', 
'vhost-vsock-pci,iommu_platform=false,id=vhost-vsock-pci0,bus=pcie.0,addr=5.0,guest-cid=3']

During my test, it always return failure in dom0 kernel in below code:

vhost_transport_do_send_pkt {
...
   nbytes = copy_to_iter(hdr, sizeof(*hdr), _iter); 
   
if (nbytes != sizeof(*hdr)) {   

vq_err(vq, "Faulted on copying pkt hdr %x %x %x %px\n", 
nbytes, sizeof(*hdr),
__builtin_object_size(hdr, 0), _iter);
kfree_skb(skb); 

break;  

}  
}

I checked copy_to_iter, it is copy data to __user addr, but it never pass, 
the copy to __user addr always return 0 bytes copied.

The asm code "sttr x7, [x6]" will trigger data abort, the kernel will run
into do_page_fault, but lock_mm_and_find_vma report it is VM_FAULT_BADMAP,
that means the __user addr is not mapped, no vma has this addr.

I am not sure what may cause this. Appreciate if any comments.

BTW: I tested blk pci, it works, so the virtio pci should work on my setup.

Thanks,
Peng.

RE: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue

[Peng Fan]

> -Original Message-
> From: Julien Grall 
> Sent: 2024年2月2日 17:20
> To: John Ernberg ; Stefano Stabellini
> ; Bertrand Marquis ;
> Michal Orzel ; Volodymyr Babchuk
> ; Peng Fan 
> Cc: Jonas Blixt ; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
> 
> On 01/02/2024 16:17, John Ernberg wrote:
> > On 2/1/24 13:20, Julien Grall wrote:
> >>
> >>
> >> On 31/01/2024 15:32, John Ernberg wrote:
> >>> Hi Julien,
> >>
> >> Hi John,
> >>
> >>> On 1/31/24 13:22, Julien Grall wrote:
> >>>> Hi,
> >>>>
> >>>> On 31/01/2024 11:50, John Ernberg wrote:
> >>>>> When using Linux for dom0 there are a bunch of drivers that need
> >>>>> to do SMC SIP calls into the PSCI provider to enable certain
> >>>>> hardware bits like the watchdog.
> >>>>
> >>>> Do you know which protocol this is under the hood. Is this SCMI?
> >>>
> >>> I think I confused myself here when I wrote the commit log.
> >>>
> >>> The EL3 code in our case is ATF, and it does not appear to be SCMI,
> >>> nor PSCI. The register usage of these SMC SIP calls are as follows:
> >>> a0 - service
> >>> a1 - function
> >>> a2-a7 - args
> >>>
> >>> In ATF the handler is declared as a runtime service.
> >>>
> >>> Would the appropriate commmit message here be something along the
> >>> lines of below?
> >>> """
> >>> When using Linux for dom0 there are a bunch of drivers that need to
> >>> do   SMC SIP calls into the firmware to enable certain hardware bits
> >>> like the watchdog.
> >>> """
> >>
> >> It reads better thanks.
> >>
> >> [...]
> >>
> >>>> But even if we restrict to dom0, have you checked that none of the
> >>>> SMCs use buffers?
> >>> I haven't found any such instances in the Linux kernel where a
> >>> buffer is used. Adding a call filtering like suggested below
> >>> additions of such functions can be discovered and adapted for if they
> would show up later.
> >>>>
> >>>> Rather than providing a blanket forward, to me it sounds more like
> >>>> you want to provide an allowlist of the SMCs. This is more
> >>>> futureproof and avoid the risk to expose unsafe SMCs to any domain.
> >>>>
> >>>> For an example, you can have a look at the EEMI mediator for Xilinx.
> >>>
> >>> Ack. Do you prefer to see only on SMCCC service level or also on
> >>> function level? (a1 register, per description earlier)
> >>
> >> I am not sure. It will depend on whether it is correct to expose
> >> *all* the functions within a service level and they have the same format.
> >>
> >> If you can't guarantee that, then you will most likely need to
> >> allowlist at the function level.
> >>
> >> Also, do you have a spec in hand that would help to understand which
> >> service/function is implemented via those SMCs?
> >
> > I don't have the spec unfortunately, but I will add a filter on both
> > service and function for V2 and we'll take it from there.
> 
> @Peng, do you have any specification you could share? How stable is the
> interface?

No specification, the use is IMX_SIP_X in linux kernel source.

Such as IMX_SIP_RTC, IMX_SIP_TIMER

It is stable and no change, we only add new SIP macro if needed
and no change the meaning or reuse old SIP.

Regards,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall

RE: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue

[Peng Fan]

> Cc: Jonas Blixt ; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH 1/2] xen/arm: Add imx8q{m,x} platform glue
> 
> Hi Julien,
> 
> On 1/31/24 13:22, Julien Grall wrote:
> > Hi,
> >
> > On 31/01/2024 11:50, John Ernberg wrote:
> >> When using Linux for dom0 there are a bunch of drivers that need to
> >> do SMC SIP calls into the PSCI provider to enable certain hardware
> >> bits like the watchdog.
> >
> > Do you know which protocol this is under the hood. Is this SCMI?
> 
> I think I confused myself here when I wrote the commit log.
> 
> The EL3 code in our case is ATF, and it does not appear to be SCMI, nor PSCI.
> The register usage of these SMC SIP calls are as follows:
> a0 - service
> a1 - function
> a2-a7 - args
> 
> In ATF the handler is declared as a runtime service.
> 
> Would the appropriate commmit message here be something along the lines
> of below?
> """
> When using Linux for dom0 there are a bunch of drivers that need to do
> SMC
> SIP calls into the firmware to enable certain hardware bits like the watchdog.
> """
> >
> >>
> >> Provide a basic platform glue that implements the needed SMC forwarding.
> >>
> >> Signed-off-by: John Ernberg 
> >> ---
> >> NOTE: This is based on code found in NXP Xen tree located here:
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
> >> hub.com%2Fnxp-imx%2Fimx-xen%2Fblob%2Flf-
> 5.10.y_4.13%2Fxen%2Farch%2Far
> >>
> m%2Fplatforms%2Fimx8qm.c=05%7C02%7Cpeng.fan%40nxp.com%7C
> 573b599a
> >>
> 4b4143ceca1d08dc2271e5be%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C
> 0%7C0%7
> >>
> C638423119777601548%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQI
> >>
> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=ZO
> 0TXjL6
> >> g0W7TIZo8x8lTNBXEZW%2BDNcLPndWlEf5D2A%3D=0
> >
> > Anything after --- will be removed while applied to the three. I think
> > this NOTE should be written down in the commit message.
> 
> Ack.
> >
> > You also possibly want a signed-off-by from Peng as this is his code.
> 
> @Peng: May I add a sign-off from you?

Yeah. You could add my sign off.

> >
> >>
> >>   xen/arch/arm/platforms/Makefile |  1 +
> >>   xen/arch/arm/platforms/imx8qm.c | 65
> >> +
> >>   2 files changed, 66 insertions(+)
> >>   create mode 100644 xen/arch/arm/platforms/imx8qm.c
> >>
> >> diff --git a/xen/arch/arm/platforms/Makefile
> >> b/xen/arch/arm/platforms/Makefile index 8632f4115f..bec6e55d1f
> 100644
> >> --- a/xen/arch/arm/platforms/Makefile
> >> +++ b/xen/arch/arm/platforms/Makefile
> >> @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
> >>   obj-$(CONFIG_ALL64_PLAT) += thunderx.o
> >>   obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
> >>   obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
> >> +obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
> >>   obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
> >>   obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o diff --git
> >> a/xen/arch/arm/platforms/imx8qm.c
> b/xen/arch/arm/platforms/imx8qm.c
> >> new file mode 100644 index 00..a9cd9c3615
> >> --- /dev/null
> >> +++ b/xen/arch/arm/platforms/imx8qm.c
> >> @@ -0,0 +1,65 @@
> >> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> >> +/*
> >> + * xen/arch/arm/platforms/imx8qm.c
> >> + *
> >> + * i.MX 8QM setup
> >> + *
> >> + * Copyright (c) 2016 Freescale Inc.
> >> + * Copyright 2018-2019 NXP
> >> + *
> >> + *
> >> + * Peng Fan 
> >> + */
> >> +
> >> +#include 
> >> +#include 
> >> +
> >> +static const char * const imx8qm_dt_compat[] __initconst = {
> >> +    "fsl,imx8qm",
> >> +    "fsl,imx8qxp",
> >> +    NULL
> >> +};
> >> +
> >> +static bool imx8qm_smc(struct cpu_user_regs *regs) {
> >
> > Your implementation below will not only forward SMC for dom0 but also
> > for any non-trusted domains. Have you investigated that all the SIP
> > calls are safe to be called by anyone?
> 
> We use pure virtualized domUs, so we do not expect any calls to this SMC
> interface from the guest. I'll limit it to dom0.

Would you mind to share what features are supported in your DomU?

Pure virtualized, you using xen pv or virtio?

Thanks,
Peng.

> >
> > But even if we restrict to dom0, have you checked that none of the
> > SMCs use buffers?
> I haven't found any such instances in the Linux kernel where a buffer is used.
> Adding a call filtering like suggested below additions of such functions can 
> be
> discovered and adapted for if they would show up later.
> >
> > Rather than providing a blanket forward, to me it sounds more like you
> > want to provide an allowlist of the SMCs. This is more futureproof and
> > avoid the risk to expose unsafe SMCs to any domain.
> >
> > For an example, you can have a look at the EEMI mediator for Xilinx.
> 
> Ack. Do you prefer to see only on SMCCC service level or also on function
> level? (a1 register, per description earlier)
> >
> > Cheers,
> >
> 
> Thanks! // John Ernberg

RE: [PATCH] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Peng Fan]

> Subject: Re: [PATCH] xen: Drop out of coroutine context
> xen_invalidate_map_cache_entry
> 
> On Tue, 16 Jan 2024, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > xen_invalidate_map_cache_entry is not expected to run in a coroutine.
> > Without this, there is crash:
> >
> > signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
> > threadid=) at pthread_kill.c:78
> > at /usr/src/debug/glibc/2.38+git-r0/sysdeps/posix/raise.c:26
> > fmt=0x9e1ca8a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
> > assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
> > file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-
> lock.c", line=line@entry=260,
> > function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3>
> "bdrv_graph_rdlock_main_loop") at assert.c:92
> > assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
> > file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-
> lock.c", line=line@entry=260,
> > function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3>
> "bdrv_graph_rdlock_main_loop") at assert.c:101
> > at ../qemu-xen-dir-remote/block/graph-lock.c:260
> > at /home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-
> remote/include/block/graph-lock.h:259
> > host=host@entry=0x742c8000, size=size@entry=2097152)
> > at ../qemu-xen-dir-remote/block/io.c:3362
> > host=0x742c8000, size=2097152)
> > at ../qemu-xen-dir-remote/block/block-backend.c:2859
> > host=, size=, max_size=)
> > at ../qemu-xen-dir-remote/block/block-ram-registrar.c:33
> > size=2097152, max_size=2097152)
> > at ../qemu-xen-dir-remote/hw/core/numa.c:883
> > buffer=buffer@entry=0x743c5000 "")
> > at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:475
> > buffer=buffer@entry=0x743c5000 "")
> > at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:487
> > as=as@entry=0xe1ca3ae8 ,
> buffer=0x743c5000,
> > len=, is_write=is_write@entry=true,
> > access_len=access_len@entry=32768)
> > at ../qemu-xen-dir-remote/system/physmem.c:3199
> > dir=DMA_DIRECTION_FROM_DEVICE, len=,
> > buffer=, as=0xe1ca3ae8 )
> > at /home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-
> remote/include/sysemu/dma.h:236
> > elem=elem@entry=0xf620aa30, len=len@entry=32769)
> > at ../qemu-xen-dir-remote/hw/virtio/virtio.c:758
> > elem=elem@entry=0xf620aa30, len=len@entry=32769,
> idx=idx@entry=0)
> > at ../qemu-xen-dir-remote/hw/virtio/virtio.c:919
> > elem=elem@entry=0xf620aa30, len=32769)
> > at ../qemu-xen-dir-remote/hw/virtio/virtio.c:994
> > req=req@entry=0xf620aa30, status=status@entry=0 '\000')
> > at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:67
> > ret=0) at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:136
> > at ../qemu-xen-dir-remote/block/block-backend.c:1559
> > --Type  for more, q to quit, c to continue without paging--
> > at ../qemu-xen-dir-remote/block/block-backend.c:1614
> > i1=) at ../qemu-xen-dir-remote/util/coroutine-
> ucontext.c:177
> > at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123
> >
> > Signed-off-by: Peng Fan 
> 
> Hi Peng! Many thanks for the patch and for the investigation!
> 
> Only one minor question below
> 
> 
> > ---
> >  hw/xen/xen-mapcache.c | 31 +--
> >  1 file changed, 29 insertions(+), 2 deletions(-)
> >
> > diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c index
> > f7d974677d..4e1bb665ee 100644
> > --- a/hw/xen/xen-mapcache.c
> > +++ b/hw/xen/xen-mapcache.c
> > @@ -481,11 +481,38 @@ static void
> xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
> >  g_free(entry);
> >  }
> >
> > -void xen_invalidate_map_cache_entry(uint8_t *buffer)
> > +typedef struct XenMapCacheData {
> > +Coroutine *co;
> > +uint8_t *buffer;
> > +int ret;
> 
> Do we need int ret? It doesn't look like we are using it.

Good catch, it is not needed, I will drop it in V2.

Thanks,
Peng.

> 
> 
> > +} XenMapCacheData;
> > +
> > +static void xen_invalidate_map_cache_entry_bh(void *opaque)
> >  {
> > +XenMapCacheData *data = opaque;
> > +
> >  mapcache_lock();
> > -xen_invalidate_map_cache_entry_unlocked(buffer);
> > +xen_invalidate_map_cache_entry_unlo

[PATCH V2] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Peng Fan (OSS)]

From: Peng Fan 

xen_invalidate_map_cache_entry is not expected to run in a
coroutine. Without this, there is crash:

signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
threadid=) at pthread_kill.c:78
at /usr/src/debug/glibc/2.38+git-r0/sysdeps/posix/raise.c:26
fmt=0x9e1ca8a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:92
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:101
at ../qemu-xen-dir-remote/block/graph-lock.c:260
at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/block/graph-lock.h:259
host=host@entry=0x742c8000, size=size@entry=2097152)
at ../qemu-xen-dir-remote/block/io.c:3362
host=0x742c8000, size=2097152)
at ../qemu-xen-dir-remote/block/block-backend.c:2859
host=, size=, max_size=)
at ../qemu-xen-dir-remote/block/block-ram-registrar.c:33
size=2097152, max_size=2097152)
at ../qemu-xen-dir-remote/hw/core/numa.c:883
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:475
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:487
as=as@entry=0xe1ca3ae8 , buffer=0x743c5000,
len=, is_write=is_write@entry=true,
access_len=access_len@entry=32768)
at ../qemu-xen-dir-remote/system/physmem.c:3199
dir=DMA_DIRECTION_FROM_DEVICE, len=,
buffer=, as=0xe1ca3ae8 )
at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/sysemu/dma.h:236
elem=elem@entry=0xf620aa30, len=len@entry=32769)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:758
elem=elem@entry=0xf620aa30, len=len@entry=32769, idx=idx@entry=0)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:919
elem=elem@entry=0xf620aa30, len=32769)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:994
req=req@entry=0xf620aa30, status=status@entry=0 '\000')
at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:67
ret=0) at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:136
at ../qemu-xen-dir-remote/block/block-backend.c:1559
--Type  for more, q to quit, c to continue without paging--
at ../qemu-xen-dir-remote/block/block-backend.c:1614
i1=) at ../qemu-xen-dir-remote/util/coroutine-ucontext.c:177
at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123

Signed-off-by: Peng Fan 
---

V2:
 Drop unused ret in XenMapCacheData (thanks Stefano)

 hw/xen/xen-mapcache.c | 30 --
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index f7d974677d..8d62b3d2ed 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -481,11 +481,37 @@ static void 
xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
 g_free(entry);
 }
 
-void xen_invalidate_map_cache_entry(uint8_t *buffer)
+typedef struct XenMapCacheData {
+Coroutine *co;
+uint8_t *buffer;
+} XenMapCacheData;
+
+static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
+XenMapCacheData *data = opaque;
+
 mapcache_lock();
-xen_invalidate_map_cache_entry_unlocked(buffer);
+xen_invalidate_map_cache_entry_unlocked(data->buffer);
 mapcache_unlock();
+
+aio_co_wake(data->co);
+}
+
+void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
+{
+if (qemu_in_coroutine()) {
+XenMapCacheData data = {
+.co = qemu_coroutine_self(),
+.buffer = buffer,
+};
+aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
+xen_invalidate_map_cache_entry_bh, );
+qemu_coroutine_yield();
+} else {
+mapcache_lock();
+xen_invalidate_map_cache_entry_unlocked(buffer);
+mapcache_unlock();
+}
 }
 
 void xen_invalidate_map_cache(void)
-- 
2.35.3

[PATCH] xen: Drop out of coroutine context xen_invalidate_map_cache_entry

[Peng Fan (OSS)]

From: Peng Fan 

xen_invalidate_map_cache_entry is not expected to run in a
coroutine. Without this, there is crash:

signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
threadid=) at pthread_kill.c:78
at /usr/src/debug/glibc/2.38+git-r0/sysdeps/posix/raise.c:26
fmt=0x9e1ca8a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:92
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:101
at ../qemu-xen-dir-remote/block/graph-lock.c:260
at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/block/graph-lock.h:259
host=host@entry=0x742c8000, size=size@entry=2097152)
at ../qemu-xen-dir-remote/block/io.c:3362
host=0x742c8000, size=2097152)
at ../qemu-xen-dir-remote/block/block-backend.c:2859
host=, size=, max_size=)
at ../qemu-xen-dir-remote/block/block-ram-registrar.c:33
size=2097152, max_size=2097152)
at ../qemu-xen-dir-remote/hw/core/numa.c:883
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:475
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:487
as=as@entry=0xe1ca3ae8 , buffer=0x743c5000,
len=, is_write=is_write@entry=true,
access_len=access_len@entry=32768)
at ../qemu-xen-dir-remote/system/physmem.c:3199
dir=DMA_DIRECTION_FROM_DEVICE, len=,
buffer=, as=0xe1ca3ae8 )
at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/sysemu/dma.h:236
elem=elem@entry=0xf620aa30, len=len@entry=32769)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:758
elem=elem@entry=0xf620aa30, len=len@entry=32769, idx=idx@entry=0)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:919
elem=elem@entry=0xf620aa30, len=32769)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:994
req=req@entry=0xf620aa30, status=status@entry=0 '\000')
at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:67
ret=0) at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:136
at ../qemu-xen-dir-remote/block/block-backend.c:1559
--Type  for more, q to quit, c to continue without paging--
at ../qemu-xen-dir-remote/block/block-backend.c:1614
i1=) at ../qemu-xen-dir-remote/util/coroutine-ucontext.c:177
at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123

Signed-off-by: Peng Fan 
---
 hw/xen/xen-mapcache.c | 31 +--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/hw/xen/xen-mapcache.c b/hw/xen/xen-mapcache.c
index f7d974677d..4e1bb665ee 100644
--- a/hw/xen/xen-mapcache.c
+++ b/hw/xen/xen-mapcache.c
@@ -481,11 +481,38 @@ static void 
xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer)
 g_free(entry);
 }
 
-void xen_invalidate_map_cache_entry(uint8_t *buffer)
+typedef struct XenMapCacheData {
+Coroutine *co;
+uint8_t *buffer;
+int ret;
+} XenMapCacheData;
+
+static void xen_invalidate_map_cache_entry_bh(void *opaque)
 {
+XenMapCacheData *data = opaque;
+
 mapcache_lock();
-xen_invalidate_map_cache_entry_unlocked(buffer);
+xen_invalidate_map_cache_entry_unlocked(data->buffer);
 mapcache_unlock();
+
+aio_co_wake(data->co);
+}
+
+void coroutine_mixed_fn xen_invalidate_map_cache_entry(uint8_t *buffer)
+{
+if (qemu_in_coroutine()) {
+XenMapCacheData data = {
+.co = qemu_coroutine_self(),
+.buffer = buffer,
+};
+aio_bh_schedule_oneshot(qemu_get_current_aio_context(),
+xen_invalidate_map_cache_entry_bh, );
+qemu_coroutine_yield();
+} else {
+mapcache_lock();
+xen_invalidate_map_cache_entry_unlocked(buffer);
+mapcache_unlock();
+}
 }
 
 void xen_invalidate_map_cache(void)
-- 
2.35.3

!qemu_in_coroutine() assert on ARM64 XEN

[Peng Fan]

Hi All,

When enabling virtio disk and virtio net on Xen, I could see qemu blk assert
and being killed sometimes,  This is not 100% reproducible. I am using
qemu master branch

7425b6277f12e82952cede1f531bfc689bf77fb1 (HEAD -> dummy, origin/staging, 
origin/master, origin/HEAD, master) Merge tag 'tracing-pull-request' 
of https://gitlab.com/stefanha/qemu into staging

The qemu built option is the one in xen tool/Makefile, I just
change to qemu-system-aarch64.

Anyone has suggestions?

The coredump stack:

Symbols already loaded for /usr/lib/libc.so.6
(gdb) bt
#0  __pthread_kill_implementation (threadid=,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x9e100568 in __pthread_kill_internal (signo=6,
threadid=) at pthread_kill.c:78
#2  0x9e0bacd0 in __GI_raise (sig=sig@entry=6)
at /usr/src/debug/glibc/2.38+git-r0/sysdeps/posix/raise.c:26
#3  0x9e0a6ef0 in __GI_abort () at abort.c:79
#4  0x9e0b43f8 in __assert_fail_base (
fmt=0x9e1ca8a8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:92
#5  0x9e0b4470 in __assert_fail (
assertion=assertion@entry=0xe0d25740 "!qemu_in_coroutine()",
file=file@entry=0xe0d301a8 "../qemu-xen-dir-remote/block/graph-lock.c", 
line=line@entry=260,
function=function@entry=0xe0e522c0 <__PRETTY_FUNCTION__.3> 
"bdrv_graph_rdlock_main_loop") at assert.c:101
#6  0xe0a66a60 in bdrv_graph_rdlock_main_loop ()
at ../qemu-xen-dir-remote/block/graph-lock.c:260
#7  0xe0a6d9e0 in graph_lockable_auto_lock_mainloop (x=)
--Type  for more, q to quit, c to continue without paging--
at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/block/graph-lock.h:259
#8  bdrv_unregister_buf (bs=bs@entry=0xf619d5a0,
host=host@entry=0x742c8000, size=size@entry=2097152)
at ../qemu-xen-dir-remote/block/io.c:3362
#9  0xe0a5ddd4 in blk_unregister_buf (blk=,
host=0x742c8000, size=2097152)
at ../qemu-xen-dir-remote/block/block-backend.c:2859
#10 0xe060aab4 in ram_block_removed (n=,
host=, size=, max_size=)
at ../qemu-xen-dir-remote/block/block-ram-registrar.c:33
#11 0xe0399318 in ram_block_notify_remove (host=0x742c8000,
size=2097152, max_size=2097152)
at ../qemu-xen-dir-remote/hw/core/numa.c:883
#12 0xe097cf84 in xen_invalidate_map_cache_entry_unlocked (
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:475
#13 0xe097dad0 in xen_invalidate_map_cache_entry (
buffer=buffer@entry=0x743c5000 "")
at ../qemu-xen-dir-remote/hw/xen/xen-mapcache.c:487
#14 0xe0993e18 in address_space_unmap (
as=as@entry=0xe1ca3ae8 , buffer=0x743c5000,
len=, is_write=is_write@entry=true,
--Type  for more, q to quit, c to continue without paging--
access_len=access_len@entry=32768)
at ../qemu-xen-dir-remote/system/physmem.c:3199
#15 0xe095cc9c in dma_memory_unmap (access_len=32768,
dir=DMA_DIRECTION_FROM_DEVICE, len=,
buffer=, as=0xe1ca3ae8 )

at 
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/include/sysemu/dma.h:236
#16 virtqueue_unmap_sg (vq=vq@entry=0x965cc010,
elem=elem@entry=0xf620aa30, len=len@entry=32769)

at ../qemu-xen-dir-remote/hw/virtio/virtio.c:758
#17 0xe095efa4 in virtqueue_fill (vq=vq@entry=0x965cc010,
elem=elem@entry=0xf620aa30, len=len@entry=32769, idx=idx@entry=0)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:919
#18 0xe095f0b8 in virtqueue_push (vq=0x965cc010,

elem=elem@entry=0xf620aa30, len=32769)
at ../qemu-xen-dir-remote/hw/virtio/virtio.c:994
#19 0xe091a608 in virtio_blk_req_complete (
req=req@entry=0xf620aa30, status=status@entry=0 '\000')

at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:67
#20 0xe091bdc8 in virtio_blk_rw_complete (opaque=,
ret=0) at ../qemu-xen-dir-remote/hw/block/virtio-blk.c:136
#21 0xe0a5a938 in blk_aio_complete (acb=acb@entry=0x880015f0)

at ../qemu-xen-dir-remote/block/block-backend.c:1559
--Type  for more, q to quit, c to continue without paging--
#22 0xe0a5b58c in blk_aio_read_entry (opaque=0x880015f0)
at ../qemu-xen-dir-remote/block/block-backend.c:1614

#23 0xe0b96c2c in coroutine_trampoline (i0=,
i1=) at ../qemu-xen-dir-remote/util/coroutine-ucontext.c:177
#24 0x9e0bfb40 in ?? ()
at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123

   from /usr/lib/libc.so.6

(gdb) thread apply all bt

Thread 10 (Thread 0x951348c0 (LWP 5460)):
#0  0x9e15d8c4 in __GI___libc_read (nbytes=16, 

RE: Xen and Qemu virtio question

[Peng Fan]

Hi Vikram,

> Subject: Re: Xen and Qemu virtio question
> 
> Hi Peng,
> Please see my comments below.
> On Wed, Jan 03, 2024 at 05:38:07AM +, Peng Fan wrote:
> > Hi Vikram, Oleksii
> >
> > I follow the slide virtio for Xen on ARM[1], but I met some issues,
> > and stuck for about two days.
> >
> > I use upstream lastest qemu repo master branch, not qemu-xen.git repo.
> This is right.
> >
> > My start command as below[2], but seems I need first `xl create
> > domu.cfg`, otherwise it will fail, because xen hypervisor
> > dm_op->rcu_lock_remote_domain_by_id will return failure if domu not
> > created.  My domu cfg is simple:
> > kernel = "/run/media/boot-mmcblk1p1/Image"
> > disk = [ '/etc/xen/disk_1.img,,xvda,specification=virtio' ] cmdline =
> > "console=hvc0 root=/dev/xvda rw"
> > name = "DomU"
> > memory = 512
> > serial = "pty"
> >
> > I drop qdisk in the disk entry, because xen tool report qdisk and
> > virtio not compatible.
> >
> > And the arg xen_domid should be same as domu domain id? Is there any
> > dynamic way to set xen_domid, start qemu when start domu?
> Yes, it should be xen_domid. Below virtio-disk patches will take care of
> invoking QEMU with right domid.
> >
> > Does the domu dts still needed to include the virtio,mmio node?
> >
> > however xl create domu.cfg relies virtio disk ready, which needs qemu
> > start first. This confuses me. How do you address this or I do
> > something wrong?
> >
> > Then after xl create,  I quickly start qemu, but met:
> > failed to create ioreq server, then I see there is mismatch.
> > qemu use HVM_IOREQSRV_BUFIOREQ_ATOMIC to create ioreq server, but
> xen
> > ioreq_server_create will return failure:
> >  if ( !IS_ENABLED(CONFIG_X86) && bufioreq_handling )
> >return -EINVAL;
> >
> There is a downstream patch for this:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2FXilinx%2Fxen%2Fcommit%2Fbe35b46e907c7c78fd23888d837475e
> b28334638=05%7C02%7Cpeng.fan%40nxp.com%7C52a0d96c7d95474
> c016408dc0c9cbc2e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%
> 7C638399114509848564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%
> 7C%7C=KylGkFrL2hOazsLjFwwj%2FIAPDQU80LhFfONQL4ytaic%3D
> served=0
> 
> > So I change HVM_IOREQSRV_BUFIOREQ_OFF in qemu, but met:
> > qemu-system-aarch64: failed to map ioreq server resources: error 2
> > handle=0xd26c7f30
> > qemu-system-aarch64: xen hardware virtual machine initialisation
> > failed
> >
> > Do you have out of box repo that works well? Including Qemu and Xen, I
> > am trying virtio disk, but will move to test virtio gpu soon.
> >
> > I am not sure there are some downstream patches in your side to
> > address and make it work well.
> >
> We have a few downstream patches for xen-tools.
> For Virtio disk backend:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2FXilinx%2Fxen%2Fcommit%2F947280803294bbb963f428423f679d0
> 74c60d632=05%7C02%7Cpeng.fan%40nxp.com%7C52a0d96c7d95474
> c016408dc0c9cbc2e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%
> 7C638399114509848564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%
> 7C%7C=%2Bfjlzz%2Bvut3WyRWARfCij03kJ5ZuiGIika8uK5MIdM8%3D&
> reserved=0
> For Virtio-net:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2FXilinx%2Fxen%2Fcommit%2F32fcc702718591270e5c8928b7687d8
> 53249c882=05%7C02%7Cpeng.fan%40nxp.com%7C52a0d96c7d95474
> c016408dc0c9cbc2e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%
> 7C638399114509848564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%
> 7C%7C=BLtX2GbY77J65H6mBF1r3nXkCWN%2B0SfRJTkH1CnTiQE%3D
> =0
> For changing the machine name to Xenpvh(to align with QEMU changes):
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
> .com%2FXilinx%2Fxen%2Fcommit%2F5f669949c9ffdb1947cb47038956b5fb
> 8eeb072a=05%7C02%7Cpeng.fan%40nxp.com%7C52a0d96c7d95474c
> 016408dc0c9cbc2e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7
> C638399114509848564%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7
> C%7C=FbmQpeyrc86qapDK8gDNr7o0fYj09t6RrBA3RdmR4EQ%3D
> erved=0
> 
> With above 4 patches it should work for you. Please re-compile Xen and Xen-
> tools first and finally compile QEMU with this Xen.

Thank

Xen and Qemu virtio question

[Peng Fan]

Hi Vikram, Oleksii

I follow the slide virtio for Xen on ARM[1], but I met some issues, and
stuck for about two days.

I use upstream lastest qemu repo master branch, not qemu-xen.git
repo.

My start command as below[2], but seems I need first `xl create domu.cfg`,
otherwise it will fail, because xen hypervisor
dm_op->rcu_lock_remote_domain_by_id will return failure if domu
not created.  My domu cfg is simple:
kernel = "/run/media/boot-mmcblk1p1/Image"
disk = [ '/etc/xen/disk_1.img,,xvda,specification=virtio' ]
cmdline = "console=hvc0 root=/dev/xvda rw"
name = "DomU"
memory = 512
serial = "pty"

I drop qdisk in the disk entry, because xen tool report qdisk and virtio
not compatible.

And the arg xen_domid should be same as domu domain id? Is
there any dynamic way to set xen_domid, start qemu when start
domu?

Does the domu dts still needed to include the virtio,mmio node?

however xl create domu.cfg relies virtio disk ready, which needs qemu
start first. This confuses me. How do you address this
or I do something wrong?

Then after xl create,  I quickly start qemu, but met:
failed to create ioreq server, then I see there is mismatch.
qemu use HVM_IOREQSRV_BUFIOREQ_ATOMIC to create ioreq server,
but xen ioreq_server_create will return failure:
 if ( !IS_ENABLED(CONFIG_X86) && bufioreq_handling )
 
   return -EINVAL;

So I change HVM_IOREQSRV_BUFIOREQ_OFF in qemu, but met:
qemu-system-aarch64: failed to map ioreq server resources: error 2
handle=0xd26c7f30
qemu-system-aarch64: xen hardware virtual machine initialisation failed

Do you have out of box repo that works well? Including Qemu and Xen,
I am trying virtio disk, but will move to test virtio gpu soon.

I am not sure there are some downstream patches in your side to
address and make it work well.

Thanks for your help.

Thanks,
Peng.

[1]https://www.youtube.com/watch?v=boRQ8UHc760

[2]qemu-system-aarch64  -xen-domid 1 \
-chardev socket,id=libxl-cmd,path=qmp-libxl-1,server=on,wait=off \
-mon chardev=libxl-cmd,mode=control \
-chardev socket,id=libxenstat-cmd,path=qmp-libxenstat-1,server=on,wait=off \
-mon chardev=libxenstat-cmd,mode=control \
-xen-attach -name guest0 -vnc none -display none -nographic \
-machine xenpvh -m 513 &

XEN virtio question

[Peng Fan]

All,

I am trying to setup xen virtio on NXP i.MX9, but when build xen tools,
I always met qemu build error, such as:

In file included from ../qemu-xen-dir-remote/hw/xen/xen-operations.c:16:
/home/Freenix/work/sw-stash/xen/upstream/tools/qemu-xen-dir-remote/
include/hw/xen/xen_native.h:5:2: error: #error In Xen native files, 
include xen_native.h before other Xen headers
5 | #error In Xen native files, include xen_native.h before other Xen 
headers
  |  ^

Anyone met this?

BTW, do u build qemu together with xen or build standalone qemu out of
xen repo? Does the default qemu-system-i386 built out works well
for virtio trying?

Thanks,
Peng.

RE: [Xen-devel] SMMU permission fault on Dom0 when init vpu_decoder

[Peng Fan]

> Subject: Re: [Xen-devel] SMMU permission fault on Dom0 when init
> vpu_decoder
> 
> (+ Stefano)
> 
> On 30/05/2022 16:21, Oleksii Moisieiev wrote:
> > Hello,
> 
> Hi Oleksii,
> 
> > I'm getting permission fault from SMMU when trying to init
> > VPU_Encoder/Decoder in Dom0 on IMX8QM board:
> > (XEN) smmu: /iommu@5140: Unhandled context fault: fsr=0x408,
> > iova=0x86000a60, fsynr=0x1c0062, cb=0 This error appears when
> > vpu_encoder/decoder tries to memcpy firmware image to
> > 0x8600 address, which is defined in reserved-memory node in xen
> > device-tree as encoder_boot/decoder_boot region.
> 
> It is not clear to me who is executing the memcpy(). Is it the device or your
> domain? If the former, where was the instruction fetch from?
> 
> The reason I am asking that is, from what you wrote, mempcy() will write to
> 0x8600. So the write should not result to a instruction abort.
> Only an instruction fetch would lead to such abort.
> 
> >
> > I'm using xen from branch xen-project/staging-4.16 + imx related
> > patches, which were taken from
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsource.c
> odeaurora.org%2Fexternal%2Fimx%2Fimx-xendata=05%7C01%7Cpeng.f
> an%40nxp.com%7C726bfc45b45747a655bc08da425351d4%7C686ea1d3bc2b
> 4c6fa92cd99c5c301635%7C0%7C0%7C637895222868692087%7CUnknown%
> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> CJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=KcbEjN80VJS5yq4KMp2g
> NjQuVtx95jKHH5T32ZCj3Do%3Dreserved=0.
> >
> > After some investigation I found that this issue was fixed by Peng Fan
> > in
> > commit: 46b3dd3718144ca6ac2c12a3b106e57fb7156554 (Hash from
> > codeaurora), but only for the Guest domains.
> > It introduces new p2m_type p2m_mmio_direct_nc_x, which differs from
> > p2m_mmio_direct_nc by XN = 0. This type is set to the reserved memory
> > region in map_mmio_regions function.
> >
> > I was able to fix issue in Dom0 by setting p2m_mmio_direct_nc_x type
> > for the reserved memory in map_regions_p2mt, which is used to map
> memory during Dom0 creation.
> > Patch can be found below.
> >
> > Based on initial discussions on IRC channel - XN bit did the trick
> > because looks like vpu decoder is executing some code from this memory.
> 
> This was a surprise to me that device could also execute memory. From the
> SMMU spec, this looks a legit things. Before relaxing the type, I would like 
> to
> confirm this is what's happenning in your case.

It is not device, VPU could execute code, it has firmware. Just like use
SMMU to isolate M4-core, M4 could execute code.

Regards,
Peng.

> 
> [...]
> 
> > ---
> > arm: Set p2m_type to p2m_mmio_direct_nc_x for reserved memory regions
> >
> > This is the enhancement of the
> 46b3dd3718144ca6ac2c12a3b106e57fb7156554.
> > Those patch introduces p2m_mmio_direct_nc_x p2m type which sets the
> > e->p2m.xn = 0 for the reserved-memory, such as vpu encoder/decoder.
> >
> > Set p2m_mmio_direct_nc_x in map_regions_p2mt for reserved-memory the
> > same way it does in map_mmio_regions. This change is for the case when
> > vpu encoder/decoder works in DomO and not passed-through to the Guest
> > Domains.
> >
> > Signed-off-by: Oleksii Moisieiev 
> > ---
> >   xen/arch/arm/p2m.c | 7 +++
> >   1 file changed, 7 insertions(+)
> >
> > diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index
> > e9568dab88..bb1f681b71 100644
> > --- a/xen/arch/arm/p2m.c
> > +++ b/xen/arch/arm/p2m.c
> > @@ -1333,6 +1333,13 @@ int map_regions_p2mt(struct domain *d,
> >mfn_t mfn,
> >p2m_type_t p2mt)
> >   {
> > +if (((long)gfn_x(gfn) >= (GUEST_RAM0_BASE >> PAGE_SHIFT)) &&
> > +(((long)gfn_x(gfn) + nr) <=
> > +((GUEST_RAM0_BASE + GUEST_RAM0_SIZE)>> PAGE_SHIFT)))
> 
> I am afraid I don't understand what this check is for. In a normal setup, we 
> don't
> know where the reserved regions are mapped. Only the caller may know that.
> 
> For dom0, this decision could be taken in map_range_to_domain(). For the
> domU, we would need to let the toolstack to chose the memory attribute.
> Stefano attempted to do that a few years ago (see [1]). Maybe we should revive
> it?
> 
> > +{
> > +p2m_remove_mapping(d, gfn, nr, mfn);
> > +return p2m_insert_mapping(d, gfn, nr, mfn,
> p2m_mmio_direct_nc_x);
> > +}
> >   return p2m_insert_mapping(d, gfn, nr, mfn, p2mt);
> >   }
> >
> 
> Cheers,
> 
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.ker
> nel.org%2Fxen-devel%2Falpine.DEB.2.10.1902261501020.20689%40sstabellini
> -ThinkPad-X260%2Fdata=05%7C01%7Cpeng.fan%40nxp.com%7C726bfc
> 45b45747a655bc08da425351d4%7C686ea1d3bc2b4c6fa92cd99c5c301635%
> 7C0%7C0%7C637895222868692087%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi
> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000
> %7C%7C%7Csdata=Y%2B%2Fxslm6wc38jM6zZWshRnTM4tx%2BMpyXaI
> r5BLV1R9Q%3Dreserved=0
> 
> --
> Julien Gral

RE: [Xen-devel] SMMU permission fault on Dom0 when init vpu_decoder

[Peng Fan]

> Subject: Re: [Xen-devel] SMMU permission fault on Dom0 when init
> vpu_decoder
> 
> On Wed, Jun 01, 2022 at 07:59:23AM +0000, Peng Fan wrote:
> > > Subject: [Xen-devel] SMMU permission fault on Dom0 when init
> > > vpu_decoder
> > >
> > > Hello,
> > >
> > > I'm getting permission fault from SMMU when trying to init
> > > VPU_Encoder/Decoder in Dom0 on IMX8QM board:
> > > (XEN) smmu: /iommu@5140: Unhandled context fault: fsr=0x408,
> > > iova=0x86000a60, fsynr=0x1c0062, cb=0 This error appears when
> > > vpu_encoder/decoder tries to memcpy firmware image to
> > > 0x8600 address, which is defined in reserved-memory node in xen
> > > device-tree as encoder_boot/decoder_boot region.
> > >
> > > I'm using xen from branch xen-project/staging-4.16 + imx related
> > > patches, which were taken from
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fur
> > >
> ldefense.com%2Fv3%2F__https%3A%2F%2Feur01.safelinks.protection.outlo
> > >
> ok.com%2F%3Furl%3Dhttps*3A*2F*2Fsource.c__%3BJSUl!!GF_29dbcQIUBPA!
> wz
> > >
> oDdJsuf4bjXMe85tA46E0tLpFG5gqHoo-OeY6o_ARroNBmX7JByHW67qEUik7bN
> ow0ST
> > >
> gvAjR4rBkRu2Ux%24data=05%7C01%7Cpeng.fan%40nxp.com%7C5abe5
> 7eece
> > >
> 404f6c017808da43aef8f7%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7
> C0%7C
> > >
> 637896716019179992%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> DAiLCJQI
> > >
> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csd
> ata=
> > >
> 47ddyB8JUz8sDHcXluhcB7RJ7bH4a33l%2FF2XzUpAPxY%3Dreserved=0
> > > [eur01[.]safelinks[.]protection[.]outlook[.]com]
> > >
> odeaurora.org%2Fexternal%2Fimx%2Fimx-xendata=05%7C01%7Cpeng.f
> > >
> an%40nxp.com%7C91e3a953942d414dcc6208da425006e7%7C686ea1d3bc2b
> > >
> 4c6fa92cd99c5c301635%7C0%7C0%7C637895208732114203%7CUnknown%
> > >
> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> > >
> CJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=no%2BV2ubjGmrsm96NP
> > > ybeeug4a3BXx3oX7xmylzZCU8E%3Dreserved=0.
> > >
> > > After some investigation I found that this issue was fixed by Peng
> > > Fan in
> > > commit: 46b3dd3718144ca6ac2c12a3b106e57fb7156554 (Hash from
> > > codeaurora), but only for the Guest domains.
> > > It introduces new p2m_type p2m_mmio_direct_nc_x, which differs from
> > > p2m_mmio_direct_nc by XN = 0. This type is set to the reserved
> > > memory region in map_mmio_regions function.
> > >
> > > I was able to fix issue in Dom0 by setting p2m_mmio_direct_nc_x type
> > > for the reserved memory in map_regions_p2mt, which is used to map
> > > memory during
> > > Dom0 creation.
> > > Patch can be found below.
> > >
> > > Based on initial discussions on IRC channel - XN bit did the trick
> > > because looks like vpu decoder is executing some code from this memory.
> > >
> > > The purpose of this email is to discuss this issue and probably
> > > produce generic solution for it.
> > >
> > > Best regards,
> > > Oleksii.
> > >
> > > ---
> > > arm: Set p2m_type to p2m_mmio_direct_nc_x for reserved memory
> > > regions
> > >
> > > This is the enhancement of the
> > > 46b3dd3718144ca6ac2c12a3b106e57fb7156554.
> > > Those patch introduces p2m_mmio_direct_nc_x p2m type which sets the
> > > e->p2m.xn = 0 for the reserved-memory, such as vpu encoder/decoder.
> > >
> > > Set p2m_mmio_direct_nc_x in map_regions_p2mt for reserved-memory the
> > > same way it does in map_mmio_regions. This change is for the case
> > > when vpu encoder/decoder works in DomO and not passed-through to the
> > > Guest Domains.
> >
> > For Dom0, there is no SMMU, so no need x. Just nc is enough.
> >
> > Regards,
> > Peng.
> 
> I would say that SMMU is not neccessary for Dom0 because it's mapped 1:1.
> But using device under SMMU in Dom0 is still valid case. For example to 
> protect
> device from reaching address, assigned to another domain, since Dom0 is
> trusted.

I mean the reason to use nc_x is that VPU DomU is accessing DRAM through SMMU.
It needs X because there is VPU firmware run in DomU.

I not see why need X for Dom0, unless you assign a SID for VPU and create SMMU
mapping for VPU in Dom0.

Regards,
Peng.

> 
> >
> > >
> > > Signed-off-by: Oleksii Moisieiev 
> > > ---
> > >  xen/arch/arm/p2m.c | 7 +++
> > >  1 file changed, 7 insertions(+)
> > >
> > > diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index
> > > e9568dab88..bb1f681b71 100644
> > > --- a/xen/arch/arm/p2m.c
> > > +++ b/xen/arch/arm/p2m.c
> > > @@ -1333,6 +1333,13 @@ int map_regions_p2mt(struct domain *d,
> > >   mfn_t mfn,
> > >   p2m_type_t p2mt)  {
> > > +if (((long)gfn_x(gfn) >= (GUEST_RAM0_BASE >> PAGE_SHIFT)) &&
> > > +(((long)gfn_x(gfn) + nr) <=
> > > +((GUEST_RAM0_BASE + GUEST_RAM0_SIZE)>> PAGE_SHIFT)))
> > > +{
> > > +p2m_remove_mapping(d, gfn, nr, mfn);
> > > +return p2m_insert_mapping(d, gfn, nr, mfn,
> > > p2m_mmio_direct_nc_x);
> > > +}
> > >  return p2m_insert_mapping(d, gfn, nr, mfn, p2mt);  }
> > >
> > > --
> > > 2.27.0

RE: [Xen-devel] SMMU permission fault on Dom0 when init vpu_decoder

[Peng Fan]

> Subject: [Xen-devel] SMMU permission fault on Dom0 when init vpu_decoder
> 
> Hello,
> 
> I'm getting permission fault from SMMU when trying to init
> VPU_Encoder/Decoder in Dom0 on IMX8QM board:
> (XEN) smmu: /iommu@5140: Unhandled context fault: fsr=0x408,
> iova=0x86000a60, fsynr=0x1c0062, cb=0 This error appears when
> vpu_encoder/decoder tries to memcpy firmware image to
> 0x8600 address, which is defined in reserved-memory node in xen
> device-tree as encoder_boot/decoder_boot region.
> 
> I'm using xen from branch xen-project/staging-4.16 + imx related patches,
> which were taken from
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsource.c
> odeaurora.org%2Fexternal%2Fimx%2Fimx-xendata=05%7C01%7Cpeng.f
> an%40nxp.com%7C91e3a953942d414dcc6208da425006e7%7C686ea1d3bc2b
> 4c6fa92cd99c5c301635%7C0%7C0%7C637895208732114203%7CUnknown%
> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
> CJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=no%2BV2ubjGmrsm96NP
> ybeeug4a3BXx3oX7xmylzZCU8E%3Dreserved=0.
> 
> After some investigation I found that this issue was fixed by Peng Fan in
> commit: 46b3dd3718144ca6ac2c12a3b106e57fb7156554 (Hash from
> codeaurora), but only for the Guest domains.
> It introduces new p2m_type p2m_mmio_direct_nc_x, which differs from
> p2m_mmio_direct_nc by XN = 0. This type is set to the reserved memory region
> in map_mmio_regions function.
> 
> I was able to fix issue in Dom0 by setting p2m_mmio_direct_nc_x type for the
> reserved memory in map_regions_p2mt, which is used to map memory during
> Dom0 creation.
> Patch can be found below.
> 
> Based on initial discussions on IRC channel - XN bit did the trick because 
> looks
> like vpu decoder is executing some code from this memory.
> 
> The purpose of this email is to discuss this issue and probably produce 
> generic
> solution for it.
> 
> Best regards,
> Oleksii.
> 
> ---
> arm: Set p2m_type to p2m_mmio_direct_nc_x for reserved memory regions
> 
> This is the enhancement of the
> 46b3dd3718144ca6ac2c12a3b106e57fb7156554.
> Those patch introduces p2m_mmio_direct_nc_x p2m type which sets the
> e->p2m.xn = 0 for the reserved-memory, such as vpu encoder/decoder.
> 
> Set p2m_mmio_direct_nc_x in map_regions_p2mt for reserved-memory the
> same way it does in map_mmio_regions. This change is for the case when vpu
> encoder/decoder works in DomO and not passed-through to the Guest
> Domains.

For Dom0, there is no SMMU, so no need x. Just nc is enough.

Regards,
Peng.

> 
> Signed-off-by: Oleksii Moisieiev 
> ---
>  xen/arch/arm/p2m.c | 7 +++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index
> e9568dab88..bb1f681b71 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -1333,6 +1333,13 @@ int map_regions_p2mt(struct domain *d,
>   mfn_t mfn,
>   p2m_type_t p2mt)
>  {
> +if (((long)gfn_x(gfn) >= (GUEST_RAM0_BASE >> PAGE_SHIFT)) &&
> +(((long)gfn_x(gfn) + nr) <=
> +((GUEST_RAM0_BASE + GUEST_RAM0_SIZE)>> PAGE_SHIFT)))
> +{
> +p2m_remove_mapping(d, gfn, nr, mfn);
> +return p2m_insert_mapping(d, gfn, nr, mfn,
> p2m_mmio_direct_nc_x);
> +}
>  return p2m_insert_mapping(d, gfn, nr, mfn, p2mt);  }
> 
> --
> 2.27.0

[PATCH V6 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

The i.MX LPUART Documentation:
https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)

Tested-by: Henry Wang 
Signed-off-by: Peng Fan 
---
 xen/arch/arm/include/asm/imx-lpuart.h |  64 ++
 xen/drivers/char/Kconfig  |   7 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 275 ++
 4 files changed, 347 insertions(+)
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

diff --git a/xen/arch/arm/include/asm/imx-lpuart.h 
b/xen/arch/arm/include/asm/imx-lpuart.h
new file mode 100644
index 00..fe859045dc
--- /dev/null
+++ b/xen/arch/arm/include/asm/imx-lpuart.h
@@ -0,0 +1,64 @@
+/*
+ * xen/arch/arm/include/asm/imx-lpuart.h
+ *
+ * Common constant definition between early printk and the LPUART driver
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_IMX_LPUART_H__
+#define __ASM_ARM_IMX_LPUART_H__
+
+/* 32-bit register definition */
+#define UARTBAUD  (0x10)
+#define UARTSTAT  (0x14)
+#define UARTCTRL  (0x18)
+#define UARTDATA  (0x1C)
+#define UARTMATCH (0x20)
+#define UARTMODIR (0x24)
+#define UARTFIFO  (0x28)
+#define UARTWATER (0x2c)
+
+#define UARTSTAT_TDRE BIT(23, UL)
+#define UARTSTAT_TC   BIT(22, UL)
+#define UARTSTAT_RDRF BIT(21, UL)
+#define UARTSTAT_OR   BIT(19, UL)
+
+#define UARTBAUD_OSR_SHIFT(24)
+#define UARTBAUD_OSR_MASK (0x1f)
+#define UARTBAUD_SBR_MASK (0x1fff)
+#define UARTBAUD_BOTHEDGE (0x0002)
+#define UARTBAUD_TDMAE(0x0080)
+#define UARTBAUD_RDMAE(0x0020)
+
+#define UARTCTRL_TIE  BIT(23, UL)
+#define UARTCTRL_TCIE BIT(22, UL)
+#define UARTCTRL_RIE  BIT(21, UL)
+#define UARTCTRL_ILIE BIT(20, UL)
+#define UARTCTRL_TE   BIT(19, UL)
+#define UARTCTRL_RE   BIT(18, UL)
+#define UARTCTRL_MBIT(4, UL)
+
+#define UARTWATER_RXCNT_OFF 24
+
+#endif /* __ASM_ARM_IMX_LPUART_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..e5f7b1d8eb 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,13 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have i.MX8QM based board, say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..2709136081
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,275 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+uint32_t baud, clock_hz, dat

[PATCH V6 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 14 
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 48 +
 2 files changed, 62 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..842d768280 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -58,6 +58,16 @@ choice
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
config EARLY_UART_CHOICE_MESON
select EARLY_UART_MESON
depends on ARM_64
@@ -186,6 +196,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +296,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..b169332932
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,48 @@
+/*
+ * xen/arch/arm/arm64/debug-imx-lpuart.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+/*
+ * Wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/*
+ * LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.35.1

[PATCH V6 0/2] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

V6:
 Fix a stale variable check per Stefano's comments.

V5:
 Align code
 drop early_uart_init

V4:
 Wrong v3 version, some BIT definition are mixed in patch 1,2. 

V3:
 Addressed Michal's comments.
 Add Henry's T-b

V2:
 Per Julien's comments, fix coding style issue, drop unneeded code

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.

Peng Fan (2):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support

 xen/arch/arm/Kconfig.debug  |  14 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  48 +
 xen/arch/arm/include/asm/imx-lpuart.h   |  64 ++
 xen/drivers/char/Kconfig|   7 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 275 
 6 files changed, 409 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

-- 
2.35.1

RE: [PATCH V5 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan]

> Subject: Re: [PATCH V5 1/2] xen/arm: Add i.MX lpuart driver
> 
> On Thu, 14 Apr 2022, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > The i.MX LPUART Documentation:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> >
> nxp.com%2Fwebapp%2FDownload%3FcolCode%3DIMX8QMIECdata=04
> %7C01%7Cp
> >
> eng.fan%40nxp.com%7Caf595d02a1dd4ed5825b08da1f2f0e78%7C686ea1d3bc
> 2b4c6
> >
> fa92cd99c5c301635%7C0%7C0%7C637856584221931187%7CUnknown%7CTW
> FpbGZsb3d
> >
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> %7C
> >
> 3000sdata=3N2TsMqzhHF5dem1Cyeo2HVdgfTJFnPHSoMW7KrNph8%3D
> rese
> > rved=0 Chatper 13.6 Low Power Universal Asynchronous Receiver/
> > Transmitter (LPUART)
> >
> > Tested-by: Henry Wang 
> > Signed-off-by: Peng Fan 
> > ---
> >  xen/arch/arm/include/asm/imx-lpuart.h |  64 ++
> >  xen/drivers/char/Kconfig  |   7 +
> >  xen/drivers/char/Makefile |   1 +
> >  xen/drivers/char/imx-lpuart.c | 276 ++
> >  4 files changed, 348 insertions(+)
> >  create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
> >  create mode 100644 xen/drivers/char/imx-lpuart.c
> >
> > diff --git a/xen/arch/arm/include/asm/imx-lpuart.h
> > b/xen/arch/arm/include/asm/imx-lpuart.h
> > new file mode 100644
> > index 00..fe859045dc
> > --- /dev/null
> > +++ b/xen/arch/arm/include/asm/imx-lpuart.h
> > @@ -0,0 +1,64 @@
> > +/*
> > + * xen/arch/arm/include/asm/imx-lpuart.h
> > + *
> > + * Common constant definition between early printk and the LPUART
> > +driver
> > + *
> > + * Peng Fan 
> > + * Copyright 2022 NXP
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#ifndef __ASM_ARM_IMX_LPUART_H__
> > +#define __ASM_ARM_IMX_LPUART_H__
> > +
> > +/* 32-bit register definition */
> > +#define UARTBAUD  (0x10)
> > +#define UARTSTAT  (0x14)
> > +#define UARTCTRL  (0x18)
> > +#define UARTDATA  (0x1C)
> > +#define UARTMATCH (0x20)
> > +#define UARTMODIR (0x24)
> > +#define UARTFIFO  (0x28)
> > +#define UARTWATER (0x2c)
> > +
> > +#define UARTSTAT_TDRE BIT(23, UL)
> > +#define UARTSTAT_TC   BIT(22, UL)
> > +#define UARTSTAT_RDRF BIT(21, UL)
> > +#define UARTSTAT_OR   BIT(19, UL)
> > +
> > +#define UARTBAUD_OSR_SHIFT(24)
> > +#define UARTBAUD_OSR_MASK (0x1f)
> > +#define UARTBAUD_SBR_MASK (0x1fff)
> > +#define UARTBAUD_BOTHEDGE (0x0002)
> > +#define UARTBAUD_TDMAE(0x0080)
> > +#define UARTBAUD_RDMAE(0x0020)
> > +
> > +#define UARTCTRL_TIE  BIT(23, UL)
> > +#define UARTCTRL_TCIE BIT(22, UL)
> > +#define UARTCTRL_RIE  BIT(21, UL)
> > +#define UARTCTRL_ILIE BIT(20, UL)
> > +#define UARTCTRL_TE   BIT(19, UL)
> > +#define UARTCTRL_RE   BIT(18, UL)
> > +#define UARTCTRL_MBIT(4, UL)
> > +
> > +#define UARTWATER_RXCNT_OFF 24
> > +
> > +#endif /* __ASM_ARM_IMX_LPUART_H__ */
> > +
> > +/*
> > + * Local variables:
> > + * mode: C
> > + * c-file-style: "BSD"
> > + * c-basic-offset: 4
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> > diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig index
> > 2ff5b288e2..e5f7b1d8eb 100644
> > --- a/xen/drivers/char/Kconfig
> > +++ b/xen/drivers/char/Kconfig
> > @@ -13,6 +13,13 @@ config HAS_CADENCE_UART
> >   This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
> >   based board, say Y.
> >
> > +config HAS_IMX_LPUART
> > +   bool "i.MX LPUART driver"
> > +   default y
> > +   depends on ARM_64
> > +   help
> > + This selects the i.MX LPUART. If you have i.MX8QM based board, say Y.
> > +
> >  config HAS_MVEBU
> > bool "Marvell MVEBU UART

[PATCH V5 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

The i.MX LPUART Documentation:
https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)

Tested-by: Henry Wang 
Signed-off-by: Peng Fan 
---
 xen/arch/arm/include/asm/imx-lpuart.h |  64 ++
 xen/drivers/char/Kconfig  |   7 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 276 ++
 4 files changed, 348 insertions(+)
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

diff --git a/xen/arch/arm/include/asm/imx-lpuart.h 
b/xen/arch/arm/include/asm/imx-lpuart.h
new file mode 100644
index 00..fe859045dc
--- /dev/null
+++ b/xen/arch/arm/include/asm/imx-lpuart.h
@@ -0,0 +1,64 @@
+/*
+ * xen/arch/arm/include/asm/imx-lpuart.h
+ *
+ * Common constant definition between early printk and the LPUART driver
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_IMX_LPUART_H__
+#define __ASM_ARM_IMX_LPUART_H__
+
+/* 32-bit register definition */
+#define UARTBAUD  (0x10)
+#define UARTSTAT  (0x14)
+#define UARTCTRL  (0x18)
+#define UARTDATA  (0x1C)
+#define UARTMATCH (0x20)
+#define UARTMODIR (0x24)
+#define UARTFIFO  (0x28)
+#define UARTWATER (0x2c)
+
+#define UARTSTAT_TDRE BIT(23, UL)
+#define UARTSTAT_TC   BIT(22, UL)
+#define UARTSTAT_RDRF BIT(21, UL)
+#define UARTSTAT_OR   BIT(19, UL)
+
+#define UARTBAUD_OSR_SHIFT(24)
+#define UARTBAUD_OSR_MASK (0x1f)
+#define UARTBAUD_SBR_MASK (0x1fff)
+#define UARTBAUD_BOTHEDGE (0x0002)
+#define UARTBAUD_TDMAE(0x0080)
+#define UARTBAUD_RDMAE(0x0020)
+
+#define UARTCTRL_TIE  BIT(23, UL)
+#define UARTCTRL_TCIE BIT(22, UL)
+#define UARTCTRL_RIE  BIT(21, UL)
+#define UARTCTRL_ILIE BIT(20, UL)
+#define UARTCTRL_TE   BIT(19, UL)
+#define UARTCTRL_RE   BIT(18, UL)
+#define UARTCTRL_MBIT(4, UL)
+
+#define UARTWATER_RXCNT_OFF 24
+
+#endif /* __ASM_ARM_IMX_LPUART_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..e5f7b1d8eb 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,13 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have i.MX8QM based board, say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..df44f91e5d
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,276 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+uint32_t baud, clock_hz, dat

[PATCH V5 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 14 
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 48 +
 2 files changed, 62 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..842d768280 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -58,6 +58,16 @@ choice
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
config EARLY_UART_CHOICE_MESON
select EARLY_UART_MESON
depends on ARM_64
@@ -186,6 +196,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +296,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..b169332932
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,48 @@
+/*
+ * xen/arch/arm/arm64/debug-imx-lpuart.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+/*
+ * Wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/*
+ * LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.35.1

[PATCH V5 0/2] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

V5:
 Align code
 drop early_uart_init

V4:
 Wrong v3 version, some BIT definition are mixed in patch 1,2. 

V3:
 Addressed Michal's comments.
 Add Henry's T-b

V2:
 Per Julien's comments, fix coding style issue, drop unneeded code

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.



Peng Fan (2):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support

 xen/arch/arm/Kconfig.debug  |  14 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  48 +
 xen/arch/arm/include/asm/imx-lpuart.h   |  64 ++
 xen/drivers/char/Kconfig|   7 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 276 
 6 files changed, 410 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

-- 
2.35.1

RE: [PATCH V3 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan]

Hi Michal,

> -Original Message-
> From: Michal Orzel [mailto:michal.or...@arm.com]
> Sent: 2022年4月8日 18:23
> To: Peng Fan (OSS) ; sstabell...@kernel.org;
> jul...@xen.org; volodymyr_babc...@epam.com;
> bertrand.marq...@arm.com
> Cc: andrew.coop...@citrix.com; george.dun...@citrix.com;
> jbeul...@suse.com; w...@xen.org; xen-devel@lists.xenproject.org;
> van.free...@gmail.com; Peng Fan 
> Subject: Re: [PATCH V3 2/2] xen/arm: Add i.MX lpuart early printk support
> 
> Hi Peng,
> 
> On 07.04.2022 04:44, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > Signed-off-by: Peng Fan 
> > ---
> >  xen/arch/arm/Kconfig.debug  | 14 +++
> >  xen/arch/arm/arm64/debug-imx-lpuart.inc | 52
> +
> >  xen/arch/arm/include/asm/imx-lpuart.h   | 22 +--
> >  3 files changed, 77 insertions(+), 11 deletions(-)  create mode
> > 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
> >
> > diff --git a/xen/arch/arm/include/asm/imx-lpuart.h
> > b/xen/arch/arm/include/asm/imx-lpuart.h
> > index 111640edb4..26e2cf0249 100644
> > --- a/xen/arch/arm/include/asm/imx-lpuart.h
> > +++ b/xen/arch/arm/include/asm/imx-lpuart.h
> > @@ -30,10 +30,10 @@
> >  #define UARTFIFO  (0x28)
> >  #define UARTWATER (0x2c)
> >
> > -#define UARTSTAT_TDRE BIT(23)
> > -#define UARTSTAT_TC   BIT(22)
> > -#define UARTSTAT_RDRF BIT(21)
> > -#define UARTSTAT_OR   BIT(19)
> > +#define UARTSTAT_TDRE BIT(23, UL)
> > +#define UARTSTAT_TC   BIT(22, UL)
> > +#define UARTSTAT_RDRF BIT(21, UL)
> > +#define UARTSTAT_OR   BIT(19, UL)
> >
> >  #define UARTBAUD_OSR_SHIFT (24)
> >  #define UARTBAUD_OSR_MASK (0x1f)
> > @@ -42,13 +42,13 @@
> >  #define UARTBAUD_TDMAE(0x0080)
> >  #define UARTBAUD_RDMAE(0x0020)
> >
> > -#define UARTCTRL_TIE  BIT(23)
> > -#define UARTCTRL_TCIE BIT(22)
> > -#define UARTCTRL_RIE  BIT(21)
> > -#define UARTCTRL_ILIE BIT(20)
> > -#define UARTCTRL_TE   BIT(19)
> > -#define UARTCTRL_RE   BIT(18)
> > -#define UARTCTRL_MBIT(4)
> > +#define UARTCTRL_TIE  BIT(23, UL)
> > +#define UARTCTRL_TCIE BIT(22, UL)
> > +#define UARTCTRL_RIE  BIT(21, UL)
> > +#define UARTCTRL_ILIE BIT(20, UL)
> > +#define UARTCTRL_TE   BIT(19, UL)
> > +#define UARTCTRL_RE   BIT(18, UL)
> > +#define UARTCTRL_MBIT(4, UL)
> >
> >  #define UARTWATER_RXCNT_OFF 24
> >
> 
> I find it a bit strange that you fix the header file in the patch 2/2 while 
> patch
> 1/2 introduces it. Could you do these fixes in the patch where you create this
> file?

Oops. I missed to merge that BIT fix into patch 1. Just send out V4.

Thanks,
Peng.

> 
> Cheers,
> Michal

[PATCH V4 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 14 +++
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 52 +
 2 files changed, 66 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..842d768280 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -58,6 +58,16 @@ choice
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
config EARLY_UART_CHOICE_MESON
select EARLY_UART_MESON
depends on ARM_64
@@ -186,6 +196,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +296,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..f68252da86
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,52 @@
+/*
+ * xen/arch/arm/arm64/debug-imx-lpuart.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+.macro early_uart_init wb wc wd
+/* Already initialized in bootloader */
+.endm
+
+/*
+ * Wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/*
+ * LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.35.1

[PATCH V4 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

The i.MX LPUART Documentation:
https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)

Tested-by: Henry Wang 
Signed-off-by: Peng Fan 
---
 xen/arch/arm/include/asm/imx-lpuart.h |  64 ++
 xen/drivers/char/Kconfig  |   7 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 276 ++
 4 files changed, 348 insertions(+)
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

diff --git a/xen/arch/arm/include/asm/imx-lpuart.h 
b/xen/arch/arm/include/asm/imx-lpuart.h
new file mode 100644
index 00..26e2cf0249
--- /dev/null
+++ b/xen/arch/arm/include/asm/imx-lpuart.h
@@ -0,0 +1,64 @@
+/*
+ * xen/arch/arm/include/asm/imx-lpuart.h
+ *
+ * Common constant definition between early printk and the LPUART driver
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_IMX_LPUART_H__
+#define __ASM_ARM_IMX_LPUART_H__
+
+/* 32-bit register definition */
+#define UARTBAUD  (0x10)
+#define UARTSTAT  (0x14)
+#define UARTCTRL  (0x18)
+#define UARTDATA  (0x1C)
+#define UARTMATCH (0x20)
+#define UARTMODIR (0x24)
+#define UARTFIFO  (0x28)
+#define UARTWATER (0x2c)
+
+#define UARTSTAT_TDRE BIT(23, UL)
+#define UARTSTAT_TC   BIT(22, UL)
+#define UARTSTAT_RDRF BIT(21, UL)
+#define UARTSTAT_OR   BIT(19, UL)
+
+#define UARTBAUD_OSR_SHIFT (24)
+#define UARTBAUD_OSR_MASK (0x1f)
+#define UARTBAUD_SBR_MASK (0x1fff)
+#define UARTBAUD_BOTHEDGE (0x0002)
+#define UARTBAUD_TDMAE(0x0080)
+#define UARTBAUD_RDMAE(0x0020)
+
+#define UARTCTRL_TIE  BIT(23, UL)
+#define UARTCTRL_TCIE BIT(22, UL)
+#define UARTCTRL_RIE  BIT(21, UL)
+#define UARTCTRL_ILIE BIT(20, UL)
+#define UARTCTRL_TE   BIT(19, UL)
+#define UARTCTRL_RE   BIT(18, UL)
+#define UARTCTRL_MBIT(4, UL)
+
+#define UARTWATER_RXCNT_OFF 24
+
+#endif /* __ASM_ARM_IMX_LPUART_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..e5f7b1d8eb 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,13 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have i.MX8QM based board, say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..df44f91e5d
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,276 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+uint32_t baud, clock_hz, data_bits, parity, stop_bits, fif

[PATCH V4 0/2] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

V4:
 Wrong v3 version, some BIT definition are mixed in patch 1,2. 

V3:
 Addressed Michal's comments.
 Add Henry's T-b

V2:
 Per Julien's comments, fix coding style issue, drop unneeded code

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.

Peng Fan (2):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support

 xen/arch/arm/Kconfig.debug  |  14 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  52 +
 xen/arch/arm/include/asm/imx-lpuart.h   |  64 ++
 xen/drivers/char/Kconfig|   7 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 276 
 6 files changed, 414 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

-- 
2.35.1

[PATCH V3 0/2] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

V3:
 Addressed Michal's comments.
 Add Henry's T-b

V2:
 Per Julien's comments, fix coding style issue, drop unneeded code

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.

Peng Fan (2):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support

 xen/arch/arm/Kconfig.debug  |  14 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  52 +
 xen/arch/arm/include/asm/imx-lpuart.h   |  64 ++
 xen/drivers/char/Kconfig|   7 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 276 
 6 files changed, 414 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

-- 
2.35.1

[PATCH V3 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 14 +++
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 52 +
 xen/arch/arm/include/asm/imx-lpuart.h   | 22 +--
 3 files changed, 77 insertions(+), 11 deletions(-)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..842d768280 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -58,6 +58,16 @@ choice
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
config EARLY_UART_CHOICE_MESON
select EARLY_UART_MESON
depends on ARM_64
@@ -186,6 +196,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +296,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..f68252da86
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,52 @@
+/*
+ * xen/arch/arm/arm64/debug-imx-lpuart.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+.macro early_uart_init wb wc wd
+/* Already initialized in bootloader */
+.endm
+
+/*
+ * Wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/*
+ * LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/include/asm/imx-lpuart.h 
b/xen/arch/arm/include/asm/imx-lpuart.h
index 111640edb4..26e2cf0249 100644
--- a/xen/arch/arm/include/asm/imx-lpuart.h
+++ b/xen/arch/arm/include/asm/imx-lpuart.h
@@ -30,10 +30,10 @@
 #define UARTFIFO  (0x28)
 #define UARTWATER (0x2c)
 
-#define UARTSTAT_TDRE BIT(23)
-#define UARTSTAT_TC   BIT(22)
-#define UARTSTAT_RDRF BIT(21)
-#define UARTSTAT_OR   BIT(19)
+#define UARTSTAT_TDRE BIT(23, UL)
+#define UARTSTAT_TC   BIT(22, UL)
+#define UARTSTAT_RDRF BIT(21, UL)
+#define UARTSTAT_OR   BIT(19, UL)
 
 #define UARTBAUD_OSR_SHIFT (24)
 #define UARTBAUD_OSR_MASK (0x1f)
@@ -42,13 +42,13 @@
 #define UARTBAUD_TDMAE(0x0080)
 #define UARTBAUD_RDMAE(0x0020)
 
-#define UARTCTRL_TIE  BIT(23)
-#define UARTCTRL_TCIE BIT(22)
-#define UARTCTRL_RIE  BIT(21)
-#define UARTCTRL_ILIE BIT(20)
-#define UARTCTRL_TE   BIT(19)
-#define UARTCTRL_RE   BIT(18)
-#define UARTCTRL_MBIT(4)
+#define UARTCTRL_TIE  BIT(23, UL)
+#define UARTCTRL_TCIE BIT(22, UL)
+#define UAR

[PATCH V3 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

The i.MX LPUART Documentation:
https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)

Tested-by: Henry Wang 
Signed-off-by: Peng Fan 
---
 xen/arch/arm/include/asm/imx-lpuart.h |  64 ++
 xen/drivers/char/Kconfig  |   7 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 276 ++
 4 files changed, 348 insertions(+)
 create mode 100644 xen/arch/arm/include/asm/imx-lpuart.h
 create mode 100644 xen/drivers/char/imx-lpuart.c

diff --git a/xen/arch/arm/include/asm/imx-lpuart.h 
b/xen/arch/arm/include/asm/imx-lpuart.h
new file mode 100644
index 00..111640edb4
--- /dev/null
+++ b/xen/arch/arm/include/asm/imx-lpuart.h
@@ -0,0 +1,64 @@
+/*
+ * xen/arch/arm/include/asm/imx-lpuart.h
+ *
+ * Common constant definition between early printk and the LPUART driver
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __ASM_ARM_IMX_LPUART_H__
+#define __ASM_ARM_IMX_LPUART_H__
+
+/* 32-bit register definition */
+#define UARTBAUD  (0x10)
+#define UARTSTAT  (0x14)
+#define UARTCTRL  (0x18)
+#define UARTDATA  (0x1C)
+#define UARTMATCH (0x20)
+#define UARTMODIR (0x24)
+#define UARTFIFO  (0x28)
+#define UARTWATER (0x2c)
+
+#define UARTSTAT_TDRE BIT(23)
+#define UARTSTAT_TC   BIT(22)
+#define UARTSTAT_RDRF BIT(21)
+#define UARTSTAT_OR   BIT(19)
+
+#define UARTBAUD_OSR_SHIFT (24)
+#define UARTBAUD_OSR_MASK (0x1f)
+#define UARTBAUD_SBR_MASK (0x1fff)
+#define UARTBAUD_BOTHEDGE (0x0002)
+#define UARTBAUD_TDMAE(0x0080)
+#define UARTBAUD_RDMAE(0x0020)
+
+#define UARTCTRL_TIE  BIT(23)
+#define UARTCTRL_TCIE BIT(22)
+#define UARTCTRL_RIE  BIT(21)
+#define UARTCTRL_ILIE BIT(20)
+#define UARTCTRL_TE   BIT(19)
+#define UARTCTRL_RE   BIT(18)
+#define UARTCTRL_MBIT(4)
+
+#define UARTWATER_RXCNT_OFF 24
+
+#endif /* __ASM_ARM_IMX_LPUART_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..e5f7b1d8eb 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,13 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have i.MX8QM based board, say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..df44f91e5d
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,276 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+uint32_t baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
+uint32_t irq;
+char 

RE: [PATCH V2 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan]

> Subject: Re: [PATCH V2 1/2] xen/arm: Add i.MX lpuart driver
> 
> Hi Peng,
> 
> On 02.04.2022 07:42, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > The i.MX LPUART Documentation:
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> >
> nxp.com%2Fwebapp%2FDownload%3FcolCode%3DIMX8QMIECdata=0
> 4%7C01%7Cp
> >
> eng.fan%40nxp.com%7Cc7d221ce800342bbd7c108da179e72ca%7C686ea1d
> 3bc2b4c6
> >
> fa92cd99c5c301635%7C0%7C1%7C637848266543058498%7CUnknown%7C
> TWFpbGZsb3d
> >
> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C
> >
> 3000sdata=3CV%2FoJViSFzqRby0h577SiusHRQnC8KiXscntQW%2BZOs
> %3D
> > reserved=0 Chatper 13.6 Low Power Universal Asynchronous Receiver/
> > Transmitter (LPUART)
> >
> > Signed-off-by: Peng Fan 
> > ---
> >  xen/drivers/char/Kconfig  |   8 +
> >  xen/drivers/char/Makefile |   1 +
> >  xen/drivers/char/imx-lpuart.c | 275
> > ++
> >  xen/include/xen/imx-lpuart.h  |  64 
> >  4 files changed, 348 insertions(+)
> >  create mode 100644 xen/drivers/char/imx-lpuart.c  create mode
> 100644
> > xen/include/xen/imx-lpuart.h
> >
> > diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig index
> > 2ff5b288e2..0efdb2128f 100644
> > --- a/xen/drivers/char/Kconfig
> > +++ b/xen/drivers/char/Kconfig
> > @@ -13,6 +13,14 @@ config HAS_CADENCE_UART
> >   This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
> >   based board, say Y.
> >
> > +config HAS_IMX_LPUART
> > +   bool "i.MX LPUART driver"
> > +   default y
> > +   depends on ARM_64
> > +   help
> > + This selects the i.MX LPUART. If you have a i.MX8QM based board,
> > + say Y.
> Why did you move "say Y" to the next line?

I configured vimrc with " set cc=75,100 ", the "." will be at 80, so I
move "say Y." to next line.

> 
> > +
> >  config HAS_MVEBU
> > bool "Marvell MVEBU UART driver"
> > default y
> > diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
> > index 7c646d771c..14e67cf072 100644
> > --- a/xen/drivers/char/Makefile
> > +++ b/xen/drivers/char/Makefile
> > @@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
> >  obj-$(CONFIG_HAS_OMAP) += omap-uart.o
> >  obj-$(CONFIG_HAS_SCIF) += scif-uart.o
> >  obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
> > +obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
> >  obj-$(CONFIG_ARM) += arm-uart.o
> >  obj-y += serial.o
> >  obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o diff --git
> > a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c new
> > file mode 100644 index 00..49330fd2f8
> > --- /dev/null
> > +++ b/xen/drivers/char/imx-lpuart.c
> > @@ -0,0 +1,275 @@
> > +/*
> > + * xen/drivers/char/imx-lpuart.c
> > + *
> > + * Driver for i.MX LPUART.
> > + *
> > + * Peng Fan 
> > + * Copyright 2022 NXP
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +
> > +#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
> > +#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs +
> > +off)
> > +
> > +static struct imx_lpuart {
> > +uint32_t baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
> > +uint32_t irq;
> > +char __iomem *regs;
> > +struct irqaction irqaction;
> > +struct vuart_info vuart;
> > +} imx8_com;
> > +
> > +static void imx_lpuart_interrupt(int irq, void *data,
> > + struct cpu_user_regs *regs) {
> > +struct serial_port *port = data;
> > +struct imx_lpuart *uart = port->uart;
> > +uint32_t sts, rxcnt;
> > +
> > +sts = imx_lpuart_

[PATCH V2 1/2] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

The i.MX LPUART Documentation:
https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)

Signed-off-by: Peng Fan 
---
 xen/drivers/char/Kconfig  |   8 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 275 ++
 xen/include/xen/imx-lpuart.h  |  64 
 4 files changed, 348 insertions(+)
 create mode 100644 xen/drivers/char/imx-lpuart.c
 create mode 100644 xen/include/xen/imx-lpuart.h

diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..0efdb2128f 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,14 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have a i.MX8QM based board,
+ say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..49330fd2f8
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,275 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+uint32_t baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
+uint32_t irq;
+char __iomem *regs;
+struct irqaction irqaction;
+struct vuart_info vuart;
+} imx8_com;
+
+static void imx_lpuart_interrupt(int irq, void *data,
+ struct cpu_user_regs *regs)
+{
+struct serial_port *port = data;
+struct imx_lpuart *uart = port->uart;
+uint32_t sts, rxcnt;
+
+sts = imx_lpuart_read(uart, UARTSTAT);
+rxcnt = imx_lpuart_read(uart, UARTWATER) >> UARTWATER_RXCNT_OFF;
+
+if ( (sts & UARTSTAT_RDRF) || (rxcnt > 0) )
+   serial_rx_interrupt(port, regs);
+
+if ( sts & UARTSTAT_TDRE )
+   serial_tx_interrupt(port, regs);
+
+imx_lpuart_write(uart, UARTSTAT, sts);
+}
+
+static void __init imx_lpuart_init_preirq(struct serial_port *port)
+{
+struct imx_lpuart *uart = port->uart;
+uint32_t ctrl, old_ctrl, bd;
+
+ctrl = old_ctrl = imx_lpuart_read(uart, UARTCTRL);
+ctrl = (old_ctrl & ~UARTCTRL_M) | UARTCTRL_TE | UARTCTRL_RE;
+bd = imx_lpuart_read(uart, UARTBAUD);
+
+while ( !(imx_lpuart_read(uart, UARTSTAT) & UARTSTAT_TC) )
+   cpu_relax();
+
+/* Disable transmit and receive */
+imx_lpuart_write(uart, UARTCTRL, old_ctrl & ~(UARTCTRL_TE | UARTCTRL_RE));
+
+/* Reuse firmware baudrate settings, only disable DMA here */
+bd &= ~(UARTBAUD_TDMAE | UARTBAUD_RDMAE);
+
+imx_lpuart_write(uart, UARTMODIR, 0);
+imx_lpuart_write(uart, UARTBAUD, bd);
+imx_lpuart_write(uart, UARTCTRL, ctrl);
+}
+
+static void __init imx_lpuart_init_postirq(struct serial_port *port)
+{
+struct imx_lpuart *uart = port->uart;
+uint32_t temp;
+
+uart->irqaction.handler = imx_lpuart_interrupt;
+uart->irqaction.name = "imx_lpuart";
+uart->irqaction.dev_id = port;
+
+if ( setup_irq(uart->irq, 0, >irqaction) != 0 )
+{
+dprintk(XENLOG_ERR, "Failed to allocate imx_lpuart IRQ %d\n",
+uart->irq);
+return;
+}
+
+/* Enable interrupts */
+temp = imx_lpuart_read(uart, UARTCTRL);
+temp |= (UARTCTRL_RIE | UARTCTRL_TIE);
+temp |= UARTCTRL_ILIE;
+imx_lpuart_write(uart, U

[PATCH V2 2/2] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 14 +++
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 52 +
 2 files changed, 66 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..842d768280 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -58,6 +58,16 @@ choice
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
config EARLY_UART_CHOICE_MESON
select EARLY_UART_MESON
depends on ARM_64
@@ -186,6 +196,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +296,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..f7ac78a781
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,52 @@
+/*
+ * xen/arch/arm/arm64/debug-imx8qm.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright (C) 2016 Freescale Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+.macro early_uart_init wb wc wd
+/* Already initialized in bootloader */
+.endm
+
+/*
+ * Wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/*
+ * LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit
+ */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.35.1

[PATCH V2 0/2] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

V2:
 Per Julien's comments, fix coding style issue, drop unneeded code

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.

Peng Fan (2):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support

 xen/arch/arm/Kconfig.debug  |  14 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  52 +
 xen/drivers/char/Kconfig|   8 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 275 
 xen/include/xen/imx-lpuart.h|  64 ++
 6 files changed, 414 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/drivers/char/imx-lpuart.c
 create mode 100644 xen/include/xen/imx-lpuart.h

-- 
2.35.1

RE: [PATCH 2/3] xen/arm: Add i.MX lpuart early printk support

[Peng Fan]

> Subject: Re: [PATCH 2/3] xen/arm: Add i.MX lpuart early printk support
> 
> Hi Peng,
> 
> On 28/02/2022 01:07, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > Signed-off-by: Peng Fan 
> > ---
> >   xen/arch/arm/Kconfig.debug  | 18 ++
> >   xen/arch/arm/arm64/debug-imx-lpuart.inc | 48
> +
> >   2 files changed, 66 insertions(+)
> >   create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
> >
> > diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
> > index 35ccd13273..9ecb446b3a 100644
> > --- a/xen/arch/arm/Kconfig.debug
> > +++ b/xen/arch/arm/Kconfig.debug
> > @@ -55,6 +55,20 @@ choice
> > selecting one of the platform specific options below if
> > you know the parameters for the port.
> >
> > +   This option is preferred over the platform specific
> > +   options; the platform specific options are deprecated
> > +   and will soon be removed.
> > +   config EARLY_UART_CHOICE_IMX_LPUART
> > +   select EARLY_UART_IMX_LPUART
> > +   depends on ARM_64
> > +   bool "Early printk via i.MX LPUART"
> > +   help
> > +   Say Y here if you wish the early printk to direct their
> > +   output to a i.MX LPUART. You can use this option to
> > +   provide the parameters for the i.MX LPUART rather than
> > +   selecting one of the platform specific options below if
> > +   you know the parameters for the port.
> 
> Plaform specific early printk are deprecated. So I would rather prefer we are
> not introducing new one. Can you adjust the description to remove any
> mention of platform specific options?

Sure, fix in v2.

> 
> > +
> > This option is preferred over the platform specific
> > options; the platform specific options are deprecated
> > and will soon be removed.
> > @@ -186,6 +200,9 @@ config EARLY_UART_CADENCE
> >   config EARLY_UART_EXYNOS4210
> > select EARLY_PRINTK
> > bool
> > +config EARLY_UART_IMX_LPUART
> > +   select EARLY_PRINTK
> > +   bool
> >   config EARLY_UART_MESON
> > select EARLY_PRINTK
> > bool
> > @@ -283,6 +300,7 @@ config EARLY_PRINTK_INC
> > default "debug-8250.inc" if EARLY_UART_8250
> > default "debug-cadence.inc" if EARLY_UART_CADENCE
> > default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
> > +   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
> > default "debug-meson.inc" if EARLY_UART_MESON
> > default "debug-mvebu.inc" if EARLY_UART_MVEBU
> >     default "debug-pl011.inc" if EARLY_UART_PL011 diff --git
> > a/xen/arch/arm/arm64/debug-imx-lpuart.inc
> > b/xen/arch/arm/arm64/debug-imx-lpuart.inc
> > new file mode 100644
> > index 00..7510210d46
> > --- /dev/null
> > +++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
> > @@ -0,0 +1,48 @@
> > +/*
> > + * xen/arch/arm/arm64/debug-imx8qm.inc
> > + *
> > + * i.MX8QM specific debug code
> > + *
> > + * Peng Fan 
> > + * Copyright (C) 2016 Freescale Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include 
> > +
> > +.macro early_uart_init wb wc wd
> > +/* Already initialized in bootloader */ .endm
> 
> NIT: I would add a newline to separate with this macro from next one.

Fix in v2.

> 
> > +/* i.MX8QM wait LPUART to be ready to transmit
> > + * rb: register which contains the UART base address
> > + * rc: scratch register
> > + */
> 
> The coding style for multi-lines comment is:

Fix in v2. Thanks.

> 
> /*
>   * Foo
>   * Bar
>   */
> 
> > +.macro early_uart_ready xb, c
> > +1:
> > +ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
> > +tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
> > +beq   1b  /* Wait for the UART to be
> ready */
> > +.endm
> > +
> > +/* i.MX8QM LPUART transmit character
> > + * rb: register which contains the UART base address
> > + * rt: register which contains the character to transmit */
> 
> Coding style:

Fix in V2.

Thanks,
Peng.
> 
> /*
>   * Foo
>   * Bar
>   */
> 
> > +.macro early_uart_transmit xb, wt
> > +str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
> > +.endm
> > +
> > +/*
> > + * Local variables:
> > + * mode: ASM
> > + * indent-tabs-mode: nil
> > + * End:
> > + */
> 
> Cheers,
> 
> --
> Julien Grall

RE: [PATCH 3/3] xen/arm: Add i.MX8QM platform support

[Peng Fan]

> Subject: Re: [PATCH 3/3] xen/arm: Add i.MX8QM platform support
> 
> Hi Peng,
> 
> On 28/02/2022 01:07, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > Signed-off-by: Peng Fan 
> > ---
> >   xen/arch/arm/Kconfig.debug  |  3 +++
> >   xen/arch/arm/platforms/Makefile |  1 +
> >   xen/arch/arm/platforms/imx8qm.c | 44
> +
> >   3 files changed, 48 insertions(+)
> >   create mode 100644 xen/arch/arm/platforms/imx8qm.c
> >
> > diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
> > index 9ecb446b3a..43ccd8fe62 100644
> > --- a/xen/arch/arm/Kconfig.debug
> > +++ b/xen/arch/arm/Kconfig.debug
> > @@ -143,6 +143,9 @@ choice
> > config EARLY_PRINTK_HIKEY960
> > bool "Early printk with pl011 with Hikey 960"
> > select EARLY_UART_PL011
> > +   config EARLY_PRINTK_IMX8QM
> > +   bool "Early printk with i.MX LPUART with i.MX8QM"
> > +   select EARLY_UART_IMX_LPUART
> 
> The goal of platform specific early printk is to select to UART address (see
> EARLY_UART_BASE_ADDRESS).
> 
> However, we have deprecated them. So we should avoid adding new ones.
> 
> > config EARLY_PRINTK_JUNO
> > bool "Early printk with pl011 on Juno platform"
> > select EARLY_UART_PL011
> > diff --git a/xen/arch/arm/platforms/Makefile
> > b/xen/arch/arm/platforms/Makefile index 8632f4115f..bec6e55d1f 100644
> > --- a/xen/arch/arm/platforms/Makefile
> > +++ b/xen/arch/arm/platforms/Makefile
> > @@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
> >   obj-$(CONFIG_ALL64_PLAT) += thunderx.o
> >   obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
> >   obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
> > +obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
> >   obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
> >   obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o diff --git
> > a/xen/arch/arm/platforms/imx8qm.c b/xen/arch/arm/platforms/imx8qm.c
> > new file mode 100644 index 00..289c18e5f9
> > --- /dev/null
> > +++ b/xen/arch/arm/platforms/imx8qm.c
> > @@ -0,0 +1,44 @@
> > +/*
> > + * xen/arch/arm/platforms/imx8qm.c
> > + *
> > + * i.MX 8QM setup
> > + *
> > + * Copyright 2022 NXP
> > + *
> > + * Peng Fan 
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +
> > +static const char * const imx8qm_dt_compat[] __initconst = {
> > +"fsl,imx8qm",
> > +NULL
> > +};
> > +
> > +PLATFORM_START(imx8qm, "i.MX 8")
> > +.compatible = imx8qm_dt_compat,
> > +PLATFORM_END
> 
> We are only adding new platform definition when quirks are necessary. Do
> you need specific quirks for the i.MX8QM?
> 
> A somewhat related question, is this series enough to boot Xen upstream on
> the board?

Boot xen upstream, no need specific quirk.

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall

RE: [PATCH 1/3] xen/arm: Add i.MX lpuart driver

[Peng Fan]

Hi Julien,

> Subject: Re: [PATCH 1/3] xen/arm: Add i.MX lpuart driver
> 
> Hi Peng,
> 
> On 28/02/2022 01:07, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > Signed-off-by: Peng Fan 
> > ---
> >   xen/drivers/char/Kconfig  |   8 +
> >   xen/drivers/char/Makefile |   1 +
> >   xen/drivers/char/imx-lpuart.c | 303
> ++
> >   xen/include/xen/imx-lpuart.h  |  64 +++
> >   4 files changed, 376 insertions(+)
> >   create mode 100644 xen/drivers/char/imx-lpuart.c
> >   create mode 100644 xen/include/xen/imx-lpuart.h
> >
> > diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig index
> > 2ff5b288e2..0efdb2128f 100644
> > --- a/xen/drivers/char/Kconfig
> > +++ b/xen/drivers/char/Kconfig
> > @@ -13,6 +13,14 @@ config HAS_CADENCE_UART
> >   This selects the Xilinx Zynq Cadence UART. If you have a Xilinx
> Zynq
> >   based board, say Y.
> >
> > +config HAS_IMX_LPUART
> > +   bool "i.MX LPUART driver"
> > +   default y
> > +   depends on ARM_64
> > +   help
> > + This selects the i.MX LPUART. If you have a i.MX8QM based board,
> > + say Y.
> > +
> >   config HAS_MVEBU
> > bool "Marvell MVEBU UART driver"
> > default y
> > diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
> > index 7c646d771c..14e67cf072 100644
> > --- a/xen/drivers/char/Makefile
> > +++ b/xen/drivers/char/Makefile
> > @@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
> >   obj-$(CONFIG_HAS_OMAP) += omap-uart.o
> >   obj-$(CONFIG_HAS_SCIF) += scif-uart.o
> >   obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
> > +obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
> >   obj-$(CONFIG_ARM) += arm-uart.o
> >   obj-y += serial.o
> >   obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o diff --git
> > a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c new
> > file mode 100644 index 00..2a30e3f21a
> > --- /dev/null
> > +++ b/xen/drivers/char/imx-lpuart.c
> > @@ -0,0 +1,303 @@
> > +/*
> > + * xen/drivers/char/imx-lpuart.c
> > + *
> > + * Driver for i.MX LPUART.
> > + *
> > + * Peng Fan 
> > + * Copyright 2022 NXP
> > + *
> > + * This program is free software; you can redistribute it and/or
> > +modify
> > + * it under the terms of the GNU General Public License as published
> > +by
> > + * the Free Software Foundation; either version 2 of the License, or
> > + * (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + */
> > +
> > +#include 
> 
> This should not be necessary.h

Will drop in v2.

> 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> 
> Please order the  alphabetically.h

Fix in V2.

> 
> > +#include 
> > +#include 
> > +
> > +#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
> > +#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs +
> > +off)
> > +
> > +static struct imx_lpuart {
> > +unsigned int baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
> > +unsigned int irq;
> > +char __iomem *regs;
> > +struct irqaction irqaction;
> > +struct vuart_info vuart;
> > +} imx8_com = {0};
> 
> This will be initialized to 0 by default. So I would drop {0}.

Fix in V2.

> 
> > +
> > +static void imx_lpuart_interrupt(int irq, void *data,
> > +  struct cpu_user_regs *regs)
> 
> Coding style: 'struct' should be aligned with 'int'.

Fix in V2.

> 
> > +{
> > +struct serial_port *port = data;
> > +struct imx_lpuart *uart = port->uart;
> > +unsigned int sts, rxcnt;
> > +
> > +sts = imx_lpuart_read(uart, UARTSTAT);
> > +rxcnt = imx_lpuart_read(uart, UARTWATER) >>
> UARTWATER_RXCNT_OFF;
> > +
> > +if ((sts & UARTSTAT_RDRF) || (rxcnt > 0)) {
> 
> Coding style:
> 
> if ( ... )
> {
> 
> But for single line block, we tend to avoid {}.

Fix in V2.

> 
> > +   serial_rx_interrupt(port, regs);
> > +}
> > +
> > +if ((sts & UARTSTAT_TDRE) &&
> > +!(imx_lpuart_read(uart, UARTBAUD

RE: [PATCH 1/3] xen/arm: Add i.MX lpuart driver

[Peng Fan]

Hi Julien,

> Subject: Re: [PATCH 1/3] xen/arm: Add i.MX lpuart driver
> 
> Hi Peng,
> 
> On 28/02/2022 01:07, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> 
> Can you give me a link to the specification and/or a similar driver in Linux?

https://www.nxp.com/webapp/Download?colCode=IMX8QMIEC 
Chatper 13.6 Low Power Universal Asynchronous Receiver/
Transmitter (LPUART)
But this requires registration to access.

Linux driver:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/tty/serial/fsl_lpuart.c


Thanks,
Peng.

> 
> This would help to review this patch.
> 
> Cheers,
> 
> --
> Julien Grall

RE: [PATCH 0/3] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan]

> Subject: Re: [PATCH 0/3] xen/arm: add i.MX lpuart and i.MX8QM initial
> support
> 
> On 28.02.2022 02:07, Peng Fan (OSS) wrote:
> > From: Peng Fan 
> >
> > Add i.MX lpuart driver and i.MX8QM platform support.
> >  - lpuart is the uart IP used in i.MX8QM/QXP/93.
> >  - Very basic i.MX8QM platform support.
> >
> > Peng Fan (3):
> >   xen/arm: Add i.MX lpuart driver
> >   xen/arm: Add i.MX lpuart early printk support
> >   xen/arm: Add i.MX8QM platform support
> >
> >  xen/arch/arm/Kconfig.debug  |  21 ++
> >  xen/arch/arm/arm64/debug-imx-lpuart.inc |  48 
> >  xen/arch/arm/platforms/Makefile |   1 +
> >  xen/arch/arm/platforms/imx8qm.c |  44 
> >  xen/drivers/char/Kconfig|   8 +
> >  xen/drivers/char/Makefile   |   1 +
> >  xen/drivers/char/imx-lpuart.c   | 303
> 
> >  xen/include/xen/imx-lpuart.h|  64 +
> >  8 files changed, 490 insertions(+)
> >  create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
> >  create mode 100644 xen/arch/arm/platforms/imx8qm.c  create mode
> > 100644 xen/drivers/char/imx-lpuart.c  create mode 100644
> > xen/include/xen/imx-lpuart.h
> 
> I guess the latter two additions want to be accompanied by an update
> to ./MAINTAINERS' ARM section.

I'll address this in v2. Let me wait more comments and address together.

Thanks,
Peng.

> 
> Jan

[PATCH 1/3] xen/arm: Add i.MX lpuart driver

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/drivers/char/Kconfig  |   8 +
 xen/drivers/char/Makefile |   1 +
 xen/drivers/char/imx-lpuart.c | 303 ++
 xen/include/xen/imx-lpuart.h  |  64 +++
 4 files changed, 376 insertions(+)
 create mode 100644 xen/drivers/char/imx-lpuart.c
 create mode 100644 xen/include/xen/imx-lpuart.h

diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 2ff5b288e2..0efdb2128f 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -13,6 +13,14 @@ config HAS_CADENCE_UART
  This selects the Xilinx Zynq Cadence UART. If you have a Xilinx Zynq
  based board, say Y.
 
+config HAS_IMX_LPUART
+   bool "i.MX LPUART driver"
+   default y
+   depends on ARM_64
+   help
+ This selects the i.MX LPUART. If you have a i.MX8QM based board,
+ say Y.
+
 config HAS_MVEBU
bool "Marvell MVEBU UART driver"
default y
diff --git a/xen/drivers/char/Makefile b/xen/drivers/char/Makefile
index 7c646d771c..14e67cf072 100644
--- a/xen/drivers/char/Makefile
+++ b/xen/drivers/char/Makefile
@@ -8,6 +8,7 @@ obj-$(CONFIG_HAS_MVEBU) += mvebu-uart.o
 obj-$(CONFIG_HAS_OMAP) += omap-uart.o
 obj-$(CONFIG_HAS_SCIF) += scif-uart.o
 obj-$(CONFIG_HAS_EHCI) += ehci-dbgp.o
+obj-$(CONFIG_HAS_IMX_LPUART) += imx-lpuart.o
 obj-$(CONFIG_ARM) += arm-uart.o
 obj-y += serial.o
 obj-$(CONFIG_XEN_GUEST) += xen_pv_console.o
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
new file mode 100644
index 00..2a30e3f21a
--- /dev/null
+++ b/xen/drivers/char/imx-lpuart.c
@@ -0,0 +1,303 @@
+/*
+ * xen/drivers/char/imx-lpuart.c
+ *
+ * Driver for i.MX LPUART.
+ *
+ * Peng Fan 
+ * Copyright 2022 NXP
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define imx_lpuart_read(uart, off)   readl((uart)->regs + off)
+#define imx_lpuart_write(uart, off, val) writel((val), (uart)->regs + off)
+
+static struct imx_lpuart {
+unsigned int baud, clock_hz, data_bits, parity, stop_bits, fifo_size;
+unsigned int irq;
+char __iomem *regs;
+struct irqaction irqaction;
+struct vuart_info vuart;
+} imx8_com = {0};
+
+static void imx_lpuart_interrupt(int irq, void *data,
+  struct cpu_user_regs *regs)
+{
+struct serial_port *port = data;
+struct imx_lpuart *uart = port->uart;
+unsigned int sts, rxcnt;
+
+sts = imx_lpuart_read(uart, UARTSTAT);
+rxcnt = imx_lpuart_read(uart, UARTWATER) >> UARTWATER_RXCNT_OFF;
+
+if ((sts & UARTSTAT_RDRF) || (rxcnt > 0)) {
+   serial_rx_interrupt(port, regs);
+}
+
+if ((sts & UARTSTAT_TDRE) &&
+!(imx_lpuart_read(uart, UARTBAUD) & UARTBAUD_TDMAE))
+   serial_tx_interrupt(port, regs);
+
+imx_lpuart_write(uart, UARTSTAT, sts);
+}
+
+static void __init imx_lpuart_init_preirq(struct serial_port *port)
+{
+struct imx_lpuart *uart = port->uart;
+u32 sbr, osr;
+u32 ctrl, old_ctrl, bd;
+u32 baud;
+
+ctrl = old_ctrl = imx_lpuart_read(uart, UARTCTRL);
+ctrl = (old_ctrl & ~UARTCTRL_M) | UARTCTRL_TE | UARTCTRL_RE;
+bd = imx_lpuart_read(uart, UARTBAUD);
+baud = uart->baud;
+
+while ( !(imx_lpuart_read(uart, UARTSTAT) & UARTSTAT_TC))
+   barrier();
+
+/* Disable trasmit and receive */
+imx_lpuart_write(uart, UARTCTRL, old_ctrl & ~(UARTCTRL_TE | UARTCTRL_RE));
+
+osr = (bd >> UARTBAUD_OSR_SHIFT) & UARTBAUD_OSR_MASK;
+sbr = uart->clock_hz / (baud * (osr + 1));
+
+bd &= ~ UARTBAUD_SBR_MASK;
+bd |= sbr & UARTBAUD_SBR_MASK;
+bd |= UARTBAUD_BOTHEDGE;
+bd &= ~(UARTBAUD_TDMAE | UARTBAUD_RDMAE);
+
+imx_lpuart_write(uart, UARTMODIR, 0);
+imx_lpuart_write(uart, UARTBAUD, bd);
+imx_lpuart_write(uart, UARTCTRL, ctrl);
+}
+
+static void __init imx_lpuart_init_postirq(struct serial_port *port)
+{
+struct imx_lpuart *uart = port->uart;
+unsigned int temp;
+
+uart->irqaction.handler = imx_lpuart_interrupt;
+uart->irqaction.name = "imx_lpuart";
+uart->irqaction.dev_id = port;
+
+if ( setup_irq(uart->irq, 0, >irqaction) != 0 )
+{
+dprintk(XENLOG_ERR, "Failed to allocate imx_lpuart IRQ %d\n",
+uart->irq);
+return;
+}
+
+ 

[PATCH 3/3] xen/arm: Add i.MX8QM platform support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  |  3 +++
 xen/arch/arm/platforms/Makefile |  1 +
 xen/arch/arm/platforms/imx8qm.c | 44 +
 3 files changed, 48 insertions(+)
 create mode 100644 xen/arch/arm/platforms/imx8qm.c

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 9ecb446b3a..43ccd8fe62 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -143,6 +143,9 @@ choice
config EARLY_PRINTK_HIKEY960
bool "Early printk with pl011 with Hikey 960"
select EARLY_UART_PL011
+   config EARLY_PRINTK_IMX8QM
+   bool "Early printk with i.MX LPUART with i.MX8QM"
+   select EARLY_UART_IMX_LPUART
config EARLY_PRINTK_JUNO
bool "Early printk with pl011 on Juno platform"
select EARLY_UART_PL011
diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
index 8632f4115f..bec6e55d1f 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -9,5 +9,6 @@ obj-$(CONFIG_ALL_PLAT)   += sunxi.o
 obj-$(CONFIG_ALL64_PLAT) += thunderx.o
 obj-$(CONFIG_ALL64_PLAT) += xgene-storm.o
 obj-$(CONFIG_ALL64_PLAT) += brcm-raspberry-pi.o
+obj-$(CONFIG_ALL64_PLAT) += imx8qm.o
 obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp.o
 obj-$(CONFIG_MPSOC_PLATFORM)  += xilinx-zynqmp-eemi.o
diff --git a/xen/arch/arm/platforms/imx8qm.c b/xen/arch/arm/platforms/imx8qm.c
new file mode 100644
index 00..289c18e5f9
--- /dev/null
+++ b/xen/arch/arm/platforms/imx8qm.c
@@ -0,0 +1,44 @@
+/*
+ * xen/arch/arm/platforms/imx8qm.c
+ *
+ * i.MX 8QM setup
+ *
+ * Copyright 2022 NXP
+ *
+ * Peng Fan 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+static const char * const imx8qm_dt_compat[] __initconst =
+{
+"fsl,imx8qm",
+NULL
+};
+
+PLATFORM_START(imx8qm, "i.MX 8")
+.compatible = imx8qm_dt_compat,
+PLATFORM_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.30.0

[PATCH 2/3] xen/arm: Add i.MX lpuart early printk support

[Peng Fan (OSS)]

From: Peng Fan 

Signed-off-by: Peng Fan 
---
 xen/arch/arm/Kconfig.debug  | 18 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc | 48 +
 2 files changed, 66 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc

diff --git a/xen/arch/arm/Kconfig.debug b/xen/arch/arm/Kconfig.debug
index 35ccd13273..9ecb446b3a 100644
--- a/xen/arch/arm/Kconfig.debug
+++ b/xen/arch/arm/Kconfig.debug
@@ -55,6 +55,20 @@ choice
selecting one of the platform specific options below if
you know the parameters for the port.
 
+   This option is preferred over the platform specific
+   options; the platform specific options are deprecated
+   and will soon be removed.
+   config EARLY_UART_CHOICE_IMX_LPUART
+   select EARLY_UART_IMX_LPUART
+   depends on ARM_64
+   bool "Early printk via i.MX LPUART"
+   help
+   Say Y here if you wish the early printk to direct their
+   output to a i.MX LPUART. You can use this option to
+   provide the parameters for the i.MX LPUART rather than
+   selecting one of the platform specific options below if
+   you know the parameters for the port.
+
This option is preferred over the platform specific
options; the platform specific options are deprecated
and will soon be removed.
@@ -186,6 +200,9 @@ config EARLY_UART_CADENCE
 config EARLY_UART_EXYNOS4210
select EARLY_PRINTK
bool
+config EARLY_UART_IMX_LPUART
+   select EARLY_PRINTK
+   bool
 config EARLY_UART_MESON
select EARLY_PRINTK
bool
@@ -283,6 +300,7 @@ config EARLY_PRINTK_INC
default "debug-8250.inc" if EARLY_UART_8250
default "debug-cadence.inc" if EARLY_UART_CADENCE
default "debug-exynos4210.inc" if EARLY_UART_EXYNOS4210
+   default "debug-imx-lpuart.inc" if EARLY_UART_IMX_LPUART
default "debug-meson.inc" if EARLY_UART_MESON
default "debug-mvebu.inc" if EARLY_UART_MVEBU
default "debug-pl011.inc" if EARLY_UART_PL011
diff --git a/xen/arch/arm/arm64/debug-imx-lpuart.inc 
b/xen/arch/arm/arm64/debug-imx-lpuart.inc
new file mode 100644
index 00..7510210d46
--- /dev/null
+++ b/xen/arch/arm/arm64/debug-imx-lpuart.inc
@@ -0,0 +1,48 @@
+/*
+ * xen/arch/arm/arm64/debug-imx8qm.inc
+ *
+ * i.MX8QM specific debug code
+ *
+ * Peng Fan 
+ * Copyright (C) 2016 Freescale Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+.macro early_uart_init wb wc wd
+/* Already initialized in bootloader */
+.endm
+/* i.MX8QM wait LPUART to be ready to transmit
+ * rb: register which contains the UART base address
+ * rc: scratch register
+ */
+.macro early_uart_ready xb, c
+1:
+ldr   w\c, [\xb, #UARTSTAT]   /* <- Flag register */
+tst   w\c, #UARTSTAT_TDRE /* Check FIFO EMPTY bit */
+beq   1b  /* Wait for the UART to be ready */
+.endm
+
+/* i.MX8QM LPUART transmit character
+ * rb: register which contains the UART base address
+ * rt: register which contains the character to transmit */
+.macro early_uart_transmit xb, wt
+str   \wt, [\xb, #UARTDATA]  /* -> Data Register */
+.endm
+
+/*
+ * Local variables:
+ * mode: ASM
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.30.0

[PATCH 0/3] xen/arm: add i.MX lpuart and i.MX8QM initial support

[Peng Fan (OSS)]

From: Peng Fan 

Add i.MX lpuart driver and i.MX8QM platform support.
 - lpuart is the uart IP used in i.MX8QM/QXP/93.
 - Very basic i.MX8QM platform support.

Peng Fan (3):
  xen/arm: Add i.MX lpuart driver
  xen/arm: Add i.MX lpuart early printk support
  xen/arm: Add i.MX8QM platform support

 xen/arch/arm/Kconfig.debug  |  21 ++
 xen/arch/arm/arm64/debug-imx-lpuart.inc |  48 
 xen/arch/arm/platforms/Makefile |   1 +
 xen/arch/arm/platforms/imx8qm.c |  44 
 xen/drivers/char/Kconfig|   8 +
 xen/drivers/char/Makefile   |   1 +
 xen/drivers/char/imx-lpuart.c   | 303 
 xen/include/xen/imx-lpuart.h|  64 +
 8 files changed, 490 insertions(+)
 create mode 100644 xen/arch/arm/arm64/debug-imx-lpuart.inc
 create mode 100644 xen/arch/arm/platforms/imx8qm.c
 create mode 100644 xen/drivers/char/imx-lpuart.c
 create mode 100644 xen/include/xen/imx-lpuart.h

-- 
2.30.0

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> 
> Sorry for the late reply -- a couple of conferences kept me busy.
> 
> 
> On Wed, 1 Jul 2020, Michael S. Tsirkin wrote:
> > On Wed, Jul 01, 2020 at 10:34:53AM -0700, Stefano Stabellini wrote:
> > > Would you be in favor of a more flexible check along the lines of
> > > the one proposed in the patch that started this thread:
> > >
> > > if (xen_vring_use_dma())
> > > return true;
> > >
> > >
> > > xen_vring_use_dma would be implemented so that it returns true when
> > > xen_swiotlb is required and false otherwise.
> >
> > Just to stress - with a patch like this virtio can *still* use DMA API
> > if PLATFORM_ACCESS is set. So if DMA API is broken on some platforms
> > as you seem to be saying, you guys should fix it before doing
> > something like this..
> 
> Yes, DMA API is broken with some interfaces (specifically: rpmesg and trusty),
> but for them PLATFORM_ACCESS is never set. That is why the errors weren't
> reported before. Xen special case aside, there is no problem under normal
> circumstances.
> 
> 
> If you are OK with this patch (after a little bit of clean-up), Peng, are you 
> OK
> with sending an update or do you want me to?

If you could help, that would be great. You have more expertise in knowing
the whole picture.

Thanks,
Peng.

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: RE: [PATCH] xen: introduce xen_vring_use_dma
> 
> On Mon, 29 Jun 2020, Peng Fan wrote:
> > > > If that is the case, how is it possible that virtio breaks on ARM
> > > > using the default dma_ops? The breakage is not Xen related (except
> > > > that Xen turns dma_ops on). The original message from Peng was:
> > > >
> > > >   vring_map_one_sg -> vring_use_dma_api
> > > >-> dma_map_page
> > > >-> __swiotlb_map_page
> > > > ->swiotlb_map_page
> > > > 
> > > > ->__dma_map_area(phys_to_virt(dma_to_phys(dev,
> > > dev_addr)), size, dir);
> > > >   However we are using per device dma area for rpmsg, phys_to_virt
> > > >   could not return a correct virtual address for virtual address in
> > > >   vmalloc area. Then kernel panic.
> > > >
> > > > I must be missing something. Maybe it is because it has to do with
> RPMesg?
> > >
> > > I think it's an RPMesg bug, yes
> >
> > rpmsg bug is another issue, it should not use dma_alloc_coherent for
> > reserved area, and use vmalloc_to_page.
> >
> > Anyway here using dma api will also trigger issue.
> 
> Is the stack trace above for the RPMesg issue or for the Trusty issue?

The stack trace you pasted is rpmsg issue.

> If it is the stack trace for RPMesg, can you also post the Trusty stack 
> trace? Or
> are they indentical?

There is no stack dump here. It successfully using swiotlb to do a map,
but we actually no need swiotlb in domu to do the map.

Thanks,
Peng.

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> 
> On Mon, Jun 29, 2020 at 06:25:41AM +0000, Peng Fan wrote:
> > > > > > Anyway, re-reading the last messages of the original thread
> > > > > > [1], it looks like Peng had a clear idea on how to fix the general 
> > > > > > issue.
> > > > > > Peng, what happened with that?
> > > >
> > > > We shrinked the rpmsg reserved area to workaround the issue.
> > > > So still use the dma apis in rpmsg.
> > > >
> > > > But here I am going to address domu android trusty issue using virtio.
> > >
> > > My suggestion is to first of all fix DMA API so it works properly.
> >
> > Could you please elaborate more details?
> >
> > You mean the DMA API usage of rpmsg? Or xen domu dma_ops?
> >
> > Thanks,
> > Peng.
> 
> Not 100% sure but I think xen dma ops.

Sorry to ask again, could you explain more details about what issues
do you see of current xen ARM domu dma_ops? 
Or Dom0 swiotlb xen dma_ops?

Thanks,
Pen.g

> 
> --
> MST

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> 
> On Mon, Jun 29, 2020 at 03:05:19AM +0000, Peng Fan wrote:
> > > Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> > >
> > > On Thu, Jun 25, 2020 at 10:31:27AM -0700, Stefano Stabellini wrote:
> > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > On Wed, Jun 24, 2020 at 02:53:54PM -0700, Stefano Stabellini wrote:
> > > > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > > > On Wed, Jun 24, 2020 at 10:59:47AM -0700, Stefano Stabellini
> wrote:
> > > > > > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > > > > > On Wed, Jun 24, 2020 at 05:17:32PM +0800, Peng Fan wrote:
> > > > > > > > > > Export xen_swiotlb for all platforms using xen swiotlb
> > > > > > > > > >
> > > > > > > > > > Use xen_swiotlb to determine when vring should use dma
> > > > > > > > > > APIs to map the
> > > > > > > > > > ring: when xen_swiotlb is enabled the dma API is required.
> > > > > > > > > > When it is disabled, it is not required.
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Peng Fan 
> > > > > > > > >
> > > > > > > > > Isn't there some way to use VIRTIO_F_IOMMU_PLATFORM for
> > > this?
> > > > > > > > > Xen was there first, but everyone else is using that now.
> > > > > > > >
> > > > > > > > Unfortunately it is complicated and it is not related to
> > > > > > > > VIRTIO_F_IOMMU_PLATFORM :-(
> > > > > > > >
> > > > > > > >
> > > > > > > > The Xen subsystem in Linux uses dma_ops via swiotlb_xen to
> > > > > > > > translate foreign mappings (memory coming from other VMs)
> > > > > > > > to
> > > physical addresses.
> > > > > > > > On x86, it also uses dma_ops to translate Linux's idea of
> > > > > > > > a physical address into a real physical address (this is
> > > > > > > > unneeded on ARM.)
> > > > > > > >
> > > > > > > >
> > > > > > > > So regardless of VIRTIO_F_IOMMU_PLATFORM, dma_ops should
> > > > > > > > be used on Xen/x86 always and on Xen/ARM if Linux is Dom0
> > > > > > > > (because it has foreign
> > > > > > > > mappings.) That is why we have the if (xen_domain) return
> > > > > > > > true; in vring_use_dma_api.
> > > > > > >
> > > > > > > VIRTIO_F_IOMMU_PLATFORM makes guest always use DMA ops.
> > > > > > >
> > > > > > > Xen hack predates VIRTIO_F_IOMMU_PLATFORM so it *also*
> > > > > > > forces DMA ops even if VIRTIO_F_IOMMU_PLATFORM is clear.
> > > > > > >
> > > > > > > Unfortunately as a result Xen never got around to properly
> > > > > > > setting VIRTIO_F_IOMMU_PLATFORM.
> > > > > >
> > > > > > I don't think VIRTIO_F_IOMMU_PLATFORM would be correct for
> > > > > > this because the usage of swiotlb_xen is not a property of
> > > > > > virtio,
> > > > >
> > > > >
> > > > > Basically any device without VIRTIO_F_ACCESS_PLATFORM (that is
> > > > > it's name in latest virtio spec, VIRTIO_F_IOMMU_PLATFORM is what
> > > > > linux calls it) is declared as "special, don't follow normal
> > > > > rules for access".
> > > > >
> > > > > So yes swiotlb_xen is not a property of virtio, but what *is* a
> > > > > property of virtio is that it's not special, just a regular
> > > > > device from DMA
> > > POV.
> > > >
> > > > I am trying to understand what you meant but I think I am missing
> > > > something.
> > > >
> > > > Are you saying that modern virtio should always have
> > > > VIRTIO_F_ACCESS_PLATFORM, hence use normal dma_ops as any other
> > > devices?
> > >
> > > I am saying it's a safe default. Clear VIRTIO_F_ACCESS_PLATFORM if
> > > you have some special needs e.g. you are very sure it's ok t

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> 
> On Thu, Jun 25, 2020 at 10:31:27AM -0700, Stefano Stabellini wrote:
> > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > On Wed, Jun 24, 2020 at 02:53:54PM -0700, Stefano Stabellini wrote:
> > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > On Wed, Jun 24, 2020 at 10:59:47AM -0700, Stefano Stabellini wrote:
> > > > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > > > On Wed, Jun 24, 2020 at 05:17:32PM +0800, Peng Fan wrote:
> > > > > > > > Export xen_swiotlb for all platforms using xen swiotlb
> > > > > > > >
> > > > > > > > Use xen_swiotlb to determine when vring should use dma
> > > > > > > > APIs to map the
> > > > > > > > ring: when xen_swiotlb is enabled the dma API is required.
> > > > > > > > When it is disabled, it is not required.
> > > > > > > >
> > > > > > > > Signed-off-by: Peng Fan 
> > > > > > >
> > > > > > > Isn't there some way to use VIRTIO_F_IOMMU_PLATFORM for
> this?
> > > > > > > Xen was there first, but everyone else is using that now.
> > > > > >
> > > > > > Unfortunately it is complicated and it is not related to
> > > > > > VIRTIO_F_IOMMU_PLATFORM :-(
> > > > > >
> > > > > >
> > > > > > The Xen subsystem in Linux uses dma_ops via swiotlb_xen to
> > > > > > translate foreign mappings (memory coming from other VMs) to
> physical addresses.
> > > > > > On x86, it also uses dma_ops to translate Linux's idea of a
> > > > > > physical address into a real physical address (this is
> > > > > > unneeded on ARM.)
> > > > > >
> > > > > >
> > > > > > So regardless of VIRTIO_F_IOMMU_PLATFORM, dma_ops should be
> > > > > > used on Xen/x86 always and on Xen/ARM if Linux is Dom0
> > > > > > (because it has foreign
> > > > > > mappings.) That is why we have the if (xen_domain) return
> > > > > > true; in vring_use_dma_api.
> > > > >
> > > > > VIRTIO_F_IOMMU_PLATFORM makes guest always use DMA ops.
> > > > >
> > > > > Xen hack predates VIRTIO_F_IOMMU_PLATFORM so it *also* forces
> > > > > DMA ops even if VIRTIO_F_IOMMU_PLATFORM is clear.
> > > > >
> > > > > Unfortunately as a result Xen never got around to properly
> > > > > setting VIRTIO_F_IOMMU_PLATFORM.
> > > >
> > > > I don't think VIRTIO_F_IOMMU_PLATFORM would be correct for this
> > > > because the usage of swiotlb_xen is not a property of virtio,
> > >
> > >
> > > Basically any device without VIRTIO_F_ACCESS_PLATFORM (that is it's
> > > name in latest virtio spec, VIRTIO_F_IOMMU_PLATFORM is what linux
> > > calls it) is declared as "special, don't follow normal rules for
> > > access".
> > >
> > > So yes swiotlb_xen is not a property of virtio, but what *is* a
> > > property of virtio is that it's not special, just a regular device from 
> > > DMA
> POV.
> >
> > I am trying to understand what you meant but I think I am missing
> > something.
> >
> > Are you saying that modern virtio should always have
> > VIRTIO_F_ACCESS_PLATFORM, hence use normal dma_ops as any other
> devices?
> 
> I am saying it's a safe default. Clear VIRTIO_F_ACCESS_PLATFORM if you have
> some special needs e.g. you are very sure it's ok to bypass DMA ops, or you
> need to support a legacy guest (produced in the window between virtio 1
> support in 2014 and support for VIRTIO_F_ACCESS_PLATFORM in 2016).
> 
> 
> > If that is the case, how is it possible that virtio breaks on ARM
> > using the default dma_ops? The breakage is not Xen related (except
> > that Xen turns dma_ops on). The original message from Peng was:
> >
> >   vring_map_one_sg -> vring_use_dma_api
> >-> dma_map_page
> >-> __swiotlb_map_page
> > ->swiotlb_map_page
> > ->__dma_map_area(phys_to_virt(dma_to_phys(dev,
> dev_addr)), size, dir);
> >   However we are using per device dma area for rpmsg, phys_to_virt
> >   could not return a correct virtual address for virtu

RE: [PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

> Subject: Re: [PATCH] xen: introduce xen_vring_use_dma
> 
> On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > On Wed, Jun 24, 2020 at 02:53:54PM -0700, Stefano Stabellini wrote:
> > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > On Wed, Jun 24, 2020 at 10:59:47AM -0700, Stefano Stabellini wrote:
> > > > > On Wed, 24 Jun 2020, Michael S. Tsirkin wrote:
> > > > > > On Wed, Jun 24, 2020 at 05:17:32PM +0800, Peng Fan wrote:
> > > > > > > Export xen_swiotlb for all platforms using xen swiotlb
> > > > > > >
> > > > > > > Use xen_swiotlb to determine when vring should use dma APIs
> > > > > > > to map the
> > > > > > > ring: when xen_swiotlb is enabled the dma API is required.
> > > > > > > When it is disabled, it is not required.
> > > > > > >
> > > > > > > Signed-off-by: Peng Fan 
> > > > > >
> > > > > > Isn't there some way to use VIRTIO_F_IOMMU_PLATFORM for this?
> > > > > > Xen was there first, but everyone else is using that now.
> > > > >
> > > > > Unfortunately it is complicated and it is not related to
> > > > > VIRTIO_F_IOMMU_PLATFORM :-(
> > > > >
> > > > >
> > > > > The Xen subsystem in Linux uses dma_ops via swiotlb_xen to
> > > > > translate foreign mappings (memory coming from other VMs) to
> physical addresses.
> > > > > On x86, it also uses dma_ops to translate Linux's idea of a
> > > > > physical address into a real physical address (this is unneeded
> > > > > on ARM.)
> > > > >
> > > > >
> > > > > So regardless of VIRTIO_F_IOMMU_PLATFORM, dma_ops should be
> used
> > > > > on Xen/x86 always and on Xen/ARM if Linux is Dom0 (because it
> > > > > has foreign
> > > > > mappings.) That is why we have the if (xen_domain) return true;
> > > > > in vring_use_dma_api.
> > > >
> > > > VIRTIO_F_IOMMU_PLATFORM makes guest always use DMA ops.
> > > >
> > > > Xen hack predates VIRTIO_F_IOMMU_PLATFORM so it *also* forces
> DMA
> > > > ops even if VIRTIO_F_IOMMU_PLATFORM is clear.
> > > >
> > > > Unfortunately as a result Xen never got around to properly setting
> > > > VIRTIO_F_IOMMU_PLATFORM.
> > >
> > > I don't think VIRTIO_F_IOMMU_PLATFORM would be correct for this
> > > because the usage of swiotlb_xen is not a property of virtio,
> >
> >
> > Basically any device without VIRTIO_F_ACCESS_PLATFORM (that is it's
> > name in latest virtio spec, VIRTIO_F_IOMMU_PLATFORM is what linux
> > calls it) is declared as "special, don't follow normal rules for
> > access".
> >
> > So yes swiotlb_xen is not a property of virtio, but what *is* a
> > property of virtio is that it's not special, just a regular device from DMA 
> > POV.
> 
> I am trying to understand what you meant but I think I am missing something.
> 
> Are you saying that modern virtio should always have
> VIRTIO_F_ACCESS_PLATFORM, hence use normal dma_ops as any other
> devices?
> 
> If that is the case, how is it possible that virtio breaks on ARM using the
> default dma_ops? The breakage is not Xen related (except that Xen turns
> dma_ops on). The original message from Peng was:
> 
>   vring_map_one_sg -> vring_use_dma_api
>-> dma_map_page
>  -> __swiotlb_map_page
>   ->swiotlb_map_page
>   ->__dma_map_area(phys_to_virt(dma_to_phys(dev,
> dev_addr)), size, dir);
>   However we are using per device dma area for rpmsg, phys_to_virt
>   could not return a correct virtual address for virtual address in
>   vmalloc area. Then kernel panic.
> 
> I must be missing something. Maybe it is because it has to do with RPMesg?

I am not going to fix the rpmsg issue with this patch. It is when ARM DomU
Android os communicate with secure world trusty os using virtio, the
vring_use_dma_api will return true for xen domu, but I no need it return
true and fall into swiotlb.

Thanks,
Peng.

> 
> 
> > > > > You might have noticed that I missed one possible case above:
> > > > > Xen/ARM DomU :-)
> > > > >
> > > > > Xen/ARM domUs don't need swiotlb_xen, it is not even
> > > > > initialized. So if
> > > > > (xen_domain) return true; would give the wrong answer i

[PATCH] xen: introduce xen_vring_use_dma

[Peng Fan]

Export xen_swiotlb for all platforms using xen swiotlb

Use xen_swiotlb to determine when vring should use dma APIs to map the
ring: when xen_swiotlb is enabled the dma API is required. When it is
disabled, it is not required.

Signed-off-by: Peng Fan 
---

V2:
 This is a modified version from Stefano's patch
 https://lore.kernel.org/patchwork/patch/1033801/#1222404
 Note: This is not to address rpmsg virtio issue, this is
 to let DomU virtio not using xen swiotlb could use non dma vring
 on ARM64 platforms.

 arch/arm/xen/mm.c  | 1 +
 arch/x86/include/asm/xen/swiotlb-xen.h | 2 --
 arch/x86/xen/pci-swiotlb-xen.c | 2 --
 drivers/virtio/virtio_ring.c   | 2 +-
 drivers/xen/swiotlb-xen.c  | 3 +++
 include/xen/swiotlb-xen.h  | 6 ++
 include/xen/xen.h  | 6 ++
 7 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index d40e9e5fc52b..6a493ea087f0 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -139,6 +139,7 @@ static int __init xen_mm_init(void)
struct gnttab_cache_flush cflush;
if (!xen_initial_domain())
return 0;
+   xen_swiotlb = 1;
xen_swiotlb_init(1, false);
 
cflush.op = 0;
diff --git a/arch/x86/include/asm/xen/swiotlb-xen.h 
b/arch/x86/include/asm/xen/swiotlb-xen.h
index 6b56d0d45d15..bb5ce02b4e20 100644
--- a/arch/x86/include/asm/xen/swiotlb-xen.h
+++ b/arch/x86/include/asm/xen/swiotlb-xen.h
@@ -3,12 +3,10 @@
 #define _ASM_X86_SWIOTLB_XEN_H
 
 #ifdef CONFIG_SWIOTLB_XEN
-extern int xen_swiotlb;
 extern int __init pci_xen_swiotlb_detect(void);
 extern void __init pci_xen_swiotlb_init(void);
 extern int pci_xen_swiotlb_init_late(void);
 #else
-#define xen_swiotlb (0)
 static inline int __init pci_xen_swiotlb_detect(void) { return 0; }
 static inline void __init pci_xen_swiotlb_init(void) { }
 static inline int pci_xen_swiotlb_init_late(void) { return -ENXIO; }
diff --git a/arch/x86/xen/pci-swiotlb-xen.c b/arch/x86/xen/pci-swiotlb-xen.c
index 33293ce01d8d..071fbe0e1a91 100644
--- a/arch/x86/xen/pci-swiotlb-xen.c
+++ b/arch/x86/xen/pci-swiotlb-xen.c
@@ -18,8 +18,6 @@
 #endif
 #include 
 
-int xen_swiotlb __read_mostly;
-
 /*
  * pci_xen_swiotlb_detect - set xen_swiotlb to 1 if necessary
  *
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index a2de775801af..768afd79f67a 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -252,7 +252,7 @@ static bool vring_use_dma_api(struct virtio_device *vdev)
 * the DMA API if we're a Xen guest, which at least allows
 * all of the sensible Xen configurations to work correctly.
 */
-   if (xen_domain())
+   if (xen_vring_use_dma())
return true;
 
return false;
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index b6d27762c6f8..25747e72e6fe 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -40,6 +40,9 @@
 
 #include 
 #define MAX_DMA_BITS 32
+
+int xen_swiotlb __read_mostly;
+
 /*
  * Used to do a quick range check in swiotlb_tbl_unmap_single and
  * swiotlb_tbl_sync_single_*, to see if the memory was in fact allocated by 
this
diff --git a/include/xen/swiotlb-xen.h b/include/xen/swiotlb-xen.h
index ffc0d3902b71..235babcde848 100644
--- a/include/xen/swiotlb-xen.h
+++ b/include/xen/swiotlb-xen.h
@@ -12,4 +12,10 @@ void xen_dma_sync_for_device(dma_addr_t handle, phys_addr_t 
paddr, size_t size,
 extern int xen_swiotlb_init(int verbose, bool early);
 extern const struct dma_map_ops xen_swiotlb_dma_ops;
 
+#ifdef CONFIG_SWIOTLB_XEN
+extern int xen_swiotlb;
+#else
+#define xen_swiotlb (0)
+#endif
+
 #endif /* __LINUX_SWIOTLB_XEN_H */
diff --git a/include/xen/xen.h b/include/xen/xen.h
index 19a72f591e2b..c51c46f5d739 100644
--- a/include/xen/xen.h
+++ b/include/xen/xen.h
@@ -52,4 +52,10 @@ bool xen_biovec_phys_mergeable(const struct bio_vec *vec1,
 extern u64 xen_saved_max_mem_size;
 #endif
 
+#include 
+static inline int xen_vring_use_dma(void)
+{
+   return !!xen_swiotlb;
+}
+
 #endif /* _XEN_XEN_H */
-- 
2.16.4

RE: UEFI support in ARM DomUs

[Peng Fan]

> Subject: Re: UEFI support in ARM DomUs
> 
> 
> On 6/24/20 10:07 AM, Peng Fan wrote:
> >> Subject: Re: UEFI support in ARM DomUs
> >>
> >>
> >> On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
> >>> On 6/23/20 4:20 AM, Stefano Stabellini wrote:
> >>>> On Mon, 22 Jun 2020, Julien Grall wrote:
> >>>>>>>> For the first part (__XEN_INTERFACE_VERSION__) I think we can
> >>>>>>>> provide it via
> >>>>>>>>
> >>>>>>>> CFLAGS or something. This can also be done for the location of
> >>>>>>>> Xen headers.
> >>>>>>> __XEN_INTERFACE_VERSION__ should work through the CFLAGS.
> An
> >>>>>>> alternative would be to allow the user to specify through the
> Kconfig.
> >>>>>> You mean specifying via Kconfig something like "0x00040d00"?
> >>>>> Possibly yes.
> >>>>>
> >>>>>> And what about the headers? How will we provide their location if
> >>>>>> we decide not to include those
> >>>>>>
> >>>>>> in the tree?
> >>>>> I would do through Kconfig as well.
> >>>> If we specify the external location of the Xen headers via Kconfig,
> >>>> it seems to me that we should be able to detect the interface
> >>>> version automatically from any Makefile as part of the build. No
> >>>> need to ask the user.
> >>>>
> >>>> However, if Oleksandr is thinking of using the Xen headers for the
> >>>> hypercalls definitions, then I think we might not need external
> >>>> headers at all because that is a stable interface, as Julien wrote.
> >>>> We could just define our own few headers for just what you need
> >>>> like Linux
> >> does.
> >>> This is a good idea: I'll try to get the minimal set of headers from
> >>> Linux
> >>>
> >>> instead of Xen as those seem to be well prepared for such a use-case.
> >>> This
> >>>
> >>> way we'll have headers in U-boot tree and guarantee that we have a
> >>> minimal
> >>>
> >>> subset which is easier to maintain. I'll keep you updated on the
> >>> progress
> >> We've managed to strip the headers and remove __XEN__ and the rest
> >> definitions
> >>
> >> we were talking about. So, these are now the minimal required set of
> >> headers
> >>
> >> that allows U-boot to build serial and block drivers. Please take a
> >> look at [1]
> >>
> >> Pull request for comments is at [2]
> > The U-Boot new merge window will be open in 2020/7/1, so I'd suggest
> > the patchset goes to U-Boot mail list for discussion if you wanna the
> > patches gonna merged soon.
> 
> We definitely want the patches to be upstreamed and merged, but before
> that
> 
> we also want to make sure that Xen community is happy with what we
> upstream
> 
> I believe we resolved most of the concerns such as headers, PIE etc, so this
> can
> 
> be done.
> 
> BTW, Peng, did you have a chance to try the pvblock driver yet?

Not yet. I could have time to give a test next Monday. I think you not
enable SPL, right? So android dual bootloader feature not enabled.
But SPL mostly not have MMU enabled..

Regards,
Peng.

> 
> >
> > Regards,
> > Peng.
> >
> >>>> If you can do that, I think it would be better because we decouple
> >>>> the UBoot build from the Xen build completely. We don't even need
> >>>> the Xen tree checked out to build UBoot. It would be a huge
> >>>> advantage because it makes it far easier to build-test changes for
> >>>> others in the community that don't know about Xen and also it
> >>>> becomes far easier to integrate into any build system.
> >> [1]
> >>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldef
> ense.com%2Fv3%2F__https%3A%2F%2Feur01.safelinks.protection.outlook.c
> om%2F%3Furl%3Dhttps*3A*2F*2Fgithub__%3BJSUl!!GF_29dbcQIUBPA!mwi
> b3un6-vYBG68zMfurW6-0MuuER5tNmJtOitDpViICNkX0lhig8iPHmZokuM-BLQ
> YpeKYAGQ%24data=02%7C01%7Cpeng.fan%40nxp.com%7C950d9c0a
> dc514927ce8b08d8180ea4ef%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C
> 0%7C0%7C637285798460563914sdata=fMrI%2FQotknCsXV0amC4BFl
> 1Hg4vPw3zOMVdAVim2Wcs%3Dreserved=0 .
> >>
> com%2Fandr2000%2Fu-boot%2Ftree%2Fpvblock_upstream_v1data=0
> >>
> 2%7C01%7Cpeng.fan%40nxp.com%7C407d8af24a36483fbdce08d81805ed88
> >> %7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637285761021
> 975
> >>
> 164sdata=5vWfBbLScICXPZWU%2BU3b7DyONcgxT8iICsxrwUbORZY%
> >> 3Dreserved=0
> >>
> >> [2]
> >>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldef
> ense.com%2Fv3%2F__https%3A%2F%2Feur01.safelinks.protection.outlook.c
> om%2F%3Furl%3Dhttps*3A*2F*2Fgithub__%3BJSUl!!GF_29dbcQIUBPA!mwi
> b3un6-vYBG68zMfurW6-0MuuER5tNmJtOitDpViICNkX0lhig8iPHmZokuM-BLQ
> YpeKYAGQ%24data=02%7C01%7Cpeng.fan%40nxp.com%7C950d9c0a
> dc514927ce8b08d8180ea4ef%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C
> 0%7C0%7C637285798460563914sdata=fMrI%2FQotknCsXV0amC4BFl
> 1Hg4vPw3zOMVdAVim2Wcs%3Dreserved=0 .
> >>
> com%2Fxen-troops%2Fu-boot%2Fpull%2F2data=02%7C01%7Cpeng.fa
> >>
> n%40nxp.com%7C407d8af24a36483fbdce08d81805ed88%7C686ea1d3bc2b4
> >>
> c6fa92cd99c5c301635%7C0%7C0%7C637285761021975164sdata=%2
> >>
> FmXheEvKssLjjaFKsHBBbqh%2B72jH3uQnE7cpN0J3k8I%3Dreserved=0

RE: UEFI support in ARM DomUs

[Peng Fan]

> Subject: Re: UEFI support in ARM DomUs
> 
> 
> On 6/23/20 8:31 AM, Oleksandr Andrushchenko wrote:
> >
> > On 6/23/20 4:20 AM, Stefano Stabellini wrote:
> >> On Mon, 22 Jun 2020, Julien Grall wrote:
> >> For the first part (__XEN_INTERFACE_VERSION__) I think we can
> >> provide it via
> >>
> >> CFLAGS or something. This can also be done for the location of
> >> Xen headers.
> > __XEN_INTERFACE_VERSION__ should work through the CFLAGS. An
> > alternative would be to allow the user to specify through the Kconfig.
>  You mean specifying via Kconfig something like "0x00040d00"?
> >>> Possibly yes.
> >>>
>  And what about the headers? How will we provide their location if
>  we decide not to include those
> 
>  in the tree?
> >>> I would do through Kconfig as well.
> >> If we specify the external location of the Xen headers via Kconfig,
> >> it seems to me that we should be able to detect the interface version
> >> automatically from any Makefile as part of the build. No need to ask
> >> the user.
> >>
> >> However, if Oleksandr is thinking of using the Xen headers for the
> >> hypercalls definitions, then I think we might not need external
> >> headers at all because that is a stable interface, as Julien wrote.
> >> We could just define our own few headers for just what you need like Linux
> does.
> >
> > This is a good idea: I'll try to get the minimal set of headers from
> > Linux
> >
> > instead of Xen as those seem to be well prepared for such a use-case.
> > This
> >
> > way we'll have headers in U-boot tree and guarantee that we have a
> > minimal
> >
> > subset which is easier to maintain. I'll keep you updated on the
> > progress
> 
> We've managed to strip the headers and remove __XEN__ and the rest
> definitions
> 
> we were talking about. So, these are now the minimal required set of headers
> 
> that allows U-boot to build serial and block drivers. Please take a look at 
> [1]
> 
> Pull request for comments is at [2]

The U-Boot new merge window will be open in 2020/7/1, so I'd suggest
the patchset goes to U-Boot mail list for discussion if you wanna the patches
gonna merged soon.

Regards,
Peng.

> 
> >
> >>
> >> If you can do that, I think it would be better because we decouple
> >> the UBoot build from the Xen build completely. We don't even need the
> >> Xen tree checked out to build UBoot. It would be a huge advantage
> >> because it makes it far easier to build-test changes for others in
> >> the community that don't know about Xen and also it becomes far
> >> easier to integrate into any build system.
> 
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
> com%2Fandr2000%2Fu-boot%2Ftree%2Fpvblock_upstream_v1data=0
> 2%7C01%7Cpeng.fan%40nxp.com%7C407d8af24a36483fbdce08d81805ed88
> %7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637285761021975
> 164sdata=5vWfBbLScICXPZWU%2BU3b7DyONcgxT8iICsxrwUbORZY%
> 3Dreserved=0
> 
> [2]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.
> com%2Fxen-troops%2Fu-boot%2Fpull%2F2data=02%7C01%7Cpeng.fa
> n%40nxp.com%7C407d8af24a36483fbdce08d81805ed88%7C686ea1d3bc2b4
> c6fa92cd99c5c301635%7C0%7C0%7C637285761021975164sdata=%2
> FmXheEvKssLjjaFKsHBBbqh%2B72jH3uQnE7cpN0J3k8I%3Dreserved=0

RE: UEFI support in ARM DomUs

[Peng Fan]

> Subject: Re: UEFI support in ARM DomUs
> 
> 
> On 6/19/20 3:59 PM, Julien Grall wrote:
> > Hi,
> >
> > On 19/06/2020 13:51, Oleksandr Andrushchenko wrote:
> >> On 6/19/20 3:47 PM, Julien Grall wrote:
> >>> They will not be available from the fdt, but you can retrieve them with an
> hypervisor call (see HVM_PARAM_STORE_PFN,
> HVM_PARAM_CONSOLE_PFN).
> >> Yes, and it used in the relevant pieces of code (hyp calls)
> >>> One question though, why do you need to map them in advance?
> Couldn't you map them on demand?
> >>
> >> Well, we need to at least estimate the pg_table size so we can reserve and
> allocate memory later,
> >
> > Oh, so U-boot doesn't support runtime page-table table allocation. Is that
> right?
> As per my understanding no, we provide a memory map and the tables are
> allocated beforehand

You are right.

Regards,
Peng.
> >
> >>
> >> so I have to provide memory range from either by coding a constant or
> looking into the devtree at
> >>
> >> hypervisor { reg = <>; }. It is a bit tricky though
> >
> > Looking for a node in the device-tree shouldn't be too difficult given that 
> > you
> have fdt_* available.
> >
> > However, please not that  doesn't refer to the guest magic pages.
> Instead, it provides a region you can use for mapping the grant-table frames
> 
> Indeed, this is in my case 0x3800, but the magic is at 0x3900
> 
> So, I need the memory range set up beforehand, but I can't as there is no cute
> way to get that.
> 
> Of course, I can issue a hyp call to get HVM_PARAM_CONSOLE_PFN and use it
> as the base address,
> 
> but this smells like a hack. I can call other HVM_PARAM_ to get their pfns and
> set up the memory regions,
> 
> but this looks a bit weird. I need that constant badly ;)
> 
> >
> > Cheers,
> >

RE: [Tee-dev] TEE with XEN

[Peng Fan]

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi Peng,
> 
> On Fri, 19 Jun 2020 at 12:06, Peng Fan  wrote:
> >
> > > Subject: Re: [Tee-dev] TEE with XEN
> > >
> > >
> > >
> > > > On 19 Jun 2020, at 09:52, Peng Fan  wrote:
> > > >
> > > > Hi Bertrand,
> > > >
> > > >> Subject: Re: [Tee-dev] TEE with XEN
> > > >>
> > > >> Hi,
> > > >>
> > > >>> On 18 Jun 2020, at 19:05, Julien Grall  wrote:
> > > >>>
> > > >>> +Bertrand and Stefano
> > > >>>
> > > >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> > > >>>> Hi Peng,
> > > >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan 
> wrote:
> > > >>>>>
> > > >>>>> Hi All,
> > > >>>>>
> > > >>>>> While enabling trusty os with xen, I took same approach as
> > > >>>>> OP-TEE, with OP-TEE running in secure world. But I am also
> > > >>>>> thinking this might introduce potential issue is that secure
> > > >>>>> world OS communicate with
> > > >> DomU.
> > > >>>>> If there are some misbehavior in secure world OS, it might let
> > > >>>>> XEN hypervisor not work proper.
> > > >>>>>
> > > >>>>> In my setup, trusty os sometimes panic in secure world, xen
> > > >>>>> will not able to control the panic core anymore.
> > > >>>
> > > >>> May I ask in which case Trusty is panicking?
> > > >>
> > > >> In any case, optee should protect itself against this and it
> > > >> should be considered that optee is more priviledged then Xen.
> > > >> So if optee is crashing we cannot expect that Xen can recover or fix 
> > > >> it.
> > > >>
> > > >> I would more consider this as a bug that optee needs to be robust
> against.
> > > >
> > > > ok. I am not using OP-TEE, currently I use google trusty OS.
> > >
> > > Sorry i should have been more generic.
> > > Please read this as “Anything running in secure world”, being optee or
> trusty.
> > >
> > > >
> > > > I have two OS, Dom0 linux + DomU android auto.
> > > >
> > > > DomU android auto needs trusty OS, Dom0 Linux not need that.
> > >
> > > But i would guess your Dom0 is more “critical” then your DomU.
> > > In this case you must make sure that any resource given to your DomU
> > > cannot affect your Dom0.
> > > For example: if the DomU is starting a very heavy operation in
> > > blocked in trusty, any interrupt for non-secure could be blocked,
> > > thus affecting the ability of your Dom0.
> > >
> > > >
> > > > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> > > >
> > > > One more case is if dom0 linux needs OP-TEE, DomU needs google
> > > > trusty, how we handle this in one SoC?
> > >
> > > You have a shared resource in this case, someone more or as trusted
> > > as the clients needs to decide how the resource can be shared.
> > > In this case could be Dom0 or Xen or Trusty or op-Tee (if i should
> > > make an order).
> > >
> > > >
> > > >>
> > > >>>
> > > >>>>>
> > > >>>>> So I am thinking whether we need to emulating secure world in
> > > >>>>> a XEN VM which is the VM running DomU. Just like what ACRN did
> > > >>>>> to run trusty os.
> > > >>>> Well, it depends on whom you are trusting more. Both XEN and
> > > >>>> TEE are minimal OS implementations with aim at security. I'm
> > > >>>> speaking about generic TEE OS, not about particular OS like OP-TEE
> or Trusty.
> > > >>>> Problem is that, if TEE is running inside VM, it will be
> > > >>>> susceptible to a hypervisor misbehaviour. You need to
> > > >>>> understand that Xen and privileged domain (dom0, mostly) can
> > > >>>> access memory of
> > > any guest.
> > > >>>> At least, in default configuration. There are means to harden
> > > >>>> this setup. But anyways, Xen can't be stopped from reading TEE's
>

RE: [Tee-dev] TEE with XEN

[Peng Fan]

> Subject: Re: [Tee-dev] TEE with XEN
> 
> 
> 
> > On 19 Jun 2020, at 09:52, Peng Fan  wrote:
> >
> > Hi Bertrand,
> >
> >> Subject: Re: [Tee-dev] TEE with XEN
> >>
> >> Hi,
> >>
> >>> On 18 Jun 2020, at 19:05, Julien Grall  wrote:
> >>>
> >>> +Bertrand and Stefano
> >>>
> >>> On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> >>>> Hi Peng,
> >>>> On Mon, 15 Jun 2020 at 05:07, Peng Fan  wrote:
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> While enabling trusty os with xen, I took same approach as OP-TEE,
> >>>>> with OP-TEE running in secure world. But I am also thinking this
> >>>>> might introduce potential issue is that secure world OS
> >>>>> communicate with
> >> DomU.
> >>>>> If there are some misbehavior in secure world OS, it might let XEN
> >>>>> hypervisor not work proper.
> >>>>>
> >>>>> In my setup, trusty os sometimes panic in secure world, xen will
> >>>>> not able to control the panic core anymore.
> >>>
> >>> May I ask in which case Trusty is panicking?
> >>
> >> In any case, optee should protect itself against this and it should
> >> be considered that optee is more priviledged then Xen.
> >> So if optee is crashing we cannot expect that Xen can recover or fix it.
> >>
> >> I would more consider this as a bug that optee needs to be robust against.
> >
> > ok. I am not using OP-TEE, currently I use google trusty OS.
> 
> Sorry i should have been more generic.
> Please read this as “Anything running in secure world”, being optee or trusty.
> 
> >
> > I have two OS, Dom0 linux + DomU android auto.
> >
> > DomU android auto needs trusty OS, Dom0 Linux not need that.
> 
> But i would guess your Dom0 is more “critical” then your DomU.
> In this case you must make sure that any resource given to your DomU cannot
> affect your Dom0.
> For example: if the DomU is starting a very heavy operation in blocked in
> trusty, any interrupt for non-secure could be blocked, thus affecting the 
> ability
> of your Dom0.
> 
> >
> > I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
> >
> > One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
> > how we handle this in one SoC?
> 
> You have a shared resource in this case, someone more or as trusted as the
> clients needs to decide how the resource can be shared.
> In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an
> order).
> 
> >
> >>
> >>>
> >>>>>
> >>>>> So I am thinking whether we need to emulating secure world in a
> >>>>> XEN VM which is the VM running DomU. Just like what ACRN did to
> >>>>> run trusty os.
> >>>> Well, it depends on whom you are trusting more. Both XEN and TEE
> >>>> are minimal OS implementations with aim at security. I'm speaking
> >>>> about generic TEE OS, not about particular OS like OP-TEE or Trusty.
> >>>> Problem is that, if TEE is running inside VM, it will be
> >>>> susceptible to a hypervisor misbehaviour. You need to understand
> >>>> that Xen and privileged domain (dom0, mostly) can access memory of
> any guest.
> >>>> At least, in default configuration. There are means to harden this
> >>>> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
> >>>
> >>> IIRC, we discussed this approach for OP-TEE in the past. There was
> >>> other
> >> potential pitfalls with it. For instance, you wouldn't be able to
> >> directly access any secure device from that guest (it is running in
> non-secure world).
> >>>
> >>> There are also issues in term of latency as you may have the
> >>> following
> >> model:
> >>>
> >>> domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) ->
> >>> Xen ->
> >> domU.
> >>>
> >>> The bit in () is if you require to call the host TEE.
> >>>
> >>> One possibility would be to use Secure-EL2 for your Trusty OS. But I
> >>> don't
> >> know whether your platform supports it.
> >>>
> >>> Depending on whether you can modify Trusty OS, alternative would be
> >>> to
>

RE: [Tee-dev] TEE with XEN

[Peng Fan]

Hi Bertrand,

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi,
> 
> > On 18 Jun 2020, at 19:05, Julien Grall  wrote:
> >
> > +Bertrand and Stefano
> >
> > On 16/06/2020 02:24, Volodymyr Babchuk wrote:
> >> Hi Peng,
> >> On Mon, 15 Jun 2020 at 05:07, Peng Fan  wrote:
> >>>
> >>> Hi All,
> >>>
> >>> While enabling trusty os with xen, I took same approach as OP-TEE,
> >>> with OP-TEE running in secure world. But I am also thinking this
> >>> might introduce potential issue is that secure world OS communicate with
> DomU.
> >>> If there are some misbehavior in secure world OS, it might let XEN
> >>> hypervisor not work proper.
> >>>
> >>> In my setup, trusty os sometimes panic in secure world, xen will not
> >>> able to control the panic core anymore.
> >
> > May I ask in which case Trusty is panicking?
> 
> In any case, optee should protect itself against this and it should be
> considered that optee is more priviledged then Xen.
> So if optee is crashing we cannot expect that Xen can recover or fix it.
> 
> I would more consider this as a bug that optee needs to be robust against.

ok. I am not using OP-TEE, currently I use google trusty OS.

I have two OS, Dom0 linux + DomU android auto.

DomU android auto needs trusty OS, Dom0 Linux not need that.

I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.

One more case is if dom0 linux needs OP-TEE, DomU needs google trusty,
how we handle this in one SoC?

> 
> >
> >>>
> >>> So I am thinking whether we need to emulating secure world in a XEN
> >>> VM which is the VM running DomU. Just like what ACRN did to run
> >>> trusty os.
> >> Well, it depends on whom you are trusting more. Both XEN and TEE are
> >> minimal OS implementations with aim at security. I'm speaking about
> >> generic TEE OS, not about particular OS like OP-TEE or Trusty.
> >> Problem is that, if TEE is running inside VM, it will be susceptible
> >> to a hypervisor misbehaviour. You need to understand that Xen and
> >> privileged domain (dom0, mostly) can access memory of any guest.
> >> At least, in default configuration. There are means to harden this
> >> setup. But anyways, Xen can't be stopped from reading TEE's secrets.
> >
> > IIRC, we discussed this approach for OP-TEE in the past. There was other
> potential pitfalls with it. For instance, you wouldn't be able to directly 
> access
> any secure device from that guest (it is running in non-secure world).
> >
> > There are also issues in term of latency as you may have the following
> model:
> >
> > domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
> domU.
> >
> > The bit in () is if you require to call the host TEE.
> >
> > One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
> know whether your platform supports it.
> >
> > Depending on whether you can modify Trusty OS, alternative would be to
> make itvirtualization aware as OP-TEE did. The core would need to be
> resilient and the panic only affect a given client.
> 
> I do not have right a clear idea of what is the status of optee and xen but in
> theory I would see 2 possible ways to handle this:
> - without optee modification, something in a guest (Dom0 or an other
> priviledged one) needs to have access to optee and emulate optee access for
> others.
> - with optee modifications, optee needs to have a concept of client and Xen
> would need to passthrough optee requests but being responsible of adding a
> “client” identifier. Maybe also informing Optee when a new client is
> created/removed.
> 
> The second scenario could then be somehow splitted in the previous one from
> Julien if some parts would need to be emulated somewhere in some kind of
> combination of the 2 models.
> 
> In any case i would always consider that anything running on optee (or in
> general in the secure world) is more trusted then Xen.

Ok, this means optee runs on all cores in secure world, but this would not
work when we need to support multiple OSes with their own TEE.

Regards,
Peng.

> 
> Regards
> Bertrand

RE: [Tee-dev] TEE with XEN

[Peng Fan]

Hi,

> Subject: Re: [Tee-dev] TEE with XEN
> 
> Hi Peng,
> 
> On Mon, 15 Jun 2020 at 05:07, Peng Fan  wrote:
> >
> > Hi All,
> >
> > While enabling trusty os with xen, I took same approach as OP-TEE,
> > with OP-TEE running in secure world. But I am also thinking this might
> > introduce potential issue is that secure world OS communicate with DomU.
> > If there are some misbehavior in secure world OS, it might let XEN
> > hypervisor not work proper.
> >
> > In my setup, trusty os sometimes panic in secure world, xen will not
> > able to control the panic core anymore.
> >
> > So I am thinking whether we need to emulating secure world in a XEN VM
> > which is the VM running DomU. Just like what ACRN did to run trusty
> > os.
> 
> Well, it depends on whom you are trusting more. Both XEN and TEE are
> minimal OS implementations with aim at security. 

XEN is targeting safety.
TEE is targeting security.

I'm speaking about generic
> TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if 
> TEE is
> running inside VM, it will be susceptible to a hypervisor misbehaviour. You
> need to understand that Xen and privileged domain (dom0, mostly) can access
> memory of any guest.
> At least, in default configuration. There are means to harden this setup. But
> anyways, Xen can't be stopped from reading TEE's secrets.

Yes. Understand.

> 
> If this is okay for your needs, then you can run TEE as a VM of course.
> 
> So, this is heavilly depends on your security threats model. There can't be
> universal solution. Also, I'm proposing to check Google's requirements for
> Trusty environment.

Let me try to ask Google guys to see any feedback.

Thanks,
Peng.

> Do they allow it to run outside of TrustZone? For example, GPD TEE System
> Architecture document clearly says that TEE should be separated from REE by
> hardware mechanisms that are not controlled by REE (section 2.2.1). I believe,
> that should be a similar document for Trusty.
> 
> --
> WBR Volodymyr Babchuk aka lorc [+380976646013]
> mailto: vlad.babc...@gmail.com

TEE with XEN

[Peng Fan]

Hi All,

While enabling trusty os with xen, I took same approach as OP-TEE, 
with OP-TEE running in secure world. But I am also thinking this might
introduce potential issue is that secure world OS communicate with DomU.
If there are some misbehavior in secure world OS, it might let XEN
hypervisor not work proper.

In my setup, trusty os sometimes panic in secure world, xen will not able
to control the panic core anymore.

So I am thinking whether we need to emulating secure world in a XEN VM
which is the VM running DomU. Just like what ACRN did to run trusty
os.

Any comments?

Thanks,
Peng.

RE: UEFI support in ARM DomUs

[Peng Fan]

Hi Stefano,

> -Original Message-
> From: Stefano Stabellini [mailto:sstabell...@kernel.org]
> Sent: 2020年6月4日 23:32
> To: Oleksandr Andrushchenko 
> Cc: Peng Fan ; Roman Shaposhnik
> ; Xen-devel ; Stefano
> Stabellini ; Julien Grall ; Nataliya
> Korovkina 
> Subject: Re: UEFI support in ARM DomUs
> 
> On Thu, 4 Jun 2020, Oleksandr Andrushchenko wrote:
> > On 6/4/20 4:57 AM, Peng Fan wrote:
> > > Grall ;
> > >> Nataliya Korovkina 
> > >> Subject: UEFI support in ARM DomUs
> > > We have made U-Boot run inside XEN DomU, but just only PV console
> > > part, not implement other frontend drivers currently. Would this
> > > help for your case if enable EFI in U-Boot?
> >
> > Well, we have a working PV block implementation on top of that on iMX8
> >
> > platform, mostly ported from mini-os. Currently we are finalizing the
> > work
> >
> > and cleaning up (it's going to take a week or so hopefully). Then, we
> > we'll post
> >
> > it on our public github. We are also thinking about upstreaming the
> > work, but it may
> >
> > take quite some time if the whole idea fits u-boot's view on such an
> extension at all.
> 
> Yes please to both of you! :-)

The simple console part

https://source.codeaurora.org/external/imx/uboot-imx/tree/drivers/serial/serial_xen.c?h=imx_v2020.04_5.4.24_2.1.0

We enable U-Boot in DomU is mostly for android auto dual bootloader case.

It has the issue that only have console output after mmu enabled in U-Boot 
stage.

Regards,
Peng.

> 
> In the meantime, while we wait for those changes to go upstream in uboot,
> could you please post a branch on github and a link on this email thread?
> 
> Maybe we should have a wikipage on wiki.xenproject.org about
> work-in-progress uboot items.
> 
> 
> 
> 
> > > Regards,
> > > Peng.
> > >
> > >> Hi!
> > >>
> > >> with a lot of help from Stefano, we're getting RPi4 support in
> > >> Project EVE pretty much on par between KVM and Xen.
> > >>
> > >> One big area that still remains is supporting UEFI boot sequence for
> DomUs.
> > >> With KVM, given the qemu virt device model this is as simple as
> > >> using either stock UEFI build for arm or even U-Boot EFI emulation
> > >> environment and passing it via -bios option.
> > >>
> > >> Obviously with Xen on ARM we don't have the device model so my
> > >> understanding is that the easiest way we can support it would be to
> > >> port UEFI's OvmfPkg/OvmfXen target to ARM (it seems to be currently
> > >> exclusively X64).
> > >>
> > >> So here's my first question: if there's anybody on this list who
> > >> had a hand in implementing OvmfPkg/OvmfXen can you please share
> > >> your thoughts on how much work that port may be (or whether it is
> > >> even feasible -- I may totally be missing something really obvious here).
> > >>
> > >> And as long as I've got your attention: two more questions:
> > >> 1.. compared to the above, would porting pvgrub to ARM be any
> > >> easier or more difficult?
> > >>
> > >> 2. same question for teaching u-boot about PV calls.
> > >>
> > >> Thanks,
> > >> Roman.
> > >>
> > >> P.S. Oh and I guess between:
> > >> 0. OvmfPkg/OvmfXen on ARM64
> > >> 1. pvgrub on ARM64
> > >> 2. u-boot/EFI emulation with PV calls backend I didn't miss any
> > >> other obvious way of making UEFI-aware VM images to boot on Xen
> > >> ARM64 DomU, right?

RE: UEFI support in ARM DomUs

[Peng Fan]

Grall ;
> Nataliya Korovkina 
> Subject: UEFI support in ARM DomUs

We have made U-Boot run inside XEN DomU, but just only PV console part,
not implement other frontend drivers currently. Would this help for your
case if enable EFI in U-Boot?

Regards,
Peng.

> 
> Hi!
> 
> with a lot of help from Stefano, we're getting RPi4 support in Project EVE
> pretty much on par between KVM and Xen.
> 
> One big area that still remains is supporting UEFI boot sequence for DomUs.
> With KVM, given the qemu virt device model this is as simple as using either
> stock UEFI build for arm or even U-Boot EFI emulation environment and
> passing it via -bios option.
> 
> Obviously with Xen on ARM we don't have the device model so my
> understanding is that the easiest way we can support it would be to port
> UEFI's OvmfPkg/OvmfXen target to ARM (it seems to be currently exclusively
> X64).
> 
> So here's my first question: if there's anybody on this list who had a hand in
> implementing OvmfPkg/OvmfXen can you please share your thoughts on how
> much work that port may be (or whether it is even feasible -- I may totally be
> missing something really obvious here).
> 
> And as long as I've got your attention: two more questions:
>1.. compared to the above, would porting pvgrub to ARM be any
>easier or more difficult?
> 
>2. same question for teaching u-boot about PV calls.
> 
> Thanks,
> Roman.
> 
> P.S. Oh and I guess between:
>0. OvmfPkg/OvmfXen on ARM64
>1. pvgrub on ARM64
>2. u-boot/EFI emulation with PV calls backend I didn't miss any other
> obvious way of making UEFI-aware VM images to boot on Xen ARM64 DomU,
> right?

RE: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

[Peng Fan]

> Subject: Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU
> 
> On 06.05.20 00:34, Stefano Stabellini wrote:
> > + Boris, Jürgen
> >
> > See the crash Roman is seeing booting dom0 on the Raspberry Pi. It is
> > related to the recent xen dma_ops changes. Possibly the same thing
> > reported by Peng here:

Yes. It is same issue.

> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmarc
> > .info%2F%3Fl%3Dlinux-kernel%26m%3D158805976230485%26w%3D2
> p;data=02%
> >
> 7C01%7Cpeng.fan%40nxp.com%7Cab98b2db94484141a8ad08d7f1803372%7
> C686ea1d
> >
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637243405233572354sd
> ata=0Yr5h
> > Rg4%2FuApsBpTwIIL4StZU%2FUA55oP5Lfnfmtj4Hg%3Dreserved=0
> >
> > An in-depth analysis below.
> >
> >
> > On Mon, 4 May 2020, Roman Shaposhnik wrote:
>  [2.534292] Unable to handle kernel paging request at virtual
>  address 0026c340
>  [2.542373] Mem abort info:
>  [2.545257]   ESR = 0x9604
>  [2.548421]   EC = 0x25: DABT (current EL), IL = 32 bits
>  [2.553877]   SET = 0, FnV = 0
>  [2.557023]   EA = 0, S1PTW = 0
>  [2.560297] Data abort info:
>  [2.563258]   ISV = 0, ISS = 0x0004
>  [2.567208]   CM = 0, WnR = 0
>  [2.570294] [0026c340] user address but active_mm is
> swapper
>  [2.576783] Internal error: Oops: 9604 [#1] SMP
>  [2.581784] Modules linked in:
>  [2.584950] CPU: 3 PID: 135 Comm: kworker/3:1 Not tainted
> 5.6.1-default #9
>  [2.591970] Hardware name: Raspberry Pi 4 Model B (DT)
>  [2.597256] Workqueue: events deferred_probe_work_func
>  [2.602509] pstate: 6005 (nZCv daif -PAN -UAO)
>  [2.607431] pc : xen_swiotlb_free_coherent+0x198/0x1d8
>  [2.612696] lr : dma_free_attrs+0x98/0xd0
>  [2.616827] sp : 800011db3970
>  [2.620242] x29: 800011db3970 x28: 000f7b00
>  [2.625695] x27: 1000 x26: 37b68410
>  [2.631129] x25: 0001 x24: f7b0
>  [2.636583] x23:  x22: 
>  [2.642017] x21: 800011b0d000 x20: 80001179b548
>  [2.647461] x19: 37b68410 x18: 0010
>  [2.652905] x17: 35d66a00 x16: deadbeef
>  [2.658348] x15:  x14: 80001179b548
>  [2.663792] x13: 800091db37b7 x12: 800011db37bf
>  [2.669236] x11: 8000117c7000 x10: 800011db3740
>  [2.674680] x9 : ffd0 x8 : 80001071e980
>  [2.680124] x7 : 0132 x6 : 80001197a6ab
>  [2.685568] x5 :  x4 : 
>  [2.691012] x3 : f7b0 x2 : fde0
>  [2.696465] x1 : 0026c340 x0 : 0246c340
>  [2.701899] Call trace:
>  [2.704452]  xen_swiotlb_free_coherent+0x198/0x1d8
>  [2.709367]  dma_free_attrs+0x98/0xd0
>  [2.713143]  rpi_firmware_property_list+0x1e4/0x240
>  [2.718146]  rpi_firmware_property+0x6c/0xb0
>  [2.722535]  rpi_firmware_probe+0xf0/0x1e0
>  [2.726760]  platform_drv_probe+0x50/0xa0
>  [2.730879]  really_probe+0xd8/0x438
>  [2.734567]  driver_probe_device+0xdc/0x130
>  [2.738870]  __device_attach_driver+0x88/0x108
>  [2.743434]  bus_for_each_drv+0x78/0xc8
>  [2.747386]  __device_attach+0xd4/0x158
>  [2.751337]  device_initial_probe+0x10/0x18
>  [2.755649]  bus_probe_device+0x90/0x98
>  [2.759590]  deferred_probe_work_func+0x88/0xd8
>  [2.764244]  process_one_work+0x1f0/0x3c0
>  [2.768369]  worker_thread+0x138/0x570
>  [2.772234]  kthread+0x118/0x120
>  [2.775571]  ret_from_fork+0x10/0x18
>  [2.779263] Code: d34cfc00 f2dfbfe2 d37ae400 8b020001
> (f8626800)
>  [2.785492] ---[ end trace 4c435212e349f45f ]---
>  [2.793340] usb 1-1: New USB device found, idVendor=2109,
>  idProduct=3431, bcdDevice= 4.20
>  [2.801038] usb 1-1: New USB device strings: Mfr=0, Product=1,
> SerialNumber=0
>  [2.808297] usb 1-1: Product: USB2.0 Hub
>  [2.813710] hub 1-1:1.0: USB hub found
>  [2.817117] hub 1-1:1.0: 4 ports detected
> 
>  This is bailing out right here:
> 
>  https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fg
> 
> it.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fstable%2Flinux.g
> 
> it%2Ftree%2Fdrivers%2Ffirmware%2Fraspberrypi.c%3Fh%3Dv5.6.1%23n125
> &
> 
> amp;data=02%7C01%7Cpeng.fan%40nxp.com%7Cab98b2db94484141a8ad0
> 8d7f18
> 
> 03372%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6372434052
> 335723
> 
> 54sdata=h1dyHkb%2FsifUqH3Z0m3uIIcqUhXVwhHS%2Ft%2FVvig%2Fo
> u4%3D
>  reserved=0
> 
>  FYIW (since I modified the source to actually print what was
>  returned right before it bails) we get:

RE: [PATCH] xen/swiotlb: correct the check for xen_destroy_contiguous_region

[Peng Fan]

> Subject: Re: [PATCH] xen/swiotlb: correct the check for
> xen_destroy_contiguous_region
> 
> On 28.04.20 09:33, peng@nxp.com wrote:
> > From: Peng Fan 
> >
> > When booting xen on i.MX8QM, met:
> > "
> > [3.602128] Unable to handle kernel paging request at virtual address
> 00272d40
> > [3.610804] Mem abort info:
> > [3.613905]   ESR = 0x9604
> > [3.617332]   EC = 0x25: DABT (current EL), IL = 32 bits
> > [3.623211]   SET = 0, FnV = 0
> > [3.626628]   EA = 0, S1PTW = 0
> > [3.630128] Data abort info:
> > [3.633362]   ISV = 0, ISS = 0x0004
> > [3.637630]   CM = 0, WnR = 0
> > [3.640955] [00272d40] user address but active_mm is
> swapper
> > [3.647983] Internal error: Oops: 9604 [#1] PREEMPT SMP
> > [3.654137] Modules linked in:
> > [3.677285] Hardware name: Freescale i.MX8QM MEK (DT)
> > [3.677302] Workqueue: events deferred_probe_work_func
> > [3.684253] imx6q-pcie 5f00.pcie: PCI host bridge to bus :00
> > [3.688297] pstate: 6005 (nZCv daif -PAN -UAO)
> > [3.688310] pc : xen_swiotlb_free_coherent+0x180/0x1c0
> > [3.693993] pci_bus :00: root bus resource [bus 00-ff]
> > [3.701002] lr : xen_swiotlb_free_coherent+0x44/0x1c0
> > "
> >
> > In xen_swiotlb_alloc_coherent, if !(dev_addr + size - 1 <= dma_mask)
> > or range_straddles_page_boundary(phys, size) are true, it will create
> > contiguous region. So when free, we need to free contiguous region use
> > upper check condition.
> 
> No, this will break PV guests on x86.

Could you share more details why alloc and free not matching for the check?

Thanks,
Peng.

> 
> I think there is something wrong with your setup in combination with the ARM
> xen_create_contiguous_region() implementation.
> 
> Stefano?
> 
> 
> Juergen
> 
> >
> > Signed-off-by: Peng Fan 
> > ---
> >   drivers/xen/swiotlb-xen.c | 4 ++--
> >   1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> > index b6d27762c6f8..ab96e468584f 100644
> > --- a/drivers/xen/swiotlb-xen.c
> > +++ b/drivers/xen/swiotlb-xen.c
> > @@ -346,8 +346,8 @@ xen_swiotlb_free_coherent(struct device *hwdev,
> size_t size, void *vaddr,
> > /* Convert the size to actually allocated. */
> > size = 1UL << (order + XEN_PAGE_SHIFT);
> >
> > -   if (!WARN_ON((dev_addr + size - 1 > dma_mask) ||
> > -range_straddles_page_boundary(phys, size)) &&
> > +   if (((dev_addr + size - 1 > dma_mask) ||
> > +   range_straddles_page_boundary(phys, size)) &&
> > TestClearPageXenRemapped(virt_to_page(vaddr)))
> > xen_destroy_contiguous_region(phys, order);
> >
> >

RE: [PATCH] xen/swiotlb: correct the check for xen_destroy_contiguous_region

[Peng Fan]

> Subject: Re: [PATCH] xen/swiotlb: correct the check for
> xen_destroy_contiguous_region
> 
> On Tue, Apr 28, 2020 at 03:33:45PM +0800, peng@nxp.com wrote:
> >
> > In xen_swiotlb_alloc_coherent, if !(dev_addr + size - 1 <= dma_mask)
> > or range_straddles_page_boundary(phys, size) are true, it will create
> > contiguous region. So when free, we need to free contiguous region use
> > upper check condition.
> >
> > Signed-off-by: Peng Fan 
> > ---
> >  drivers/xen/swiotlb-xen.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> > index b6d27762c6f8..ab96e468584f 100644
> > --- a/drivers/xen/swiotlb-xen.c
> > +++ b/drivers/xen/swiotlb-xen.c
> > @@ -346,8 +346,8 @@ xen_swiotlb_free_coherent(struct device *hwdev,
> size_t size, void *vaddr,
> > /* Convert the size to actually allocated. */
> > size = 1UL << (order + XEN_PAGE_SHIFT);
> >
> > -   if (!WARN_ON((dev_addr + size - 1 > dma_mask) ||
> > -range_straddles_page_boundary(phys, size)) &&
> > +   if (((dev_addr + size - 1 > dma_mask) ||
> > +   range_straddles_page_boundary(phys, size)) &&
> > TestClearPageXenRemapped(virt_to_page(vaddr)))
> 
> No need for the inner braces.
> 
> But more importantly please factor our a helper that can be used by alloc and
> free to make sure that they always stay in sync.  Something

Thanks for reviewing. I'll take your suggestion in v2. Before that,
I would wait to see if there are more comments in this patch,
because there are several history commits touching this place.

Thanks,
Peng.

> like:
> 
> static inline bool xen_swiotlb_need_contiguous_region(struct device *dev,
>   phys_addr_t phys, size_t size)
> {
> 
>   return xen_phys_to_bus(phys) + size - 1 > dev->coherent_dma_mask ||
>   range_straddles_page_boundary(phys, size)) }

[PATCH] xen/swiotlb: correct the check for xen_destroy_contiguous_region

[peng . fan]

From: Peng Fan 

When booting xen on i.MX8QM, met:
"
[3.602128] Unable to handle kernel paging request at virtual address 
00272d40
[3.610804] Mem abort info:
[3.613905]   ESR = 0x9604
[3.617332]   EC = 0x25: DABT (current EL), IL = 32 bits
[3.623211]   SET = 0, FnV = 0
[3.626628]   EA = 0, S1PTW = 0
[3.630128] Data abort info:
[3.633362]   ISV = 0, ISS = 0x0004
[3.637630]   CM = 0, WnR = 0
[3.640955] [00272d40] user address but active_mm is swapper
[3.647983] Internal error: Oops: 9604 [#1] PREEMPT SMP
[3.654137] Modules linked in:
[3.677285] Hardware name: Freescale i.MX8QM MEK (DT)
[3.677302] Workqueue: events deferred_probe_work_func
[3.684253] imx6q-pcie 5f00.pcie: PCI host bridge to bus :00
[3.688297] pstate: 6005 (nZCv daif -PAN -UAO)
[3.688310] pc : xen_swiotlb_free_coherent+0x180/0x1c0
[3.693993] pci_bus :00: root bus resource [bus 00-ff]
[3.701002] lr : xen_swiotlb_free_coherent+0x44/0x1c0
"

In xen_swiotlb_alloc_coherent, if !(dev_addr + size - 1 <= dma_mask) or
range_straddles_page_boundary(phys, size) are true, it will
create contiguous region. So when free, we need to free contiguous
region use upper check condition.

Signed-off-by: Peng Fan 
---
 drivers/xen/swiotlb-xen.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index b6d27762c6f8..ab96e468584f 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -346,8 +346,8 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t 
size, void *vaddr,
/* Convert the size to actually allocated. */
size = 1UL << (order + XEN_PAGE_SHIFT);
 
-   if (!WARN_ON((dev_addr + size - 1 > dma_mask) ||
-range_straddles_page_boundary(phys, size)) &&
+   if (((dev_addr + size - 1 > dma_mask) ||
+   range_straddles_page_boundary(phys, size)) &&
TestClearPageXenRemapped(virt_to_page(vaddr)))
xen_destroy_contiguous_region(phys, order);
 
-- 
2.16.4

Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

> Subject: Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER
> range
> 
> Hi Stefano,
> 
> On 28/11/2019 01:12, Stefano Stabellini wrote:
> > On Wed, 27 Nov 2019, Jürgen Groß wrote:
> >> On 27.11.19 10:31, Peng Fan wrote:
> >>>> Subject: Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix
> >>>> GICD_ISACTIVER range
> >>>>
> >>>> On 27.11.19 01:01, Julien Grall wrote:
> >>>>> Hi,
> >>>>>
> >>>>> On 26/11/2019 23:17, Stefano Stabellini wrote:
> >>>>>> On Tue, 26 Nov 2019, Julien Grall wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> On 26/11/2019 20:43, Stefano Stabellini wrote:
> >>>>>>>> + Juergen
> >>>>>>>>
> >>>>>>>> I missed that you weren't in CC to the original patch, sorry.
> >>>>>>>> I think this patch should go in, as otherwise Linux 5.4 could
> >>>>>>>> run into problems. It is also a pretty straightforward 4 lines patch.
> >>>>>>>
> >>>>>>> 5.5 (or 5.6) is not going to run on Xen for other reasons (still
> >>>>>>> in the vGIC)... So I would not view this as critical.
> >>>>>>
> >>>>>> 5.5 is not out yet, in fact, the dev window has just opened.
> >>>>>> Isn't your statement a bit premature?
> >>>>>
> >>>>> The GICv4.1 work [1] is going to prevent Linux booting on all
> >>>>> current versions of Xen. While I can't confirm this is going to be
> >>>>> merged in 5.5, I can tell you this will break.
> >>>>>
> >>>>>>
> >>>>>> In any case, even if potential future Linux releases could have
> >>>>>> other additional issues, I don't think it should change our
> >>>>>> current view on this specific issue which affects 5.4, just released.
> >>>>>
> >>>>> The patch is definitely not as straightforward as you may think.
> >>>>> Please refer to the discussion we had on the first version. I
> >>>>> voiced concern about this approach and gave point what could go
> >>>>> wrong with
> >>>> happen.
> >>>>>
> >>>>> This patch may be better than the current state (i.e crashing),
> >>>>> but this wasn't tested enough to confirm this is the correct
> >>>>> things to do and no other bug will appear (I don't believe reading
> >>>>> I*ACTIVER was ever tested before).
> >>>>>
> >>>>> It is an annoying bug, but this is only affecting 5.4 which has
> >>>>> just been released. It feels to me this is a fairly risky choice
> >>>>> to merge it qutie late in the release without a good graps of the
> >>>>> problem (see above).
> >>>>>
> >>>>> So I would definitly, prefer if this patch is getting through
> >>>>> backport once we get more testing.
> >>>>>
> >>>>> We can still document the bug in the release note and point people
> >>>>> to the patch.
> >>>>>
> >>>>> Anyway, this is Juergen choice here. But at least now he has the
> >>>>> full picture...
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> [1]
> >>>>>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.
> >>>>>
> >>>>
> net%2FArticles%2F800494%2Fdata=02%7C01%7Cpeng.fan%40nxp.co
> >>>> m%7Cdca
> >>>>>
> >>>>
> dfb39240749ee675e08d772fcd3ba%7C686ea1d3bc2b4c6fa92cd99c5c30163
> >>>> 5%7C0%7
> >>>>>
> >>>>
> C0%7C637104302519996592sdata=7Jv2IhI8HZgBTSuYzkEplFyhX1lzmv
> >>>> d73xb5
> >>>>> 2d6ERVQ%3Dreserved=0
> >>>>>
> >>>>
> >>>> Thanks, Julien, for sharing your opinion.
> >>>>
> >>>> With that statement I'd like to defer this patch to 4.14.
> >>>
> >>> But without this patch, 5.4 kernel will crash. So you prefer we
> >>> develop the solution as Julien suggested for 4.13?
> >>
> >> I certainly won't take a patch for 4.13 when a maintainer of the
> &g

Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

> Subject: Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER
> range
> 
> On 27.11.19 10:31, Peng Fan wrote:
> >> Subject: Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix
> >> GICD_ISACTIVER range
> >>
> >> On 27.11.19 01:01, Julien Grall wrote:
> >>> Hi,
> >>>
> >>> On 26/11/2019 23:17, Stefano Stabellini wrote:
> >>>> On Tue, 26 Nov 2019, Julien Grall wrote:
> >>>>> Hi,
> >>>>>
> >>>>> On 26/11/2019 20:43, Stefano Stabellini wrote:
> >>>>>> + Juergen
> >>>>>>
> >>>>>> I missed that you weren't in CC to the original patch, sorry.
> >>>>>> I think this patch should go in, as otherwise Linux 5.4 could run
> >>>>>> into problems. It is also a pretty straightforward 4 lines patch.
> >>>>>
> >>>>> 5.5 (or 5.6) is not going to run on Xen for other reasons (still
> >>>>> in the vGIC)... So I would not view this as critical.
> >>>>
> >>>> 5.5 is not out yet, in fact, the dev window has just opened. Isn't
> >>>> your statement a bit premature?
> >>>
> >>> The GICv4.1 work [1] is going to prevent Linux booting on all
> >>> current versions of Xen. While I can't confirm this is going to be
> >>> merged in 5.5, I can tell you this will break.
> >>>
> >>>>
> >>>> In any case, even if potential future Linux releases could have
> >>>> other additional issues, I don't think it should change our current
> >>>> view on this specific issue which affects 5.4, just released.
> >>>
> >>> The patch is definitely not as straightforward as you may think.
> >>> Please refer to the discussion we had on the first version. I voiced
> >>> concern about this approach and gave point what could go wrong with
> >> happen.
> >>>
> >>> This patch may be better than the current state (i.e crashing), but
> >>> this wasn't tested enough to confirm this is the correct things to
> >>> do and no other bug will appear (I don't believe reading I*ACTIVER
> >>> was ever tested before).
> >>>
> >>> It is an annoying bug, but this is only affecting 5.4 which has just
> >>> been released. It feels to me this is a fairly risky choice to merge
> >>> it qutie late in the release without a good graps of the problem (see
> above).
> >>>
> >>> So I would definitly, prefer if this patch is getting through
> >>> backport once we get more testing.
> >>>
> >>> We can still document the bug in the release note and point people
> >>> to the patch.
> >>>
> >>> Anyway, this is Juergen choice here. But at least now he has the
> >>> full picture...
> >>>
> >>> Cheers,
> >>>
> >>> [1]
> >>>
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.
> >>>
> >>
> net%2FArticles%2F800494%2Fdata=02%7C01%7Cpeng.fan%40nxp.co
> >> m%7Cdca
> >>>
> >>
> dfb39240749ee675e08d772fcd3ba%7C686ea1d3bc2b4c6fa92cd99c5c30163
> >> 5%7C0%7
> >>>
> >>
> C0%7C637104302519996592sdata=7Jv2IhI8HZgBTSuYzkEplFyhX1lzmv
> >> d73xb5
> >>> 2d6ERVQ%3Dreserved=0
> >>>
> >>
> >> Thanks, Julien, for sharing your opinion.
> >>
> >> With that statement I'd like to defer this patch to 4.14.
> >
> > But without this patch, 5.4 kernel will crash. So you prefer we
> > develop the solution as Julien suggested for 4.13?
> 
> I certainly won't take a patch for 4.13 when a maintainer of the related code
> has reservations against it.
> 
> I think the best thing to do is to develop a proper patch the maintainers are
> happy with and don't try to force it into 4.13 now.
> Such a patch can still be backported to 4.13 later.

Ok.

Julien,

What's your suggestion to fix the issue? Do you have a rough idea?

Thanks,
Peng.

> 
> 
> Juergen
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

> Subject: Re: [Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER
> range
> 
> On 27.11.19 01:01, Julien Grall wrote:
> > Hi,
> >
> > On 26/11/2019 23:17, Stefano Stabellini wrote:
> >> On Tue, 26 Nov 2019, Julien Grall wrote:
> >>> Hi,
> >>>
> >>> On 26/11/2019 20:43, Stefano Stabellini wrote:
>  + Juergen
> 
>  I missed that you weren't in CC to the original patch, sorry.
>  I think this patch should go in, as otherwise Linux 5.4 could run
>  into problems. It is also a pretty straightforward 4 lines patch.
> >>>
> >>> 5.5 (or 5.6) is not going to run on Xen for other reasons (still in
> >>> the vGIC)... So I would not view this as critical.
> >>
> >> 5.5 is not out yet, in fact, the dev window has just opened. Isn't
> >> your statement a bit premature?
> >
> > The GICv4.1 work [1] is going to prevent Linux booting on all current
> > versions of Xen. While I can't confirm this is going to be merged in
> > 5.5, I can tell you this will break.
> >
> >>
> >> In any case, even if potential future Linux releases could have other
> >> additional issues, I don't think it should change our current view on
> >> this specific issue which affects 5.4, just released.
> >
> > The patch is definitely not as straightforward as you may think.
> > Please refer to the discussion we had on the first version. I voiced
> > concern about this approach and gave point what could go wrong with
> happen.
> >
> > This patch may be better than the current state (i.e crashing), but
> > this wasn't tested enough to confirm this is the correct things to do
> > and no other bug will appear (I don't believe reading I*ACTIVER was
> > ever tested before).
> >
> > It is an annoying bug, but this is only affecting 5.4 which has just
> > been released. It feels to me this is a fairly risky choice to merge
> > it qutie late in the release without a good graps of the problem (see 
> > above).
> >
> > So I would definitly, prefer if this patch is getting through backport
> > once we get more testing.
> >
> > We can still document the bug in the release note and point people to
> > the patch.
> >
> > Anyway, this is Juergen choice here. But at least now he has the full
> > picture...
> >
> > Cheers,
> >
> > [1]
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.
> >
> net%2FArticles%2F800494%2Fdata=02%7C01%7Cpeng.fan%40nxp.co
> m%7Cdca
> >
> dfb39240749ee675e08d772fcd3ba%7C686ea1d3bc2b4c6fa92cd99c5c30163
> 5%7C0%7
> >
> C0%7C637104302519996592sdata=7Jv2IhI8HZgBTSuYzkEplFyhX1lzmv
> d73xb5
> > 2d6ERVQ%3Dreserved=0
> >
> 
> Thanks, Julien, for sharing your opinion.
> 
> With that statement I'd like to defer this patch to 4.14.

But without this patch, 5.4 kernel will crash. So you prefer
we develop the solution as Julien suggested for 4.13?

Thanks,
Peng.

> 
> 
> Juergen
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] vsnd issue

[Peng Fan]

> Cc: sstabell...@kernel.org; jul...@xen.org; xen-de...@lists.xen.org
> Subject: Re: [Xen-devel] vsnd issue
> 
> On 11/25/19 12:40 PM, Artem Mygaiev wrote:
> > Hello Peng Fan
> >
> > Please contact Oleksandr Andrushchenko (added to this thread) on this
> > issue.
> >
> >   -- Artem
> >
> > On Mon, 2019-11-25 at 10:24 +, Julien Grall wrote:
> >> On 25/11/2019 10:19, Peng Fan wrote:
> >>> Hi All,
> >> Hi,
> >>
> >>> I am trying vsnd from xen-troops with xen 4.13 and Linux 5.4, but
> >>> domu reports:
> >> xen-troops is not part of Xen Project. Please contact the owner of
> >> the repo for any help here.
> >>
> >> I have CCed Artem who should be able to point to the author of vsnd.
> >>
> >> Best regards,
> >>
> >>> aplay compl.mp3
> Hm, could you please try the same with wav and not mp3 and get back with
> the logs?

Same issue.
aplay hu01_48k.wav
ALSA lib 
../../../alsa-lib-1.1.9/src/pcm/pcm_direct.c:1156:(snd1_pcm_direct_initialize_slave)
 slave plugin does not support mmap interleaved or mmap noninterleaved access
ALSA lib ../../../alsa-lib-1.1.9/src/pcm/pcm_dmix.c:1120:(snd_pcm_dmix_open) 
unable to initialize slave
aplay: main:828: audio open error: Invalid argument

Is there any limitation with current vsnd? Is there any plan to upstream 
libxenbe and snd_be
to xen?

Thanks,
Peng.

> >>> ALSA lib ../../../alsa-lib-
> >>> 1.1.9/src/pcm/pcm_direct.c:1156:(snd1_pcm_direct_initialize_slave)
> >>> slave plugin does not support mmap interleaved or mmap
> >>> noninterleaved access
> ALSA frontend does not support mmap because of [1], so this is expected
> >>> ALSA lib ../../../alsa-lib-
> >>> 1.1.9/src/pcm/pcm_dmix.c:1120:(snd_pcm_dmix_open) unable to
> >>> initialize slave
> >>> aplay: main:828: audio open error: Invalid argument
> >>>
> >>> When executing aplay in domu, dom0 side log:
> >>> root@imx8qmmek:~# 13.11.19 08:24:57.484 | XenEvtchn   | DBG
> -
> >>> Event received, port: 10
> >>> 13.11.19 08:24:57.491 | StreamRing  | DBG - Request received,
> >>> id: alsa, cmd:9
> >>> 13.11.19 08:24:57.500 | CommandHandler  | DBG - Handle command
> >>> [QUERY_HW_PARAM]
> >>> 13.11.19 08:24:57.508 | AlsaPcm | DBG - Query pcm device
> >>> hw:2,0 for HW parameters
> >>> 13.11.19 08:24:57.516 | CommandHandler  | DBG - Return status: [0]
> >>> 13.11.19 08:24:57.523 | XenEvtchn   | DBG - Notify event
> >>> channel, port: 10
> >>> 13.11.19 08:24:57.531 | XenEvtchn   | DBG - Event received,
> >>> port: 10
> >>> 13.11.19 08:24:57.538 | StreamRing  | DBG - Request received,
> >>> id: alsa, cmd:9
> >>> 13.11.19 08:24:57.546 | CommandHandler  | DBG - Handle command
> >>> [QUERY_HW_PARAM]
> >>> 13.11.19 08:24:57.554 | AlsaPcm | DBG - Query pcm device
> >>> hw:2,0 for HW parameters
> >>> 13.11.19 08:24:57.563 | CommandHandler  | DBG - Return status: [0]
> >>> 13.11.19 08:24:57.570 | XenEvtchn   | DBG - Notify event
> >>> channel, port: 10
> >>> 13.11.19 08:24:57.577 | XenEvtchn   | DBG - Event received,
> >>> port: 10
> >>> 13.11.19 08:24:57.584 | StreamRing  | DBG - Request received,
> >>> id: alsa, cmd:9
> >>> 13.11.19 08:24:57.593 | CommandHandler  | DBG - Handle command
> >>> [QUERY_HW_PARAM]
> >>> 13.11.19 08:24:57.601 | AlsaPcm | DBG - Query pcm device
> >>> hw:2,0 for HW parameters
> >>> 13.11.19 08:24:57.610 | CommandHandler  | DBG - Return status: [0]
> >>> 13.11.19 08:24:57.616 | XenEvtchn   | DBG - Notify event
> >>> channel, port: 10
> >>> 13.11.19 08:24:57.624 | XenEvtchn   | DBG - Event received,
> >>> port: 10
> >>> 13.11.19 08:24:57.631 | StreamRing  | DBG - Request received,
> >>> id: alsa, cmd:9
> >>> 13.11.19 08:24:57.640 | CommandHandler  | DBG - Handle command
> >>> [QUERY_HW_PARAM]
> >>> 13.11.19 08:24:57.647 | AlsaPcm | DBG - Query pcm device
> >>> hw:2,0 for HW parameters
> >>> 13.11.19 08:24:57.656 | CommandHandler  | DBG - Return status: [0]
> >>> 13.11.19 08:24:57.663 | XenEvtchn   | DBG - Notify event
> >>> channel, port: 10
> >>> 13.11.19 08:24:57.671 | XenEvtchn   | DBG - Event received,
> >>> port: 10
> >>> 13.11.19 08:24:57.678

[Xen-devel] vsnd issue

[Peng Fan]

Hi All,

I am trying vsnd from xen-troops with xen 4.13 and Linux 5.4, but domu reports:
aplay compl.mp3
ALSA lib 
../../../alsa-lib-1.1.9/src/pcm/pcm_direct.c:1156:(snd1_pcm_direct_initialize_slave)
 slave plugin does not support mmap interleaved or mmap noninterleaved access
ALSA lib ../../../alsa-lib-1.1.9/src/pcm/pcm_dmix.c:1120:(snd_pcm_dmix_open) 
unable to initialize slave
aplay: main:828: audio open error: Invalid argument

When executing aplay in domu, dom0 side log:
root@imx8qmmek:~# 13.11.19 08:24:57.484 | XenEvtchn   | DBG - Event 
received, port: 10
13.11.19 08:24:57.491 | StreamRing  | DBG - Request received, id: 
alsa, cmd:9
13.11.19 08:24:57.500 | CommandHandler  | DBG - Handle command [QUERY_HW_PARAM]
13.11.19 08:24:57.508 | AlsaPcm | DBG - Query pcm device hw:2,0 for HW 
parameters
13.11.19 08:24:57.516 | CommandHandler  | DBG - Return status: [0]
13.11.19 08:24:57.523 | XenEvtchn   | DBG - Notify event channel, port: 10
13.11.19 08:24:57.531 | XenEvtchn   | DBG - Event received, port: 10
13.11.19 08:24:57.538 | StreamRing  | DBG - Request received, id: 
alsa, cmd:9
13.11.19 08:24:57.546 | CommandHandler  | DBG - Handle command [QUERY_HW_PARAM]
13.11.19 08:24:57.554 | AlsaPcm | DBG - Query pcm device hw:2,0 for HW 
parameters
13.11.19 08:24:57.563 | CommandHandler  | DBG - Return status: [0]
13.11.19 08:24:57.570 | XenEvtchn   | DBG - Notify event channel, port: 10
13.11.19 08:24:57.577 | XenEvtchn   | DBG - Event received, port: 10
13.11.19 08:24:57.584 | StreamRing  | DBG - Request received, id: 
alsa, cmd:9
13.11.19 08:24:57.593 | CommandHandler  | DBG - Handle command [QUERY_HW_PARAM]
13.11.19 08:24:57.601 | AlsaPcm | DBG - Query pcm device hw:2,0 for HW 
parameters
13.11.19 08:24:57.610 | CommandHandler  | DBG - Return status: [0]
13.11.19 08:24:57.616 | XenEvtchn   | DBG - Notify event channel, port: 10
13.11.19 08:24:57.624 | XenEvtchn   | DBG - Event received, port: 10
13.11.19 08:24:57.631 | StreamRing  | DBG - Request received, id: 
alsa, cmd:9
13.11.19 08:24:57.640 | CommandHandler  | DBG - Handle command [QUERY_HW_PARAM]
13.11.19 08:24:57.647 | AlsaPcm | DBG - Query pcm device hw:2,0 for HW 
parameters
13.11.19 08:24:57.656 | CommandHandler  | DBG - Return status: [0]
13.11.19 08:24:57.663 | XenEvtchn   | DBG - Notify event channel, port: 10
13.11.19 08:24:57.671 | XenEvtchn   | DBG - Event received, port: 10
13.11.19 08:24:57.678 | StreamRing  | DBG - Request received, id: 
alsa, cmd:9
13.11.19 08:24:57.686 | CommandHandler  | DBG - Handle command [QUERY_HW_PARAM]
13.11.19 08:24:57.694 | AlsaPcm | DBG - Query pcm device hw:2,0 for HW 
parameters
13.11.19 08:24:57.703 | CommandHandler  | DBG - Return status: [0]
13.11.19 08:24:57.709 | XenEvtchn   | DBG - Notify event channel, port: 10


My xl.cfg:
vsnd = [
['CARD, short-name=Main, sample-formats=s16_le;s8;u32_be',
'PCM, name=Main',
'STREAM, unique-id=alsa, type=p',
'STREAM, unique-id=alsa, type=c, channels-max=2'
],
]

The audio device on my board:
aplay -l
 List of PLAYBACK Hardware Devices 
card 0: imxaudmix [imx-audmix], device 0: HiFi-AUDMIX-FE (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: imxaudmix [imx-audmix], device 1: HiFi-AUDMIX-FE (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: cs42888audio [cs42888-audio], device 0: HiFi cs42888-0 [HiFi cs42888-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: cs42888audio [cs42888-audio], device 1: HiFi-ASRC-FE (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: wm8960audio [wm8960-audio], device 0: HiFi wm8960-hifi-0 []
  Subdevices: 0/1
  Subdevice #0: subdevice #0

Is there something wrong in my configuration?

Thanks,
Peng.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

The end should be GICD_ISACTIVERN not GICD_ISACTIVER,
and also print a warning for the unhandled read.

Signed-off-by: Peng Fan 
---

V2:
 Add a warning message

 xen/arch/arm/vgic-v3.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 422b94f902..a15b9f6441 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -706,7 +706,10 @@ static int __vgic_v3_distr_common_mmio_read(const char 
*name, struct vcpu *v,
 goto read_as_zero;
 
 /* Read the active status of an IRQ via GICD/GICR is not supported */
-case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVER):
+case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
+printk(XENLOG_G_ERR "%pv: vGICD: unhandled read from ISACTIVER%d\n",
+   v, (reg - GICD_ISACTIVER) / 4);
+goto read_as_zero;
 case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
 goto read_as_zero;
 
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

> Do you have a call stack trace for this?

(XEN) d0v1: vGICD: unhandled read r1 offset 0x000324
(XEN) traps.c:1994:d0v1 HSR=0x93810006 pc=0x8000104f2bb4 
gva=0x800010010324 gpa=0x0051a00324
[1.780564] Unhandled fault at 0x800010010324
[1.785771] Mem abort info:
[1.788899]   ESR = 0x9600
[1.792320]   EC = 0x25: DABT (current EL), IL = 32 bits
[1.798196]   SET = 0, FnV = 0
[1.801615]   EA = 0, S1PTW = 0
[1.805124] Data abort info:
[1.808350]   ISV = 0, ISS = 0x
[1.812620]   CM = 0, WnR = 0
[1.815943] swapper pgtable: 4k pages, 48-bit VAs, pgdp=a17f6000
[1.823344] [800010010324] pgd=00095003, pud=00095fffe003, 
pmd=00095fffd003, pte=00e851a00707
[1.835016] Internal error: ttbr address size fault: 9600 [#1] PREEMPT 
SMP
[1.842983] Modules linked in:
[1.846403] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 
5.4.0-rc5-03191-g20646f88034a-dirty #1774
[1.855980] Hardware name: Freescale i.MX8QM MEK DOM0 (DT)
[1.862059] pstate: a085 (NzCv daIf -PAN -UAO)
[1.867374] pc : gic_peek_irq+0x6c/0xb0
[1.871638] lr : gic_peek_irq+0x28/0xb0
[1.875899] sp : 80001002ba80
[1.879599] x29: 80001002ba80 x28: 
[1.885483] x27:  x26: 800011800514
[1.891369] x25:  x24: 
[1.897246] x23: 0008dc3e5400 x22: 0008dc3e5458
[1.903129] x21: 800011d798c8 x20: 0001
[1.909011] x19: 80001002bb07 x18: 0004
[1.914893] x17: 8000112243c0 x16: deadbeef
[1.920780] x15: 0008dc01b490 x14: 0008dbcde5c0
[1.926658] x13: ff00 x12: 
[1.932540] x11: 0008 x10: 
[1.938422] x9 : c1c1c1c1c1c1c1c1 x8 : 
[1.944304] x7 :  x6 : 0008dc3e5458
[1.950186] x5 : 800011d798c8 x4 : 0008
[1.956069] x3 : 0324 x2 : fd03
[1.961951] x1 : 800010010324 x0 : 0300
[1.967835] Call trace:
[1.970588]  gic_peek_irq+0x6c/0xb0
[1.974481]  gic_irq_get_irqchip_state+0x70/0xa0
[1.979607]  __synchronize_hardirq+0xd8/0xe0
[1.984343]  __free_irq+0x13c/0x2b8
[1.988232]  free_irq+0x30/0x70
[1.991749]  devm_irq_release+0x14/0x20
[1.996019]  release_nodes+0x1b0/0x220
[2.000189]  devres_release_all+0x34/0x50
[2.004652]  really_probe+0x1b8/0x308
[2.008728]  driver_probe_device+0x54/0xe8
[2.013281]  device_driver_attach+0x6c/0x78
[2.017929]  __driver_attach+0x54/0xd0
[2.022106]  bus_for_each_dev+0x70/0xc0
[2.026372]  driver_attach+0x20/0x28
[2.030359]  bus_add_driver+0x178/0x1d8
[2.034630]  driver_register+0x60/0x110
[2.038900]  __platform_driver_register+0x44/0x50
[2.044121]  fec_driver_init+0x18/0x20
[2.048294]  do_one_initcall+0x74/0x1b0
[2.052564]  kernel_init_freeable+0x194/0x22c
[2.057403]  kernel_init+0x10/0x100
[2.061293]  ret_from_fork+0x10/0x18
[2.065279] Code: 53057c63 d37e6863 8b204063 8b030021 (b9400021)
[2.072024] ---[ end trace 62c8d1d1d10ae3de ]---
[2.077158] note: swapper/0[1] exited with preempt_count 1
[2.083296] Kernel panic - not syncing: Attempted to kill init! 
exitcode=0x000b
[2.091659] SMP: stopping secondary CPUs
[2.096037] Kernel Offset: disabled
[2.099914] CPU features: 0x0002,2000200c
[2.104371] Memory Limit: none
[2.107788] ---[ end Kernel panic - not syncing: Attempted to kill init! 
exitcode=0x000b ]---

Regards,
Peng.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

Hi Julien,

Inline marked with [Peng Fan]

From: Julien Grall  
Sent: 2019年11月9日 6:44
To: Stefano Stabellini ; Andre Przywara 

Cc: Peng Fan ; Jürgen Groß ; 
julien.gr...@arm.com; xen-de...@lists.xen.org
Subject: Re: [Xen-devel] [PATCH] arch: arm: vgic-v3: fix GICD_ISACTIVER range

Hi,

Sorry for the formatting.
On Sat, 9 Nov 2019, 04:27 Stefano Stabellini, <mailto:sstabell...@kernel.org> 
wrote:
On Thu, 7 Nov 2019, Peng Fan wrote:
> The end should be GICD_ISACTIVERN not GICD_ISACTIVER.
> 
> Signed-off-by: Peng Fan <mailto:peng@nxp.com>

Reviewed-by: Stefano Stabellini <mailto:sstabell...@kernel.org>

To be honest, I am not sure the code is correct. A read to those registers 
should tell you the list of interrupts active. As we always return 0, this will 
not return the correct state of the GIC.

I know that returning the list of actives interrupts is complicated with the 
old vGIC, but I don't think silently ignoring it is a good idea.

The question here is why the guest accessed those registers? What is it trying 
to figure out?

[Peng Fan] I am running Linux 5.4 kernel dom0, gic_peek_irq triggers abort.



Juergen, I think this fix should be in the release (and also
backported to stable trees.)

Without an understanding of the problem, I disagree with this request (see 
above).

As an aside, the range ISPENDR  has the same issue.

[Peng Fan] Should I include this change in v2? Or develop new method to fix the 
issue?
But at least dom0 abort when boot.

Thanks,
Peng.

Cheers,






> ---
>  xen/arch/arm/vgic-v3.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
> index 422b94f902..e802f2055a 100644
> --- a/xen/arch/arm/vgic-v3.c
> +++ b/xen/arch/arm/vgic-v3.c
> @@ -706,7 +706,7 @@ static int __vgic_v3_distr_common_mmio_read(const char 
> *name, struct vcpu *v,
>          goto read_as_zero;
>  
>      /* Read the active status of an IRQ via GICD/GICR is not supported */
> -    case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVER):
> +    case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
>      case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
>          goto read_as_zero;
>  
> -- 
> 2.16.4
> 

___
Xen-devel mailing list
mailto:Xen-devel@lists.xenproject.org
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.xenproject.org%2Fmailman%2Flistinfo%2Fxen-devel=02%7C01%7Cpeng.fan%40nxp.com%7C33f2e907cdc84ed0a48608d7649d359e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637088498678782239=G3FA2vefr56FeUX5QVZQwSzG22nfv1m%2F0fKIDOnfuFQ%3D=0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] arch: arm: vgic-v3: fix GICD_ISACTIVER range

[Peng Fan]

The end should be GICD_ISACTIVERN not GICD_ISACTIVER.

Signed-off-by: Peng Fan 
---
 xen/arch/arm/vgic-v3.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 422b94f902..e802f2055a 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -706,7 +706,7 @@ static int __vgic_v3_distr_common_mmio_read(const char 
*name, struct vcpu *v,
 goto read_as_zero;
 
 /* Read the active status of an IRQ via GICD/GICR is not supported */
-case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVER):
+case VRANGE32(GICD_ISACTIVER, GICD_ISACTIVERN):
 case VRANGE32(GICD_ICACTIVER, GICD_ICACTIVERN):
 goto read_as_zero;
 
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V3] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

From: Peng Fan 

arm64 shares some code under arch/arm/xen, including mm.c.
However ZONE_DMA is removed by commit
ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").

So add a check if CONFIG_ZONE_DMA32 is enabled use __GFP_DMA32.

Signed-off-by: Peng Fan 
---

V3:
 Use IS_ENABLED(CONFIG_ZONE_DMA32) and drop xen_set_gfp_dma.

V2:
 Follow suggestion from Stefano,
 introduce static inline void xen_set_gfp_dma(gfp_t *flags) for arm32/arm64, and
 for arm64 using __GFP_DMA for the former and __GFP_DMA32 for the latter.


 arch/arm/xen/mm.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index 2b2c208408bb..38fa917c8585 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -28,7 +28,10 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)
 
for_each_memblock(memory, reg) {
if (reg->base < (phys_addr_t)0x) {
-   flags |= __GFP_DMA;
+   if (IS_ENABLED(CONFIG_ZONE_DMA32))
+   flags |= __GFP_DMA32;
+   else
+   flags |= __GFP_DMA;
break;
}
}
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH V2] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

> Subject: Re: [PATCH V2] arm: xen: mm: use __GPF_DMA32 for arm64
> 
> + Juergen, Boris
> 
> On Fri, 30 Aug 2019, Christoph Hellwig wrote:
> > Can we take a step back and figure out what we want to do here?
> >
> > AFAICS this function allocates memory for the swiotlb-xen buffer, and
> > that means it must be <= 32-bit addressable to satisfy the DMA API
> > guarantees.  That means we generally want to use GFP_DMA32
> everywhere
> > that exists, but on systems with odd zones we might want to dip into
> > GFP_DMA.  This also means swiotlb-xen doesn't actually do the right
> > thing on x86 at the moment.  So shouldn't we just have one common
> > routine in swiotlb-xen.c that checks if we have CONFIG_ZONE_DMA32 set,
> > then try GFP_DMA32, and if not check if CONFIG_ZONE_DMA is set and
> > then try that, else default to GFP_KERNEL?
> 
> Yes, for ARM/ARM64 it makes a lot of sense given that dom0 is 1:1 mapped
> (pseudo-physical == physical).  I'll let Juergen and Boris comment on the x86
> side of things, but on x86 PV Dom0 is not 1:1 mapped so
> GFP_DMA32 is probably not meaningful.

If we only take ARM/ARM64, so could the following patch be ok?

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index d33b77e9add3..e5a6a73b2e06 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -28,7 +28,11 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)

for_each_memblock(memory, reg) {
if (reg->base < (phys_addr_t)0x) {
+#ifdef CONFIG_ZONE_DMA32
+   flags |= __GFP_DMA32;
+#else
flags |= __GFP_DMA;
+#endif
break;
}
}

Thanks,
Peng.

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH V2] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

From: Peng Fan 

arm64 shares some code under arch/arm/xen, including mm.c.
However ZONE_DMA is removed by commit
ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
So introduce xen_set_gfp_dma for arm32/arm64 and using __GFP_DMA
for the former and __GFP_DMA32 for the latter.

Signed-off-by: Peng Fan 
---

V2:
 Follow suggestion from Stefano,
 introduce static inline void xen_set_gfp_dma(gfp_t *flags) for arm32/arm64, and
 for arm64 using __GFP_DMA for the former and __GFP_DMA32 for the latter.

 arch/arm/include/asm/xen/page.h   | 5 +
 arch/arm/xen/mm.c | 2 +-
 arch/arm64/include/asm/xen/page.h | 5 +
 3 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/xen/page.h b/arch/arm/include/asm/xen/page.h
index 31bbc803cecb..d08309c45e6c 100644
--- a/arch/arm/include/asm/xen/page.h
+++ b/arch/arm/include/asm/xen/page.h
@@ -1 +1,6 @@
 #include 
+
+static inline void xen_set_gfp_dma(gfp_t *flags)
+{
+   *flags |= __GFP_DMA;
+}
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index d33b77e9add3..828f49dc95f9 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -28,7 +28,7 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)
 
for_each_memblock(memory, reg) {
if (reg->base < (phys_addr_t)0x) {
-   flags |= __GFP_DMA;
+   xen_set_gfp_dma();
break;
}
}
diff --git a/arch/arm64/include/asm/xen/page.h 
b/arch/arm64/include/asm/xen/page.h
index 31bbc803cecb..5eeabf2c6494 100644
--- a/arch/arm64/include/asm/xen/page.h
+++ b/arch/arm64/include/asm/xen/page.h
@@ -1 +1,6 @@
 #include 
+
+static inline void xen_set_gfp_dma(gfp_t *flags)
+{
+   *flags |= __GFP_DMA32;
+}
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

Hi Stefano,

> Subject: RE: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64
> 
> On Wed, 28 Aug 2019, Peng Fan wrote:
> > Hi Robin,
> >
> > > Subject: Re: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64
> > >
> > > On 09/07/2019 09:22, Peng Fan wrote:
> > > > arm64 shares some code under arch/arm/xen, including mm.c.
> > > > However ZONE_DMA is removed by commit
> > > > ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
> > > > So to ARM64, need use __GFP_DMA32.
> 
> Hi Peng,
> 
> Sorry for being so late in replying, this email got lost in the noise.
> 
> 
> > > > Signed-off-by: Peng Fan 
> > > > ---
> > > >   arch/arm/xen/mm.c | 2 +-
> > > >   1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c index
> > > > e1d44b903dfc..a95e76d18bf9 100644
> > > > --- a/arch/arm/xen/mm.c
> > > > +++ b/arch/arm/xen/mm.c
> > > > @@ -27,7 +27,7 @@ unsigned long
> > > > xen_get_swiotlb_free_pages(unsigned
> > > > int order)
> > > >
> > > > for_each_memblock(memory, reg) {
> > > > if (reg->base < (phys_addr_t)0x) {
> > > > -   flags |= __GFP_DMA;
> > > > +   flags |= __GFP_DMA | __GFP_DMA32;
> > >
> > > Given the definition of GFP_ZONE_BAD, I'm not sure this combination
> > > of flags is strictly valid, but rather is implicitly reliant on only
> > > one of those zones ever actually existing. As such, it seems liable
> > > to blow up if the plans to add ZONE_DMA to arm64[1] go ahead.
> >
> > How about this, or do you have any suggestions?
> > diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c index
> > d33b77e9add3..f61c29a4430f 100644
> > --- a/arch/arm/xen/mm.c
> > +++ b/arch/arm/xen/mm.c
> > @@ -28,7 +28,11 @@ unsigned long xen_get_swiotlb_free_pages(unsigned
> > int order)
> >
> > for_each_memblock(memory, reg) {
> > if (reg->base < (phys_addr_t)0x) {
> > +#ifdef CONFIG_ARM64
> > +   flags |= __GFP_DMA32; #else
> > flags |= __GFP_DMA;
> > +#endif
> > break;
> > }
> > }
> 
> Yes I think this is the way to go, but we are trying not to add any #ifdef
> CONFIG_ARM64 under arch/arm. Maybe you could introduce a static inline
> function to set GFP_DMA:
> 
>   static inline void xen_set_gfp_dma(gfp_t *flags)
> 
> it could be implemented under arch/arm/include/asm/xen/page.h for arm
> and under arch/arm64/include/asm/xen/page.h for arm64 using __GFP_DMA
> for the former and __GFP_DMA32 for the latter.

Thanks for your suggestion. I'll use this in V2.

Thanks,
Peng.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

Hi Robin,

> Subject: Re: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64
> 
> On 09/07/2019 09:22, Peng Fan wrote:
> > arm64 shares some code under arch/arm/xen, including mm.c.
> > However ZONE_DMA is removed by commit
> > ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
> > So to ARM64, need use __GFP_DMA32.
> >
> > Signed-off-by: Peng Fan 
> > ---
> >   arch/arm/xen/mm.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c index
> > e1d44b903dfc..a95e76d18bf9 100644
> > --- a/arch/arm/xen/mm.c
> > +++ b/arch/arm/xen/mm.c
> > @@ -27,7 +27,7 @@ unsigned long xen_get_swiotlb_free_pages(unsigned
> > int order)
> >
> > for_each_memblock(memory, reg) {
> > if (reg->base < (phys_addr_t)0x) {
> > -   flags |= __GFP_DMA;
> > +   flags |= __GFP_DMA | __GFP_DMA32;
> 
> Given the definition of GFP_ZONE_BAD, I'm not sure this combination of flags
> is strictly valid, but rather is implicitly reliant on only one of those 
> zones ever
> actually existing. As such, it seems liable to blow up if the plans to add
> ZONE_DMA to arm64[1] go ahead.

How about this, or do you have any suggestions?
diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index d33b77e9add3..f61c29a4430f 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -28,7 +28,11 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)

for_each_memblock(memory, reg) {
if (reg->base < (phys_addr_t)0x) {
+#ifdef CONFIG_ARM64
+   flags |= __GFP_DMA32;
+#else
flags |= __GFP_DMA;
+#endif
break;
}
}

Thanks,
Peng.

> 
> Robin.
> 
> [1]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.ke
> rnel.org%2Flinux-arm-kernel%2F20190820145821.27214-1-nsaenzjulienne%
> 40suse.de%2Fdata=02%7C01%7Cpeng.fan%40nxp.com%7C5d2a641b1
> e3f449562f908d72ae95d85%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0
> %7C0%7C637025054169859035sdata=1ZPGH0gZnvgmlMpX7VrjUNME
> XbEjiv4%2FZ9pYwTQTWxY%3Dreserved=0
> 
> > break;
> > }
> > }
> >
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

Ping again..

+Julien

> Subject: RE: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64
> 
> Hi Russell, Stefano
> 
> > Subject: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64
> 
> Any comments?
> 
> >
> > arm64 shares some code under arch/arm/xen, including mm.c.
> > However ZONE_DMA is removed by commit
> > ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
> > So to ARM64, need use __GFP_DMA32.
> >
> > Signed-off-by: Peng Fan 
> > ---
> >  arch/arm/xen/mm.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c index
> > e1d44b903dfc..a95e76d18bf9 100644
> > --- a/arch/arm/xen/mm.c
> > +++ b/arch/arm/xen/mm.c
> > @@ -27,7 +27,7 @@ unsigned long xen_get_swiotlb_free_pages(unsigned
> int
> > order)
> >
> > for_each_memblock(memory, reg) {
> > if (reg->base < (phys_addr_t)0x) {
> > -   flags |= __GFP_DMA;
> > +   flags |= __GFP_DMA | __GFP_DMA32;
> > break;
> > }
> > }
> 
> Thanks,
> Peng.

Thanks,
Peng.

> 
> > --
> > 2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

Hi Russell, Stefano

> Subject: [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

Any comments?

> 
> arm64 shares some code under arch/arm/xen, including mm.c.
> However ZONE_DMA is removed by commit
> ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
> So to ARM64, need use __GFP_DMA32.
> 
> Signed-off-by: Peng Fan 
> ---
>  arch/arm/xen/mm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c index
> e1d44b903dfc..a95e76d18bf9 100644
> --- a/arch/arm/xen/mm.c
> +++ b/arch/arm/xen/mm.c
> @@ -27,7 +27,7 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int
> order)
> 
>   for_each_memblock(memory, reg) {
>   if (reg->base < (phys_addr_t)0x) {
> - flags |= __GFP_DMA;
> + flags |= __GFP_DMA | __GFP_DMA32;
>   break;
>   }
>   }

Thanks,
Peng.

> --
> 2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] arm: xen: mm: use __GPF_DMA32 for arm64

[Peng Fan]

arm64 shares some code under arch/arm/xen, including mm.c.
However ZONE_DMA is removed by commit
ad67f5a6545("arm64: replace ZONE_DMA with ZONE_DMA32").
So to ARM64, need use __GFP_DMA32.

Signed-off-by: Peng Fan 
---
 arch/arm/xen/mm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c
index e1d44b903dfc..a95e76d18bf9 100644
--- a/arch/arm/xen/mm.c
+++ b/arch/arm/xen/mm.c
@@ -27,7 +27,7 @@ unsigned long xen_get_swiotlb_free_pages(unsigned int order)
 
for_each_memblock(memory, reg) {
if (reg->base < (phys_addr_t)0x) {
-   flags |= __GFP_DMA;
+   flags |= __GFP_DMA | __GFP_DMA32;
break;
}
}
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH] xen/arm: setup_xenheap_mappings: BUG when alloc_boot_pages

[Peng Fan]

On ARM64, bootmem is initialized after setup_xenheap_mappings,
so we should not call alloc_boot_pages in setup_xenheap_mappings.

We could not simply move setup_xenheap_mappings after init_boot_pages,
because in bootmem_region_add, when bootmem_region_list is NULL,
it will assign the first virtual address of the first bootmem region
to bootmem_region_list. If the bootmem is not mapped, it will trigger
data abort when writting to the bootmem_region_list[] area.

Currently we did not met issue, because FIRST_SIZE is 1GB on ARM64,
and xenheap_first_first table could hold up to 512GB virtual memory
region. we do not have SoC support such large DRAM now, but we might
have in future.

Add BUG() to let people be aware of this issue.

Signed-off-by: Peng Fan 
---
 xen/arch/arm/mm.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index 01ae20..dcc72ef199 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -850,8 +850,14 @@ void __init setup_xenheap_mappings(unsigned long base_mfn,
 }
 else
 {
-mfn_t first_mfn = alloc_boot_pages(1, 1);
-
+mfn_t first_mfn;
+
+   /*
+* TODO: bootmem not ready, need init_boot_pages before alloc
+* boot pages.
+*/
+BUG();
+first_mfn = alloc_boot_pages(1, 1);
 clear_page(mfn_to_virt(first_mfn));
 pte = mfn_to_xen_entry(first_mfn, MT_NORMAL);
 pte.pt.table = 1;
-- 
2.16.4


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12] xen/arm: gic-v2: deactivate interrupts during initialization

[Peng Fan]

> -Original Message-
> From: Stefano Stabellini [mailto:sstabell...@kernel.org]
> Sent: 2019年2月6日 5:39
> To: xen-devel@lists.xenproject.org
> Cc: sstabell...@kernel.org; Stefano Stabellini ;
> julien.gr...@arm.com; Peng Fan ; jgr...@suse.com
> Subject: [PATCH for-4.12] xen/arm: gic-v2: deactivate interrupts during
> initialization
> 
> Interrupts could be ACTIVE at boot. Make sure to deactivate them during
> initialization.
> 
> Signed-off-by: Stefano Stabellini 
> CC: julien.gr...@arm.com
> CC: peng@nxp.com
> CC: jgr...@suse.com
> ---
>  xen/arch/arm/gic-v2.c | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c index
> e9fb8a0..256988c 100644
> --- a/xen/arch/arm/gic-v2.c
> +++ b/xen/arch/arm/gic-v2.c
> @@ -379,7 +379,10 @@ static void __init gicv2_dist_init(void)
> 
>  /* Disable all global interrupts */
>  for ( i = 32; i < nr_lines; i += 32 )
> +{
>  writel_gicd(~0x0, GICD_ICENABLER + (i / 32) * 4);
> +writel_gicd(~0x0, GICD_ICACTIVER + (i / 32) * 4);
> +}
> 
>  /* Turn on the distributor */
>  writel_gicd(GICD_CTL_ENABLE, GICD_CTLR); @@ -394,6 +397,7 @@
> static void gicv2_cpu_init(void)
>  /* The first 32 interrupts (PPI and SGI) are banked per-cpu, so
>   * even though they are controlled with GICD registers, they must
>   * be set up here with the other per-cpu state. */
> +writel_gicd(0x, GICD_ICACTIVER); /* Diactivate PPIs and
> + SGIs */
>  writel_gicd(0x, GICD_ICENABLER); /* Disable all PPI */
>  writel_gicd(0x, GICD_ISENABLER); /* Enable all SGI */
> 
> --

Reviewed-by: Peng Fan 

> 1.9.1

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI during initialization

[Peng Fan]

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@arm.com]
> Sent: 2019年2月3日 0:04
> To: Peng Fan ; sstabell...@kernel.org; jgr...@suse.com
> Cc: xen-devel@lists.xenproject.org
> Subject: Re: [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI during
> initialization
> 
> 
> 
> On 2/2/19 12:54 PM, Peng Fan wrote:
> > Hi Julien
> 
> Hi Peng,
> 
> >> -Original Message-
> >> From: Julien Grall [mailto:julien.gr...@arm.com]
> >> Sent: 2019年2月1日 18:41
> >> To: Peng Fan ; sstabell...@kernel.org;
> >> jgr...@suse.com
> >> Cc: xen-devel@lists.xenproject.org
> >> Subject: Re: [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI
> >> during initialization
> >>
> >> Hi,
> >>
> >> We spoke about SPIs in the previous version. Why aren't they
> >> de-activated here?
> >
> > According to GIC-V3 " 8.9.5 GICD_ICACTIVER, Interrupt Clear-Active
> Registers, n = 0 - 31"
> > "
> > Some or all RW fields of this register have defined reset values.
> > When this register has an architecturally-defined reset value, this field
> resets to 0.
> > "
> 
> I can't find this wording in the last spec (IHI0069E). Can you please give the
> version of the specific version of the spec when quoting it?

The spec I use is IHI0069D_gic_architecture_specification. Will add the version 
in
future patches when needed.

> 
> >
> > So I think we no need to deactivated, because it has reset values 0
> 
> The specification is about the reset value from the hardware, it does not tell
> you how the firmware (or the previous kernel when using kexec) is going to
> leave the interrupts. For instance, as I pointed out in a different thread, 
> Xen
> may reboot with SPIs activated. This can happen if you receive the
> reboot/power off request while handling an SPI.

Thanks, just sent out v3 including the deactivation of SPI.

> 
> >
> >>
> >> On 1/30/19 2:00 PM, Peng Fan wrote:
> >>> On i.MX8, we implemented partition reboot which means Cortex-A
> >>> reboot will not impact M4 cores and System control Unit core.
> >>> However GICv3 is not reset because we also need to support A72
> >>> Cluster reboot without affecting A53 Cluster.
> >>>
> >>> The gic-v3 controller is configured with EOImode to 1, so during xen
> >>> reboot, there is a function call "smp_call_function(halt_this_cpu, NULL,
> 0);"
> >>> , but halt_this_cpu never return, that means other CPUs have no
> >>> chance to deactivate the SGI interrupt, because the deactivate_irq
> >>> operation is at the end of do_sgi. During the next boot of Xen, CPU0
> >>> will issue GIC_SGI_CALL_FUNCTION to other CPUs. As the Active state
> >>> for SGI is left untouched during the reboot, the
> >>> GIC_SGI_CALL_FUNCTION will still be active on the non-boot CPUs.
> >>> This means the interrupt cannot be triggered again until it get
> deactivated.
> >>>
> >>> And according to IHI0069D_gic_architecture_specification, chapter
> >>> "8.11.3 GICR_ICACTIVER0, Interrupt Clear-Active Register 0", the RW
> >>> field of GICR_ICACTIVER0 resets to a value that is architecturally
> >> UNKNOWN.
> >>>
> >>> So set a fixed value during gic-v3 initialization to make sure
> >>> interrupts are in deactivated state.
> >>
> >> It is a bit unclear what you mean by "fixed value" here. The only
> >> thing you do is clearing active state. So a better wording is "So
> >> make sure all interrupts are deactivated at during initialization by 
> >> clearing
> the state".
> >>
> >> How about SPIs?
> >
> > Replied above. I could add the following to the patch if you think it
> > needed, I am not sure whether it is valid that SPI is in active state when 
> > Xen
> booting.
> 
> See above, I think this is valid and can happen today.
> 
> >
> > diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index
> > 1c1d2604f3..5b9c5559a7 100644
> > --- a/xen/arch/arm/gic-v3.c
> > +++ b/xen/arch/arm/gic-v3.c
> > @@ -622,9 +622,12 @@ static void __init gicv3_dist_init(void)
> >   writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
> >   }
> >
> > -/* Disable all global interrupts */
> > +/* Disable/deactivate all global interrupts */
> >   for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
> > +{
> >   writel_relaxed(0x, GICD + GICD_ICENABLER + (i / 32)
> > * 4);
> > +writel_relaxed(0x, GICD + GICD_ICACTIVER + (i / 32) * 4);
> > +}
> >
> >   /*
> >* Configure SPIs as non-secure Group-1. This will only matter
> 
> This chunk looks good. I will also write a patch for GICv2 doing the same.

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v3 for-4.12] arm: gic-v3: deactivate interrupts during initialization

[Peng Fan]

On i.MX8, we implemented partition reboot which means Cortex-A reboot
will not impact M4 cores and System control Unit core. However GICv3
is not reset because we also need to support A72 Cluster reboot without
affecting A53 Cluster.

The gic-v3 controller is configured with EOImode to 1, so during xen
reboot, there is a function call "smp_call_function(halt_this_cpu, NULL, 0);"
, but halt_this_cpu never return, that means other CPUs have no chance to
deactivate the SGI interrupt, because the deactivate_irq operation is at
the end of do_sgi. During the next boot of Xen, CPU0 will issue
GIC_SGI_CALL_FUNCTION to other CPUs. As the Active state for SGI is left
untouched during the reboot, the GIC_SGI_CALL_FUNCTION will still be active
on the non-boot CPUs. This means the interrupt cannot be triggered again
until it get deactivated.

And according to IHI0069D_gic_architecture_specification, chapter
"8.11.3 GICR_ICACTIVER0, Interrupt Clear-Active Register 0", the RW
field of GICR_ICACTIVER0 resets to a value that is architecturally UNKNOWN.
So make sure all interrupts are deactivated at during initialization by
clearing the state.

Signed-off-by: Peng Fan 
---

V3:
  SGI/PPI->interrupts.
  refine commit log.
  Deactivate SPI
V2:
  refine commit log
  deactivate SGI/PPI.
  No need for SPI, because Some or all RW fields of this GICD_ICACTIVER
  have defined reset values

 xen/arch/arm/gic-v3.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index c9200d24e1..0f6cbf6224 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -626,9 +626,12 @@ static void __init gicv3_dist_init(void)
 writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
 }
 
-/* Disable all global interrupts */
+/* Disable/deactivate all global interrupts */
 for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
+{
 writel_relaxed(0x, GICD + GICD_ICENABLER + (i / 32) * 4);
+writel_relaxed(0x, GICD + GICD_ICACTIVER + (i / 32) * 4);
+}
 
 /*
  * Configure SPIs as non-secure Group-1. This will only matter
@@ -834,6 +837,11 @@ static int gicv3_cpu_init(void)
 writel_relaxed(priority,
 GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
 
+/*
+ * The activate state is unknown at boot, so make sure all
+ * SGIs and PPIs are de-activated.
+ */
+writel_relaxed(0x, GICD_RDIST_SGI_BASE + GICR_ICACTIVER0);
 /*
  * Disable all PPI interrupts, ensure all SGI interrupts are
  * enabled.
-- 
2.14.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI during initialization

[Peng Fan]

Hi Julien

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@arm.com]
> Sent: 2019年2月1日 18:41
> To: Peng Fan ; sstabell...@kernel.org; jgr...@suse.com
> Cc: xen-devel@lists.xenproject.org
> Subject: Re: [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI during
> initialization
> 
> Hi,
> 
> We spoke about SPIs in the previous version. Why aren't they de-activated
> here?

According to GIC-V3 " 8.9.5 GICD_ICACTIVER, Interrupt Clear-Active 
Registers, n = 0 - 31"
"
Some or all RW fields of this register have defined reset values.
When this register has an architecturally-defined reset value, this field 
resets to 0.
"

So I think we no need to deactivated, because it has reset values 0.

> 
> On 1/30/19 2:00 PM, Peng Fan wrote:
> > On i.MX8, we implemented partition reboot which means Cortex-A reboot
> > will not impact M4 cores and System control Unit core. However GICv3
> > is not reset because we also need to support A72 Cluster reboot
> > without affecting A53 Cluster.
> >
> > The gic-v3 controller is configured with EOImode to 1, so during xen
> > reboot, there is a function call "smp_call_function(halt_this_cpu, NULL, 
> > 0);"
> > , but halt_this_cpu never return, that means other CPUs have no chance
> > to deactivate the SGI interrupt, because the deactivate_irq operation
> > is at the end of do_sgi. During the next boot of Xen, CPU0 will issue
> > GIC_SGI_CALL_FUNCTION to other CPUs. As the Active state for SGI is
> > left untouched during the reboot, the GIC_SGI_CALL_FUNCTION will still
> > be active on the non-boot CPUs. This means the interrupt cannot be
> > triggered again until it get deactivated.
> >
> > And according to IHI0069D_gic_architecture_specification, chapter
> > "8.11.3 GICR_ICACTIVER0, Interrupt Clear-Active Register 0", the RW
> > field of GICR_ICACTIVER0 resets to a value that is architecturally
> UNKNOWN.
> >
> > So set a fixed value during gic-v3 initialization to make sure
> > interrupts are in deactivated state.
> 
> It is a bit unclear what you mean by "fixed value" here. The only thing you do
> is clearing active state. So a better wording is "So make sure all interrupts 
> are
> deactivated at during initialization by clearing the state".
> 
> How about SPIs?

Replied above. I could add the following to the patch if you think it needed, I 
am
not sure whether it is valid that SPI is in active state when Xen booting.

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 1c1d2604f3..5b9c5559a7 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -622,9 +622,12 @@ static void __init gicv3_dist_init(void)
 writel_relaxed(priority, GICD + GICD_IPRIORITYR + (i / 4) * 4);
 }

-/* Disable all global interrupts */
+/* Disable/deactivate all global interrupts */
 for ( i = NR_GIC_LOCAL_IRQS; i < nr_lines; i += 32 )
+{
 writel_relaxed(0x, GICD + GICD_ICENABLER + (i / 32) * 4);
+writel_relaxed(0x, GICD + GICD_ICACTIVER + (i / 32) * 4);
+}

 /*
  * Configure SPIs as non-secure Group-1. This will only matter

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH for-4.12 v2] arm: gic-v3: deactivate SGI/PPI during initialization

[Peng Fan]

On i.MX8, we implemented partition reboot which means Cortex-A reboot
will not impact M4 cores and System control Unit core. However GICv3
is not reset because we also need to support A72 Cluster reboot without
affecting A53 Cluster.

The gic-v3 controller is configured with EOImode to 1, so during xen
reboot, there is a function call "smp_call_function(halt_this_cpu, NULL, 0);"
, but halt_this_cpu never return, that means other CPUs have no chance to
deactivate the SGI interrupt, because the deactivate_irq operation is at
the end of do_sgi. During the next boot of Xen, CPU0 will issue
GIC_SGI_CALL_FUNCTION to other CPUs. As the Active state for SGI is left
untouched during the reboot, the GIC_SGI_CALL_FUNCTION will still be active
on the non-boot CPUs. This means the interrupt cannot be triggered again
until it get deactivated.

And according to IHI0069D_gic_architecture_specification, chapter
"8.11.3 GICR_ICACTIVER0, Interrupt Clear-Active Register 0", the RW
field of GICR_ICACTIVER0 resets to a value that is architecturally UNKNOWN.

So set a fixed value during gic-v3 initialization to make sure
interrupts are in deactivated state.

Signed-off-by: Peng Fan 
---

V2:
  refine commit log
  deactivate SGI/PPI.
  No need for SPI, because Some or all RW fields of this GICD_ICACTIVER
  have defined reset values

 xen/arch/arm/gic-v3.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index 6fbc106757..1c1d2604f3 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -833,6 +833,11 @@ static int gicv3_cpu_init(void)
 writel_relaxed(priority,
 GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 + (i / 4) * 4);
 
+/*
+ * The activate state is unknown at boot, so make sure all
+ * SGIs and PPIs are de-activated.
+ */
+writel_relaxed(0x, GICD_RDIST_SGI_BASE + GICR_ICACTIVER0);
 /*
  * Disable all PPI interrupts, ensure all SGI interrupts are
  * enabled.
-- 
2.14.1


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: gic-v3: clear GICR active interrupts

[Peng Fan]

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@arm.com]
> Sent: 2019年1月30日 21:49
> To: Peng Fan ; sstabell...@kernel.org
> Cc: xen-devel@lists.xenproject.org; Andre Przywara
> 
> Subject: Re: [PATCH] arm: gic-v3: clear GICR active interrupts
> 
> Hi,
> 
> On 30/01/2019 13:36, Peng Fan wrote:
> >>> Each ICACTIVER0 registers hold the active bit for 32 interrupts.
> >>> However, this code assumes the register only hold 4 interrupts. So
> >>> this will write to unwanted area.
> >>>
> >>> There are only 16 SGIs, so it fits in one write to ICACTIVER0. As
> >>> wrote above, you also need to deactivate the PPIs. So the following
> >>> should
> >> be enough:
> >>>
> >>> /*
> >>>* The activate state is unknown at boot, so make sure all SGIs
> >>> and PPIs
> >> are
> >>>* de-activated.
> >>>*/
> >>> writel_relaxed(0x, GICD_RDIST_SGI_BASE + GICR_ICACTIVER0);
> >
> > Just have a following up question, according to
> > IHI0069D_gic_architecture_specification
> > "8.11.3 GICR_ICACTIVER0, Interrupt Clear-Active Register 0"
> > This field resets to a value that is architecturally UNKNOWN, Do we
> > need to take SPI into consideration as the following in gic_cpu_init?
> > for (i = 0; i < nr_lines; i +=32)
> > writel_relaxed(0x, GICD_RDIST_SGI_BASE + GICR_ICACTIVER0 + (i
> > / 32) * 4);
> >
> > and move nr_lines out from gic_dist_init to a static global varaiable.
> 
> You can't clear SPIs through GICD_RDIST_SGI_BASE. You have to use the
> GICD.

Oops. I got wrong.

> 
> Furthermore, gic_cpu_init is called for every CPU so this is not a good place
> for clearing shared interrupts. Shared interrupts should be cleared in
> gic_dist_init as this is called only once.

Yes.

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] arm: gic-v3: clear GICR active interrupts

[Peng Fan]

Hi Julien

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@arm.com]
> Sent: 2019年1月30日 20:06
> To: Peng Fan ; sstabell...@kernel.org
> Cc: xen-devel@lists.xenproject.org; Andre Przywara
> 
> Subject: Re: [PATCH] arm: gic-v3: clear GICR active interrupts
> 
> Hi
> 
> Replying to this thread again.
> 
> On 22/01/2019 10:54, Julien Grall wrote:
> > Hi Peng,
> >
> > The commit title is a bit confusing. It suggests that all interrupts
> > should be deactivated at boot, however you are only deactivating the SGIs.
> >
> > On 1/22/19 2:35 AM, Peng Fan wrote:
> >> On i.MX8, we implemented partition reboot which means Cortex-A reboot
> >> will not impact M4 cores and System control Unit core. However GICv3
> >> is not reset because hardware design.
> >
> > What do you mean by hardware design? Is it a defect?
> >
> >>
> >> The gic-v3 controller is configured with EOImode to 1, so during xen
> >> reboot, GIC_SGI_CALL_FUNCTION interrupt from CPU0 to other CPUs, but
> >> stop_cpu never return, that means other CPUs have no chance to
> >> deactive the interrupt. During xen booting again, CPU0 will issue
> >> GIC_SGI_CALL_FUNCTION to other CPUs. Because
> GIC_SGI_CALL_FUNCTION of
> >> other CPUs are active during the last reboot, interrupts could not be
> >> triggered unless we deactive the interrupt first.
> >
> >  From the description here, I think it not very sane to go to sleep
> > with an interrupt activate.
> 
> It looks like I was wrong here. There are case where you can't avoid that (see
> my answer to your other patch) and the boot code cannot rely on the activate
> state of interrupt. So they have to be cleaned at boot.
> 
> This also have to be done for PPI and SPIs. Peng, would you mind to do that
> patch?

I'll send out v2 patch soon.

> 
> >
> > A better solution would be to move the deactivation earlier on in
> > do_sgi (maybe after eoi_irq) so we call stop_cpu() with all interrupts
> disabled.
> >
> >>
> >> So let's deactive the interrupts during GICv3 initialization to fix
> >
> > s/deactivate/
> >
> >> this issue.
> >
> > Similarly to the commit title, you wrote the commit message very
> generically.
> >
> >
> >>
> >> Signed-off-by: Peng Fan 
> >> ---
> >>   xen/arch/arm/gic-v3.c | 4 
> >>   1 file changed, 4 insertions(+)
> >>
> >> diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c index
> >> 6fbc106757..643d4a33f0 100644
> >> --- a/xen/arch/arm/gic-v3.c
> >> +++ b/xen/arch/arm/gic-v3.c
> >> @@ -824,8 +824,12 @@ static int gicv3_cpu_init(void)
> >>   priority = (GIC_PRI_IPI << 24 | GIC_PRI_IPI << 16 | GIC_PRI_IPI
> >> << 8 |
> >>   GIC_PRI_IPI);
> >>   for (i = 0; i < NR_GIC_SGI; i += 4)
> >> +    {
> >>   writel_relaxed(priority,
> >>   GICD_RDIST_SGI_BASE + GICR_IPRIORITYR0 +
> (i / 4) *
> >> 4);
> >> +    writel_relaxed(0x,
> >> +    GICD_RDIST_SGI_BASE + GICR_ICACTIVER0 + (i /
> 4) *
> >> +4);
> 
> Each ICACTIVER0 registers hold the active bit for 32 interrupts. However, this
> code assumes the register only hold 4 interrupts. So this will write to
> unwanted area.
> 
> There are only 16 SGIs, so it fits in one write to ICACTIVER0. As wrote above,
> you also need to deactivate the PPIs. So the following should be enough:
> 
> /*
>   * The activate state is unknown at boot, so make sure all SGIs and PPIs are
>   * de-activated.
>   */
> writel_relaxed(0x, GICD_RDIST_SGI_BASE + GICR_ICACTIVER0);

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH for-4.12] arm: gic: deactivate sgi immediately after eoi

[Peng Fan]

Hi Julien

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@arm.com]
> Sent: 2019年1月29日 23:57
> To: Peng Fan ; sstabell...@kernel.org; jgr...@suse.com
> Cc: xen-devel@lists.xenproject.org; Andre Przywara
> 
> Subject: Re: [PATCH for-4.12] arm: gic: deactivate sgi immediately after eoi
> 
> (+ Andre)
> 
> Hi Peng,
> 
> On 24/01/2019 12:43, Peng Fan wrote:
> > On i.MX8, we implemented partition reboot which means Cortex-A reboot
> > will not impact M4 cores and System control Unit core. However GICv3
> > is not reset because we also need to support A72 Cluster reboot
> > without affecting A53 Cluster.
> 
> How about the other interrupts? For instance, it would be theoretically
> possible to have a PPI/SPI active and receive the order to turn it off. Is 
> this
> going to be an issue?

Currently we only met this issue when doing Xen reboot, because halt_this_cpu 
never return.
For other interrupts, I think they are deactivated properly.

> 
> >
> > The gic-v3 controller is configured with EOImode to 1, so during xen
> > reboot, there is a function call "smp_call_function(halt_this_cpu, NULL, 
> > 0);"
> > ,but halt_this_cpu never return, that means other CPUs have no chance
> > to
> 
> s/,/, /
> 
> > deactive the SGI interrupt, because the deactivate_irq operation is at
> 
> s/deactive/deactivate/
> 
> > the end of do_sgi. During xen booting again, CPU0 will issue
> 
> s/xen/Xen/
> 
> Also:
> 
> "During the next boot of Xen,"
> 
> > GIC_SGI_CALL_FUNCTION to other CPUs. Because
> GIC_SGI_CALL_FUNCTION of
> > other CPUs are active during the last reboot, interrupts could not be
> 
> s/of other/on the others/
> 
> > triggered unless we deactivate the interrupt first.
> 
> I think it would be clearer if you say: "As the Active state for SGI is left
> untouched during the reboot, the GIC_SGI_CALL_FUNCTION will still be
> active on the non-boot CPUs. This means the interrupt cannot be triggered
> again until it get deactivated."
> 
> >
> > To fix this issue, let's move the deactivate_irq operation just after
> > eoi_irq, then the SGI interrupt will be in deactive state when
> 
> s/deactive/deactivate/
> 
> > smp_call_function_interrupt.
> 
> I would add: "This is fine because the interrupts are masked while handling
> the
> SGI.".
> 
> >
> > Signed-off-by: Peng Fan 
> 
> I think the patch makes sense in itself. This is a saner behavior than turning
> off the CPU with active interrupts. I want to double-check a few things before
> committing it. But I will definitely commit it by the end of the week.
> 
> No need to resend the patch if there are no changes in the code. I can take
> care
> for the commit message.

Thanks,
Peng.

> 
> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain

[Peng Fan]

> -Original Message-
> From: h...@infradead.org [mailto:h...@infradead.org]
> Sent: 2019年1月28日 16:00
> To: Peng Fan 
> Cc: h...@infradead.org; Stefano Stabellini ;
> m...@redhat.com; jasow...@redhat.com; xen-devel@lists.xenproject.org;
> linux-remotep...@vger.kernel.org; linux-ker...@vger.kernel.org;
> virtualizat...@lists.linux-foundation.org; l...@kernel.org; jgr...@suse.com;
> boris.ostrov...@oracle.com; Andy Duan 
> Subject: Re: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain
> 
> On Fri, Jan 25, 2019 at 09:45:26AM +, Peng Fan wrote:
> > Just have a question,
> >
> > Since vmalloc_to_page is ok for cma area, no need to take cma and per
> > device cma into consideration right?
> 
> The CMA area itself it a physical memory region.  If it is a non-highmem
> region you can call virt_to_page on the virtual addresses for it.  If it is in
> highmem it doesn't even have a kernel virtual address by default.
> 
> > we only need to implement a piece code to handle per device specific
> > region using RESERVEDMEM_OF_DECLARE, just like:
> > RESERVEDMEM_OF_DECLARE(rpmsg-dma, "rpmsg-dma-pool",
> > rmem_rpmsg_dma_setup); And implement the device_init call back and
> > build a map between page and phys.
> > Then in rpmsg driver, scatter list could use page structure, no need
> > vmalloc_to_page for per device dma.
> >
> > Is this the right way?
> 
> I think this should work fine.  If you have the cycles for it I'd actually 
> love to
> be able to have generic CMA DT glue for non DMA API driver allocations, as
> there obviously is a need for it.  So basically the same as above, just added
> to kernel/cma.c as a generic API.

Thanks for the hints. I'll try to add that.

Thanks,
Peng.


___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain

[Peng Fan]

Hi,

> -Original Message-
> From: h...@infradead.org [mailto:h...@infradead.org]
> Sent: 2019年1月24日 5:14
> To: Stefano Stabellini 
> Cc: h...@infradead.org; Peng Fan ; m...@redhat.com;
> jasow...@redhat.com; xen-devel@lists.xenproject.org;
> linux-remotep...@vger.kernel.org; linux-ker...@vger.kernel.org;
> virtualizat...@lists.linux-foundation.org; l...@kernel.org; jgr...@suse.com;
> boris.ostrov...@oracle.com
> Subject: Re: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain
> 
> On Wed, Jan 23, 2019 at 01:04:33PM -0800, Stefano Stabellini wrote:
> > If vring_use_dma_api is actually supposed to return true when
> > dma_dev->dma_mem is set, then both Peng's patch and the patch I wrote
> > are not fixing the real issue here.
> >
> > I don't know enough about remoteproc to know where the problem
> > actually lies though.
> 
> The problem is the following:
> 
> Devices can declare a specific memory region that they want to use when the
> driver calls dma_alloc_coherent for the device, this is done using the
> shared-dma-pool DT attribute, which comes in two variants that would be a
> little to much to explain here.
> 
> remoteproc makes use of that because apparently the device can only
> communicate using that region.  But it then feeds back memory obtained
> with dma_alloc_coherent into the virtio code.  For that it calls
> vmalloc_to_page on the dma_alloc_coherent, which is a huge no-go for the
> ĐMA API and only worked accidentally on a few platform, and apparently
> arm64 just changed a few internals that made it stop working for remoteproc.
> 
> The right answer is to not use the DMA API to allocate memory from a
> device-speficic region, but to tie the driver directly into the DT reserved
> memory API in a way that allows it to easilt obtain a struct device for it.

Just have a question, 

Since vmalloc_to_page is ok for cma area, no need to take cma and per device
cma into consideration right? 

we only need to implement a piece code to handle per device specific region
using RESERVEDMEM_OF_DECLARE, just like:
RESERVEDMEM_OF_DECLARE(rpmsg-dma, "rpmsg-dma-pool", 
rmem_rpmsg_dma_setup);
And implement the device_init call back and build a map between page and phys.
Then in rpmsg driver, scatter list could use page structure, no need 
vmalloc_to_page
for per device dma.

Is this the right way?

Thanks
Peng.

> 
> This is orthogonal to another issue, and that is that hardware virtio devices
> really always need to use the DMA API, otherwise we'll bypass such features
> as the device specific DMA pools, DMA offsets, cache flushing, etc, etc.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel