Hi!
I couldn't figure out what to log in mhi_init_irq_setup but i put a logline
on in mhi_init_dev_ctxt:
oskar@oskar-ThinkPad-X1-Nano-Gen-1:~/src/linux-kernel/linux-upstream-5.12.0+/drivers/bus/mhi$
git diff .
diff --git a/drivers/bus/mhi/core/init.c b/drivers/bus/mhi/core/init.c
index c81b377fca8f..732affde7a20 100644
--- a/drivers/bus/mhi/core/init.c
+++ b/drivers/bus/mhi/core/init.c
@@ -266,6 +266,7 @@ int mhi_init_dev_ctxt(struct mhi_controller *mhi_cntrl)
struct mhi_cmd *mhi_cmd;
u32 tmp;
int ret = -ENOMEM, i;
+ struct device *dev = &mhi_cntrl->mhi_dev->dev;
atomic_set(&mhi_cntrl->dev_wake, 0);
atomic_set(&mhi_cntrl->pending_pkts, 0);
@@ -333,6 +334,9 @@ int mhi_init_dev_ctxt(struct mhi_controller *mhi_cntrl)
er_ctxt->ertype = MHI_ER_TYPE_VALID;
er_ctxt->msivec = mhi_event->irq;
+ dev_warn(dev, "er_ctxt->msivec::%d\n",
+ er_ctxt->msivec);
+
mhi_event->db_cfg.db_mode = true;
ring->el_size = sizeof(struct mhi_tre);
With intel vt-d disabled:
[ 64.428320] mhi: loading out-of-tree module taints kernel.
[ 64.428455] mhi: module verification failed: signature and/or required
key missing - tainting kernel
[ 98.714766] mhi-pci-generic 0000:08:00.0: BAR 0: assigned [mem
0xbc200000-0xbc200fff 64bit]
[ 98.714900] mhi-pci-generic 0000:08:00.0: enabling device (0000 -> 0002)
[ 98.715245] mhi-pci-generic 0000:08:00.0: using shared MSI
[ 98.715611] mhi mhi0: er_ctxt->msivec::0
[ 98.715614] mhi mhi0: er_ctxt->msivec::0
[ 98.715617] mhi mhi0: er_ctxt->msivec::0
[ 98.715623] mhi mhi0: er_ctxt->msivec::0
[ 98.715631] mhi mhi0: Requested to power ON
[ 98.716156] mhi mhi0: Power on setup success
[ 98.750461] mhi mhi0: Wait for device to enter SBL or Mission mode
With intel vt-d enabled:
[ 64.750646] mhi: loading out-of-tree module taints kernel.
[ 64.750776] mhi: module verification failed: signature and/or required
key missing - tainting kernel
[ 71.229020] mhi-pci-generic 0000:08:00.0: BAR 0: assigned [mem
0xbc200000-0xbc200fff 64bit]
[ 71.229153] mhi-pci-generic 0000:08:00.0: enabling device (0000 -> 0002)
[ 71.230017] mhi mhi0: er_ctxt->msivec::1
[ 71.230021] mhi mhi0: er_ctxt->msivec::2
[ 71.230024] mhi mhi0: er_ctxt->msivec::3
[ 71.230030] mhi mhi0: er_ctxt->msivec::4
[ 71.230038] mhi mhi0: Requested to power ON
[ 71.231107] mhi mhi0: Power on setup success
[ 71.264612] mhi mhi0: Wait for device to enter SBL or Mission mode
Do i understand it correctly that this is correctly telling the modem to
use shared irq and that this might be a modem firmware-bug?
/Oskar Stenman
On Tue, 18 May 2021 at 12:16, Loic Poulain <loic.poul...@linaro.org> wrote:
> Hi Oskar,
>
> On Tue, 18 May 2021 at 11:33, Oskar Stenman <os...@cetex.se> wrote:
> >
> > Hi!
> >
> > I'm way out in the deep end of the pool here poking at things i
> shouldn't to see what happens and if i can learn something, or find a bit
> more info where things go wrong.. So I patched
> drivers/bus/mhi/pci_generic.c to give some more debug info:
> >
> > oskar@oskar-ThinkPad-X1-Nano-Gen-1:~/src/linux-kernel/linux-upstream-5.12.0+$
> git diff
> > diff --git a/drivers/bus/mhi/pci_generic.c
> b/drivers/bus/mhi/pci_generic.c
> > index 7c810f02a2ef..819c0f17993d 100644
> > --- a/drivers/bus/mhi/pci_generic.c
> > +++ b/drivers/bus/mhi/pci_generic.c
> > @@ -492,9 +492,13 @@ static int mhi_pci_get_irqs(struct mhi_controller
> *mhi_cntrl,
> > * Alloc one MSI vector for BHI + one vector per event ring,
> ideally...
> > * No explicit pci_free_irq_vectors required, done by
> pcim_release.
> > */
> > + dev_warn(&pdev->dev, "mhi_cntrl_config->num_events: %d\n",
> mhi_cntrl_config->num_events);
> > mhi_cntrl->nr_irqs = 1 + mhi_cntrl_config->num_events;
> > + dev_warn(&pdev->dev, "mhi_cntrl->nr_irqs: %d\n",
> mhi_cntrl->nr_irqs);
> > + dev_warn(&pdev->dev, "PCI_IRQ_MSI: %d\n", PCI_IRQ_MSI);
> >
> > nr_vectors = pci_alloc_irq_vectors(pdev, 1, mhi_cntrl->nr_irqs,
> PCI_IRQ_MSI);
> > + dev_warn(&pdev->dev, "nr_vectors: %d\n", nr_vectors);
> > if (nr_vectors < 0) {
> > dev_err(&pdev->dev, "Error allocating MSI vectors %d\n",
> > nr_vectors);
> >
> > Compiling and loading that module prints the following in dmesg:
> > [ 5553.096754] mhi-pci-generic 0000:08:00.0: BAR 0: assigned [mem
> 0xbc200000-0xbc200fff 64bit]
> > [ 5553.097033] mhi-pci-generic 0000:08:00.0:
> mhi_cntrl_config->num_events: 4
> > [ 5553.097036] mhi-pci-generic 0000:08:00.0: mhi_cntrl->nr_irqs: 5
> > [ 5553.097038] mhi-pci-generic 0000:08:00.0: PCI_IRQ_MSI: 2
> > [ 5553.097314] mhi-pci-generic 0000:08:00.0: nr_vectors: 1 <- requested
> 5, got 1..
> > [ 5553.097316] mhi-pci-generic 0000:08:00.0: using shared MSI
> >
> > requesting 5 nr_vectors, getting one which then seems to trigger another
> codepath..
> >
> > Whlie researching pci_alloc_irq_vectors I then found this semi-related
> thread
> https://stackoverflow.com/questions/49821599/multiple-msi-vectors-linux-pci-alloc-irq-vectors-return-one-while-the-devi
> where Riton38 mentions that enabling intel vt-d "solved the problem"
> (default from factory on this Thinkpad X1 Nano it has all virtualization
> disabled).
> >
> > Since i can't figure out what's wrong with the actual irq-setup as it's
> possibly bug in modem firmware (throwing interrupts over some channel when
> it shouldn't), or maybe a bug in how irq's are allocated in mhi_pci_generic
> i just tried enabling Intel VT-D in bios (security -> virtualization) and
> what would you know, it works!!
> >
> > [ 81.291874] mhi_pci_generic: loading out-of-tree module taints kernel.
> > [ 81.291970] mhi_pci_generic: module verification failed: signature
> and/or required key missing - tainting kernel
> > [ 81.292527] mhi-pci-generic 0000:08:00.0: BAR 0: assigned [mem
> 0xbc200000-0xbc200fff 64bit]
> > [ 81.292560] mhi-pci-generic 0000:08:00.0: enabling device (0000 ->
> 0002)
> > [ 81.292603] mhi-pci-generic 0000:08:00.0:
> mhi_cntrl_config->num_events: 4
> > [ 81.292605] mhi-pci-generic 0000:08:00.0: mhi_cntrl->nr_irqs: 5
> > [ 81.292606] mhi-pci-generic 0000:08:00.0: PCI_IRQ_MSI: 2
> > [ 81.292713] mhi-pci-generic 0000:08:00.0: nr_vectors: 5 <- We got 5
> vectors now!
> > [ 81.292955] mhi mhi0: Requested to power ON
> > [ 81.293180] mhi mhi0: Power on setup success
> > [ 81.325854] mhi mhi0: Wait for device to enter SBL or Mission mode
> >
> > After running qmicli and connecting, no more errors in kernel log.
> >
> > oskar@oskar-ThinkPad-X1-Nano-Gen-1:~/src/linux-kernel/linux-upstream-5.12.0+/drivers/bus/mhi$
> ping ftp.sunet.se
> > PING ftp.sunet.se(tutankhamon.ftp.acc.umu.se (2001:6b0:19::163)) 56
> data bytes
> > 64 bytes from tutankhamon.ftp.acc.umu.se (2001:6b0:19::163): icmp_seq=1
> ttl=51 time=46.1 ms
> > 64 bytes from tutankhamon.ftp.acc.umu.se (2001:6b0:19::163): icmp_seq=2
> ttl=51 time=43.9 ms
> > 64 bytes from tutankhamon.ftp.acc.umu.se (2001:6b0:19::163): icmp_seq=3
> ttl=51 time=42.7 ms
> > 64 bytes from tutankhamon.ftp.acc.umu.se (2001:6b0:19::163): icmp_seq=4
> ttl=51 time=49.0 ms
> > ^C
> > --- ftp.sunet.se ping statistics ---
> > 4 packets transmitted, 4 received, 0% packet loss, time 3005ms
> > rtt min/avg/max/mdev = 42.708/45.419/49.019/2.404 ms
> >
> > Speedtest now gives decent speed, 50Mbit/s down and 13.4Mbit/ up where i
> before only got +1000ms latency and 7Mbit/s down where I am (just outside
> of central stockholm)
> > https://www.speedtest.net/result/11435773983
> >
> > Anything i can do to test / debug interrupts without intel vt-d enabled
> while I have this fresh in memory?
>
> Yes, without intel VT-D (and IRQ_REMAP kernel config), it's not possible
> to request multiple MSI, that's indeed why you get the warning. Normally,
> each MHI event ring has its own dedicated MSI (to isolate events). But
> because we noticed vt-d is not necessarily enabled by default, we
> introduced a fallback to use a single shared MSI. That also why you get
> multiple irq names (bhi, mhi, mhi....) for the same IRQ index:
>
> >> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> CPU6 CPU7
> >> 143: 0 0 0 0 0 6
> 0 192 PCI-MSI 4194304-edge bhi, mhi, mhi, mhi, mhi
>
> related patch:
> https://git.kernel.org/pub/scm/linux/kernel/git/mani/mhi.git/commit/?h=mhi-next&id=b91c3b30e2267265cd7e67cb3d0c99c48c02b001
>
> This shared IRQ mechanism has been validated with Telit FN980m WWAN module.
>
> In your case, the modem seems to trigger a different interrupt than the
> shared one... I would suggest checking in the MHI core, especially in
> mhi_init_irq_setup() for the host side and in mhi_init_dev_ctxt() for the
> modem side, that misvec index is set to the same value (0).
>
> If everything is correct, it's possibly a bug on modem/firmware side.
>
> Regards,
> Loic
>
_______________________________________________
ModemManager-devel mailing list
ModemManager-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/modemmanager-devel
