On Tue, Feb 10, 2026 at 05:30:10PM +0100, Niklas Cassel wrote: > On Tue, Feb 10, 2026 at 11:07:16PM +0900, Koichiro Den wrote: > > On Tue, Feb 10, 2026 at 01:24:29PM +0100, Niklas Cassel wrote: > > > On Mon, Feb 09, 2026 at 09:53:08PM +0900, Koichiro Den wrote: > > > > Tested on > > > > ========= > > > > > > > > I tested the embedded (DMA) doorbell fallback path (via pci-epf-test) on > > > > R-Car Spider boards: > > > > > > > > $ ./pci_endpoint_test -t DOORBELL_TEST > > > > TAP version 13 > > > > 1..1 > > > > # Starting 1 tests from 1 test cases. > > > > # RUN pcie_ep_doorbell.DOORBELL_TEST ... > > > > # OK pcie_ep_doorbell.DOORBELL_TEST > > > > ok 1 pcie_ep_doorbell.DOORBELL_TEST > > > > # PASSED: 1 / 1 tests passed. > > > > # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 > > > > > > > > with the following message observed on the EP side: > > > > > > > > [ 80.464653] pci_epf_test pci_epf_test.0: Using embedded (DMA) > > > > doorbell fallback > > > > > > > > (Note: for the test to pass on R-Car Spider, one of the following was > > > > required: > > > > - echo 1048576 > functions/pci_epf_test/func1/pci_epf_test.0/bar2_size > > > > - apply > > > > https://lore.kernel.org/all/[email protected]/) > > > > > > I applied this series on top of branch pci/controller/dwc > > > on Rock 5B (pcie-dw-rockchip.c). > > > > > > On EP side: > > > [ 39.218533] pci_epf_test pci_epf_test.0: Can't find MSI domain for EPC > > > [ 39.219125] pci_epf_test pci_epf_test.0: Using embedded (DMA) doorbell > > > fallback > > > > > > On RC side: > > > # RUN pcie_ep_doorbell.DOORBELL_TEST ... > > > [ 40.297892] pci-endpoint-test 0000:01:00.0: Failed to trigger doorbell > > > in endpoint > > > # pci_endpoint_test.c:279:DOORBELL_TEST:Expected 0 (0) == ret (-22) > > > # pci_endpoint_test.c:279:DOORBELL_TEST:Test failed for Doorbell > > > > > > # DOORBELL_TEST: Test failed > > > # FAIL pcie_ep_doorbell.DOORBELL_TEST > > > not ok 23 pcie_ep_doorbell.DOORBELL_TEST > > > > > > Any suggestions? > > > > > > (All BARs in pcie-dw-rockchip.c is marked as BAR_RESIZABLE.) > > > > Thank you for testing. > > > > If the failure was observed in a scenario other than a plain > > `./pci_endpoint_test -t DOORBELL_TEST`, could you please try again with [1] > > applied as well? > > > > [1] > > https://lore.kernel.org/linux-pci/[email protected]/ > > I applied that series, but I got the same problem. > > I added debug, and the EP side does use the correct address for the eDMA: > [ 26.279457] msg_addr: 0xa403800a0 > [ 26.279898] phys_addr: 0xa40300000 offset: 0x800a0 > > > If I write to the msg_addr directly on the EP using devmem, I do see the print > that I added in the IRQ handler: > # devmem 0xa403800a0 32 0 > [ 155.861989] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 158.809160] dw_edma_interrupt_emulated:696 > [ 158.809543] pci_epf_test_doorbell_primary:729 > # [ 158.809986] pci_epf_test_doorbell_handler:703 > # devmem 0xa403800a0 32 0 > [ 161.241326] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 163.466054] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 167.378662] dw_edma_interrupt_emulated:696 > [ 167.379045] pci_epf_test_doorbell_primary:729 > # [ 167.379512] pci_epf_test_doorbell_handler:703 > # devmem 0xa403800a0 32 0 > [ 168.880179] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 170.492176] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 171.729154] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 173.481271] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 174.985787] dw_edma_interrupt_emulated:696 > # devmem 0xa403800a0 32 0 > [ 176.517131] dw_edma_interrupt_emulated:696 > [ 176.517511] pci_epf_test_doorbell_primary:729 > # [ 176.517963] pci_epf_test_doorbell_handler:703 > > But not on every write.... > > I'm not sure, but could this perhaps be because we are missing this patch: > https://lore.kernel.org/dmaengine/[email protected]/
Thank you for the detailed debugging. I don't have a Rock 5B to reproduce locally, but your log indicates that the emulated interrupt is not always delivered on the same eDMA IRQ line. On RK3588 (rk3588-extra.dtsi) there are multiple eDMA IRQs (dma0-4), while pci-epf-test requests only epf->db_msg[0].virq (IRQ 53 in your /proc/interrupts). For R-Car S4 Spider, chip->nr_irqs == 1 and I wasn't able to verify whether my earlier concern here: https://lore.kernel.org/linux-pci/p4ommmpcjegvb4lafzecf67tzmdodtuqexeoifcn5eh7xqyp2y@ss76d3ubbsw7/ > The proposed dmaengine_{register,unregister}_selfirq() APIs are > device-wide (i.e. not per channel), so I'm not sure which "channel" you > refer to here. Also, when chip->nr_irqs > 1 on EP, dw-edma distributes > channels across multiple IRQ vectors, and it's unclear (at least to me) > which IRQ vector the emulated interrupt ("fake irq") is expected to be > delivered on. actually holds true in practice. Your report makes it clear that the emulated interrupt can indeed be delivered on different IRQ vectors. One hypothesis is that: we currently program msg.data = 0 for the "embedded" doorbell, and we write to DMA_READ_INT_STATUS_OFF. The register field (RD_DONE_INT_STATUS) is defined per-channel, and the register supports interrupt emulation by writes, so it might be that writing BIT(n) selects the channel/irq line, while writing 0 does not consistently map to a specific one. Could you try a quick experiment on the Rock 5B EP side? devmem 0xa403800a0 32 1 devmem 0xa403800a0 32 2 devmem 0xa403800a0 32 4 devmem 0xa403800a0 32 8 and see if the interrupt consistently lands on a specific one of IRQ53-56 for each value? If that is the case, we can make msg.data non-zero value instead of always 0. If that is not the case, then we may need to consider two less ideal options: - switch back to the ~v3 approach, where we run the registered hook exactly at the time when the emulated interrupt is deasserted. (ref. https://lore.kernel.org/linux-pci/[email protected]/) - or, require users to request_irq() for all irq vectors associated with all channels. However, this would not be very attractive from a design perspective. > > # dmesg | grep eDMA > [ 1.243339] rockchip-dw-pcie a40000000.pcie-ep: eDMA: unroll T, 2 wr, 2 rd > > # cat /proc/interrupts | grep edma > 53: 8 0 0 0 0 0 > 0 0 GICv3 303 Level dw-edma-core:a40000000.pcie-ep, > pci-ep-test-doorbell > 54: 7 0 0 0 0 0 > 0 0 GICv3 304 Level dw-edma-core:a40000000.pcie-ep > 55: 15 0 0 0 0 0 > 0 0 GICv3 301 Level dw-edma-core:a40000000.pcie-ep > 56: 7 0 0 0 0 0 > 0 0 GICv3 302 Level dw-edma-core:a40000000.pcie-ep > > > > Anyway, I was still curious why this did never worked when writing from the > host side, even when running the test case many, many times. > AFAICT, the inbound translation looks correct. > > RK3588 (pcie-dw-rockchip.c) exposes the DMA registers in BAR4 by default. > If I hack pci-epf-test on top of your patch to unconditionally return BAR4 > with > offset 0xa0, it works. So my best guess is that the fixed inbound translation > in BAR4 (to the eDMA registers) somehow messes with the inbound translation if > another BAR tries to use an inbound translation to the eDMA registers as well. Thanks a lot for letting me know that. I see two possible ways forward: (a) extend PCI_EPC_AUX_DMA_CTRL_MMIO to optionally describe that the DMA MMIO window is already mapped to a fixed BAR and should be reused, so EPFs avoid creating a second mapping to the same target. I guess it could be treated as a quirk for "rockchip,rk3588-pcie-ep". (b) alternatively, clear the default BAR4 mapping on RK3588 at least temporarily when using the pci-epf-msi doorbell fallback. Koichiro > > > Kind regards, > Niklas >
