Re: swiotlb/caamjr regression (Was: [GIT PULL] (swiotlb) stable/for-linus-5.12)

> Jianxiong Gao, before spending more time on this, could you also try > Chanho Park's patch? > https://lore.kernel.org/linux-iommu/20210510091816.ga2...@lst.de/T/#m0d0df6490350a08dcc24c9086c8edc165b402d6f > I have tested Chanho Parks's patch and it works for us. The NVMe driver performs correctly

Re: swiotlb/caamjr regression (Was: [GIT PULL] (swiotlb) stable/for-linus-5.12)

On Fri, Jun 11, 2021 at 3:35 AM Konrad Rzeszutek Wilk wrote: > > On Fri, Jun 11, 2021 at 08:21:53AM +0200, Christoph Hellwig wrote: > > On Thu, Jun 10, 2021 at 05:52:07PM +0300, Horia Geantă wrote: > > > I've noticed the failure also in v5.10 and v5.11 stable kernels, > > > since the patch set

Re: preserve DMA offsets when using swiotlb v2

> Sorry for being a little pushy, any chance we could get this reviewed > in time for the 5.12 merge window? > Tested and it looks good. Thanks for clearing it up! Gao ___ iommu mailing list iommu@lists.linux-foundation.org

Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

> > Please try with this extra patch: > I have tried with the extra patch and it still fails to boot. I have attached dmesg output for the error: -dmesg starts here- [6.357755] XFS (nvme0n1p2): Mounting V5 Filesystem [6.430092] XFS (nvme0n1p2): Torn write (CRC

Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

> Why is this setting being done and undone on each IO? Wouldn't it be > more efficient to set it once during device initialization? I agree that setting it once is the right way of doing it. So I have changed the patch to enable the mask once in nvme_probe. drivers/nvme/host/pci.c | 3 +++ 1

Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

> Why is this setting being done and undone on each IO? Wouldn't it be > more efficient to set it once during device initialization? > > And more importantly, this isn't thread safe: one CPU may be setting the > device's dma alignment mask to 0 while another CPU is expecting it to be >

Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

> > On Mon, Feb 01, 2021 at 10:30:17AM -0800, Jianxiong Gao wrote: > > @@ -868,12 +871,24 @@ static blk_status_t nvme_map_data(struct nvme_dev > *dev, struct request *req, > > if (!iod->nents) > > goto out_free_sg; > > > > + offset_ret = dma_set_min_align_mask(dev->dev,

Re: [PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

On Mon, Feb 1, 2021 at 10:56 AM Andy Shevchenko wrote: > > On Mon, Feb 01, 2021 at 10:30:17AM -0800, Jianxiong Gao wrote: > > NVMe driver relies on the address offset to function properly. > > This patch adds the offset preserve mask to NVMe driver when mapping > > via dma_map_sg_attrs and

[PATCH V2 1/3] Adding page_offset_mask to device_dma_parameters

Some devices rely on the address offset in a page to function correctly (NVMe driver as an example). These devices may use a different page size than the Linux kernel. The address offset has to be preserved upon mapping, and in order to do so, we need to record the page_offset_mask first.

[PATCH V2 2/3] Add swiotlb offset preserving mapping when dma_dma_parameters->page_offset_mask is non zero.

For devices that need to preserve address offset on mapping through swiotlb, this patch adds offset preserving based on page_offset_mask and keeps the offset if the mask is non zero. This is needed for device drivers like NVMe. Signed-off-by: Jianxiong Gao --- kernel/dma/swiotlb.c | 27

[PATCH V2 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

NVMe driver relies on the address offset to function properly. This patch adds the offset preserve mask to NVMe driver when mapping via dma_map_sg_attrs and unmapping via nvme_unmap_sg. The mask depends on the page size defined by CC.MPS register of NVMe controller. Signed-off-by: Jianxiong Gao

[PATCH V2 0/3] SWIOTLB: Preserve swiotlb map offset when needed.

NVMe driver and other applications may depend on the data offset to operate correctly. Currently when unaligned data is mapped via SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. This patch adds an option to make sure the mapped data preserves its offset of the orginal addrss.

Re: [PATCH 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

I have tried to set it once at probe and then leave it in place, however the NVMe driver does not seem to like it, and the VM does not boot correctly. I have spent a couple days debugging but I am a bit lost now. Basically whenever nvme_setup_prp_simple

Re: [PATCH 2/3] Add swiotlb offset preserving mapping when dma_dma_parameters->page_offset_mask is non zero.

The error can't be fixed by just updating the NVMe driver. The NVMe spec (and as pointed out by Chirstoph, some other drivers) rely on the offset of address to copy data correctly. When data is mapped via swiotlb, the current implementation always copy the data at 2k/4k aligned address. On Thu,

[PATCH 0/3] Adding offset keeping option when mapping data via SWIOTLB.*

NVMe driver and other applications may depend on the data offset to operate correctly. Currently when unaligned data is mapped via SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. This patch adds an option to make sure the mapped data preserves its offset of the orginal addrss.

[PATCH 1/3] Adding page_offset_mask to device_dma_parameters

Some devices rely on the address offset in a page to function correctly (NVMe driver as an example). These devices may use a different page size than the Linux kernel. The address offset has to be preserved upon mapping, and in order to do so, we need to record the page_offset_mask first.

[PATCH 2/3] Add swiotlb offset preserving mapping when dma_dma_parameters->page_offset_mask is non zero.

For devices that need to preserve address offset on mapping through swiotlb, this patch adds offset preserving based on page_offset_mask and keeps the offset if the mask is non zero. This is needed for device drivers like NVMe. Signed-off-by: Jianxiong Gao --- kernel/dma/swiotlb.c | 25

[PATCH 3/3] Adding device_dma_parameters->offset_preserve_mask to NVMe driver.

NVMe driver relies on the address offset to function properly. This patch adds the offset preserve mask to NVMe driver when mapping via dma_map_sg_attrs and unmapping via nvme_unmap_sg. The mask depends on the page size defined by CC.MPS register of NVMe controller. Signed-off-by: Jianxiong Gao

Re: [PATCH] [PATCH] Keep offset when mapping data via SWIOTLB.

Hi Konrad, Thanks for testing it out. I have updated the patch and tested on 5.11.0-rc2+ with dhclient successfully. Could you please help me verify if the patch works on your side? Thank you NVMe driver and other applications depend on the data offset to operate correctly. Currently when

[PATCH] [PATCH] Keep offset when mapping data via SWIOTLB.

NVMe driver and other applications depend on the data offset to operate correctly. Currently when unaligned data is mapped via SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. When booting with --swiotlb=force option and using NVMe as interface, running mkfs.xfs on Rhel fails because

[PATCH] [PATCH] Adding offset keeping option when mapping data via SWIOTLB.

NVMe driver and other applications may depend on the data offset to operate correctly. Currently when unaligned data is mapped via SWIOTLB, the data is mapped as slab aligned with the SWIOTLB. When booting with --swiotlb=force option and using NVMe as interface, running mkfs.xfs on Rhel fails