Gaurav Batra <[email protected]> writes: > In powerPC, if Dynamic DMA Window is big enough, RAM is pre-mapped. To > determine the size of RAM, a PAPR+ property "ibm,lrdr-capacity" is used. > This OF property dictates what is the max size of RAM an LPAR can have, > including DR added memory. > > In PowerPC, 16GB pages can be allocated at machine level and then > assigned to LPARs. These 16GB pages are added to LPAR memory at the time > of boot. The address range for these 16GB pages is above MAX RAM an LPAR > can have (ibm,lrdr-capacity). In the current implementation, these 16GB > pages are being excluded from pre-mapped TCEs. A driver can have DMA > buffers allocated from 16GB pages. This results in platform to raise an > EEH when DMA is attempted on buffers in 16GB memory range. > > commit 6aa989ab2bd0 ("powerpc/pseries/iommu: memory notifier incorrectly > adds TCEs for pmemory") > > Prior to the above patch, memblock_end_of_DRAM() was being used to > determine the MAX memory of an LPAR. This included 16GB pages as well. > The issue with using memblock_end_of_DRAM() is that when pmemory is > converted to RAM via daxctl command, the DDW engine will incorrectly try > to add TCEs for pmemory as well. > > Below is the address distribution of RAM, 16GB pages and pmemory for an > LPAR with max memory of 256GB, memory allocated 64GB, 2 16GB pages and > assigned pmemory of 8GB. > > RANGE SIZE STATE REMOVABLE BLOCK > 0x0000000000000000-0x0000000fffffffff 64G online yes 0-255 > 0x0000004000000000-0x00000047ffffffff 32G online yes 1024-1151 > > cat /sys/bus/nd/devices/region0/resource > 0x40100000000 > cat /sys/bus/nd/devices/region0/size > 8589934592 > > The approach to fix this problem is to revert back the code changes > introduced by the above patch and to stash away the MAX memory of an > LPAR, including 16GB pages, at the LPAR boot time. This value is then > used whenever TCEs are needed to be pre-mapped - enable_DDW() or, > iommu_mem_notifier() > > Fixes: 6aa989ab2bd0 ("powerpc/pseries/iommu: memory notifier incorrectly adds > TCEs for pmemory") > Cc: [email protected] > Signed-off-by: Gaurav Batra <[email protected]>
Thanks for addressing the review comments. BTW - we should really cleanup enable_ddw() function. It has so many corner cases and has started to become confusing. But as for this patch it looks good to me. So, please feel free to add: Reviewed-by: Ritesh Harjani (IBM) <[email protected]>
