Re: [PATCH] [ath10k] go back to using dma_alloc_coherent() for firmware scratch memory.

[Kalle Valo]

Adrian Chadd  writes:

> This reverts b057886524be060021e3cfad0ba8458c850330cd in 2015
> which converted this allocation from dma_map_coherent() to
> kzalloc() / dma_map_single().
>
> The current problem manifests when using later model NICs with larger
> (>700KiB) scratch spaces in memory.  Although the kzalloc call
> succeeds, the software IOMMU TLB code (via dma_map_single()) panics
> because it can't find 700KiB of linear physmem bounce buffers for DMA.
> Now, this is a bit of a silly failure mode for the dma map API,
> but it's what we currently have to play with.
>
> In these cases, doing kzalloc() works fine, but the dma_map_single()
> call fails.
>
> After chatting with Linus briefly about this, it indeed should be
> using dma_alloc_coherent() for doing larger device memory allocation
> that requires some kind of physical address mapping.
>
> You're not supposed to be using kzalloc and dma_map_* calls for
> large memory regions, and I'm guessing not for long-held mappings
> either.  Typically dma mappings should be temporary for DMA,
> not long held like these.
>
> Now, since hopefully the major annoying underlying problem has also been
> addressed (ie, ath10k is no longer tears down all of these allocations
> and reallocates them every time the vdevs are brought down) fragmentation
> should stop being such a touchy issue.  If it is though, using
> dma_alloc_coherent() use gets us access to the CMB APIs too relatively
> easily and ideally we would be allocating memory early in boot for
> exactly these reasons.
>
> Signed-off-by: Adrian Chadd 

So there are now three positive test results (including mine) so I would
like to queue this to 4.12 as an important bug fix.

But I think all of them was with x86 and I'm worried if this patch
creates problems with other architectures, especially on ARM? In the
original commit Felix wrote that coherent memory is constrained some
architectures. I would hate to go ping pong with this and reverting this
patch soon after, instead I would prefer to find a proper solution which
works for everyone.

Felix, what do you think? Can someone test this patch on LEDE with the
most problematic boards?

commit b057886524be060021e3cfad0ba8458c850330cd
Author: Felix Fietkau 
Date:   Mon Nov 30 19:32:01 2015 +0100

ath10k: do not use coherent memory for allocated device memory chunks

Coherent memory is more expensive to allocate (and constrained on some
architectures where it has to be pre-allocated). It is also completely
unnecessary, since the host has no reason to even access these allocated
memory spaces

Signed-off-by: Felix Fietkau 
Signed-off-by: Kalle Valo 

-- 
Kalle Valo

Re: [PATCH] [ath10k] go back to using dma_alloc_coherent() for firmware scratch memory.

[Sven Eckelmann]

On Montag, 1. Mai 2017 14:43:27 CEST Adrian Chadd wrote:
> This reverts b057886524be060021e3cfad0ba8458c850330cd in 2015
> which converted this allocation from dma_map_coherent() to
> kzalloc() / dma_map_single().
> 
> The current problem manifests when using later model NICs with larger
> (>700KiB) scratch spaces in memory.  Although the kzalloc call
> succeeds, the software IOMMU TLB code (via dma_map_single()) panics
> because it can't find 700KiB of linear physmem bounce buffers for DMA.
> Now, this is a bit of a silly failure mode for the dma map API,
> but it's what we currently have to play with.
[]
> 
> Signed-off-by: Adrian Chadd 
> ---
>  drivers/net/wireless/ath/ath10k/wmi.c | 35 
> ++-
>  1 file changed, 10 insertions(+), 25 deletions(-)

Thanks for investigating this. This partial revert fixes following crash for
me with QCA99X0 on amd64 with SWIOMMU:

[9.167281] DMA: Out of SW-IOMMU space for 689816 bytes at device 
:02:00.0
[9.174719] Kernel panic - not syncing: DMA: Random memory could be DMA 
read
[9.174719] 
[9.183560] CPU: 0 PID: 133 Comm: kworker/u32:6 Tainted: P   OE  
 4.9.0-0.bpo.2-amd64 #1 Debian 4.9.13-1~bpo8+1
[9.194666] Hardware name: Intel Corporation SandyBridge 
Platform/ETXe-SC T2 i3-2310E, BIOS CHR2R110 05/12/2011
[9.205033] Workqueue: ath10k_aux_wq ath10k_wmi_event_service_ready_work 
[ath10k_core]
[9.213151]   bd329cd5 0200 
a7b5c1247cb8
[9.220827]  bd17bc24 0008 a7b5c1247cc8 
a7b5c1247c60
[9.228523]  6036bf02 00020018 91fde700dfc8 

[9.236177] Call Trace:
[9.238692]  [] ? dump_stack+0x5c/0x77
[9.244165]  [] ? panic+0xe8/0x236
[9.251161]  [] ? swiotlb_map_page+0x16c/0x190
[9.259219]  [] ? 
ath10k_wmi_event_service_ready_work+0x4d5/0x680 [ath10k_core]
[9.272150]  [] ? process_one_work+0x14b/0x410
[9.280190]  [] ? worker_thread+0x65/0x4a0
[9.287869]  [] ? rescuer_thread+0x340/0x340
[9.295689]  [] ? kthread+0xe0/0x100
[9.302742]  [] ? __switch_to+0x2bb/0x700
[9.310213]  [] ? kthread_park+0x60/0x60
[9.317556]  [] ? ret_from_fork+0x25/0x30
[9.324992] Kernel Offset: 0x3c00 from 0x8100 
(relocation range: 0x8000-0xbfff)
[9.339429] ---[ end Kernel panic - not syncing: DMA: Random memory 
could be DMA read
[9.339429] 
[9.353900] [ cut here ]
[9.360274] WARNING: CPU: 0 PID: 133 at 
/home/zumbi/linux-4.9.13/arch/x86/kernel/smp.c:127 
update_process_times+0x40/0x50
[9.374724] Modules linked in: tpm_infineon iTCO_wdt iTCO_vendor_support 
ppdev wl(POE) evdev intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp 
kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel 
cryptd intel_cstate intel_uncore ath10k_pci ath10k_core intel_rapl_perf ath 
mac80211 i2c_kempld btusb btrtl btbcm btintel pcspkr bluetooth cfg80211 
serio_raw rfkill snd_hda_codec_realtek snd_hda_codec_generic battery 
snd_hda_intel parport_pc parport snd_hda_codec i915 snd_hda_core snd_hwdep 
shpchp video drm_kms_helper snd_pcm snd_timer snd drm soundcore mei_me button 
mei i2c_i801 i2c_smbus lpc_ich tpm_tis tpm_tis_core ac tpm autofs4 ext4 crc16 
jbd2 fscrypto mbcache sg sd_mod ata_generic ahci pata_jmicron libahci 
kempld_core mfd_core libata ehci_pci ehci_hcd scsi_mod crc32c_intel usbcore 
psmouse e1000e usb_common igb i2c_algo_bit dca ptp pps_core fan thermal fjes
[9.476736] CPU: 0 PID: 133 Comm: kworker/u32:6 Tainted: P   OE  
 4.9.0-0.bpo.2-amd64 #1 Debian 4.9.13-1~bpo8+1
[9.491320] Hardware name: Intel Corporation SandyBridge 
Platform/ETXe-SC T2 i3-2310E, BIOS CHR2R110 05/12/2011
[9.505260] Workqueue: ath10k_aux_wq ath10k_wmi_event_service_ready_work 
[ath10k_core]
[9.517103]   bd329cd5  

[9.528682]  bd0778a4 91fde101a140  
a7b5c1247b98
[9.540304]  bd0f7270 0003 91fde7012580 
bd0e7dc0
[9.551962] Call Trace:
[9.556381]   [9.558370]  [] ? 
dump_stack+0x5c/0x77
[9.567776]  [] ? __warn+0xc4/0xe0
[9.574863]  [] ? tick_sched_do_timer+0x30/0x30
[9.583059]  [] ? update_process_times+0x40/0x50
[9.591302]  [] ? tick_sched_handle.isra.13+0x20/0x50
[9.600021]  [] ? tick_sched_timer+0x38/0x70
[9.607872]  [] ? __hrtimer_run_queues+0xdc/0x250
[9.616158]  [] ? hrtimer_interrupt+0x99/0x190
[9.624100]  [] ? smp_apic_timer_interrupt+0x39/0x50
[9.632618]  [] ? apic_timer_interrupt+0x82/0x90
[9.640700]   [9.642681]  [] ? 
panic+0x1f2/0x236
[9.649615]  [] ? swiotlb_map_page+0x16c/0x190

Re: [PATCH] [ath10k] go back to using dma_alloc_coherent() for firmware scratch memory.

[Kalle Valo]

Su Kang Yin  writes:

> On 11 May 2017 at 11:55, Kalle Valo wrote:
>> Su Kang Yin writes:
>>
>>> Without this patch QCA9888 is not working. Also I have to update
>>> board2.bin from Kalle's git repo.
>>
>> More details would be good to know, as the behaviour seems to vary quite
>> a lot. What platform are you using, x86, some ARM board or what? And
>> what kernel exactly?
>
> Kernel 4.9.25 on X86 Intel Atom
>
> Panic message is something like that:

Thanks, this was very helpful.

-- 
Kalle Valo

Re: [PATCH] [ath10k] go back to using dma_alloc_coherent() for firmware scratch memory.

[Su Kang Yin]

On 11 May 2017 at 11:55, Kalle Valo wrote:
> Su Kang Yin writes:
>
>> Without this patch QCA9888 is not working. Also I have to update
>> board2.bin from Kalle's git repo.
>
> More details would be good to know, as the behaviour seems to vary quite
> a lot. What platform are you using, x86, some ARM board or what? And
> what kernel exactly?

Kernel 4.9.25 on X86 Intel Atom

Panic message is something like that:

Workqueue: ath10k_aux_wq ath10k_wmi_event_service_ready_work [ath10k_core]
task: 880036433a00 ti: 88003644 task.ti: 88003644
RIP: 0010:[]  [] new_slab+0x39a/0x410
RSP: 0018:880036443b58  EFLAGS: 00010092
RAX: 0006 RBX: 024082c4 RCX: 
RDX: 0006 RSI: 88021e30dd08 RDI: 88021e30dd08
RBP: 880036443b90 R08:  R09: 
R10:  R11: 0372 R12: 88021dc01200
R13: 88021dc00cc0 R14: 88021dc01200 R15: 0001
FS:  () GS:88021e30() knlGS:
CS:  0010 DS:  ES:  CR0: 80050033
CR2: 7f3e65c1c730 CR3: 01e06000 CR4: 001406e0
Stack:
  8127a4fc 0a01ff10 024082c4 88021dc01200
  88021dc00cc0 88021dc01200 0001 880036443c58
  81247ac6 88021e31b360 880036433a00 880036433a00
Call Trace:
  [] ? __d_lookup+0x9c/0x160
  [] ___slab_alloc+0x396/0x4a0
  [] ?
ath10k_wmi_event_service_ready_work+0x5ad/0x800 [ath10k_core]
  [] ? alloc_kmem_pages+0x9/0x10
  [] ? kmalloc_order+0x13/0x40
  [] ?
ath10k_wmi_event_service_ready_work+0x5ad/0x800 [ath10k_core]

-- Yin

Re: [PATCH] [ath10k] go back to using dma_alloc_coherent() for firmware scratch memory.

[Kalle Valo]

Su Kang Yin  writes:

> Without this patch QCA9888 is not working. Also I have to update
> board2.bin from Kalle's git repo.

More details would be good to know, as the behaviour seems to vary quite
a lot. What platform are you using, x86, some ARM board or what? And
what kernel exactly?

-- 
Kalle Valo