On 08/02/2017 11:02 AM, Tariq Toukan wrote:
On 07/02/2017 5:50 PM, Tariq Toukan wrote:
Hi Eric,
Thanks for your series.
On 07/02/2017 5:02 AM, Eric Dumazet wrote:
As mentioned half a year ago, we better switch mlx4 driver to order-0
allocations and page recycling.
This reduces vulnerability surface thanks to better skb->truesize
tracking
and provides better performance in most cases.
The series makes significant change in the RX data-path, that
requires deeper checks, in addition to code review.
We applied your series and started running both our functional and
performance regression.
We will have results by tomorrow morning, and will analyze them
during the day. I'll update about that.
We hit a kernel panic when running traffic after configuring a large
MTU (9000).
I will take deeper look into this soon and will keep you updated.
Doesn't happen before applying patch 9/9:
mlx4: add page recycling in receive path
[56136.982183] BUG: unable to handle kernel paging request at
000000022f9e7020
[56136.990426] IP: mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en]
[56136.995303] PGD 220b7c067
[56136.995304] PUD 0
[56136.997941]
[56137.001807] Oops: 0000 [#1] SMP
[56137.004540] Modules linked in: netconsole mlx4_ib mlx4_en(E)
mlx4_core(E) nfsv3 nfs fscache rpcrdma ib_isert iscsi_target_mod
ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp
scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm
dm_mirror dm_region_hash dm_log ib_cm dm_mod iw_cm ppdev parport_pc
i2c_piix4 sg virtio_balloon parport pcspkr acpi_cpufreq nfsd
auth_rpcgss nfs_acl lockd grace sunrpc ip_tables mlx5_ib sd_mod
ata_generic pata_acpi ib_core mlx5_core cirrus drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy
libata ptp e1000 crc32c_intel virtio_pci pps_core serio_raw
virtio_ring i2c_core virtio [last unloaded: netconsole]
[56137.046028] CPU: 1 PID: 16 Comm: ksoftirqd/1 Tainted: G
E 4.10.0-rc6+ #26
[56137.051501] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[56137.055817] task: ffff880236245200 task.stack: ffffc90000d04000
[56137.060154] RIP: 0010:mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en]
[56137.064712] RSP: 0018:ffffc90000d07c90 EFLAGS: 00010282
[56137.068646] RAX: 0000000000000003 RBX: 000000022f9e7000 RCX:
ffff880234988880
[56137.073588] RDX: ffff8802349888e0 RSI: 0000000000000000 RDI:
ffff880235dad0a0
[56137.078563] RBP: ffffc90000d07ce0 R08: 0000000000000000 R09:
ffff8802225a08c0
[56137.083370] R10: ffff8802335c7800 R11: 0000000000000000 R12:
ffffc90001da1048
[56137.088123] R13: 0000000000000b36 R14: ffff8802225af040 R15:
0000000000000b36
[56137.092837] FS: 0000000000000000(0000) GS:ffff88023fc40000(0000)
knlGS:0000000000000000
[56137.098495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[56137.102626] CR2: 000000022f9e7020 CR3: 000000023441f000 CR4:
00000000000006e0
[56137.107581] Call Trace:
[56137.109955] mlx4_en_process_rx_cq+0x35c/0xda0 [mlx4_en]
[56137.113894] ? mlx4_en_free_tx_desc+0x14e/0x350 [mlx4_en]
[56137.117992] ? load_balance+0x1ac/0x900
[56137.121285] mlx4_en_poll_rx_cq+0x30/0xa0 [mlx4_en]
[56137.125023] net_rx_action+0x23d/0x3a0
[56137.128146] __do_softirq+0xd1/0x2a2
[56137.131178] run_ksoftirqd+0x29/0x50
[56137.134180] smpboot_thread_fn+0x110/0x160
[56137.137530] kthread+0x101/0x140
[56137.140330] ? sort_range+0x30/0x30
[56137.143255] ? kthread_park+0x90/0x90
[56137.146304] ? __kthread_parkme+0x50/0x70
[56137.149466] ret_from_fork+0x2c/0x40
[56137.152426] Code: c0 8b 45 cc 41 8b 8a cc 00 00 00 48 63 d0 49 03
8a d0 00 00 00 48 83 c2 03 48 c1 e2 04 48 01 ca 48 89 1a 44 89 5a 08
44 89 7a 0c <48> 8b 53 20 f6 c2 01 0f 85 76 01 00 00 48 89 da 48 83 7a
10 ff
[56137.164855] RIP: mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en]
RSP: ffffc90000d07c90
[56137.170211] CR2: 000000022f9e7020
[56137.175430] ---[ end trace 6a259f16967a0cff ]---
Worth noting this patch series deletes more than 100 lines of code ;)
Eric Dumazet (9):
mlx4: use __skb_fill_page_desc()
mlx4: dma_dir is a mlx4_en_priv attribute
mlx4: remove order field from mlx4_en_frag_info
mlx4: get rid of frag_prefix_size
mlx4: rx_headroom is a per port attribute
mlx4: reduce rx ring page_cache size
mlx4: removal of frag_sizes[]
mlx4: use order-0 pages for RX
mlx4: add page recycling in receive path
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 350
+++++++++------------------
drivers/net/ethernet/mellanox/mlx4/en_tx.c | 4 +-
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 28 +--
3 files changed, 129 insertions(+), 253 deletions(-)
Thanks,
Tariq
