Re: [PATCH] crypto, qat, use generic numa functions
On 10/08/2014 11:50 AM, Tadeusz Struk wrote: Hi Prarit, On 10/07/2014 05:12 PM, Prarit Bhargava wrote: The method in which the qat code determines the numa node for memory allocations is a bit clunky. On 2 socket, single node systems it is possible that adf_get_dev_node_id() returns node 1, even though node 1 doesn't exist. This code transitions the qat code to the generic numa functions. Changing adf_get_dev_node_id() to a simple call to dev_get_node() results in a change to the adf_accel_dev struct as well. The problem with that is we don't want to use any valid numa node, but the node we are connected to or we don't want to use the accelerator at all. Otherwise, when the first valid numa node happens to be the remote node the dma transactions we be slow and instead of accelerating we will slow things down. A patch that enforces this is on it's way. Yeah, I was actually wondering if dev_get_node() returns NO_NODE, then we should just default to 0? I'll wait for your patch ... P. -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] crypto, qat, use generic numa functions
Hi Prarit, On 10/07/2014 05:12 PM, Prarit Bhargava wrote: The method in which the qat code determines the numa node for memory allocations is a bit clunky. On 2 socket, single node systems it is possible that adf_get_dev_node_id() returns node 1, even though node 1 doesn't exist. This code transitions the qat code to the generic numa functions. Changing adf_get_dev_node_id() to a simple call to dev_get_node() results in a change to the adf_accel_dev struct as well. The problem with that is we don't want to use any valid numa node, but the node we are connected to or we don't want to use the accelerator at all. Otherwise, when the first valid numa node happens to be the remote node the dma transactions we be slow and instead of accelerating we will slow things down. A patch that enforces this is on it's way. -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] crypto, qat, use generic numa functions
While testing, the following panic was seen: IP: [8115b8d7] __alloc_pages_nodemask+0x97/0x420 PGD 0 Oops: [#1] SMP Modules linked in: aesni_intel ptp lrw qat_dh895xcc(+) intel_qat pps_core i2c_algo_bit authenc gf128mul iTCO_wdt ioatdma glue_helper sb_edac i2c_i801 ablk_helper serio_raw iTCO_vendor_support pcspkr edac_core shpchp i2c_core cryptd dca lpc_ich mfd_core wmi xfs libcrc32c sd_mod crc_t10dif crct10dif_common ahci libahci libata dm_mirror dm_region_hash dm_log dm_mod CPU: 0 PID: 1235 Comm: systemd-udevd Not tainted 3.10.0-165.el7.x86_64 #1 Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS CCFRCLC0.019.1308201516 08/20/2013 task: 88006d068000 ti: 88006ca0c000 task.ti: 88006ca0c000 RIP: 0010:[8115b8d7] [8115b8d7] __alloc_pages_nodemask+0x97/0x420 RSP: 0018:88006ca0f928 EFLAGS: 00010246 RAX: 2000 RBX: RCX: 88006ca0ffd8 RDX: RSI: 0002 RDI: 002052d0 RBP: 88006ca0f9c8 R08: 0008 R09: 0002 R10: 0068 R11: ffc4 R12: 002052d0 R13: R14: 0002 R15: FS: 7f999a6f9880() GS:880076a0() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 2008 CR3: 6c916000 CR4: 001407f0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Stack: 88007ac07700 88006ca0f940 811a43d9 88006ca0fa00 811a4a0a 88007ac00e30 88007ac00e10 880076a17 8802 2000 000180d0 Call Trace: [811a43d9] ? discard_slab+0x39/0x50 [811a4a0a] ? deactivate_slab+0x35a/0x3c0 [811a3521] new_slab+0x91/0x300 [815ee9ed] __slab_alloc+0x2bb/0x482 [8101b923] ? native_sched_clock+0x13/0x80 [8101b999] ? sched_clock+0x9/0x10 [a01b8177] ? adf_probe+0xb7/0x5a0 [qat_dh895xcc] [812cce6f] ? idr_get_empty_slot+0x16f/0x3c0 [812cce6f] ? idr_get_empty_slot+0x16f/0x3c0 [811a690b] kmem_cache_alloc_node_trace+0x9b/0x220 [a01b8177] adf_probe+0xb7/0x5a0 [qat_dh895xcc] [81237bd2] ? sysfs_addrm_finish+0x42/0xe0 [812379b1] ? __sysfs_add_one+0x61/0x100 [812fee25] local_pci_probe+0x45/0xa0 [81300295] ? pci_match_device+0xc5/0xd0 [813003d9] pci_device_probe+0xf9/0x150 [813caee7] driver_probe_device+0x87/0x390 [813cb2c3] __driver_attach+0x93/0xa0 [813cb230] ? __device_attach+0x40/0x40 [813c8c73] bus_for_each_dev+0x73/0xc0 [813ca93e] driver_attach+0x1e/0x20 [813ca490] bus_add_driver+0x200/0x2d0 [813cb944] driver_register+0x64/0xf0 [812ffe95] __pci_register_driver+0xa5/0xc0 [a01be000] ? 0xa01bdfff [a01be03a] adfdrv_init+0x3a/0x1000 [qat_dh895xcc] [810020b8] do_one_initcall+0xb8/0x230 [810da32a] load_module+0x131a/0x1b20 [812ee3e0] ? ddebug_proc_write+0xf0/0xf0 [810d68c3] ? copy_module_from_fd.isra.43+0x53/0x150 [810dace6] SyS_finit_module+0xa6/0xd0 [81601a69] system_call_fastpath+0x16/0x1b Code: c1 eb 02 c1 e8 13 83 e3 02 83 e0 01 09 c3 44 23 25 cf 22 8a 00 48 c7 45 c0 00 00 00 00 41 f6 c4 10 0f 85 55 02 00 00 48 8b 45 b0 48 83 78 08 00 0f 84 a3 01 00 00 0f 1f 44 00 00 48 8b 45 b0 44 The method in which the qat code determines the numa node for memory allocations is a bit clunky. On 2 socket, single node systems it is possible that adf_get_dev_node_id() returns node 1, even though node 1 doesn't exist. This code transitions the qat code to the generic numa functions. Changing adf_get_dev_node_id() to a simple call to dev_get_node() results in a change to the adf_accel_dev struct as well. In addition to that change, qat_crypto_get_instance_node() must check for any node as a valid numa_node value. Cc: Tadeusz Struk tadeusz.st...@intel.com Cc: Herbert Xu herb...@gondor.apana.org.au Cc: David S. Miller da...@davemloft.net Cc: Bruce Allan bruce.w.al...@intel.com Cc: Prarit Bhargava pra...@redhat.com Cc: John Griffin john.grif...@intel.com Cc: qat-li...@intel.com Cc: linux-crypto@vger.kernel.org Signed-off-by: Prarit Bhargava pra...@redhat.com --- drivers/crypto/qat/qat_common/adf_accel_devices.h |2 +- drivers/crypto/qat/qat_common/qat_algs.c |7 +-- drivers/crypto/qat/qat_common/qat_crypto.c|4 +++- drivers/crypto/qat/qat_dh895xcc/adf_drv.c | 19 ++- 4 files changed, 7 insertions(+), 25 deletions(-) diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h index 9282381..025f52f 100644 --- a/drivers/crypto/qat/qat_common/adf_accel_devices.h +++
