Re: [PATCH V2] blk-mq: re-build queue map in case of kdump kernel

2018-12-07 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 1/2] blk-mq: Export iterating all tagged requests

2018-12-04 Thread Sagi Grimberg
Yes, I'm very much in favour of this, too. We always have this IMO slightly weird notion of stopping the queue, set some error flags in the driver, then _restarting_ the queue, just so that the driver then sees the error flag and terminates the requests. Which I always found quite

Re: [PATCH 1/2] blk-mq: Export iterating all tagged requests

2018-12-04 Thread Sagi Grimberg
Yes, I'm very much in favour of this, too. We always have this IMO slightly weird notion of stopping the queue, set some error flags in the driver, then _restarting_ the queue, just so that the driver then sees the error flag and terminates the requests. Which I always found quite

Re: [PATCH 10/13] nvme-mpath: remove I/O polling support

2018-12-04 Thread Sagi Grimberg
If it really becomes an issue we should rework the nvme code to also skip the multipath code for any private namespace, even if that could mean some trouble when rescanning. This requires some explanation? skip the multipath code how? We currently always go through the multipath node as

Re: [PATCH 05/13] nvme-pci: consolidate code for polling non-dedicated queues

2018-12-04 Thread Sagi Grimberg
+static int nvme_poll_irqdisable(struct nvme_queue *nvmeq, unsigned int tag) Do we still need to carry the tag around? Yes, the timeout handler polls for a specific tag. Does it have to? the documentation suggests that we missed an interrupt, so it is probably waiting on the completion

Re: [PATCH 02/13] nvme-pci: use atomic bitops to mark a queue enabled

2018-12-04 Thread Sagi Grimberg
@@ -2173,6 +2157,8 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) if (nr_io_queues == 0) return 0; + + clear_bit(NVMEQ_ENABLED, >flags); This is a change of behavior, looks correct though as we can fail nvme_setup_irqs after we freed the

Re: [PATCH 01/13] block: move queues types to the block layer

2018-12-04 Thread Sagi Grimberg
Nit, there seems to be an extra newline that can be omitted here before the else if statement (if I'm reading this correctly)... Empty lines can always be ommited, but in this case I actually like it as it seems to help readability.. If you think its useful I'm fine with it as is...

Re: [PATCH 7/7] blk-mq: use plug for devices that implement ->commits_rqs()

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 6/7] blk-mq: use bd->last == true for list inserts

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 4/7] virtio_blk: implement mq_ops->commit_rqs() hook

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 5/7] ataflop: implement mq_ops->commit_rqs() hook

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 2/7] blk-mq: add mq_ops->commit_rqs()

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 1/7] block: improve logic around when to sort a plug list

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 1/2] blk-mq: Export iterating all tagged requests

2018-12-03 Thread Sagi Grimberg
A driver may wish to iterate every tagged request, not just ones that satisfy blk_mq_request_started(). The intended use is so a driver may terminate entered requests on quiesced queues. How about we just move the started check into the handler passed in for those that care about it? Much

Re: [PATCH 13/13] block: enable polling by default if a poll map is initalized

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 12/13] block: only allow polling if a poll queue_map exists

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 11/13] block: remove ->poll_fn

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 10/13] nvme-mpath: remove I/O polling support

2018-12-03 Thread Sagi Grimberg
If it really becomes an issue we should rework the nvme code to also skip the multipath code for any private namespace, even if that could mean some trouble when rescanning. This requires some explanation? skip the multipath code how? Other than that, Reviewed-by: Sagi Grimberg

Re: [PATCH 08/13] nvme-pci: remove the CQ lock for interrupt driven queues

2018-12-03 Thread Sagi Grimberg
of disabling interrupts. With that we can stop taking the cq_lock for normal queues. Nice, Reviewed-by: Sagi Grimberg

Re: [PATCH 07/13] nvme-pci: don't poll from irq context when deleting queues

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 06/13] nvme-pci: refactor nvme_disable_io_queues

2018-12-03 Thread Sagi Grimberg
@@ -2428,7 +2426,8 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) nvme_stop_queues(>ctrl); if (!dead && dev->ctrl.queue_count > 0) { - nvme_disable_io_queues(dev); + if (nvme_disable_io_queues(dev, nvme_admin_delete_sq)) +

Re: [PATCH 05/13] nvme-pci: consolidate code for polling non-dedicated queues

2018-12-03 Thread Sagi Grimberg
+static int nvme_poll_irqdisable(struct nvme_queue *nvmeq, unsigned int tag) Do we still need to carry the tag around? Other than that, Reviewed-by: Sagi Grimberg

Re: [PATCH 04/13] nvme-pci: only allow polling with separate poll queues

2018-12-03 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 03/13] nvme-pci: cleanup SQ allocation a bit

2018-12-03 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 02/13] nvme-pci: use atomic bitops to mark a queue enabled

2018-12-03 Thread Sagi Grimberg
@@ -2173,6 +2157,8 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) if (nr_io_queues == 0) return 0; + + clear_bit(NVMEQ_ENABLED, >flags); This is a change of behavior, looks correct though as we can fail nvme_setup_irqs after we freed the admin

Re: [PATCH 01/13] block: move queues types to the block layer

2018-12-03 Thread Sagi Grimberg
AD)) + type = HCTX_TYPE_READ; Nit, there seems to be an extra newline that can be omitted here before the else if statement (if I'm reading this correctly)... Otherwise looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 2/2] nvme: create slaves/holder entries for multipath devices

2018-11-27 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 1/2] block: move holder tracking from struct block_device to hd_struct

2018-11-27 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 2/7] block: have ->poll_fn() return number of entries polled

2018-11-27 Thread Sagi Grimberg
We currently only really support sync poll, ie poll with 1 IO in flight. This prepares us for supporting async poll. Hey Jens, So are we sure that this is fine to simply replace the poll functionality? you say that that we support poll with only 1 I/O inflight but is it entirely true?

Re: [PATCH 2/7] block: have ->poll_fn() return number of entries polled

2018-11-25 Thread Sagi Grimberg
We currently only really support sync poll, ie poll with 1 IO in flight. This prepares us for supporting async poll. Hey Jens, So are we sure that this is fine to simply replace the poll functionality? you say that that we support poll with only 1 I/O inflight but is it entirely true?

Re: [PATCH 6/7] block: make blk_poll() take a parameter on whether to spin or not

2018-11-20 Thread Sagi Grimberg
This opportunistic poll is pretty bogus now as we never set the HIPRI flag and it should probably be removed in a prep patch. We should then later try to use a scheme similar to your aio polling for the nvme target as well. I'll kill it in a pre-patch. Agreed..

Re: [GIT PULL] nvme fixes for 4.20

2018-11-09 Thread Sagi Grimberg
That is because your for-linus was still 4.20-rc based when I created it. It's never been 4.20-rc based. After the initial merge, it got based on a random commit in the merge window: commit 3acbd2de6bc3af215c6ed7732dfc097d1e238503 Merge: d49f8a52b15b de7d83da84bd Author: Linus Torvalds

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-30 Thread Sagi Grimberg
Subject: Re: [PATCH v2] block: fix rdma queue mapping Sagi - From what I can tell, i40iw is also exposed to this same issue if the IRQ affinity is configured by user. managed affinity does not allow setting smp_affinity from userspace. OK. But our device IRQs are not set-up as to be

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-23 Thread Sagi Grimberg
Sagi - From what I can tell, i40iw is also exposed to this same issue if the IRQ affinity is configured by user. managed affinity does not allow setting smp_affinity from userspace.

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-23 Thread Sagi Grimberg
Christoph, Sagi: it seems you think /proc/irq/$IRP/smp_affinity shouldn't be allowed if drivers support managed affinity. Is that correct? Not just shouldn't, but simply can't. But as it stands, things are just plain borked if an rdma driver supports ib_get_vector_affinity() yet the admin

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-23 Thread Sagi Grimberg
Christoph, Sagi: it seems you think /proc/irq/$IRP/smp_affinity shouldn't be allowed if drivers support managed affinity. Is that correct? Not just shouldn't, but simply can't. But as it stands, things are just plain borked if an rdma driver supports ib_get_vector_affinity() yet the admin

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-15 Thread Sagi Grimberg
Can we please make forward progress on this? Christoph, Sagi:  it seems you think /proc/irq/$IRP/smp_affinity shouldn't be allowed if drivers support managed affinity. Is that correct? Perhaps that can be codified and be a way forward?  IE: Somehow allow the admin to choose either "managed

Re: [PATCH v2] block: fix rdma queue mapping

2018-10-03 Thread Sagi Grimberg
Can we please make forward progress on this? Christoph, Sagi: it seems you think /proc/irq/$IRP/smp_affinity shouldn't be allowed if drivers support managed affinity. Is that correct? Perhaps that can be codified and be a way forward? IE: Somehow allow the admin to choose either "managed

Re: [PATCH] block: fix rdma queue mapping

2018-08-24 Thread Sagi Grimberg
Hi Sagi, did you have a chance to look on Israel's and mine fixes that we attached to the first thread ? there are few issues with this approach. For example in case you don't have a "free" cpu in the mask for Qi and you take cpu from Qi+j mask. Also in case we have a non-symetrical

Re: [PATCH v2] block: fix rdma queue mapping

2018-08-24 Thread Sagi Grimberg
nvme-rdma attempts to map queues based on irq vector affinity. However, for some devices, completion vector irq affinity is configurable by the user which can break the existing assumption that irq vectors are optimally arranged over the host cpu cores. IFF affinity is configurable we should

Re: [PATCH v2] block: fix rdma queue mapping

2018-08-24 Thread Sagi Grimberg
I guess this way still can't fix the request allocation crash issue triggered by using blk_mq_alloc_request_hctx(), in which one hw queue may not be mapped from any online CPU. Not really. I guess we will need to simply skip queues that are mapped to an offline cpu. Maybe this patch isn't

[PATCH v2] block: fix rdma queue mapping

2018-08-20 Thread Sagi Grimberg
, we map it to any unmapped cpu we can find. Then, map the remaining cpus in the possible cpumap naively. Tested-by: Steve Wise Signed-off-by: Sagi Grimberg --- Changes from v1: - fixed double semicolon typo block/blk-mq-cpumap.c | 39 +++- block/blk-mq-rdma.c| 80

[PATCH] block: fix rdma queue mapping

2018-08-17 Thread Sagi Grimberg
, we map it to any unmapped cpu we can find. Then, map the remaining cpus in the possible cpumap naively. Tested-by: Steve Wise Signed-off-by: Sagi Grimberg --- See thread that started here: https://marc.info/?l=linux-rdma=153172982318299=2 block/blk-mq-cpumap.c | 39

Re: [PATCH v2 2/2] nvme: Fix race conditions related to creation of /dev/nvme0n

2018-08-17 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH] block: Fix cloning of requests with a special payload

2018-06-28 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH] block: pass failfast and driver-specific flags to flush requests

2018-06-07 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCHv2] block: always set partition number to '0' in blk_partition_remap()

2018-06-07 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 12/12] nvme: limit warnings from nvme_identify_ns

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 11/12] nvme: use the changed namespaces list log to clear ns data changed AENs

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 10/12] nvme: mark nvme_queue_scan static

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 08/12] nvmet: mask pending AERs

2018-05-30 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg

Re: [PATCH 09/12] nvme: submit AEN event configuration on startup

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 07/12] nvmet: Add AEN configuration support

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 06/12] nvmet: implement the changed namespaces log

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 05/12] nvmet: split log page implementation

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 04/12] nvmet: add a new nvmet_zero_sgl helper

2018-05-30 Thread Sagi Grimberg
+u16 nvmet_zero_sgl(struct nvmet_req *req, off_t off, size_t len) Would personally prefer nvmet_clear_sgl, but not hard headed on it, Reviewed-by: Sagi Grimberg

Re: [PATCH 03/12] nvme.h: add AER configuration symbols

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 02/12] nvme.h: add the changed namespace list log

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH 01/12] nvme.h: untangle AEN notice definitions

2018-05-30 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg

Re: [PATCH v5] blk-mq: Avoid that a completion can be ignored for BLK_EH_RESET_TIMER

2018-04-11 Thread Sagi Grimberg
static void __blk_mq_requeue_request(struct request *rq) { struct request_queue *q = rq->q; + enum mq_rq_state old_state = blk_mq_rq_state(rq); blk_mq_put_driver_tag(rq); trace_block_rq_requeue(q, rq); wbt_requeue(q->rq_wb, >issue_stat); - if

Re: [PATCH v4] blk-mq: Fix race conditions in request timeout handling

2018-04-11 Thread Sagi Grimberg
Ah, yeah, I was moving it out of add_timer but forgot to actully add it to the issue path. Fixed patch below. BTW, no matter what we do w/ the request handover between normal and timeout paths, we'd need something similar. Otherwise, while we can reduce the window, we can't get rid of it.

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-11 Thread Sagi Grimberg
diff --git a/block/blk-mq.c b/block/blk-mq.c index 75336848f7a7..81ced3096433 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -444,6 +444,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q, return ERR_PTR(-EXDEV); } cpu =

Re: [PATCH] blk-mq: Revert "blk-mq: reimplement blk_mq_hw_queue_mapped"

2018-04-11 Thread Sagi Grimberg
Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH v5] blk-mq: Avoid that a completion can be ignored for BLK_EH_RESET_TIMER

2018-04-11 Thread Sagi Grimberg
ITE_ONCE(rq->gstate, new_val); + return cmpxchg(>__deadline, old_val, new_val) == old_val; } Can you explain why this takes the old_state of the request? Otherwise this looks good to me, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH v5] blk-mq: Avoid that a completion can be ignored for BLK_EH_RESET_TIMER

2018-04-11 Thread Sagi Grimberg
The blk-mq timeout handling code ignores completions that occur after blk_mq_check_expired() has been called and before blk_mq_rq_timed_out() has reset rq->aborted_gstate. If a block driver timeout handler always returns BLK_EH_RESET_TIMER then the result will be that the request never

Re: [PATCH 8/8] blk-mq: remove code for dealing with remapping queue

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 7/8] blk-mq: reimplement blk_mq_hw_queue_mapped

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 6/8] blk-mq: don't check queue mapped in __blk_mq_delay_run_hw_queue()

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 5/8] blk-mq: remove blk_mq_delay_queue()

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 4/8] blk-mq: introduce blk_mq_hw_queue_first_cpu() to figure out first cpu

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 3/8] blk-mq: avoid to write intermediate result to hctx->next_cpu

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 2/8] blk-mq: don't keep offline CPUs mapped to hctx 0

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH 1/8] blk-mq: make sure that correct hctx->next_cpu is set

2018-04-09 Thread Sagi Grimberg
Looks good, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-09 Thread Sagi Grimberg
Hi Sagi Sorry for the late response, bellow patch works, here is the full log: Thanks for testing! Now that we isolated the issue, the question is if this fix is correct given that we are guaranteed that the connect context will run on an online cpu? another reference to the patch (we can

Re: [PATCH] blk-mq: Fix recently introduced races in the timeout handling code

2018-04-09 Thread Sagi Grimberg
If a completion occurs after blk_mq_rq_timed_out() has reset rq->aborted_gstate and the request is again in flight when the timeout expires then a request will be completed twice: a first time by the timeout handler and a second time when the regular completion occurs. Additionally, the blk-mq

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-09 Thread Sagi Grimberg
My device exposes nr_hw_queues which is not higher than num_online_cpus so I want to connect all hctxs with hope that they will be used. The issue is that CPU online & offline can happen any time, and after blk-mq removes CPU hotplug handler, there is no way to remap queue when CPU topo is

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-08 Thread Sagi Grimberg
On 04/08/2018 03:57 PM, Ming Lei wrote: On Sun, Apr 08, 2018 at 02:53:03PM +0300, Sagi Grimberg wrote: Hi Sagi Still can reproduce this issue with the change: Thanks for validating Yi, Would it be possible to test the following: -- diff --git a/block/blk-mq.c b/block/blk-mq.c index

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-08 Thread Sagi Grimberg
Hi Sagi Still can reproduce this issue with the change: Thanks for validating Yi, Would it be possible to test the following: -- diff --git a/block/blk-mq.c b/block/blk-mq.c index 75336848f7a7..81ced3096433 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -444,6 +444,10 @@ struct request

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-08 Thread Sagi Grimberg
Hi Sagi Still can reproduce this issue with the change: Thanks for validating Yi, Would it be possible to test the following: -- diff --git a/block/blk-mq.c b/block/blk-mq.c index 75336848f7a7..81ced3096433 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -444,6 +444,10 @@ struct request

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-08 Thread Sagi Grimberg
Hi Sagi Still can reproduce this issue with the change: Thanks for validating Yi, Would it be possible to test the following: -- diff --git a/block/blk-mq.c b/block/blk-mq.c index 75336848f7a7..81ced3096433 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -444,6 +444,10 @@ struct request

Re: BUG at IP: blk_mq_get_request+0x23e/0x390 on 4.16.0-rc7

2018-04-04 Thread Sagi Grimberg
On 03/30/2018 12:32 PM, Yi Zhang wrote: Hello I got this kernel BUG on 4.16.0-rc7, here is the reproducer and log, let me know if you need more info, thanks. Reproducer: 1. setup target #nvmetcli restore /etc/rdma.json 2. connect target on host #nvme connect-all -t rdma -a $IP -s 4420during

Re: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB

2018-03-05 Thread Sagi Grimberg
- if (nvmeq->sq_cmds_io) - memcpy_toio(>sq_cmds_io[tail], cmd, sizeof(*cmd)); - else - memcpy(>sq_cmds[tail], cmd, sizeof(*cmd)); + memcpy(>sq_cmds[tail], cmd, sizeof(*cmd)); Hmm, how safe is replacing memcpy_toio() with regular memcpy()? On PPC

Re: [PATCH v2 10/10] nvmet: Optionally use PCI P2P memory

2018-03-01 Thread Sagi Grimberg
On 01/03/18 04:03 AM, Sagi Grimberg wrote: Can you describe what would be the plan to have it when these devices do come along? I'd say that p2p_dev needs to become a nvmet_ns reference and not from nvmet_ctrl. Then, when cmb capable devices come along, the ns can prefer to use its own cmb

Re: [PATCH v2 05/10] block: Introduce PCI P2P flags for request and request queue

2018-03-01 Thread Sagi Grimberg
Looks fine, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH v2 08/10] nvme-pci: Add support for P2P memory in requests

2018-03-01 Thread Sagi Grimberg
. This looks fine to me, Reviewed-by: Sagi Grimberg <s...@grimberg.me> Any plans adding the capability to nvme-rdma? Should be straight-forward... In theory, the use-case would be rdma backend fabric behind. Shouldn't be hard to test either...

Re: [PATCH v2 10/10] nvmet: Optionally use PCI P2P memory

2018-03-01 Thread Sagi Grimberg
We create a configfs attribute in each nvme-fabrics target port to enable p2p memory use. When enabled, the port will only then use the p2p memory if a p2p memory device can be found which is behind the same switch as the RDMA port and all the block devices in use. If the user enabled it an no

Re: [PATCH v2 09/10] nvme-pci: Add a quirk for a pseudo CMB

2018-03-01 Thread Sagi Grimberg
Looks fine, Reviewed-by: Sagi Grimberg <s...@grimberg.me>

Re: [PATCH v2 06/10] IB/core: Add optional PCI P2P flag to rdma_rw_ctx_[init|destroy]()

2018-03-01 Thread Sagi Grimberg
On 03/01/2018 01:40 AM, Logan Gunthorpe wrote: In order to use PCI P2P memory pci_p2pmem_[un]map_sg() functions must be called to map the correct DMA address. To do this, we add a flags variable and the RDMA_RW_CTX_FLAG_PCI_P2P flag. When the flag is specified use the appropriate map function.

Re: [PATCH v2 00/10] Copy Offload in NVMe Fabrics with P2P PCI Memory

2018-03-01 Thread Sagi Grimberg
Hi Everyone, Hi Logan, Here's v2 of our series to introduce P2P based copy offload to NVMe fabrics. This version has been rebased onto v4.16-rc3 which already includes Christoph's devpagemap work the previous version was based off as well as a couple of the cleanup patches that were in v1.

Re: [PATCH rfc 3/5] irq_poll: wire up irq_am

2018-02-12 Thread Sagi Grimberg
Hey Bart, +void irq_poll_init_am(struct irq_poll *iop, unsigned int nr_events, +unsigned short nr_levels, unsigned short start_level, irq_poll_am_fn *amfn) +{ + iop->amfn = amfn; + irq_am_init(>am, nr_events, nr_levels, start_level, irq_poll_am); +} This function has a

Re: [PATCH] blk: optimization for classic polling

2018-02-08 Thread Sagi Grimberg
I think it'd be simpler to have blk_poll set it back to running if need_resched is true rather than repeat this patter across all the callers: --- diff --git a/block/blk-mq.c b/block/blk-mq.c index df93102e2149..40285fe1c8ad 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -3164,6 +3164,7

Re: [PATCH rfc 0/5] generic adaptive IRQ moderation library for I/O devices

2018-02-06 Thread Sagi Grimberg
Hi Tal, I think Tal has idea/s on how the existing library can be changed to support more modes/models What I was thinking is allowing DIM algorithm to disregard data which is 0. Currently if bytes == 0 we return "SAME" immediately. We can change it to simply move to the packets check (which

Re: [PATCH rfc 0/5] generic adaptive IRQ moderation library for I/O devices

2018-02-06 Thread Sagi Grimberg
Hey Or, not any more, Andy and Tal made the mlx5 AM code a kernel library which is called DIM f4e5f0e MAINTAINERS: add entry for Dynamic Interrupt Moderation 6a8788f bnxt_en: add support for software dynamic interrupt moderation 8115b75 net/dim: use struct net_dim_sample as arg to net_dim

[PATCH rfc 1/5] irq-am: Introduce library implementing generic adaptive moderation

2018-02-05 Thread Sagi Grimberg
/decrement its current level (step) and schedules a program dispatch work. Signed-off-by: Sagi Grimberg <s...@grimberg.me> --- include/linux/irq-am.h | 116 +++ lib/Kconfig| 5 ++ lib/Makefile | 1 + lib/irq-am.c

[PATCH rfc 2/5] irq-am: add some debugfs exposure on tuning state

2018-02-05 Thread Sagi Grimberg
Useful for local debugging Signed-off-by: Sagi Grimberg <s...@grimberg.me> --- include/linux/irq-am.h | 2 + lib/irq-am.c | 109 + 2 files changed, 111 insertions(+) diff --git a/include/linux/irq-am.h b/include/linux/irq-am.h

[PATCH rfc 3/5] irq_poll: wire up irq_am

2018-02-05 Thread Sagi Grimberg
Update online stats for fired event and completions on each poll cycle. Also expose am initialization interface. The irqpoll consumer will initialize the irq-am context of the irq-poll context. Signed-off-by: Sagi Grimberg <s...@grimberg.me> --- include/linux/irq_poll.h | 9

[PATCH rfc 4/5] IB/cq: add adaptive moderation support

2018-02-05 Thread Sagi Grimberg
Currently activated via modparam, obviously we will want to find a more generic way to control this. Signed-off-by: Sagi Grimberg <s...@grimberg.me> --- drivers/infiniband/core/cq.c | 48 1 file changed, 48 insertions(+) diff --git a/d

[PATCH rfc 0/5] generic adaptive IRQ moderation library for I/O devices

2018-02-05 Thread Sagi Grimberg
is interested :) Sagi Grimberg (5): irq-am: Introduce helper library for adaptive moderation implementation irq-am: add some debugfs exposure on tuning state irq_poll: wire up irq_am and allow to initialize it IB/cq: add adaptive moderation support IB/cq: wire up adaptive moderation

[PATCH rfc 5/5] IB/cq: wire up adaptive moderation to workqueue based completion queues

2018-02-05 Thread Sagi Grimberg
Signed-off-by: Sagi Grimberg <s...@grimberg.me> --- drivers/infiniband/core/cq.c | 25 - include/rdma/ib_verbs.h | 9 +++-- 2 files changed, 27 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c

Re: [PATCH 00/24] InfiniBand Transport (IBTRS) and Network Block Device (IBNBD)

2018-02-05 Thread Sagi Grimberg
Hi Bart, My another 2 cents:) On Fri, Feb 2, 2018 at 6:05 PM, Bart Van Assche wrote: On Fri, 2018-02-02 at 15:08 +0100, Roman Pen wrote: o Simple configuration of IBNBD: - Server side is completely passive: volumes do not need to be explicitly exported.

  1   2   3   4   5   >