Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
On 12/1/2017 12:34 AM, Johannes Thumshirn wrote: James Smartwrites: On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes Thumshirn Cc: James Smart This is unnecessary and can create weird configurations. It assumes connections are manually created. a) connections can (and will) be maually created and for this the users have to know the topology or connection establishment will fail b) there is no need that the connections are manually created. Sagi did post an RFC systemd service which calls nvme connect-all and this is what should be done regardless if we're running on FC-NVME or NVMe over RDMA any new transport that may come in the future. What I want is a consistent user experience within NVMe, as I am the one that has to answer a documentation team's inquiries on how to configure NVMf, support QA in testing and fix end user bugs. The least thing I want ot do is tell them "well if you use rdma you have to use the nvme-connect.service, if you use FC you have to have some magic udev rules and auto-connect scripts, if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada". I don't really like that we have to manully connect either but this behaviour was first in NVMe so we should either stick to that or convert RDMA over to using some sort of udev magic as well (which wont work as far as I know, and it definitively won't work with TCP if and when it's there). Sagi's RFC is certainly fine to use with FC-NVME as well. But the mechanism does require the admin to have apriori knowledge and populate the discovery.conf file with the discovery controller information. It also requires updates on any dynamic change. All of that is unnecessary with the FC-NVME auto-connect scripts. Note: there's nothing wrong with using both mechanisms simultaneously with FC-NVME. I certainly understand the sentiment of not doing something N ways. But, in this case, the functionality is so useful and valuable, that constraining the solution to the least common denominator is a bad idea. You are also ignoring the user-base coming from SCSI on FC (which is much larger than nvmf on RDMA) that have completely different expectations on how the system finds targets and connects to storage. They will be confused and have lots of questions about why SCSI is so automatic yet (even on the same machines/adapters/fabric) they now must have all kinds of upfront knowledge and make manual changes for NVME. I get bombarded on this and I know you will too. Although it's not nice to state transport-specific methods, I believe the message is still rather short and simple. We seem to have gotten away from the desired patch into a completely different area. In truth, adding the patch isn't a big deal, but to me it doesn't provide any value over the existing scripts. You could say it adds yet another one off in FC-specific behavior that could complicate the documentation. -- james
Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
On 12/1/2017 12:34 AM, Johannes Thumshirn wrote: James Smart writes: On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes Thumshirn Cc: James Smart This is unnecessary and can create weird configurations. It assumes connections are manually created. a) connections can (and will) be maually created and for this the users have to know the topology or connection establishment will fail b) there is no need that the connections are manually created. Sagi did post an RFC systemd service which calls nvme connect-all and this is what should be done regardless if we're running on FC-NVME or NVMe over RDMA any new transport that may come in the future. What I want is a consistent user experience within NVMe, as I am the one that has to answer a documentation team's inquiries on how to configure NVMf, support QA in testing and fix end user bugs. The least thing I want ot do is tell them "well if you use rdma you have to use the nvme-connect.service, if you use FC you have to have some magic udev rules and auto-connect scripts, if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada". I don't really like that we have to manully connect either but this behaviour was first in NVMe so we should either stick to that or convert RDMA over to using some sort of udev magic as well (which wont work as far as I know, and it definitively won't work with TCP if and when it's there). Sagi's RFC is certainly fine to use with FC-NVME as well. But the mechanism does require the admin to have apriori knowledge and populate the discovery.conf file with the discovery controller information. It also requires updates on any dynamic change. All of that is unnecessary with the FC-NVME auto-connect scripts. Note: there's nothing wrong with using both mechanisms simultaneously with FC-NVME. I certainly understand the sentiment of not doing something N ways. But, in this case, the functionality is so useful and valuable, that constraining the solution to the least common denominator is a bad idea. You are also ignoring the user-base coming from SCSI on FC (which is much larger than nvmf on RDMA) that have completely different expectations on how the system finds targets and connects to storage. They will be confused and have lots of questions about why SCSI is so automatic yet (even on the same machines/adapters/fabric) they now must have all kinds of upfront knowledge and make manual changes for NVME. I get bombarded on this and I know you will too. Although it's not nice to state transport-specific methods, I believe the message is still rather short and simple. We seem to have gotten away from the desired patch into a completely different area. In truth, adding the patch isn't a big deal, but to me it doesn't provide any value over the existing scripts. You could say it adds yet another one off in FC-specific behavior that could complicate the documentation. -- james
Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
James Smartwrites: > On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: >> One major usability difference between NVMf RDMA and FC is resolving >> the default host transport address in RDMA. This is perfectly doable >> in FC as well, as we already have all possible lport <-> rport >> combinations pre-populated so we can pick the first lport that has a >> connection to our desired rport per default or optionally use the user >> supplied lport if we have one. >> >> Signed-off-by: Johannes Thumshirn >> Cc: James Smart > > This is unnecessary and can create weird configurations. It assumes > connections are manually created. a) connections can (and will) be maually created and for this the users have to know the topology or connection establishment will fail b) there is no need that the connections are manually created. Sagi did post an RFC systemd service which calls nvme connect-all and this is what should be done regardless if we're running on FC-NVME or NVMe over RDMA any new transport that may come in the future. What I want is a consistent user experience within NVMe, as I am the one that has to answer a documentation team's inquiries on how to configure NVMf, support QA in testing and fix end user bugs. The least thing I want ot do is tell them "well if you use rdma you have to use the nvme-connect.service, if you use FC you have to have some magic udev rules and auto-connect scripts, if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada". I don't really like that we have to manully connect either but this behaviour was first in NVMe so we should either stick to that or convert RDMA over to using some sort of udev magic as well (which wont work as far as I know, and it definitively won't work with TCP if and when it's there). > The weirdness is: a) an admin has to > know there are multiple paths in order to connect them and be > intelligent on how to get the complex name strings and try to know > what connections are already in existence; b) if a users has a > connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the > controller is terminated, they must manually issue the connec commands > again; and c) those un-knowledgeable users will unknowingly find that > their multiple paths aren't connected and the system will gang up on > the host adapter detected on the system with connectivity. All things > unexpected and not what occurs with FC and SCSI and which will result > in system support calls. > > If the system uses the FC auto-connect scripts things will be properly > connected across all paths connected to the subsystem - automatically, > including resume after an extended connectivity loss - and the system > will behave just like FC does with SCSI. > > I see no reason to add this patch. Please move away from manual > configuration. OK so the please help with moving NVMe away from manual configuration. I'm fine with either way I just don't want to have N different ways because it is a documentation and usability nightmare. Johannes -- Johannes Thumshirn Storage jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
James Smart writes: > On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: >> One major usability difference between NVMf RDMA and FC is resolving >> the default host transport address in RDMA. This is perfectly doable >> in FC as well, as we already have all possible lport <-> rport >> combinations pre-populated so we can pick the first lport that has a >> connection to our desired rport per default or optionally use the user >> supplied lport if we have one. >> >> Signed-off-by: Johannes Thumshirn >> Cc: James Smart > > This is unnecessary and can create weird configurations. It assumes > connections are manually created. a) connections can (and will) be maually created and for this the users have to know the topology or connection establishment will fail b) there is no need that the connections are manually created. Sagi did post an RFC systemd service which calls nvme connect-all and this is what should be done regardless if we're running on FC-NVME or NVMe over RDMA any new transport that may come in the future. What I want is a consistent user experience within NVMe, as I am the one that has to answer a documentation team's inquiries on how to configure NVMf, support QA in testing and fix end user bugs. The least thing I want ot do is tell them "well if you use rdma you have to use the nvme-connect.service, if you use FC you have to have some magic udev rules and auto-connect scripts, if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada". I don't really like that we have to manully connect either but this behaviour was first in NVMe so we should either stick to that or convert RDMA over to using some sort of udev magic as well (which wont work as far as I know, and it definitively won't work with TCP if and when it's there). > The weirdness is: a) an admin has to > know there are multiple paths in order to connect them and be > intelligent on how to get the complex name strings and try to know > what connections are already in existence; b) if a users has a > connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the > controller is terminated, they must manually issue the connec commands > again; and c) those un-knowledgeable users will unknowingly find that > their multiple paths aren't connected and the system will gang up on > the host adapter detected on the system with connectivity. All things > unexpected and not what occurs with FC and SCSI and which will result > in system support calls. > > If the system uses the FC auto-connect scripts things will be properly > connected across all paths connected to the subsystem - automatically, > including resume after an extended connectivity loss - and the system > will behave just like FC does with SCSI. > > I see no reason to add this patch. Please move away from manual > configuration. OK so the please help with moving NVMe away from manual configuration. I'm fine with either way I just don't want to have N different ways because it is a documentation and usability nightmare. Johannes -- Johannes Thumshirn Storage jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes ThumshirnCc: James Smart This is unnecessary and can create weird configurations. It assumes connections are manually created. The weirdness is: a) an admin has to know there are multiple paths in order to connect them and be intelligent on how to get the complex name strings and try to know what connections are already in existence; b) if a users has a connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the controller is terminated, they must manually issue the connec commands again; and c) those un-knowledgeable users will unknowingly find that their multiple paths aren't connected and the system will gang up on the host adapter detected on the system with connectivity. All things unexpected and not what occurs with FC and SCSI and which will result in system support calls. If the system uses the FC auto-connect scripts things will be properly connected across all paths connected to the subsystem - automatically, including resume after an extended connectivity loss - and the system will behave just like FC does with SCSI. I see no reason to add this patch. Please move away from manual configuration. -- james
Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr
On 11/30/2017 7:12 AM, Johannes Thumshirn wrote: One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes Thumshirn Cc: James Smart This is unnecessary and can create weird configurations. It assumes connections are manually created. The weirdness is: a) an admin has to know there are multiple paths in order to connect them and be intelligent on how to get the complex name strings and try to know what connections are already in existence; b) if a users has a connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the controller is terminated, they must manually issue the connec commands again; and c) those un-knowledgeable users will unknowingly find that their multiple paths aren't connected and the system will gang up on the host adapter detected on the system with connectivity. All things unexpected and not what occurs with FC and SCSI and which will result in system support calls. If the system uses the FC auto-connect scripts things will be properly connected across all paths connected to the subsystem - automatically, including resume after an extended connectivity loss - and the system will behave just like FC does with SCSI. I see no reason to add this patch. Please move away from manual configuration. -- james
[PATCH v2] nvme-fc: don't require user to enter host_traddr
One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes ThumshirnCc: James Smart -- Changes to v1: - Both laddr.pn and laddr.nn must be present _if_ we take the laddr into account (Hannes) --- drivers/nvme/host/fc.c | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 7ab0be55c7d0..063b9c3438ce 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -3328,15 +3328,19 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) if (ret || !raddr.nn || !raddr.pn) return ERR_PTR(-EINVAL); - ret = nvme_fc_parse_traddr(, opts->host_traddr, NVMF_TRADDR_SIZE); - if (ret || !laddr.nn || !laddr.pn) - return ERR_PTR(-EINVAL); + if (opts->mask & NVMF_OPT_HOST_TRADDR) { + ret = nvme_fc_parse_traddr(, opts->host_traddr, + NVMF_TRADDR_SIZE); + if (ret || !laddr.nn || !laddr.pn) + return ERR_PTR(-EINVAL); + } /* find the host and remote ports to connect together */ spin_lock_irqsave(_fc_lock, flags); list_for_each_entry(lport, _fc_lport_list, port_list) { - if (lport->localport.node_name != laddr.nn || - lport->localport.port_name != laddr.pn) + if (!(laddr.nn && laddr.pn) && + (lport->localport.node_name != laddr.nn || +lport->localport.port_name != laddr.pn)) continue; list_for_each_entry(rport, >endp_list, endp_list) { @@ -3364,8 +3368,9 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) static struct nvmf_transport_ops nvme_fc_transport = { .name = "fc", - .required_opts = NVMF_OPT_TRADDR | NVMF_OPT_HOST_TRADDR, - .allowed_opts = NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO, + .required_opts = NVMF_OPT_TRADDR, + .allowed_opts = NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO + | NVMF_OPT_HOST_TRADDR, .create_ctrl= nvme_fc_create_ctrl, }; -- 2.13.6
[PATCH v2] nvme-fc: don't require user to enter host_traddr
One major usability difference between NVMf RDMA and FC is resolving the default host transport address in RDMA. This is perfectly doable in FC as well, as we already have all possible lport <-> rport combinations pre-populated so we can pick the first lport that has a connection to our desired rport per default or optionally use the user supplied lport if we have one. Signed-off-by: Johannes Thumshirn Cc: James Smart -- Changes to v1: - Both laddr.pn and laddr.nn must be present _if_ we take the laddr into account (Hannes) --- drivers/nvme/host/fc.c | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 7ab0be55c7d0..063b9c3438ce 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -3328,15 +3328,19 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) if (ret || !raddr.nn || !raddr.pn) return ERR_PTR(-EINVAL); - ret = nvme_fc_parse_traddr(, opts->host_traddr, NVMF_TRADDR_SIZE); - if (ret || !laddr.nn || !laddr.pn) - return ERR_PTR(-EINVAL); + if (opts->mask & NVMF_OPT_HOST_TRADDR) { + ret = nvme_fc_parse_traddr(, opts->host_traddr, + NVMF_TRADDR_SIZE); + if (ret || !laddr.nn || !laddr.pn) + return ERR_PTR(-EINVAL); + } /* find the host and remote ports to connect together */ spin_lock_irqsave(_fc_lock, flags); list_for_each_entry(lport, _fc_lport_list, port_list) { - if (lport->localport.node_name != laddr.nn || - lport->localport.port_name != laddr.pn) + if (!(laddr.nn && laddr.pn) && + (lport->localport.node_name != laddr.nn || +lport->localport.port_name != laddr.pn)) continue; list_for_each_entry(rport, >endp_list, endp_list) { @@ -3364,8 +3368,9 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts) static struct nvmf_transport_ops nvme_fc_transport = { .name = "fc", - .required_opts = NVMF_OPT_TRADDR | NVMF_OPT_HOST_TRADDR, - .allowed_opts = NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO, + .required_opts = NVMF_OPT_TRADDR, + .allowed_opts = NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO + | NVMF_OPT_HOST_TRADDR, .create_ctrl= nvme_fc_create_ctrl, }; -- 2.13.6