On Tue, 2013-06-11 at 13:26 -0500, Mike Christie wrote: > On 06/11/2013 12:43 PM, Eddie Wai wrote: > > > > On Tue, 2013-06-11 at 12:25 -0500, Mike Christie wrote: > >> On 6/10/13 10:47 PM, Vikas Chaudhary wrote: > >>> > >>> > >>> -----Original Message----- > >>> From: Eddie Wai <[email protected]> > >>> Date: Wednesday 5 June 2013 7:00 AM > >>> To: "[email protected]" <[email protected]> > >>> Cc: "[email protected]" <[email protected]>, Vikas > >>> <[email protected]>, Lalit Chandivade > >>> <[email protected]>, Ravi Anand <[email protected]>, > >>> Poornima Vonti <[email protected]>, Manish Rangankar > >>> <[email protected]>, Adheer Chandravanshi > >>> <[email protected]> > >>> Subject: RE: [RFC_V5 PATCH 1/3] scsi_transport_iscsi: Add flash node mgmt > >>> support > >>> > >>>> > >>>> On Wed, 2013-05-29 at 14:37 -0700, [email protected] wrote: > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: [email protected] [mailto:[email protected]] > >>>>>> On Behalf Of Mike Christie > >>>>>> Sent: Wednesday, May 29, 2013 1:30 PM > >>>>>> To: [email protected] > >>>>>> Cc: Eddie Wai; [email protected]; > >>>>> [email protected]; > >>>>>> [email protected]; [email protected]; > >>>>>> [email protected]; Adheer Chandravanshi > >>>>>> Subject: Re: [RFC_V5 PATCH 1/3] scsi_transport_iscsi: Add flash node > >>>>> mgmt > >>>>>> support > >>>>>> > >>>>>> On 05/29/2013 12:23 PM, Eddie Wai wrote: > >>>>>>> > >>>>>>> On Wed, 2013-05-29 at 10:26 -0500, Mike Christie wrote: > >>>>>>>> On 05/28/2013 07:35 PM, Eddie Wai wrote: > >>>>>>>>> Hello Mike, Vikas, and all, > >>>>>>>>> > >>>>>>>>> Thanks for the great work in creating the flash node mechanism! > >>>>>>>>> > >>>>>>>>> To extend the conversation we had to add support for software and > >>>>>>>>> other offload solutions that requires iscsid/iscsiadm to create > >>>>> the > >>>>>>>>> sessions, the following needs to be further discussed: > >>>>>>>>> > >>>>>>>>> 1. Flash node creation. > >>>>>>>>> > >>>>>>>>> The current solution relies on the transport driver to initiate > >>>>> the > >>>>>>>>> flash node sysfs creation upon iscsi_host allocation. This > >>>>> presents > >>>>>>>>> a fundamental problem for software iSCSI as new iscsi_host > >>>>> instance > >>>>>>>>> won't get created until there's a session initiation. > >>>>>>>>> > >>>>>>>>> Per our conversation, I think it makes sense to move the flash > >>>>> node > >>>>>>>>> initiation code altogether to a separate module like how its done > >>>>>>>>> for ibft. The new module shall cycle through each existing iSCSI > >>>>>>>>> host and query the corresponding transport to fill the flash node > >>>>>>>>> sysfs entries specific to that host. Perhaps via a new transport > >>>>> callback > >>>>>> or so. > >>>>>>>> > >>>>>>>> Agree. > >>>>>>>> > >>>>>>>>> > >>>>>>>>> Since there won't be any pre-existing host created for software > >>>>>>>>> iSCSI, this needs to be handled differently. Instead, the new > >>>>>>>>> module will also need to cycle through each network interfaces > >>>>>>>>> present and query for the flash node content separately. > >>>>>>>>> > >>>>>>>>> To accomplish this, each network interface will need to be made > >>>>>>>>> aware of flash nodes and also provide a way for the new module to > >>>>>>>>> read out the flash node content. Per our conversation, this can > >>>>> be > >>>>>>>>> done via an extension of the ethtool utility. For unsupported > >>>>>>>>> network drivers, this > >>>>>>>> > >>>>>>>> I do not remember that discussion. Has it been discussed with the > >>>>>>>> netdev people already? > >>>>>>> This has only been discussed internally so far, but we believe > >>>>> adding > >>>>>>> a new ethtool extension for this flash memory access is one logical > >>>>>>> way that the netdev community can accept. > >>>>>>>> > >>>>>>>> Why does the new module need to cycle through each net device? > >>>>> Can't > >>>>>>>> a net driver that knows it supports this just call into the new > >>>>>>>> module to register itself when it is loaded? When it registers it > >>>>> can > >>>>>>>> create any sysfs/netlink stuff needed so userspace can detect it > >>>>> and > >>>>>> access it. > >>>>>>> That would work too, but our proposal essentially is tailored toward > >>>>>>> minimizing any storage specific code in the network drivers. > >>>>>>> > >>>>>>> Note that our proposal is to add an ethtool extension which will > >>>>>>> provide read/write access directly to the flash memory. It will not > >>>>>>> do any sysfs creation or parameter qualification. It only provides > >>>>> a > >>>>>>> gateway to the flash memory, that's it. It will be up to the new > >>>>>>> module to initiate, create, and manage over those sysfs parameters. > >>>>>> > >>>>>> Sounds ok to me. > >>>>>> > >>>>>>> > >>>>>>> Perhaps we can add some minor initiation code in the network driver > >>>>> to > >>>>>>> perhaps 'register' some flag so the new module will only have to > >>>>> cycle > >>>>>>> through a list of supported network interfaces only. > >>>>>> > >>>>>> It is ok. I was just thinking along the lines of either ethtool or > >>>>> iscsi mod only. I > >>>>>> cannot think of a major issue to probe with ethtool from userspace > >>>>> like you > >>>>>> suggested before. The only issue might be if we have to do some sort > >>>>> of > >>>>>> probing and checking if this is supported takes a while (like if we > >>>>> have to do > >>>>>> some fw command that takes several to 10-15 seconds each time). For > >>>>> that > >>>>>> case we do not want to have to probe every device during boot or the > >>>>>> boot/distro people will not be happy. > >>>>>> > >>>>> > >>>>> Second that.. > >>>>> > >>>>> One alternative is that the network driver registers to the new module. > >>>>> I would prefer that the new module is loaded already so it can > >>>>> enumerate the /sysfs entries correctly. > >>>>> > >>>> Although we have not proposed this via the netdev ML yet, however, the > >>>> current suggestion to only add ethtool extension to support this is > >>>> gaining traction. > >>>> > >>>> As far as latency goes, a simple for_each_netdev loop to probe each > >>>> netdev's ethtool_ops for these new extensions (like get/set_flash_node) > >>>> shouldn't incur too much cost for unsupported nics. However, for the > >>>> case when there are many supported nics in the system, we would then > >>>> have to decide how to best handle the delay. We might have to rely on > >>>> defer tactics or impose a hard limit on the number of flash targets to > >>>> support on a system. This should be easy to govern via the bios. > >>>> > >>>> How about for HBAs? We probably want to employ the same top-down > >>>> approach and have the new module cycle through each created host and > >>>> pull for the flash node targets. The alternative is to call into the > >>>> new module to create those flash node sysfs entries upon iscsi_host > >>>> allocation. What would be the best way to do this? > >>> > >>> > >>> I think iSCSI HBA Driver call into the new module to create flash node > >>> sysfs entries upon iscsi_host allocation is better way to do it. > >>> > >>> > >> > >> I agree. > >> > > With HBAs, the bottom-up approach seems to be more logical and probably > > would incur lesser of an impact for all vendors. But for non-HBAs, it > > makes better sense for the new module to query through the netdevs > > instead since the networking drivers are passive in this sense. > > > > So, it looks like the best approach would be to fundamentally handle > > them differently. > > > > One related question. Are we also going to have the iscsi hbas's > implement some ethtool ops for this, or will only drivers that interact > with a netdev do that? If the iscsi hbas do it will it use a dummy > netdev or are we going to modify the ethtool ops to not use a netdev. > I imagine the ethtool ops flashnode extension would only be applicable to non-HBA nics. For HBAs, I think a simple transport callback would be sufficient.
So upon iscsi_host_alloc, the HBA would call into the flashnode module to start the sysfs creation process. The flashnode module would alloc the memory and call the HBA driver's get_flashnode callback and pump out the target info. For non-HBA nics, the process would probably be done upon the flash node's module init. It'll also have to handle netdev reg/unreg changes accordingly. > > -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
