Robert Love wrote:
> On Mon, 2010-03-08 at 11:17 -0800, Joe Eykholt wrote:
>> Hi All,
>>
>> I've started work on the implementation of a proposal in T11 BB-6 to
>> support VN_port to VN_port (VN2VN) operation with FIP. This is similar to
>> but way better than the current point-to-point operation.
>>
>> There's info on this at
>> http://www.t11.org/ftp/t11/pub/fc/bb-6/10-019v2.pdf
>> http://www.t11.org/ftp/t11/pub/fc/bb-6/10-060v0.pdf
>>
>> Note these aren't finalized and will change a bit before becoming
>> standard. There are links to other proposals on the fcoe.com web site.
>>
> Do you have a feel for how far along the standardization is? fcoe.com
> shows the first discussion happened in Aug '09 (but the history doesn't
> go much further back). It seems complicated and it's rolled in to FIP
> which seems to suggest it's fairly mature. :) Do you know if there is
> general agreement on the protocol described in the links above?
I think its pretty far along, but it's hard to say how much longer
until it's adopted, I'd guess 3-4 months, but it could be more.
I'm told it has fairly good general agreement.
>> Anyway, the point of this post is to ask how we should select the
>> mode when creating an FCoE interface.
>>
>> VN2VN allows an FCoE initiator to talk to a number of targets over
>> the ethernet target without any FCF or fabric infrastructure.
>>
>> This could co-exist with a fabric as well, so we could have multiple lports
>> on the same VLAN (even without NPIV) with some of them talking to
>> various FCFs and one (at most) using VN2VN mode on that VLAN.
>>
>> That could be implemented separately, but there's always a chance
>> when running VN2VN mode that a fabric's FCF could appear later, and
>> the proposal doesn't allow for switching from VN2VN mode to fabric
>> mode for an lport. So, this is a long way of saying we need to have
>> a way of setting the mode (fabric or VN2VN) when we first create
>> the fcoe interface.
>>
>> There are some other options we may want to set. One is VN2VN, but
>> only allowing one neighbor (single point-to-point).
>>
>> We may also want to set the world-wide names explicitly, rather than
>> letting them default to the ones generated from the MACs or gotten
>> from the NICs. This would be necessary if handling multiple FCFs.
>>
> To extend on this I think we should have a way to pass in whether the
> lport is in initiator or target mode. Right now we're not hooked into
> stgt, but if that happens I'd rather have the initiator/target decision
> made upfront before any discovery or FLOGI frames go out.
Yes, that's one of the things on my list, too. That should be done at
the libfc level so the code is common, maybe. We could pass the create
options up to libfc but have a way of getting back LLD-specific options.
Depending on how we do it.
>> I see multiple ways to do this and wonder which ones would be acceptable:
>>
>> 1. We could do separate Create and Enable operations, allowing parameters
>
> The ndo_enable() call to the Ethernet driver may reset the DCB
> connection when the HW is reconfigured for FCoE. We may be seeing races
> where discovery frames are hitting the wire before DCB is
> re-established. That could be prevented by delaying the enable call
> until fcoemon has re-confirmed DCB's stability. Splitting enable/create
> may not be the only fix, more investigation is needed, but it seems like
> a reasonable solution to me.
>
>> to be set before the new instance is used. Create would make the
>> instance but not start any network operations until Enable was set.
>> The parameters (mode, WWNs) would be writable attributes somewhere,
>> perhaps in /sys/class/fcoe_host? This breaks compatibility with
>> the user utilities.
>>
> /sys/class/fcoe_host was rejected on linux-scsi in favor of a sysfs
> reorganization for FC to make room for FCoE. I did some work on that,
> but haven't had a chance to complete the work- I hope to get back to it
> soon.
>
> If we do add new sysfs knobs they should be added to that reorganization
> as opposed to being fcoe.ko module parameters. I would like to move
> enable/disable too.
>
>> 2. We could pass extra parameters after the interface name when
>> writing /sys/module/fcoe/parameters/create. For example,
>>
>> # echo eth0 --mode vn2vn --wwpn 1000abcd12345678 >
>> /sys/module/fcoe/parameters/create
>>
>> Parsing could be simpler and less general than getopt, but you get
>> the idea, some set of name/value pairs after the interface name.
>> This maintains compatibility with existing user utilities.
>>
>> 3. Like 2 with simpler parsing could be as simple as:
>>
>> <nic> [<flags> [<wwpn> [<wwnn>]]]
>>
>> Where flags could be single-letter flags for various modes and options
>> like: f for fabric, m for vn2vn multipoint, p for vn2vn point-to-point, etc.
>> This would use less code in the kernel, and maintains compatibility.
>>
> I don't find the option passing to sysfs very nice. We'd have to
> implement option parsing in the kernel which I can imagine would get
> ugly quickly if we need other options later. I'd prefer multiple sysfs
> files over one that takes multiple options. Multiple sysfs files could
> cause problems if you touch one after the enable call (assumes
> create/enable split). Is there any precedence here? I'm not familiar
> with any modules that do option parsing.
Of course the kernel itself does option parsing from the boot options.
I didn't find those routines exported, but there are file systems that
do option parsing on the mount syscall, and those routines are exported
from lib/parser.c. That seems like an OK mechanism. One example use is in
udf_parse_options() in fs/udf/super.c. Its also used by other modules
like selinux and infiniband.
It seems like most of them use comma separated options like:
param=value,param=value,...
The code to use another separator is easy, but comma has some precedence,
and won't appear in interface names, I think.
We could make the first value be the interface name for compatibility,
and do:
eth0,port_name=1234567812345678,fip=ptp,fcp=[target|initiator|both]
or something like that:
The full set of options I can think of now are:
port_name=<value> // set the port name to the supplied hex value
node_name=<value>
fip=[ptmp|ptp|fabric|off|auto] // FIP mode to use
fcp=[target|initiator|both]
disable // don't enable, just create?
> netlink is something else we need to consider. It would allow us to
> implement an easily extensible structure that could define all flags the
> kernel needs to know about. Also providing a mechanism to pass
> information back to user space. This way we'd have a guaranteed format.
> It's what Open-iSCSI and DCB use.
>
> netlink would not solve the HW resetting issue I described above but I
> look at that as a reason to not enable by default on create, not really
> a deciding factor in the communication channel.
>
> //Rob
netlink is OK but I like sysfs since it is working now and we don't need to
distribute programs along with the kernel to use it. I notice everyone
doing a lot of work on the fcoeadm/fcoemon lately, which is good, but
it's nice to be able to test the low level stuff with just echo and cat.
Joe
_______________________________________________
devel mailing list
[email protected]
http://www.open-fcoe.org/mailman/listinfo/devel
