>-----Original Message----- >From: [email protected] [mailto:[email protected]] >Sent: Tuesday, December 15, 2009 9:35 AM >To: Multanen, Eric W; [email protected] >Cc: [email protected] >Subject: RE: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring > >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] >> On Behalf Of Eric Multanen >> Sent: Tuesday, December 15, 2009 10:31 AM >> To: [email protected] >> Subject: [Open-FCoE] [fcoemon PATCH v2 00/11] rfcoemon restructuring >> >> v2 - restore code in the service script which destroys FCoE interfaces >> for >> which DCB is not required when the service is stopped. >> >> This patch set provides a fairly significant redesign of fcoemon. >> fcoemon >> was suffering from a number of issues, including being too reactive to >> events. This resulted in unstable behavior. This note describes the >> high >> level functions of the redesigned fcoemon. The following patches >> provide the >> implementation. >> >> fcoemon will now have the following high level structure: >> 1. Read in FCoE interface configuration files. This information is >> used >> to create a list of FCoE interfaces. As link and dcbd events >> occur and dcbd queries are resolved, the FCoE interface list will >> manage >> the creation and destruction of FCoE interfaces. Each FCoE >> interface >> element in the list keeps track of its network interface (from the >> config file and could be a VLAN), its real network interface >(found >> from link events during runtime), and its next action (create, >> destroy, >> etc.). >> >> 2. As link events are received from the system, fcoemon determines >> which ones >> are relevent for the FCoE interfaces. Relevent link events are >> those which >> occur on interfaces which are configured to be FCoE interfaces >(see >> the FCoE interface list above), or on the underlying real network >> interface, >> if the FCoE interface is configured on a VLAN. A list of relevant >> real >> network interfaces is maintained. The list is used to track the >> operstate of the real network interfaces and, if DCB is required, >> the status of dcbd queries for the real network interface. >> >> 3. The core loop of fcoemon: >> - listens for link events from rtnetlink and updates the state >of >> the >> real network interface list. Link down events will terminate >> any >> pending dcbd query sequences and a subsequent link up will >> start the >> sequence over from the beginning. Delete link events (VLAN >> deleted, >> driver unloaded) will mark the FCoE interface for destruction. >> - listens for dcbd events and response messages. If the >response >> matches the current dcbd query state, then move to the next >> state. >> If a dcbd event indicates a change in a DCB feature of >interest, >> then start the dcbd query sequence over. >> - manages a timeout for maintaining a connection to dcbd. If >the >> dcbd >> service stops, this timeout will clean up as needed and re- >> establish >> the dcbd connection once dcbd starts up again. >> - at the end of each event/timeout cycle, after all link and >dcbd >> and network interface events have been handled, perform any >> pending next actions. >> For network interface elements, that could mean sending the >> next >> dcbd query, or, if the dcbd queries are complete, analyze the >> results and determine if the FCoE interface should be >> will now handle all FCoE interfaces whether or not DCB is >> required. >> For FCoE interfaces, this means creating or destroying the >FCoE >> interface. >> >> 4. All FCoE interfaces, whether DCB is required or not, will be >> managed by >> fcoemon now. Previously, the init.d start script would handle >> configured >> FCoE interfaces which were configured without DCB required. >> >> >> dcbd query sequence >> =================== >> When DCB is required for an FCoE interface, the real network device >> performs a >> series of dcbd queries to obtain the current DCB configuration. The >> sequence >> is as follows: >> 1. DCB state - is DCB enabled on the interface? >> 2. PFC configuration (Priority Flow Control) >> 3. FCoE configuration >> 4. PFC operational state >> 5. FCoE operational state >> >> If an error occurs at any step, or if all the steps are completed and >> the >> dcbd state does not match the conditions to either create or destroy >> the >> FCoE interface (see next sections), then a retry timer is started. >> Once the >> timer expires, the dcbd query sequence is reinitiated from the >> beginning. The >> timer begins at 0.2 seconds and increases by multiples of that amount >> up to >> ten retries. Once the max retries has been reached, without a >> successful >> completion of the dcbd query sequence, then the FCoE interface is >> marked for >> destruction. >> >> The function of this retry mechanism is to protect against destroying >> the >> FCoE interface due to intermittent problems. Such as when links go >> down >> momentarily due to configuration changes. When this occurs, dcbd may >> take >> a couple seconds to resynchronize with the peer device. >> >> >> Required conditions to create an FCoE interface >> =============================================== >> 1. Link is up on the network interfaces required for the FCoE >> interface, and >> >> 2a. DCB is not required. >> OR >> 2b. DCB is required >> AND DCB is enabled >> AND PFC is enabled >> AND App:FCoE is enabled >> AND PFC and App:FCoE are operational >> AND PFC and App:FCoE operational values match >> >> >> Required conditions to destroy an FCoE interface >> ================================================ >> 1. The network interface required for the FCoE interface is removed >> Such as driver is unloaded or VLAN interface is destroyed. >> This is detected by DELLINK rtnetlink events. >> >> 2. The network interfaces required for the FCoE interface are up, >> AND DCB is required, >> AND on root network interface >> DCB is disabled >> OR >> App:FCoE is disabled >> OR >> PFC and App:FCoE are operational >> AND PFC and App:FCoE operational values mismatch >> >> 3. When DCB is required and the conditions to create the FCoE >> interface are >> not satisfied after going through the maximum number of retry >> sequences, >> as described in the "dcbd query sequence" section, then the FCoE >> interface >> will be destroyed. >> >> --- >> >> Eric Multanen (11): >> Update version string of fcoemon. >> Remove FCoE interface management from service start script >> Add dcbd query timeout and retry mechanism >> Update the fcoemon state machine to be insensitive to >> intermittent dcb changes >> Update the FCoE and network interface lists. >> Add separate arguments to specify FCoE and network interface >> Fix the print_errors() routine >> Remove extraneous dcb members from the network interface >> structure. >> Fixup dcbd connection timeout code >> Remove unused dcbd routine. >> Obtain the real device of a VLAN interface using an ioctl. >> >> >> etc/initd/initd.fedora | 66 -- >> etc/initd/initd.suse | 37 - >> fcoemon.c | 1504 >++++++++++++++++++++++------------------ >> -------- >> fcoemon.h | 67 +- >> fcoeplumb.in | 121 ++-- >> 5 files changed, 789 insertions(+), 1006 deletions(-) >> >> -- >> Signature: Eric Multanen <[email protected]> >> _______________________________________________ >> devel mailing list >> [email protected] >> http://www.open-fcoe.org/mailman/listinfo/devel > >Shouldn't the requirement to administer DCB be FCoE independent ? After >all DCB is required for other protocols like iscsi as well. > >How about using a library interface to manage DCB over a network >interface so that both open-iscsi and open-fcoe could administer DCB?
dcbd is an independent process and can be administered by client programs (e.g. dcbtool). fcoemon is also a dcbd client, but currently only queries DCB configuration and status. It does not actively administer the DCB configuration. open-iscsi could become a client of dcbd as well if there is a need. _______________________________________________ devel mailing list [email protected] http://www.open-fcoe.org/mailman/listinfo/devel
