Cathy Zhou wrote: >> This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the >> right granularity? It seems odd to me that we could potentially >> configure the system to have some addresses that respond and some that >> do not. The point of "no accept" mode is to behave as a real VRRP >> system. If you want that, I don't think you'd want it just some of >> the time, would you? > You are right. I can add a per-ill flag. But since I would also like > this flag to be observable by the administrator, I will keep the > IPIF_NOACCEPT flag so that ifconfig bge0:1 will be able to show this > address is set to be "no_accept".
I don't understand. The IFF_* flag space is subdivided into flags on ipifs (IPIF_*), flags on ills (ILLF_*), and flags on phyints (PHYINT_*). Those are all OR'd together when delivered to user space. Thus, at least for observability, it does not matter where you put the flag. What does matter is that you choose the right granularity based on what the flag is supposed to report and/or control. If it's something that describes the behavior of the ill, then it should be an ILLF_* flag. If it's something that needs to be set separately for each address on the ill, then it needs to be an IPIF_* flag. Having an ill flag and "keeping" the IPIF_NOACCEPT definition sounds like a syntax error to me. (I don't honestly know whether it needs to be on the ill or the ipif. My guess is ill, but I brought up the question because it didn't seem to be stated clearly in the materials.) >>> The IFF_NOACCEPT flag can only be set on a IP addresses if its >>> IFF_VRRP flag is set. Further, the same SO_VRRP socket option must >>> be set to change the IFF_NOACCEPT flag over a IP address. >> >> What's the rationale for requiring SO_VRRP? Nothing else knows about >> IFF_NOACCEPT, so (unlike IFF_UP) it can't be set "accidentally" by >> anyone else. (And you clearly can't *stop* appropriately privileged >> applications from setting SO_VRRP on their own.) > First, an application could potentially make a mistake by not getting > ifflag before setting ifflag, which could potentially clear > IFF_NOACCEPT. Applications can do that with any flag, and we haven't needed to go to that sort of protection mechanism in the past. Moreover, as a new flag, this would likely be in the upper 32 bits. That means that the standard BSD ioctls can't touch it at all. You need the Solaris-specific SIOC[GS]LIF* family instead. That issue alone ropes off a large population of potentially troublesome applications. (It just seems odd to me. Why is there no SO_IPMP wrapper around IFF_IPMP or SO_DHCP around IFF_DHCPRUNNING?) > Second, this is also related to setting/clearing the > IFF_UP flag over an VRRP VNIC: a non-accept VRRP router usually sets > IFF_UP and IFF_NOACCEPT using the same socket when it becomes master, > having SO_VRRP is to make the socket setting both flags to becomes a > VRRP control socket session. It feels like too much complication to me, and very much unlike all the other cases where we've had to deal with subsystems that manage IP interfaces ... but I'm not going to argue it further. >> A separate question on the IFF_UP limitations: I'm not sure why that's >> being controlled by the VRRP daemon. Why not allow IFF_UP to be the >> normal administrative control on the address, and just have the VRRP >> daemon control whether the address is "live" on the network (i.e., >> turning on and off ARP and ND for the address, as well as controlling >> input and output)? >> >> This proposal looks workable, but it still looks a little more complex >> to administer than I'd expected. > You mean another flag to control the liveness of the network? We thought > about that option, and eventually we choose today's approach is that > there is ambiguity of the "IFF_UP" flag. It can be seens as an > administrative flag, but some application also believes it shows the > current state of the address: once it is UP, the address should be usable. Yes. My argument, though, is that it doesn't actually matter if the address is marked "UP" but (for reasons outside the application's control) is not currently usable on the wire. At most, lack of network connection is a reason to avoid that address when doing source address selection -- but, then, you almost certainly should be avoiding these VRRP-controlled addresses when doing source address selection for outbound connections anyway. >>> The RTM_FREEADDR event will be generated when a IP address is >>> removed from the configuration. >> >> These don't look like messages on other systems. Are there any >> parallels with existing BSD extensions? > I looked but didn't see things similar to it. BSD does provide a > IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface > but that cannot satisfy my needs. It'd probably help to describe that message and why it doesn't provide the information needed. (In fact, that message is exactly what I'd expected was needed ...) >>> Note that both events will not report the unspecified (all-zero) >>> IP addresses. >> >> I'm not sure what that means. Why would those need to be special? Or >> is that just an implementation limitation? >> >> Aren't all ipifs constructed with zero address when first plumbed ... ? > It is related to implementation. Usually, the first zero-address is the > result of the SIOCLIFADDIF, but the IP address is set as the result of > the SIOCSLIFADDR. We only generate the RTM_CHGADDR as a result of the > second operation. > > Also, I can imagine that application could care about the plumbing event > of the interface, but I assume that no application would care about the > first zero-IP addresses, to me, that zero-address is a side effect of > our implementation. Hmm. OK. >>> # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ >>> -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 >> >> When the user does this, where is the information stored? Does >> "create-router" write the address into /etc/hostname.*, and so on? > The vrrp configuration stores in /etc/vrrp.conf, and we will make use of > libipadm to do the ip configuration persistence. Ah, ok. That latter part is what I was after. >>> - priv_net_rawaccess >>> >>> Required to hold the physical data-link (which owns the >>> primary IP address) and the vnic (which owns the virtual >>> IP addresses) open to prevent them from being deleted. >> >> Why is that necessary? > To prevent the data-link from being deleted or renamed. Note that we > don't require the underlying interface and VNIC to be plumbed when the > router is enabled. Instead, the PF_ROUTE socket will track all the IP > addresses configured over the underlying interface and VNIC and > determine the primary IP address and virtual IP addresses. Why do you care whether the underlying link is deleted or renamed? How does that work with DR? >> That seems reasonable. Is there some way that SMF or Quagga could be >> configured to signal VRRP when a routing daemon fails? > VRRP also needs to know that which virtual addresses those services are > using and find out the specific VRRP router. > > In theory, I think that can be done and can be a useful feature, but > this will be out of the scope of the current VRRP/ILB project. OK.
