Darren Reed wrote: > On 26/07/10 08:18 AM, James Carlson wrote: >> darren.r...@oracle.com wrote: >>> Author: Darren Reed<darren.r...@oracle.com> >>> Repository: /hg/onnv/onnv-gate >>> Latest revision: 2794a0c9cce102961d08f075ee1f569073b99786 >>> Total changesets: 1 >>> Log message: >>> 6965774 bound the interface index in IP to [1,65535] >>> >>> Files: >>> update: usr/src/uts/common/inet/ip/ip_if.c >>> update: usr/src/uts/common/net/if.h >> >> I suspect this change will have two serious effects: >> >> - SNMP is now potentially broken. The interface IDs cannot be reused >> without an engine reboot indication, and restricting to a 16 bit range >> makes reuse very much more likely than before. > > Which interface IDs cannot be used?
The SNMP ifIndex value itself cannot be recycled without reporting a reboot in the SNMP engine. (It's not necessary that the *system* itself be rebooted, but that at least the SNMP engine behave as though the system were rebooted by bumping up the generation number.) See RFC 2863 section 3.1.5. > Do you mean to say that SNMP cannot handle two different > network interfaces having the same interface ID during the > lifetime of the SNMP agent? Correct. > But to me this sounds like a fundamental problem with SNMP and > that choosing 32bits for a network interface identifier does not > represent a better design or solution, only a pushing out the > problem to some point in the future and hoping that it never > appears on your watch. Correct. At 2^32 IDs, "not on my watch" means that a system churning away at (say) 5 plumb/unplumb sequences per second would last a bit over 27 years before the feared roll-over occurs. My plans at that time include being either retired or dead. Maybe both. 2^16 IDs passes by much more quickly. >> - Systems with large numbers of virtual interfaces (tunnels and PPP >> links, for example) may now run out of identifiers when this wasn't >> previously possible. > > This was considered. > > 65,535 is a LOT of network interfaces. > Further, the limit is only per instance of IP. A "lot" depends on what you're doing. At just one replumb a minute, you'd wrap in 45 days. > A given installation of Solaris can only support 1024 zones and > it would require every zone to be using a shared network instance > and 64 network interfaces for it to be a problem in that direction. Simultaneously, yes, that's one possibility. Sequentially, though, it's much easier to get there. And things like tunnels can get you there much faster. I'm *SURE* that punchin has rolled over that 2^16 limit many times over. > Can you imagine how long "ifconfig -a" would take to run on a > system with that many network interfaces? The numbers are intentionally like PIDs; they're not reused until the worst happens. I believe that's the point you may be missing here. > I suspect that anyone who is pushing beyond using 1000 network > interfaces at any particular point in time is going to be facing > all sorts of performance issues. Yes. That's why we did the SolarMAX work many years ago. 1000 was once quite hard. Post-SolarMAX, we demonstrated scaling to tens of thousands of interfaces with little trouble. >> Was any consideration given to fixing the applications that rely on >> these old BSD interfaces? That's what we had been doing since at least >> 2.6 -- making the applications aware that ifIndex numbers could be >> ambiguous, and using other means to verify the data. There are numerous >> examples of this work in the source base. Let me know if you need >> pointers; I can google it for you. ;-} >> >> In general, having aliases in the ifIndex space seen by the old BSD >> interfaces is "not a problem" for most applications, because getting >> routing socket messages just means that it's time to use the ioctls to >> get the real current information. The routing socket messages >> themselves contain too little data to make a fully functioning program >> on Solaris anyway. >> >> Or perhaps "fixing" these old interfaces with some new mechanism? > > In fixing the situation to support more than 16bits for a network > interface identifier, changes to the routing message would be > required. That change would then break our compatibility with > every open source application that exists and uses those messages. > It would also break compatibility with older applications built for > Solaris. If a new interface was introduced and the old one left in > place for compatibility reasons, that doesn't stop the ones using > the old interface from causing strange behaviour when the index > exceeds 65535. The sockaddr_dl message is used by applications to > both send and receive routing messages. Indeed. That's exactly why this interface has been left alone for decades, and instead the fixes were put into the applications using the interfaces. > In the quick survey that I did of source code that used this interface, > only the IKE daemon appeared to be able to changed easily to use > SIOCGLIFINDEX. See for example lookup_pif_by_uindex() in dhcpagent and its caller, lookup_smach_by_uindex(). What the code does is to assume that if the kernel reports an event on index number 0xYYYY, then the daemon should (and will) check for changed values on every interface where (index & 0xFFFF) is equal to that value. I don't believe that the fix applied for CR 6965774 is really the right idea. It perhaps makes some sense in a Windows-like environment where you're encouraged (sometimes forcefully) to reboot every few hours or so, but not so much for an OS that runs for a long period of time. Tossing away the 32-bit counter that we put into IP decades ago seems like a step backwards. -- James Carlson 42.703N 71.076W <carls...@workingcode.com> _______________________________________________ networking-discuss mailing list networking-discuss@opensolaris.org
