Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery (DHCPv6 Bulk Leasequery) to Proposed Standard
I wanted to send a message about my conclusion of this thread, as the draft is now progressing to IESG review. I sympathize with the desire to pull as much data as possible with a simple request and no prior knowledge of what sessions exist. I think it is obvious that with a better understanding of how to use SNMP, we can avoid the silliest approach to pulling data -- I hope there is a way to provide better education on how to use SNMP for something like this. But I'm not expert enough to say whether you can reach exactly an equivalent model as this draft has. Maybe. Nevertheless, it is clear to me that there is consensus in the WG to do this. They are chartered to do infrastructure improvements (such as this one) to DHCP. There are customers who need to do this. The last call discussion has raised alternative ways to implement this but no specific problems in the current approach. I believe that in this case we need to let the WG use the approach they have chosen. More generally, I believe that if there are specific data access needs in applications for large volume, bulk, etc -- the WGs in charge of those applications can choose the approach they want to use for the access to the data. While most of our management needs are fulfilled by MIBs, I do not think we can preclude other solutions for specialized situations, such as recovering DHCP session data. Jari ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery(DHCPv6 Bulk Leasequery) to Proposed Standard
On Wed, Oct 22, 2008 at 09:15:01PM -0700, Randy Presuhn wrote: Uh... One of the useful features of tables is to organize related information into conceptual rows with common INDEX values. I'm not sure where a single table of a single variable at a time comes from - GetBulk certainly has no such limitation. I don't remember referring to GetBulk, and I don't understand why I would have. If you don't know where a single table of a single variable at time comes from, then walk the 'interfaces' mib on any system at your convenience. Note how 'ifDescr', 'ifInOctets', and etc are all organized with spatial locality, but the data for interface 5 is not. This spatial locality does not track with common application temporal locality requirements, which wants all the details for one conceptual object at once so that it may continue with processing it, rather than waiting. For example, you start by walking a table of index advertisements, where you receive an 'index number' that can be used into other MIBs to find variables associated with that database entry. For each of these indexes you discover, you could then queue single GET PDU's for each separate variable you were interested in (lease state, lease expiration time, ...). That would be a spectacularly inefficient implementation strategy. I should hope there's nothing in the SNMP RFCs that would be read as encouraging such wasteful behaviour. Efficiency is often subjective. Are you optimizing for simplicity of design (using one walk to traverse an entire mib), or are you optimizing for the lowest latency to complete configurations? Are you optimizing out variables you have no interest in (most MIBs are half extra data you don't require)? What kind of device are you querying? On some models of router, for example, a SNMP query causes a message to be sent over their own 'control bus' to fetch all statistics for a single interface. The most recent control command is cached, because it was designed to be used for a CLI command to display a single interface's data. The typical snmpwalk or even bulkwalk scenario means every interface is queried multiple times; once for each table they appear in. This kills the cached object, by using none of its locality, and overworks the command bus. Even if the router's cache were larger, it would have to be equal in size to the number of (physical and virtual) interfaces in order to have even one hit. An application trying not to live in 1985 might find itself doing, as I describe, a kind of dance to get out of SNMP's spatial locality mindset. In this case, you would query all the data you required pertaining to one interface at a time, before moving on to the next interface. An application requiring not to live in 1985 definitely will find itself queuing more than one packet to a single SNMP server at a time. In this case, walking some number of tables in one session, while queuing query packets filled to the brim with GET's for all specific relevant data for each interface in series. Unless you're querying a Cisco Catalyst, in which case queuing more than one packet at a time is foolish; the Cat will drop the second packet if it is busy with a request already, and you'll have to sit in retransmission hell. But I'm getting sidetracked. The advantages here I am describing are twofold; 1) The object for interface 5 is read in the least time possible, even though the time to read the entire dataset remains constant (due more to RTT than to bandwidth). interface 5 can move on to the next stage in processing while SNMP data continues to flow, rather than waiting for the walk to finally reach the last table of variables. 2) The router is not flagellated needlessly, resulting in a lower time to complete the operation overall. I'm not sure what you're trying to say here. An SNMP message (which would normally be carried in a single UDP datagram) by definition contains exactly one SNMP PDU. Possibly this is a kind of brain damage that arises from the SNMP library I once used, which describes multiple query commands in one packet as PDU's, even if they are labeled internally to SNMP as being 'variable bindings' attached to the end of the PDU. I wind up using the terms interchangeably as a result. This depends on the design of the MIB module in general, and the selection of the INDEX elements in particular. I don't see how any SNMP MIB design cleverness can get around the SNMP way of requiring strictly ordered datasets for GETNEXT support. But good luck with that, and I look forward to reading your DHCP MIB's draft. -- Ash bugud-gul durbatuluk agh burzum-ishi krimpatul. Why settle for the lesser evil? https://secure.isc.org/store/t-shirt/ -- David W. HankinsIf you don't do it right the first time, Software Engineeryou'll just have to do it again. Internet Systems Consortium, Inc. -- Jack T. Hankins pgppkdaTI6HQu.pgp
Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery (DHCPv6 Bulk Leasequery) to Proposed Standard
On Wed, Oct 22, 2008 at 08:36:22AM +0300, Pekka Savola wrote: $ snmpwalk -m IP-FORWARD-MIB -v 2c -c foo foo-rtr This essentially would be identical to DHCP leasequery, minus the bulk. Even if transported via TCP, on the wire it would look like a single client-server GETNEXT, waiting for a server-client reply before sending the next one. One PDU and one OID in it at a time. snmpbulkwalk is a minor improvement; a single UDP reply can contain many iterated GETNEXT's, or similarly over TCP. There is still a 'pause' between the client's request and reply. What the leasequery bulk methods are looking for is all at once. The objective is to fill the socket at maximum window size. I am not aware of a means to do this with SNMP considering the kinds of data in DHCP lease tables, and the usual ways MIBs are constructed. ip.ipForward.ipCidrRouteTable.ipCidrRouteEntry.ipCidrRouteProto.128.214.46 IP-FORWARD-MIB::ipCidrRouteProto.128.214.46.0.255.255.255.0.0.0.0.0.0 = INTEGER: netmgmt(3) IP-FORWARD-MIB::ipCidrRouteProto.128.214.46.254.255.255.255.255.0.0.0.0.0 = INTEGER: local(2) This kind of underlines a qualm with SNMP data management (as opposed to network management). For each iterated GETNEXT or GETBULK both rely upon a fixed point in the database (node n), which was present in the database and was the last PDU in the server's reply in the previous packet. But it may not be present in the database at the time the next request is made. This requires the database models used by the servers to be able to find a reliable sorting, where the previously-valid-now-invalid OID can still be used as an index to provide continuation; it can still grant the next OID. This essentially is a problem, or at least a facility not present in some DHCP databases, which caused us to motivate away from UDP based bulk queries (the original bulk leasequery proposal was more similar to snmpwalk in this regard, and this was a concern raised by implementers). In addition... SNMP likes to present a single table of a single variable at a time. I suppose we could overcome this by having the DHCP lease information in an 'blob of octets' rather than in classical SNMP variable form (INTEGER etc), so you only have one MIB to walk. But it seems foreign to SNMP to do so. The problem is that most leasequery clients are not positioned to allocate fields of temporary memory in order to make sense of SNMP's kind of scatter-gather approach to this kind of data transfer. To make sense of SNMP MIBs you have to develop some strategy to receive multiple datapoints from different locations and times. For example, you start by walking a table of index advertisements, where you receive an 'index number' that can be used into other MIBs to find variables associated with that database entry. For each of these indexes you discover, you could then queue single GET PDU's for each separate variable you were interested in (lease state, lease expiration time, ...). There are 'performance alternatives' from there, and they are fantastic to entertain because so many SNMP server implementations will outright crash if too many PDUs are queued in a single packet (others corrupt their replies if there are more than single PDU's). This becomes more problematic when you consider that some leasequery clients are going to want only a subset of the MIB's total contents. The question truly is what leases did I have in my table before I rebooted? Such filtration in an SNMP MIB model I think would be done on the client end, not on the server end, meaning the client still must traverse some entire MIB one PDU (GETNEXT or GETBULK) at a time. This is different from the proposed bulk leasequery models, where the server writes to the TCP socket all at once, with all data for a given lease spatially located in the same position in the TCP stream, and a primitive query language (by query type) to provide subsets. This doesn't mean a standard DHCP MIB isn't a bad idea for entirely different reasons. -- Ash bugud-gul durbatuluk agh burzum-ishi krimpatul. Why settle for the lesser evil? https://secure.isc.org/store/t-shirt/ -- David W. HankinsIf you don't do it right the first time, Software Engineeryou'll just have to do it again. Internet Systems Consortium, Inc. -- Jack T. Hankins pgpWCakKMUnRv.pgp Description: PGP signature ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery(DHCPv6 Bulk Leasequery) to Proposed Standard
Hi - From: David W. Hankins [EMAIL PROTECTED] To: DHC WG [EMAIL PROTECTED] Cc: ietf@ietf.org Sent: Wednesday, October 22, 2008 10:17 AM Subject: Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery(DHCPv6 Bulk Leasequery) to Proposed Standard ... SNMP likes to present a single table of a single variable at a time. I suppose we could overcome this by having the DHCP lease information in an 'blob of octets' rather than in classical SNMP variable form (INTEGER etc), so you only have one MIB to walk. But it seems foreign to SNMP to do so. Uh... One of the useful features of tables is to organize related information into conceptual rows with common INDEX values. I'm not sure where a single table of a single variable at a time comes from - GetBulk certainly has no such limitation. ... For example, you start by walking a table of index advertisements, where you receive an 'index number' that can be used into other MIBs to find variables associated with that database entry. For each of these indexes you discover, you could then queue single GET PDU's for each separate variable you were interested in (lease state, lease expiration time, ...). That would be a spectacularly inefficient implementation strategy. I should hope there's nothing in the SNMP RFCs that would be read as encouraging such wasteful behaviour. There are 'performance alternatives' from there, and they are fantastic to entertain because so many SNMP server implementations will outright crash if too many PDUs are queued in a single packet (others corrupt their replies if there are more than single PDU's). I'm not sure what you're trying to say here. An SNMP message (which would normally be carried in a single UDP datagram) by definition contains exactly one SNMP PDU. This becomes more problematic when you consider that some leasequery clients are going to want only a subset of the MIB's total contents. The question truly is what leases did I have in my table before I rebooted? Such filtration in an SNMP MIB model I think would be done on the client end, not on the server end, meaning the client still must traverse some entire MIB one PDU (GETNEXT or GETBULK) at a time. This depends on the design of the MIB module in general, and the selection of the INDEX elements in particular. Choosing INDEX elements for a MIB module is *not* the same problem as selecting indexes for a database. The use cases for information access, such as what leases did I have in my table during time period X are also important. Sometimes it makes to have shadow tables that do nothing but provide re-ordered access to the table with the real data - but this requires careful thought about what the high-frequency or high-value use cases are. Randy ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery (DHCPv6 Bulk Leasequery) to Proposed Standard
On Tue, Oct 21, 2008 at 8:09 AM, Pekka Savola [EMAIL PROTECTED] wrote: On Mon, 20 Oct 2008, The IESG wrote: The IESG has received a request from the Dynamic Host Configuration WG (dhc) to consider the following document: - 'DHCPv6 Bulk Leasequery ' draft-ietf-dhc-dhcpv6-bulk-leasequery-04.txt as a Proposed Standard The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2008-11-03. Exceptionally, comments may be sent to [EMAIL PROTECTED] instead. In either case, please retain the beginning of the Subject line to allow automated sorting. The file can be obtained via http://www.ietf.org/internet-drafts/draft-ietf-dhc-dhcpv6-bulk-leasequery-04.txt First there was DHC Leasequery (RFC4388), next DHCv6 Leasequery (RFC5007), now we have DHCv6 Bulk Leasequery. And someone seems to be proposing DHCPv4 bulk leasequery as well (draft-dtv-dhc-dhcpv4-bulk-leasequery). RFC4388 S4.2 described reasons why SNMP was deemed inappropriate. And if you look at the reasoning there, some of these are not even valid anymore for bulk leasequeries. I remain unconvinced. A far better solution would seem to be define a smaller MIB just for querying leases so implementing it would be trivial. Bulk leasequeries just underline the fact that SNMP and MIB data models are being reinvented inside DHCP. -- Pekka Savola You each name yourselves king, yet the Netcore Oykingdom bleeds. Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings I am not an expert on SNMP, but the only way I could imagine that working, would be by using queries for MIBs which would look like this: get MIB.querytype As the query type can be a relay id, link-address or remote id, this would look a bit strange to me. I know and use SNMP mostly for querying specific, predefined counters or tables, not variable entries in the MIB tree. Also all implementations I know, use UDP not TCP for SNMP queries and replies. The DHCPv6 Bulk Leasquery proposal looks like a logical next step to me. Marcus ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: [dhcwg] Last Call: draft-ietf-dhc-dhcpv6-bulk-leasequery (DHCPv6 Bulk Leasequery) to Proposed Standard
I was hoping some folks more intimate with SNMP and IETF management frameworks would chime in here, but I'll respond to clarify: On Tue, 21 Oct 2008, Marcus wrote: I am not an expert on SNMP, but the only way I could imagine that working, would be by using queries for MIBs which would look like this: get MIB.querytype As the query type can be a relay id, link-address or remote id, this would look a bit strange to me. I know and use SNMP mostly for querying specific, predefined counters or tables, not variable entries in the MIB tree. If I understand what you want to achieve, what you want is supported and indeed used by lots of MIB modules out there. This is useful and indeed we use it regularly with e.g. BGP and IP forwarding MIBs: $ snmpwalk -m IP-FORWARD-MIB -v 2c -c foo foo-rtr ip.ipForward.ipCidrRouteTable.ipCidrRouteEntry.ipCidrRouteProto.128.214.46 IP-FORWARD-MIB::ipCidrRouteProto.128.214.46.0.255.255.255.0.0.0.0.0.0 = INTEGER: netmgmt(3) IP-FORWARD-MIB::ipCidrRouteProto.128.214.46.254.255.255.255.255.0.0.0.0.0 = INTEGER: local(2) Also all implementations I know, use UDP not TCP for SNMP queries and replies. The DHCPv6 Bulk Leasquery proposal looks like a logical next step to me. An open-source SNMP implementation (net-snmp) is at least available. SNMP over TCP is defined in RFC3430. The systems under consideration already support TCP, just the TCP SNMP server part is missing; this should be trivial to implement if there is a need -- the lack of implementation efforts seems like an indication that UDP with retransmissions is usually good enough. -- Pekka Savola You each name yourselves king, yet the Netcore Oykingdom bleeds. Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
