Jamal, I'd like to get some good numbers for use-cases to put in.
For operations, it's not clear to me that they will always be batched in bulk as much as is done for down to the forwarding plane - I think aspiring to 100,000s ops/sec is a bit unlikely unless it is carefully crafted with good bulk operations. I'm putting in 10,000 as being more than we can generally do now with NetConf or CLI - but I'd be happy to see a use-case that justifies more. Alia On Wed, May 7, 2014 at 9:13 AM, Jamal Hadi Salim <[email protected]> wrote: > On Tue, May 6, 2014 at 7:05 PM, Dean Bogdanovic <[email protected]> wrote: > >> High-throughput is always nice to have, > > Is it "nice to have" or "must have"? > >>but the quantification is a bit wishy washy. >> > > Agreed. > >> High-Throughput: At a minimum, the I2RS Agent and associated router >> should be able to handle a considerable number of operations per >> second above what basic Netconf or a propretiary CLI can. >> >> What does mean basic NETCONF? Today I can do couple hundred RPCs/sec over >> NETCONF. Question: what is RPC doing? Is it simple operational get or large >> config push? >> Or some other complex RPC? >> There is also difference in creating logical interfaces and adding term to a >> filter or adding route to a routing table. Creating interface is most taxing >> one and adding term is fastest create operation. >>IMO, NETCONF has no problem >> with throughput, it is the instrumentation and that is something we should >> focus in I2RS. >> What constructs do we want to control via I2RS and depending on that, make >> sure that the protocol can sustain max throughput. >> > > We need to pick something as a metric since as you say it will depend on the > object type. The RIB table sounds like a reasonable object to standardize on. > Pick a number like 1M table entries (large enough so as to be measurable). > > Throughput: > This refers to some bulk table create/updates/dumps. Very few transactions/sec > are needed, essentially in this case the agent and client are engaged > in batching the entries > they send to the other party. > To handwave some numbers performance numbers: > I am going to pick say 10s to 100s of thousands rows per/sec. > We need bidirectional numbers. > > Transactions rate: > This is a measure of request/response. Again using the RIB example - > this would mean taking > each of the 1M rows and send them one at a time (i.e window of 1). Eg > a request from the > client to create/update is sent to the agent and when a success > response is received by the > client the next request is issued ..... until all of 1M transactions > are complete > >> In the next chapter you are mentioning >> >> Responsive: It should be possible to complete simple operations >> within a sub-second time-scale. >> >> >> Here you are stating simple operation, so why not define few simple >> operations and put some quantification with it (in the below examples I'm >> pulling the numbers out of my hat) >> >> example: >> >> read routes - single prefix per transaction < 0.5sec >> - multiple prefixes per transaction < time on the wire + 0.5sec >> add route - single prefix per transaction < 0.5 sec >> - multiple prefixes per transaction 1000 routes/sec >> >> get filter info - single filter per transaction < 0.5sec >> - multiple filters and terms per transaction < time on the wire + 0.5sec >> create filter - single filter per transaction < 0.5 sec >> - multiple filters per transaction 2000/sec >> add term to filter - single term per transaction < 0.5sec >> - multiple terms per transaction 3000/sec >> >> get interface info - single interface per transaction < 0.5 sec >> - multiple interfaces per transaction < time on the wire + 0.5 sec >> create interface - single interface per transaction < 0.5 sec >> - multiple interfaces per transaction 100/sec >> > > We need to have some quantification numbers. They dont have to be precise > but could be in the ranges. I think what you describe above matches my > thinking. > Your "multiple per transaction" is essentially a batching artifact > which falls under > "throughput" metric. The " single per transaction" is a transaction metric. > > Having said that: I know you gave those numbers as examples, but they > are way too low ;-> > >> I'm thinking how can you do assurance for i2rs. It looks as it is covered in >> following paragraph >> >> Scalable, Filterable Information Access: To extract information in a >> scalable fashion that is more easily used by applications, the >> ability to specify filtering constructs in an operation requesting >> data or requesting an asynchronous notification is very valuable. >> >> >> but it is not spelled out. >> > > I would consider the throughput, transaction rate and latency in that > category as > well. > > cheers, > jamal > > _______________________________________________ > i2rs mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/i2rs _______________________________________________ i2rs mailing list [email protected] https://www.ietf.org/mailman/listinfo/i2rs
