On Sun, Dec 2, 2012 at 3:16 PM, Magnus Danielson <[email protected] > wrote:
> Erich, > > > On 12/02/2012 08:54 PM, Erich Heine wrote: > >> Jonathan, >> >> My research group has had some good experiences using products from >> Endace ( >> http://www.endace.com/) for network timing measurement at the ethernet >> level. I don't have a pointer immediately to the work, but if there is >> interest can ask tomorrow at work. The gist of it though was to understand >> precise timing characteristics of network switches for better simulation. >> Examining the time "in switch" for various packets at the microsecond >> level >> was needed to understand various delay curves for different network loads, >> with an ultimate goal of proper statistical modeling reflecting reality as >> close as possible. >> > > This is a bold challenge, it's a difficult task (clear speak: there is a > reason for this to be a research field, industry never *really* got it > under control). > > It is a challenge. I'll have to re evaluate my understanding of the timing characteristics of the card in light of my new and growing knowledge on timing. However, this is what I understand now: I'm sure the timing isn't perfect, but we mitigated some of the potential clock issues by doing many runs of tests. Further a single endace DAG card has 2 or 4 network ports on it, so timing can be measured across a network to the same card with the same reference clock which helps simplify error source. (The card has a feature which allows time stamping packets on the card from that reference clock). Also there is a mechanism by which the endace cards can be connected directly to each other to synchronize their clocks to each other, so packet timings across cards can well measured. I wasn't very involved in the details of the project I'm describing, so I can't speak to how well any one of those functionalities of the cards performed, nor how much error we saw or tolerated, I'll ask those folks later today (as I happen to have a meeting with them about other stuff anyway :)) One thing I can speak to though, is that in simulation of the type they are doing - the general granularity of the simulation results are on the order of 10^-3, so understanding the switches at 10^-6 isn't used directly in the simulation, but rather to build probability distribution functions that capture the behavior of the switches well enough to see cumulative results comparable to observed networking conditions (e.g. the result of N switches handling serializing events under certain loads will get packet trains like $X and other loads like $Y and so on). One thing the PI on that project mentioned was the shape of those functions being correct is the highest priority. I understand this to mean that the errors "average out" (not an actual mean, but through some statistics I don't understand all the way yet they stop being significant compared to other issues). Perhaps I'll need to point those researchers at this list to find out ways to better get the timings they want :) > I personally have also used endace products to measure packet timings for >> research, but I didn't need so much precision for that work - however I >> can >> say they have a good API and decent tech support for interacting with >> their >> cards and products. >> > > Is there native support with Linux kernels? > It would be very nice to have the support using SO_TIMESTAMP and friends. > > It's been a couple of years since I got deep on this, so I don't remember details. I do know all the work we've done with the cards was on Linux, and it worked nicely - I had no issues that were OS related. One thing about the paradigm for the DAG cards is they are not in the standard networking stack. They expose themselves from the kernel in a different way via an API (which Endace provides the source for). To do IP networking with them, you need to provided your own network stack in userland - however this is out of the scope of the companies goal. They are really about providing high speed layer2 access. I believe their primary use cases are research like we did, and extremely low-latency communications between machines in clusters (targeting high frequency traders and the like). > Cheers, > Magnus > > > ______________________________**_________________ > time-nuts mailing list -- [email protected] > To unsubscribe, go to https://www.febo.com/cgi-bin/** > mailman/listinfo/time-nuts<https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts> > and follow the instructions there. > _______________________________________________ time-nuts mailing list -- [email protected] To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
