the early research on passive backbone network measurement: http://www.ece.ucdavis.edu/~chuah/classes/eec274/eec274-w09/refs/FML03-ipmon.pdf
[ED: at the time, the working storage of 330GB could potentially keep ~1,300 continuous days of compressed voice capture. (or mere hours of a lightly utilized OC12 if capturing it all like MYSTIC) ] """ The IPMON monitoring infrastructure ... consists of three elements: a set of passive monitoring entities which collect the packet traces; a data repository that stores the traces once they have been collected; and an analysis platform which performs off-line analysis. Analysis is performed off-line for two reasons. The primary reason is that the data is used in many different research projects, each of which has its own set of custom analysis tools. It is more efficient to perform the multiple types of analysis on a computing cluster in the lab where many systems can access the data simultaneously. The second reason is we archive the traces for use in future projects. 1) Monitoring entities ... are responsible for collecting the packet traces. Each trace is a sequence of packet records that contain the first 40 bytes of each packet, which are just the IP and UDP/TCP headers, as well as a sub-microsecond timestamp which indicates the time at which the packet was observed. The source and destination IP addresses are not anonymized, since they are needed in routing-related analysis. Each monitoring entity is a dual-processor Linux server (Dell PowerEdge 6000 series) with 1 GB main memory, a large disk array (100 to 330 GB), and a POS network interface card, known as the DAG card. Existing DAG cards are capable of monitoring links ranging in speed from OC-3 to OC-48... The DAG card captures, timestamps, and transfers the POS HDLC framing information and the IP packet headers to the main memory of the Linux server where a driver software then transfers the data to the disk array. An optical splitter is installed on the monitored link, and one output of the splitter is connected to the DAG card in the server... Each monitoring entity has a removable disk array of up to 330 GB. This amount of disk space allows us to capture a minimum of several hours of trace data at full link utilization. We can either schedule trace collection for a pre-defined interval or allow it to run until space on the hard disks is exhausted. By Sprint engineering design, the network links are not fully loaded (except in extreme failure scenarios) and we are typically able to collect several days of measurement data... A total of 60 monitoring entities are installed at 4 different POPs, chosen on the basis of geographic diversity and connectivity. They monitor the traffic on OC-3, OC-12, and OC-48 links which connect access routers, backbone routers and several of the private peering links. 2) Data Repository... involves two levels of storage, consisting of a 12 TB removable tape library and a 10 TB disk storage array. It is located at the Sprint Advanced Technology Laboratory (ATL). For short traces, a dedicated OC-3 link is available for transferring the data from the monitoring entities back to the ATL. Given that a full multi-POP trace set consists of approximately 10TB when trace collection is allowed to run until the disks fill up, the best method for transferring full traces back to the data repository is by physically shipping the removable hard disks. As a result of these constraints on transferring trace data, we do not schedule new traces until the previous trace data is either transferred or deleted. 3) Data Analysis Platform: Data analysis is performed on a cluster of 17 high-end servers connected to a Storage Area Network (SAN) with a capacity of 10 TB. """ On Tue, Mar 18, 2014 at 7:54 PM, coderman <[email protected]> wrote: > ... [ lots of tapping, everywhere! ] -- Liberationtech is public & archives are searchable on Google. Violations of list guidelines will get you moderated: https://mailman.stanford.edu/mailman/listinfo/liberationtech. Unsubscribe, change to digest, or change password by emailing moderator at [email protected].
