Another application for the "deep packet inspection" technique.. On Jun 9, 2013 6:32 PM, "Gregory Maxwell" <[email protected]> wrote:
> On Fri, Jun 7, 2013 at 6:47 AM, Eugen Leitl <[email protected]> wrote: > > but the ability to assemble intelligence out of taps on providers' > internal connections > > would require reverse engineering the ever changing protocols of all of > those providers. > > This is somewhat less difficult than some people think. > > Various equipment manufacturers have implemented passive monitoring > support on their interfaces specifically for these applications. You > configure the interface to go into UP/UP state and to listen in a half > duplex manner. This way you get the compatibility advantage of using > standard network equipment to implement the interception, and so it > will likely speak the same link-layer protocols the device being > intercepted speaks. > > (E.g. here is some of the relevant documentation for Juniper: > http://kb.juniper.net/InfoCenter/index?page=content&id=KB23036 and > > https://www.juniper.net/techpubs/en_US/junos11.2/topics/concept/flowmonitoring-passive-overview-solutions.html > ) > > A lot of the mechanisms— the protocols, techniques, equipment > features— for mass surveillance are easily visible to the public but > the things visible to the public are all technical minutia dealing > with the practical engineering challenges (Like the one you raise > here— how the heck do you keep up with the ever changing layer 1/2 > protocols used by service providers) that most people wouldn't even > think to ask about. > > Using commodity hardware gets you compatibility, lower costs, and fast > deployment. Even though budgets for massive surveillance no doubt > allow for all kinds of specialized hardware— you can get more of it > faster if you use commodity stuff with a few tweaks where you can. > > Here's another tidbit in public docs: > > Another challenge in implementing massive surveillance is the sheer > volumes of traffic involved. People do seem to be aware of this one, > but they argue that it makes the task impossible.... but there are few > technical challenges which can't be solved by the suitable application > of ingenuity and money. (_Lots_ of money: but keep in mind "defense" > spending is just on another order of magnitude from regular spending. > How much does a fighter jet cost? A one time use smart munition? How > much more valuable is a good network tap than these devices? In light > of that— how much is a fair defense industry price for one?) > > One way that the traffic volume problems gets solved is to realize > that the vast majority of traffic is uninteresting. If you can > rapidly filter the traffic you can throw out the 99% of uninteresting > stuff and capture all of the rest. Filtering is, of course, > computationally expensive— but it turns out that the power of > 'commodity' technology can come to the rescue again: The same > standard networking equipment that you need to speak the L1/L2 > protocols on your optical taps also has built in line-rate packet > filtering with scalability to millions of filter criteria (at no extra > cost! :) ). > > The filtering in these devices has not historically been DPI grade: > you can do stateless range/prefix matches on the packet headers, not > free-form regex (although this is changing and the latest generation > of hardware is more powerful— the need for NAT everywhere, if nothing > else, is mandating it). But, if you can update those filters very > fast— say, in under 50ms— then it doesn't matter that the filters > aren't very powerful: Configure the filters to catch all known > interesting hosts, the beginning of every new connection, and some > small fraction (say, 1:10000 of all packets) and then feed that data > to analysis systems which trigger updates to the filters when they > spot something interesting. They only need to be powerful enough to > limit a terabit of traffic to tens of gigabits, and that level of > filtering can be accomplished just on 5-tuples.. > > You can go even further, then, by having two sets of filters with a > delay line— say implemented using the >100ms of delay-product packet > buffers in high end commodity networking hardware— in between them. > The first set of filters catches enough so that your analysis systems > can identify and track interesting flows, and by the time the traffic > makes it through the delay line the second set of filters has been > updated to capture the entirety of the interesting flow. ... though > the persistence of traffic and the delay created by the TCP three way > handshake make going this far not terribly necessary. > > Of course, using filtering in this way would require a protocol > between the network elements and the analysis systems so that they > could rapidly and dynamically 'task' the filters like you task > surveillance satellites... And it sure would be convenient if the > protocol was standardized so you could get many kinds of devices > speaking it. ... something like: > http://tools.ietf.org/html/draft-cavuto-dtcp-03 > (and here is one vendor's helpful documentation on applying it, > > https://www.juniper.net/techpubs/en_US/junos/information-products/topic-collections/nce/lawful-intercept-flow-tap/lawful-intercept-using-flow-tap.pdf > ) > > I've been continually amazed at how poorly the public has been doing > at figuring out the mechanisms used for this stuff— You don't need > some insider to tell you how it works, you could have just looked up > the authors of packet interception standards and looked for people > people who worked for major providers and advertise TS/SCI clearance > on their resumes— then put together the pieces. People have observed > that building high-tech infrastructure makes keeping secrets very > costly but then fail to notice all of the completely open building > blocks than just require the suitable application of big money. > > You're obviously not going to find smoking guns— things like > production system core-dumps showing that which newspapers were being > targeted for total surveillance— because the people building this > stuff are not idiots, and it's not customary to publish that > information generally. ... but the underlying technology and > documentation needed to build this infrastructure, or at least the > parts built out of commodity hardware. You absolutely can find that if > you bother looking. > > But at the end of the day the technical details behind how it works > really don't matter much. Just assume that a hostile party is at > least passively monitoring all network links that aren't completely in > your control and you'll be making the right kind of security decisions > regardless of the technical minutia. > -- > Too many emails? Unsubscribe, change to digest, or change password by > emailing moderator at [email protected] or changing your settings at > https://mailman.stanford.edu/mailman/listinfo/liberationtech
-- Too many emails? Unsubscribe, change to digest, or change password by emailing moderator at [email protected] or changing your settings at https://mailman.stanford.edu/mailman/listinfo/liberationtech
