FYI the HTTPS site does not work for me here on vacation while the normal HTTP site does. Could be a simple config issue.
On Fri, Jul 4, 2014 at 5:04 PM, Paul Vixie <[email protected]> wrote: > > > Roland Dobbins wrote: >> I know that some DNS operators disable logging of queries/responses due to >> the overhead of doing so - are most folks on this list with large-scale DNS >> recursive and/or authoritative DNS infrastructure disabling logging, >> enabling it, and/or logging queries/responses out-of-band via packet-capture >> taps, databases, etc.? > > we've been using PCAP to do passive dns collection since 2008 or so, and > we've determined that it is unsuitable. the overhead of reassembling > fragmented IP datagrams is fairly low, but the overhead and complexity > of reassembling TCP streams is extreme -- post-processor i know is > willing to pick through a TCP stream inside a PCAP file processing each > transaction. > > our first attempt was NCAP, which was better in that it represented DNS > messages rather than IP packets. however this approach quickly showed > its age and naivety when we tried to express "orphan query, no response > seen" or "orphan response, no query seen" or "this is a query and > response pair". > > our next attempt was NMSG, which was better in that it was based on > google protocol buffers, which are extensible without fork lift upgrades > to your producers and consumers, and which are extremely performant. our > PCAP sensor today generates NMSG output, which is then post-processed > into many intermediate real time channels and ultimately informs the > DNSDB passive DNS database. however, the reliance on PCAP turns out to > be a problem for appliances, and the problem of collecting TCP remains > difficult, and there are many events we care about which don't appear on > the wire at all (cache expiry vs. cache purge, for example). > > therefore, 'dnstap', as presented by robert edmonds of farsight during > the recent dns-oarc meeting in warsaw. 'dnstap' output can be created by > PCAP feeds, but it is optimized for in-nameserver data collection, and > it supports many different message types, and is extensible, and very, > very performant. 'dnstap' tries hard to offload its CPU and memory costs > to other processes, and is unlike syslog() completely asynchronous -- no > queries are ever dropped because a 'dnstap' consumer couldn't keep up, > and thus the primary purpose of "name service" is not threatened by the > secondary purpose of "gather telemetry". > > see https://dnstap.info/ for more details. so far this is live in knot > (including kdig, which can now read and generate 'dnstap' format > telemetry streams), and has been implemented on a development branch of > unbound. we hope to get it into BIND and NSD in the next few months. we > received an indication of no interest from powerdns, but i'm hoping bert > will relent later on. > > dnstap is completely open source, with a BSD-style license (Apache 2.0). > it is sponsored by farsight because we need a uniform DNS telemetry > format for our business purposes. we are giving it away in order to make > the world better, primarily for our own products and customers, but by > necessary extension, better for everybody including our competitors. > > vixie > > _______________________________________________ > dns-operations mailing list > [email protected] > https://lists.dns-oarc.net/mailman/listinfo/dns-operations > dns-jobs mailing list > https://lists.dns-oarc.net/mailman/listinfo/dns-jobs -- ~ Andrew "lathama" Latham [email protected] http://lathama.net ~ _______________________________________________ dns-operations mailing list [email protected] https://lists.dns-oarc.net/mailman/listinfo/dns-operations dns-jobs mailing list https://lists.dns-oarc.net/mailman/listinfo/dns-jobs
