SV: APNIC dns glitch ?
APNIC made an announcement on an operator list this morning that is probably relevant to your issue: +++include Some services provided by APNIC were unavailable this morning due to a disruption to our international connectivity. This occurred between 07:00 Australian Eastern Standard Time (AEST) and approximately 11:20 AEST. Major services affected were: - www.apnic.net http://www.apnic.net/ - MyAPNIC - email - Whois - Reverse DNS (partial) Services provided by ns3.apnic.net http://ns3.apnic.net/ and sec3.apnic.net http://sec3.apnic.net/ remained resolvable. Unfortunately, we were not able to announce this situation when it occurred due to the loss of connectivity. +++end -- Martin Hannigan [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Verne Global http://www.verneglobal.com http://www.verneglobal.com/ Keflavik, Iceland Fra: Danny Thomas [mailto:[EMAIL PROTECTED] Sendt: ma 23-jun-08 22:54 Til: [EMAIL PROTECTED] Emne: APNIC dns glitch ? I thought I'd sent this a couple of hours ago APNIC are aware of the problem and things have partially recovered though the arin and ripe name-servers still SERVFAIL the second run of our delegation-checking script this morning started complaining about our 203.in-addr zones and it seems there is an issue with apnic.net the delegation shows 4 entries spread across 3 domains which is good, albeit all are under the same registry. Sometimes cumin.apnic.net and innie.apnic.net. are not reachable, or give a REFUSED response, or give a response with no A records nor any additional section. Unfortunately both tinnie.arin.net and ns-sec.ripe.net return SERVFAIL, as if they had not been able to perform a zone transfer for a while (assuming AXFR is the replication mechanism). I don't have ipv6 connectivity, but that's not likely to help. I don't think this will significantly impact reverse dns lookups as I think the dns is spread across other RIR's seems there was a different type of issue in May http://www.bauani.org/thinkings/2008/05/issue-with-apnic-dns-nameservers.html Danny Thomas # dig @I.GTLD-SERVERS.net apnic.net +norec ; DiG 9.4.2 @I.GTLD-SERVERS.net apnic.net +norec ; (1 server found) ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 5460 ;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 5 ;; QUESTION SECTION: ;apnic.net. IN A ;; AUTHORITY SECTION: apnic.net. 172800 IN NS cumin.apnic.net. apnic.net. 172800 IN NS ns-sec.ripe.net. apnic.net. 172800 IN NS tinnie.apnic.net. apnic.net. 172800 IN NS tinnie.arin.net. ;; ADDITIONAL SECTION: cumin.apnic.net.172800 IN A 202.12.29.59 ns-sec.ripe.net.172800 IN A 193.0.0.196 ns-sec.ripe.net.172800 IN 2001:610:240:0:53::4 tinnie.apnic.net. 172800 IN A 202.12.29.60 tinnie.arin.net.172800 IN A 168.143.101.18 dig @202.12.29.59 apnic.net any +norec ; DiG 9.4.2 @202.12.29.59 apnic.net any +norec ; (1 server found) ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: REFUSED, id: 33930 ;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 # dig @202.12.29.59 apnic.net any ; DiG 9.4.2 @202.12.29.59 apnic.net any ; (1 server found) ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 40744 ;; flags: qr aa rd; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;apnic.net. IN ANY ;; ANSWER SECTION: apnic.net. 3600IN SOA cumin.apnic.net. dns-admin.apnic.net. 2008062101 3600 1800 604800 3600 apnic.net. 3600IN NS cumin.apnic.net. apnic.net. 3600IN NS ns-sec.ripe.net. apnic.net. 3600IN NS tinnie.arin.net. apnic.net. 3600IN NS tinnie.apnic.net. apnic.net. 3600IN MX 10 kombu.apnic.net. apnic.net. 3600IN MX 25 karashi.apnic.net. apnic.net. 3600IN MX 35 fennel.apnic.net. ;; Query time: 3 msec ;; SERVER: 202.12.29.59#53(202.12.29.59) ;; WHEN: Tue Jun 24 10:35:13 2008 ;; MSG SIZE rcvd: 235 dig @193.0.0.196 apnic.net any +norec ; DiG 9.4.2 @193.0.0.196 apnic.net any +norec ; (1 server found) ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 37668 ;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;apnic.net. IN ANY # dig @168.143.101.18 apnic.net ns ; DiG 9.4.2
Re: Cloud service [was: RE: EC2 and GAE means end of ip address reputation industry? (Re: Intrustion attempts from Amazon EC2 IPs)]
Source IP blocking makes up a large portion of today's spam arrest approach, so we shouldn't discount the CPU benefits of that approach too quickly. I'm not sure where today's technology is in regards for caching the first 1 to 10kB of a sessiononce enough information is garnered to block, issue TCP RSETs. If it's good, free the contents of the cache. What's your interest in mopping up spam in the middle of the network? Usually spam is viewed as a leaf-node problem (much to the chagrin of receivers, actually). Regards, Ken -- Ken Simpson CEO MailChannels - Reliable Email Delivery http://mailchannels.com 604 685 7488 tel
Re: Techniques for passive traffic capturing
On Mon, Jun 23, 2008 at 10:00:06PM -0500, Kevin Kadow wrote: We started out with SPAN ports, then moved on to Netoptics taps. Lately we've been using a combination of Cisco Netflow (from remote routers), and native Argus flows (from local taps) where we need more details. Flows are useful to answer What happened X minutes/hours/days ago?, and where you do not need/want to capture full packet bodies (though with Argus you can choose whether to include payload data). http://qosient.com/argus/ Cool - good to know that the Netoptics gear is good. Seems like there's a few resounding approvals of them. Netflow would be lovely to export from our border routers. Unfortunately, we are somewhat married to the 6500 platform which has absolutely awful netflow support. Very small TCAM, export is CPU expensive, and sampling makes both problems worse. So a mirrored copy of the transit link is being sent to a pmacct box for flow generation. -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Re: easy way to scan for issues with path mtu discovery?
Dear Patrick, Does anyone know of an easy way to scan for issues with path mtu discovery along a hop path? E.g. if you think someone is ICMP black-holing along a route, or even on the endpoint host, could you use some obscure nmap flag to find out for sure, and also to identify the offending hop/router/host? What tool would you use to test for this, and how would you do such a test? Is there any probing tool that does checks like this automatically? Seems to me this happens often enough that someone has probably already figured it out, so I am trying not to reinvent the wheel. All I can think of would be to handcraft packets of steadily increasing sizes and look for replies from each hop on the route (which would be laborious at best). Google has not been kind to my researches so far. If you have a cisco router: ping Protocol [ip]: Target IP address: x.x.x.x Repeat count [5]: Datagram size [100]: 1500 Timeout in seconds [2]: 1 Extended commands [n]: y Source address or interface: Type of service [0]: Set DF bit in IP header? [no]: yes Validate reply data? [no]: yes Data pattern [0xABCD]: Loose, Strict, Record, Timestamp, Verbose[none]: Sweep range of sizes [n]: y Sweep min size [36]: Sweep max size [18024]: 1500 Sweep interval [1]: Kind regards, Ingo Flaschberger
Re: Techniques for passive traffic capturing
On Tue, Jun 24, 2008 at 01:19:03PM +1200, Nathan Ward wrote: I see little point in aggregating tapped traffic, unless you have only a small amount of it and you're doing it to save cost on monitoring network interfaces - but is that saved cost still a saving when you factor in the cost of the extra 3750s in the middle? I'd guess no. Thanks for all the info Nathan - lots of good leads in your email. Let me include some more information. The problem is finding a way to multiplex that traffic from the optical tap to multiple things that want to peek at it. The remote-span trick solves that, as well as integrating media converters. 3750 is nice since you can stack em up and mix/match the SFP and copper ports. For example - we have an FCP box from Internap. It wants to see mirrored traffic so it can watch for TCP setup problems and try to find blackholes. It takes 10G feeds of aggregated transit links. Then, we want to do some passive IDS analysis. But snort can only really only handle 600-800Mbps before it starts saturating CPU (not multithreaded...) - so one collector per gigE transit seems logical. We'd like to generate flow data out of our forwarding plane since we use 6500s to pull in border transit links. The Netflow on those boxes is terrible. pmacct does a much better job, but it needs to see all the traffic out of band. Note that for a single GE link, you'd need 2GE of remote span backhaul (one GE in each direction). We're mostly a content network, very few eyeballs. Our ingress traffic is negligable compared to egress, which makes the problem easier. Matrix switches aren't useful for your case, as you're talking about monitoring for trending etc. I think. Matrix switches are good when you have lots of links, and want to be able to switch between them. Is the cost of matrix switch ports worth the saving in GE interfaces on PCs? I guess what made me look at them is their ability to multiplex the stream of data. Take it from an optical tap, spit the same data out of multiple ports. The remote-span trick seems to do the same thing, so I'm wondering where the gotcha is. If there's an advantage to using something like the Matrix switches, I'd love to know that now. The above is based on the assumption you're using PCs for monitoring, the economics of aggregating tap traffic may make more sense if you're using some fancy monitoring platform. Yea - the fact that we have both makes the aggregation method look good. The FCP takes 10G aggregated feeds. The PCs will want single gig views of the transit links. If you find that you need lots of GE interfaces per PC or something, and are saturating the PCI bus, look at DAG cards from Endace. They're designed for passive monitoring, and will send you only headers and do BPF in hardware. I looked at these for a similar project, but didn't bother as it was cheaper to buy more PC chassis' and commodity GE cards. They can do 10GE monitoring, so if you need several 10GE's per chassis I'd recommend these. Ah the Endace gear looks really interesting. Thanks for the pointer! -- Ross Vandegrift [EMAIL PROTECTED] The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell. --St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
Re: easy way to scan for issues with path mtu discovery?
Darden, Patrick S. wrote: Hi all, Does anyone know of an easy way to scan for issues with path mtu discovery along a hop path? E.g. if you think someone is ICMP black-holing along a route, or even on the endpoint host, could you use some obscure nmap flag to find out for sure, and also to identify the offending hop/router/host? What tool would you use to test for this, and how would you do such a test? Is there any probing tool that does checks like this automatically? Seems to me this happens often enough that someone has probably already figured it out, so I am trying not to reinvent the wheel. All I can think of would be to handcraft packets of steadily increasing sizes and look for replies from each hop on the route (which would be laborious at best). Google has not been kind to my researches so far. Take a look at tracepath. http://www.google.com/search?hl=enq=tracepathbtnG=Google+Search I haven't done much of anything with it but it may be of use to you. Justin
RE: Cloud service [was: RE: EC2 and GAE means end of ip address reputation industry? (Re: Intrustion attempts from Amazon EC2 IPs)]
For the reason you stated, much to the chagrin of receivers. Easier to sell a service to customers downstream if it's being done in the network, without MX changing. Frank -Original Message- From: Ken Simpson [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 24, 2008 8:38 AM To: [EMAIL PROTECTED] Cc: 'Christopher Morrow'; nanog@nanog.org Subject: Re: Cloud service [was: RE: EC2 and GAE means end of ip address reputation industry? (Re: Intrustion attempts from Amazon EC2 IPs)] Source IP blocking makes up a large portion of today's spam arrest approach, so we shouldn't discount the CPU benefits of that approach too quickly. I'm not sure where today's technology is in regards for caching the first 1 to 10kB of a sessiononce enough information is garnered to block, issue TCP RSETs. If it's good, free the contents of the cache. What's your interest in mopping up spam in the middle of the network? Usually spam is viewed as a leaf-node problem (much to the chagrin of receivers, actually). Regards, Ken -- Ken Simpson CEO MailChannels - Reliable Email Delivery http://mailchannels.com 604 685 7488 tel
RE: easy way to scan for issues with path mtu discovery?
Look at mturoute: http://www.elifulkerson.com/projects/mturoute.php Frank -Original Message- From: Darden, Patrick S. [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 24, 2008 9:28 AM To: nanog@nanog.org Subject: easy way to scan for issues with path mtu discovery? Hi all, Does anyone know of an easy way to scan for issues with path mtu discovery along a hop path? E.g. if you think someone is ICMP black-holing along a route, or even on the endpoint host, could you use some obscure nmap flag to find out for sure, and also to identify the offending hop/router/host? What tool would you use to test for this, and how would you do such a test? Is there any probing tool that does checks like this automatically? Seems to me this happens often enough that someone has probably already figured it out, so I am trying not to reinvent the wheel. All I can think of would be to handcraft packets of steadily increasing sizes and look for replies from each hop on the route (which would be laborious at best). Google has not been kind to my researches so far. I appreciate any help! --Patrick Darden
Comcast
Anyone from Comcast on-list? I'm getting hit with phishing emails that have a link to a Wachovia look alike page that's hosted on a comcast HSI account in South Bend Indiana.(At least thats what the SWIP says). Thanks! Ed
XO contact
Can someone from XO who handles this neighbor 65.46.253.157 help me out with a BGP session going down? This is the second time within a week where a misconfiguration of an ACL on XO end is bringing down my BGP session with you and its frustrating to go through the normal tech support chain. Zaid
Re: Level3 IPv6 availability?
Level 3 provides best effort IPv6 support with no SLA to current Internet customers. As mentioned IPv6 is currently being provided via tunnels to the customer's existing router. There is a simple service agreement addendum and form to fill out for relevant config bits. Sorry you get such a response from people that should know. *sigh* regards -Craig (Level 3 architecture) * Jay Hennigan was thought to have said: Is anyone at Level3 who is familiar with IPv6, or anyone who is a Level3 IPv6 customer lurking here? We are a Level3 BGP customer and our contacts are giving us a deer-in-the-headlights stare when we want to bring up our /32, claiming that they don't do IPv6 at all. Not native, not tunneled, zip, nada. Yet, I see lots of AS3356 in the ipv6 routing tables, and there's this from three years ago...
Re: Level3 IPv6 availability?
Is anyone at Level3 who is familiar with IPv6, or anyone who is a Level3 IPv6 customer lurking here? We are a Level3 BGP customer and our contacts are giving us a deer-in-the-headlights stare when we want to bring up our /32, claiming that they don't do IPv6 at all. Not native, not tunneled, zip, nada. We've recently brought up IPv6 with Level3. It's done as an IPv6IP tunnel to their nearest IPv6 router. I may be able to dig out the form you need... See http://www.bogons.net/dl/level3_ipv6.doc brandon
Re: easy way to scan for issues with path mtu discovery?
On Tue, Jun 24, 2008 at 10:28:12AM -0400, Darden, Patrick S. wrote: Hi all, Does anyone know of an easy way to scan for issues with path mtu discovery along a hop path? E.g. if you think someone is ICMP black-holing along a route, or even on the endpoint host, could you use some obscure nmap flag to find out for sure, and also to identify the offending hop/router/host? What tool would you use to test for this, and how would you do such a test? Is there any probing tool that does checks like this automatically? Seems to me this happens often enough that someone has probably already figured it out, so I am trying not to reinvent the wheel. All I can think of would be to handcraft packets of steadily increasing sizes and look for replies from each hop on the route (which would be laborious at best). Google has not been kind to my researches so far. scamper is the best tool I've found: http://www.wand.net.nz/scamper/ Bill.
Re: EC2 and GAE means end of ip address reputation industry? (Re:
On Tue, 24 Jun 2008 00:03:20 -, Paul Vixie said: [EMAIL PROTECTED] writes: One could argue that the botnets for rent business model is in more widespread use than either EC2 or gridserver... I'm unclear whether that statement needs a smiley or not... i'd say that since EC2 won't be shut down when it's found out about, that you need a smiley. widespread use is too narrow a term. none of us expects white-hat e-commerce business to move into rented botnets, and Umm.. Paul? Take a reality check here - nobody actually *cares* where white-hat services come from. Which is bigger, the RBN or EC2's yearly revenues? rented botnets aren't all going to be in the same address space or ASN. Hey - it ain't much of a cloud if it's all in one ASN. :) pgp9V8HKVUMAN.pgp Description: PGP signature
