Re: It's the end of the world as we know it -- REM
On 4/30/2013 10:36 PM, Jimmy Hess wrote: On 4/30/13, Owen DeLong o...@delong.com wrote: With all due respect, this is a reference in section 8.3 to call out that the policies in section 4 regarding qualification of recipients are to be followed when determining eligibility for an 8.3 transfer. I don't read a reference to section 4 there. don't think it's a reasonable belief, that a network operator supplicating for transfer of IPv4 resources, would come to this conclusion -- there is no reason to select a specific section to apply because no section is mentioned, reading the policy on their own, and what you are seeing there -- may be a result of bias from your prior exposure to another interpretation of the language. Its also possible, that all of us who were reviewing the proposed transfer policy language read some rationale statement at one time or another, and just (incorrectly) assumed that the final language accomplished our intended effect. I don't think this issue should effect any network operators at this time, but nonetheless i'm concerned about the RIR policy having confusing, surprising, or hidden ramifications built into it, which are problematic and not previously considered. I was the original policy modification author. The first version I submitted said: Add a subsection to Section 8 (Transfers) of the NRPM: Justified Need for resources associated with a transfer shall be determined using the 4.2.4 ISP Additional Requests criteria applied as though the recipient has been a subscriber member of ARIN for at least one year (whether or not that is the case). Then In a followup email I pointed out: ... Section 8.3 has NO language exempting itself from the 3 month rule. That's what I hear on the list, but I looked it up, and it isn't there. That's how I ended up writing this proposal, after all. The only exemption is in 4.2.4.4. That exemption ONLY works if you are not getting an initial assignment through transfer (a likely scenario for new orgs post-runout) AND you are not a new member who only recently got their initial 3 month supply (where you'd be restricted to using transfer in 3-month increments for the first year in order to grow). AND there's other bugs in that 4.2.2.1.1 and 4.2.2.1.3 and 4.2.2.2 (at the very least) call out specific block sizes that might be *smaller* than the block you're trying to qualify under transfer. I then followed up to that with some specific scenarios... A. My hypothetical ISP provides service to a small town. I presently get two /24s of IPv4 space from my upstream provider and I'm using them at about 85%. ARIN has run completely out of addresses. A benefactor arrives and offers to transfer a /22 to me and pay for me to multihome. I attempt to use 4.2.2.2 (Initial Allocation to ISPs, Multihomed) for my justification. I need to demonstrate that I am efficiently using the two /24s. Done. I comply with 4.2.2.2.1 (SWIP). I attempt to comply with 4.2.2.2.2, but my growth shows that I won't really need more than a /23 for about 7 months. Transfer would be denied because 4.2.2.2.2 has a three month rule (as I claimed above). Benefactor takes his space elsewhere, and I lose out. B. My hypothetical ISP provides service to a single data center. I presently have a /20 that I was able to obtain from ARIN a few months ago, and I wasn't an ARIN subscriber member prior to this. I have the opportunity to acquire another ISP in town which has a /20 of its own, but which it isn't using very well because their growth plans failed after I opened up. I can demonstrate that my /20 and the second /20 from the acquisition would be filled within a year if I complete this transfer under section 8.2, but I'll only be able to fill out my existing /20 over the next three months. However, because I am under 4.2.4.3 (Subscriber Members Less Than One Year) my 8.2 transfer is denied, again because 4.2.4.3 has a three month rule (as I claimed above). on the flip side... C. My hypothetical ISP provides service to a small city that is served by only one transit provider, so I cannot multihome. It has been using a /20 from the upstream ISP and is efficiently using 16 /24s worth of space with reassignment documentation (satisfying 4.2.2.1.1 and 4.2.2.1.2). I provide detailed information showing that I can use a /20 within the next three months (satisfying 4.2.2.1.3). Now that I have met all the tests, I complete a section 8.3 transfer for a /14 worth of space (because I have loads of money I won in the lottery). As far as I can tell, there's nothing in the NRPM that blocks that transfer... because I've met all the tests in 4.2.2.1. Then, after a bunch of comment and feedback (particularly around how end-users shouldn't be judged by the ISP criteria during a transfer), I created the simplified language: Add to Section 8.3 ...they can justify under current ARIN policies showing how the addresses will be utilized
Could not send email to office 365
Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Tier1 blackholing policy?
Joel, Am 30.04.2013 18:00, schrieb joel jaeggli: On 4/30/13 8:23 AM, Thomas Schmid wrote: On 30.04.2013 17:07, Chris Boyd wrote: On Tue, 2013-04-30 at 10:59 -0400, ML wrote: 1) Do nothing - They're supposed deliver any and all bits (Disregarding a DoS or similiar situation which impedes said network) 2) Prefix filter - Don't be a party (at least in one direction) to the bad actors traffic. 3 - Deliver all packets unless I've signed up for an enhanced security offering? right - I see this really as something that should be decided at the edge of the internet (Tier2+) and not in the core. You seem to have odd ideas about what it means to be a settlement free provider. Most of their customers are not smaller internet service providers. I know what it means to be a customer of $LargeGlobalISPthatsellsTransittootherISPs since 1995 and I have *never* seen one of these guys blackholing single IPs on their own (and I'm not talking about RTB, botnet controllers that threaten to kill the internet etc.). Now since a few weeks we get regular complaints about this. So something has changed. The sensitive approach would really be to make this an opt-in service for their customers and not a default service without opt-out option. In times of CGN and hundrets or thousands of websites behind one IP, blocking addresses is not the right answer to the phishing problem. Thomas smime.p7s Description: S/MIME Kryptografische Unterschrift
Re: It's the end of the world as we know it -- REM
On May 1, 2013, at 3:51 AM, Matthew Kaufman matt...@matthew.at wrote: Now, the actual language that is in the NRPM says The recipient must demonstrate the need for up to a 24-month supply* of IP address resources under current ARIN policies and sign an RSA. ... if someone thinks that demonstrate the need...under current ARIN policies means not just demonstrate the need but also fall into compliance with every nuance of section 4 that might be applied if they were getting the addresses from ARIN (ex. 4.2.1.5 requiring a /20 minimum for ISPs) then I guess we need another policy modification. Correct, if one considers that a problem (particularly at runout) Is that really how ARIN staff is interpreting it? Also correct; as noted in prior email and per the staff assessments since this language was first introduced, demonstrate the need ... under current ARIN policies first requires assessment against current ARIN policies (only with the longer horizon) to determine if one is a valid recipient. And why is this discussion here and not on arin-ppml? Indeterminate; it appears to be follow-up to discussion about IPv4 runout in the region being potentially earlier than expected. arin-ppml is definitely a more appropriate list for such discussions. FYI, /John John Curran President and CEO ARIN
Re: Tier1 blackholing policy?
On May 1, 2013, at 4:40 PM, Thomas Schmid wrote: Now since a few weeks we get regular complaints about this. So something has changed. Yes, things have changed. There are reasons that some of the transit ISPs are performing this blocking. They aren't doing it for kicks. For example, there are non-insignificant numbers of servers/accounts which have been compromised and used to launch large-scale, high-impact DDoS attacks. The negative impact of allowing these servers to emit attack traffic far outweighs the inconvenience experienced by a few end-customers trying to access these servers (which are compromised, anyways, and therefore it isn't a good idea to try and access them in the first place). Suggest you ask the transit ISPs in question directly. You aren't likely to get an authoritative answer on a public email list. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Luck is the residue of opportunity and design. -- John Milton
Re: Mitigating DNS amplification attacks
On Tue, Apr 30, 2013 at 8:35 PM, Jared Mauch ja...@puck.nether.net wrote: Please provide advice and insights as well as directing customers to the openresolverproject.org website. We want to close these down, if you need an accurate list of IPs in your ASN, please email me and I can give you very accurate data. I think that a public list of open-resolvers is probably overdue, and the only way to get them fixed. It is trivial to scan the entire IPv4 address space for DNS servers that do no throttling even without the resources of a malicious botnet. Smurf was only fixed because, as there were fewer networks not running `no ip directed-broadcast,` the remaining amplification sources were flooded with huge amounts of malicious traffic. The public list of smurf amplifiers turned out to be the only way to really deal with it. I predict the same will be true with DNS. -- Jeff S Wheeler j...@inconcepts.biz Sr Network Operator / Innovative Network Concepts
Re: Mitigating DNS amplification attacks
On May 1, 2013, at 5:42 PM, Jeff Wheeler wrote: The public list of smurf amplifiers turned out to be the only way to really deal with it. It certainly helped; but the real solution was to get Cisco, et. al. to disable directed broadcasts by default. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Luck is the residue of opportunity and design. -- John Milton
Re: Tier1 blackholing policy?
On Tue, Apr 30, 2013 at 12:47:40PM -0400, Jared Mauch wrote: If the phishing attack is against an enterprise that is also an ISP, surely you can imagine a case where they might block traffic to prevent folks from being phished. This is not an effective anti-phishing tactic, any more than user education is an effective anti-phishing tactic. (Let me quote Marcus Ranum on the latter: if it was going to work, it would have worked by now. And let me observe: it's never worked; it's not working; it's never going to work.) i think it's great that someone is blocking folks from being infected with either malware or giving up their private details improperly. One person's malware is merely an interesting collection of inert bits to someone else, just as email virus has no operational meaning to anyone clueful enough to run a sensible mail client on a sensible operating system. Thus one undesirable effect of such blocking is that it denies access to researchers who are at nearly zero risk of negative consequences *and* who might be the very people in a position to understand the threat (phishing, malware, etc.) and figure out how to mitigate it. Another is that it presents a false sense of security to the ignorant, the lazy, and the careless. While in the short term that may seem benevolent and useful, I think in the long term it has a deleterious effect on security as a whole. And if we've arrived at a point in time where people are actually considering making routing decisions based on longstanding design and implementation defects in consumer operating systems and applications, then I think long term equates to right now. ---rsk
Re: Tier1 blackholing policy?
On May 1, 2013, at 7:44 AM, Rich Kulawiec r...@gsp.org wrote: On Tue, Apr 30, 2013 at 12:47:40PM -0400, Jared Mauch wrote: If the phishing attack is against an enterprise that is also an ISP, surely you can imagine a case where they might block traffic to prevent folks from being phished. This is not an effective anti-phishing tactic, any more than user education is an effective anti-phishing tactic. (Let me quote Marcus Ranum on the latter: if it was going to work, it would have worked by now. And let me observe: it's never worked; it's not working; it's never going to work.) We're talking about denying access to what is typically a compromised end-host which is in violation of an AUP. Speaking about my employer, we typically don't see something null0'ed for more than a few hours until we have confirmed the host is offline being repaired. I don't know about other networks practices which is what started the thread. i think it's great that someone is blocking folks from being infected with either malware or giving up their private details improperly. One person's malware is merely an interesting collection of inert bits to someone else, just as email virus has no operational meaning to anyone clueful enough to run a sensible mail client on a sensible operating system. Thus one undesirable effect of such blocking is that it denies access to researchers who are at nearly zero risk of negative consequences *and* who might be the very people in a position to understand the threat (phishing, malware, etc.) and figure out how to mitigate it. Another is that it presents a false sense of security to the ignorant, the lazy, and the careless. While in the short term that may seem benevolent and useful, I think in the long term it has a deleterious effect on security as a whole. And if we've arrived at a point in time where people are actually considering making routing decisions based on longstanding design and implementation defects in consumer operating systems and applications, then I think long term equates to right now. I think many people understand these risks and tradeoffs. We could stop mitigating DDoS attacks or responding to security complaints as well with this line of reasoning as it could be interfering with law-enforcement actions, or a researcher. Just because the house has been broken into, doesn't mean as the provider of the roads that we're going to let everyone visit it until the owner has a chance to secure it properly. I don't like that role, but it becomes necessary at times. What you are suggesting is a slippery slope to no mitigation of any badness which will lead to a lack of trust and confidence in the market. That to me is a plain and simple reason to do the right thing, even if it causes a problem for a few hours or a day or two. - Jared
Re: Tier1 blackholing policy?
On 05/01/2013 05:40 AM, Thomas Schmid wrote: Joel, Am 30.04.2013 18:00, schrieb joel jaeggli: On 4/30/13 8:23 AM, Thomas Schmid wrote: On 30.04.2013 17:07, Chris Boyd wrote: On Tue, 2013-04-30 at 10:59 -0400, ML wrote: 1) Do nothing - They're supposed deliver any and all bits (Disregarding a DoS or similiar situation which impedes said network) 2) Prefix filter - Don't be a party (at least in one direction) to the bad actors traffic. 3 - Deliver all packets unless I've signed up for an enhanced security offering? right - I see this really as something that should be decided at the edge of the internet (Tier2+) and not in the core. You seem to have odd ideas about what it means to be a settlement free provider. Most of their customers are not smaller internet service providers. I know what it means to be a customer of $LargeGlobalISPthatsellsTransittootherISPs since 1995 and I have *never* seen one of these guys blackholing single IPs on their own (and I'm not talking about RTB, botnet controllers that threaten to kill the internet etc.). Now since a few weeks we get regular complaints about this. So something has changed. The sensitive approach would really be to make this an opt-in service for their customers and not a default service without opt-out option. In times of CGN and hundrets or thousands of websites behind one IP, blocking addresses is not the right answer to the phishing problem. ... or perhaps on an internet where many network owners block / police / throttle packets by source or destination, implementing CGN or stacking thousands of websites behind one IP address are poor solutions to the connectivity problem. My only issue is the lack of information provided when blocks go into place. I would love to see networks provide information publicly that shows what is being blocked along with a description of why. A history that extends for a few days would be a bonus. -DMM
Re: Andros Island Connectivity?
On 2013-05-01, at 01:23, joseph.sny...@gmail.com wrote: Doesn't cable Bahamas sell in andros BaTelCo has five retail stores on Andros too, which suggests they offer some kind of service there (whether wireline or wireless). For a list whose subscribers have generally more experience on the ground in the Caribbean, perhaps try http://www.caribnog.org/mailing-list/. Joe
Re: Could not send email to office 365
The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Mitigating DNS amplification attacks
Well, I was going more for a public list of ISP that refuse to BCP38 their networks. But that's just me =D On point: (If your corporation is massive enough) Basically: . Mirror DST Port 53; . Write some software to stats who's spamming the same DST IP with the same query; . Dynamic ACL them; then . Give a talk to your customers =D - Alain Hebertaheb...@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.netFax: 514-990-9443 On 05/01/13 06:42, Jeff Wheeler wrote: On Tue, Apr 30, 2013 at 8:35 PM, Jared Mauch ja...@puck.nether.net wrote: Please provide advice and insights as well as directing customers to the openresolverproject.org website. We want to close these down, if you need an accurate list of IPs in your ASN, please email me and I can give you very accurate data. I think that a public list of open-resolvers is probably overdue, and the only way to get them fixed. It is trivial to scan the entire IPv4 address space for DNS servers that do no throttling even without the resources of a malicious botnet. Smurf was only fixed because, as there were fewer networks not running `no ip directed-broadcast,` the remaining amplification sources were flooded with huge amounts of malicious traffic. The public list of smurf amplifiers turned out to be the only way to really deal with it. I predict the same will be true with DNS.
Re: Andros Island Connectivity?
On Wed, May 01, 2013 at 09:20:59AM -0400, Joe Abley wrote: On 2013-05-01, at 01:23, joseph.sny...@gmail.com wrote: Doesn't cable Bahamas sell in andros BaTelCo has five retail stores on Andros too, which suggests they offer some kind of service there (whether wireline or wireless). its been a long time since i've done much in the caribbean, but i'd be quite surprised if the bahamas didn't have some decent amount of terrestrial connectivity by this time. back in the day when Cable and Wireless controlled everything, it was a pretty costly environment to work in. don't know if that's still the case. -- Jim Mercer Reptilian Research j...@reptiles.org+1 416 410-5633 He who dies with the most toys is nonetheless dead
RE: Could not send email to office 365
I am also having the some issues going on 3 weeks now. I cannot access my e-mail via Outlook and my MX records keep changing. It is nuts support has been unable to help. From: JoeSox Sent: Wednesday, May 01, 2013 9:24 AM To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Google Public DNS Problems?
Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Blair
Re: Could not send email to office 365
Just another day for them.. Office 365 has been broken for us for a long time. I've been thinking about a rackspace hosted exchange instead.. Have you guys looked into alternatives? Sent from my T-Mobile 4G LTE Device Original message From: JoeSox joe...@gmail.com Date: 05/01/2013 6:27 AM (GMT-08:00) To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Google Public DNS Problems?
Works fine from here, Philadelphia, PA .edu and FIOS networks Cheers, Harry On 05/01/2013 12:09 PM, Blair Trosper wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Blair
Re: Could not send email to office 365
Ryan, Is your Office 365 account also in an upgrade status? If not, have you completed the upgrade? -- Thanks, Joe On Wed, May 1, 2013 at 8:42 AM, Ryan Finnesey r...@finnesey.com wrote: I am also having the some issues going on 3 weeks now. I cannot access my e-mail via Outlook and my MX records keep changing. It is nuts support has been unable to help.
Re: Google Public DNS Problems?
On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.net visualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
Re: Google Public DNS Problems?
That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL On Wed, May 1, 2013 at 9:34 AM, Joe Abley jab...@hopcount.ca wrote: On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.net visualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
Re: Google Public DNS Problems?
No issues on Comcast cable in the bay area, either Comcast business or Comcast home. Scott $ nslookup gmail.com 8.8.4.4 Server: 8.8.4.4 Address:8.8.4.4#53 Non-authoritative answer: Name: gmail.com Address: 74.125.239.149 Name: gmail.com Address: 74.125.239.150 On Wed, May 1, 2013 at 9:09 AM, Blair Trosper blair.tros...@gmail.comwrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Blair
Re: Google Public DNS Problems?
On Wed, May 1, 2013 at 9:38 AM, Blair Trosper blair.tros...@gmail.com wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL If you set the CD (checking disabled) in the request, a response that would normally be SERVFAIL due to DNSSEC validation failure will return with the non-authenticated answer. With dig the flag to add is +cd. I don't know if there's an equivalent for nslookup. For example: dig +cd @8.8.8.8 google.com Casey
Re: Google Public DNS Problems?
Goes all the way up to the A root server before failing spectacularly. Europa:~ blair$ dig +cd @8.8.8.8 google.com A ; DiG 9.8.3-P1 +cd @8.8.8.8 google.com A ; (1 server found) ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 47332 ;; flags: qr rd ra cd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;google.com. IN A ;; AUTHORITY SECTION: . 467 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2013050100 1800 900 604800 86400 ;; Query time: 46 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Wed May 1 10:05:46 2013 ;; MSG SIZE rcvd: 104 On Wed, May 1, 2013 at 9:58 AM, Casey Deccio ca...@deccio.net wrote: On Wed, May 1, 2013 at 9:38 AM, Blair Trosper blair.tros...@gmail.com wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL If you set the CD (checking disabled) in the request, a response that would normally be SERVFAIL due to DNSSEC validation failure will return with the non-authenticated answer. With dig the flag to add is +cd. I don't know if there's an equivalent for nslookup. For example: dig +cd @8.8.8.8 google.com Casey
Re: Google Public DNS Problems?
8.8.4.4 is now replying SERVFAIL whereas 8.8.8.8 is suddenly working fine again... On Wed, May 1, 2013 at 10:07 AM, Blair Trosper blair.tros...@gmail.comwrote: Goes all the way up to the A root server before failing spectacularly. Europa:~ blair$ dig +cd @8.8.8.8 google.com A ; DiG 9.8.3-P1 +cd @8.8.8.8 google.com A ; (1 server found) ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 47332 ;; flags: qr rd ra cd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;google.com. IN A ;; AUTHORITY SECTION: . 467 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2013050100 1800 900 604800 86400 ;; Query time: 46 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Wed May 1 10:05:46 2013 ;; MSG SIZE rcvd: 104 On Wed, May 1, 2013 at 9:58 AM, Casey Deccio ca...@deccio.net wrote: On Wed, May 1, 2013 at 9:38 AM, Blair Trosper blair.tros...@gmail.com wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL If you set the CD (checking disabled) in the request, a response that would normally be SERVFAIL due to DNSSEC validation failure will return with the non-authenticated answer. With dig the flag to add is +cd. I don't know if there's an equivalent for nslookup. For example: dig +cd @8.8.8.8 google.com Casey
Re: Google Public DNS Problems?
Your IPs may have been rate limited... Andy Andrew Fried andrew.fr...@gmail.com On 5/1/13 12:38 PM, Blair Trosper wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL On Wed, May 1, 2013 at 9:34 AM, Joe Abley jab...@hopcount.ca wrote: On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.net visualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
RE: Could not send email to office 365
After our upgrade, we started to see the body of received PLAIN TEXT emails truncated at less than 256 bytes, which frequently truncated emails in the middle of a word, the fix was a setting change. Since being standardized in 1982 with RFC 822, you would think PLAIN TEXT emails would just work out of the box. Tony -Original Message- From: Ryan Finnesey [mailto:r...@finnesey.com] Sent: Wednesday, May 01, 2013 11:42 AM To: JoeSox; nanog@nanog.org Subject: RE: Could not send email to office 365 I am also having the some issues going on 3 weeks now. I cannot access my e-mail via Outlook and my MX records keep changing. It is nuts support has been unable to help. From: JoeSox Sent: Wednesday, May 01, 2013 9:24 AM To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Google Public DNS Problems?
Blair Trosper blair.tros...@gmail.com wrote: Goes all the way up to the A root server before failing spectacularly. That is an extremely weird response. Are you sure your queries are not being intercepted by a middlebox? What happens if you use dig +vc ? Do you get a similar round-trip time when pinging 8.8.8.8 to the one reported by dig? Europa:~ blair$ dig +cd @8.8.8.8 google.com A ; DiG 9.8.3-P1 +cd @8.8.8.8 google.com A ; (1 server found) ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 47332 ;; flags: qr rd ra cd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;google.com. IN A ;; AUTHORITY SECTION: . 467 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2013050100 1800 900 604800 86400 ;; Query time: 46 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Wed May 1 10:05:46 2013 ;; MSG SIZE rcvd: 104 Tony. -- f.anthony.n.finch d...@dotat.at http://dotat.at/ Forties, Cromarty: East, veering southeast, 4 or 5, occasionally 6 at first. Rough, becoming slight or moderate. Showers, rain at first. Moderate or good, occasionally poor at first.
Re: It's the end of the world as we know it -- REM
I believe Jimmy's confusion results primarily as follows: From NRPM 8.3: 8.3. Transfers between Specified Recipients within the ARIN Region In addition to transfers under section 8.2, IPv4 numbers resources and ASNs may be transferred according to the following conditions. Conditions on source of the transfer: The source entity must be the current registered holder of the IPv4 address resources, and not be involved in any dispute as to the status of those resources. The source entity will be ineligible to receive any further IPv4 address allocations or assignments from ARIN for a period of 12 months after a transfer approval, or until the exhaustion of ARIN's IPv4 space, whichever occurs first. The source entity must not have received a transfer, allocation, or assignment of IPv4 number resources from ARIN for the 12 months prior to the approval of a transfer request. This restriction does not include MA transfers. The minimum transfer size is a /24 Conditions on recipient of the transfer: The recipient must demonstrate the need for up to a 24-month supply of IP address resources under current ARIN policies and sign an RSA. The resources transferred will be subject to current ARIN policies. Note that the minimum /24 restriction he so often references is a restriction on the minimum that a transferor can provide. It does not mean that there are not more specific restrictions on recipients. It is quite clear in the policy, as quoted above, that the only reference to a /24 is in the section controlling the source of the transfer. The only exception to the rest of ARIN policy for the recipient is that it allows a 24 month supply rather than the current 3 month (ISP) or 12 month (End-User) limitation when obtaining resources from the free pool. Owen
Re: Google Public DNS Problems?
Once upon a time, Joe Abley jab...@hopcount.ca said: Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Does Google (and OpenDNS for that matter) support any special queries that identify the host/cluster/etc.? Something like BIND's CHAOS HOSTNAME.BIND (which also works for Unbound and some other servers), or UltraDNS's WHOAREYOU.ULTRADNS.NET. -- Chris Adams cmad...@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Re: Google Public DNS Problems?
On May 1, 2013, at 1:39 PM, Tony Finch d...@dotat.at wrote: Blair Trosper blair.tros...@gmail.com wrote: Goes all the way up to the A root server before failing spectacularly. That is an extremely weird response. Are you sure your queries are not being intercepted by a middlebox? What happens if you use dig +vc ? Do you get a similar round-trip time when pinging 8.8.8.8 to the one reported by dig? Some places like Wayport/attwifi intercept all udp/53 traffic and direct it to their local server. You won't notice this with a ping, but you will see it in the 0 or 1ms reply :) - Jared
Re: Google Public DNS Problems?
Traceroute is getting the right place and 8.8.8.8 is working, though :) On Wed, May 1, 2013 at 11:23 AM, Jared Mauch ja...@puck.nether.net wrote: On May 1, 2013, at 1:39 PM, Tony Finch d...@dotat.at wrote: Blair Trosper blair.tros...@gmail.com wrote: Goes all the way up to the A root server before failing spectacularly. That is an extremely weird response. Are you sure your queries are not being intercepted by a middlebox? What happens if you use dig +vc ? Do you get a similar round-trip time when pinging 8.8.8.8 to the one reported by dig? Some places like Wayport/attwifi intercept all udp/53 traffic and direct it to their local server. You won't notice this with a ping, but you will see it in the 0 or 1ms reply :) - Jared
Re: Google Public DNS Problems?
On 5/1/13 2:23 PM, Jared Mauch ja...@puck.nether.net wrote: On May 1, 2013, at 1:39 PM, Tony Finch d...@dotat.at wrote: Blair Trosper blair.tros...@gmail.com wrote: Goes all the way up to the A root server before failing spectacularly. That is an extremely weird response. Are you sure your queries are not being intercepted by a middlebox? What happens if you use dig +vc ? Do you get a similar round-trip time when pinging 8.8.8.8 to the one reported by dig? Some places like Wayport/attwifi intercept all udp/53 traffic and direct it to their local server. You won't notice this with a ping, but you will see it in the 0 or 1ms reply :) FWIW, no DNS intercepts happen on the Comcast networkŠ Jason (Comcast)
Data Center Installations
Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren
Re: Data Center Installations
Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren
Re: Data Center Installations
On the cheap Lowes/Home Depot are awesome, and they're everywhere. On 05/01/2013 03:23 PM, Warren Bailey wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren
Re: Data Center Installations
I'm a huge fan of netig... they bend over backwards for you. -chris Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren
RE: Data Center Installations
-Original Message- From: Warren Bailey [mailto:wbai...@satelliteintelligencegroup.com] Sent: Wednesday, May 01, 2013 2:24 PM To: nanog@nanog.org Subject: Data Center Installations Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren We've used both CSC and Graybar, more frequently CSC better deals in our case. For very nice affordable Cat 5e/6A patch cords iofast.com we've never purchased a patch from anywhere else since we found them.
Re: Data Center Installations
Seconded Graybar. If necessary, in the absence of Graybar or for tiny stuff, a Frys or Home Depot or Lowes. On Wed, May 1, 2013 at 12:32 PM, Joe Hamelin j...@nethead.com wrote: Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren -- -george william herbert george.herb...@gmail.com
Re: Data Center Installations
Tessco or Hutton. Lil pricey unless you buy in bulk. Good fit for you though since they carry RF and antenna stuff too. -mike Sent from my iPhone On May 1, 2013, at 12:45, Otis L. Surratt, Jr. o...@ocosa.com wrote: -Original Message- From: Warren Bailey [mailto:wbai...@satelliteintelligencegroup.com] Sent: Wednesday, May 01, 2013 2:24 PM To: nanog@nanog.org Subject: Data Center Installations Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren We've used both CSC and Graybar, more frequently CSC better deals in our case. For very nice affordable Cat 5e/6A patch cords iofast.com we've never purchased a patch from anywhere else since we found them.
Re: Comcast Launches IPv6 for Business Customers
On 04/29/13 15:38, Brzozowski, John wrote: FYI for folks that are interested: http://corporate.comcast.com/comcast-voices/comcast-launches-ipv6-for-business-customers Great news! Strangely, I (a Comcast Business customer at home) have noticed RAs coming across my wire for several months now. I use a Comcast-supplied Arris cable modem, with no extra routing functionality. About 4-5 months ago, I thought I'd see if there was a Comcast DHCPv6 server on that wire, and I sent it ia-na and ia-pd requests--and got responses back, and I have had a DHCPv6 client running ever since. (See the headers of this message for more info.) So from my perspective, IPv6 on Comcast Business Internet has been just working for at least 4 months. I have signed up for the pilot, as a /56 would be really useful so that I can run v6 on all of my networks (wireless, wired, etc.). Any idea how it will work with a simple Arris cable modem and a reclaimed Mac Mini running *BSD as the router, given that I have already proven that it works? :) thanks and keep up the good work... michael
Re: Mitigating DNS amplification attacks
On 04/30/2013 05:28 PM, Thomas St-Pierre wrote: The large majority of the servers being used in the attacks are not open resolvers. Just DNS servers that are authoritative for a few domains, and the default config of the dns application does referrals to root for anything else. It sounds like you're already aware that this is the default behavior for an authoritative-only server, and while the referral to the roots is a largeish response and has been used for amplification attacks, it's also rather difficult to mitigate against. A BIND server can be configured to not do that, but contacting each of your customers about it might not have a good ROI. See https://www.dns-oarc.net/oarc/articles/upward-referrals-considered-harmful for more information. Meanwhile, thank you very much for being proactive in this regard. Would that more SPs were as net.responsible as you. :) Doug
Re: Google Public DNS Problems?
It is very courteous to reply a SERVFAIL for requests being rate limited. On Wed, May 1, 2013 at 1:17 PM, Andrew Fried andrew.fr...@gmail.com wrote: Your IPs may have been rate limited... Andy Andrew Fried andrew.fr...@gmail.com On 5/1/13 12:38 PM, Blair Trosper wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL On Wed, May 1, 2013 at 9:34 AM, Joe Abley jab...@hopcount.ca wrote: On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.net visualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
Re: Data Center Installations
On Wednesday 01 May 2013 12:23, Warren Bailey wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. Would largely depend on your definition of datacenter (customer or operator) and what your corporate policies are. Speaking as the small ISP datacenter maintainer: Tie wraps, misc hardware - HomeDepot/Lowes Patch cords/bulk cable - Monoprice Patch panels/punchdown/custom fiber - Gruber Electrical (equip)/fiber/wall enclosures - Graybar/Tesco Electrical (power) - Local electrical shops/Home depot Adrian
RE: Could not send email to office 365
We have fixed the problem. I had to complete a clear install of Outlook and remove the credentials that where on the computer in the control panel under credential manager. This may also fix/help with your issue. Cheers Ryan -Original Message- From: Tony Patti [mailto:t...@swalter.com] Sent: Wednesday, May 1, 2013 1:19 PM To: Ryan Finnesey; 'JoeSox'; nanog@nanog.org Subject: RE: Could not send email to office 365 After our upgrade, we started to see the body of received PLAIN TEXT emails truncated at less than 256 bytes, which frequently truncated emails in the middle of a word, the fix was a setting change. Since being standardized in 1982 with RFC 822, you would think PLAIN TEXT emails would just work out of the box. Tony -Original Message- From: Ryan Finnesey [mailto:r...@finnesey.com] Sent: Wednesday, May 01, 2013 11:42 AM To: JoeSox; nanog@nanog.org Subject: RE: Could not send email to office 365 I am also having the some issues going on 3 weeks now. I cannot access my e-mail via Outlook and my MX records keep changing. It is nuts support has been unable to help. From: JoeSox Sent: Wednesday, May 01, 2013 9:24 AM To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Tier1 blackholing policy?
Le 01/05/2013 14:46, David Miller a écrit : On 05/01/2013 05:40 AM, Thomas Schmid wrote: Joel, Am 30.04.2013 18:00, schrieb joel jaeggli: On 4/30/13 8:23 AM, Thomas Schmid wrote: On 30.04.2013 17:07, Chris Boyd wrote: On Tue, 2013-04-30 at 10:59 -0400, ML wrote: 1) Do nothing - They're supposed deliver any and all bits (Disregarding a DoS or similiar situation which impedes said network) 2) Prefix filter - Don't be a party (at least in one direction) to the bad actors traffic. 3 - Deliver all packets unless I've signed up for an enhanced security offering? right - I see this really as something that should be decided at the edge of the internet (Tier2+) and not in the core. You seem to have odd ideas about what it means to be a settlement free provider. Most of their customers are not smaller internet service providers. I know what it means to be a customer of $LargeGlobalISPthatsellsTransittootherISPs since 1995 and I have *never* seen one of these guys blackholing single IPs on their own (and I'm not talking about RTB, botnet controllers that threaten to kill the internet etc.). Now since a few weeks we get regular complaints about this. So something has changed. The sensitive approach would really be to make this an opt-in service for their customers and not a default service without opt-out option. In times of CGN and hundrets or thousands of websites behind one IP, blocking addresses is not the right answer to the phishing problem. ... or perhaps on an internet where many network owners block / police / throttle packets by source or destination, implementing CGN or stacking thousands of websites behind one IP address are poor solutions to the connectivity problem. My only issue is the lack of information provided when blocks go into place. I would love to see networks provide information publicly that shows what is being blocked along with a description of why. A history that extends for a few days would be a bonus. I agree with that. While some blocking and policing may be judged good thing there is a well-known potential for other kinds of policing... Cheers, mh -DMM
Per Site QOS policy with Cisco IOS-XE
I have a question for the QOS gurus out there. We are having some problems with packet loss for our smaller MPLS locations. This packet loss is due to the large speed differential on our Hub site(150mb/s) in comparison the the branch office locations(single T-1 to 4.5mb/s multilinks). This packet loss only seems to impact really bursty applications like our Web Proxy. I have been around and around with WindStream to give me some extra buffer or enable random early detection on the smaller interfaces in my MPLS network. So far they are unwilling to do a custom policy and none of their standard policies have enough buffer to handle the bursts. They do FIFO tail drop in every queue, so I can’t even choose a policy that has WRED implemented. I am looking for a way to solve the problem on my side. I can create a shaper for the proxy and match off an access-list to the smaller sites, but I am either forced to do bandwidth reservations for each site, or have multiple sites share the same shaper. Here is an example of what I was playing around with: ip access-list extended ProxyT1Sites permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255 permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255 class-map match-any ProxyShaperT1 match access-group name ProxyT1Sites policy-map WindStream class VOICE priority percent 25 set dscp ef class AF41 bandwidth percent 40 set dscp af41 queue-limit 1024 packets class ProxyShaperT1 shape average 1536 bandwidth percent 1 set dscp af21 queue-limit 1024 packets class class-default fair-queue set dscp af21 queue-limit 1024 packets Another Idea I had was to create a bunch of shaper classes all feeding the same child policy for priority queuing and bandwidth reservations based on DSCP markings. I’m just not exactly sure that this is allowed or supported. I also would run out of bandwidth allocation on the policy if I use the true bandwidth number of 150mb/s. It is on a Gig port so I could just take the bandwidth statement off of the interface to give myself enough room for all of the shaper allocations. Something like this(I am omitting the access-list that matches the branch subnet and class map for brevity): policy-map PerSiteShaper class FtSmith shape average 1536 bandwidth 1536 service policy Scheduler class Dallas shape average 4500 bandwidth 4500 service policy Scheduler class NYC shape average 10 bandwidth 10 service policy Scheduler class-default service-policy Scheduler policy-map Scheduler class VOICE priority percent 25 set dscp ef class AF41 bandwidth percent 40 set dscp af41 queue-limit 1024 packets class class-default fair-queue set dscp af21 queue-limit 1024 packets Just looking for some ideas that do not involve building tunnels to our remote offices. Thanks in advance, *Wes Tribble*
Re: Google Public DNS Problems?
On Wed, May 1, 2013 at 4:14 PM, Yang Yu yang.yu.l...@gmail.com wrote: It is very courteous to reply a SERVFAIL for requests being rate limited. I believe the 'rate-limit' response is actually 'no response' ... though I haven't tested this myself :) On Wed, May 1, 2013 at 1:17 PM, Andrew Fried andrew.fr...@gmail.com wrote: Your IPs may have been rate limited... Andy Andrew Fried andrew.fr...@gmail.com On 5/1/13 12:38 PM, Blair Trosper wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL On Wed, May 1, 2013 at 9:34 AM, Joe Abley jab...@hopcount.ca wrote: On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.netvisualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
Re: Could not send email to office 365
- Original Message - From: Tony Patti t...@swalter.com After our upgrade, we started to see the body of received PLAIN TEXT emails truncated at less than 256 bytes, which frequently truncated emails in the middle of a word, the fix was a setting change. Since being standardized in 1982 with RFC 822, you would think PLAIN TEXT emails would just work out of the box. This is Microsoft, Tony. They don't understand text, and would prefer that all emails be a MIME attachment of a Word document. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
RE: Data Center Installations
-Original Message- From: Blake Pfankuch - Mailing List [mailto:blake.mailingl...@pfankuch.me] Sent: Wednesday, May 01, 2013 6:18 PM To: Otis L. Surratt, Jr.; Warren Bailey; nanog@nanog.org Subject: RE: Data Center Installations Along this same line of questioning... favorite Velcro? I used to get spools of about 500 8 inch strips for a reasonable amount however the vendor went out of business. The cloth tabs are nice, but then end up getting in the way... Thanks, Blake You should be able to get a roll from Graybar. Never checked with CSC for that. We bought a large roll from Graybar and simply cut what we need. It's not precut and pretty but it works.
Re: Data Center Installations
For bulk velcro, I found Uline to be fairly cheap. On Wed, May 1, 2013 at 4:30 PM, Otis L. Surratt, Jr. o...@ocosa.com wrote: -Original Message- From: Blake Pfankuch - Mailing List [mailto:blake.mailingl...@pfankuch.me] Sent: Wednesday, May 01, 2013 6:18 PM To: Otis L. Surratt, Jr.; Warren Bailey; nanog@nanog.org Subject: RE: Data Center Installations Along this same line of questioning... favorite Velcro? I used to get spools of about 500 8 inch strips for a reasonable amount however the vendor went out of business. The cloth tabs are nice, but then end up getting in the way... Thanks, Blake You should be able to get a roll from Graybar. Never checked with CSC for that. We bought a large roll from Graybar and simply cut what we need. It's not precut and pretty but it works. -- Mike Lyon 408-621-4826 mike.l...@gmail.com http://www.linkedin.com/in/mlyon
Re: Data Center Installations
On Wed, May 1, 2013 at 4:33 PM, Mike Lyon mike.l...@gmail.com wrote: For bulk velcro, I found Uline to be fairly cheap. I have to ask, is this an April fools joke? ULine isn't cheap for anything. Monoprice, $13, around $25 delivered depending on where you're at and how yu ship it, for 5x black hook and loop 5yd per roll... vs. ULine $28 (1x black hook and loop 75') and probably about same SH. No easy way to get them to quote SH but last time I ordered from them (they're about the only place to get some stuff) ULine is over 2x as much. Oh and Monoprice has it in quite a few colors if you don't care for black. If you're going for pre-made cable wrap type stuff it's a bit more, but still half or less than ULine. ULine is definitely a supplier of last resort, but they've got a lot of different stuff.
Re: Data Center Installations
Is hard to beat Monoprice :) But no, I have purchased velcro in bulk from ULine (not the kind for wrapping cable though) and found it to be cheaper and I usually got it the next day for not that much shipping. -Mike On Wed, May 1, 2013 at 4:49 PM, Michael Loftis mlof...@wgops.com wrote: On Wed, May 1, 2013 at 4:33 PM, Mike Lyon mike.l...@gmail.com wrote: For bulk velcro, I found Uline to be fairly cheap. I have to ask, is this an April fools joke? ULine isn't cheap for anything. Monoprice, $13, around $25 delivered depending on where you're at and how yu ship it, for 5x black hook and loop 5yd per roll... vs. ULine $28 (1x black hook and loop 75') and probably about same SH. No easy way to get them to quote SH but last time I ordered from them (they're about the only place to get some stuff) ULine is over 2x as much. Oh and Monoprice has it in quite a few colors if you don't care for black. If you're going for pre-made cable wrap type stuff it's a bit more, but still half or less than ULine. ULine is definitely a supplier of last resort, but they've got a lot of different stuff. -- Mike Lyon 408-621-4826 mike.l...@gmail.com http://www.linkedin.com/in/mlyon
Re: Data Center Installations
Zip ties have no reason to be in a dc grr Sent from my iPhone On 2013-05-01, at 6:57 PM, Mike Lyon mike.l...@gmail.com wrote: Is hard to beat Monoprice :) But no, I have purchased velcro in bulk from ULine (not the kind for wrapping cable though) and found it to be cheaper and I usually got it the next day for not that much shipping. -Mike On Wed, May 1, 2013 at 4:49 PM, Michael Loftis mlof...@wgops.com wrote: On Wed, May 1, 2013 at 4:33 PM, Mike Lyon mike.l...@gmail.com wrote: For bulk velcro, I found Uline to be fairly cheap. I have to ask, is this an April fools joke? ULine isn't cheap for anything. Monoprice, $13, around $25 delivered depending on where you're at and how yu ship it, for 5x black hook and loop 5yd per roll... vs. ULine $28 (1x black hook and loop 75') and probably about same SH. No easy way to get them to quote SH but last time I ordered from them (they're about the only place to get some stuff) ULine is over 2x as much. Oh and Monoprice has it in quite a few colors if you don't care for black. If you're going for pre-made cable wrap type stuff it's a bit more, but still half or less than ULine. ULine is definitely a supplier of last resort, but they've got a lot of different stuff. -- Mike Lyon 408-621-4826 mike.l...@gmail.com http://www.linkedin.com/in/mlyon
Re: Data Center Installations
Bring your lacing skills? Sent from my T-Mobile 4G LTE Device Original message From: Mark Gauvin mgau...@dryden.ca Date: 05/01/2013 5:01 PM (GMT-08:00) To: Mike Lyon mike.l...@gmail.com Cc: NANOG nanog@nanog.org Subject: Re: Data Center Installations Zip ties have no reason to be in a dc grr Sent from my iPhone On 2013-05-01, at 6:57 PM, Mike Lyon mike.l...@gmail.com wrote: Is hard to beat Monoprice :) But no, I have purchased velcro in bulk from ULine (not the kind for wrapping cable though) and found it to be cheaper and I usually got it the next day for not that much shipping. -Mike On Wed, May 1, 2013 at 4:49 PM, Michael Loftis mlof...@wgops.com wrote: On Wed, May 1, 2013 at 4:33 PM, Mike Lyon mike.l...@gmail.com wrote: For bulk velcro, I found Uline to be fairly cheap. I have to ask, is this an April fools joke? ULine isn't cheap for anything. Monoprice, $13, around $25 delivered depending on where you're at and how yu ship it, for 5x black hook and loop 5yd per roll... vs. ULine $28 (1x black hook and loop 75') and probably about same SH. No easy way to get them to quote SH but last time I ordered from them (they're about the only place to get some stuff) ULine is over 2x as much. Oh and Monoprice has it in quite a few colors if you don't care for black. If you're going for pre-made cable wrap type stuff it's a bit more, but still half or less than ULine. ULine is definitely a supplier of last resort, but they've got a lot of different stuff. -- Mike Lyon 408-621-4826 mike.l...@gmail.com http://www.linkedin.com/in/mlyon
Re: Data Center Installations
On 5/1/2013 7:57 PM, Mark Gauvin wrote: Zip ties have no reason to be in a dc grr They have their place, but decidedly not in data center racks where **nothing** is permanent/fixed very long :) Jeff
RE: Could not send email to office 365
http://email-guru.com/ ? --- () ascii ribbon campaign against html e-mail /\ www.asciiribbon.org -Original Message- From: Warren Bailey [mailto:wbai...@satelliteintelligencegroup.com] Sent: Wednesday, 01 May, 2013 10:12 To: JoeSox; nanog@nanog.org Subject: Re: Could not send email to office 365 Just another day for them.. Office 365 has been broken for us for a long time. I've been thinking about a rackspace hosted exchange instead.. Have you guys looked into alternatives? Sent from my T-Mobile 4G LTE Device Original message From: JoeSox joe...@gmail.com Date: 05/01/2013 6:27 AM (GMT-08:00) To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
Re: Data Center Installations
Warren Bailey wbai...@satelliteintelligencegroup.com writes: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? For stuff that they carry, Deep Surplus has been acceptable quality and the price is generally right. Others have recommended monoprice but I've found them a bit variable on quality. http://deepsurplus.com -r
Re: Per Site QOS policy with Cisco IOS-XE
On 5/1/13, Wes Tribble westrib...@gmail.com wrote: I have a question for the QOS gurus out there. cisco-nsp might be a better place to post your question. But in any case, this option looks right: Another Idea I had was to create a bunch of shaper classes all feeding the same child policy for priority queuing and bandwidth reservations based on DSCP markings. I’m just not exactly sure that this is allowed or supported. see http://puck.nether.net/pipermail/cisco-nsp/2007-October/044508.html So they just shaped at the hub towards the spoke to prevent overrunning the PE-CE link at the spoke. Another advantage was they didnt' waste hub-PE bandwidth for traffic that would be dropped at the spoke PE-CE link anyway. which has nothing to do with IOS-XE but does sound like what you're wanting to do. Regards, Lee We are having some problems with packet loss for our smaller MPLS locations. This packet loss is due to the large speed differential on our Hub site(150mb/s) in comparison the the branch office locations(single T-1 to 4.5mb/s multilinks). This packet loss only seems to impact really bursty applications like our Web Proxy. I have been around and around with WindStream to give me some extra buffer or enable random early detection on the smaller interfaces in my MPLS network. So far they are unwilling to do a custom policy and none of their standard policies have enough buffer to handle the bursts. They do FIFO tail drop in every queue, so I can’t even choose a policy that has WRED implemented. I am looking for a way to solve the problem on my side. I can create a shaper for the proxy and match off an access-list to the smaller sites, but I am either forced to do bandwidth reservations for each site, or have multiple sites share the same shaper. Here is an example of what I was playing around with: ip access-list extended ProxyT1Sites permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255 permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255 class-map match-any ProxyShaperT1 match access-group name ProxyT1Sites policy-map WindStream class VOICE priority percent 25 set dscp ef class AF41 bandwidth percent 40 set dscp af41 queue-limit 1024 packets class ProxyShaperT1 shape average 1536 bandwidth percent 1 set dscp af21 queue-limit 1024 packets class class-default fair-queue set dscp af21 queue-limit 1024 packets Another Idea I had was to create a bunch of shaper classes all feeding the same child policy for priority queuing and bandwidth reservations based on DSCP markings. I’m just not exactly sure that this is allowed or supported. I also would run out of bandwidth allocation on the policy if I use the true bandwidth number of 150mb/s. It is on a Gig port so I could just take the bandwidth statement off of the interface to give myself enough room for all of the shaper allocations. Something like this(I am omitting the access-list that matches the branch subnet and class map for brevity): policy-map PerSiteShaper class FtSmith shape average 1536 bandwidth 1536 service policy Scheduler class Dallas shape average 4500 bandwidth 4500 service policy Scheduler class NYC shape average 10 bandwidth 10 service policy Scheduler class-default service-policy Scheduler policy-map Scheduler class VOICE priority percent 25 set dscp ef class AF41 bandwidth percent 40 set dscp af41 queue-limit 1024 packets class class-default fair-queue set dscp af21 queue-limit 1024 packets Just looking for some ideas that do not involve building tunnels to our remote offices. Thanks in advance, *Wes Tribble*
Re: Data Center Installations
On Wed, May 1, 2013 at 7:04 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Bring your lacing skills Flex cross twist knot flex cross twist flex cross twist knot flex cross twist... -Blake
Re: Google Public DNS Problems?
On May 1, 2013 5:09 PM, Christopher Morrow morrowc.li...@gmail.com wrote: On Wed, May 1, 2013 at 4:14 PM, Yang Yu yang.yu.l...@gmail.com wrote: It is very courteous to reply a SERVFAIL for requests being rate limited. I believe the 'rate-limit' response is actually 'no response' ... though I haven't tested this myself :) Yes if someone has a misbehaving program or is trying to DOS you, you don't really want to reply with anything. On Wed, May 1, 2013 at 1:17 PM, Andrew Fried andrew.fr...@gmail.com wrote: Your IPs may have been rate limited... Andy Andrew Fried andrew.fr...@gmail.com On 5/1/13 12:38 PM, Blair Trosper wrote: That's all well and good, but I certainly wouldn't expect nslookup gmail.com or for nslookup google.com to return SERVFAIL On Wed, May 1, 2013 at 9:34 AM, Joe Abley jab...@hopcount.ca wrote: On 2013-05-01, at 12:09, Blair Trosper blair.tros...@gmail.com wrote: Is anyone else seeing this? From Santa Clara, CA, on Comcast Business...I'm getting SERVFAIL for any query I throw at 8.8.8.8 and 8.8.4.4... Level 3's own public resolvers are fine for me, as are OpenDNS's resolvers. Google just turned on validation across the whole of 8.8.8.8 and 8.8.4.4. The expected behaviour in the case where a response does not validate is to return SERVFAIL to the client. You could check that the queries you are sending are not suffering from poor signing hygiene (e.g. use the handy-dandy dnsviz.netvisualisation). If this is a repeatable, consistent problem even for unsigned zones (or for zones that you've verified are signed correctly) and especially if it's widespread you might want to call google on the nanog courtesy phone and have them look for collateral damage from their recent foray into 8.8.8.8 validation. Raw output from dig/drill and traceroutes to 8.8.8.8/8.8.4.4 are highly recommended if you need to take this further. Joe
RE: Could not send email to office 365
Yes we are just working out our licensing agreement and moving Exchange back in house. From talking with people BPOS (Exchange 2007) was a mess. We were on Office 365 with Exchange 2010 without issue service worked very well. Then they upgraded us to Exchange 2013 and it is just broken we are unable to connect. I thought the problem is fixed but now it is back. -Original Message- From: Warren Bailey [mailto:wbai...@satelliteintelligencegroup.com] Sent: Wednesday, May 1, 2013 12:12 PM To: JoeSox; nanog@nanog.org Subject: Re: Could not send email to office 365 Just another day for them.. Office 365 has been broken for us for a long time. I've been thinking about a rackspace hosted exchange instead.. Have you guys looked into alternatives? Sent from my T-Mobile 4G LTE Device Original message From: JoeSox joe...@gmail.com Date: 05/01/2013 6:27 AM (GMT-08:00) To: nanog@nanog.org Subject: Re: Could not send email to office 365 The company I work for has been having Outlook connectivity issues (intermittent for only a few end users) for the past 7 days for Office 365. We are in an upgrade status (on the 18th days or so; have been told it can last 30 days) and they changed our MX records without formal notification. We updated those on a Thursday or Friday and it worked until the following Monday where we observed it again, then magically fixed itself late afternoon that Monday. We had some more reports yesterday. The Microsoft technical support has not been helpful troubleshooting this for us. I am hoping it is related to our upgrade status but I cannot get an answer from anyone. -- Thanks, Joe On Wed, May 1, 2013 at 2:21 AM, ISHII, Yuji y...@ugec.net wrote: Hello folks, Have you ever seen DNS issues on Office 365? MX record of Office 365 is example.mail.eo.outlook.com. I can get the MX record, however, I could not get the A record of the MX record, got Timeout. Does anyone have the same issue? Sincerely, Yuji
RE: Data Center Installations
Wish there was Frys in the east -Original Message- From: George Herbert [mailto:george.herb...@gmail.com] Sent: Wednesday, May 1, 2013 3:42 PM To: Joe Hamelin Cc: nanog@nanog.org Subject: Re: Data Center Installations Seconded Graybar. If necessary, in the absence of Graybar or for tiny stuff, a Frys or Home Depot or Lowes. On Wed, May 1, 2013 at 12:32 PM, Joe Hamelin j...@nethead.com wrote: Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren -- -george william herbert george.herb...@gmail.com
RE: Could not send email to office 365
It was in upgrade status for about 15 days. We had to open a separate ticket to fix the upgrade but even after they completed the upgrade I was unable to connect. -Original Message- From: JoeSox [mailto:joe...@gmail.com] Sent: Wednesday, May 1, 2013 12:28 PM To: nanog@nanog.org Subject: Re: Could not send email to office 365 Ryan, Is your Office 365 account also in an upgrade status? If not, have you completed the upgrade? -- Thanks, Joe On Wed, May 1, 2013 at 8:42 AM, Ryan Finnesey r...@finnesey.com wrote: I am also having the some issues going on 3 weeks now. I cannot access my e-mail via Outlook and my MX records keep changing. It is nuts support has been unable to help.
RE: Data Center Installations
Graybar is great -Original Message- From: Joe Hamelin [mailto:j...@nethead.com] Sent: Wednesday, May 1, 2013 3:32 PM To: Warren Bailey Cc: nanog@nanog.org Subject: Re: Data Center Installations Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren
RE: Data Center Installations
I'm more impressed with MicroCenter than Frys (at least the Frys south if SF). If you need RF I used to order from Davis RF all the time. On May 2, 2013 12:57 AM, Ryan Finnesey r...@finnesey.com wrote: Wish there was Frys in the east -Original Message- From: George Herbert [mailto:george.herb...@gmail.com] Sent: Wednesday, May 1, 2013 3:42 PM To: Joe Hamelin Cc: nanog@nanog.org Subject: Re: Data Center Installations Seconded Graybar. If necessary, in the absence of Graybar or for tiny stuff, a Frys or Home Depot or Lowes. On Wed, May 1, 2013 at 12:32 PM, Joe Hamelin j...@nethead.com wrote: Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren -- -george william herbert george.herb...@gmail.com
Re: Data Center Installations
Never been to MicroCenter Sent from my iPad mini On May 2, 2013, at 1:05 AM, shawn wilson ag4ve...@gmail.commailto:ag4ve...@gmail.com wrote: I'm more impressed with MicroCenter than Frys (at least the Frys south if SF). If you need RF I used to order from Davis RF all the time. On May 2, 2013 12:57 AM, Ryan Finnesey r...@finnesey.commailto:r...@finnesey.com wrote: Wish there was Frys in the east -Original Message- From: George Herbert [mailto:george.herb...@gmail.commailto:george.herb...@gmail.com] Sent: Wednesday, May 1, 2013 3:42 PM To: Joe Hamelin Cc: nanog@nanog.orgmailto:nanog@nanog.org Subject: Re: Data Center Installations Seconded Graybar. If necessary, in the absence of Graybar or for tiny stuff, a Frys or Home Depot or Lowes. On Wed, May 1, 2013 at 12:32 PM, Joe Hamelin j...@nethead.commailto:j...@nethead.com wrote: Graybar. -- Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474tel:360-474-7474 On Wed, May 1, 2013 at 12:23 PM, Warren Bailey wbai...@satelliteintelligencegroup.commailto:wbai...@satelliteintelligencegroup.com wrote: Do any of you have a go to resource for materials used in installations? Tie wraps, cable management, blahblahblah? I have found several places, but I'm curious to know what the nanog ninja's have to say. //warren -- -george william herbert george.herb...@gmail.commailto:george.herb...@gmail.com
