Re: ipv6 @ sprint, somebody home?
:- Jeroen == Jeroen Massar [EMAIL PROTECTED] writes: [EMAIL PROTECTED]: host kay.sprintlink.net[199.0.233.8] said: 553 5.3.0 [EMAIL PROTECTED]... User Unknown (in reply to RCPT TO command) It's must be 6/6/6 that it ain't working. I guess they are scared that IPv6 might scare their fisherprice routers ;) Anybody a *working* contact so that they can also be nicely reminded of the fact that the 6bone has come to an end and that they should nicely ask their paying customers to stop announcing 6bone space? http://www.sixxs.net/tools/grh/lg/?=prefixfind=3ffe::/16 And http://www.sprintv6.net/ doesn't contain any contact info before you say google it. Then again the following url clearly shows their 'interrest' http://www.sprintv6.net/aspath/bgp-page-complete.html Last change on the tree detected on Sun DEC 11 2005, h.22:50 Fisherprice, fisherprice I just got a mail from them with the new addresses to use. They may be running a bit late, but they are doing it. Pf -- --- Pierfrancesco Caci | Network System Administrator - INOC-DBA: 6762*PFC [EMAIL PROTECTED] | Telecom Italia Sparkle - http://etabeta.noc.seabone.net/ Linux clarabella 2.6.12-10-686-smp #1 SMP Sat Mar 11 16:41:12 UTC 2006 i686 GNU/Linux
Re: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.
The only part that I don't get is that you can mtr to him without packetloss. Although the path in-between may be different, the final hop packetloss should exactly equal what he sees when mtring you. A round-trip is a round-trip, and results should be identical regardless of who originates. I can't think of any way this would be different unless echo and echo-reply were being rate limited independently. If the time was different then the packet loss would be different. Perhaps the customer runs the tests during his busy period when he is concerned about making sure there is no delay. Then, later in the day, after his busy period is over he takes the time to contact his ISP. The ISP then runs some tests which show there is no packet loss at all. To be sure this is not happening, synchronize the tests and run simultaneously. Try tcptraceroute because this more accurately reflects the traffic that is flowing. http://michael.toren.net/code/tcptraceroute/ http://tracetcp.sourceforge.net/ is a windows tool that is similar. The open source tool LFT can be built to run on Windows under cygwin http://pwhois.org/lft/ but they have this warning on their page: Many people have complained about various problems on the Windows platform. Both LFT and the WhoB client compile and run well under Cygwin environments on Windows. Unfortunately, Microsoft's changes to the Windows IP stack (as of XP Service Pack 2) reduced their raw socket functionality significantly as part of their security bolstering process. These changes have effectively stopped LFT from working properly while using TCP. LFT's UDP tracing and other advanced features still work properly. For more information on Windows raw sockets, consult www.microsoft.com/technet/prodtechnol/winxppro/maintain/sp2netwk.mspx#EIAA This may have nothing to do with your MTR issue but it does make one wonder whether a Windows machine is safe to do performance testing. In any case, the LFT people think that their non-TCP features still work properly on Windows and this is a tool that you can also run on your end. Worth a try? --Michael Dillon
Re: Zebra/linux device production networking?
First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. It is primarily small companies that use zebra or Quagga or openbgpd or Xorp or the Click Modular Router project. There is more than one choice so do your research. The main drawback of all of these is that you cannot get PCI-bus cards that support some common circuit types and the PCI bus cannot handle switching high traffic volumes. Many people build and sell routers based on a PC server running UNIX. They work fine if they are no stretched beyond the role intended. Cisco routers are the same. Look at the limitations of the 2500/2600 series for instance. Some URLs of interest: http://www.read.cs.ucla.edu/click/ http://www.xorp.org/ http://www.openbgpd.org/ http://www.quagga.net/ http://www.zebra.org/ Has there been any discussion (or musings) of moving towards such a solution? I've seen a lot of articles talking about it, but I've not actually seen many network operators chiming in. This tends to be a list focused on the cult of the BIG IRON, namely Cisco and Juniper. The people who use PC-based routers have their own hangouts. My main piece of advice is to seek out those hangouts and ask your questions there. Here's the article that started it all (this was featured on /., so likely you've read it already). Sorry, haven't seen these. --Michael Dillon
2006.06.06 NANOG-NOTES DNS reflector attacks
(I was going to try to get all the notes from today's panels out before going to bed, but I fell asleep on my keyboard finishing up these notes, so I think I'm going to wait and send the batch of Tuesday and Wednesday notes out after things wrap up on Wednesday. Sorry about the delay, but I need a bit more sleep I think. ^_^;; --Matt) 2006.06.06 Morning welcome, and introduction of Chris Morrow, panelist Please fill out survey today if you're going to be leaving! Frank Scalzo, Verisign Recent DNS reflector attacks. Attacker breaks into innocent authoritative DNS server, publishes large text record; then does queries from zombie army against that record, with sources spoofed with victim IP. 5 gig attack, 2.2G made it, 3gig didn't. E.TN.CO.ZA DNS attack, 64 byte query, 63:1 amplification, 4028 byte answer 34,668 reflectors. Victim sees 5G of traffic, 144,142bps per reflector, 13.5packets per second 4.5DNS answers per second. reflectors won't see this as anomolous for the most part; top talker only sent 8.5 answers per second. No visibility into the attacker at all, but best guess was 79Mb of source generated 5GB of responses. Record was maliciously installed; 2 auth servers, 1 compromised; 65% response, 35% name error. Answer comes in 3 fragments, larger than normal MTU. Attack came in 3 phases. first port 666, then port 53 and 666, then all 53. Port shifts are nearly instant, so fast command and control system in place for it. Filter out open recursive DNS servers; you can't put ACL in for 500,000 DNS servers. What about limiting DNS packets to 512 bytes? will break things. What about blocking 53 outside of your network hierarchy, force people to use your resolvers? What about discarding fragments? Challenge is getting your upstream to implement it, unless you have hardware and pipes to handle the flood coming at you to start with. Some ISPs won't do it unless they see live attack traffic, and a 24 minute attack is too short lived for ISPs to see and react to. data from Jan 11 - Feb 27 this year. Attack queries/second consistent with avg reflector qps. one reflector sent 1.9M DNS answers to 1593 victims, 605 different queries to generate answers. 180TB of attack traffic on Feb 1st. after feb 15th, ramped down. Assume 4KB response packet, see attacks between 3G and 7G, the scary part is that it only took 130Mb to generate the 7G attack, and the 3 gig attacks are all from less than a fastE connected compromised web server. 500,000 reflectors with 2G source could generate a 120GB DoS attack. Top victim got over 130Tb of attack traffic, top bunch are all over 100Tb 65,461 ports used, Top port is less than 10% of traffic though top 20 domains used, mostly innocent bystanders. Internet root . was second highest domain used; certainly can't filter *that* out. Fundamental challenge; UDP lacks 3 way handshake, easy to spoof DNS is easy target, so many unsecured DNS servers Other UDP servers need to be evaluated as well DNS closing 500,000 open recursive DNS servers will be very, very painful. poor separation between authoritative and recursive DNS servers. BIND allow-query ACLs, recursive DNS servers should not accept queries from outside. What if it's an embedded system like a wireless gateway? We depend on large records for DNSSec, etc. Beyond open recursive DNS servers root domain . was used most authoritative name servers will answer with an upward referral doesn't include actual IPs, but it's still 438 bytes, and pretty much every DNS server responds to it. Source validation IETF BCP 38 How do you manage 70,000 ACLs on 500 routers? what about people who are multi-homed with static routes? what about legacy stuff that works but shouldn't? strict RPF breaks with traffic asymmetry; loose RPF doesn't help with this. ISPs see the problem as long, hard, expensive to overcome, and they're right. If we never start trying, we'll never fix it! Close open recursive DNS servers DNS servers should include filtering SOHO router vendors should fix their DNS proxy code, don't listen on outside interface BCP 38 otherwise we'll be jumping from protocol to protocol. Questions? Q: What does verisign do to protect their DNS servers? A: Anycast, massive peering and transit capacity Q: Jared Mauch, NTT/America; he turned on unicast rpf on the NANOG upstream link. 372,000 packets that people here have sent failed the RPF check. BCP 38 is hard Paul Quinn asked what percentage of the traffic that is. Bora Akyul, Broadcomm--any data on source ranges on the packets being seen? He could look at the 1 in 10,000 netflow sampling to see, but the individual link is a /30, looks like a normal customer link. The Merit router isn't RPF'ing either. Q: Ren Provo asks when they will peer; A: not yet, next few months, Miami Terremark, and other sites domestically and internationally in next year and a half.
RE: Zebra/linux device production networking?
I would be interested to know how many software (for want of a better description) routers are in live production in this kind of environment i.e. the 99.% Uptime variety, from speaking to people albeit randomly in data centres it would seem to be more common than one might expect. It is indeed very common. That is why there are several implementations of BGP and routing software available. These are used in dozens and dozens of commercial products some of which are sold as IP routers, plain and simple. In any case, 5 nines and 6 nines are not always what the marketing department claims. They often exclude planned maintenance periods so if you reboot once a week or you have a crash after changing a config, that doesn't count against the 5 nines. In addition, the 5 nines figure generally applies to the network, not to individual devices within it. Networks can be designed so that the failure of a device does not cause a network outage. This whole issue is so complex that you just can't make blanket recommendations. Even the biggest networks don't just buy and deploy big iron. They run every new router model and software release through an extensive battery of tests. Then they write operational guidelines telling people which features can be used in which situations. They do this to avoid crashes and network outages because the big iron (Cisco/Juniper) simply cannot provide that on its own. A smart small company can get excellent results from Linux routers (although I would take a serious look at FreeBSD or OpenBSD for this). Process is as important as hardware. --Michael Dillon
2006.06.06 NANOG-NOTES network-level spam behaviour
2006.06.06 Nick Feamster, Network-level spam behaviour [slides are at: http://www.nanog.org/mtg-0606/pdf/nick-feamster.pdf Spam unsolicited commercial email feb 2005, 90% of all email is spam common filtering techniques are content based DNS balcklist queries are significant fraction of DNS traffic today. (DNSbls) Using IP address based spam black lists isn't so useful. How spammers evade blacklists will be discussed as well. Problems with content-based filters ...uh oh, some technical glitches... Content-based properties are malleable low cost to evasion altering content based on scripts is too easy customized emails are easy to generate content based filters need fuzzy hashes over content, etc. high cost to filter maintainers as content changes, filters need to be updated. constantly tweaking spamassasain rules is a pain. false positives are always an issue. Content-based filters are applied at the destination too little, too late -- wasted network bandwidth, storage, etc. ; many users recieve and store the same spam content. Network level spam filtering is robust (hypothesis) network-level propeerties are more fixed hosting or upstream ISP (as number) botnet membership location in the network IP address block country? are there common ISPs that host the spammers, for example? Avoid receiving mail from machines that are part of botnets. Challenge--which properties are most useful for distinguishing spam traffic from legitimate email? very little if anything is known about these characteristics yet! Randy gave a lightning talk last NANOG about some of this. Some properties listed. Spamming techniques mostly botnets, of course other techniques too we're trying to quantify this coordination characteristics how we're doing this correlations with Bobax victims from georgia tech botnet sinkhole other possilities: heuristics distance of client IP from the MX record coordinated, low-bandwidth sending looked at pcaps coming in from hijacked command and control station from bots trying to talk to it; spamming bots, Bobax drone botnet, exclusively used to send spam. Collection two domains instrumented with MailAvenger (both on the same network) sinkhole domain 1 continuous spam collection since aug 2004 no real email addresses--sink everything 10 million + pieces of spam sinkhole domain #2 recently registered Nov 2005 clean control domain posted at a few places not much spam yet--perhaps being too conservative contact page with random email contact, look at who crawls, and then who spams the unique email addresses Monitoring BGP route advertisments from same network Also capturing traceroutes, DNSBL results, passive TCP host fingerprinting, simultaneous with spam arrival (results in this talk focus on BGP+ spam only) Mail Avenger, not an MTA, it forks to sendmail or postfix, it sits in front of MTA, does things like do DNSBL lookups, add headers, passive OS fingerprinting, as the spam is arriving. Also logged BGP routes from same network that got the spam; see connectivity to the spamming machine at the time. Picture of collection up at MIT network. Mail Collection: MailAvenger X-Avenger header. best guess at operating system, POF, DNSBL lookups, traceroutes back to mail relay at the time the mail was sent (used for debugging BGP) distribution across IP space plot /24 prefix vs how much spam coming from it. steeper lines mean more spam from that part of the IP space; you can see where spam is coming from. bunch comes from apnic, cable modem space, etc. few interesting things to note; still redoing legitimate mail characteristics. from georgia tech mail machines, it's legit plus spam, need to split out better. between 90.* and 180.*, legitimate mail mainly. Is IP-based blacklisting enough? Probably not: more than half of spamming client IPs appear less than twice. Roughly 50% of the IPs showed up less than twice; but that's a single sinkhole domain, would help more across multiple domains. emphasizes need to collaborate across multiple domains to build blacklists; any one domain won't see repeated patterns of IPs. Distribution across ASes 40% of spam coming from the US BGP spectrum agility Log IP addresses of SMTP relays Join with BGP route advertisements seen at network where spam trap is co-located. A small club of persistent players appears to be using this technique 61.0.0.0/8 AS4678 66.0.0.0/8 AS21562 82.0.0.0/8 AS8717 somewhere between 1-10% of all spam (some clearly intentional, others might be flapping) about 10 minute announcement time of the /8 while spam is flooded out. Might be interesting to couple this with route hijacking alerting to filter out if this is really a hijacking vs a flapping legitimate route. A slightly different pattern; announce-spam-withdraw on a minute-by-minute basis. really really egregious! Why such big prefixes? flexibility: client IPs can be scattered throughout dark space within a large /8 same sender usually returns with
Re: Zebra/linux device production networking?
Nick Burke wrote: Greetings fellow nanogers, How many of you have actually use(d) Zebra/Linux as a routing device (core and/or regional, I'd be interested in both) in a production (read: 99.999% required, hsrp, bgp, dot1q, other goodies) environment? Just have a look for MTU. If you connect home - aDSL - someplace and your MTU is smaller than the aDSL packetsize then your connection is home - adsl - tunnel - someplace That tunnel consists of two routers, linux or whatever. Behind the tunnel you might find some 200 hosts. The speed is 2Meg through the tunnel. It used to connect one /18 and a handful of /24 The two linux boxes were maintained by a guru. They almost never gave problems. Mostly the hardware router behind that tunnel did. I dont know what kind of device it is. All I know is, it seems to know some 8 or more interfaces, hardware or virtual. The installation, a nuclear bunker, used to house some websites and services. (And an XTC-lab :) There are a lot of network bunkers arround. I guess half of them looks the same. Cheers Peter and Karin Dambier -- Peter and Karin Dambier Cesidian Root - Radice Cesidiana Graeffstrasse 14 D-64646 Heppenheim +49(6252)671-788 (Telekom) +49(179)108-3978 (O2 Genion) +49(6252)750-308 (VoIP: sipgate.de) mail: [EMAIL PROTECTED] mail: [EMAIL PROTECTED] http://iason.site.voila.fr/ https://sourceforge.net/projects/iason/
Re: ipv6 @ sprint, somebody home?
On Tue, Jun 06, 2006 at 07:38:51PM -0400, Jared Mauch wrote: On Tue, Jun 06, 2006 at 09:45:18PM +0200, Jeroen Massar wrote: And http://www.sprintv6.net/ doesn't contain any contact info before you say google it. Then again the following url clearly shows their 'interrest' http://www.sprintv6.net/aspath/bgp-page-complete.html Last change on the tree detected on Sun DEC 11 2005, h.22:50 those people at PAIX Palo Alto i think are still waiting for the nap lan to number out of 3ffe space. It's the same as the IPv4 lan (vlan6) you just set up the v6 ips there.. I suspect in another few days all these routes will go away and will start to be filtered more effectively. - jared -- Jared Mauch | pgp key available via finger from [EMAIL PROTECTED] clue++; | http://puck.nether.net/~jared/ My statements are only mine. they should not be waiting for those numbers, they have had them for a couple of years now. --bill
Re: Zebra/linux device production networking?
On Wed, 7 Jun 2006 [EMAIL PROTECTED] wrote: First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. We are a small company, so naturally penny pinching is the primary motivation. It is primarily small companies that use zebra or Quagga or openbgpd or Xorp or the Click Modular Router project. There is more than one choice so do your research. The main drawback of all of these is that you cannot get PCI-bus cards that support some common circuit types and the PCI bus cannot handle switching high traffic volumes. I've talked to people using PC-based system on OC48 and analyzing that entire data. Sounded unbelievable to me but their numbers of how much data PCI(Express) can handle support that PC-based router would be able to do it. How reliable this is and if cost of supporting such router is worth going forward is another matter. Also both Linux and Freebsd are fairly equivalent as bases for such routers and if you have knowledgeable people (and you should if you're considering going with PC router), you should be able to set linux that is secure as freebsd. There are some differences in the routing code whereas Linux is designed with per-flow based switching in mind (which works very well when used as a server) and has extensive packet classification mechanism (which I strongly advise you test in the lab before trying in production). Freebsd has what I consider to be simpler code design for which many believe works better if you receive unusual packets, but personally I've used Linux as packet firewall at Gb rate and it handled DoS fine. Linux also supports multiple routing tables in the kernel, which I think latest quagga can take advantage of and it can make a difference whe selecting linux vs freebsd. Now do remember that biggest headache is going to be supporting this as such custom solution will require custom coding of tools and good engineer who really knows well both linux and networking and finding more such people to support your infrastructure if you grow maybe difficult. -- William Leibzon Elan Networks [EMAIL PROTECTED]
Re: 2006.06.05 NANOG-NOTES BGP tools BOF notes
Matthew Petach wrote: Q: Randy Bush. Common problem we all face. I'm at 42 peering points; my neighbors are X. I have route views dumps, I have my BGP dumps. I have my netflow data. Want a whatifatron that shows what happens to my traffic if depeer someone, or add someone, or peer with SingTel in singapore, or stop peering with Joe in SF. That's a question many operators ask every day. We have such a whatifatron. We used it for instance to investigate the impact of peering/depeering on routing and on traffic in various ISP networks including a large european transit network. Our tool is called C-BGP and some of the what-if scenarios we performed on the GEANT network were described recently in an IEEE Network paper entitled Modeling the routing of an Autonomous System with C-BGP (November 2005). Our tool is able to eat BGP dumps (in MRT format), Cisco/Juniper configs and NetFlow data. It's open-source and released under LGPL. It is still a command-line tool but we are working towards a more user-friendly interface. Some useful links: http://cbgp.info.ucl.ac.be http://www.info.ucl.ac.be/~standel/bgp-converter/- Cisco/Juniper parser http://cbgp.info.ucl.ac.be/gui-totem.html- upcoming GUI A: Matt notes that if they can solve that question/write something that does all that, they'll have Arbor and others beating on their door. ^_^ If any of you is interested in testing it on its data, please feel free to contact me :-) Bruno -- CSE Dept. UCL, Belgium - http://www.info.ucl.ac.be/~bqu Phone: ++32 10 47 24 04 GSM: ++32 498 28 12 21
Re: Zebra/linux device production networking?
On 6/7/06, Peter Dambier [EMAIL PROTECTED] wrote: The installation, a nuclear bunker, used to house some websites and services. (And an XTC-lab :) Ah, I sometimes wonder about how people get the idea of deploying alternate roots. Then I see that email from Peter and it all becomes blindingly clear. :) --srs -- Suresh Ramasubramanian ([EMAIL PROTECTED])
Subject: Found power supply at NANOG37
Found: HP laptop power supply left on a large round table late tuesday night in the main hallway. Here's hoping you have enough juice left to read this email...
Re: Zebra/linux device production networking?
On 6/7/06, Nick Burke [EMAIL PROTECTED] wrote: First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. This looks reasonable .. http://www.linux-vpn.de/lr101.php -- Suresh Ramasubramanian ([EMAIL PROTECTED])
2006.06.06 NANOG-NOTES DDoS attack information collection
Information collection on DDoS attacks, Anna Claiborne, Prolexic Technologies. [slides are at: http://www.nanog.org/mtg-0606/pdf/anna-claiborne.pdf DDoS mitigation service. personal experience mitigating over 150 DDoS attacks. Popular topic, but nobody talks about how you can defend yourself or take legal action; only thing you can do is collect information. 0.1% of DDoS attacks end in an arrest, that's out of the reported number to the US Secret Service, and that's out of the ones that fall into their jurisdiction. These are real losses: A major US corp lost over $2mil in a 20 hour outage An offshore gambling comp. lost estimated $4m in 3 days Online payment processor lost $400,000 in 72 hours online retailer lost $20K/day over 3 weeks. These are directly reported losses; doesn't include lost PR, etc. Canadian retailer spend 50K on hardware mitigation, they got kicked out of 3 datacenters due to the DDoS attacks, spent 20K on IT and security consultants, and another $6K on a different mitigation that also failed. Basic Information Collection Get packet captures--either from machine being attacked, or a span port, or from upstream device, tcpdump -n -s0 -C (get full length of raw packet, limit pcap file to 5MB or smaller) take 3 or 4 over 15 minutes, to start, and then repeat every hour Determine the type of attack and duration (ex SYN flood lasting 6 hours) Obtain as complete a list as possible of source IP addresses Save bandwidth graphs, flow data, pps graphs, any and all visual material relating to the attack Save any contact with the attacker, email, chat conversation, phone calls, etc. Get loss figures from management--downtime, per hour losses, per day losses, section 18 of some law, have to substantiate losses over $5k before you can take legal action against someone. Recommendations have a plan! DDoS is stressful Put all attack information in a central location God monitoring doesn't have to be expensive, a simple fiber card in a 1u box can be a mirror port for a large volume of traffic Don't have to have expensive hardware like arbor boxes. Limit to 100mb to prevent killing your capture box. Graphs and flow data can be retrieved from upstream Find the source Use list of source addresses, find a reputable hosting company, you may even see a friend's IP Approach the network with the infected machine, give them as much information as possible, it can take time finding someone willing to help Obtaining information is dependent on who you are dealing with, be as helpful as possible. Get information from the infected machine netstat, tcpdumps, who is logged in, web logs, access logs Get and save the source code responsible process can take hours to weeks--prolexic has huge contact list, and even for them can be really difficult And SAVE all your information to a central location! and back it up! Examine the source code scripts are best, you know exactly what's going on compiled code, run strings on it best case, you can get a name or identification for who wrote it, passwords, domain names, port usage worst case you can obtain information that doesn't make sense...yet (it may fit into a bigger context later) Locate controlling server Examine TCP connection table or source code to find the controlling server verify your information, scan or connect to the suspect machine contact abuse where the server is hosted, explain the situation have as much information possible to verify your conclusion and validate your identity Good luck, most abuse contacts are less than helpful Raises a good question: how to improve awareness and legitimate requests answered. (may be able to get FBI to provide warrants to seize machines that are being used to control attacks against you, but takes time and documentation) Hunting the attacker (not for the faint of heart!) Review all information gathered so far on the attack contact the attacker, establish a report save all information and/or conversations (important note, if conversations aren't on a public server, they can't be used) Piecing the information together to form a high level view of the exploit, attack, and attacker A long process, most attackers are highly motivated and skilled, you usuallly have to wait for them to slip up! Resources: local FBI field office department of cybercrime department of homeland security CERT Cymru--great guys, if they have to help you NHTCU--EU, cyber crime divisions in local offices Local US secret service--division of electronic crimes DDoSDB.org -- under development at the moment. how to identify/recognize different types of attacks may be able to put their attack database open to the public up there. A success story The tracking of x3m1st/eXe responsible for hundreds of extortion based DDoS attacks tracked for months eventually lead to his arrest. hid behind four levels of compromised servers. eXe and his group only talked on private IRC servers; made the mistake of connecting from his home domain, from a
Re: Zebra/linux device production networking? (summary)
Thanks to all for all the feedback! It seems what a lot of people are saying is that it's almost acceptable (in that, you shouldn't if you can afford other devices), given the right time and engineering. The cost of supporting seems to be unanimously higher then going with a specific vendor. A number of people have noted that some of the support that the various packages of software for handling routing protocols may not play correctly with the os layer or even other packages. (IE: routing) I've seen confliction on if *bsd or linux is better, this (hopefully) isn't that surprising to anyone. The consensus is that when something breaks it takes longer to fix and requires greater technical aptitude. Finally, it appears as if, contrary to what the articles are saying, not many people are actively considering such a move. However, it is more common in smaller businesses starting new locations or building out. A lot of people seemed to of assumed the absolute worse case (which, might I add, is generally what I was looking for) scenario: a dusty box with interesting hardware out-of-the-box kernel no research a MSMD approach What about better case situations?* IE: toe cards custom kernel no moving parts (ie: hard drive, maybe fans if possible) up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages hardware would be best *This deviates from operational and gets into the more technical issues, so it's actually a not a question I'm looking for you kind folks to answer. But I feel I have to vindicate myself a little bit as my technical skills were called into question for even posting the original email... ;) Once again, thanks everyone!
Re: Zebra/linux device production networking? (summary)
On Wed, 7 Jun 2006, Nick Burke wrote: What about better case situations?* IE: toe cards custom kernel no moving parts (ie: hard drive, maybe fans if possible) up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages hardware would be best I didn't notice anyone mention Imagestream, who sell Linux based routers using a custom distro and no moving parts other than fans. Storage is flash. I've helped a client manage several of them for several years. IMO, they're not bad as CPE, but I don't think we could use them if we wanted to on most of our network. Some of the features we need just aren't available. As others have mentioned, I wouldn't recommend it unless you have some people very comfortable with Linux and IP routing on Linux on staff. At one point, they had 4 full BGP feeds going into one Imagestream Gateway router, which is a P4, upgraded to 512MB RAM. With 2 full views now, they have 308MB free. It's an older installation, predating the addition of zebra/quagga to their distro, so it's still running gated_public, which works, but is fairly lacking in BGP knobs. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
Check out the IAR for Potential Prefix Hijacks and if you're coming to this more than 24 hours after the post, do a search on AS 23520 as the hijacking AS. I don't know how long the routes were announced, but they seem to be gone now. Or maybe the IAR is horribly broken, in which case I will be lynched :) IAR: http://cs.unm.edu/~karlinjf/IAR/ Josh
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
On Wed, 7 Jun 2006, Josh Karlin wrote: I don't expect better from NW Network Cable, but I definitely expect better from Sprint (their upstream). But this hasn't been the first and most unfortunately, not the last, cuz, almost no one gives a f*ck anymore. -Hank Check out the IAR for Potential Prefix Hijacks and if you're coming to this more than 24 hours after the post, do a search on AS 23520 as the hijacking AS. I don't know how long the routes were announced, but they seem to be gone now. Or maybe the IAR is horribly broken, in which case I will be lynched :) IAR: http://cs.unm.edu/~karlinjf/IAR/ Josh +++ This Mail Was Scanned By Mail-seCure System at the Tel-Aviv University CC.
Re: Zebra/linux device production networking?
In article [EMAIL PROTECTED], william(at)elan.net [EMAIL PROTECTED] wrote: you should be able to set linux that is secure as freebsd. There are some differences in the routing code whereas Linux is designed with per-flow based switching in mind (which works very well when used as a server) Nobody noticed, but Linux 2.6 has alternative FIB code you can select when compiling the kernel. Yes, it is fairly new and I'm not sure it is production quality, but still. The config option is IP_FIB_TRIE, for the LC-trie algorithm. It's supposed to be something like CEF. Mike.
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
On Wed, 7 Jun 2006, Hank Nussbacher wrote: On Wed, 7 Jun 2006, Josh Karlin wrote: I don't expect better from NW Network Cable, but I definitely expect better from Sprint (their upstream). But this hasn't been the first and most unfortunately, not the last, cuz, almost no one gives a f*ck anymore. Well, when all we do about a situation is b*tch about it, waste time running after other people's tails, and repeat the same process all over again - we can't really expect for anything to change. If we won't take hold of these issues, eventually someone will take hold of them for us. They will most likely do a bad job at it but they will *do*, making our lives extremely difficult in the process. Which Government is your first bet? Unless this ISP is cut from the net by its uplink, and then called to answer before a judge or say, the FCC, nothing will change. The day this will happen will be a very sad say, as the uplink will not be making money and the Government will likely miss the whole point, but still. How can we complain about China when most of the problems are our own? Gadi. -Hank Check out the IAR for Potential Prefix Hijacks and if you're coming to this more than 24 hours after the post, do a search on AS 23520 as the hijacking AS. I don't know how long the routes were announced, but they seem to be gone now. Or maybe the IAR is horribly broken, in which case I will be lynched :) IAR: http://cs.unm.edu/~karlinjf/IAR/ Josh +++ This Mail Was Scanned By Mail-seCure System at the Tel-Aviv University CC.
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
Wonder if it was intentional or a 'classful' issue. This is why we (Level 3) and ATT announce the /9s of 4/8, 8/8, and 12/8 :) -Kevin The /9s were stolen too, as well as a host of other prefixes. I just listed the biggies that I was pretty sure didn't belong to 23520. No clue if it was intentional or not, but I would also like to know. Josh
Re: Zebra/linux device production networking? (summary)
I've seen confliction on if *bsd or linux is better, this (hopefully) isn't that surprising to anyone. You should do a PPS throughput analysis of your own to see which OS works better on the hardware that you plan to use. Drivers, and the susceptibility of the kernel to livelock, are where there may be differences in performance. Finally, it appears as if, contrary to what the articles are saying, not many people are actively considering such a move. However, it is more common in smaller businesses starting new locations or building out. DEC's gateway to the Internet ran on host-based routers - DEC Alphas running Digital UNIX with turbochannel FDDI cards - from 1994 to sometime in 1999-ish (I stopped being responsible for it in 1998). I started with a pair and had suffered one all-night upgrade to eight when the PPS load of some AltaVista announcement pushed the pair over the edge into livelock. What about better case situations?* IE: toe cards TOE won't help you, you aren't terminating TCP sessions on the box. At least you shouldn't be. Don't let anyone talk you into also running a web server. custom kernel This could be useful, if the kernel is able to handle all packet forwarding in the interrupt or polling input service routine. no moving parts (ie: hard drive, maybe fans if possible) That'll certainly help with reliability, as well as dual power supplies. up-to-date software packages with internal coders to fix ugly bugs, etc actual research into what packages hardware would be best Both of those things, or a support agreement from one of the vendors that's trying to make the host-based open-source router business model work. Stephen
Re: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.
On Tue, Jun 06, 2006 at 05:19:33PM +0200, Gunther Stammwitz wrote: Hallo colleagues, Maybe someone of you can help me to understand the phenomenon of pack loss when using asynchronous routing? I have customers who are complaining about packet loss and they are providing me with MTRs and pathpings (that's some sort of traceroute that pings every hop it sees several times - comes with windows xp) that show the loss starting at my routers and ending at their server (=the last hop). All users are coming from a (dialup-)network where the way from them to our servers are going via a carrier different than the carrier we are using to route the traffic back to the dial user. The interesting thing is that there is no loss at all when the users either use a ping instead of this pathping/mtr-stuff or when I perform a ping or even an mtr on my server in direction of the dialup customer. The nasty thing is that there is de facto NO LOSS on the line but the users is seeing some sort of phantom loss. The problem immediately disappears when I change to way back to the same carrier as the way to us so that we have synchronous routing again. My assumption is that pathping and mtr somehow get irritated by the icmp messages due to a wrong timing or something like that. Any ideas? I can't tell you what is going on. But I can ask, (a) why are you doing asymmetrical routing in the first place? and, (b) is it possible that the MicroSoft versions of these tools are reporting errors BECAUSE of the asynchronous routing? -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.
On 7-Jun-2006, at 12:35, Joseph S D Yao wrote: I can't tell you what is going on. But I can ask, (a) why are you doing asymmetrical routing in the first place? For any non-trivial path, it seems to me that asymmetry in forward and return paths is normal. Symmetrical paths are the exception. From another angle, how can anybody hope to ensure that all forward and return paths are identical when the only exit under their control is the one on the outbound path, at their own border? Joe
Re: Zebra/linux device production networking?
I'm running ImageStream routers for the Internet distribution side of my network (2 edge routers, 2 core routers) and I'm extremely happy... This is a datacenter network and my customers are happy, I guess that's all that counts. In my opinion, I prefer to go with a open-source based solution because of pricing and customizability... I can build a script and load it into the equipment to give me any type of statistic I want... And I don't have to wait for a new IOS release. JP On 6/7/06, Miquel van Smoorenburg [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], william(at)elan.net [EMAIL PROTECTED] wrote: you should be able to set linux that is secure as freebsd. There are some differences in the routing code whereas Linux is designed with per-flow based switching in mind (which works very well when used as a server) Nobody noticed, but Linux 2.6 has alternative FIB code you can select when compiling the kernel. Yes, it is fairly new and I'm not sure it is production quality, but still. The config option is IP_FIB_TRIE, for the LC-trie algorithm. It's supposed to be something like CEF. Mike. -- Justin W. Pauler Baton Rouge, LA
Re: Phantom packet loss is being shown when using pathping in connection with asynchronous routing - although there is no real loss.
On Wed, Jun 07, 2006 at 12:49:04PM -0700, Joe Abley wrote: On 7-Jun-2006, at 12:35, Joseph S D Yao wrote: I can't tell you what is going on. But I can ask, (a) why are you doing asymmetrical routing in the first place? For any non-trivial path, it seems to me that asymmetry in forward and return paths is normal. Symmetrical paths are the exception. From another angle, how can anybody hope to ensure that all forward and return paths are identical when the only exit under their control is the one on the outbound path, at their own border? Joe If this is for their customers, it wasn't clear that the path went outside their zone of control. I did wonder. -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: Zebra/linux device production networking?
On Wed, Jun 07, 2006 at 09:31:51PM +0530, Suresh Ramasubramanian wrote: On 6/7/06, Nick Burke [EMAIL PROTECTED] wrote: First, a little background.. My CTO made my stomach curdle today when he announced that he wanted to do away with all our cisco [routers] and instead use Linux/zebra boxen. This looks reasonable .. http://www.linux-vpn.de/lr101.php LEAF http://leaf.sourceforge.net/ and Coyote http://www.coyotelinux.com/ are often cited live branches off the Linux Router Project. -- Joe Yao --- This message is not an official statement of OSIS Center policies.
Re: Zebra/linux device production networking?
On Wed, 7 Jun 2006, Justin W. Pauler wrote: I'm running ImageStream routers for the Internet distribution side of my network (2 edge routers, 2 core routers) and I'm extremely happy... This is a datacenter network and my customers are happy, I guess that's all that counts. In my opinion, I prefer to go with a open-source based solution because of pricing and customizability... I can build a script and load it into the equipment to give me any type of statistic I want... And I don't have to wait for a new IOS release. Note that imagestream is the worst of both worlds. it is ghetto like opensores but you don't get the source to fix it yourself if vendor is not being helpful. -alex
Re: a fun hijack: 1/8, 2/8, 3/8, 4/8, 5/8, 7/8, 8/8, 12/8 briefly announced by AS 23520 (today)
At 01:58 PM 07-06-06 -0500, Gadi Evron wrote: On Wed, 7 Jun 2006, Hank Nussbacher wrote: On Wed, 7 Jun 2006, Josh Karlin wrote: I don't expect better from NW Network Cable, but I definitely expect better from Sprint (their upstream). But this hasn't been the first and most unfortunately, not the last, cuz, almost no one gives a f*ck anymore. Well, when all we do about a situation is b*tch about it, waste time running after other people's tails, and repeat the same process all over again - we can't really expect for anything to change. I have seen hijacks of 192.0.0.0/2 and 128.0.0.0/1 recently and no one seems to care much - no matter how many emails I may send out. 5% usually answer with who made you the net-police? I'll continue to send directed emails to those that have configuration errors in the hope that some (and some do) really do care. -Hank
