Re: Network Storage
Andrew Thrift writes: If you want something from a Tier1 the new Dell R720XD's will take 24x 900GB SAS disks or 12x 2TB 3.5 cheap slow SATA disks or 12x 3TB 3.5 more expensive slightly faster SAS disks - if you take the (cheaper) 3.5-disk variant of the R720xd chassis. or 12x 3TB 3.5 cheapslow SATA disks if you buy them directly rather than from Dell. (Presumably you'd have to buy Dell hot-swap trays) -- Simon. and have 16 cores. If you order it with a SAS6-HBA you can add up to 8 trays of 24 x 900GB SAS disks to provide 194TB of raw space at quite a reasonable cost.
RE: Most energy efficient (home) setup
From: Joe Greco [mailto:jgr...@ns.sol.net] I'd have to say that that's been the experience here as well, ECC is great, yes, but it just doesn't seem to be something that is absolutely vital on an ongoing basis, as some of the other posters here have implied, to correct the constant bit errors that are(n't) showing up. Maybe I'll get bored one of these days and find some devtools to stick on one of the Macs. In all the years I've been playing with high end hardware, the best sample machine I have is an SGI Origin 200 that I had in production for over ten years, with the only downtime during that time being once to add more memory, once to replace a failed drive, once to move the rack and the occasional OS upgrade (I tended to skip a 6.5.x release or two between updates, and after 6.5.30 there were of course no more). That machine was down less than 24 hours cumulative for that entire period. In that ten year span, I saw TWO ECC parity errors (both single bit correctable). On any machine that saw regular ECC errors it was a sign of failing hardware (usually, but not necessarily the memory, there are other parts in there that have to carry that data too). As much as I prefer ECC, it's not a show stopper for me if it's not there. Jamie
Re: Most energy efficient (home) setup
In a message written on Sun, Apr 15, 2012 at 09:54:14PM -0400, Luke S. Crawford wrote: On my current fleet (well under 100 servers) single bit errors are so rare that if I get one, I schedule that machine for removal from production. In a previous life, in a previous time, I worked at a place that had a bunch of Cisco's with parity RAM. For the time, these boxes had a lot of RAM, as they had distributed line cards each with their own processor memory. Cisco was rather famous for these parity errors, mostly because of their stock answer: sunspots. The answer was in fact largely correct, but it's just not a great response from a vendor. They had a bunch of statistics though, collected from many of these deployed boxes. We ran the statistics, and given hundreds of routers, each with many line cards the math told us we should have approximately 1 router every 9-10 months get one parity error from sunspots and other random activity (e.g. not a failing RAM module with hundreds of repeatable errors). This was, in fact, close to what we observed. This experience gave me two takeaways. First, single bit flips are rare, but when you have enough boxes rare shows up often. It's very similar to anyone with petabytes of storage, disks fail every couple of days because you have so many of them. At the same time a home user might not see a failure in their lifetime (of disk or memory). Second though, if you're running a business, ECC is a must because the message is so bad. This was caused by sunspots is not a customer inspiring response, no matter how correct. We could have prevented this by spending an extra $50 on proper RAM for your $1M box is even worse. Some quick looking at Newegg, 4GB DDR3 1333 ECC DIMM, $33.99. 4GB DDR3 1333 Non-ECC DIMM, $21.99. Savings, $12. (Yes, I realize the Motherboard also needs some extra circuitry, I expect it's less than $1 in quantity though). Pretty much everyone I know values their data at more than $12 if it is lost. -- Leo Bicknell - bickn...@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ pgpxNc88GvYD9.pgp Description: PGP signature
Re: Most energy efficient (home) setup
Some quick looking at Newegg, 4GB DDR3 1333 ECC DIMM, $33.99. 4GB DDR3 1333 Non-ECC DIMM, $21.99. Savings, $12. (Yes, I realize the Motherboard also needs some extra circuitry, I expect it's less than $1 in quantity though). Pretty much everyone I know values their data at more than $12 if it is lost. The problem is that if you want to move past the 4GB modules, things can get expensive. Bearing in mind the subject line, consider for example the completely awesome Intel Sandy Bridge E3-1230 with a board like the Supermicro X9SCL+-F, which can be built into a low power system that idles around 45W if you're careful. Problem is, the 8GB modules tend to cost an arm and a leg; http://www.google.com/products/catalog?q=MEM-DR380L-CL01-EU13oe=utf-8rls=org.mozilla:en-US:officialclient=firefox-aum=1hl=enbav=on.2,or.r_gc.r_pw.r_qf.,cf.osbbiw=1043bih=976ie=UTF-8tbm=shopcid=8556948603121267780sa=Xei=HxmMT5btB8_PgAfLs5TvCQved=0CD8Q8wIwAA to outfit a machine with 32GB several months ago cost around *$400* per module, or $1600 for the machine, whereas the average cost for a 4GB module was only around $30. So then you start looking at the less expensive options. When the average going price for 8GB non-ECC modules is between $50 and $100, then you're only looking at a cost premium of $1200 for ECC. For $1200, I'm willing to at least consider non-ECC. You can infer from this message that I'm actually waiting for more reasonable ECC prices to show up; we're finally seeing somewhat more reasonable prices, but by that I mean only around $130/8GB. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Communal Dining
Folks, You are all invited to an extremely informal dinner at our house at 6PM on Saturday, April 21. Spouses and children are all invited. I will bake bread and put on a huge pot of soup. If your kids are picky eaters, feel free to bring whatever they will eat. Our house is located at: 241 West Meadowland Lane Sterling, Virgina 20164 703 430 8379 -- Ron and Nancy Bonica vcard: www.bonica.org/ron/ronbonica.vcf
FW: Communal Dining
Folks, Sorry, you are not all invited to dinner. I apologize for the spam. MS mail address completion helped me a little more than I wanted. Ron -Original Message- From: Ronald Bonica Sent: Monday, April 16, 2012 10:05 AM To: 'frbi...@aol.com'; 'Nicholas Hinko'; 'Susan Hinko'; jay cuasay; 'William Richey'; Will Ress; 'maria torres'; 'landre...@gmail.com'; nanog@nanog.org Subject: Communal Dining Folks, You are all invited to an extremely informal dinner at our house at 6PM on Saturday, April 21. Spouses and children are all invited. I will bake bread and put on a huge pot of soup. If your kids are picky eaters, feel free to bring whatever they will eat. Our house is located at: 241 West Meadowland Lane Sterling, Virgina 20164 703 430 8379 -- Ron and Nancy Bonica vcard: www.bonica.org/ron/ronbonica.vcf
Re: FW: Communal Dining
Shoot I was half way there already! :-) On Mon, Apr 16, 2012 at 10:11:44AM -0400, Ronald Bonica wrote: Folks, Sorry, you are not all invited to dinner. I apologize for the spam. MS mail address completion helped me a little more than I wanted. Ron -Original Message- From: Ronald Bonica Sent: Monday, April 16, 2012 10:05 AM To: 'frbi...@aol.com'; 'Nicholas Hinko'; 'Susan Hinko'; jay cuasay; 'William Richey'; Will Ress; 'maria torres'; 'landre...@gmail.com'; nanog@nanog.org Subject: Communal Dining Folks, You are all invited to an extremely informal dinner at our house at 6PM on Saturday, April 21. Spouses and children are all invited. I will bake bread and put on a huge pot of soup. If your kids are picky eaters, feel free to bring whatever they will eat. Our house is located at: 241 West Meadowland Lane Sterling, Virgina 20164 703 430 8379 -- Ron and Nancy Bonica vcard: www.bonica.org/ron/ronbonica.vcf
RE: Communal Dining
Is this going to be like when teenagers advertise their parties on facebook? -Original Message- From: Ronald Bonica [mailto:rbon...@juniper.net] Sent: 16 April 2012 15:09 To: frbi...@aol.com; Nicholas Hinko; Susan Hinko; jay cuasay; William Richey; Will Ress; maria torres; landre...@gmail.com; nanog@nanog.org Subject: Communal Dining Folks, You are all invited to an extremely informal dinner at our house at 6PM on Saturday, April 21. Spouses and children are all invited. I will bake bread and put on a huge pot of soup. If your kids are picky eaters, feel free to bring whatever they will eat. Our house is located at: 241 West Meadowland Lane Sterling, Virgina 20164 703 430 8379 -- Ron and Nancy Bonica vcard: www.bonica.org/ron/ronbonica.vcf __ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com __ __ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com __
RE: Network Storage
I'd like to point out that you can actually do 26 2.5 disks on an R720xd if you use the flexbay +1 SD card for your os install if you're being a maximalist. =) -Drew -Original Message- From: Simon Leinen [mailto:simon.lei...@switch.ch] Sent: Monday, April 16, 2012 5:38 AM To: Andrew Thrift Cc: nanog@nanog.org Subject: Re: Network Storage Andrew Thrift writes: If you want something from a Tier1 the new Dell R720XD's will take 24x 900GB SAS disks or 12x 2TB 3.5 cheap slow SATA disks or 12x 3TB 3.5 more expensive slightly faster SAS disks - if you take the (cheaper) 3.5-disk variant of the R720xd chassis. or 12x 3TB 3.5 cheapslow SATA disks if you buy them directly rather than from Dell. (Presumably you'd have to buy Dell hot-swap trays) -- Simon. and have 16 cores. If you order it with a SAS6-HBA you can add up to 8 trays of 24 x 900GB SAS disks to provide 194TB of raw space at quite a reasonable cost.
RE: Communal Dining
There used to be a modification to the WWIV BBS software that when you entered the 'boards' section (wow I am so old, by the way) it would display 'Party at my house' and show all of the user's information in it's best ascii representation; of course it showed only that user's information to themselves so it looked like everyone was having a party, many complaints were filed =) -Original Message- From: Leigh Porter [mailto:leigh.por...@ukbroadband.com] Sent: Monday, April 16, 2012 11:05 AM To: Ronald Bonica; nanog@nanog.org Subject: RE: Communal Dining Is this going to be like when teenagers advertise their parties on facebook? -Original Message- From: Ronald Bonica [mailto:rbon...@juniper.net] Sent: 16 April 2012 15:09 To: frbi...@aol.com; Nicholas Hinko; Susan Hinko; jay cuasay; William Richey; Will Ress; maria torres; landre...@gmail.com; nanog@nanog.org Subject: Communal Dining Folks, You are all invited to an extremely informal dinner at our house at 6PM on Saturday, April 21. Spouses and children are all invited. I will bake bread and put on a huge pot of soup. If your kids are picky eaters, feel free to bring whatever they will eat. Our house is located at: 241 West Meadowland Lane Sterling, Virgina 20164 703 430 8379 -- Ron and Nancy Bonica vcard: www.bonica.org/ron/ronbonica.vcf __ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com __ __ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com __
Automatic IPv6 due to broadcast
Hello everyone Just got a awfully crazy issue. I heard from our support team about failure of whois during domain registration. Initially I thought of port 43 TCP block or something but found it was all ok. Later when ran whois manually on server via terminal it failed. Found problem that server was connecting to whois server - whois.verisign-grs.com. I was stunned! Server got IPv6 and not just that one - almost all. This was scary - partial IPv6 setup and it was breaking things. In routing tables, routes were all going to a router which I recently setup for testing. That router and other servers are under same switch but by no means I ever configured that router as default gateway for IPv6. I found option of broadcast was enabled on router for local fe80... address and I guess router broadcasted IPv6 and somehow (??) all servers found that they have a IPv6 router on LAN and started using it - automated DHCP IPv6? I wonder if anyone else also had similar issues? Also, if my guesses are correct then how can we disable Red Hat distro oriented servers from taking such automated configuration - simple DHCP in IPv6 disable? Thanks -- Anurag Bhatia anuragbhatia.com or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected network! Twitter: @anurag_bhatia https://twitter.com/#!/anurag_bhatia Linkedin: http://linkedin.anuragbhatia.com
Re: Most energy efficient (home) setup
Have you looked at the HP ProLiant MicroServer? Cheers, Henk On 13-04-12 12:06, Jeroen van Aart wrote: Leo Bicknell wrote: But what's really missing is storage management. RAID5 (and similar) require all drives to be online all the time. I'd love an intelligent file system that could spin down drives when not in use, and even for many workloads spin up only a portion of the drives. It's easy to imagine a system with a small SSD and a pair of disks. Reads spin one disk. Writes go to that disk and the SSD until there are enough, which spins up the second drive and writes them out as a proper mirror. In a home file server drive motors, time you have 4-6 drives, eat most of the power. CPU's speed step down nicely, drives don't. Late reply by me, but excellent points. A combination of mdadm and hdparm on linux should suffice to have a raid that will spin down the disks when not in use. I have used for years a G4 system with a mdadm raid1 (and a separate boot disk) and hdparm configured to spin the raid disks down after 10 minutes and it worked great. I think in a raid10 this would only spin up the disk pair that has the data you need, but leave the rest asleep. But I didn't try that yet. What I'd like is to have small disk enclosuer that includes a whole (low power) computer capable of having linux installed on some flash memory. Say you have an enclosure with space for 4 2.5 inch disks, install linux, set it up as a raid10, connect through USB to your computer for back up purposes. Greetings, Jeroen
RE: Automatic IPv6 due to broadcast
To completely disable ipv6 in Redhat: 1) Modify /etc/modprobe.conf (add) alias ipv6 off alias net-pf-10 off options ipv6 disable=1 2) Modify /etc/sysconfig/network (add) NETWORKING_IPV6=no I usually also add NOZEROCONF=yes That should completely disable ipv6 in Redhat 5.x Matthew Huff | 1 Manhattanville Rd Director of Operations | Purchase, NY 10577 OTA Management LLC | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 -Original Message- From: Anurag Bhatia [mailto:m...@anuragbhatia.com] Sent: Monday, April 16, 2012 2:10 PM To: NANOG Mailing List Subject: Automatic IPv6 due to broadcast Hello everyone Just got a awfully crazy issue. I heard from our support team about failure of whois during domain registration. Initially I thought of port 43 TCP block or something but found it was all ok. Later when ran whois manually on server via terminal it failed. Found problem that server was connecting to whois server - whois.verisign-grs.com. I was stunned! Server got IPv6 and not just that one - almost all. This was scary - partial IPv6 setup and it was breaking things. In routing tables, routes were all going to a router which I recently setup for testing. That router and other servers are under same switch but by no means I ever configured that router as default gateway for IPv6. I found option of broadcast was enabled on router for local fe80... address and I guess router broadcasted IPv6 and somehow (??) all servers found that they have a IPv6 router on LAN and started using it - automated DHCP IPv6? I wonder if anyone else also had similar issues? Also, if my guesses are correct then how can we disable Red Hat distro oriented servers from taking such automated configuration - simple DHCP in IPv6 disable? Thanks -- Anurag Bhatia anuragbhatia.com or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected network! Twitter: @anurag_bhatia https://twitter.com/#!/anurag_bhatia Linkedin: http://linkedin.anuragbhatia.com smime.p7s Description: S/MIME cryptographic signature
Re: Automatic IPv6 due to broadcast
On Mon, 16 Apr 2012 23:39:46 +0530, Anurag Bhatia said: More a host config issue than a NANOG issue, but what the heck... I wonder if anyone else also had similar issues? Also, if my guesses are correct then how can we disable Red Hat distro oriented servers from taking such automated configuration - simple DHCP in IPv6 disable? The *right* answer is, of course, to hurry up and deploy proper IPv6. It sounds like the distro is shipping some apps that don't do happy-eyeballs yet - you probably want to file bug reports about that. (It's easy enough to disable IPv6 if you haven't deployed it - in /etc/sysconfig/network-scripts/ifcfg-interface add: IPV6INIT=no then ifdown/ifup the interface and you should be good to go. Make sure you remember to take that line out when you get around to deploying IPv6. (The difference between this and Matthew Huff's suggestion is that this way, it disables IPv6 on the interface, and leaves the ::1 loopback address in place.) pgpFnOWIYX75j.pgp Description: PGP signature
Re: Most energy efficient (home) setup
On Mon, Apr 16, 2012 at 11:22:20AM -0700, Henk Hesselink wrote: Have you looked at the HP ProLiant MicroServer? Notice it takes up to 8 GByte ECC memory and supports zfs via napp-it/Illumos. A hacked BIOS was required to use the 5th internal SATA port in AHCI mode, maybe that's no longer necessary with N40L.
Re: Automatic IPv6 due to broadcast
Anurag, You have a rogue RA in your network. Now is just an annoying DoS, but it can easily be turned in a real security concern. I suggest to either deploy properly IPv6 or disable it. I am more on the former, but it is your choice. Regards -as On 16 Apr 2012, at 15:09, Anurag Bhatia wrote: Hello everyone Just got a awfully crazy issue. I heard from our support team about failure of whois during domain registration. Initially I thought of port 43 TCP block or something but found it was all ok. Later when ran whois manually on server via terminal it failed. Found problem that server was connecting to whois server - whois.verisign-grs.com. I was stunned! Server got IPv6 and not just that one - almost all. This was scary - partial IPv6 setup and it was breaking things. In routing tables, routes were all going to a router which I recently setup for testing. That router and other servers are under same switch but by no means I ever configured that router as default gateway for IPv6. I found option of broadcast was enabled on router for local fe80... address and I guess router broadcasted IPv6 and somehow (??) all servers found that they have a IPv6 router on LAN and started using it - automated DHCP IPv6? I wonder if anyone else also had similar issues? Also, if my guesses are correct then how can we disable Red Hat distro oriented servers from taking such automated configuration - simple DHCP in IPv6 disable? Thanks -- Anurag Bhatia anuragbhatia.com or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected network! Twitter: @anurag_bhatia https://twitter.com/#!/anurag_bhatia Linkedin: http://linkedin.anuragbhatia.com
Re: Most energy efficient (home) setup
On Mon, Apr 16, 2012 at 11:22:20AM -0700, Henk Hesselink wrote: Have you looked at the HP ProLiant MicroServer? Notice it takes up to 8 GByte ECC memory and supports zfs via napp-it/Illumos. A hacked BIOS was required to use the 5th internal SATA port in AHCI mode, maybe that's no longer necessary with N40L. The MicroServer is actually a nice little platform, one little bright spot in the small-home-server market. It does have some other issues though: 1) It's not particularly low-power, as in, I managed to build some Xeon based systems that run rings around it for only maybe a dozen watts more, and some of the NAShead guys over at one of the Linux based projects have a similar but lower-power platform for a lower price, 2) While it has a remote management card available, it's known to not work with certain things, including FreeBSD, 3) Various problems noted with the eSATA port, such as the inability to use an external port multiplier. On the flip side, some people have tossed one of those 4-2.5-in-a-5.25 bay racks into the optical bay, along with a PCI controller, to allow the addition of SSD's or whatever for NAS use. Pretty cool and the thing *is* pretty compact. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Automatic IPv6 due to broadcast
I know you mentioned RedHat, but not if it was the router or other servers. Were you playing with Microsoft's Direct Access and turn on the dns entry (isatap.domain.com) internally? At my current place of employment, we had a security student (at the direction of our security analyst) turn up a DA test server. When they enabled the DNS entry, just about every Windows 7 and 2008 server setup a v6 tunnel back to this little tiny VM. This also included the DNS entries in AD, so all of the sudden, servers have v6 addresses. Needless to say, everything was horribly slow, and some things even flat out broke. Sadly this event left a really sour taste for IPv6 with Networking department (whom I was occasionally bugging about v6). If you weren't testing this, did you possibly setup something similar where it would automatically generate a tunnel? Brandon Penglase On Mon, 16 Apr 2012 23:39:46 +0530 Anurag Bhatia m...@anuragbhatia.com wrote: Hello everyone Just got a awfully crazy issue. I heard from our support team about failure of whois during domain registration. Initially I thought of port 43 TCP block or something but found it was all ok. Later when ran whois manually on server via terminal it failed. Found problem that server was connecting to whois server - whois.verisign-grs.com. I was stunned! Server got IPv6 and not just that one - almost all. This was scary - partial IPv6 setup and it was breaking things. In routing tables, routes were all going to a router which I recently setup for testing. That router and other servers are under same switch but by no means I ever configured that router as default gateway for IPv6. I found option of broadcast was enabled on router for local fe80... address and I guess router broadcasted IPv6 and somehow (??) all servers found that they have a IPv6 router on LAN and started using it - automated DHCP IPv6? I wonder if anyone else also had similar issues? Also, if my guesses are correct then how can we disable Red Hat distro oriented servers from taking such automated configuration - simple DHCP in IPv6 disable? Thanks -- Anurag Bhatia anuragbhatia.com or simply - http://[2001:470:26:78f::5] if you are on IPv6 connected network! Twitter: @anurag_bhatia https://twitter.com/#!/anurag_bhatia Linkedin: http://linkedin.anuragbhatia.com
Re: Automatic IPv6 due to broadcast
On Mon, 16 Apr 2012 17:38:07 -0400, Brandon Penglase said: flat out broke. Sadly this event left a really sour taste for IPv6 with Networking department (whom I was occasionally bugging about v6). Talking point: If you guys had deployed a proper IPv6 infrastructure, those tunnels wouldn't have happened and there would have been no problem. The best defense against a user accidentally deploying some infrastructure incorrectly is to do a pre-emptive correct deployment. pgpUuBOw74Kn8.pgp Description: PGP signature
