Re: [SLUG] attaching lots of disks to PowerEdge 860?
On Tue, 2009-09-29 at 21:46 +1000, Dean Hamstead wrote: > ATAoE is l2 protocol so no its not routable, but ATAoE is a published > standard and the drivers are in the kernel since 2.6.11. A published specification, not a published standard. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
Dean Hamstead writes: >> From what I hear they are proprietary and their technology is not >> routable. I prefer to stick to an open standard, preferably something which >> comes as part of CentOS 5. > > ATAoE is l2 protocol so no its not routable ...unless you package it in MPLS, or L2TP, or some other technology designed to route layer 2 packets; various people I know who use production ATAoE swear by this strategy. > but ATAoE is a published standard and the drivers are in the kernel since > 2.6.11. *nod* I won't offer an opinion on ATAoE myself, but I can say that concerns about the protocol compared to iSCSI are less than compelling to me. Now, having only a single vendor, that might worry a body. Daniel -- ✣ Daniel Pittman✉ dan...@rimspace.net☎ +61 401 155 707 ♽ made with 100 percent post-consumer electrons Looking for work? Love Perl? In Melbourne, Australia? We are hiring. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
Kevin Shackleton writes: > or there is: > http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ > > 45 discs, 67 TB, under US$8k, in one box on SATA port multipliers. 50% over > the cost of raw drives. I don't suppose blazing speed was their primary > goal, but security was up there. Yeah, but. Specifically, but, did you actually read about their plan for data integrity? They expect those storage things to fail left, right and center, and use higher level software to replicate data safely on multiple nodes. Not exactly the level of reliability you want if you only have a half-dozen of those things, let alone less. It works, in part, because "designed to fail" is really good at a large scale, but not so great at small scale. Daniel -- ✣ Daniel Pittman✉ dan...@rimspace.net☎ +61 401 155 707 ♽ made with 100 percent post-consumer electrons Looking for work? Love Perl? In Melbourne, Australia? We are hiring. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
From what I hear they are proprietary and their technology is not routable. I prefer to stick to an open standard, preferably something which comes as part of CentOS 5. ATAoE is l2 protocol so no its not routable, but ATAoE is a published standard and the drivers are in the kernel since 2.6.11. If routing iscsi is a good idea or not, is a totally different discussion :) Dean -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
2009/9/29 Dean Hamstead : > >> These servers have space for only two internal disks and I'd like to >> try to convert a couple of them into servers of shared storage. >> I'm thinking of just setting them up to sync their disks using DRBD >> and providing access to the rest of the network via iSCSI. > > I am not sure that you want to use it as an iSCSI host. This is a block > level device and isnt really (easily) shared between more than one client > system. Most likely you want NFS or CIFS/SMB. There are a few reasons but the main one is that I'm looking at moving our data centre to use iSCSI storage instead of the current arrangement (application servers sharing disks in pairs via DRBD) and would like to use same technology on our development network both to get familiar with it and to test the options, while taking advantage of the old(er) disks we have laying around for extra storage space. > External eSATA or USB devices is really the only low low cost option. I suppose so, of course the PowerEdge has USB ports and I suspect also even a built-in eSATA controller, but so far I didn't find a large box to connect to this eSATA port. > Using an external iSCSI or ATAoE (or even FCoE) is an option if the disk > array is the right price. But then most of them come with an NFS/CIFS head > on them anyway. Once there is a iSCSI target somewhere I assume I'll be able to share parts of it over NFS/CIFS by one (or a cluster of) iSCSI clients. > > Take a look at Coraid. These guys made up ATAoE and their disk arrays are > surprisingly cheap. http://www.coraid.com/ >From what I hear they are proprietary and their technology is not routable. I prefer to stick to an open standard, preferably something which comes as part of CentOS 5. > > > Keep in mind though, it may be cheaper still to just guy a cheap case with > lots of disk space (perhaps this behemoth > http://www.auspcmarket.com.au/show_product_info.php?input[product_code]=EL-PC-343B&input[category_id]=277 > or this much cheaper icute case, of which i have its little brother > http://www.mwave.com.au/newAU/mwaveAU/productdetail.asp?SKU=16010594) > > and then drop in a cheap CPU, a motherboard with a lot of sata connectors > and some cheap 4 port sata2 cards. Thanks! Both for the advise and the web sites. These looks like what I had in mind, or actually http://www.auspcmarket.com.au/index.php?redir=http://www.auspcmarket.com.au/show_product_info.php?input[product_code]=EL-PC-343B&input[category_id]=277 since I'm not sure how to connect to the LIAN LI PC-343B. I discussed this option with a co-worker and he suggested something like this too (using some of our decommissioned hardware). We have some MoBo's/CPU's/RAM laying around and such cases might do the trick. Cheers, --Amos -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
or there is: http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ 45 discs, 67 TB, under US$8k, in one box on SATA port multipliers. 50% over the cost of raw drives. I don't suppose blazing speed was their primary goal, but security was up there. Kevin. On Tue, 2009-09-29 at 16:26 +1000, pe...@chubb.wattle.id.au wrote: > > "Amos" == Amos Shapira writes: > > Amos> Is there another economical (and sane, speed-wise) way to get > Amos> lots of disks on these system's bus? > > > You could put in a couple of eSATA controllers and use boxes of disks > with SATA port multipliers in them. We're doing that; you have to > watch which port multiplier you use as some are a bit slow, but apart > from that it's an easy way to get lots of disc on a machine with just > a couple of spare PCI slots. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
On Tue, Sep 29, 2009, Dean Hamstead wrote: > Take a look at Coraid. These guys made up ATAoE and their disk arrays > are surprisingly cheap. http://www.coraid.com/ The last time I read the ATAoE spec, i found it had no real guarantees for data integrity or specifying how to handle packet loss on the ethernet segments. But then, I'm a stickler for this sort of thing.. :) > http://www.auspcmarket.com.au/show_product_info.php?input[product_code]=EL-PC-343B&input[category_id]=277 > or this much cheaper icute case, of which i have its little brother > http://www.mwave.com.au/newAU/mwaveAU/productdetail.asp?SKU=16010594) > > and then drop in a cheap CPU, a motherboard with a lot of sata > connectors and some cheap 4 port sata2 cards. I bought a pair of 5 disk SATA hotplug(ish!) boxes from auspcmarket for experimentation. They work fine enough. The trouble was finding a PC case with sufficient 5.25" halfheight bays which let you mount a > half height disk in it. ;) Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA - -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
These servers have space for only two internal disks and I'd like to try to convert a couple of them into servers of shared storage. I'm thinking of just setting them up to sync their disks using DRBD and providing access to the rest of the network via iSCSI. I am not sure that you want to use it as an iSCSI host. This is a block level device and isnt really (easily) shared between more than one client system. Most likely you want NFS or CIFS/SMB. External eSATA or USB devices is really the only low low cost option. Using an external iSCSI or ATAoE (or even FCoE) is an option if the disk array is the right price. But then most of them come with an NFS/CIFS head on them anyway. Take a look at Coraid. These guys made up ATAoE and their disk arrays are surprisingly cheap. http://www.coraid.com/ Keep in mind though, it may be cheaper still to just guy a cheap case with lots of disk space (perhaps this behemoth http://www.auspcmarket.com.au/show_product_info.php?input[product_code]=EL-PC-343B&input[category_id]=277 or this much cheaper icute case, of which i have its little brother http://www.mwave.com.au/newAU/mwaveAU/productdetail.asp?SKU=16010594) and then drop in a cheap CPU, a motherboard with a lot of sata connectors and some cheap 4 port sata2 cards. Dean -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
On Mon, Sep 28, 2009 at 09:46:01PM +1000, Amos Shapira wrote: > Hello, > > We have a few PowerEdge 860's which I mostly converted from our > previous-generation Windows based system to Xen hosts for development > on top of CentOS 5 (the production system is now mostly hosted > abroad). > > These servers have space for only two internal disks and I'd like to > try to convert a couple of them into servers of shared storage. > I'm thinking of just setting them up to sync their disks using DRBD > and providing access to the rest of the network via iSCSI. > > I'm looking for a way to attach lots of disks to them. So far the only > options I found are MD1000/3000 from Dell, which are a bit too > expensive for such a side-project. > > Is there another economical (and sane, speed-wise) way to get lots of > disks on these system's bus? > > Thanks, How sane does the speed have to be exactly? :-) If the answer is not completely insanse then you could add a lot of esata/usb disks externally. If you run out of sata or usb, add more via pci cards. You might not get sata speeds as you might saturate the pci bus well before hitting the sata limit. If you want a little more speed (well, lower latency perhaps) at the cost of reliability you could raid0 them. BTW, an alternative to iSCSI might be ATA over ethernet. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] attaching lots of disks to PowerEdge 860?
> "Amos" == Amos Shapira writes: Amos> Is there another economical (and sane, speed-wise) way to get Amos> lots of disks on these system's bus? You could put in a couple of eSATA controllers and use boxes of disks with SATA port multipliers in them. We're doing that; you have to watch which port multiplier you use as some are a bit slow, but apart from that it's an easy way to get lots of disc on a machine with just a couple of spare PCI slots. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] attaching lots of disks to PowerEdge 860?
Hello, We have a few PowerEdge 860's which I mostly converted from our previous-generation Windows based system to Xen hosts for development on top of CentOS 5 (the production system is now mostly hosted abroad). These servers have space for only two internal disks and I'd like to try to convert a couple of them into servers of shared storage. I'm thinking of just setting them up to sync their disks using DRBD and providing access to the rest of the network via iSCSI. I'm looking for a way to attach lots of disks to them. So far the only options I found are MD1000/3000 from Dell, which are a bit too expensive for such a side-project. Is there another economical (and sane, speed-wise) way to get lots of disks on these system's bus? Thanks, --Amos -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
