Re: CPU RAM in a storage box

2010-10-16 Thread Maxim Veksler
Not really sure what's the current topic of this long lasting discussion is
but if it's backup we're talking about I don't think any service can beat S3
in pricing, S3 reduced storage even cheaper, see pricing
http://aws.amazon.com/s3/#pricing

http://aws.amazon.com/s3/#pricingAnd you have great tools to work with the
service

   - http://code.google.com/p/s3fs/
   - http://s3tools.org/s3cmd
   - http://sourceforge.net/projects/elasticfox/
   - and some proprietary like bucket explorer and co.

HTH

Maxim.

On Tue, Sep 28, 2010 at 5:28 PM, Nadav Har'El n...@math.technion.ac.ilwrote:

 On Tue, Sep 28, 2010, Nadav Har'El wrote about Re: CPU  RAM in a storage
 box:
  Thanks for the recommendation, I'll look into it. Their price is a bit of
  a turn-off, though - at $0.80 GB/mo, backing up 20 GB costs $16 a month,
 ..
  I wonder what is rsync.net's discount for open source developers ;-)

 I asked them, and the answer is that a free software developer, I get a 50%
 discount, i.e, 40 cents per GB per month. There's still a minimum of $5.6
 a month, for which I get to store 14 GB.

 --
 Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri
 5771
 n...@math.technion.ac.il
 |-
 Phone +972-523-790466, ICQ 13349191 |Cement mixer collided with a prison
 van.
 http://nadav.harel.org.il   |Look out for sixteen hardened
 criminals.

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Nadav Har'El
On Mon, Sep 27, 2010, Michael Tewner wrote about Re: CPU  RAM in a storage 
box:
 Have you seen this?
...
  At Backblaze, we provide unlimited storage to our customers for only $5 per
  month, so we had to figure out how to store hundreds of petabytes of
...

Looking at their site, it appears that while their systems run on Linux,
they don't give service to Linux machines. Is that true? Is there a similar
online backup service which does support Linux? I'd love to use such a service
for my home Linux computer.

Nadav.

-- 
Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri 5771
n...@math.technion.ac.il |-
Phone +972-523-790466, ICQ 13349191 |This box was intentionally left blank.
http://nadav.harel.org.il   |

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Ori Idan
I think dropbox.com can be used as a backup system to Linux.
It has a daemon for Linux, however the daemon itself is propriatry, the
GNOME Nautilus extension is not.

-- 
Ori Idan


On Tue, Sep 28, 2010 at 8:51 AM, Nadav Har'El n...@math.technion.ac.ilwrote:

 On Mon, Sep 27, 2010, Michael Tewner wrote about Re: CPU  RAM in a
 storage box:
  Have you seen this?
 ...
   At Backblaze, we provide unlimited storage to our customers for only $5
 per
   month, so we had to figure out how to store hundreds of petabytes of
 ...

 Looking at their site, it appears that while their systems run on Linux,
 they don't give service to Linux machines. Is that true? Is there a similar
 online backup service which does support Linux? I'd love to use such a
 service
 for my home Linux computer.

 Nadav.

 --
 Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri
 5771
 n...@math.technion.ac.il
 |-
 Phone +972-523-790466, ICQ 13349191 |This box was intentionally left blank.
 http://nadav.harel.org.il   |

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


RE: CPU RAM in a storage box

2010-09-28 Thread ronys
DropBox is nice for what it does, but I'm not sure I'd consider them a
backup service.
 
For Linux-friendliness, I don't think that you can find something better
than  http://rsync.net/ http://rsync.net/, but I'd be happy to be proven
wrong. Note that they have discounts for Open Source developers. Disclaimer:
I have never used their products, and I am not associated with them in any
way.
 
Rony

  _  

From: linux-il-boun...@cs.huji.ac.il [mailto:linux-il-boun...@cs.huji.ac.il]
On Behalf Of Ori Idan
Sent: Tuesday, September 28, 2010 9:24 AM
To: Nadav Har'El
Cc: linux-il
Subject: Re: CPU  RAM in a storage box


I think dropbox.com can be used as a backup system to Linux.
It has a daemon for Linux, however the daemon itself is propriatry, the
GNOME Nautilus extension is not.

-- 
Ori Idan



On Tue, Sep 28, 2010 at 8:51 AM, Nadav Har'El n...@math.technion.ac.il
wrote:


On Mon, Sep 27, 2010, Michael Tewner wrote about Re: CPU  RAM in a storage
box:
 Have you seen this?
...

  At Backblaze, we provide unlimited storage to our customers for only $5
per
  month, so we had to figure out how to store hundreds of petabytes of

...

Looking at their site, it appears that while their systems run on Linux,
they don't give service to Linux machines. Is that true? Is there a similar
online backup service which does support Linux? I'd love to use such a
service
for my home Linux computer.

Nadav.

--
Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri
5771
n...@math.technion.ac.il
|-
Phone +972-523-790466, ICQ 13349191 |This box was intentionally left blank.
http://nadav.harel.org.il   |


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il



___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Hetz Ben Hamo
Hi,

Not that I want to open a flame war, but.. $.80 per gigabyte? even I (as 
hetz.biz, my business) sell equivalent service for less then that price,
and my servers are located in Israel. They brag about international, but
looking at their FAQ, they do a backup to some international location, so as
a user, if you're connected from Israel or you're connecting from US, you'll
be connected to the same server, with all the latencies etc..

Few days ago I wrote about this a post in my business blog, you can read it
here: http://blog.hetz.biz/?p=50

Hetz

2010/9/28 ronys ro...@gmx.net

  DropBox is nice for what it does, but I'm not sure I'd consider them a
 backup service.

 For Linux-friendliness, I don't think that you can find something better
 than http://rsync.net/, but I'd be happy to be proven wrong. Note that
 they have discounts for Open Source developers. Disclaimer: I have never
 used their products, and I am not associated with them in any way.

 Rony

  --
 *From:* linux-il-boun...@cs.huji.ac.il [mailto:
 linux-il-boun...@cs.huji.ac.il] *On Behalf Of *Ori Idan
 *Sent:* Tuesday, September 28, 2010 9:24 AM
 *To:* Nadav Har'El
 *Cc:* linux-il
 *Subject:* Re: CPU  RAM in a storage box

  I think dropbox.com can be used as a backup system to Linux.
 It has a daemon for Linux, however the daemon itself is propriatry, the
 GNOME Nautilus extension is not.

 --
 Ori Idan


 On Tue, Sep 28, 2010 at 8:51 AM, Nadav Har'El n...@math.technion.ac.ilwrote:

 On Mon, Sep 27, 2010, Michael Tewner wrote about Re: CPU  RAM in a
 storage box:
  Have you seen this?
 ...
   At Backblaze, we provide unlimited storage to our customers for only
 $5 per
   month, so we had to figure out how to store hundreds of petabytes of
 ...

 Looking at their site, it appears that while their systems run on Linux,
 they don't give service to Linux machines. Is that true? Is there a
 similar
 online backup service which does support Linux? I'd love to use such a
 service
 for my home Linux computer.

 Nadav.

 --
 Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri
 5771
 n...@math.technion.ac.il
 |-
 Phone +972-523-790466, ICQ 13349191 |This box was intentionally left
 blank.
 http://nadav.harel.org.il   |

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il



 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il




-- 

*חץ בן חמו
חץ-ביז (הוסטינג)
*השכרה ואירוח של שרתים פיזיים
השכרת שרתים וירטואליים מקצועיים וגדולים במחירים *קטנים*
בקרו באתרנו בכתובת hetz.biz http://www.hetz.biz/ ובבלוג שלנו:
blog.hetz.biz
טלפוןן: 078113/4/5, אימייל: sa...@hetz.biz
מסנג'ר: sa...@hetz.biz - סקייפ: heunique
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Nadav Har'El
On Tue, Sep 28, 2010, ronys wrote about RE: CPU  RAM in a storage box:
 For Linux-friendliness, I don't think that you can find something better
 than  http://rsync.net/ http://rsync.net/, but I'd be happy to be proven
 wrong. Note that they have discounts for Open Source developers. Disclaimer:
 I have never used their products, and I am not associated with them in any
 way.

Thanks for the recommendation, I'll look into it. Their price is a bit of
a turn-off, though - at $0.80 GB/mo, backing up 20 GB costs $16 a month,
while the same amount of storage on Google (where I back up all my digital
photos) costs me 40 cents a month. With the other service mentioned earlier,
the price is $5 per month for unlimited storage, so in theory I could back
up a terabyte of movies for the same price (of course, it would probably take
a year to upload a terabyte ;-)).

I wonder what is rsync.net's discount for open source developers ;-)

Maybe I should back up my photos to one place (Picasa Web, at $5 per 20GB per
year and a fantastic web interface to view and share these backed up photos)
and the rest of the content - not more than 7 GB (rsync.net's minimum) to
rsync.net...

-- 
Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri 5771
n...@math.technion.ac.il |-
Phone +972-523-790466, ICQ 13349191 |May you live as long as you want - and
http://nadav.harel.org.il   |never want as long as you live.

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread geoffrey mendelson


On Sep 28, 2010, at 1:18 PM, Nadav Har'El wrote:


 so in theory I could back
up a terabyte of movies for the same price (of course, it would  
probably take

a year to upload a terabyte ;-)).



If my arithmetic and assumptions are correct, you can upload a  
megabyte in 8 seconds with an 800k bits per second NGN line.
This comes out to around 3 hours a gigabyte or 8 gigabytes a day.  
Multiply that out and a terabyte would take 128 days (slightly over 4  
months).


Since this does not include protocol overhead, retansmissions, network  
slow downs, etc, a year would IMHO be a reasonable estimate.


Geoff.

--
Geoffrey S. Mendelson,  N3OWJ/4X1GM
To help restaurants, as part of the stimulus package, everyone must  
order dessert. As part of the socialized health plan, you are  
forbidden to eat it. :-)









___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Ori Berger

Nadav Har'El wrote:

At Backblaze, we provide unlimited storage to our customers for only $5 per
month, so we had to figure out how to store hundreds of petabytes of

Looking at their site, it appears that while their systems run on Linux,
they don't give service to Linux machines. Is that true? Is there a similar
online backup service which does support Linux? I'd love to use such a service
for my home Linux computer.


Not quite as cheap, but http://www.tarsnap.com/ is 30c/(compressed GB) 
storage or bandwidth, and support Linux. windows support is exclusively 
through Cygwin.


(Not a user or affiliated in any way; can't comment on how good/bad they 
are)


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-28 Thread Nadav Har'El
On Tue, Sep 28, 2010, Nadav Har'El wrote about Re: CPU  RAM in a storage box:
 Thanks for the recommendation, I'll look into it. Their price is a bit of
 a turn-off, though - at $0.80 GB/mo, backing up 20 GB costs $16 a month,
..
 I wonder what is rsync.net's discount for open source developers ;-)

I asked them, and the answer is that a free software developer, I get a 50%
discount, i.e, 40 cents per GB per month. There's still a minimum of $5.6
a month, for which I get to store 14 GB.

-- 
Nadav Har'El| Tuesday, Sep 28 2010, 20 Tishri 5771
n...@math.technion.ac.il |-
Phone +972-523-790466, ICQ 13349191 |Cement mixer collided with a prison van.
http://nadav.harel.org.il   |Look out for sixteen hardened criminals.

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-27 Thread Michael Tewner
Have you seen this?

http://www.backblaze.com/petabytes-on-a-budget-how-to-build-cheap-cloud-storage.html

At Backblaze, we provide unlimited storage to our customers for only $5 per
 month, so we had to figure out how to store hundreds of petabytes of
 customer data in a reliable, scalable way—and keep our costs low. After
 looking at several overpriced commercial solutions, we decided to build our
 own custom Backblaze Storage Pods: 67 terabyte 4U servers for $7,867.


Cute.

-Mike


2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi people,
 I'm planning to add some big storage solution to my VPS 
 businesshttp://hetz.biz.
 I did some checking and calculated the costs, and figured out that if I want
 to have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).

 I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and how
 much RAM should I put in such a machine? Xeon is overkill IIRC.

 What do you suggest?

 Thanks and Shana Tova
 Hetz

 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-10 Thread Erez D
here is my 3 cents:

In two companies i worked for, i designed similiar servers.

1. i wanted to have as less as a down time, and i didn't want to buy another
server just to sit and wait for a failure, so i decided it should work on
any pc with any raid controller - i decided to do the raid in software, so
it will be hardware independant - in case of hardware failure - just get any
pc with any raid controller ( the disks may even be connected via usb if you
do not have a raid controller) and everything is up in minutes.

2. I wanted to have snapshots like in a net-app - making snapshots every
hour or so. using the most up to date redhat distributions of that time -
when creating a snapshot, once in a while the server got locked (dead).
i tried it again when LVM2 was officially released, and got similiar
results. so as they say once bitten, twice shy - I will take a lot of
proof LVM is stable enough for snapshots, before i try it again.

3. linux caches the disk data before it writes it to disk. this is very good
for performance, but may create a big data loss in case of power failure...


cheers,
erez.

2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi people,
 I'm planning to add some big storage solution to my VPS 
 businesshttp://hetz.biz.
 I did some checking and calculated the costs, and figured out that if I want
 to have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).

 I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and how
 much RAM should I put in such a machine? Xeon is overkill IIRC.

 What do you suggest?

 Thanks and Shana Tova
 Hetz

 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-10 Thread Etzion Bar-Noy
Linux LVM2 has been around for several years now. It can take and use
snapshots, and I do it for the last three or so years on *production*
 sites.
There are limitations, such as space utilization and performance, but the
most significant one is that LVM snapshots are nowhere near NetApp
snapshots. Check BTRFS or ZFS for such behavior.

Ez

2010/9/10 Erez D erez0...@gmail.com

 here is my 3 cents:

 In two companies i worked for, i designed similiar servers.

 1. i wanted to have as less as a down time, and i didn't want to buy
 another server just to sit and wait for a failure, so i decided it should
 work on any pc with any raid controller - i decided to do the raid in
 software, so it will be hardware independant - in case of hardware failure -
 just get any pc with any raid controller ( the disks may even be connected
 via usb if you do not have a raid controller) and everything is up in
 minutes.

 2. I wanted to have snapshots like in a net-app - making snapshots every
 hour or so. using the most up to date redhat distributions of that time -
 when creating a snapshot, once in a while the server got locked (dead).
 i tried it again when LVM2 was officially released, and got similiar
 results. so as they say once bitten, twice shy - I will take a lot of
 proof LVM is stable enough for snapshots, before i try it again.

 3. linux caches the disk data before it writes it to disk. this is very
 good for performance, but may create a big data loss in case of power
 failure...


 cheers,
 erez.

 2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi people,
 I'm planning to add some big storage solution to my VPS 
 businesshttp://hetz.biz.
 I did some checking and calculated the costs, and figured out that if I want
 to have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).

 I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and how
 much RAM should I put in such a machine? Xeon is overkill IIRC.

 What do you suggest?

 Thanks and Shana Tova
 Hetz

 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il



 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-10 Thread Amos Shapira
2010/9/10 Etzion Bar-Noy eza...@tournament.org.il:
 Linux LVM2 has been around for several years now. It can take and use
 snapshots, and I do it for the last three or so years on production sites.
 There are limitations, such as space utilization and performance, but the
 most significant one is that LVM snapshots are nowhere near NetApp
 snapshots. Check BTRFS or ZFS for such behavior.
 Ez

I use LVM2 for a few years too but BTRFS or ZFS on Linux for production??

--Amos

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-10 Thread Etzion Bar-Noy
Nope. I did not say production. I said that if you want NetApp-style
snapshots, use one of the mentioned file systems. I never said I actually
used them for production, nor for anything at all (I don't)

Ez

On Fri, Sep 10, 2010 at 2:52 PM, Amos Shapira amos.shap...@gmail.comwrote:

 2010/9/10 Etzion Bar-Noy eza...@tournament.org.il:
  Linux LVM2 has been around for several years now. It can take and use
  snapshots, and I do it for the last three or so years on
 production sites.
  There are limitations, such as space utilization and performance, but the
  most significant one is that LVM snapshots are nowhere near NetApp
  snapshots. Check BTRFS or ZFS for such behavior.
  Ez

 I use LVM2 for a few years too but BTRFS or ZFS on Linux for production??

 --Amos

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


CPU RAM in a storage box

2010-09-09 Thread Hetz Ben Hamo
Hi people,
I'm planning to add some big storage solution to my VPS
businesshttp://hetz.biz.
I did some checking and calculated the costs, and figured out that if I want
to have a decent 12TB solution NAS box, it would be best if I would roll my
own. (12 TB before all the RAID stuff, after that it would lot less). All
other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
without a single hard disk).

I'm planning to use hardware based RAID card, minimal Linux distribution and
have some offers like iSCSI, NFS, CIFS - the usual suspects.

My question is: since I'll use hardware RAID card, which processor and how
much RAM should I put in such a machine? Xeon is overkill IIRC.

What do you suggest?

Thanks and Shana Tova
Hetz

-- 
my blog (hebrew): http://benhamo.org
Skype: heunique
MSN: hetz-b...@benhamo.org
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread geoffrey mendelson


On Sep 9, 2010, at 6:35 PM, Hetz Ben Hamo wrote:



I'm planning to add some big storage solution to my VPS business. I  
did some checking and calculated the costs, and figured out that if  
I want to have a decent 12TB solution NAS box, it would be best if I  
would roll my own. (12 TB before all the RAID stuff, after that it  
would lot less). All other solutions are very expensive (example:  
IBM EXP 3000 costs here 6K nis without a single hard disk).




The question you should be asking yourself, IMHO, is what can I buy  
that will be as reliable as a commerical, industrial grade server?






I'm planning to use hardware based RAID card, minimal Linux  
distribution and have some offers like iSCSI, NFS, CIFS - the usual  
suspects.


My question is: since I'll use hardware RAID card, which processor  
and how much RAM should I put in such a machine? Xeon is overkill  
IIRC.


For example, a system which costs under 900 NIS would do the job. You  
can get them from Ivory or KSP. They have a dual core ATOM processor,
one PCI slot and one DDR2 memory slot. The power supply is not very  
big, but it will power a bunch of 5400 rpm green disks.


How well will it work? How long will it last? Will it be fast enough?

And the killer question, how much will it cost to replace, in the  
value of downtime, your time to replace it, bad will among your  
customers, etc?


Geoff.

--
Geoffrey S. Mendelson,  N3OWJ/4X1GM
To help restaurants, as part of the stimulus package, everyone must  
order dessert. As part of the socialized health plan, you are  
forbidden to eat it. :-)









___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Kfir Lavi
Hi Hetz,
My experience is with raid backup servers, that need to keep data but don't
have to be the fastest.
I wouldn't go so  far with it at the start.
Take some old server box.
Buy a killer power supply for hard drives stability.
Now test the setup using mdadm as raid.
This will most likely be good for you.
If it's not mission critical, hardware raid is not mandatory, because with
mdadm you get updates, which are very easy to apply.
Now you just have to watch the server load and ram to decide if you need
better hardware.

Regards,
Kfir


2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi people,
 I'm planning to add some big storage solution to my VPS 
 businesshttp://hetz.biz.
 I did some checking and calculated the costs, and figured out that if I want
 to have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).

 I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and how
 much RAM should I put in such a machine? Xeon is overkill IIRC.

 What do you suggest?

 Thanks and Shana Tova
 Hetz

 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


RE: CPU RAM in a storage box

2010-09-09 Thread Baruch Shpirer
I am using a dual core machine with 1GB ram just for the OS even though its
not scratching the 50% usage
My raid controller is 3ware 9550SX-12LP 64-bit/133MHz PCI-X with 256 MB DDR2
400 memory with ECC protection supporting all raid types and with RISC
PowerPC cpu to provide real HW raid
This host is handling more then 8TB in Raid1 and Raid5 units easy and you
can leave hot spare drives and setup fast rebuild
Notifications both in snmp and mail will provide you alerts once a unit is
degraded or drive is giving errors (pre-fail)
I do recomment using real 64-bit interface and well... this card is old, you
can get newer and better similar cards.. so do the math
 
Shana tova to all

  _  

From: linux-il-boun...@cs.huji.ac.il [mailto:linux-il-boun...@cs.huji.ac.il]
On Behalf Of Hetz Ben Hamo
Sent: Thursday, September 09, 2010 18:36
To: linux-il
Subject: CPU  RAM in a storage box


Hi people,
I'm planning to add some big storage solution to my VPS business
http://hetz.biz . I did some checking and calculated the costs, and
figured out that if I want to have a decent 12TB solution NAS box, it would
be best if I would roll my own. (12 TB before all the RAID stuff, after that
it would lot less). All other solutions are very expensive (example: IBM EXP
3000 costs here 6K nis without a single hard disk). 

I'm planning to use hardware based RAID card, minimal Linux distribution and
have some offers like iSCSI, NFS, CIFS - the usual suspects.


My question is: since I'll use hardware RAID card, which processor and how
much RAM should I put in such a machine? Xeon is overkill IIRC.

What do you suggest?

Thanks and Shana Tova
Hetz

-- 
my blog (hebrew): http://benhamo.org
Skype: heunique
MSN: hetz-b...@benhamo.org

!DSPAM:4c88ff2c14471277820751! 
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Hetz Ben Hamo
Hi,

2010/9/9 geoffrey mendelson geoffreymendel...@gmail.com


 On Sep 9, 2010, at 6:35 PM, Hetz Ben Hamo wrote:


 I'm planning to add some big storage solution to my VPS business. I did
 some checking and calculated the costs, and figured out that if I want to
 have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).


 The question you should be asking yourself, IMHO, is what can I buy that
 will be as reliable as a commerical, industrial grade server?


Not looking for industrial grade one.


  I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and how
 much RAM should I put in such a machine? Xeon is overkill IIRC.


 For example, a system which costs under 900 NIS would do the job. You can
 get them from Ivory or KSP. They have a dual core ATOM processor,
 one PCI slot and one DDR2 memory slot. The power supply is not very big,
 but it will power a bunch of 5400 rpm green disks.



This storage will be mainly used for backups. If someone wants to do a
colocation to my rack, I want to give him a bonus, something that you can't
find today with my competitors: I want to give him 50-100GB for storage.
You'll get an NFS/CIFS/iSCSI and you mount it to your machine and use it for
your backup/rsync/whatever. By comparison, when you colocate a server to
Netvision's farm, you get ... 5GB backup space.. yippee..


 How well will it work? How long will it last? Will it be fast enough?


Fast doesn't matter much when you're doing backups or storing some
temporary stuff, does it really matter when it take 20 seconds instead of 10
when you're doing rsync? I don't think so..


 And the killer question, how much will it cost to replace, in the value
 of downtime, your time to replace it, bad will among your customers, etc?


Really depends. I'm not planning to fully use all the disks, some will be
disconnected or out of the RAID, perhaps I'll put a redundant PSU.

Hetz


 Geoff.

 --
 Geoffrey S. Mendelson,  N3OWJ/4X1GM
 To help restaurants, as part of the stimulus package, everyone must order
 dessert. As part of the socialized health plan, you are forbidden to eat it.
 :-)









-- 
my blog (hebrew): http://benhamo.org
Skype: heunique
MSN: hetz-b...@benhamo.org
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Etzion Bar-Noy
This is a joke, right? You want someone to host your system, which, by
design, will not be rack-mountable, and would be large, due to the amount of
disks you are to place there. It is possible, but extremely expensive to
host a non-1-U server nowadays. Who would give it to you?

An industrial-grade, 2U system could host, today, about 6 3.5 SATA disks. A
3 U can do much more, with up to 12-14 disks, depending on the system.

And RAM is extremely important. Since you will not invest in an
industial-class RAID controller (3ware, LSI-Logic, Adaptec, Intel, etc)
which will cost several hundreds of dollars, as I see it, you would want to
compensate for the high write latency with a large amount of RAM and fully
buffered writes (not secure, but good enough). Especially with 7200RPM SATA
drives with low seek speed.
NFS shares, in async mode would give great performance, provided you give
the system enough RAM. Then your RAM will actually become the disk write
cache.

Ez

2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi,

 2010/9/9 geoffrey mendelson geoffreymendel...@gmail.com


 On Sep 9, 2010, at 6:35 PM, Hetz Ben Hamo wrote:


 I'm planning to add some big storage solution to my VPS business. I did
 some checking and calculated the costs, and figured out that if I want to
 have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).


 The question you should be asking yourself, IMHO, is what can I buy that
 will be as reliable as a commerical, industrial grade server?


 Not looking for industrial grade one.


  I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and
 how much RAM should I put in such a machine? Xeon is overkill IIRC.


 For example, a system which costs under 900 NIS would do the job. You can
 get them from Ivory or KSP. They have a dual core ATOM processor,
 one PCI slot and one DDR2 memory slot. The power supply is not very big,
 but it will power a bunch of 5400 rpm green disks.



 This storage will be mainly used for backups. If someone wants to do a
 colocation to my rack, I want to give him a bonus, something that you can't
 find today with my competitors: I want to give him 50-100GB for storage.
 You'll get an NFS/CIFS/iSCSI and you mount it to your machine and use it for
 your backup/rsync/whatever. By comparison, when you colocate a server to
 Netvision's farm, you get ... 5GB backup space.. yippee..


 How well will it work? How long will it last? Will it be fast enough?


 Fast doesn't matter much when you're doing backups or storing some
 temporary stuff, does it really matter when it take 20 seconds instead of 10
 when you're doing rsync? I don't think so..


 And the killer question, how much will it cost to replace, in the value
 of downtime, your time to replace it, bad will among your customers, etc?


 Really depends. I'm not planning to fully use all the disks, some will be
 disconnected or out of the RAID, perhaps I'll put a redundant PSU.

 Hetz


 Geoff.

 --
 Geoffrey S. Mendelson,  N3OWJ/4X1GM
 To help restaurants, as part of the stimulus package, everyone must
 order dessert. As part of the socialized health plan, you are forbidden to
 eat it. :-)









 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Hetz Ben Hamo
Hi,

2010/9/9 Etzion Bar-Noy eza...@tournament.org.il

 This is a joke, right? You want someone to host your system, which, by
 design, will not be rack-mountable, and would be large, due to the amount of
 disks you are to place there. It is possible, but extremely expensive to
 host a non-1-U server nowadays. Who would give it to you?


Huh? Of course it will be rack-mountable. I'm planning to put it on a 2U or
3U chassis.  I'm also not looking for someone to host my hardware, I already
got a rack in Netvision today.


 An industrial-grade, 2U system could host, today, about 6 3.5 SATA disks.
 A 3 U can do much more, with up to 12-14 disks, depending on the system.


IBM EXP 3000 can host 12 3.5 hard drives on a 2U chassis. See
thisftp://public.dhe.ibm.com/common/ssi/ecm/en/tsd03025usen/TSD03025USEN.PDFPDF
file. There are other 2U cases which can  host 12 3.5 disks.

And RAM is extremely important. Since you will not invest in an
 industial-class RAID controller (3ware, LSI-Logic, Adaptec, Intel, etc)
 which will cost several hundreds of dollars, as I see it, you would want to
 compensate for the high write latency with a large amount of RAM and fully
 buffered writes (not secure, but good enough). Especially with 7200RPM SATA
 drives with low seek speed.
 NFS shares, in async mode would give great performance, provided you give
 the system enough RAM. Then your RAM will actually become the disk write
 cache.


Since this machine will not be using any Xeon with it's expensive RAM, I
could put some gigs of RAM quite easily.

Thanks for your points.
Hetz



Ez

 2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi,


 2010/9/9 geoffrey mendelson geoffreymendel...@gmail.com


 On Sep 9, 2010, at 6:35 PM, Hetz Ben Hamo wrote:


 I'm planning to add some big storage solution to my VPS business. I did
 some checking and calculated the costs, and figured out that if I want to
 have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).


 The question you should be asking yourself, IMHO, is what can I buy that
 will be as reliable as a commerical, industrial grade server?


 Not looking for industrial grade one.


  I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 My question is: since I'll use hardware RAID card, which processor and
 how much RAM should I put in such a machine? Xeon is overkill IIRC.


 For example, a system which costs under 900 NIS would do the job. You can
 get them from Ivory or KSP. They have a dual core ATOM processor,
 one PCI slot and one DDR2 memory slot. The power supply is not very big,
 but it will power a bunch of 5400 rpm green disks.



 This storage will be mainly used for backups. If someone wants to do a
 colocation to my rack, I want to give him a bonus, something that you can't
 find today with my competitors: I want to give him 50-100GB for storage.
 You'll get an NFS/CIFS/iSCSI and you mount it to your machine and use it for
 your backup/rsync/whatever. By comparison, when you colocate a server to
 Netvision's farm, you get ... 5GB backup space.. yippee..


 How well will it work? How long will it last? Will it be fast enough?


  Fast doesn't matter much when you're doing backups or storing some
 temporary stuff, does it really matter when it take 20 seconds instead of 10
 when you're doing rsync? I don't think so..


 And the killer question, how much will it cost to replace, in the value
 of downtime, your time to replace it, bad will among your customers, etc?


 Really depends. I'm not planning to fully use all the disks, some will be
 disconnected or out of the RAID, perhaps I'll put a redundant PSU.

 Hetz


 Geoff.

 --
 Geoffrey S. Mendelson,  N3OWJ/4X1GM
 To help restaurants, as part of the stimulus package, everyone must
 order dessert. As part of the socialized health plan, you are forbidden to
 eat it. :-)









 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: hetz-b...@benhamo.org

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il




-- 
my blog (hebrew): http://benhamo.org
Skype: heunique
MSN: hetz-b...@benhamo.org
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Amos Shapira
2010/9/10 Hetz Ben Hamo het...@gmail.com

 Hi people,
 I'm planning to add some big storage solution to my VPS business. I did some 
 checking and calculated the costs, and figured out that if I want to have a 
 decent 12TB solution NAS box, it would be best if I would roll my own. (12 TB 
 before all the RAID stuff, after that it would lot less). All other solutions 
 are very expensive (example: IBM EXP 3000 costs here 6K nis without a single 
 hard disk).
 I'm planning to use hardware based RAID card, minimal Linux distribution and 
 have some offers like iSCSI, NFS, CIFS - the usual suspects.

How about looking at openfiler (http://www.openfiler.com/), consider
whether you want to use it for that solution and see what kind of
hardware they recommend?

I have never used it but considered it before and as far as I remember
it's an integration of SAN management interface on top of CentOS.

Cheers,

--Amos

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Etzion Bar-Noy
On top of rpath Linux, which is the root of all evil. Also - although I have
implemented quite a few of these, OpenFiler suffers from various bugs
and shortcomings. Still - in the free like beer area - it is good enough
for most purposes.

Ez

On Fri, Sep 10, 2010 at 1:08 AM, Amos Shapira amos.shap...@gmail.comwrote:

 2010/9/10 Hetz Ben Hamo het...@gmail.com
 
  Hi people,
  I'm planning to add some big storage solution to my VPS business. I did
 some checking and calculated the costs, and figured out that if I want to
 have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K nis
 without a single hard disk).
  I'm planning to use hardware based RAID card, minimal Linux distribution
 and have some offers like iSCSI, NFS, CIFS - the usual suspects.

 How about looking at openfiler (http://www.openfiler.com/), consider
 whether you want to use it for that solution and see what kind of
 hardware they recommend?

 I have never used it but considered it before and as far as I remember
 it's an integration of SAN management interface on top of CentOS.

 Cheers,

 --Amos

 ___
 Linux-il mailing list
 Linux-il@cs.huji.ac.il
 http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il

___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


Re: CPU RAM in a storage box

2010-09-09 Thread Etzion Bar-Noy
I happen to have an IBM SAN storage at home, so I am familiar with IBM line
of storage products. The EXP3000 is an expansion to IBM storage, which can
perform for itself (JBoD), however - it contains no CPU, or RAID abilities
internally. You will have to connect it to an additional server (1U, I
assume) which will include the RAID hardware, if you decide to use it, and
OS.

I have not seen any commodity ATOM 1U systems. I know someone in my or IT
Experts forum in Tapuz has asked about such a solution, and it is not simple
to find. ATOMs are limited with the max amount of RAM. Desktop-class CPUs
are also rather rare in the 1-2U markets, and are far from trivial to
obtain, and worse - get service for.

Ez

On Fri, Sep 10, 2010 at 12:27 AM, Hetz Ben Hamo het...@gmail.com wrote:

 Hi,

 2010/9/9 Etzion Bar-Noy eza...@tournament.org.il

 This is a joke, right? You want someone to host your system, which, by
 design, will not be rack-mountable, and would be large, due to the amount of
 disks you are to place there. It is possible, but extremely expensive to
 host a non-1-U server nowadays. Who would give it to you?


 Huh? Of course it will be rack-mountable. I'm planning to put it on a 2U or
 3U chassis.  I'm also not looking for someone to host my hardware, I already
 got a rack in Netvision today.


 An industrial-grade, 2U system could host, today, about 6 3.5 SATA disks.
 A 3 U can do much more, with up to 12-14 disks, depending on the system.


 IBM EXP 3000 can host 12 3.5 hard drives on a 2U chassis. See 
 thisftp://public.dhe.ibm.com/common/ssi/ecm/en/tsd03025usen/TSD03025USEN.PDFPDF
  file. There are other 2U cases which can  host 12 3.5 disks.

 And RAM is extremely important. Since you will not invest in an
 industial-class RAID controller (3ware, LSI-Logic, Adaptec, Intel, etc)
 which will cost several hundreds of dollars, as I see it, you would want to
 compensate for the high write latency with a large amount of RAM and fully
 buffered writes (not secure, but good enough). Especially with 7200RPM SATA
 drives with low seek speed.
 NFS shares, in async mode would give great performance, provided you
 give the system enough RAM. Then your RAM will actually become the disk
 write cache.


 Since this machine will not be using any Xeon with it's expensive RAM, I
 could put some gigs of RAM quite easily.

 Thanks for your points.
 Hetz



 Ez

 2010/9/9 Hetz Ben Hamo het...@gmail.com

 Hi,


 2010/9/9 geoffrey mendelson geoffreymendel...@gmail.com


 On Sep 9, 2010, at 6:35 PM, Hetz Ben Hamo wrote:


 I'm planning to add some big storage solution to my VPS business. I did
 some checking and calculated the costs, and figured out that if I want to
 have a decent 12TB solution NAS box, it would be best if I would roll my
 own. (12 TB before all the RAID stuff, after that it would lot less). All
 other solutions are very expensive (example: IBM EXP 3000 costs here 6K 
 nis
 without a single hard disk).


 The question you should be asking yourself, IMHO, is what can I buy that
 will be as reliable as a commerical, industrial grade server?


 Not looking for industrial grade one.


  I'm planning to use hardware based RAID card, minimal Linux
 distribution and have some offers like iSCSI, NFS, CIFS - the usual
 suspects.

 My question is: since I'll use hardware RAID card, which processor and
 how much RAM should I put in such a machine? Xeon is overkill IIRC.


 For example, a system which costs under 900 NIS would do the job. You
 can get them from Ivory or KSP. They have a dual core ATOM processor,
 one PCI slot and one DDR2 memory slot. The power supply is not very big,
 but it will power a bunch of 5400 rpm green disks.



 This storage will be mainly used for backups. If someone wants to do a
 colocation to my rack, I want to give him a bonus, something that you can't
 find today with my competitors: I want to give him 50-100GB for storage.
 You'll get an NFS/CIFS/iSCSI and you mount it to your machine and use it for
 your backup/rsync/whatever. By comparison, when you colocate a server to
 Netvision's farm, you get ... 5GB backup space.. yippee..


 How well will it work? How long will it last? Will it be fast enough?


  Fast doesn't matter much when you're doing backups or storing some
 temporary stuff, does it really matter when it take 20 seconds instead of 10
 when you're doing rsync? I don't think so..


 And the killer question, how much will it cost to replace, in the
 value of downtime, your time to replace it, bad will among your customers,
 etc?


 Really depends. I'm not planning to fully use all the disks, some will be
 disconnected or out of the RAID, perhaps I'll put a redundant PSU.

 Hetz


 Geoff.

 --
 Geoffrey S. Mendelson,  N3OWJ/4X1GM
 To help restaurants, as part of the stimulus package, everyone must
 order dessert. As part of the socialized health plan, you are forbidden to
 eat it. :-)









 --
 my blog (hebrew): http://benhamo.org
 Skype: heunique
 MSN: